Luigi – Profiles, Jobs, and Resources for Python Developers

Luigi is a Python framework for building complex pipelines of batch jobs.

It handles dependency resolution, workflow management, and monitoring, simplifying the development and execution of large-scale data processing tasks.