Spark – Profiles, Jobs, and Resources for Python Developers

Apache Spark is a unified analytics engine for large-scale data processing.

It provides high-level APIs in Python, Java, Scala, R, and SQL, and executes tasks in parallel across a cluster of computers.