PySpark Tutorials

Every PySpark tutorial on PySpark.in (54 articles), grouped by topic. Each article covers one subject in depth with runnable examples you can try in the free browser compiler.

Your first DataFrame learning path

Start with basic Python and SQL. Follow the course categories for the full lessons, then use these short guides to practise each DataFrame skill with visible results.

1. Load and inspect data
Understand schemas and try a small CSV parsing example.

Read CSV records
2. Transform and clean
Create columns, choose missing-value defaults and define which duplicate should survive.

Add or replace columns

Handle nulls and NaN

Remove duplicates
3. Combine data and inspect partitions
Choose a join that preserves the rows you need, then inspect how rows are partitioned.

Choose a join

Repartition or coalesce