PySpark: read a CSV into a DataFrame

In PySpark you load a CSV with `spark.read.csv(path)` or `spark.read.option(...).csv(path)`. Set `header=True` to use the first row as column names and `inferSchema=True` to detect types automatically — or pass an explicit schema for speed and correctness in production.

Header and schema

spark.read.option("header", True).option("inferSchema", True).csv("data.csv") reads names and infers types. inferSchema scans the file, so for large data provide an explicit StructType schema instead.

Common options

Useful options: sep (delimiter), nullValue, quote, escape, mode (PERMISSIVE / DROPMALFORMED / FAILFAST) to control how malformed rows are handled.

Example (PySpark)

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

df = (
    spark.read
         .option("header", True)
         .option("inferSchema", True)
         .csv("path/to/data.csv")
)
df.printSchema()
df.show(5)

Reads a CSV using the header row for names and inferring column types.

Run this example in the free online PySpark compiler

Frequently asked questions

How do I read a CSV file in PySpark?

Use spark.read.option("header", True).csv("path.csv"). Add .option("inferSchema", True) to auto-detect column types.

What does inferSchema do and is it slow?

inferSchema scans the data to detect column types. It's convenient but adds an extra pass over the file; for large or production data, supply an explicit schema instead.

How do I read a CSV with a different delimiter?

Set the sep option, e.g. spark.read.option("sep", ";").csv(path) for a semicolon-delimited file.

Open the free PySpark compiler · Data Engineering challenges · Data Engineering jobs