PySpark fillna(): replace null values

PySpark `fillna(value)` (alias `df.na.fill(value)`) replaces null values in a DataFrame. Pass a single value to fill all compatible columns, or a dict to fill specific columns with different values. Use `dropna()` to remove rows containing nulls instead of filling them.

Fill per column

df.fillna({"age": 0, "city": "unknown"}) fills each named column with its own default; a single value like fillna(0) fills all numeric columns.

Drop vs fill vs coalesce

dropna() removes rows with nulls; fillna() substitutes a value; coalesce(col1, col2) returns the first non-null across columns.

Example (PySpark)

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()
df = spark.createDataFrame([(1, None, "Delhi"), (2, 30, None)], ["id", "age", "city"])

df.fillna({"age": 0, "city": "unknown"}).show()

Fills null age with 0 and null city with 'unknown' using a per-column dict.

Run this example in the free online PySpark compiler

Frequently asked questions

How do I replace null values in PySpark?

Use df.fillna(value) or df.na.fill(value). Pass a dict to fill specific columns with different defaults.

What is the difference between fillna and dropna in PySpark?

fillna substitutes a value for nulls; dropna removes rows that contain nulls entirely.

How do I get the first non-null value across columns?

Use coalesce(col1, col2, ...) from pyspark.sql.functions, which returns the first non-null value per row.

Open the free PySpark compiler · Data Engineering challenges · Data Engineering jobs