What is the difference between Transformation and Action in Spark?

Published 2026-09-01 in PySpark

👉 What Is the Difference Between Transformation and Action in Apache Spark? Understanding the difference between Transformations and Actions is one of the most important fundamentals of Apache Spark—and a very common topic in Data Engineering interviews . 🔹 Transformation A Transformation creates a new DataFrame or RDD from an existing dataset. Transformations are lazy , which means Spark does not execute them immediately. Instead, Spark builds an execution plan and optimizes it before the computation is actually performed. 📌 Common Examples: filter() select() withColumn() join() groupBy() map() 🔹 Action An Action triggers the actual execution of the Spark computation. Actions either return a result to the driver or initiate an operation that writes data to external storage. 📌 Common Examples: count() collect() show() take() write() 💡 Simple Example df2 = df.filter(df.age > 18) ➡️ Transformation — Spark does not execute the filtering operation immediately. It records it as part of the execution plan. df2.count() ➡️ Action — Spark now executes the required computation and returns the count. 🧠 Interview Tip A simple way to remember it: Transformation = Builds the execution plan 🏗️…

More PySpark articles · All collections · Practice challenges