Maximum Difference Between Consecutive Values (PySpark)

PYSPARK coding challenge · Difficulty: easy · Topic: Window Functions · +70 XP

Problem

Sort the values in ascending order. Walk through that sorted list and compare

every value with the one immediately before it. Return the largest of those

differences.

Produce a single row with one column named max_diff.

Sort by `value`, not by `id`

"Consecutive" means consecutive in value order. The rows are not stored

sorted, so ordering by id gives a different — and wrong — answer.

Schema — `numbers`

| Column | Type |
| --- | --- |
| id | int |
| value | int |

Example Input — `numbers`

| id | value |
| --- | --- |
| 1 | 25 |
| 2 | 10 |
| 3 | 50 |
| 4 | 15 |
| 5 | 30 |

Expected Output

| max_diff |
| --- |
| 20 |

Explanation

Step 1 — sort the values ascending: 10, 15, 25, 30, 50

Step 2 — pair each value with the one before it:

| value | previous value | difference |
| --- | --- | --- |
| 10 | null | null |
| 15 | 10 | 5 |
| 25 | 15 | 10 |
| 30 | 25 | 5 |
| 50 | 30 | 20 |

The first row has nothing before it, so its difference is null. F.max() skips

nulls, so it needs no special handling.

Step 3 — the largest difference is 20.

Sorting by id instead would give the values 25, 10, 50, 15, 30, differences

-15, 40, -35, 15, and an answer of 40. That is why the ordering matters.

Notes

• The DataFrame is created for you — do not recreate it
• Build a DataFrame called df_result and finish with df_result.show()
• The single output column must be named max_diff
• Same problem in SQL: [Maximum Difference Between Consecutive Values](/challenges/maximum-difference-between-consecutive-values)

What this PYSPARK challenge teaches you

“Maximum Difference Between Consecutive Values (PySpark)” is a easy-level PYSPARK challenge focused on Window Functions. Working through it gives you hands-on practice with lag, Window, max, Consecutive, Difference — the kind of transformation you are asked to write in real data engineering work and in technical interviews. You can solve it directly in the browser: the dataset is pre-loaded, so you write the query or DataFrame code, run it, and compare your output against the expected result immediately.

Concepts covered

• lag
• Window
• max
• Consecutive
• Difference

How to approach it

If you get stuck, work through these steps in order before looking at a full solution — each one narrows the problem down:

1. F.lag("value").over(Window.orderBy("value")) gives the previous row's value. Order by value, not by id.
2. Add prev_value as a column first, then select F.max(F.col("value") - F.col("prev_value")).alias("max_diff").

How to practise it on PySpark.in

Open the challenge, write your PySpark code in the editor and press Run to execute it against the sample dataset. Submitting checks your output against every test case, including hidden ones, so you find out straight away whether your logic holds up. You can retry as often as you like, and each solved challenge adds to your XP.

Related PYSPARK challenges

• Top 3 Products per Category
• Running Total Revenue
• Median Salary per Department
• Latest Order Per Customer
• 3-Day Rolling Sum of Sales
• 7-Day Rolling Purchase Amount by Customer

Frequently asked questions

Do I need to install Spark or a database to solve this?

No. The PYSPARK environment runs in your browser with the sample data already loaded, so there is nothing to install or configure.

Is this challenge free?

Yes - the problem, the sample dataset, the hints and unlimited test runs are free.

What level is it?

It is rated easy and covers Window Functions.

Solve this challenge free on PySpark.in