D1, D7 and D30 Retention per User (PySpark)

PYSPARK coding challenge · Difficulty: hard · Topic: Aggregation · +150 XP

Problem

A user is 'retained on day N' if they did anything exactly N days after they signed up. Report each user's D1, D7 and D30 retention as 1 or 0.

Schema — `signups`

| Column |
| --- |
| user_id |
| signup_date |

Schema — `user_events`

| Column |
| --- |
| user_id |
| event_date |

Example Input — `signups`

| user_id | signup_date |
| --- | --- |
| 1 | 2024-01-01 |
| 2 | 2024-01-01 |
| 3 | 2024-01-05 |

Example Input — `user_events`

| user_id | event_date |
| --- | --- |
| 1 | 2024-01-02 |
| 1 | 2024-01-08 |
| 1 | 2024-01-31 |
| 2 | 2024-01-02 |
| 3 | 2024-01-12 |

Expected Output

| user_id | retained_d1 | retained_d7 | retained_d30 | retention_pattern |
| --- | --- | --- | --- | --- |
| 1 | 1 | 1 | 1 | 1-1-1 |
| 2 | 1 | 0 | 0 | 1-0-0 |
| 3 | 0 | 1 | 0 | 0-1-0 |

Explanation

Retention is measured against each user's OWN signup date, not a fixed calendar date — user 3 signed up on 2024-01-05, so their day 7 is 2024-01-12.

Worked through: user 1 returns on day 1, day 7 and day 30, so 1/1/1. User 2 returns on day 1 and never again, so 1/0/0. User 3 returns only on day 7, so 0/1/0.

Every user must appear, including those who never came back — which is why the join has to be a LEFT JOIN.

Notes

• The DataFrame is created for you — do not recreate it
• Build a DataFrame called df_result and finish with df_result.show()
• Same problem in SQL: [D1, D7 and D30 Retention per User](/challenges/d1-d7-and-d30-retention-per-user)

What this PYSPARK challenge teaches you

“D1, D7 and D30 Retention per User (PySpark)” is a hard-level PYSPARK challenge focused on Aggregation. Working through it gives you hands-on practice with join, datediff, when, Retention — the kind of transformation you are asked to write in real data engineering work and in technical interviews. You can solve it directly in the browser: the dataset is pre-loaded, so you write the query or DataFrame code, run it, and compare your output against the expected result immediately.

Concepts covered

• join
• datediff
• when
• Retention

How to approach it

If you get stuck, work through these steps in order before looking at a full solution — each one narrows the problem down:

1. F.datediff(F.to_date('event_date'), F.to_date('signup_date')) is the day offset.
2. F.max(F.when(F.col('day_offset') == 7, 1).otherwise(0)) is the D7 flag.
3. Join with how='left' so users with no events survive the join.
4. F.concat_ws('-', 'retained_d1', 'retained_d7', 'retained_d30')

How to practise it on PySpark.in

Open the challenge, write your PySpark code in the editor and press Run to execute it against the sample dataset. Submitting checks your output against every test case, including hidden ones, so you find out straight away whether your logic holds up. You can retry as often as you like, and each solved challenge adds to your XP.

Related PYSPARK challenges

• Find Duplicate Emails
• HR: Average Salary by Department
• Logistics: Count Shipments by Status
• Count Total Orders Placed by Each Customer
• Find Average Order Amount for Each Customer
• Find Customers Who Placed More Than 5 Orders (HAVING)

Frequently asked questions

Do I need to install Spark or a database to solve this?

No. The PYSPARK environment runs in your browser with the sample data already loaded, so there is nothing to install or configure.

Is this challenge free?

Yes - the problem, the sample dataset, the hints and unlimited test runs are free.

What level is it?

It is rated hard and covers Aggregation.

Solve this challenge free on PySpark.in