Customers Who Purchased From Every Category (PySpark)

PYSPARK coding challenge · Difficulty: medium · Topic: Aggregation · +100 XP

Problem

Topic: Aggregation · COUNT(DISTINCT ...) · comparing with a total (relational division)

The marketing team wants to reward true all-rounders: customers who have bought from every category the store sells.

Using the PySpark DataFrame API (or Spark SQL), find those customers.

Assign the result to df_result with the single column customer_id, sorted ascending, and finish with df_result.show().

Schema

DataFrame: products

| Column | Type | Meaning |
| --- | --- | --- |
| product_id | integer | Unique id of the product |
| category | string | The product's category |

DataFrame: purchases

| Column | Type | Meaning |
| --- | --- | --- |
| customer_id | integer | The customer who bought |
| product_id | integer | The product bought (one row per purchase) |

Example 1

Input: products

| product_id | category |
| --- | --- |
| 101 | Electronics |
| 102 | Electronics |
| 103 | Books |
| 104 | Books |
| 105 | Clothing |

Input: purchases

| customer_id | product_id |
| --- | --- |
| 1 | 101 |
| 1 | 103 |
| 1 | 105 |
| 2 | 101 |
| 2 | 102 |
| 2 | 103 |
| 3 | 103 |
| 3 | 104 |
| 3 | 105 |
| 3 | 101 |
| 4 | 105 |
| 4 | 105 |
| 4 | 105 |
| 5 | 102 |
| 5 | 104 |

Output:

`

+-----------+

|customer_id|

+-----------+

|          1|
|          3|

+-----------+

`

Explanation:

There are 3 available categories: Electronics (101, 102), Books (103, 104) and Clothing (105).

Constraints

What this PYSPARK challenge teaches you

“Customers Who Purchased From Every Category (PySpark)” is a medium-level PYSPARK challenge focused on Aggregation. Working through it gives you hands-on practice with countDistinct, groupBy, crossJoin, Relational division, E-commerce — the kind of transformation you are asked to write in real data engineering work and in technical interviews. You can solve it directly in the browser: the dataset is pre-loaded, so you write the query or DataFrame code, run it, and compare your output against the expected result immediately.

Concepts covered

How to approach it

If you get stuck, work through these steps in order before looking at a full solution — each one narrows the problem down:

  1. total = products.select(F.countDistinct('category')).first()[0] -- from the catalogue, not from purchases.
  2. purchases.join(products, 'product_id') gives each purchase its category.
  3. groupBy('customer_id').agg(F.countDistinct('category')) counts each customer's different categories.
  4. Keep the rows equal to total, select('customer_id').orderBy('customer_id'), then df_result.show().

Interview practice

This problem is good practice for data engineering interviews at companies such as Infosys, IBM, Capgemini. Interviewers use problems like it to check whether you can express the logic cleanly and reason about correctness on edge cases such as ties, nulls and empty groups.

How to practise it on PySpark.in

Open the challenge, write your PySpark code in the editor and press Run to execute it against the sample dataset. Submitting checks your output against every test case, including hidden ones, so you find out straight away whether your logic holds up. You can retry as often as you like, and each solved challenge adds to your XP.

Related PYSPARK challenges

Helpful resources

Frequently asked questions

Do I need to install Spark or a database to solve this?

No. The PYSPARK environment runs in your browser with the sample data already loaded, so there is nothing to install or configure.

Is this challenge free?

Yes - the problem, the sample dataset, the hints and unlimited test runs are free.

What level is it?

It is rated medium and covers Aggregation.

Solve this challenge free on PySpark.in