IBM: Average VM Uptime Hours per Region (PySpark)
PYSPARK coding challenge · Difficulty: medium · Topic: Date Functions · +100 XP
Problem
*IBM-style interview practice.*
Topic: Date Functions · timestamp differences · date-range filter · GROUP BY / AVG / ROUND
IBM's cloud operations team wants to compare regions: compute the average uptime in hours of the virtual-machine sessions in each region over the last 90 days, with the PySpark DataFrame API (or Spark SQL).
- Each row is one uptime session of a virtual machine, from
start_timetoend_time. A VM can have many sessions. - A session's uptime is
end_time - start_timein hours, including minutes and seconds (90 minutes = 1.5 hours). - The report runs on 30 June 2025. "The last 90 days" are the sessions whose
start_timeis from2025-04-01 00:00:00up to, but not including,2025-06-30 00:00:00. - A session with no
end_time(NULL) is still running and is not counted. - For each region,
avg_uptime_hoursis the average uptime per session (every session counts once), rounded to 2 decimals. - Regions with no counted session have no row.
Assign the result to df_result with the columns region and avg_uptime_hours, sorted by avg_uptime_hours descending, then region ascending, and finish with df_result.show().
Schema
DataFrame: vm_uptime
| Column | Type | Meaning | | --- | --- | --- | | vm_id | integer | The virtual machine | | region | string | The VM's region | | start_time | string | When the session started, `yyyy-MM-dd HH:mm:ss` | | end_time | string | When it ended, `yyyy-MM-dd HH:mm:ss`; null while still running |
Example 1
Input: vm_uptime
| vm_id | region | start_time | end_time | | --- | --- | --- | --- | | 1 | US-East | 2025-06-01 08:00:00 | 2025-06-01 16:00:00 | | 2 | US-East | 2025-06-10 09:00:00 | 2025-06-10 16:00:00 | | 1 | US-East | 2025-05-20 22:00:00 | 2025-05-21 05:30:00 | | 2 | US-East | 2025-03-15 06:00:00 | 2025-03-16 02:00:00 | | 1 | US-East | 2025-06-29 10:00:00 | null | | 3 | Europe | 2025-06-05 10:00:00 | 2025-06-05 12:30:00 | | 4 | Europe | 2025-04-01 00:00:00 | 2025-04-01 05:30:00 | | 4 | Europe | 2025-06-30 01:00:00 | 2025-06-30 03:00:00 | | 5 | APAC | 2025-03-31 23:00:00 | 2025-04-01 02:00:00 |
Output:
`
+-------+----------------+
| region|avg_uptime_hours|
+-------+----------------+
|US-East| 7.5| | Europe| 4.0|
+-------+----------------+
`
Explanation:
- US-East counted sessions: 8 h (1 Jun 08:00-16:00), 7 h (10 Jun 09:00-16:00) and 7.5 h (20 May 22:00 to 21 May 05:30, across midnight). Average = 22.5 / 3 = 7.50. The 15 March session is before the window, and the session from 29 June has not ended.
- Europe counted sessions: 2.5 h (5 Jun) and 5.5 h (starts exactly 1 April 00:00:00, which is inside). Average = 4.00. The session on 30 June is the report day, so it is outside the window.
- APAC has one session, but it started on 31 March (before the window), so APAC has no row.
Constraints
end_timeis null or later thanstart_time.- A session belongs to the window by its start time; a session that started inside the window counts in full even if it ends after it.
F.unix_timestamp("end_time") - F.unix_timestamp("start_time")gives seconds; divide by 3600 for hours.vm_uptime,spark,F(pyspark.sql.functions) andWindoware ready. Do not recreatevm_uptime.- Your code is also run on hidden datasets with other regions and sessions, so it must compute the answer from
vm_uptime. The row order is graded.
What this PYSPARK challenge teaches you
“IBM: Average VM Uptime Hours per Region (PySpark)” is a medium-level PYSPARK challenge focused on Date Functions. Working through it gives you hands-on practice with unix_timestamp, to_timestamp, Date range, groupBy, Cloud operations — the kind of transformation you are asked to write in real data engineering work and in technical interviews. You can solve it directly in the browser: the dataset is pre-loaded, so you write the query or DataFrame code, run it, and compare your output against the expected result immediately.
Concepts covered
- unix_timestamp
- to_timestamp
- Date range
- groupBy
- Cloud operations
How to approach it
If you get stuck, work through these steps in order before looking at a full solution — each one narrows the problem down:
- Convert with F.to_timestamp and filter start_time >= 2025-04-01 00:00:00 and < 2025-06-30 00:00:00.
- (F.unix_timestamp(end) - F.unix_timestamp(start)) / 3600.0 is the session's uptime in hours.
- Drop running sessions: F.col('end_time').isNotNull().
- groupBy('region').agg(F.round(F.avg(...), 2)) and orderBy(avg desc, region).
Interview practice
This problem is good practice for data engineering interviews at companies such as IBM. Interviewers use problems like it to check whether you can express the logic cleanly and reason about correctness on edge cases such as ties, nulls and empty groups.
How to practise it on PySpark.in
Open the challenge, write your PySpark code in the editor and press Run to execute it against the sample dataset. Submitting checks your output against every test case, including hidden ones, so you find out straight away whether your logic holds up. You can retry as often as you like, and each solved challenge adds to your XP.
Related PYSPARK challenges
- Monthly Revenue Report Using DATE_FORMAT
- Count Employees Hired in Each Year
- Find the Month With the Highest Number of New Hires
- Zomato: A User's Orders from the Last 30 Days
- Amazon: Customers Active on Exactly 3 Days Last Month
- Amazon: Customers Active on Exactly 3 Days Last Month (PySpark)
Helpful resources
Frequently asked questions
Do I need to install Spark or a database to solve this?
No. The PYSPARK environment runs in your browser with the sample data already loaded, so there is nothing to install or configure.
Is this challenge free?
Yes - the problem, the sample dataset, the hints and unlimited test runs are free.
What level is it?
It is rated medium and covers Date Functions.