Context

During the COVID-19 pandemic, Pi developed an e-learning platform for NovoNordisk called “Diabetes Academy,” designed to introduce doctors to new diabetes treatments. After three months of online courses, we created a 40-minute immersive experience as the program’s culmination.

In this activity, doctors, organized into groups of 4 and guided by 2 actors, took on the role of special agents and solved puzzles in three virtual rooms. This journey allowed them to apply what they had learned in a fun way and collaborate with other doctors, providing valuable social interaction during a time of distancing.

The experience was highly appreciated by both the doctors and NovoNordisk, highlighting the combination of learning, entertainment, and the opportunity to connect in a safe virtual environment.

Journey

Welcome

As the elevator doors open, the 4 doctors are greeted by 2 actors who present the mission and invite them to choose a character.

Using their phones, participants scan a QR code that takes them to a unique interface linked to their role. Throughout the experience, they must collaborate to solve codes and puzzles using this interface.

Once everyone has selected a character, a “hack” is triggered. Alarms go off, red flashing lights appear, and smoke fills the room, simulating an attack on the facilities. The participants must work as a team to stop the hackers and save lives. The team is then led to the first room, where the challenge begins.

Room 1

In the first room, they are presented with a critical medical case: a patient whose life is at risk.

Due to the hacker attack, the patient’s medical profile is encrypted. The goal of this room is to save the patient’s life. To do this, the doctors must decode the information displayed on the screen and match it with the correct medical profile. Using their phones, they must enter the correct codes to reveal crucial information about the patient. With limited time, they must organize themselves to decode only the relevant information.

Once the correct profile is identified, they stabilize the patient by assembling a box with the appropriate medication.

Finally, they complete the medical prescription by filling in the missing words, using clues provided in the projection, applying the knowledge gained during the course.

Room 2

In the second room, participants face another patient, but this time they can “travel through time.”

Exploring the patient’s apartment, they discover lifestyle habits that led to the crisis.

After gathering clues, they must collaborate to answer multiple-choice questions on their phones to prescribe a treatment plan based on the patient’s history.

Room 3

In the final room, each doctor stands under a light corresponding to the color of their character. A congratulatory video is played, celebrating their success in saving both patients. After the video, they interact with a timeline showing the evolution of diabetes treatments over the years. As a final touch, participants receive a printed group photo before leaving the experience.

Control Interface

The “function” was monitored live with security cameras and a graphical interface made in TouchDesigner, which allowed operators to control the state of each room.

Prototype

To present the project to the client and iterate quickly, an interactive demo was made in Unreal Engine.

My Role

Some of my key responsibilities were:

• Assembling and leading the programming team.
• Proposing, defining, and executing the technical solutions to implement the project’s creative ideas.
• Developing the UI that allowed real-time control of the game, enabling backstage operators to trigger events and manage the progress of each group of participants.
• Designing the project architecture in TouchDesigner, distributing the processing load across multiple computers to ensure optimal performance during the experience. Special thanks to Matthew Ragan for his course on modular architectures which was instrumental.
• Designing and programming the integration and communication between various systems: computers, phones, m5sticks, sound systems, lighting, projections, smoke machine, and Arduinos.
• Developing the project in Unreal Engine used in room 2.

Additionally, I worked closely with other programmers to effectively integrate the audiovisual content of the rooms, developed in TouchDesigner.

Team

Companies

Design and programming - Pi

Production and construction - Prokart

Sound and lighting - Dedaluzproducciones

Programming Team

• Benjamín Benavides
• Paola Olea, Tolch: audiovisual development of the rooms in TouchDesigner and phone UI.
• Sebastián Felbol: IoT programming (M5stick)