Fernet-Branca Pop-up Experience

2024.7

Unreal Engine-Based Interactive Installation for Brand Activation

Interactive Installation/Unreal Engine/Motion Capture/Captury/Python/LED Wall

During my internship, we collaborated with the liqueur brand Fernet-Branca to create a pop-up shop experience in New Orleans. 

Using a markerless motion capture system and a high-definition LED wall, we built an immersive experience where fans can meet the brand’s mascot, Fernando the crocodile. Up to two people at a time can use their motion to control the mascots and interact with each other.

To help fans remember the moment, and also help spread the word about Fernando, we built an automated system that records their experience and sends it to participants’ emails, ready for sharing on social media.

My Work

This is a team project.

  • I was the lead for setting up the motion capture system, including designing the placement of cameras, setting up the cameras, calibration, and Unreal Engine integration.

  • Together with another engineer, I proposed the dual-view recording function and the tech stack for the system using OBS and Gmail API.

  • I also participated in the development of the Unreal Engine scene.

Captury Markerless Motion Capture System

Markerless motion capture system is ideal for interactive installations because the participants do not need to wear any equipment.

We chose Captury, which is an RGB, multi-camera-based solution. Based on our testing, this system not only provides good accuracy, but it can also cover a wider range of area in a small room, compared to single 3D camera solutions like Kinect.

The system also has native support for multi-person tracking, which enables participants to interact with each other, elevating the experience.

Working with Constraints through Collaboration

The space for the experience is a vintage pizza oven. Upon arrival, we found out the brick walls were too brittle, and the camera mounts cannot be drilled into.

To overcome this problem, we worked closely with fabricators, analyzing places where we can safely set up the cameras. Also, we created innovative ways to mount the cameras on the wall, by hanging plywood from pipes with zip ties.

Additionally, I analyzed the space and figured out a camera placement plan that would avoid unavailable places and require the least amount of cameras. In the end, we were able to use only 6 cameras, as opposed to the originally planned 8, while still covering the entire area of the space.

Sharing the Experience

With the client’s goal being to increase the recognition and online presence of their mascot Fernando, and also connecting with the community, we designed a system that automatically records the experience for the participants to share on social media.

Two viewpoints: the Unreal Engine rendering, and the participants, are being recorded simultaneously, and composed in realtime using OBS, controlled via WebSocket with Python. The users enter their email address before the experience, and we use the Gmail API to send the video to, which the client can also utilize for further targeted engagements.

Enlin Li

© 2023-2026. All Rights Reserved.

Kanagawa, Japan