genai-ki

2023.9 - 2023.12

Ai-powered endless journey

Generative AI, Unity, Python

This project explores how generative AI can enhance, or take over, the storytelling process. I created a visual novel style 2D travel/talking simulator game where every single place, NPC, conversation, and even game objectives are created by AI (including GPT and leonardo.ai) dynamically as you play. Because game assets are generated by AI, you'll never run out of new content. The journey can continue for as long as you want.

Challenges include managing API calls and generated assets in Unity, as well as ensuring correct and consistent outputs.

Demo

Inspiration

I was inspired by 風雨来記/Fuuraiki (hence the name of the project), a Japanese travel adventure game series where a journalist travels freely across a region in Japan on a bike, writing stories and meeting people along the way. The game is unique in that it features real-world locations and riding footage to increase the immersion.
I love the idea of traveling and exploring the stories and changes that surround a place, but of course, it would be very resource-intensive to create a world that's expansive and has variety. Therefore, I decided to use generative AI to "augment" the storytelling.

The Game

The game has a simple mechanism. The player plays as a traveler, and wherever they arrive, they will meet a local, and their objective is, through conversation, to uncover as much information as possible about the person and the place -- the history, traditions, socio-economical changes, and so on. The number of conversations is limited before the player has to hit the road again, arriving at the next unexpected destination.

AI handles almost the entirety of the gameplay. The settings for the place and the NPC, the objectives that the player needs to discover, the NPC's conversation responses, and the final score-counting, are all processed by generative AI services through API calls.

Generative AI

4 types of generative AI sessions were utilized in the game. In the order from start to end:

1. "the writer" (GPT4): Using prompt templates, GPT will be instructed to create a setting for a location, a character that lives in the place, and 10 key points of information that are related to the character and the place for the player to discover. It will also generate prompts for the image generator for both the place and the character that will be used in the next step. GPT4 is used for the best comprehension and creativity.

2. "the painter" (leonardo.ai): After fetching the settings, the prompts are provided to leonardo.ai, an AI image generator platform, to generate a landscape shot and a close-up character shot image that will be used as background visuals for the gameplay.

3. "the actor" (GPT4-Turbo): The location and character settings, as well as the key information, were provided, and a new GPT session was created and instructed to act as the in-game NPC, engaging the conversation with the player. GPT4-Turbo version is used for its lower API cost and faster generation speed.

4. "the gamemaster" (GPT4-Turbo): After the conversation ends, the entire transcript, along with all the settings and key information is provided to another GPT session, which was instructed to analyze the transcript for how many points were discovered by the player. The results were formatted as an array so that it could be parsed by Unity. GPT4-Turbo is used for its longer context window.

Enlin Li

Brooklyn, New York, United States