Transcript of Lies: Detective Mystery Game
Bridging Gaming and Accessibility Through Speech-to-Text Technology
Transcript of Lies is an interactive detective mystery game that combines engaging gameplay with accessibility education. Players step into the role of a detective investigating the disappearance of scientists, using a speech-to-text device as their primary tool for gathering and interpreting clues.
Product design
Interaction Design
User Research
Lead Developer & Designer (Primary coder with AI assistance)
General public lacks understanding of communication challenges faced by speech-to-text users. STT technology has significant accuracy issues with accents, dialects, and environmental noise, requiring users to develop complex error-correction strategies that most people don't realize exist.
SOLUTION
Transcript of Lies
Created a detective mystery where players experience STT challenges firsthand. Detective steps in green substance that simulates speech processing difficulties, forcing reliance on STT device to solve the case and rescue missing scientist.
THE CHALLENGE
Create an entertainment-focused game (not an edugame) that:
Create an entertainment-focused game (not an edugame) that:
Encourages understanding of how STT technologies assist communication
Raises awareness of the limitations and frustrations faced by STT users
Maintains an engaging and enjoyable gameplay experience without invoking pity or turning the experience into a lecture
Gaming Industry Gap: Voice-enabled games historically focused on "novelty or gimmicky aspect" rather than authentic accessibility representation
Cultural Impact: Research shows "different cultural backgrounds or language families" affect STT effectiveness, with "oriental cultural backgrounds less active in learning interaction"
Accuracy Barriers: "Most studies report that although STR technology is useful for learning, a greater accuracy in the system's recognition of speech is required". Accuracy drops significantly with accents, dialects, and background noise.
My team analyzed 12+ academic sources spanning STT technology, accessibility gaming, and cross-cultural communication barriers. Here are the key research insights we found:
RESEARCH
The detective steps into a glowing green substance at the lab. Suddenly, his communication abilities are altered, he must now rely on an STT device to interpret speech.
🟣 UX Goal: Introduce the “STT impairment” concept through a narrative device rather than text explanation.
The detective steps into a glowing green substance at the lab. Suddenly, his communication abilities are altered, he must now rely on an STT device to interpret speech.
🟣 UX Goal: Introduce the “STT impairment” concept through a narrative device rather than text explanation.
🧪 Scene 2 – Green Goo & New Reality
🧪 Scene 2 – Green Goo & New Reality
The detective begins in his office. A message appears from his boss with an urgent mission: a missing scientist with critical secrets. This scene uses a text messaging UI to build narrative immersion.
🟣 UX Goal: Ground players in story context while setting up STT as a future gameplay mechanic.
The detective begins in his office. A message appears from his boss with an urgent mission: a missing scientist with critical secrets. This scene uses a text messaging UI to build narrative immersion.
🟣 UX Goal: Ground players in story context while setting up STT as a future gameplay mechanic.
🕵️ Scene 1 – The Briefing
🕵️ Scene 1 – The Briefing
After going through several rounds of research, brainstorming, prototyping, and iteration, our team arrived at a low-fidelity game that simulated the experience of relying on speech-to-text (STT) technology. Below is an overview of the final scenes and the key user experience goals behind each one.
Players encounter a voice recording with a vital message from the scientist, but background noise corrupts the audio. Using the STT device produces a broken transcript, which the player must decode.
🟣 UX Goal: Simulate real-world STT failures due to noise and make users experience the cognitive load of deciphering misheard speech.
🎙️ Scene 3 – Garbled Recording
🎙️ Scene 3 – Garbled Recording
The detective texts his partner for info on Dr. Reid, but the green goo affects his hands — he can no longer type. He switches to sending an audio message using the STT device.
🟣 UX Goal: Simulate how physical impairments force users to rely on speech input for communication.
The detective texts his partner for info on Dr. Reid, but the green goo affects his hands — he can no longer type. He switches to sending an audio message using the STT device.
🟣 UX Goal: Simulate how physical impairments force users to rely on speech input for communication.
📱 Scene 4 – Switch to STT
📱 Scene 4 – Switch to STT
User testing didn’t just improve existing interactions, it also helped us expand the game by identifying areas where players wanted more clarity, challenge, or variety.
User testing didn’t just improve existing interactions, it also helped us expand the game by identifying areas where players wanted more clarity, challenge, or variety.
Feedback That Inspired New Scenes
Focus on experience over difficulty: Don’t stress about making the puzzles “hard” or “easy”, what matters is that they reflect the real struggle of interpreting faulty STT transcriptions.
The mechanic is the message: The act of figuring out what someone meant to say , even when the system gets it wrong, is the core experience.
Puzzle = empathy: Turning speech-to-text struggles into puzzle mechanics effectively recreates what many real users go through.
No need for overcomplication: The mechanic (e.g., combining settings to get a correct transcription) speaks for itself, it doesn’t need extra layers like a “Speak” button if it adds friction.
Key Takeaways
We also received supportive and constructive feedback from peers and instructors that helped us clarify our focus and feel confident in the game’s direction:
We also received supportive and constructive feedback from peers and instructors that helped us clarify our focus and feel confident in the game’s direction:
DESIGN VALIDATION & DIRECTION
The detective learns the STT device is incomplete. Players test different gears—one adds emojis, one translates to French, one fixes punctuation—to restore proper transcription.
🟣 UX Goal: Show how missing features in STT devices can distort meaning and affect usability.
The detective learns the STT device is incomplete. Players test different gears—one adds emojis, one translates to French, one fixes punctuation—to restore proper transcription.
🟣 UX Goal: Show how missing features in STT devices can distort meaning and affect usability.
Players were curious if the STT device could malfunction or be incomplete → We added a scene where you must repair the STT device by testing different gears.
At a voice-activated lock, players must choose the right mic sensitivity and dialect settings before saying the password. Only correct calibration unlocks the vault.
🟣 UX Goal: Teach how system settings directly impact transcription accuracy and user success.
At a voice-activated lock, players must choose the right mic sensitivity and dialect settings before saying the password. Only correct calibration unlocks the vault.
🟣 UX Goal: Teach how system settings directly impact transcription accuracy and user success.
Testers wanted to better understand how settings impact STT accuracy → We added a voice lock puzzle where players calibrate mic sensitivity and dialect mode.
🎛️ Scene 6 – Calibration Challenge
🎛️ Scene 6 – Calibration Challenge
FULL GAMEPLAY WALKTHROUGH
FULL GAMEPLAY WALKTHROUGH
What I Learned
How to design gameplay mechanics based on real user research
The importance of testing and iteration to simplify complex concepts for players
How to use puzzle design to simulate the real cognitive load experienced by STT users
That frustration can be a design tool—when used carefully to create understanding, not just difficulty
What I Learned
How to design gameplay mechanics based on real user research
The importance of testing and iteration to simplify complex concepts for players
How to use puzzle design to simulate the real cognitive load experienced by STT users
That frustration can be a design tool—when used carefully to create understanding, not just difficulty
What We Achieved
Created a narrative game that helped players feel the frustration of unreliable speech-to-text technology
Balanced entertainment with educational value without turning the game into an “edugame”
Successfully communicated the importance of STT accuracy, configuration, and accessibility through mechanics
What We Achieved
Created a narrative game that helped players feel the frustration of unreliable speech-to-text technology
Balanced entertainment with educational value without turning the game into an “edugame”
Successfully communicated the importance of STT accuracy, configuration, and accessibility through mechanics
This project pushed us to think beyond usability and dive into empathy-driven design. By turning real-world accessibility challenges into interactive puzzles, we created an experience that was both meaningful and engaging.
OUTCOMES AND LEARNINGS
