Gesture Quest
Gesture Quest is a semester long project at Entertainment Technology Center at Carnegie Mellon University. The project aims to develop an educational game dedicated to teaching American sign language through entertaining and engaging experiences.
The program utilizes Google Mediapipe for hand marker tracking and machine learning for static and motion hand gesture recognition, which are then communicated from Python to Unity as game input.
-
Time: Jan. 2024 - May 2024
-
Team Project
-
Role: Programmer, 3D Artist, Storyboard/Comic Artist
-
-
Software and Technology: Maya, Unity, Python, Google MediaPipe
Official Project Website: ETC Questure Website
Full Game Demo
Game Concept Design
The project consists of 3 mini games, each focusing on a different aspect of ASL learning.
Mini 1 - Tutorial Level
The player learns sign language letters to pop the balloon and feed the cat.
-
Slow pace, low pressure
-
Encourage accuracy and memorization
Mini 2 - Speed Level
The player learns sign language letters to remove the obstacle in front of the cat
-
Requires fast reaction
-
Encourage repeated practice of learned alphabet
Mini 3 - Application Level
The player spell out words using the learned alphabet to earn a little stamp from the cat.
-
Encourage reinforcement of learned alphabet
