Inspired by the team’s love for salsa and dance, Studio Rumba is an expression of two different passions: movement and machines. The team wanted to explore how we can co-create — or even dance  — with machines. 

Studio Rumba is a Machine Learning-based dance teacher that helps users dance salsa and find their groove. Designed for those who’ve never danced salsa, Studio Rumba breaks down a primary salsa step to help users learn the basics. 

The project uses PoseNet machine learning technology to track user’s movements. The ML "teacher" tracks the user's movement, instructing them to move their right and left feet in time with the music. For example, if the user steps to the right, Studio Rumba reads that they’ve moved their left leg and instructs them to move their leg back to the center position. 

The project set-up consists of a split screen monitor that displays a live video of the dancer with projected PoseNet outlines and a looping recorded video of a dancer doing a basic salsa step. The user is able to look at both the looping dance video and PoseNet projection of themselves as they learn the step.

When the user has successfully mastered the moves after ten consecutive steps, Studio Rumba will reward their student with the presence of famed singer, Celia Cruz. 

The team used different tools to bring this project to life: Google’s Teachable Machine, Runway, ML5, and p5. The challenges and joys of Studio Rumba were figuring out how to stitch these codes together to a multi-sensory experience. 

The team would love to continue to build on this project, by creating training for other dance moves and integrating ‘dynamic time warping’ to track dancer's movement in real-time.