"This approach will enable the robots to play along with human musicians seamlessly"
The primary motivation for our research is to explore the feasibility of creating a rock band that does not rely on intricate mechanical designs. By utilizing our existing humanoid robots, which are capable of performing various instruments together, we aim to develop a system that is both robust and versatile.
In the Press
Humanoid robots join human musicians for synchronized musical performances
When Machines Play in Perfect Harmony: Humanoid Robots Perform with Human Musicians
Newo.ai
.
Humanoid robots and human musicians jam together to perfect future symphonies
“This new work helps us understand the ecological relationships between dinosaurs in the Jurassic and reveals that the habits of the larger carnivores then were closer to that of the tyrannosaurs than previously thought. It’s another important step in reconstructing the behaviour of these ancient animals.”
Prof. MengCheng Lau
For All Readers - AI Explainer
What is this research about?
This research explores how humanoid robots can perform music alongside human musicians in a synchronized and natural way. The goal is to create a robot-human musical band where robots and humans collaborate seamlessly during performances.
Why is this important?
Humanoid robots are becoming more advanced in entertainment, but true synchronization and collaboration with humans in live performances remains a challenge. By improving robots’ ability to follow rhythm, adjust to musical timing, and interact with human performers, this research pushes the boundaries of what robots can do in creative fields like music.
Which robots were used, and what were their roles?
Two humanoid robots were part of the musical band:
- Polaris – A mid-sized humanoid robot that played the drums.
- Oscar – A Robotis-OP3 humanoid robot that played the keyboard.
These robots worked alongside human musicians to create a synchronized performance.
How did the researchers make the robots play music with humans?
The technical setup included:
- Visual servoing – Using cameras and sensors to track movement and timing.
- Human-robot interaction – Ensuring the robots could respond to human musicians.
- Robot Operating System (ROS) – A framework that allowed the robots to communicate and coordinate their actions smoothly.
By combining these technologies, the robots could adjust their timing and movements in real-time to stay in sync with the human musicians.
What were the main achievements of this research?
The project successfully created a collaborative musical performance where robots and humans played together in harmony. The result was a demonstration of real-time synchronization, which can be seen and heard in the following video: YouTube link.
What are the next steps for this research?
The team aims to further improve:
- Synchronization – Enhancing the robots’ ability to predict musical timing more accurately.
- Musical improvisation – Allowing robots to respond dynamically to changes in the music, similar to human musicians.
- Beat recognition – Helping robots detect and follow rhythm more effectively while minimizing self-generated noise.
- Social interaction – Equipping robots with the ability to interpret musical notes and engage meaningfully with audiences.
The ultimate goal is to make humanoid robots more expressive, reliable, and engaging in live performances, sparking public interest in robotics through the universal language of music.
What are Article Spotlights?
Spotlighted articles are press released, and feature author interviews, AI explainers and more.
If you have published in Peer J and would like to be featured in an Article Spotlight please contact PeerJ.