Using AR technology to bring a virtual pianist to life with a real piano.



You can direct the camera of your phone to various pianos and almost instantly create an AR pianist who can play works from different performers, while sheet music of the corresponding notes scrolls across the bottom in real-time.

The app uses machine learning to detect the piano, and then figures out the exact 3D position and orientation of your piano in a 3D space down to 1 cm accuracy. It does this just by looking at a single frame and without the use of 3D depth cameras.

This AR application works with any model or brand of piano under any lighting conditions, without the need to use special targets, image markers or the need to scan the piano model in advance.

This application will help many users in music education. Any user will be able to slow down the song up to 100 times without losing information on the movement of the hands, which is impossible with a traditional recorded video.

The full version of the application will also use the microphone of your smartphone to listen to you while you practice, and through machine learning, give you instant feedback on your playing; plus use the AR pianist to offer you tips on how to exactly move your hands during your performance.

Back to all news

Innovations in AR

AR will become one of the main marketing tools in 2020
30.03.2020

AR will become one of the main marketing tools in 2020

Read  
The global giants of the mass market are increasingly using AR.This time Pepsi.
30.03.2020

The global giants of the mass market are increasingly using AR.This time Pepsi.

Read  
HTC held to host annual conference in VR to mitigate coronavirus risk
30.03.2020

HTC held to host annual conference in VR to mitigate coronavirus risk

Read  
AR Museum at your home
30.03.2020

AR Museum at your home

Read  
Lebron James scores a point with the help of augmented reality
20.02.2020

Lebron James scores a point with the help of augmented reality

Read