Using AR technology to bring a virtual pianist to life with a real piano.



You can direct the camera of your phone to various pianos and almost instantly create an AR pianist who can play works from different performers, while sheet music of the corresponding notes scrolls across the bottom in real-time.

The app uses machine learning to detect the piano, and then figures out the exact 3D position and orientation of your piano in a 3D space down to 1 cm accuracy. It does this just by looking at a single frame and without the use of 3D depth cameras.

This AR application works with any model or brand of piano under any lighting conditions, without the need to use special targets, image markers or the need to scan the piano model in advance.

This application will help many users in music education. Any user will be able to slow down the song up to 100 times without losing information on the movement of the hands, which is impossible with a traditional recorded video.

The full version of the application will also use the microphone of your smartphone to listen to you while you practice, and through machine learning, give you instant feedback on your playing; plus use the AR pianist to offer you tips on how to exactly move your hands during your performance.

Back to all news

Innovations in AR

Auto Giant Uses AR for Promotion
03.09.2020

Auto Giant Uses AR for Promotion

Read  
AR Marketing Goes To Space
03.09.2020

AR Marketing Goes To Space

Read  
New AR Hit from the Pokemon GO Developer
03.09.2020

New AR Hit from the Pokemon GO Developer

Read  
Bbox Sports Has Released an ARApp for Football Fans
03.09.2020

Bbox Sports Has Released an ARApp for Football Fans

Read  
Auto Giant Uses AR for Promotion
02.09.2020

Auto Giant Uses AR for Promotion

Read