There’s been lots of talk about VR and 5G as the perfect combination when it comes to cloud gaming services in the future. But with VR still struggling to be anything more than a fun but niche way to enjoy games and movies, you can understand why we wanted to experiment with some other technologies.
That’s why Blacknut was happy to partner with InterDigital and Nvidia to do R&D work to develop this industry-first: a way to control cloud-based streaming games with gestures and body movement.
How does it work? By taking a video feed of a gamer's movements and processing them in the cloud using AI to convert the video into vectors, coordinates and motion data. The collected data can then be applied to control the game that's also streaming from the cloud, either on a phone or PC, without having to use or purchase any additional equipment.
In our proof of concept a player can control a snowboarding avatar on-screen in almost real-time, leaning left and right to steer, and making moves that are mirrored in the game.
Of course, we can remember how Microsoft launched the Xbox Kinect, while rival Sony tried the Playstation Move. Both were quite expensive at the time and only worked with a limited number of games. This project with InterDigital and Nvidia gets rid of all the boxes and cables and makes it happen right on the same TV you might have at home.
Blacknut’s mission is to bring to our customers unlimited hours of gaming fun in the simplest manner. Our unique cloud gaming solution allows to free games from dedicated consoles or hardware. Using AI and machine learning to transform the human body itself in a full-fledged game controller was challenging but Blacknut’s close collaboration with Interdigital and NVidia led to outstanding performances. And yes, it is addictive and fun to play this way!
Said Pascal Manchon, CTO at Blacknut.
This collaboration has created a world-first. We are excited about how this video-based AI technology will be developed further in the future, although it is just one of the many R&D areas we are currently working on, in addition to new voice, adaptive and accessibility solutions. We hope all these developments will make the gamers’ experience even richer and more engaging - while also potentially unlocking new gaming creative opportunities. Stay tuned for more!