Haptic Foviated Rendering within Unreal Engine for Human Robot Interfaces
Touch enabled robotic interfaces for simulation is a complex multidisciplinary field that exists at the intersection of robotics, gaming, and biology. At Haply, we are developing new force-feedback controllers into physical simulation consoles that will enable surgeons to train in virtual and augmented reality. The surgical simulation market is currently $1B and is growing at a compound annual rate between 17% and 22% per year and is expected to reach $30B by 2050.
There are still many hurdles to overcome. One such hurdle is that when running real-time simulation with force-feedback, we must control the haptic interface at rates greater than 1000Hz. This means that the complex virtual surgical environment, often consisting of large deformable meshes, has to solve the simulation physics and return values to the haptic system at millisecond timescales. This puts an incredible demand on the simulation engine and computation resources.
There has been research into this area to overcome the challenges. We have been in discussions with a world expert in "cosimulation“ and "reduced order modelling", at the leading university in Canada, McGill, to carry out a 3-year project where we explore these techniques and pioneer the implementation of novel software architectures to share data across different sample data systems in an intelligent way to reduce the computational complexity required to generate high fidelity haptic rendering.
This is a profound research topic as it is fundamental to the merging of the physical and virtual world as these requirements are determined by our sensory perception and physiology.
Today we are selecting a simulation engine to move forward with this project on as we will take the next 24-36 months to target development on a specific engine, either the Unreal Engine or Unity.
We believe a contribution from Epic Games to help us fund the project will bring a significant benefit to the Unreal Engine as well as to Haply. The project will generate meaningful IP that will be integrated into the Unreal Engine, which has the potential to bring significant improvement to force-feedback performance as we pioneer touch enabled technologies in enterprise. In addition, the project, anticipated to have upwards of 6 post-doctoral fellows, PhDs, and Masters students will generate new experts in the highly skilled intersecting domains of simulation physics and the Unreal Engine. As we are located in Montreal and Epic Games has a growing division here, these individuals would be highly capable of jumping into Epic’s hiring pipeline and are likely to become future Epic Games employees. And finally, by building out of product and the requisite SDK on the Unreal Engine, we will facilitate the creation of cutting-edge surgical simulation software in the Unreal Engine using our physical simulation console. This will lead to revenue generation for Epic Games as the simulation industry grows to higher levels of prevalence in our 21st century lives and beyond.