Human-computer interaction is increasingly utilizing multiple modes of simultaneous sensory feedback that includes video and animation for the eyes, audio for the ears, and haptics for the sense of touch. The increasing use of haptics has led to growing awareness and demand for haptic technology and content. However, haptics faces unique challenges. There exists no common media format or tooling to enable content creators, no standard platform APIs to enable developers to support haptic hardware, and no standard mechanical configurations for OEMs to drop into their devices.
Because these problems are interdependent, most products with haptics are vertically integrated with components that cannot interoperate with other systems. For experience creators who are used to content being platform-neutral, the haptic landscape is highly fragmented and inconsistent, which impedes creative exploration of the use of haptics to make novel user experiences.
In the domains of audio and video, standards have been developed that simultaneously enforce baseline fidelity levels and define a design space for creative exploration. Such standards exist for media coding, platform APIs, and physical components. These standards enabled a rich ecosystem of content creators who could rely on a large enough content market to justify the cost of production. The existence of this market in turn triggered investment in design tools, content libraries, streaming technologies, and so on, and has resulted in a robust standards-based media ecosystem.
At the founding meeting of the Haptics Industry Forum in December of 2019, the assembled industry experts aligned on a common need to provide a standard mechanism for abstracting devices and platforms from content. Since that meeting, Immersion has led the development of content standards in the context of MPEG, with these results:
This standard will enable interoperable content encoding which will allow the ecosystem to develop standardized haptic libraries and creation tools that can be used by designers in multiple markets. This will reduce the barriers to entry for haptic design and enable more and more impactful haptic experiences.
There is no standard platform API that allows developers to access the functionality of a broad range of haptic hardware without writing device-specific code. The development of a broadly applicable device protocol would encourage abstraction between end-user experiences and the physical hardware that renders them. The haptics ecosystem needs to define common APIs with standard functionality for applications running on Android, iOS, Windows, and other platforms.
Haptics Industry Forum has recognized the need to drive these APIs and is engaged in a Liaison relationship with Khronos Group, specifically targeting the OpenXR and glTF working groups. The goal of entering into this relationship is to ensure OpenXR has sufficient API functionality and abstraction to support a wide range of hardware capabilities. HIF will also engage the IEEE P2861.3 group working on mobile platform API and presentation standards for haptics.
This standard will reduce the artificial barriers between different products by providing a consistent API that developers and toolmakers can rely upon.
On a haptic peripheral, the communication between the microcontroller and the overall system commonly uses USB-defined HID (human interface device) protocols. Although there is a HID protocol for haptics that was ratified in 2016, it is primarily intended for single haptic effects that are triggered by a single event; it does not contemplate streaming of haptic effects, multiple channels of haptics, synchronization of haptics with other modalities, high-definition effects, kinesthetic effects, and so on.
An extensible device protocol is needed to support legacy devices such as game controllers with rumble features, haptic mice, and so on, as well as near-future devices that can provide an endpoint for platform APIs and rich MPEG content. The existence of such a protocol will enable 3rd party peripheral vendors to compete and innovate to generate amazing new peripherals that can immediately access experiences already developed for the current best-of-breed peripherals.
Haptics Industry Forum has recognized this need and formed a HID Working Group in April of 2021. This group is expected to register a new haptic usage page for HID before June 30th, 2021.
This standard will enable plug-and-play compatibility of disparate haptic peripherals and reduce the cost of implementing advanced haptic functionality for device makers. This will enable innovative and competitive hardware to come to market that meaningfully leverages existing content but also provides new experience opportunities for content creators.
There are several initiatives being developed by standards development organizations that have relevance to haptic technology. Below is simplified overview of the standards in development that relate to PC/XR and to mobile devices. Haptics Industry Forum and its members are actively involved in collaborating with SDOs on many of these.
If you are a member of the haptics ecosystem and interested in contributing to these activities, you can request to become member of the Haptics Industry Forum on our Membership page.