• 3855 SW 153rd Drive, Beaverton, OR 97006
  • hello@hapticsif.org
Standards

Haptic Industry Forum strategy for haptic standards

Current state of the haptic ecosystem

Human-computer interaction is increasingly utilizing multiple modes of simultaneous sensory feedback that includes video and animation for the eyes, audio for the ears, and haptics for the sense of touch. The increasing use of haptics has led to growing awareness and demand for haptic technology and content. However, haptics faces unique challenges. There exists no common media format or tooling to enable content creators, no standard platform APIs to enable developers to support haptic hardware, and no standard mechanical configurations for OEMs to drop into their devices.

Because these problems are interdependent, most products with haptics are vertically integrated with components that cannot interoperate with other systems. For experience creators who are used to content being platform-neutral, the haptic landscape is highly fragmented and inconsistent, which impedes creative exploration of the use of haptics to make novel user experiences.

Our vision for haptic standards

  1. Standards for haptic representation and coding enable content creators to create haptic content and monetize it across a wide range of end-user devices. Content creators can expect that the end-user experience will preserve the creative intent behind the haptics.
  2. Standards for device manufacturers allow them to showcase rich haptic experiences on their devices and rely on a robust pipeline of compelling content to support device sales.
  3. Haptic standards enable broader adoption of haptics in the mass-market overall, which grows the ecosystem and the commercial opportunities of the entire market.

In the domains of audio and video, standards have been developed that simultaneously enforce baseline fidelity levels and define a design space for creative exploration. Such standards exist for media coding, platform APIs, and physical components. These standards enabled a rich ecosystem of content creators who could rely on a large enough content market to justify the cost of production. The existence of this market in turn triggered investment in design tools, content libraries, streaming technologies, and so on, and has resulted in a robust standards-based media ecosystem.

Agenda item 1: Haptic experience encoding

At the founding meeting of the Haptics Industry Forum in December of 2019, the assembled industry experts aligned on a common need to provide a standard mechanism for abstracting devices and platforms from content. Since that meeting, Immersion has led the development of content standards in the context of MPEG, with these results:

  1. Haptics is being considered as a first-order media type, along with video and audio. This means that MPEG files could contain just haptics, haptics and audio, or any other combination of first-order media tracks. This activity is in ballot and may become formally adopted as early as October of 2021.
  2. MPEG has issued a call for proposals for a coded representation of haptic signals. This will become a component of the MPEG-Immersive multi-year program.
  3. MPEG has mandated an ad-hoc group to develop requirements and eventually a call for proposals for coded representation of interactive haptic experiences. This will become a component of the MPEG-Immersive multi-year program.
  4. Haptics Industry Forum is seeking to become a Liaison to the International Standards Organization (ISO) to facilitate contributions from HIF members to MPEG. All MPEG ad-hoc group activities are open to participation from any people with expertise in haptics, regardless of their membership status in ISO or MPEG.

This standard will enable interoperable content encoding which will allow the ecosystem to develop standardized haptic libraries and creation tools that can be used by designers in multiple markets. This will reduce the barriers to entry for haptic design and enable more and more impactful haptic experiences.

Agenda item 2: Haptic platform API

There is no standard platform API that allows developers to access the functionality of a broad range of haptic hardware without writing device-specific code. The development of a broadly applicable device protocol would encourage abstraction between end-user experiences and the physical hardware that renders them. The haptics ecosystem needs to define common APIs with standard functionality for applications running on Android, iOS, Windows, and other platforms.

Haptics Industry Forum has recognized the need to drive these APIs and is engaged in a Liaison relationship with Khronos Group, specifically targeting the OpenXR and glTF working groups. The goal of entering into this relationship is to ensure OpenXR has sufficient API functionality and abstraction to support a wide range of hardware capabilities. HIF will also engage the IEEE P2861.3 group working on mobile platform API and presentation standards for haptics.

This standard will reduce the artificial barriers between different products by providing a consistent API that developers and toolmakers can rely upon.

Agenda item 3: Device protocol

On a haptic peripheral, the communication between the microcontroller and the overall system commonly uses USB-defined HID (human interface device) protocols. Although there is a HID protocol for haptics that was ratified in 2016, it is primarily intended for single haptic effects that are triggered by a single event; it does not contemplate streaming of haptic effects, multiple channels of haptics, synchronization of haptics with other modalities, high-definition effects, kinesthetic effects, and so on.

An extensible device protocol is needed to support legacy devices such as game controllers with rumble features, haptic mice, and so on, as well as near-future devices that can provide an endpoint for platform APIs and rich MPEG content. The existence of such a protocol will enable 3rd party peripheral vendors to compete and innovate to generate amazing new peripherals that can immediately access experiences already developed for the current best-of-breed peripherals.

Haptics Industry Forum has recognized this need and formed a HID Working Group in April of 2021. This group is expected to register a new haptic usage page for HID before June 30th, 2021.

This standard will enable plug-and-play compatibility of disparate haptic peripherals and reduce the cost of implementing advanced haptic functionality for device makers. This will enable innovative and competitive hardware to come to market that meaningfully leverages existing content but also provides new experience opportunities for content creators.

SDO-driven standards initiatives

There are several initiatives being developed by standards development organizations that have relevance to haptic technology. Below is simplified overview of the standards in development that relate to PC/XR and to mobile devices. Haptics Industry Forum and its members are actively involved in collaborating with SDOs on many of these.

How you can contribute

If you are a member of the haptics ecosystem and interested in contributing to these activities, you can request to become member of the Haptics Industry Forum on our Membership page.

MPEG Call for Proposals for the Coded Representation of Haptics – Phase 1

Below is the document suite for the MPEG Call for Proposals for the Coded Representation of Haptics – Phase 1