>> Electronic Design Resources
.. >> Library: Article Series
.. .. >> Topic: System Design
.. .. .. >> Series: Touching on Haptics
With new use cases in markets such as gaming, Internet of Things, PC, and virtual reality, there’s an increasing demand for richer haptic experiences. High-definition (HD) haptics, with effects ranging from subtle to sharp, textured effects that simulate different surfaces and sensations, and increasingly efficient vibration motors, are becoming the norm in many products already on the marketplace.
As excited as we all are about haptic technology coming into its own, it’s difficult to ignore a serious problem that comes with this increased adoption—market fragmentation. Different proprietary APIs, different hardware-to-software interfaces, and different motor characteristics effectively create “walled gardens,” making it difficult for application developers to incorporate haptics easily into their applications. Worse, they must repeat the exercise for multiple platforms and devices.
Changes to lower levels of the haptic stack (e.g., replacing the vibration motor) typically necessitate changes to the upper levels (e.g., the codecs and APIs used). The high cost of such changes often discourages OEMs and stifles innovation. This ultimately hurts adoption by end users, the most important link in the haptic value chain.
Need for Standardization
This is where standardization can help. Standardized APIs, standardized evaluation criteria for haptics, and standardized vibration motor performance ranges, to name a few, can only help harmonize haptic offerings from different vendors—without compromising vendor differentiation. This, in turn, will incentivize content and application creators to incorporate richer haptic experiences into their offerings, leveraging the standard interfaces throughout the stack.
Current walled gardens will give way to a flourishing haptic ecosystem that benefits all stakeholders. In other words, the time is indeed right for standardization across all layers of the haptic technology stack (Fig. 1). Also check out this article on the haptic stack.
In Figure 2, the haptic technology stack is annotated with the opportunities for standards at each layer, along with the questions that standardization is likely to address. Several international standards development organizations (SDOs) would be ideal forums for standardizing various parts of the haptic stack.
However, standardization is really a community-based activity that goes beyond SDOs. It requires buy-in from all members of the haptic community across the entire value chain to drive the standards, regardless of the SDO. A standard that best reflects the interests of all stakeholders, regardless of their role in the value chain, has the best chance of succeeding in the marketplace. To this end, it’s important that the haptics community provides input on the questions that need to be addressed in standardization. The list of questions in the diagram should be considered as just a starting point for further discussions.
Haptic Technology Stack and the Opportunities for Standardization
To be clear, some ratified haptic standards are already in existence. However, none of them specifically pertain to the haptic technology stack, which is the focus of this article. By creating haptic standards based on the haptic technology stack, we’re acknowledging the interdependency of the entire system in order to deliver consistent and high-quality end-user experiences. This is a shortcoming of existing standards efforts. Briefly:
- MPEG-V: This standard provides an architecture and specifies associated information representations to enable the interoperability between virtual worlds (such as in gaming, AR/VR, etc.). Part 3 (Sensory Information) addresses haptics, among several other sensory effects.
- SMPTE ST 2100-1:2017: This standard deals with the definition and representation of haptic-tactile essence for broadcast production applications.
- ISO 9241-940:2017: This standard deals with the evaluation of haptic and tactile interactions with devices such as 2D and 3D gesture sensors, desktop haptic interfaces, and tactile mobile displays (phones).
While we may want to approach these SDOs at a later date to integrate the new haptic standards, other SDOs may be a better fit for what we’re trying to accomplish with standardization of the haptic stack. Below is a brief overview of these SDOs:
MPEG
MPEG (Motion Pictures Expert Group) needs no introduction, as it’s the SDO responsible for standards such as MP3, AAC, MP4, etc., that pervade our digital media lives. Immersion is working on an MPEG4+HAPT specification proposal to introduce a ‘hapt’ box into the ISO Base Media File Format (ISO BMFF). The company plans on presenting this proposal at the MPEG-130 meeting in Alpbach, Austria. Adding a ‘hapt’ box and related fields to ISO BMFF will allow content creators to more easily incorporate one or more timed haptic tracks into their media files. A MPEG4+HAPT standard is an important first step in making haptics ubiquitous in all media files.
IEEE
The IEEE Standards Association has an ongoing standardization effort called P1918.1 (Tactile Internet Working Group) that focuses on the Tactile Internet as a 5G and beyond application across a range of different user groups. A sub-group of P1918.1, called P1918.1.1 (Haptic Codecs for the Tactile Internet), focuses on standardizing haptic codecs for both open-loop (vibrotactile) and closed-loop (kinesthetic) communication in the Tactile Internet. Perceptual evaluation of haptic codecs is also under the purview of this sub-group. We’re actively working with both groups to start the standards discussion.
ATSC
ATSC (Advanced Television Systems Committee) recently released its next-generation TV standard, called ATSC 3.0, which is a quantum leap compared to the existing ATSC 1.0 (and ATSC 2.0) standards that run our existing digital TV sets. We’re not able to get into all of the different aspects of the ATSC 3.0 standards suite in this article. However, it’s important to point out that one of the promises of ATSC 3.0 is the ability to view TV broadcasts as well as supplemental content on “companion devices” (i.e., smartphones and tablets) along with “primary devices” (regular ATSC 3.0-compliant TV sets).
Since nearly all companion devices come with haptic actuators built into them, there is a real opportunity here to enhance the viewers’ experience with haptics when they’re watching ATSC 3.0 content on their companion devices. Live sports, feature films, music videos, etc. can all benefit from sophisticated (and customizable) haptics when played back on these devices.
HD Motor Performance Standards
While HD motor specifications themselves are best left to the motor manufacturers, motor performance characteristics can benefit from standardization so that upper layers of the haptic technology stack can get a level of assurance of motor performance.
Immersion has developed its own HD motor performance criteria to ensure that certain performance levels are met before they are qualified as HD. It’s been used as the baseline to build an understanding between the company, its partners, and its customers on the level of experience to expect. Lofelt has also published a specification (VT-1) that provides parameters and requirements for creating high-quality, realistic user experiences through vibrotactile (VT) haptic feedback. These two documents are good starting points for the development of a standard for HD motors.
Call to Action
The standardization of haptics has the potential to lead to better experiences, faster integration, and greater adoption. Now is the right time for it, as more consumer-facing products (automobile dashboards, smart appliances, advanced game controllers, AR/VR headsets) incorporate HD haptics. The success of any standardization effort depends on the active participation of all stakeholders—customers, business partners, technology vendors, and market competitors alike. It can’t be done alone. We at Immersion hope that other members of the haptics community will join us in this effort.
Yeshwant Muthusamy, Ph.D., is Senior Director of Standards at Immersion Corp.
>> Electronic Design Resources
.. >> Library: Article Series
.. .. >> Topic: System Design
.. .. .. >> Series: Touching on Haptics