HMD system

A pilot flying a combat mission requires a high level of situational awareness to maintain safe altitude and airspeed, while avoiding obstructions, evading enemy weapons and, at the same time, executing the mission. To achieve the mission success in such a scenario the pilot may not risk the situational awareness while looking at flight critical instruments in the cockpit very frequently. Therefore to keep the pilots attention intact, militaries around the globe utilize the advanced helmet mounted display (HMD) technology.

A HMD system provides information that increases situational awareness because the pilot does not need to break attention from the outside scene to look at instruments in the cockpit. Although this same capability exists in a head-up display (HUD), the fixed nature of this instrument requires the pilot to look forward to see the symbology. The future could also see the Head Up Display (HUD) fitted on the aircraft being replaced by sophisticated smart helmets that offer all the information provided by the HUD along with information from other sensors on the aircraft for increased situational awareness. HMD systems are now in widespread use on combat aircraft, delivering that vital edge when engaging a target with highly agile short range missiles within visual range arena. With a helmet mounted sight the pilot just has to look at the target to designate it and launch a missile.

In the past few decades, technological advances have significantly improved the design and engineering of aircraft. These developments have contributed to make pilot systems more complex and, hence, have increased the demand on the human operator. To add to this, pilots often fly in degraded visual environments. Poor visibility due to adverse weather or stirred up dust or sand during landing or take-off can induce spatial disorientation, high workload, and loss of situation awareness. These factors pose a major problem for military pilots.

To counter situation of accidents in such a scenario there is a need for enhanced visual support, which is these days achieved by superimposing information on a helmet-mounted display system (HMD).

Early role

The first devices in this category emerged during the late seventies, as an aid to targeting second generation heat seeking missiles. Given the limitations of both sight and missile technology of that period, the HMS slipped into obscurity for several years, only to be resurrected with the advent of fourth generation heat seeking missiles (WVR AAMs). At this time the HMS and newer, more capable HMDs are seeing a resurgence in the marketplace and have now become a standard feature in the cockpit of any new build fighter aircraft.

The various militaries across the world have actively pursued the research, development, application, and fleet introduction of a variety of helmet-mounted technologies for decades. The concept and the potential applications of HMDs in aircraft cockpits have fascinated military aviation strategists for decades. The idea of placing a virtual image focused at infinity in the visual path of the pilot and overlaying computer-generated images so that mission critical information is always available with “eyes-out,” has mobilized incredible technical and financial resources over the last decades. It is generally acknowledged that an HMD, when part of a Visually Coupled System (VCS), is among the most valuable visual aids in the arsenal of a military pilot.

Outside the United States, the first “modern” helmet-mounted sight (HMS) was the optically-sensed Russian design, developed to support the Vympel R-73/AA-11 Archer high off-boresight seeker, air-to-air missile, carried by the MiG-29 Fulcrum and the Su-27 Flanker, and built to attach to the ZSh-5 series Russian helmet.

Displays and their ability to provide information are a distinct advantage in any operational setting. There are operational settings where certain displayed information is critical on a second-to-second basis. For example, in fast-moving aircraft flying close to the ground, the operational environment changes so rapidly that even the brief time it takes a pilot to glance down at one or more displays to obtain aircraft flight status information may severely degrade his situation awareness. This short-coming of “head-down” displays gave rise to the development of head-up displays (HUDs) or HMDs.

Using the HMD, the pilot can quickly “tag” the enemy aircraft, launch a missile, and then turn to the next target and repeat the procedure. Sequential targeting enables a pilot to deal with multiple threats simultaneously, by eliminating the limitation posed by aircraft maneuverability.

The concept

The HMD concept can be extended and transferred to other areas where a wide field-of-regard is beneficial. While early HMD development was aviation driven, their utility beyond aviation has not been overlooked. Tank commanders can benefit by staying in touch with the “outside world” while remaining protected. Dismounted soldiers can also maintain constant situation awareness of the digital battlefield as well as expanded and enhanced sensory inputs via HMDs.

For this reason, HMDs offer potentially greater direct access to critical visual information, while offering greater flexibility of head movement, less total system weight, and greater flexibility in use of vehicular interior space. More importantly, it is argued that HMDs provide users with increased situational awareness. Situational awareness encompasses the total information available, used to create an accurate picture of a battle theater, including spatial position and orientation of the aircraft, the surrounding areas, and any aircraft-relevant information. The pilot has to be aware of many different forms of information which is used to make judgments on how to respond to a given situation, which can be vital for success in most situations.

By centralizing critical flight information within a user’s line-of-sight, overall performance is increased and operational safety is enhanced. HMDs offer users the advantage of monitoring critical information without having to repeatedly look down to scan instrument displays.

Over the years, the military has interfaced helmet-mounted sights and HMDs to a wide variety of vehicle systems and weapons. They have been linked with radars, electro-optical/TV missile systems, reconnaissance sensors, long-range target identification sensors, pilotage sensors, head-slaved guns (both air-to-ground and surface-to-air), and angle-rate bombing sensors. They have been interfaced with distributed aperture sensor systems for a total coverage “windowless cockpit” synthetic vision system capability for both aircraft and ground vehicles. They have been used to present spatially-referenced “highway-in-the-sky” type flight control information for both fixed-wing ejection seat aircraft and rotary-wing operations and for shipboard landings, and to present “predictor” fire control dynamic symbology.

When considering HMD implementation, it is crucial to take into account numerous human-factor issues, including attentional, perceptual, visual, and somatic aspects. Positioning elements relevant for a specific task in close spatial proximity enhances task performance. With regard to HMDs, this is one of the basic ideas behind superimposing instrument information on the pilot’s forward field of view.

An HMD is composed of a modulated light source with drive electronics viewed through an optical system, which, combined with a housing, is mounted on a user’s head via a headband or a helmet. The positioning of light sources, optics, and optomechanics with respect to the head inflicts tight requirements on the overall system design. Moreover, to create appropriate viewpoints to the users based on their head position and possibly gaze point, visual coupling systems (i.e., trackers) must be employed.

Many head-mounted displays include speakers or headphones so that it can provide both video and audio output.  HMDs almost always include a tracking device so that the point of view displayed in the monitors changes as the user moves his head.

Applications

Because of their broad application domains, HMDs are designed and targeted at key applications and task specification, and are tested with the targeted end users for advanced tasks. Beside military applications that dominated the market of HMDs for several decades, more recent applications include medical, user interface design, visual aid for everyday life, manufacturing, and distributed collaborative environments.

While since the 1960s military simulators have driven HMD designs with key tasks in far field visualization with collimated optics, many other applications from medical to education have emerged that are driving new concepts for HMDs across multiple tasks up to near field visualization.

Many companies in the world are developing advance helmet mounted display systems while adding new technologies as per the changing battlefield requirements.

BAE Systems Striker II is the all new digital helmet-mounted display system with integrated night vision camera.

With decades of combat-proven experience, the new platform-agnostic Striker II helmet-mounted display (HMD) builds upon BAE Systems’ current Striker HMD, which has been successfully deployed in theatre on Eurofighter typhoon and Gripen fleets. Striker II is a fully digital solution that provides today’s combat pilot with exceptional night vision and target tracking technology within a fully integrated visor-projected HMD system.

BAE Systems’ Striker helmet-mounted display system (HMDS) is based on the company’s unique two-part helmet design. It provides comfort, protection, and helmet stability for fixed-and rotary-wing platforms. The HMD is lightweight having detachable night vision camera which provides high accuracy.

This next-generation HMD also includes a cutting-edge tracking system that ensures the pilot’s exact head position and the aircraft computer system are continuously in sync, reducing problems common to other HMDs. The Striker II tracking system, with new hybrid opto-inertial technology, eliminates any delay in determining where the pilot is looking and can therefore perfectly position symbology onto the visor. The results are high-precision target tracking and engagement as well as superior situational awareness and mission effectiveness.

Joint Helmet Mounted Cueing System (JHMCS) is a product of Vision Systems International (VSI), a joint venture of Elbit Systems with Rockwell Collins. The JHMCS provides the pilot with “first look, first shoot” high off-boresight weapons engagement capabilities. The system, also available in a night configuration, enables the pilot to accurately cue onboard weapons and sensors against enemy aircraft and ground targets without the need to aggressively turn the aircraft or place the target in the Head-Up Display (HUD) for designation. Critical information and symbology such as targeting cues and aircraft performance parameters are graphically displayed directly on the pilot’s visor.

The JHMCS is a modified HGU-55/P helmet that incorporates a visor-projected Heads-Up Display (HUD) to cue weapons and sensors to the target. This new cueing system improves effectiveness in both Air-to-Air and Air-to-Ground missions. In close combat, a pilot must currently align the aircraft to shoot at a target. JHMCS allows the pilot to simply look at a target to shoot. This system projects visual targeting and aircraft performance information on the back of the helmet’s visor, enabling the pilot to monitor this information without interrupting his field of view through the cockpit canopy. The system uses a magnetic transmitter unit fixed to the pilot’s seat and a magnetic field probe mounted on the helmet to define helmet pointing positioning. A Helmet Vehicle Interface (HVI) interacts with the aircraft system bus to provide signal generation for the helmet display. This provides significant improvement for close combat targeting and engagement.

In the rapidly growing Helmet Mounted Systems market Elbit Systems is operating from a position of strength and global leadership after innovating and integrating three generations of HMSs for both fixed and rotary wing aircraft. Elbit Systems and its subsidiary Vision Systems International (VSI), jointly owned with Kaiser Electronics (a Rockwell Collins Company) have more production and operational experience than any other company in the field.

Its HMSs incorporate tracking and display systems for target designation, weapon and sensor slaving and processing and display of tactical information day and night. They are supplied as part of upgrade programs as well as on a stand-alone basis. Elbits’ Display and Sight Helmet System (DASH) enables pilots to aim their weapons simply by looking at the target. DASH measures the pilot’s Line-Of-Sights (LOS) relative to the aircraft, and transfers its information to other aircraft systems. Aircraft, sensors, avionics and weapons are thus enslaved to the target. DASH is adaptable to any fighter/attack aircraft and will accommodate advanced missiles and smart weapon lock-on envelopes. There are over 1000 helmets and 750 systems operational on 4 continents and onboard 15  different platforms.

F-35 Gen III Helmet Mounted Display System (HMDS) is the world’s most advanced helmet-mounted display system bringing unprecedented capability to the world’s most advanced tactical aircraft. Its integrated head-up display is the first to provide pilots with all the critical information they need on the helmet’s visor ensuring that every mission, day or night, is supported with unsurpassed situational awareness, tactical capability and safety. The F-35’s Helmet Mounted Display Systems provide pilots with unprecedented situational awareness. All the information pilots need to complete their missions-airspeed, heading, altitude, targeting information and warnings-is projected on the helmet’s visor, rather than on a traditional Heads-up Display. This approach greatly reduces the pilot’s workload and increases responsiveness. Additionally, the F-35’s Distributed Aperture System (DAS) streams real-time imagery from six infrared cameras mounted around the aircraft to the helmet, allowing pilots to “look through” the airframe. The helmet also provides pilots night vision through the use of an integrated camera.

By fully integrating three advanced technologies-head-up display, helmet-mounted display and visor-projected night vision-the F-35 Gen III HMDS provides revolutionary capability to the fighter cockpit. For night missions, the HMDS projects the night vision scene directly onto the visor, eliminating the need for separate night-vision goggles. The HMDS provides a lightweight helmet, with optimized center of gravity and maximum comfort for reduced pilot fatigue.

L-3 Link’s Advanced Helmet Mounted Display (AHMD) represents a major breakthrough in visual display capability.

L-3 Link’s AHMD has been designed to provide the ultimate immersive experience for both virtual training and augmented operational reality. In support of virtual training, the AHMD can be used to meet deployable training and traditional simulation requirements. In addition, the AHMD can support augmented reality awareness for unmanned aerial vehicle, air traffic control and C4ISR operators.

L-3 Link’s AHMD proprietary design provides users with a 360° field-of-regard of both out-the-window imagery and systems symbology. The instantaneous user field-of-view spans 100° horizontally by 50° vertically and has a 30° binocular overlap region that induces a sense of immersion. The AHMD’s greater than 60 percent see-through capability enables users to clearly view their surrounding cockpit controls and read maps in high ambient lighting.

A revolutionary optics and illumination design, in addition to use of solid-state micro displays, enables the AHMD to provide unmatched contrast, brightness and vivid color for all types of imagery. Because the L-3 Link AHMD has an eye relief greater than 50 millimeters, users also are able to wear eyeglasses. The claustrophobic, closed-in feeling prevalent in other helmet mounted displays is not an issue with the L-3 Link AHMD.

The AHMD attaches to a user’s own helmet in a matter of seconds, provides a lightweight, balanced center of gravity display and includes image alignment controls that ensure all users obtain optimum display viewing.

Future prospect

In future it is expected that fighters and helicopters will be replaced by unmanned aerial vehicles to some extent for certain specific mission. Keeping that in mind researchers are trying to utilize the potential of HMDs for UAVs as well.

Today, UAVs are operated from Ground Control Stations (GCS) that use computer monitors and joysticks for controlling UAV payload sensors (cameras). Since the development of affordable HMDs, however, interest has been expressed in determining whether they can be incorporated into UAV GCS. HMDs provide ecologically relevant cues to operators  and facilitate awareness of areas already searched for target identification, thereby potentially reducing the rescanning of those same areas.

HMDs also offer an egocentric, augmented reality display presentation throughout the full field of regard of the pilot’s natural vision, potentially offering increased spatial and situational awareness.

The application of advanced technologies like HMDs to the domain of UAV GCS operation will rely on the systematic analysis of their effects (costs and benefits) on human operators. Research is being conducted to study the performance, workload, motion sickness, and situational awareness factors associated with HMDs in this domain.

Future HMD designs for UAVs will require high-resolution sensors and brighter displays to allow for the integration of night vision cameras.

The modern air combat arena has changed considerably in recent years. Today, having sophisticated radar and missiles and being able to outmaneuver the opponent are no longer guarantees for air combat superiority.

In addition, due to the fact that in most cases a large number of aircraft take part in air engagements, it is known that the longer one stays in the air, the less the chance of survival. Being able to lock the weapon system, fire, and egress the area as quickly as possible is therefore of paramount importance.

The pilot is deluged with vast amounts of spherical oriented information. In both air to air, air to ground, in the way to and from the mission core, the pilot’s situation awareness is a crucial factor in achieving the mission goals.  Therefore to achieve the optimal situational awareness displaying the flight data and three dimensional-location based systems and sensors information to the pilot and, designating and exploiting the pilot’s LOS (Line Of Sight) for the purpose of slaving the aircraft sensors and systems and thus enabling the pilot to achieve lock-on of sensors, avionics systems and missiles, simply by looking at the target, is necessary. And helmet mounted display systems are definitely helping the pilots to achieve the required expertise into such a scenario.

There is no doubt that the wide scale deployment of the HMD will further reduce cockpit workloads and increase the lethality of modern fighters in close quarters combat.