A dose of virtual reality: Pilots enhance 316th OSS helicopter simulator for training needs

  • Published
  • By 316th Wing Public Affairs

First Lt. Joseph Mull, a 1st Helicopter Squadron pilot at Joint Base Andrews, Md., was flying one of two UH-1N Hueys in formation over the National Capital Region.

The scenario was a routine 30-minute mission in late October where Mull was shadowing the lead Huey taking off from Andrews and navigating through a series of checkpoints before returning to base. However, nobody was actually in the lead helicopter, and Mull never left the ground the entire time.

Instead of a real helicopter, Mull had donned a virtual reality headset and was piloting a UH-1N in a flight simulator assembled by the 316th Operations Support Squadron to support the 1st HS mission. The VR simulator was assembled after the regular simulator was sent off for repairs, and everything including the software, desktop computer, monitor, seat, VR headset and flight controls was purchased commercially, but pilots have tweaked the components to customize the simulator as their own.

The simulator Mull was piloting is one of seven VR stations setup in the 1st HS headquarters building.  They utilize X-Plane flight simulator software, and linking the stations together allows pilots to train as they fly in multi-ship formations.

Maj. Mitchell Clapp, 316th OSS assistant director of operations, has been able to channel his lifelong passion for video games as the architect of the VR simulator system.

“I was an avid gamer as a kid, and I still play, so I’ve been keeping up with the technology,” said Clapp, who also has a degree in computer science.  “When I was younger and playing, I wanted to modify and edit the games.”

The VR simulator project began about a year-and-a-half ago when the regular helicopter simulator was crated up and freighted out of state for repairs and updates.  That simulator takes up an entire room because it’s modeled after a real Huey cockpit and is mounted in front of a giant projection screen.  Because of the length of time required for restoration, the current system utilizing commercial software and components was devised as a stopgap by the 316th OSS.  The VR system was just in its infancy stage when Clapp – who flew UH-1Ns here from 2010 to 2014 – returned to JBA for a second assignment. Initially squadron leadership wanted to find a contractor to address system needs, but with his background in gaming, they handed Clapp the reins.

“When the need arose here, we had the basic system, but things just didn’t look right,” he recalled.  “The game engine comes with a back-end support program we can use to edit buildings or add new ones. It wasn’t complicated to begin.  I did the basic scenery and then jumped into more elaborate stuff.  After thousands of YouTube videos, I’ve essentially been going to a master’s school now for this.”

The X-Plane computer program has a library of elements such as buildings and trees that can be added to the simulator landscape, and Clapp said they used another 3D modeling software program to make objects from scratch to ensure detail and accuracy on important navigation points.

One such building that was added was a power plant the aircrews use for reference when flying routes around the NCR.  This building wasn’t part of the VR world, so 1st Lt. Daniel Herndon, a 316th OSS pilot, studied images from Google Earth and others found on the internet to render an accurate likeness.

“I built it over a few days,” Herndon recalled.  “It was one of my quicker ones.”

The latest scratch-built structure added to the map was the new Frederick Douglass Memorial Bridge spanning the Anacostia River in Washington, D.C. The bridge opened in 2021 to replace the old South Capitol Street Bridge, and the new bridge has taller arches and spans the river in a slightly different location because it was erected while the old bridge was still in use.

“Not only are all those pilots building the scenery and tweaking the map, as they're doing it, they're becoming more familiar with our mission,” Clapp said.  “That translates directly to their knowledge of this city, which is part of their job when they get flying.”

A pilot like Mull who just completed flight school hasn’t been here long enough to complete the helicopter training required before taking to the skies in a real-world mission.  He said he puts in about eight to 10 hours a week modeling scenery and buildings to place in the VR map, and that coupled with flying in the simulator will benefit him once he takes the controls in a real Huey.

“I started out studying the routes on paper but learning through VR gives me a better idea of routes and zones,” he said.  “Seeing an actual geographic representation and knowing what everything looks like visually is super helpful.”

In addition to training over the NCR, the VR simulator is also ideal for familiarization if pilots are called to fly out of the area, Clapp said.  From regular missions to community engagement events like flyovers and static display requests, pilots can pre-fly the route before leaving the ground.

“For example, [when] the squadron gets tasked to support an event, we don’t have to go in there blind,” he said.  “The area that we're going to land, we can put the trees in the proper spot, make them the proper height, and make the buildings for the proper footprint.  Before the real mission starts, we've already flown some of the routes, looked at the city and memorized it.”

To enhance training, in the middle of the room with the seven VR stations is what the pilots call “The Sand Table.”  Assembled like an elevated children’s sandbox, it’s a large square constructed of wood that Clapp built at home.  It’s filled with sand which can be hand-shaped to render the NCR terrain with its elevations and depressions of the Potomac and Anacostia rivers.  Dimming the lights in the room, a projector overhead beams down a geographical map, and icons represent squadron helicopters as the VR missions are flown in real-time.  Missions are recorded to play back for debriefings and training purposes. The base’s SparkX Cell Innovation and Idea Center is 3D printing miniature models of the White House, Washington Monument, U.S. Capitol and Lincoln Memorial to contribute to “The Sand Table” landscape.

With seven piloting stations and three other development stations for mapping and modeling, the VR system tallied about $70,000 to assemble, Clapp said.  With the hands-on approach – part of the training for new pilots is to build scenery in the VR world – updates and fixes happen quickly.

Clapp doesn’t see any reason why other flying units wouldn’t be able to duplicate the VR setup.  While X-Plane only offers civilian aircraft (and only one is a helicopter), the program is bundled with software to design and edit aircraft for use in the simulator.

X-Plane is the flight simulator program of choice compared to other applications because it’s easier to customize, and the virtual world is more complete, Herndon added.  The first X-Plane program was released in 1995, and it has gone through several variations and updates over the years.  The squadron currently uses X-Plane 11 and will upgrade soon to version 12, which was released in mid-October in early access status with full release planned near Christmas.

“I've already got the demo on there and imported some things over, so I already know it works,” Clapp said.  “It should be a relatively easy transition.  It's just a matter of getting it here and running.”

Repair on the old simulator has been completed; it returned to service at Joint Base Andrews a few months ago and still has some advantages.  It has been certified by the Federal Aviation Administration and the Air Force District of Washington, so pilots can log real flight hours.  Since it is modeled after a Huey cockpit, it physically looks like the real thing with gauges and switches, and it has two seats for the pilot and co-pilot (or pilot and trainer) sitting next to each other.

The old simulator is also supported by a contractor on-site for routine maintenance and updates.  With Clapp as the lead on the VR system, someone would have to take over when he receives permanent change of station orders, which he estimates will be in July.  Another possibility is hiring a civilian for continuity.

The VR system is useful for training in multi-ship formations and pilots taking responsibility for enhancements, Clapp said.  Upgrades such as adding the GPS satellite and tweaking the helicopter’s flying characteristics are among future possibilities to add to the simulator’s realism.

“The technology has gotten to a point where this stuff is getting better and better,” Clapp said.  “There’s infinite potential.”