Technological advances in virtual and augmented reality mean that service personnel have a world of training environments quite literally at their fingertips. Andrew Tunnicliffe looks at the latest developments in synthetic training environments.

The last decade has brought about significant changes in the way personnel are trained, thanks to the development of simulated or synthetic training capabilities. Virtual reality (VR) and augmented reality (AR) – layering visuals over a real environment view – are exciting advances shaping the way military service men and women, from all of the armed forces, are trained.

“VR and AR technology has nearly an unlimited amount of practical applications,” says David Reddick, chief product officer at VR Vision, a Canadian virtual and augmented reality specialist solutions provider. “We have used it to entertain in the past, but as technology gets better we have seen a steady increase in developing it to mimic real-world environments and simulations.”

In a relatively short period of time, armed forces the world over have adopted VR and AR technologies, making them a key part of their training mix. From flight simulation to medical training, the battlefield to boot camp, billions are being invested annually in simulation training, and that figure set to rise further.

Research carried out by Samsung suggests the US spends as much as $14bn a year on VR/AR development, while the UK spends £1.1bn according to 2016 MOD figures. The Chinese VR market has also grown by more than 8% in the past year as a result of government-backed spending in developments, making it worth $8.5bn according to Bloomberg.

For almost 20 year the UK’s Defence Science and Technology Laboratory (Dstl) has sought to prove the rich and immersive experience that synthetic training could be, one which fills identified training gaps for the armed services.

The success of exercise Enduring Sunrise in 2014 went some way to doing that. Involving facilities across the UK, the exercise brought together synthetic Typhoons and Tornados at the Air Battlespace Training Centre, synthetic rotary wing aircraft at Middle Wallop, Typhoons at RAF Lossiemouth, and maritime assets at HMS Collingwood, as part of the largest ever synthetic exercise research trial to be held in the UK.

Ultimately, the experiment proved it possible to simulate exercises across different locations, meaning VR and AR could achieve training value. Today Dstl continues its work in developing technologies that will facilitate training in this way, known as distributed synthetic environment training – much like a US programme to develop the Synthetic Training Environment (STE).

“Training using mobile apps, VR, and AR technology will eventually replace regular training techniques.”

According to the US Army, the STE is designed to provide a cognitive, collective, multi-echelon training and mission rehearsal capability for the operational, institutional, and self-development training domains. The environment will “keep pace with and adapt to the rapid development of technologies” as part of the Army’s ‘Big Six’ modernisation priorities and builds on the US Department of Defense’s annual spend of $14bn in this field.

“Training using mobile apps, VR, and AR technology will eventually replace regular training techniques,” says Reddick, reflecting on how this type of training is changing the way we learn. “In VR and AR, you can accomplish any task with no risk to your body. We're able to recreate almost any task or mission that is asked of us in the virtual world and provide multiple outcomes, feedback, and analytics for each and every motion the user does.”

The ability to conduct exercises without risks to personnel or assets explains the appeal of this type of training for the armed forces.

“As technology progresses, we are noticing an increase in realism and interaction formats that allow users to experience training almost identically to how they would experience it in real life, with the advantage of being able to stop, pause and reset the training experience,” Reddick says, adding that the possibility to simulate and reset on a constant basis, for a nearly unlimited user base and low cost, is where much of the appetite for VR and AR comes from across both the private and public sectors.

“The ability to interact with a training workflow, from start to finish, with the ability to reset that workflow instantly for the next user, or have a group of people play out the experience at the same time and track all their results, is where the value of computer training comes in”, he adds. “We can simulate any military experience and, with our analytics and unique eye tracking technology, even determine how it is effecting the users mentally and physically.”

“It’s only a matter of time before developments in this field will enable mobile VR and AR – meaning the immersive experience will become even more realistic.”

In virtual combat training scenarios the soldiers, for example, wear headsets that place them into a 3D environment, which changes with their movements. Wearing body armour and much of the kit they will be expected to carry in real-world scenarios, individuals are tracked and the data is used to alter events around them. This tracking is where the next big evolution is likely to be.

“Currently inside-out tracking uses sensors that are commonly placed around rooms to sense the position of the user. In the coming years, we're going to see this tracking built inside the headsets, using cameras to sense the position of users without the need for external sensors to be put on walls,” Reddick explains.

Aiding this will be further advances in mobile power supply, historically a limiting factor. But it’s only a matter of time before developments in this field will enable mobile VR and AR according to Reddick – meaning the immersive experience will become even more realistic.

With the likes of the US and UK making continual advances in their pursuit of a fully-fledged collective training environment, the future of armed forces training looks to be on the cusp of a major revolution, if it hasn’t seen one already.

The US STE and its British equivalent, the Defence Operational Training Capability which utilises the Dstl’s so-called synthetic environment tower, means there is an exciting future ahead for VR and AR in military training.

“We believe the future of synthetic training is the ability to have complete haptic feedback to the user of anything they do in VR and be able to eliminate accessories to use your actual hands that are completely tracked in any VR experience using upgraded tracking technology,” Reddick says.

“We already have the ability to distribute, on a massive scale, using wireless headsets, so producing for a large amount of sites at once is already becoming the present. Adoption of the technology for training has never been better.”