How NVIDIA helped 3D researchers bring naval history to life

Museumgoers will be able to explore two sunken WWII ships as if they were scuba divers on the ocean floor, thanks to work at Curtin University in Perth, Australia.

Exhibits in development, for display in Australia and potentially further afield, will use exquisitely detailed 3D models the researchers are creating to tell the story of one of the nation's greatest naval battles. NVIDIA technology has a key role.

On November 19, 1941, Australia's HMAS Sydney (II) and Germany's HSK Kormoran lobbed hundreds of shells in a duel that lasted less than an hour. More than 700 died, including every sailor on the Sydney. Both ships sank to 8,000 feet, 130 miles off the coast of Western Australia, not to be discovered for decades.

Andrew Woods, an expert in stereoscopic 3D visualisation and associate professor at Curtin, built an underwater rig with more than a dozen video and still cameras to capture details of the wrecks in 2015.

Ash Doshi, a computer vision specialist and senior research officer at Curtin, is developing and running software on NVIDIA GPUs that stitches the half-million pictures and 300 hours of video they took into virtual and printed 3D models.

3D at battleship scale

It's difficult pioneering work in a process called photogrammetry. Commercially available software maxes out at around 10,000 images.

"It's highly computationally intensive — when you double the number of images, you quadruple the compute requirements," said Woods, who manages the Curtin HIVE, a lab with four advanced visualisation systems.

"It would've taken a thousand years to process with our existing systems, even though they are fairly fast," he said.

When completed next year, the work will have taken less than three years, thanks to systems at the nearby Pawsey Supercomputing Centre using NVIDIA V100 and prior-generation GPUs.

Speed enables iteration

Accelerated computing is critical because the work is iterative. Images must be processed, manipulated and then reprocessed.

For example, Woods said a first pass on a batch of 400 images would take 10 hours on his laptop. By contrast, he could run a first pass in 10 minutes on his system with two NVIDIA RTX A6000 GPUs awarded through NVIDIA's Applied Research Accelerator Program.

It would take a month to process 8,000 images on the lab's fast PCs, work the supercomputer could handle in a day. "Rarely would anyone in industry wait a month to process a dataset," said Woods.

From films to VR

Local curators can't wait to get the Sydney and Kormoran models on display. Half the comments on their Tripadvisor page already celebrate 3D films the team took of the wrecks.

The digital models will more deeply engage museumgoers with interactive virtual and augmented reality exhibits and large-scale 3D prints.

"These 3D models really help us unravel the story, so people can appreciate the history," Woods said.

In a video call, Woods and Doshi show how forces embedded an anchor in the Kormoran's hull as it sank.

The exhibits are expected to tour museums in Perth and Sydney, and potentially cities in Germany and the U.K., where the ships were built.

When the project is complete, the researchers aim to make their code available so others can turn historic artifacts on the seabed into rare museum pieces. Woods expects the software could also find commercial uses monitoring undersea pipelines, oil and gas rigs and more.

A real-time tool

On the horizon, the researchers want to try Instant NeRF, an inverse rendering tool NVIDIA researchers developed to turn 2D images into 3D models in real time.

Woods imagines using it on future shipwreck surveys, possibly running on an NVIDIA DGX System on the survey vessel. It could provide previews in near real time based on images gathered by remotely operated underwater vehicles on the ocean floor, letting the team know when it has enough data to take back for processing on a supercomputer.

"We really don't want to return to base to find we've missed a spot," said Woods.

Woods' passion for 3D has its roots in the sea.

"I saw the movie Jaws 3D when I was a teenager, and the images of sharks exploding out of the screen are in part responsible for taking me down this path," he said.

The researchers released the video here to commemorate the 81st anniversary of the sinking of the WWII ships.