Nearmap decreases AI model training time by 98% with AWS

Nearmap presents at AWS Summit Innovation Day in Sydney, April 11, 2024

Nearmap is on the cusp of launching its sixth-generation AI model, which has been trained with AWS SageMaker, a service that builds, trains, and deploys ML models at scale. Leveraging AWS, Nearmap has cut down the training time needed from up to eight weeks down to just 26 hours, while also having more than 40 times more compute power, a decrease of 98% in AI model training time.

As global businesses realise the transformative nature of artificial intelligence (AI) and machine learning (ML) to automate and streamline tasks at a level previously unachievable through human input alone, Nearmap is the only aerial intelligence company with an AI system that runs routinely on every survey flight to build automated high-resolution maps of AI and ML content covering up to 95% of the Australian population.

Satellite companies capture the globe, but they don’t provide the resolution that’s needed to fully understand what’s happening in cities, in suburbs and at an individual property level. With aircraft-mounted cameras, Nearmap imagery delivers the level of detail and coverage turns imagery into actionable insights including understanding surface imperfections, inventory assets, or precisely measuring sites and buildings.

The power of this AI data has helped organisations like local councils map vegetation cover to make more effective community-focused decisions. Property and casualty insurers are also gaining additional insights to make underwriting and claims decisions with more accuracy and at faster speeds. Nearmap’s AI capabilities are transforming the insurance industry and setting new fulfilment standards for policyholders.

But this all relies on the speed of ML innovation, which is often bogged-down by complex infrastructure setups, slow and disjointed experimentation cycles, and inefficient hyperparameter (ML training) optimisation. DataRobot, an enterprise AI platform, has disclosed that data scientists iterate on models 5 to 10 times on average before deployment – that’s a lot of expensive lost time.

Michael Bewley, Vice President of AI and Computer Vision at Nearmap, faced similar challenges when his team began applying machine learning on more than 50 petabytes of imagery data.

“From an aerial perspective, no-one has an AI system that runs at high resolution, routinely, on every survey flight to build automated maps – except us,” said Michael Bewley.

In 2019, Nearmap spent about $1 million on AWS compute, and produced one million square kilometres of deep-learning results in Australia and the US. That first-generation of Nearmap AI comprised 12 layers, with layers including swimming pools, buildings, trees, construction sites, and more.

“Inspired by the possibilities, I asked our team to figure out how to put all the layers in a single model. We knew the benefits would be huge, so our mantra became: build for a thousand layers.”

But the AI training came at a cost. By generative five, Nearmap had amassed around 1 million labelled images, with a total catalogue of more than 50 petabytes of data. The models were being trained on a rack of servers in a data centre, and would take six to eight weeks to train.

The speed of deployment and execution and the ability to experiment is critical for innovation and tech evolution – so Nearmap took transformative action to enable this. It involved a full rewrite of the training stack using PyTorch Lightning, switching from on-premise data centres to AWS, leveraging AWS SageMaker. At the same time, Nearmap doubled the size of its model architecture and its labeled data set.

Amazon SageMaker met the challenging brief to combine the optimal balance between speed of innovation, and efficient resource allocation with rigorous governance and security controls. In 2024, Nearmap is on the cusp of launching its sixth-generation AI, which could include as many as 150 layers.

Michael Bewley reflects on what that means for Nearmap: “We’ve gone from spending six to eight weeks’ on a single long running model training job, down to just 26 hours, leveraging 40x the compute power. That’s a decrease in time of 98%, while producing a bigger, more powerful model trained on a larger data set. We’ve made huge strides on our training systems, model architecture and internal processes.”

“This simply wouldn’t have been possible without switching from on-premise data centres to the cloud,” he said. “Amazon SageMaker has allowed us to submit jobs to train efficiently on 40x the compute we used to use, and were just getting started. Moving from training on a single machine to many is a big step. It unlocks a potential future of using hundreds of GPUs at once, without having to worry about all the infrastructure and hardware management.”

“The training of AI models can be costly and time consuming; if companies can gain some efficiency, it will help them accelerate the introduction of new features to the market.  We are so pleased to partner with Nearmap, who are leveraging Amazon SageMaker to reduce their model training time by 98% while simultaneously increasing their model's size and performance.   This is a testament to the power and scalability of cloud as AWS allows companies like Nearmap to focus on their core competencies and bring ground-breaking AI applications to market faster than ever before,” said Sarah Bassett, Head of Software, AWS Australia & New Zealand.

The Nearmap data science team can now run multiple experiments concurrently, testing various hypotheses and model configurations against partitioned data on Amazon Simple Storage Service (S3) on more than 60 petabytes of imagery. This fan-out strategy eliminates bottlenecks associated with sequential iterations and finite resources, and continues to help Nearmap accelerate the discovery of optimal models during the early development stage.

When asked about what the future holds for his AI team, Michael Bewley says: “It’s perfect timing really – we’ve got a mature semantic segmentation pipeline and core product. Now we have the tooling and compute at our fingertips to explore what value we can unlock from our massive data set.”

“The latest Nearmap camera system, HyperCamera 3 (HC3) has richer data, near infrared, and higher-quality multi-angle views, which opens up exciting possibilities from an ML research perspective. We’ve set an ambitious plan with AWS SageMaker over the next few years purely for training models – I can’t wait to see what the team comes up with!”

Nearmap PR Contact
Dave Slattery
dave.slattery@nearmap.com