Semantic segmentation is the classification of every pixel in an image/video. The segmentation partitions a digital image into multiple objects to simplify/change the representation of the image into something that is more meaningful and easier to analyze . The technique has a wide variety of applications ranging from perception in autonomous driving scenarios to cancer cell segmentation for medical diagnosis.
Exponential growth in the datasets that require such segmentation is driven by improvements in the accuracy and quality of the sensors generating the data extending to 3D point cloud data. This growth is further compounded by exponential advances in cloud technologies enabling the storage and compute available for such applications. The need for semantically segmented datasets is a key requirement to improve the accuracy of inference engines that are built upon them.
Streamlining the accuracy and efficiency of these systems directly affects the value of the business outcome for organizations that are developing such functionalities as a part of their AI strategy.
This presentation details workflows for labeling, preprocessing, modeling, and evaluating performance/accuracy. Scientists and engineers leverage domain-specific features/tools that support the entire workflow from labeling the ground truth, handling data from a wide variety of sources/formats, developing models and finally deploying these models. Users can scale their deployments optimally on GPU-based cloud infrastructure to build accelerated training and inference pipelines while working with big datasets. These environments are optimized for engineers to develop such functionality with ease and then scale against large datasets with Spark-based clusters on the cloud.
Arvind Hosagrah…: Hello, Hi, my name is Arvind Hosagrahara. And I’m here to talk about processing large data sets for automated driving for ADAS applications using Spark. I’m a Solutions Architect at the Mathworks. As an outline of today’s talk, we will start with the big picture, stating a problem and the challenges in the space. Talk about the architecture of a system for tackling these problems and compassing from the extract transform and load of these recorded artifacts, recorded data sets, going through training, training of deep learning networks and building out inference engine.
And then quickly dive into a deep dive into a few of the Spark workflows. How do you manage clusters? How do you perform, how do you interact with the explore your data sets, while it’s on the cloud behind Spark clusters and how to push down the analytics. Move your program to the analytics. Talk a little bit about experiment and metadata management. And finally, touch upon a few design considerations in building these systems. Scaling, accuracy, interoperability, security and whatnot, and conclude with a few question and answers.
As a very big picture, as we try building automated driving features to get to automate transportation, one of the things that people in this space deal with is answering how to analyze and simulate the scenarios, how to build perception stacks to let the computers that are driving these cars know where they are, plan where to go and control it. And how do you design and deploy these algorithms? And finally, how do you make it safe? How do I integrate, test it and actually take this out into a production system?
For the purposes of this conversation, which talks a little bit about the data analytics and the processing side of it, we’re going to focus on the perception bit. And within the perception, there are many, many different kinds of sensors in a modern automobile. What we will be looking at is the camera system. There are lighters, there are radar, solars, there’s there’s a whole bunch of rich sensors on these newer vehicles that are capable of driving themselves. We will focus on the vision aspects of this and in particular, a problem known as semantic segmentation. Which is the classification of every pixel in an image or video.
Why do you do this? The higher quality of perception outcomes are achievable if you can simplify the processing and being able to make these datasets easier to analyze and validate localization modules, allows us to actually simplify the processing behind what is a very complex image space. For example, it makes no sense looking for lane markers in the sky. So if you can distinguish your road from your sky, you’re already up ahead because you can actually analyze your data better. And finally, this is a rapidly evolving space with newer models appearing yearly. The pace of evolution here is so rapid that it is imperative to have faster design iterations, to go from experimentation to operationalization, by turning the crank very quickly.
As a quote, there’s a research study which says autonomous vehicles would have to be driven hundreds of millions of miles. And sometimes hundreds of billions of miles, with a B. To demonstrate their reliability in terms of safety. Driving a billion mile takes a lots of data as well as a lot of time. Sometimes untenably so. And that is where aspects, technological advances such as simulation in that space begin to appear. So boiling that down to a problem statement, how can we accelerate development of these models, from prototype to production to build better perception software? How to scale the workflows, capabilities and throughputs of engineering processes to enable teams to leverage automation, the cloud storage, compute and other infrastructure, excuse me.
As challenges, technical challenges, the richness of the sensors means that datasets grow. They grow very quickly and very large. They come in a variety of specialized formats. And sometimes these systems, especially the cloud based systems, leverage the latest best in class storage and compute infrastructure to be able to handle the computation needs. And to enable self serve analytics for the data and engineer. This becomes the holy grail of people who would like to enable their end engineers that their domain experts and the data scientists to self serve the writing needs. And if you boil it up to a business level, we are looking at going quicker from prototype to production to verify and validate these algorithms quickly, and be able to deal with the development of the right workforce and integrate engineers with very disparate backgrounds required to actually build out such stacks.
So I’ll cut to the chase and give a very high level demonstrative example. As what I have here is a quick video. What we have here is front camera from a vehicle driving a busy street. This video was captured in Spain by one of our colleagues, showing approximately what the car is seeing from the front camera. And it’s a very small section of the video, what we will do is take this through the process of semantic segmentation. The first of these models is fairly old. VGG 16 is approximately five or six years old. And as you can see if this is what my car was perceiving the world to be, I don’t know if I would feel comfortable sitting in it. It tries to distinguish the road from other vehicles and the sky, it doesn’t do a very good job.
But this process of iteration, as you start applying color space corrections, as you start to pre-processing the video, making it easier to analyze, you will see the quality of the semantic segmentation improved. As a second iteration, you will see that I’m slightly better and observing the sky, but not noticeably so. Color correcting the contrast correcting, we get a little better, it’s able to quite reliably spot the sky from the road. But there is still a lot of noise in this. And even at this point, you know it is nowhere close to be, if this is the way a car is actually looking at it, there is scope for improvement. Switching to the more modern models for semantic segmentation, the deep lab V3 model, now begins to locate the other motorists, locate the road in a slightly better way.
And if you take a look at the fourth iteration here, it’s less noisier than the earlier ones. And if you look at it as something which has the appropriate adjustments, what happens is the need for actually retraining these models and turning the crank with newer models. In this one you can see the sky is pretty reliable, it’s able to spot a few signboards, being able to look at the semantic segmentation problem. It is a process of continuous improvement. And it will take a lot of data, a lot of advances in the way these models are constructed, to be able to build something that will serve the needs.
So a billion miles of that kind of recorded data is very difficult to actually crunch. And sometimes it’s not possible to really drive so many miles. That’s where simulation comes into the picture. So in this case, you can actually use techniques such as generative adversarial networks or GANs, and use 3D simulation data to train the network and be able to handle a different domain as it actually and find tune it for real world data. As a query quick example, this example shows a training set that has been documented by 3D simulation data. And trained to be able to do the semantic segmentation. It’s a standardized video set. But you can see the quality of the segmentation get a whole lot better, with it reliably detecting the sky from the ground, detecting pedestrians, detecting other cars. And this becomes the process of iteration as people are developing these algorithms. The End outcome is a better perception stack for the vehicles that are actually operating on the streets, without human intervention.
So as a very big picture, let’s take a look at the architecture of a system to aid in the development of these kinds of algorithms. We set up a system on AWS, it’s a very complicated diagram here, and I don’t expect people to be able to see all the details. But as a very high level what you are looking at, to simplify this, is a development environment to allow you to quickly iterate on algorithms and do it on desktop environments, be able to access local datasets. And if necessary, connect up to your large datasets. Usually a cloud based data set onto the storage layers. Allowing you to build inference engines, allowing you to train these models, allowing you to be able to build inference engines to enrich new data as it comes in and monitor the performance accuracy of the predictions that these models are making.
A front end and visualization piece will aid with that, giving you a quick insight into how well these models are performing. And back it by cloud based training systems. This, for example, in this particular case was Databricks Spark. And we were running Spark clusters, which are GPU enabled, which are backed by fast networks on the cloud, to be able to burst up our needs for training and be able to train quicker, faster and turn the crank quicker. And finally, the ability to ingest recorded and simulated data, both batch and streaming into the system. So to simplify it, we can think of it as a simplified view, where you have an integrated system that will go all the way from development to operationalization of these models.
Through the entire process, you manage your experiments, you perform a certain amount of governance of what data was used to train what and build it against the best security practices. You can think of this as a couple of sub components. The first one is ETL, of the extract, transform and load pipeline. Recorded and simulation data can be ingested into your storage system, enriched, cleansed as they come in. Being able to apply these engines to enrich the data before it gets stored will make query later on a whole lot easier. Will make post processing a whole lot easier. As opposed to dealing with a large data as the data comes in, it is possible to actually clean it up, get it ready for analysis.
The next would be a training pipeline. Ways to quickly develop, experiment, turn the crank and run through design iterations, experimenting with different hyper parameters and being able to do this with the ability to burst the compute into arbitrarily sized trial based training systems, faster machines with more memory and GPU support on fast networks. All of this connecting up to a storage. In our case, it was Amazon S3. And that would give practically an unlimited amount of storage for you to build out here, your data lake. The third being the inference pipeline, which is to be able to very quickly take the train models and build inference engines, either to plug into other software systems or be operationalized to go into the vehicles themselves and to be monitored to be able to monitor the accuracy of these.
So as a quick start software stack, what we did was actually built this as a development environment we use MATLAB and Simulink. As the training we use the Databricks Spark environment, MATLAB capable clusters running on Spark. We also connected up to S3 as a storage layer and inference engines hosted on production server and web app server. The experiment management dovetailed into ml flow, and it did connect to the best in the cloud based security, identity and access management systems.
Another way to look at the same architecture is from a DevOps perspective. And what we really are trying to do is to turn the crank quickly. To start with the planning stage where you decide the capacity of this, go through cluster management, be able to enable domain experts to connect up to their data sources, to be able to interactively explore the data sets, experiment and help with the modeling. Be able to connect up to CICD pipelines, to automate the build and test by creating the appropriate Spark jobs and the libraries, and managing the motors as the models and experiments through their lifecycle.
Finally, operationalization for performance, taking it down to code, taking it down to CUDA code that will run in a highly optimal way, for some embedded applications or for building these inference engines that you can then operationalize some of them running 24/7, 365, while maintaining their centralized configurations and centralized view of their performance. With this, we’ll get into a deep dive into some of these building blocks, the Spark workflows that allows us to do this. As the first workflow, a way of quickly orchestrating clusters for us to be able to build up these Databricks clusters, these MATLAB capable Databricks clusters in the cloud. We enable that by connecting up the restful endpoints that Databricks provides and provisioning either in their scripts, so dockerized images and create these clusters at scale, very quickly.
As a quick example, let’s create a Databricks cluster. This is the MATLAB environment and we’re creating an object, a Databricks cluster object, and giving it a name. In a few lines of MATLAB code, everything that I’m typing here can be put into scripts and functions. You can interrogate Databricks and ask for available Spark versions, picking DPR our Databricks runtime of your choice. In this case, we’ll pick up 8.2. scalar, 2.12. Configuring these objects gives a very quick way in a few lines of code. You don’t have to do this, just the defaults will work, but if you wanted to tailor it, you can tailor your cluster for example, I’m looking at different node types, and picking either large instances here based on my needs for that particular experiment, or GPU optimized instances, very quickly configured as object properties.
Once you have a configured cluster, we can decide what scale to run this at. This is a bit of capacity planning, depending on the kind of training you need. We can set the number of workers to just the driver, setting it to zero. Or a four node cluster, or if necessary, an auto scaling cluster that will go between four and 10 workers. At which point, what happens is you have a definition in code. This is infrastructural code, of a cluster which you can then enable the MATLAB runtime, make it a MATLAB enabled Spark cluster. This will set up the inner scripts to make sure that the Spark cluster is capable of running MATLAB code on it. At this point, it is possible to actually take it through the entire lifecycle of the cluster. You can create these clusters, we can certainly manage it through code. In this case, I’ll pick a slightly smaller example just to keep cost down, creating the cluster will actually create a Databricks cluster and orchestrate it in this case between four and 10 worker nodes.
I can also terminate this cluster, running off billing on the Databricks side, and if necessary, permanently delete it. Freeing up underlying cloud resources. So all of that managers code gives me a very clean way to refresh and build these experiments so that it leads up to reproducible experiments, as I actually submit them into the Databricks system. One part of getting these onto the Databricks is being able to connect up to the data layers. In this case, abstractions for the Databricks file system, will give you the ability to look into folders, push and pull, make folders necessary push and pull, small slices of data. This is a way of moving your program to where the data is, you can either upload it, or you can take result sets and download them. Like in this case, I’m picking down a couple of files and recursively pulling down a fork of reserves for me to actually analyze on my local environment.
So once you have your cluster, we enable a variety of workflows to query, to explore the data on the cluster. You could connect up to the tables to teach Databricks like a database and connect up to the ODBC JDBC drivers that they provide, allowing you to slice and dice. Submit your SQL queries to slice and dice the data. And this applies not just to SQL, but if you had other data sources in the backend. Other relational databases or delta tables, you should be able to connect up either through JDBC ODBC or through the Spark session abstractions, for you to be able to slice and dice the data. And that enables scientists and engineers to be able to study their data interactively. One other mechanism to do this is to use the package that Databricks releases called Databricks Connect.
And what this does is takes what looks like a Spark API locally, but transfers it over to the Spark master and executes it on the cluster. As a quick example, what we’re going to do is connect up to a very quick script. This again is the MATLAB interface. And we’re going to connect up to data source underneath. Specify the data source and import Spark properties in a Spark configuration, allowing us to create a Spark session. At this point, what you have is a handle to your data set, which you can then read and slice and dice. This is done with a fairly rich interface, you have the ability to move slider bars and see the different kinds. For example, our video contained acceleration sensors and you have the ability to slice and dice and interactively explore the data set by seeing what the sensor is doing. Build it out in the form of interactive live script, as you’re seeing here.
And as you are performing these steps of slicing and dicing, you can connect up, read and write results, intermediate results into data tables if required. Which are stored in the backend as optimized parquet files, connect up to various other kinds of data storage on Databricks Spark. So at which point what you have is the ability to read and write these parquet files. In this case, it’s a parquet file on a cloud based system. That when you read it, you can actually perform Spark API operations like in this case, I count the number of records that I have in that particular set, there’s just over 2 million records in this particular data set. On the backend, what this is doing, if I looked inside my Databricks system and looked inside my Spark UI, every step through my code is actually triggering the appropriate actions on the Spark UI.
So I can go to the Spark UI, take a look at my executors, take a look at what is happening across the cluster and operations such as count will kick off Spark jobs. So where there’s 1000s of jobs on this particular example, but drill down into the task, drill down to each job and see exactly what the Spark job is doing and how it’s being staged. So with those building blocks, with those workflows as building blocks, what we can do is look at the local experimentation phase, a quick view of the high level API that you can get to write less and do more. So essentially, low code ways of actually providing the right language abstractions for you to augment your input data set. For you to be able to train the network and for you to be able to perform the training exercise, executing this on the Databricks Spark system.
These workflows are designed for interviews and debug and there’re a whole variety of apps for selecting models to perform forensic studies on accuracy, loss and training performance, and many examples with good documentation on how to do this. Once we have our model, we can then take that model, push it down to execute either a Spark submit jobs on new clusters or on libraries, accessible to others systems. As libraries on Databricks Spark. We have a couple of semantics to do that, either exposing an API for Spark itself, the MATLAB API for Spark, or abstracting out certain MATLAB users in the form of Tolar is much more natural, a very high level way of actually letting the tool build the bag for you.
What we will do is take a look at for the semantic segmentation example that we were discussing, what it takes to train a fully convoluted network. We build a small function that processes a folder of data, specifying where to write out the model and hyper parameters such as batch size in the box. In this particular case, the code here references VGG 16. But if you had a different model structure, you could certainly load those in and leverage accelerated environments like the CUDA toolkit, to be able to generate code and run this as native CUDA kernels. Once you actually have your training code, there’s a whole variety of applications for domain specific work. In this particular case, I’m looking at building the library of this epic a Java package. I point it to my training code. And once I have my training code in place, I configure it and with a few clicks, these are apps that are designed for engineers and scientists, with a few clicks be able to package it.
And it will go through the process of building out my JAR files, which I can then use the DBF as API, to move to Databricks, and execute at scale on the cloud. Like in this case, the output file is an example, .JAR file, but I can use the DBF as API to push up to the cloud. Once you have that, you can either do it interactively through the app, or in this case, the MCC application can be called either at the MATLAB command line or on system script. Without telling it your CICD mechanisms, allowing developers to just check in the artifacts and let the CICD tooling build and deploy the artifacts. So the build and test stages of packaging your algorithm can be automated, and we dovetail into a whole variety of both on premise systems that you may have spun up as a part of your organization, or cloud based CICD services, like circle CI or Travis CI, as your dev pipelines, to be able to improve the quality of the code and models, by continuously testing them, packaging them and enable your users to adhere to agile development practices.
For building out training code, let’s take a peek at what the insides of the training algorithm and the submission process looks like. Building on top of the building blocks that we just discussed, you should be able to define your infrastructure as code, leveraging the latest hardware, optimized drivers and toolkits. And there is an API for you to be able to connect and submit this to a Spark cluster. In fact, what you’re looking at on the right hand side is almost the entirety of the entire submission process, where I point it to an input location, specify what kind of application, my Spark app over here being the compiled artifact. Dealing with training hyper parameters and defining my infrastructure’s code.
Once I have my infrastructure’s code, I can then configure a job as a Databricks job and define that to be a Spark submit task, set the task and control the execution. That is create the task on the database system, execute it and refresh. Minimally, it’s almost the complete submission code. And what you are doing in this case is very quickly building an environment for repeatability. You can actually repeat this experiment and what this code will do is actually spin up the cluster with the Databricks runtime and the MATLAB enabled Spark cluster. Execute the training, pick the results of the training, store it out into a storage layer and then tear down the Databricks resources, tear down the underlying storage resources, and you can go and fetch your answers of you’re training model.
What that looks like when it executes is some visibility. In this case, I’m looking at the Spark driver logs where I dump things like the accuracy. But there are cleaner ways of actually dealing with this that I’ll be talking about in the next few slides. That involves the management of experiments and tracking of performance. There are graphical features like the experiment manager in MATLAB that will allow you to train these networks and to be able to tweak the hyper parameters. Either by using exhaustive sweep or by using hyper parameter optimizations. All of these call a certain setup function, which gets called for each trial of your experiment, you can define what kind of experiment and how to sweep the parameters.
At which point, a whole bunch of ways of getting started quickly, the ability to see confusion matrices and to study the performance and the progress of these tasks as they execute. All of this will allow people who are doing local experimentation on their computers to be able to work very quickly, but these features dovetail into bigger systems and as an example, what we’ll do here is correct the app to ml flow. In a couple of lines of code, I can give it a name. I’m going to create an experiment object and give it a name. In this case, I’ll call it D-A-I-S 2021 demo, a few lines of code, three lines of code here, I have created an experiment on ml flow. This gives us a way for us to very quickly handle our artifacts to track our performance, to register runs and all of that exposed through a very simple API.
This API connects up to the restful endpoints that ml flow actually exposes. So for example, if I were to create run, I would actually create a new run. And if I was collaborating with my colleagues, I could specify who is running this, at which point I have created a run for that particular experiment on ml flow, and within the run, it becomes possible for me to lock my parameters, lock by metrics, to be able to operate ml flow to track progress as well as track accuracies. In this case, I’m just as an example, showing you how we can log what kind of model it is, log metrics. In this case, I’ll measure my training loss and set it down as API calls. And in this case I just picking some dummy values to show you the building block of very quickly leveraging APIs like this to connect up to back end systems that will centralize the management of these experiments.
So both local experimentation, experiment manager. And if you wanted centralized experimentation tracking, both of those should be possible from all the MATLAB contexts. Going to what this looks like, what you can do is very quickly connect up to the experiment that we just created. This is the experiment that was created in those few lines. And there’s the run that we just created. Take a look at the parameters that were logged, as well as metrics. If I clicked on the metrics, possible for me to see this and see what training loss accuracies, training losses look like. Which map to the API calls that I made for it. Finally, these building blocks, these API can be plugged into existing resources. For example, experiment manager has a setup function that we talked about.
That setup function gives a very quick way to splice in these API calls. At the same time, both on the input as well as the completion of the training, it becomes possible to exercise API’s and log the results to ml flow. Once again, this takes very few lines of code. What I’m doing here is, for this particular experiment, I’m going to actually lock the parameters and metrics, by first locating the existing experiment that I just created, manually. And I get a handle to the experiment by name, at which point I can create my runs, as well as log any metrics pertaining parameters and metrics. Like in this particular case, I specify what the initial learning rate is, matching it up with the training options that I used for that particular trial.
All of this becomes programmatically accessible, and you can script and build your software around this. So all this leads up to better agility in building and running these models. These training runs on the cloud, on Databricks on Spark clusters. A quick look at the training benchmarks and performance, we did our little sweeping of parameters. If you were curious how mini batch sides varies, I can take one of the hyper parameters being the mini batch size for training the network, it gets better, my training time improves up to a point and then gets worse depending on limits in memory and limits in processing. And interestingly, as you look at what it does, in terms of trying the same training on different machines, the most powerful local machine I have access to with an Nvidia Titan X.
And the training time took nearly five hours. Moving to single GPU Spark instances, you can see the improvement in the training time, all the way down to the most R, for local machine, local workstation class card would bring it down substantially, but not as fast as almost a 3X, as fast as a G40. An extra large party GPU instance, this is a very beefy machine. And if I have to purchase it, it would be a very expensive machine. But I can actually burst the execution and perform the training on the Spark cluster, mounting against very much more powerful hardware that I have physically access to. So boiling the conversation back up to the higher level, the inference pipeline allows you to take those models and react.
Retrain those models based on a directed acyclic graph to resolve changes to any changes in the model structure. To any changes in the ETL pipelines, to changes in the data, or new data is added to the cache. And these train models can be deployed as cloud services. So we publish cloud reference architectures on github.com to integrate with existing systems. So if you have parts of this already in play, you can actually use those blueprints to make it easier. At this point, you have an inference engine that connects up to your storage layers and allows you to view if your performance and monitor it. This, we operationalize on MATLAB production server, allowing you to scale gracefully to be able to make optimal usage of underlying CPU and GPU resources and increase the throughput as desired.
Desired meaning to say based on performance and based on budget, you should be able to actually scale up to the size that you require. And pay as you go models are available, for when you’re on the cloud. On the Azure or AWS or on public cloud. Which brings us to the last part of this, these train models become a part of the ETL pipeline. And one of the things that is possible is that we have a whole array of specialized formats. Ross bags, MP4, MDF, automotive specific formats and essentially exposing it as restful client based access, allowing you to connect up to cloud storage and distributed streaming systems like Kefka, giving you a way of taking these inference engines and horizontally scaling it to run 24/7, 365, these models can be operationalized on headless cloud systems.
They can be accelerated to run on Nvidia GPUs and run proximal to the data when you’re running on the cloud. And what you’re seeing on the right here is, for example, a know parquet of MP4 artifacts, and this is almost the entirety of the code that is being used. So compact syntax is a very high level API. As I said, write less and do more. Once you actually have ETL code like this, that can be packaged and provision on the same production server instances. And giving you the ability to reuse that infrastructure piece for more than one use, for you to be able to handle inference as well as a system ETL process. So finally, we put it all together, we have a system here that will allow you to turn the crank quickly and improve algorithm development from prototype all the way up to production. One part of what you would want to consider is the ability to react, to be able to have systems like this to be triggered manually or automatically, based on changes.
Changes to the code, changes to the models or cache data, at which point you would be able to set it to run automatically in motion. So as design considerations, we want to make sure that we enable scientists and engineers with the best workflows to self serve their analysis, in this case, a few lines of code allowing them to self serve their own machines and infrastructures, optimized for agility along the maturity framework. And connecting up to the best in class cloud storage systems, to ingest larger data sets and best in class compute in the form of Spark clusters, to slice and dice and compute quicker training iterations both locally and on the cloud.
One artifact that comes out of it is that your capital costs and your operating costs, your capital costs, which could be very substantial, it gets moved to operating costs, where you can perform different runs of these based on what you need, based on spot instances and spot pricing on the cloud. It also addresses one problem in this domain, which is the siloed nature of research and engineering. So by adopting these DevOps practices, DevOps at its very core is more of a philosophy, you are enabling your research to be able to do a certain part of data engineering, and removing the symptomatic issues that result as a part of multiple steps, but keeping it all in one integrated experience.
We’ll look at accuracy. If you looked at the little video that I had in the beginning, you’ll notice it didn’t do a very good job of locating certain traffic signs, which isn’t surprising given that only a very small fraction of our training data actually have sign symbols. So as a part of it, she would look at it and say, “Yeah, my recorded data, I have many miles of it. But I don’t have enough sign symbols to be able to reliably detect a sign symbol.” And that’s where the simulation bits come into place. The ability to do scene generation and simulate, there is no dearth of synthetic data that you can actually augment to your training data to improve the quality of your predictions.
Ground truthing as well as scenario generation, will give a very quick way to be able to build synthetic scenarios and be able to train these networks quicker and faster. And if you have to get to a billion miles of data, this is one very essential tool that will get you to better outcomes for perception. Finally, systems like this need to interoperate with the rest of the software ecosystem around it to be able to talk to interact with onyx based standards. We enable transfer learning based techniques to be able to ingest models built in other platforms, and important export to a variety of industry specific formats. Opendrive, FPX, etc. We can also connect up to automated driving system simulators and our software packages for actually handling this kind of work, as well as other game engines such as Unity and Unreal.
Having your motor in that form, will then also allow you to exercise the embedded targeting features. This is a very mature workflows. That’s a completely different topic by itself. But what you can do is produce processor optimized code, to go down to an embedded system that you place in the vehicle itself. In this case, GPU code, targeting GPU, the Nvidia drive and the Jetson hardware platforms, gives you a quick way to deploy your models onto any network on the ADH processor for best in class performance. And it will run in an optimized fashion. So with that, for turning the crank, you also want to put a little bit of thought into governance and lifecycle management, to make it easy for people to work locally with no connectivity at all, to work locally.
But use the same work to enable centralized tracking to be able to scale as they work across bigger organizations. And finally, since we are building this on Spark classes and database Spark clusters, we inherit their cloud storage best practices, their controllers based permissions for the data set as well as identity and access to control who can see what data and what they can do with it. Which brings me to the conclusion of this talk, which is how to build well architectured systems to accelerate the development of, and automate parts of semantic segmentation and other applications against large data sets. Local development provides smooth workflows, the ability to stop and debug, to be able to very quickly iterate on prototypes for the development and refinement of these deep learning models, which you can then scale up elegantly on cloud based systems and leverage systems like Databricks to enable self serve analytics.
These techniques are not just relevant for radar’s development, which is the topic of this particular talk. But they’re equally applicable across other domains such as medical, geo exploration, etc. With that, if there were key takeaways, it would be to use DevOps ready development tools to build upon proven ways of actually accelerating development of these algorithms, by leveraging best in class simulation and integration and modeling tools for faster time to market outcomes. And finally, write less and do more, providing the right level of API and the right level of tooling will allow people to spend less time writing code and doing more. And if you did find this talk interesting, I can request you to contact us at [email protected] for more details or to get started. With that, we can switch to question and answers. Thank you very much.
Arvind Hosagrahara leads a team that helps organizations deploy MATLAB algorithms in critical engineering applications, with a focus on integrating MATLAB into the enterprise IT/OT systems. Arvind has...