Andrej Karpathy Gives a Technical Update on Tesla Self Driving AI

Andrej Karpathy, Tesla Senior Director of AI gave a technical talk on Tesla’s self-driving AI at the Matroid Scaled Machine Learning Conference 2020.

If Tesla is the first to succeed in the development of full self-driving, then this will enable Tesla to become a multi-trillion company

Tesla has a few dozen developers in the AI team but they have a lot of tools for automation of problem analysis and case harvesting from the data from the car fleet.

Tesla is working on pseudo LIDAR using vision only systems. Tesla is working to achieve primarily neural net-based software and algorithms. They want the neural net systems to learn how to make more accurate depth predictions.

Tesla has a fleet of over million autopilot cars that are uploading and handling many traffic cases. Tesla’s AI team is working through the long tail of exception cases to determine how to drive with complete safety.

There is a lot of complexity. Tesla has tens to hundreds of cases every day of pedestrians moving from between parked cars or being hidden before while in traffic.

There is a lot of complexity with stop signs. Stop signs can be held in someone’s hand and it is inactive until it is raised.

Tesla is curating the test datasets. There are many examples of partially obscured stop signs. They will work to say improve performance on a particular dataset from say 40% to 99%.

They are working towards neural net components that generate a bird-eye view of the vehicle and its surroundings. This is useful for tracking the road edge problems.