A new software system developed at the University of Michigan uses video game technology to help solve one of the most daunting hurdles facing self-driving and automated cars—the high cost of the laser scanners they use to determine their location.
Ryan Wolcott, a U-M doctoral candidate in computer science and engineering, estimates that it could shave thousands of dollars from the cost of these vehicles. The technology enables them to navigate using a single video camera, delivering the same level of accuracy as laser scanners at a fraction of the cost. His paper detailing the system recently was named best student paper at the Conference on Intelligent Robots and Systems in Chicago.
“The laser scanners used by most self-driving cars in development today cost tens of thousands of dollars, and I thought there must be a cheaper sensor that could do the same job,” he said. “Cameras only cost a few dollars each and they’re already in a lot of cars. So they were an obvious choice.”
Last year Oxford modified a Nissan Leaf to use off the shelf cameras for self driving. At Oxford, lasers and cameras were subtly mounted around the vehicle and taking up some of the boot space is a computer which performs all the calculations necessary to plan, control speed and avoid obstacles. Externally it’s hard to tell this car apart from any other on the road. It was designed to take over driving while traveling on frequently used routes. There were three computers onboard the Oxford robocar. The iPad, the LLC (Low Level Controller) and the MVC (Main Vehicle Computer). The iPad runs the user interface and demands constant attention from the LLC
Visual Localization within LIDAR Maps for Automated Urban Driving (8 pages)
His system builds on the navigation systems used in other self-driving cars that are currently in development, including Google’s vehicle. They use three-dimensional laser scanning technology to create a real-time map of their environment, then compare that real-time map to a pre-drawn map stored in the system. By making thousands of comparisons per second, they’re able to determine the vehicle’s location within a few centimeters.
Wolcott’s system uses the same approach, with one crucial difference—his software converts the map data into a three-dimensional picture much like a video game. The car’s navigation system can then compare these synthetic pictures with the real-world pictures streaming in from a conventional video camera.
Ryan Eustice, a U-M associate professor of naval architecture and marine engineering who is working with Wolcott on the technology, said one of the key challenges was designing a system that could process a massive amount of video data in real time.
“Visual data takes up much more space than any other kind of data,” he said. “So one of the challenges was to build a system that could do that heavy lifting and still deliver an accurate location in real time.”
To do the job, the team again turned to the world of video games, building a system out of graphics processing technology that’s well known to gamers. The system is inexpensive, yet able to make thousands of complex decisions every second.
“When you’re able to push the processing work to a graphics processing unit, you’re using technology that’s mass-produced and widely available,” Eustice said. “It’s very powerful, but it’s also very cost-effective.”
The team has successfully tested the system on the streets of downtown Ann Arbor. While they kept the car under manual control for safety, the navigation system successfully provided accurate location information. Further testing is slated for this year at U-M’s new M City test facility, set to open this summer.
The system won’t completely replace laser scanners, at least for now—they are still needed for other functions like long-range obstacle detection. But the researchers say it’s an important step toward building lower-cost navigation systems. Eventually, their research may also help self-driving vehicle technology move past map-based navigation and pave the way to systems that see the road more like humans do.
“Map-based navigation is going to be an important part of the first wave of driverless vehicles, but it does have limitations—you can’t drive anywhere that’s not on the map,” Eustice said. “Putting cameras in cars and exploring what we can do with them is an early step toward cars that have human-level perception.”
The camera-based system still faces many of the same hurdles as laser-based navigation, including how to adapt to varying weather conditions and light levels, as well as unexpected changes in the road. But it’s a valuable new tool in the still-evolving arsenal of technology that’s moving driverless cars toward reality.
The paper is titled “Visual Localization within LIDAR Maps for Automated Urban Driving.” This work was supported by a grant from Ford Motor Co. via the Ford-UM Alliance under award N015392; Wolcott was supported by the SMART Scholarship for Service Program by the U.S. Department of Defense.
This video demonstrates our robotic platform visually localizing in a 3D LIDAR map. Scenes are synthetically generated using OpenGL and compared against the live camera feed using Normalized Mutual Information.
Abstract – Visual Localization within LIDAR Maps for Automated Urban Driving
This paper reports on the problem of map-based visual localization in urban environments for autonomous vehicles. Self-driving cars have become a reality on roadways and are going to be a consumer product in the near future. One of the most significant road-blocks to autonomous vehicles is the prohibitive cost of the sensor suites necessary for localization. The most common sensor on these platforms, a three-dimensional (3D) light detection and ranging (LIDAR) scanner, generates dense point clouds with measures of surface reflectivity—which other state-of-the-art localization methods have shown are capable of centimeter-level accuracy. Alternatively, we seek to obtain comparable localization accuracy with significantly cheaper, commodity cameras. We propose to localize a single monocular camera within a 3D prior groundmap, generated by a survey vehicle equipped with 3D LIDAR scanners. To do so, we exploit a graphics processing unit to generate several synthetic views of our belief environment. We then seek to maximize the normalized mutual information between our real camera measurements and these synthetic views. Results are shown for two different datasets, a 3.0 km and a 1.5 km trajectory, where we also compare against the state-of-the-art in LIDAR map-based localization.
SOURCES – University of Michigan, Youtube, Oxford
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.