Autonomous vehicles: University of Toronto researchers make advances with new algorithm

Researchers at the University of Toronto’s department of computer science and Uber’s Advanced Technologies Group (ATG) in Toronto have developed an algorithm that jointly reasons about all these tasks –  the first to bring them all together. Importantly, their solution takes as little as 30 milliseconds per frame.

“We try to optimize as a whole so we can correct mistakes between each of the tasks themselves,” says Wenjie Luo, a PhD student in computer science. “When done jointly, uncertainty can be propagated and computation shared.”

Luo and Bin Yang, a PhD student in computer science, along with their graduate supervisor, Raquel Urtasun, an associate professor of computer science and head of Uber ATG Toronto, will present their paper, Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net, at this week's Computer Vision and Pattern Recognition (CVPR) conference in Salt Lake City, the premier annual computer vision event.

To start, Uber collected a large-scale dataset of several North American cities using roof-mounted Li-DAR scanners that emit laser beams to measure distances. The dataset includes more than a million frames, collected from 6,500 different scenes.

Urtasun says the output of the LiDAR is a point-cloud in three dimensional space that needs to be understood by an artificial intelligence (AI) system. This data is unstructured in nature, and is thus considerably different from structured data typically fed into AI systems, such as images.

“If the task is detecting objects, you can try to detect objects everywhere but there's too much free space, so a lot of computation is done for nothing. In bird's eye view, the objects we try to recognize sit on the ground and thus it's very efficient to reason about where things are,” says Urtasun.

To deal with large amounts of unstructured data, PhD student Shenlong Wang and researchers from Uber ATG developed a special AI tool.