AI system lets robots teach themselves to see

Technology News | September 11, 2018

By Rich Pell

Optoelectronics Software & Embedded tools Electromechanical Data Acquisition Memory & Data Storage MPUs/MCUs Digital Signal Processing Sensing / Conditioning

The system is designed to address the fact that while robots in controlled environments like assembly lines are able to pick up the same object over and over again, the systems don’t truly understand objects’ shapes. While advances in computer vision have enabled robots to make basic distinctions between objects, there’s still “little the robots can do after a quick pick-up.”

The new system lets robots inspect random objects, and visually understand them enough to accomplish specific tasks without ever having seen them before. The system, called Dense Object Nets (DON), looks at objects as collections of points that serve as sort of visual roadmaps.

This approach, say the researchers, lets robots better understand and manipulate items, and, most importantly, allows them to even pick up a specific object among a clutter of similar objects. For example, the system might be used to get a robot to grab onto a specific spot on an object, such as the tongue of a shoe. In that case, it can look at a shoe it has never seen before, and successfully grab its tongue.

“Many approaches to manipulation can’t identify specific parts of an object across the many orientations that object may encounter,” says PhD student Lucas Manuelli, co-author of a paper about the system. “For example, existing algorithms would be unable to grasp a mug by its handle, especially if the mug could be in multiple orientations, like upright, or on its side.”

Two common approaches to robot grasping involve either task-specific learning, or creating a general grasping algorithm. Such techniques have their drawbacks: Task-specific methods are difficult to generalize to other tasks, and general grasping doesn’t get specific enough to deal with the nuances of particular tasks, like putting objects in specific spots.

However, the DON system, say the researchers, essentially creates a series of coordinates on a given object, which serve as a kind of visual roadmap, to give the robot a better understanding of what it needs to grasp, and where. The system is trained to look at objects as a series of points that make up a larger coordinate system, at which point it can then map different points together to visualize an object’s 3D shape.

After training, if a person specifies a point on a object, the robot can take a photo of that object, and identify and match points to be able to then pick up the object at that specified point. The system is “self-supervised,” not requiring any human annotations.

During testing of the system, a Kuka robotic arm powered by DON could grasp a soft caterpillar toy’s right ear from a range of different configurations, showing that the system has the ability to distinguish left from right on symmetrical objects. When testing on a bin of different baseball hats, DON could pick out a specific target hat despite all of the hats having very similar designs – and having never seen pictures of the hats in training data before.

“In factories robots often need complex part feeders to work reliably,” says PhD student Pete Florence, lead author of the paper on the research. “But a system like this that can understand objects’ orientations could just take a picture and be able to grasp and adjust the object accordingly.”

Potential applications however are not just in manufacturing settings, say the researchers, but also in homes. For example, the system could be provided with an image of a tidy house, and then left to clean the house while the owners are at work, or given an image of dishes so that the system puts plates away while its owners are on vacation.

Looking ahead, the researchers hope to improve the system so that it can perform specific tasks with a deeper understanding of the corresponding objects, such as learning how to grasp an object and move it with the ultimate goal of, for example, cleaning a desk. For more, see “Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation.”