​

A-Eye Can See Millions of Colors for Better Automation

by

Researchers at Northeastern have built a device that can recognize “millions of colors” using new artificial intelligence techniques. “In the world of automation, shapes and colors are the most commonly used items by which a machine can recognize objects,” Kar says.

The breakthrough is twofold. Researchers were able to engineer two-dimensional material whose special quantum properties, when built into an optical window used to let light into the machine, can process a rich diversity of color with “very high accuracy”—something practitioners in the field haven’t been able to achieve before.

Additionally, A-Eye is able to “accurately recognize and reproduce ‘seen’ colors with zero deviation from their original spectra” thanks, also, to the machine-learning algorithms developed by a team of AI researchers, helmed by Sarah Ostadabbas, an assistant professor of electrical and computer engineering at Northeastern. The project is a result of unique collaboration between Northeastern’s quantum materials and Augmented Cognition labs.

Machines typically recognize color by breaking it down, using conventional RGB (red, green, blue) filters, into its constituent components, then use that information to essentially guess at, and reproduce, the original color. When you point a digital camera at a colored object and take a photo, the light from that object flows through a set of detectors with filters in front of them that differentiate the light into those primary RGB colors.

You can think about these color filters as funnels that channel the visual information or data into separate boxes, which then assign “artificial numbers to natural colors,” Kar says.

“So if you’re just breaking it down into three components [red, green, blue], there are some limitations,” Kar says.

Instead of using filters, Kar and his team used “transmissive windows” made of the unique two-dimensional material.

“We are making a machine recognize color in a very different way,” Kar says. “Instead of breaking it down into its principal red, green and blue components, when a colored light appears, say, on a detector, instead of just seeking those components, we are using the entire spectral information. And on top of that, we are using some techniques to modify and encode them, and store them in different ways. So it provides us with a set of numbers that help us recognize the original color much more uniquely than the conventional way.”

Materials Today – Dispersion-free highly accurate color recognition using excitonic 2D materials and machine learning

bstract
Dispersion is accepted as a fundamental step required for analyzing broadband light. The recognition of color by the human eye, its digital reproduction by a camera, or detailed analysis by a spectrometer all utilize dispersion; it is also an inherent component of color detection and machine vision. Here, we present a device (called artificial eye or, A-Eye) that accurately recognizes and reproduces tested colors, without any spectral dispersion. Instead, A-Eye uses N = 3–12 transmissive windows each with unique spectral features resulting from the broadband transmittance and excitonic peak-features of 2D transition metal dichalcogenides. Colored light passing through (and modified by) these windows and incident on a single photodetector generated different photocurrents, and these were used to create a reference database (training set) for 1337 “seen” and 0.55 million synthesized “unseen” colors. By “looking” at test colors modified by these windows, A-Eye can accurately recognize and reproduce “seen” colors with zero deviation from their original spectra and “unseen” colors with only ∼1 % median deviation, using the k-NN algorithm. A-Eye can continuously improve color estimation by adding any corrected guesses to its training database. A-Eye’s accurate color recognition dispels the notion that dispersion of colors is a prerequisite for color identification and paves the way for ultra-reliable color-recognition by machines with reduced engineering complexity.

7 thoughts on “A-Eye Can See Millions of Colors for Better Automation”

  2. So instead of a normal hyperspectral imager measuring thousands of frequencies, they’re just measuring 12. That doesn’t sound much better than than the common 3.

    Then they claim that the AI can guess what it’s missing. Ok. But that would depend entirely on the training set and test set. If you tested it on light from a free electron laser tunable to thousands of colors, choosing one randomly with a uniform probability, then obviously it would fail. Because it simply doesn’t have enough information.

    So their “success” would have to be entirely in how they constrained their training set and test set. The article gives no clue how they did that. So it’s impossible to tell whether their system has even a little bit of value.

  3. Kind of unclear, without access to the original paper. (It seems to be paywalled.) So, this is a hyperspectral camera with a fairly limited spectral resolution, but providing enough spectral data that it can emulate the frequency response of a human eye?

    I think hyperspectral cameras have a lot of potential for giving machines superior color vision compared to humans, providing information humans just can’t see. But you’d need more spectral channels than just 12 to get the full potential there. While just emulating human vision I suppose has utility in an environment designed for humans.

Comments are closed.