Their system, called Brainput, is designed to recognize when a person’s workload is excessive and then automatically modify a computer interface to make it easier. The researchers used a lightweight, portable brain monitoring technology, called functional near-infrared spectroscopy (fNIRS), that determines when a person is multitasking. Analysis of the brain scan data was then fed into a system that adjusted the user’s workload at those times. A computing system with Brainput could, in other words, learn to give you a break.
Mental load: A user tries the Brainput system. Erin Treacy Solovey
There are other ways that a computer could detect when a person’s mental workload is becoming overwhelming. It could, for example, log errors in typing or speed of keystrokes. It could also use computer vision to detect facial expressions. “Brainput tries to get to closer to the source, by looking directly at brain activity,” says Erin Treacy Solovey, a postdoctoral researcher at MIT. She presented the results last Wednesday at the Computer Human Interaction Conference in Austin, Texas.
Brainput was used with into virtual robots designed to adapt to the mental state of their human controller. The main goal was for each operator, capped with fNIRS headgear, to guide two different robots through a maze to find a location where a Wi-Fi signal was strong enough to send a message. But here’s what made it tough: the drivers had to constantly switch between the two robots, trying to keep track of both their locations and keep them from crashing into walls.
As the research subjects drove their robots toward the strongest Wi-Fi signal, their fNIRS sensors transmitted information about their mental state to the robots. The robots, for their part, were programmed to focus on a state of mind called branching, in which a person is simultaneously working on two goals that require attention. (Previous studies have correlated certain fNIRS signals to this sort of mental state.) When the robots sensed that the driver was branching, they took on more of the navigation themselves.
The researchers found that when the robots’ autonomous mode kicked in, the overall performance of the human-robot team improved. The drivers didn’t seem to notice or get frustrated by the autonomous behavior of the robot when they were multitasking. The researchers also tried increasing the autonomy of the robots when Brainput did not indicate that users were mentally overloaded. When they did this, they found that overall performance decreased. In other words, increased autonomy only helped when users were struggling to cope.