Happy robots, happy cities? New learning system mimics human emotion to control autonomous systems

Hao Xu and students outdoors with two aerial robots and one ground-based rover

More than 20 billion Internet-connected devices are projected to be in use by 2020, according to forecasts from technology research firm Gartner, Inc.
That means everything from your light bulbs to your refrigerator can be online, happily adjusting your lighting preferences and adding eggs to your shopping list. Outside the home, power grids and traffic lights will be connected to smart monitoring systems, conserving energy and speeding commuters home with optimized signal timing.

But a world of proliferating devices requires new approaches to networked control, particularly in applications where real-time responses are critical.

Hao Xu

Hao Xu

"Autonomous systems are not independent systems anymore," said Hao Xu, assistant professor of electrical engineering. "They have become big systems because the computation, communication and physical systems have been mixed together. They have also been given different names - cyber-physical systems, or networked control systems, or some people call them the Internet of things or smart cities. Everything has been connected."

The benefits of large, intelligent networks are obvious, but designing autonomous systems capable of acting in such complex, interconnected environments poses unique challenges for researchers. Xu is focused on the emerging area of real-time machine learning, in which autonomous systems are able to adapt immediately to changes in their environment.

"Learning has been studied very well, but a challenge is most learning technologies are offline, which means their computational complexity is really high. You have to run your computer for years or days or hours to get a solution, and that's too late for real-time applications," Xu said. "The research we are doing is real-time, learning-based intelligent networked control."

Novel learning system based on human brain, emotion

Xu's research is based on a wonderfully simple concept - just let the machine figure it out. Xu's model learning system has two parts: an upper-level learning system that focuses on developing an optimal plan to complete a mission and a lower-level learning system that can effectively achieve that optimal plan, even in complicated and harsh environments.

Leaving so much up to the robot may seem risky, but Xu's model is based on what he believes to be the most efficient real-time learning system out there - the human brain.

"The specific problem is the environment, a harsh environment or complicated environment, how do we handle that? And some kind of fault or problem from the sensor or from the robot itself, can we handle it and make sure the application is still optimal and resilient?" Xu said. "It's actually inspired from biology, from human beings, who can react to this kind of complex situation very well."

A human brain is a complex structure, but Xu's essential insight is to mimic the feedback capacity of human emotion as way to help robots learn if their behavior is optimal.

"If you're happy, you keep doing good. If you're sad, you think about a way to make it better. That's the basic idea," Xu said. "It's simple, because if it's complicated, it's not easy to implement in real machines because the computational complexity is going to get really high."

Graduate students in Hao Xu's lab with numerous autonomous robots

Graduate students in Hao Xu's lab are testing their learning system with both aerial and ground-based robots

Xu's approach offers at least one major advantage over traditional approaches. Instead of control techniques that require autonomous agents to generate a model of their environment - and update that model in response to continuously changing circumstances - the emotion-based learning only requires robots to focus on outputs, namely whether its actions generate a positive emotional state.

From a human perspective, Xu's methodology is also fairly hands-off. Thanks to the robot's two-level learning system, the complex task planning and real-time control are handled by the machine. A human supervisor can monitor the system at a higher level to ensure everything is functioning reliably and intervene if needed, but Xu anticipates robots will be intelligent enough to adapt to changing conditions.

"For a smart city, you don't need to worry about, ‘Okay this traffic light agent, is he thinking about how to pick up different traffic solutions?' He will realize, ‘Okay, today is a busy day, so we have to change our traffic light timing. Today is not a busy day, so I have to change it to always green,'" Xu said.

In fact, Xu sees real-time learning as transformative for the development of smart cities.

Real-time learning could transform cities, manufacturing, space exploration

"This technology is a game changer, because everything in the city is going to be included together," Xu said. "So of course you can train everybody using high-performance computing clusters, but it's still going to take tremendous time and you cannot completely predict what the real environment looks like. So the best way is to let them run and let them try to think by themselves. It's more like an ecosystem."

The ecosystem Xu envisions is one in which smart robots have earned the trust of human supervisors, who, in turn, are invested in maximizing the almost human-sounding potential of those robots.

"A lot of robots have been used for smart manufacturing," Xu said. "It's best to let the robot figure out how to adjust their manufacturing ability. You don't treat the robot like a machine, you treat them like a worker. Then you have this intelligent person, they are going to do a better job and they are going to significantly improve their performance."

In fact, Xu believes that better robots might help humans solve one of the most daunting challenges in our future - space colonization.

"For Mars, for outer space exploration, robots will be a very important part of that," Xu said. "This is a very big step for human beings to try to live on another planet, and before that you have to build a base station. Nobody knows what's going on on Mars. No matter how many times you send the rover there to collect data, there's no way you can predict what's really going to happen in the next second. The best way is to let the robot figure it out. They're going to live there for years to help us build the base. You have to give them some intelligence, so let them do the job."

To some, Xu's robotic systems - capable of learning from their environment and adapting to it in pursuit of their goals - might sound reminiscent of long-standing fears of artificial intelligence outsmarting its human creators. But Xu believes smarter robots ultimately result in smarter humans and smarter societies.

"I'm not really worried about artificial intelligence because I think if we do it the right way, robots are going to assist people and it's going to give us a revolution for ourselves," Xu said. "Everybody's thinking about robots getting better, humans getting worse. No. They're getting better; we're getting better. What we want to do is give them a little bit of intelligence they can use to better help or serve people."