Toyota Develops Domestic Help Robots In Drive To Provide Mobility For All

The concept of robots doing everyday chores around the house was once the dream of a leisure-focused future, but for Toyota it’s part of a mobility mission that can help less able people lead independent lives in their own homes.

Toyota Research Institute (TRI) is taking the lead in developing advanced robotics that can provide practical help for elderly and disabled people and those recovering from illness or injury.

Robots that know how to load the dishwasher, can recognise the different rooms in a house and have a “bubble grip” hand that can safely move delicate objects all feature in the latest advances developed by the teams at TRI’s research centres in the USA.

Their work is a perfect expression of how Toyota as a mobility company wants to make the freedom of movement and the opportunities that brings available to everyone in society.

TRI is working on home-based assistance concepts to help address the challenges presented by the world’s aging population. According to the United Nations, during the next three decades, the global population aged over 65 is predicted to more than double, reaching a total of more than 1.5 billion by 2050. TRI believes its research into advanced human-assist robots will help people grow older with dignity, in their own homes.

The research is guided by the Japanese notion of ikagi, that everyone’s life should have meaning and purpose. This human-centred approach turns typical AI (artificial intelligence) philosophy on its head: instead of replacing human beings, TRI is using AI to amplify human ability – an approach known as Intelligence Amplification (IA), where humans and machines work in synergy to do things better than either could on their own.

Dr Gill Pratt, CEO of TRI and Chief Scientist of Toyota Motor Corporation, explained: “To enable more people to achieve their ikigai, TRI is pursuing new forms of ‘automation with a human touch (what’s known as jidoka in the Toyota Production System) to develop capabilities that amplify, rather than replace, human ability with the goal of bringing happiness and fulfilment to all people.”

The potential and challenges of human amplification robots

As societies age, there will be huge demand for care-giving, systems that allow people to live independently for longer and assistance for an aging workforce. Robots and automation can play a key role in freeing up more time for people to spend with their families, helping people with tasks they enjoy, or assisting them to perform work tasks.

TRI believes robots aren’t seen in these roles today because scientists haven’t yet figured out how to make robots operate reliably in the complex, unstructured environments people function in every day. Unlike a factory, where the environment is structured and programmable, people’s homes are all different, with objects in different and constantly changing configurations.

“TRI robotics research is focused on the home because it’s in that environment that robots can provide the greatest assistance and achieving human fulfilment. It is also one of the complex environments for robots to master,” said Max Bajracharya, Vice President of Robotics at TRI. “Our work focuses on two key challenges: teaching robots from human behaviour and using simulation to train and validate robot behaviours.”

TRI describes this as fleet learning, where when one machine learns something, they all learn something – something which will be key to making the use of robots practical in human environments.

Teaching robots

To deal with the diversity it will encounter in a home environment, TRI teaches the robot to perform arbitrary tasks with a variety of objects, rather than programme it to do predefined tasks with specific objects. In this way, the robot learns to link what it sees with what it has been taught, so that if it sees an object and scenario again, even if the scene has changed slightly, it will know what it actions it can perform with respect to what it can see.

TRI is using humans as teachers to enable robots to learn and acquire real-world skills. Using virtual reality, a human trainer sees what the robot sees in real time, then commands it to perform different actions. The aim is to generalise this type of learning, so if one robot learns a wiping task in a mock home kitchen, it could do the same task in any kitchen. More information about TRI’s teaching process and fleet learning is available here.

Simulation for manipulation research

TRI has made a significant investment in simulation for engineering and validating robot behaviours. The mechanics of the way a robot hand interacts with objects is very complicated to simulate, so simulations have traditionally not been used for robotic manipulation research.

TRI’s simulation software provides a way to understand a robot’s performance without having to physically perform all the tasks every time a change is made, when teaching a robot a new behaviour, or refining one that it already knows. Simulation results are tested in mock-up kitchens in the TRI lab. Using simulation tools can also advance robotic development, even when there is limited access to hardware and testing facilities.

“We’ve used our dish-loading robot and clutter-clearing experiments to automatically improve our behaviours in simulation and then have that result in improved performance on the real robots,” said Russ Tedrake, TRI Vice President of Robotics Research. The work of the TRI robotic manipulation, team based in Cambridge, Massachusetts, is featured here.

New robotic hardware and software

TRI is also looking at more radical ideas for the home, including a “gantry robot” that would descend from an overhead framework to do tasks such as loading the dishwasher and wiping surfaces. By travelling on the ceiling, it avoids the problem of negotiating household floor clutter and cramped spaces. When not in use, it tucks itself neatly out of the way. The team has built a prototype of this concept that can do all the same tasks as a mobile floor robot.

Another innovation is a set of soft grippers with a high-density tactile-sensing capability. Each gripper’s surface is covered by a bubble that gives gently when it is in contact with an object. The degree of compliance can be controlled by changing the pressure in the bubble, while a depth camera inside the bubble senses the shape of the bubble skin and tracks motion to estimate shear forces on the surface. More details can be found here.

TRI’s “mock home”

A new “mock home” complete with kitchen, bathroom, dining room and living space has been created at TRI’s California headquarters as a robotics testing facility. The research team can change the floorplan and move objects to help the development of basic robot capabilities before testing them in a real home.

Meeting real user needs

For robotics to be successful in the home, TRI believes it is important to discover and take into account individual human tastes, needs and means of fulfilment. It takes a “fail fast” approach to technology development so that its work can more quickly make a positive impact on actual lives and improve social good. Rather than simply developing capabilities that researchers find interesting or think will push the field forward, TRI has a Robotics User Experience and Industrial Design group to uncover and probe real user needs.

“We rely heavily on observational research techniques such as contextual inquiries,” said Steffi Paepcke, a TRI User Experience Team Lead. “Before Covid-19, we went to Japan to work with our research partners to visit the homes of older adults and observe them going about their daily lives, making note of friction points, challenges, and opportunities.

“We observed that cooking is a beloved activity for many, though it can get more strenuous over time. Sharing meals and feeding loved ones also can serve as a focal point for social connection…, so giving elderly people a fully automated cooking robot or pre-cooked meals might be physically beneficial but emotionally detrimental.”

The challenge is to understand how to develop human-centred robotics when each individual user is a unique case. The goal is not to just give people gadgets, but to find ways to enable people to achieve fulfilment and help society, each in their own way.

Toyota collaboration

TRI’s robotics charter is to develop new robotics capabilities for Toyota that can contribute to solving real world problems and aiding global societies. To create real-world impact, it is working closely with other groups inside of Toyota, including Toyota AI Ventures, Toyota’s first corporate venture capital firm that strategically invests in early stage start-ups.

Another group is the Woven Planet Holdings Group – previously TRI-AD – which focuses on taking new capabilities from TRI and other parts of Toyota and, in collaboration with them, developing them into product concepts.

ENDS

Note to editors: a full TRI robotics press kit including videos is available here.