The team at the Chair of Learning Systems and Robotics at TUM (Technical University of Munich) has developed a modest-looking robot that resembles a stick on wheels with a camera at the top. However, don’t let the looks fool you. It could be one of the most useful robots designed for everyday people.
Led by Prof. Angela Schoellig, the team has built a robot that can find lost objects by creating and analyzing a spatial map of its surroundings. Next time you can’t find your keys or your glasses, don’t lose your mind because this robot will find them for you.
How does the robot find things?
The camera provides two-dimensional images, but these pixels also contain depth information. From this, the robot creates a centimeter-precise 3D map of its surroundings and constantly updates it as things change.
Technical University of Munich
A challenge with this approach is that objects are constantly being moved or replaced, causing the map to quickly become outdated. This causes the robot to scan the entire area again. To solve this problem, the researchers used an LLM-powered model to not only map the environment but also maintain and update the data.
It tracks objects and assigns a relevant rating. The score, time since the object was last seen, and other data points are then used to build a probabilistic model and decide which areas to scan and maintain.
Technical University of Munich
What makes it really clever is the layer of internet knowledge embedded in it. The robot understands that glasses are likely to be left on the table or windowsill, not on the stove or in the sink.
A language model then translates these real-world considerations into search probabilities, helping the robot focus on areas where the missing object is most likely to be located. This means the robot searches almost 30% more efficiently than when randomly scanning rooms.
What awaits the future of this robot?
Currently the robot is limited to open areas. The next challenge the team faces is teaching him to open drawers and cupboards so he can search in enclosed spaces.
It’s still early days, but a robot that truly understands your home and helps you find things in it feels more useful as a home robot than other AI robot projects we’ve seen in the past.
