Robots That Talk Too Much and Don't Listen Enough
Organizations: CDM Interact and University of Helsinki, Helsinki, Finland · University of Helsinki and CNRS-AIST Joint Robotics Lab, Tsukuba, Japan · Also Visiting Professor at Institute of Science Tokyo, Tokyo, Japan.
Abstract
Until recently, the biggest obstacles to natural spoken interaction with social robots were speech recognition errors and turn-taking failures. Advances in speech recognition have enabled much better spoken interaction, but turn-taking failures and other misunderstandings continue to be far too prevalent. This short paper looks back at earlier methods for improving speech recognition and turn-taking in interaction with social robots, and then looks at what can be done today, arguing that multimodal bidirectional streaming is needed to reduce the frequency of turn-taking failures.
Figures & tables
| Robot: | Welcome to LangGraph agent. How can I help you? |
| User: | I’m Kristiina, I’m planning a trip to Naples. |
| Robot: | Hi, Kristiina, that sounds exciting. What do you have planned for your trip to Naples? |
| User: | Well, I’m interested in history, so what would you recommend? |
| Robot: | For history, definitely visit the National Archaeological Museum and the ruins of Pompeii. The Castel Nuovo and the catacombs of San Gennaro are also fascinating. |
| User: | Tell me more about Pompeii. |
| Robot: | Pompeii is an ancient Roman city that was buried under volcanic ash from Mount Vesuvius in 79 AD. It’s incredibly well preserved, offering a glimpse into daily life in ancient Rome. You can explore homes, streets, and even frescoes. The site is vast, so consider taking a guided tour to get the most out of your visit. |
| User: | What have you learned about me in this conversation? |
| Robot: | I’ve learned that you’re Kristiina, planning a trip to Naples, and you’re particularly interested in historical sites like Pompeii and Castel Nuovo. If there’s anything else you’d like to share or ask, feel free. |
| User: | How long will you store this information? |
| Robot: | I’ll remember this information for our future conversations so I can help you better. If you ever want to update or remove anything, just let me know. |
| User: | Alright, if I want to delete some information, can you do it? |
| Robot: | If you have any more questions or need assistance, just ask. Enjoy your planning. |
| User: | Can you also delete your information if I want to? |
Human-Robot Interaction
Human-robot interaction studies how people and robots communicate, coordinate, and safely work or live together.
See all 475 papers