[HEART#1] Social robotics Today and Tomorrow: Extra Q&A
On 12th October 2020 SBRE hosted the first webinar of the HEART series, Social Robotics Today and Tomorrow: an Overview. In the webinar, 3 international experts of social robotics unraveled the why, what and how of the present and the future of social robots, and discussed the many opportunities and challenges offered by the multidisciplinarity of this domain.
You can watch the replay and read the summary of the webinar in our recent blog post here!
The webinar finished with the questions from the audience. In this page you can find the speakers’ answers to these questions:
1. Do you think that one day robot companions such as NAO and Pepper will be deployed at home for consumers? Until now, those wonderful technologies have been used in B2B, education or for research. For many years, people expected to get those robots at home. According to you, what is missing for a massive deployment and what will be the expected time frame ( 5/10/15 years)?
Victor: “I do believe that NAO could be deployed if it was super intelligent. I do not know how far our business partners went down the AI road, but I am afraid we are not there yet. At SBRE, I think we are years away, and we rather focus selling it to more proven markets. Gamification could also lead to a consumer product, like Cozmo, but the price of NAO is prohibitive for such use”.
2. What should be a future function of the robot to interact with people and other products in a connected city?
Ioana: “Maybe services? We already use automatic devices in the city and some experiments were done for the robots as guides in malls, trash bins in train stations etc. Another reference comes in my mind from the series Maniac where you have this cute poop bot (https://vimeo.com/294118081) - that type of service cleaning streets could be done by an autonomous robot... At Strate we also imagined a future with robotic furniture in a smart city: a bench, a lamp and a trash bin. These objects still serve their original function (to act as seats, to provide light and to discard waste), but they also reflect Marivaux's theatrical style. We called these objects “robjects” (robot+objects). Read more about this project here and about “robjects” here.
Victor: “And let's not forget about interoperability. If every service and product spoke the same web language, it would enable many more developments! Please use and respect standards, like W3C's or IEEE's, it will empower us all”.
3. Nao and Pepper have Concept and Knowledge modules linked to their conversational abilities, do you use them in your projects?
Victor: “I used the Knowledge service in my PhD! The API is well bridged with QiChat (our script language for writing dialogues), so you can store and retrieve data seamlessly. However it is limited: it does not support SPARQL, and has limited performance. So if you need to scale up, consider a web service to give you a hand. Check out triple stores, RDF and OWL (these are W3C standards)”.
4. In some studies (such as the prisoner's dilemma) even humans couldn't detect social cues from other humans' behavior / intent and most of the time they tried to solve it by intuition. How could that apply to a robot?
Victor: “To me, intuition is a form of "analogical" reasoning as well based on observations and experience as logics. Algorithms like neural networks, Q-Learning, etc, are doing exactly that. I do not mean they are as performant as human's intuition, but intuition-like mechanisms are used all around in AI. Also, it seems that in your example you suppose there are social cues that are observable, but humans can't detect them. As of today, image recognition algorithms have outsmarted human image recognition. We're not there yet for social cues, but there are good reasons to believe that robots can outsmart human behavior recognition in the future”.
Chloe: “You're right. The information to predict is indeed an intricate phenomenon that even humans have difficulties to analyze and the way the current research is addressing this issue follows a different paradigm. To map the collected multimodal signals to the emotions, the computational models are learning from data that was annotated by humans that are often external observers of the interaction. The emotion annotation of such data, for which it is difficult to obtain a consensus even when it is carried out by several external observers of the interaction, is more delicate than the annotation of cats in images. The current research community of affective computing relies on research in both artificial intelligence and the humanities and social sciences based on psychological models of emotions or cognitive models for human-human communication, in order to strengthen the reliability of the used annotations.
5. Pepper came out about 5 years ago. Can you please give us a short outlook of what we can expect from SoftBank in the next months and years?
Victor: “I do not think I am allowed to disclose much. But I think it's safe to assume that we are working on more powerful versions of our robots. In terms of software, I have seen visible improvements in the next NAOqi version. Also, my work on teaching behaviors is being applied little by little on Pepper…”
6. What is, in your opinion, the reason we still have not seen a rise or real success in social robotics? Is the research still not ready or is the market not ready? What can be done to reach this rise?
Victor: “I think we took AI for granted, and simply got carried over when we realized robots were real. Businesses have underestimated the time scales required to develop the AI for social interaction for robots. Nonetheless, there are social robots on the shelves of shops for Christmas. Also, smart home assistants are definitely on the rise, and their respective assistants get more and more social skills. AIs for social interaction are actually under development, but the rise will remain slow until we have at least some products that "get it”.
7. How will social robots with these enhanced social skills become affordable for people?
Victor: “With the help of optimization, miniaturization, and scaling up. ARM produces processors that are more and more optimized for machine learning. It enables more intelligent small robots. Since Aldebaran / now SBRE was born, the price of small electrical motors went down as the industry ramped up. Then there is the software. The answer really depends on what you expect from your social robot. But top research is usually done on powerful computers, so it takes many years before being embedded in small devices.”
8. Do you think that industry and the market are ready for social robots?
Victor: “Yes, but that depends on which market you are talking about, and for which function. NAO and Pepper have been proven profitable for several markets, and we're working on opening new ones”.
9. Is there a focus on further developing speech recognition in these social robots, e.g.for people with ALS and who would love to be accompanied by a robot?
Victor: “We're not directly working on that, but our speech recognition providers improve all the time. I don't know about the results of speech recognition in the case of people who have ALS.”
Chloe: “It is not my direct speciality either but what I can see from the last conferences on speech processing is that the speech recognition community is still working on how to improve speech recognition systems for low-resource languages, noisy environments, and speech-language pathologies”.
10. In the past 5 years, what has been the biggest milestone or improvement in social robotics SBRE has seen? How did you use it to improve your robots?
Victor: “I can only think of emotion detection, but I think it has been more than 5 years. The other improvements did not land in the product yet. Besides social robotics, and more recently, I can think of the visual SLAM navigation that has finally become a reality in Pepper”.
11. For a school project we are designing a social robot. The purpose of this robot is to increase the involvement of sick students in education. Based on your expertise, can you name aspects that are extra relevant in this design process? In addition, how can we best convey emotions through a social robot?
Chloe: “This is an interesting question and I have no direct answers from my side but I can just share what we have done in a similar context. Within the ANIMATAS european project, Softbank and Telecom-Paris are developing robots with social skills for schools, to assist teaching staff.At Telecom-Paris, with my PhD student Tanvi Dinkar, we chose to focus on the children side and to study how a robot can detect a level of user's confidence based on speech analysis. The long-term objective is that, with such information, the robot could be able to identify when the child is experiencing difficulties (Dinkar, T., Vasilescu, I., Pelachaud, C. and Clavel, C., 2020, May. How confident are you? Exploring the role of fillers in the automatic prediction of a speaker’s confidence. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8104-8108). IEEE.)”.
Ioana: “Maybe the first question to ask is why you need to convey emotion through a social robot in this case. Also, what is the problem with the lack of involvement of sick students and why do you think social robots are the solution? Any project, even a robotic one, should start from a need from real life. Now if you want to work on social robotics and learning at a distance (for sick and other students - COVID situation is a perfect time to experiment) you could ask yourself: what can you do with robots at home to enrich the distance learning experience? Is the robot a student like you, a classmate, is it a teacher...? Interesting scenarios could be designed!”
12. Design question: Should social robots look/act like humans?
Ioana: “As designers we always need a context in order to answer a question. So the answer is it depends :)”
13. What would be your advice for those working on emotion recognition of a user?
Chloe: “The recognition of an emotion by a human is a subjective and intricate process. In order to help human labelers, we provide annotation schemes  that are built from linguistic and psychological literature - using meaningful emotional categories or dimensions such as arousal, valence and dominance. The key to having efficient user's emotion recognition systems is to rely on reliable human annotations to train the emotion computational models.”
Victor: “Think about scaffolding the interaction thoroughly, so that you can tell what the emotion is about. For instance, try to induce an emotionally neutral state before asking a question which may induce an emotional response”.
14. Secluded elderly that don't talk about themselves to their friends and family, all of a sudden open up to a social robot. Are there other examples you came across of this behavior that seems useful and unique to these kinds of robots?
Victor: “Kids do that too ! You may end up saying things to a robot that you are too shy to tell to people. A bit like it happens with plushies or toys. Real humans are scary, in fact... For people with ASD, it is so strong that NAO is used regularly in specialized schools. The robot has a predictable behavior, and it is not loaded with that judgemental stance”.
15. Are social robotic strategies being used in industrial settings? For instance: do cleaning robots in hospitals, or cobots in factories, talk to people more sensibly?
Victor: “Not that I know of, apart from research robots like Care-o-bot, or Baxter. But I read from Dyson that they are getting interested in adding social interaction to their vacuum cleaners!”
16. We are working on neurorehabilitation applications with a humanoid robot (ebrain-science.de). In 2022 we will organize an international symposium on the topic of social robots. Do you have some ideas for the symposium?
Victor: “A humanoid robot can serve as an avatar for the disabled patient. If hopefully their brain area for motion is still working and its activity can be tracked, then they might be able to use it to move a robot. Could seeing the robot do the movement as they do it might stimulate their pathways to actual motion, and help reconstructing them? If brain-reading technology was portable, a disabled patient could actually use that avatar in their everyday life?Check the work of Veronique Aubergé on socio-affective ”glue” cues in elderly-robot interactions in a Smart Home with the EmOz platform ”
17. How may the visually impaired people benefit from social robots? Do you believe that the social robots will help in creating more equity, diversity and inclusion in our society?
Victor: “The first thing that comes to my mind is their vocal interface. Computers are rarely designed for visually impaired people, and this is a shame. Vocal interfaces should be affordable. But this is not robotics. Robots could bring more because they are embodied, they share the physical space with the human. We can make robots talk about the environment, and guide the human who might get lost, or lose objects. But I'm not sure I know any robot that would really be able to do that today. It is feasible, but still not affordable.”
18. If Pepper asked something personal to a human, do you think that the human will be more honest to Pepper than to another human? Why?
Victor: “Yes! Because the truth is not loaded for a robot. The robot does not judge you. Right now it actually does not get it... Anyway, a robot is a tool, so if you want its help, you have to tell it the truth. Note how objective truth can be relied on for constructive matters (whereas lies or fake news have a destructive power). If the robot works for you and just for you, you will tell the truth just because it's more powerful for you. Whereas if the robot works for someone with competing interests, dishonesty becomes convenient again.It depends how it asks, when it asks, why it asks, ... The fact that you know that the robot will not judge you is indeed something that could ease this type of discussion, but one will feel "safe" enough when one knows where the data will be stored and who will have access to it... Maybe like looking for info on the net, talking with an intelligent chat bot, ... ? “
19. What can we learn from virtual agents for social agents?
Victor: “Everything from decision making, dialogues and emotional stances can virtually be transposed to social robots, who are virtual agents too.”
20. Can you define what is a "social behavior"?
Victor: “A behavior involving social constructs such as language, relationship or emotion. Looking at a face is not as is a social behavior. Doing it to read the emotion, to express our attention, or to direct our speech to it, becomes a social behavior.”
 Langlet, C., Duplessis, G.D. and Clavel, C., 2017, August. A Web-Based Platform for Annotating Sentiment-Related Phenomena in Human-Agent Conversations. In International Conference on Intelligent Virtual Agents (pp. 239-242)
 Véronique Aubergé, Yuko Sasa, Nicolas Bonnefond, Brigitte Meillon, Tim Robert, et al.. The EEE corpus: socio-affective ”glue” cues in elderly-robot interactions in a Smart Home with the EmOz platform. 5th International Workshop on EMOTION, SOCIAL SIGNALS, SENTIMENT & LINKED OPEN DATA, May 2014, Reykjavik, Iceland. pp.LREC. ffhal-01003910. https://hal.archives-ouvertes.fr/file/index/docid/1003910/filename/ES3LOD_Auberge_al.pdf