Characteristics of interactive communication between Pepper robot, patients with schizophrenia, and healthy persons


delivery of health care


How to Cite

Betriana, F., Tanioka, R., Yokotani, T., Matsumoto, K., Zhao, Y., Osaka, K., Miyagawa, M., Kai, Y., Schoenhofer, S., Locsin, R. C., & Tanioka, T. (2022). Characteristics of interactive communication between Pepper robot, patients with schizophrenia, and healthy persons. Belitung Nursing Journal, 8(2), 176–184.
Share |

Funder Registry Funding data - Crosreff


Background: Expressing enjoyment when conversing with healthcare robots is an opportunity to enhance the value of human robots with interactive capabilities. In clinical practice, it is common to find verbal dysfunctions in patients with schizophrenia. Thus, interactive communication characteristics may vary between Pepper robot, persons with schizophrenia, and healthy persons.

Objective: Two case studies aimed to describe the characteristics of interactive communications, 1) between Pepper as a healthcare robot and two patients with schizophrenia, and 2) between Pepper as a healthcare robot and two healthy persons.

Case Report: The “Intentional Observational Clinical Research Design” was used to collect data. Using audio-video technology, the conversational interactions between the four participants with the Pepper healthcare robot were recorded. Their interactions were observed, with significant events noted. After their interactions, the four participants were interviewed regarding their experience and impressions of interacting with the Pepper healthcare robot. Audio-video recordings were analyzed following the analysis and interpretation protocol, and the interview data were transcribed, analyzed, and interpreted.

Discussion: There were similarities and differences in the interactive communication characteristics between the Pepper robot and the two participants with schizophrenia and between Pepper and the two healthy participants. The similarities were experiences of human enjoyment while interacting with the Pepper robot. This enjoyment was enhanced with the expectancy of the Pepper robot as able to entertain, and possessing interactive capabilities, indicating two-way conversational abilities. However, different communicating characteristics were found between the healthy participants’ impressions of the Pepper robot and the participants with schizophrenia. Healthy participants understood Pepper to be an automaton, with responses to questions often constrained and, on many occasions, displaying inaccurate gaze.

Conclusion: Pepper robot showed capabilities for effective communication pertaining to expressing enjoyment. The accuracy and appropriateness of gaze remained a critical characteristic regardless of the situation or occasion with interactions between persons with schizophrenia, and between healthy persons. It is important to consider that in the future, for effective use of healthcare robots with multiple users, improvements in the areas of the appropriateness of gaze, response time during the conversation, and entertaining functions are critically observed.

Supporting Agencies

JSPS KAKENHI Grant Number JP17H01609


Copyright (c) 2022 Feni Betriana, Ryuichi Tanioka, Tomoya Yokotani, Kazuyuki Matsumoto, Yueren Zhao, Kyoko Osaka, Misao Miyagawa, Yoshihiro Kai, Savina Schoenhofer, Rozzano C. Locsin, Tetsuya Tanioka

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Article Metrics

Abstract views: 94 | PDF views: 46 |

Dimensions Metrics


Download data is not yet available.

PlumX Metrics

Declaration of Conflicting Interest

All authors have declared no actual or potential conflict of interest.



Authors’ Contributions

TT, RL, and SS developed the initial idea. FB, TT, and RL together drafted the initial manuscript. FB, RT, TY, MY, TT collected the data. FB, RL, TT conducted data analysis. TT, FB, RT, TY, KM, YZ, KO, MM, YK, SS, and RL reviewed, revised, and contributed additional information toward the manuscript. All authors critically reviewed and approved the final version.

Data Availability

The datasets generated during and/or analyzed during the current study are not publicly available due to ethical considerations but are available from the corresponding author on reasonable request.


Aungsuroch, Y., & Gunawan, J. (2019). Viewpoint: Nurses preparation in the era of the Fourth Industrial Revolution. Belitung Nursing Journal, 5(1), 1-2.

Bazin, N., Sarfati, Y., Lefrère, F., Passerieux, C., & Hardy-Baylé, M.-C. (2005). Scale for the evaluation of communication disorders in patients with schizophrenia: A validation study. Schizophrenia Research, 77(1), 75-84.

Betriana, F., Tanioka, T., Locsin, R., Malini, H., & Lenggogeni, D. P. (2020). Are Indonesian nurses ready for healthcare robots during the covid-19 pandemic? Belitung Nursing Journal, 6(3), 63-66.

Cañigueral, R., & Hamilton, A. F. d. C. (2019). The role of eye gaze during natural social interactions in typical and autistic people. Frontiers in Psychology, 10, 560.

Chikaraishi, T., Minato, T., & Ishiguro, H. (2008, 2008). Development of an android system integrated with sensor networks. IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France.

de la Asuncion, J., Docx, L., Sabbe, B., Morrens, M., & De Bruijn, E. R. A. (2015). Converging evidence of social avoidant behavior in schizophrenia from two approach-avoidance tasks. Journal of Psychiatric Research, 69, 135-141.

He, Q., Hong, X., Chai, X., Holappa, J., Zhao, G., Chen, X., & Pietikäinen, M. (2015). OMEG: Oulu Multi-Pose Eye Gaze Dataset. In R. R. Paulsen & K. S. Pedersen (Eds.), Image Analysis. SCIA 2015. Lecture Notes in Computer Science (Vol. 9127, pp. 418-427). Springer International Publishing.

Hooker, C., & Park, S. (2005). You must be looking at me: The nature of gaze perception in schizophrenia patients. Cognitive Neuropsychiatry, 10(5), 327-345.

Kahn, R. S., Sommer, I. E., Murray, R. M., Meyer-Lindenberg, A., Weinberger, D. R., Cannon, T. D., O'Donovan, M., Correll, C. U., Kane, J. M., van Os, J., & Insel, T. R. (2015). Schizophrenia. Nature Reviews Disease Primers, 1(1), 15067.

Kobayashi, T. (2016). A robot-based enjoyable conversation system. The Journal of the Acoustical Society of America, 140(4), 3005-3005.

Kuperberg, G. R. (2010). Language in schizophrenia part 1: An introduction. Language and Linguistics Compass, 4(8), 576-589.

Locsin, R. C. (2005). Technological competency as caring in nursing: A model for practice. Indianapolis, IN: Sigma Theta Tau International

Matsusaka, Y., Tojo, T., & Kobayashi, T. (2003). Conversation robot participating in group conversation. IEICE Transactions on Information and Systems, 86(1), 26-36.

Ogawa, K., Nishio, S., Koda, K., Balistreri, G., Watanabe, T., & Ishiguro, H. (2011). Exploring the natural reaction of young and aged person with telenoid in a real world. Journal of Advanced Computational Intelligence and Intelligent Informatics, 15(5), 592-597.

Ozeki, T., Mouri, T., Sugiura, H., Yano, Y., & Miyosawa, K. (2020). Use of communication robots to converse with people suffering from schizophrenia. ROBOMECH Journal, 7(1), 1-14.

Raffard, S., Bortolon, C., Cohen, L., Khoramshahi, M., Salesse, R. N., Billard, A., & Capdevielle, D. (2018). Does this robot have a mind? Schizophrenia patients' mind perception toward humanoid robots. Schizophrenia Research, 197, 585-586.

Sabanovic, S., Bennett, C. C., Chang, W.-L., & Huber, L. (2013, 2013). PARO robot affects diverse interaction modalities in group sensory therapy for older adults with dementia. 13th international conference on rehabilitation robotics (ICORR), Seattle, WA, USA.

Seymour, K., Rhodes, G., Stein, T., & Langdon, R. (2016). Intact unconscious processing of eye contact in schizophrenia. Schizophrenia Research: Cognition, 3, 15-19.

Sirithunge, C., Jayasekara, A. G., & Chandima, D. P. (2021). An evaluation of human conversational preferences in social human-robot interaction. Applied Bionics and Biomechanics, 3648479.

Tanioka, T., Betriana, F., Yokotani, T., Osaka, K., Locsin, R. C., King, B., & Schoenhofer, S. (2021). The experience of older persons with mental health conditions who interact with healthcare robots and nurse intermediaries: The qualitative case studies. Belitung Nursing Journal, 7(4), 346-353.

Tanioka, T., Locsin, R. C., Betriana, F., Kai, Y., Osaka, K., Baua, E., & Schoenhofer, S. (2021). Intentional Observational Clinical Research Design: Innovative design for complex clinical research using advanced technology. International Journal of Environmental Research and Public Health, 18(21), 11184.

Tanioka, T., Yokotani, T., Tanioka, R., Betriana, F., Matsumoto, K., Locsin, R., Zhao, Y., Osaka, K., Miyagawa, M., & Schoenhofer, S. (2021). Development issues of healthcare robots: Compassionate communication for older adults with dementia. International Journal of Environmental Research and Public Health, 18(9), 4538.

Ujike, S., Yasuhara, Y., Osaka, K., Sato, M., Catangui, E., Edo, S., Takigawa, E., Mifune, Y., Tanioka, T., & Mifune, K. (2019). Encounter of Pepper-CPGE for the elderly and patients with schizophrenia: an innovative strategy to improve patient's recreation, rehabilitation, and communication. The Journal of Medical Investigation, 66(1.2), 50-53.

Van Wynsberghe, A. (2013). Designing robots for care: Care centered value-sensitive design. Science and Engineering Ethics, 19(2), 407-433.

Walther, S., Stegmayer, K., Sulzbacher, J., Vanbellingen, T., Müri, R., Strik, W., & Bohlhalter, S. (2015). Nonverbal social communication and gesture control in schizophrenia. Schizophrenia Bulletin, 41(2), 338-345.

World Health Organization. (2019). Schizophrenia.