Development and psychometric testing of the Professional Interpersonal Competency Assessment Scale for Novice nurses (PICASN)

Literature Review

In a complex healthcare system, effective teamwork is essential for patient safety by minimizing adverse events caused by miscommunication with other healthcare providers and misunderstanding of roles and responsibilities (Babiker et al., 2014). Thus, novice nurses require soft skills and competency. According to literature, soft skills are the personal attributes and social intelligence that enable someone to interact effectively and harmoniously with others. Soft skills for the healthcare field can be learned and developed over time through practice (Taylor, 2022). The areas covered include empathy, communication skills, teamwork, work ethic, stress management, positive attitude, flexibility, time management, professionalism, and self-development (Oduenyi, 2022). The concept of social competency involves an individual’s ability to interact effectively and appropriately with others. In Japan, people are expected to know without explicit communication what the other person wants to express or desires (Matsudaira et al., 2008). The difference between perceptual and behavioral aspects is critical. An imbalance in an individual’s perceptual abilities can lead to difficulties at the social level, that is, at the interpersonal level (Bolis & Schilbach, 2018).

In 1992, Roach identified the five Cs of caring (compassion, competency, commitment, confidence, and conscience) and added a sixth C (comportment) in a second edition in 2002 (Baillie, 2015; Roach, 2002). Nurses are expected to demonstrate the six Cs of nursing in “everything we do” (Duffy, 2015). Patient-centered care is defined as “care that respects and responds to individual patient preferences, needs, and values, and ensures that patient’s values guide all clinical decisions” (Newell & Jordan, 2015). Institute of Medicine et al. (2003) propose a set of core competencies for health professionals: providing patient-centered care-identifying, respecting, and responding to patients’ differences, values, preferences, and expressed needs; alleviating pain and suffering; coordinating continuity of care; listening to, clearly informing, communicating with, and educating patients; sharing in decision making and management; and continually advocating for disease prevention, wellness, and promotion of healthy lifestyles, including a focus on population health (Institute of Medicine et al., 2003).

It is noteworthy that establishing a therapeutic patient-nurse relationship anchors on a positive bedside manner, including effective communication, empathy, respect, and professionalism, while practicing attending skills and offering honest, encouraging, and integrity-driven care (Davidhizar & Shearer, 1998). Each person grows in their ability to express caring throughout their life. Each nurse grows in their competency to express themselves as a caring person (Boykin & Schoenhofer, 2001). The recent version of the Technological Competency as Caring in Nursing theory (Locsin & Purnell, 2015) explains that in the process of knowing, persons are appreciated as participants rather than objects of care.

Step 1. Concept Identification

To comprehend the concept under study, the researchers conducted a previous study exploring the necessary professional interpersonal skills required by novice nurses (Sato & Yasuhara, 2023). Two Professional Interpersonal Competency for Novice Nurses were identified: 1) Soft skills (empathy, communication skills, teamwork, work ethic, stress management, positive attitude, flexibility, time management, professionalism, and self-development), and 2) Interpersonal competency (person-centered care, compassion, competency, commitment, confidence, and conscience, behavior, and knowing the patient).

Semi-structured interviews were also conducted among nurses with 3-5 years of experience in two acute care hospitals. Nurses in their 3^rd to 5^th year of practice have both the experience of being a novice and a senior nurse. Eleven nurses (one male and ten females) were invited to participate in this research, and consent was obtained from all of them. The mean age of the nurses was 25.0 years. The interviews ranged from 22 to 39 minutes per person and were conducted using an interview guide. Participants were asked a) about their own experiences as new nurses, including difficulties in communication, and b) what skills are necessary for new nurses to establish good relationships during training. Additionally, a literature review was conducted using published resources to search for possible items in the scale. Overall, the in-depth interviews and literature search generated a total of 42 items. The process of developing and using the PICASN is illustrated in Figure 1.

Step 2. Item Construction

For this step, a panel of experts was assembled, comprising five clinical nurses with 3-10 years of experience, one professor, and four faculty members from nursing universities in Japan. They were tasked with determining the item format and scoring schema of the instrument. The consensus was that a 5-point Likert-type scale should be used to describe scaling responses. Participants responded by selecting one of five options: 1) Not Important, 2) Slightly Important, 3) Moderately Important, 4) Important, and 5) Very Important. Among the ten experts, this step had an 80% agreement rate.

Step 3. Validity Measure

The validity of the instrument was assessed through face, content, and construct validity. For face and content validity, the panel of experts (same experts as Step 2) reviewed the 42 draft items of PICASN. Linguistic item modifications were made to fit the cultural context. In terms of content validity, each item is rated as 1 (not relevant) or 2 (highly relevant). The item-content validity index (I-CVI) and the scale-content validity index (S-CVI) were then calculated, and items with a content validity index of 0.8 or higher were used and considered acceptable (Polit & Beck, 2010). Davis (1992) also recommends a CVI value higher than 0.70.

For construct validity, exploratory factor analysis (EFA) using maximum likelihood estimation with Promax rotation was used to assess the factor structure of the perceptual dimension of the PICASN. The CVI and the item-rest (I-R) correlation were examined in parallel with the EFA to determine the number of factors.

The participants in this study included novice nurses with 1-2 years of experience from 19 Japan Council for Quality Health Care accredited hospitals in the Chugoku and Shikoku regions of Japan.

The survey was conducted from February to April 2023 using Survey Monkey for data collection, especially for psychometric testing with EFA. Letters of invitation to participate were sent to the directors of nursing, who agreed to distribute the document containing the URL to facilitate survey participants’ access to the survey instrument. The invitation letter included the study’s purpose, the research collaboration process, ethical considerations, and a URL and QR code to access the survey website.

The questionnaire collected data on the respondents’ age, gender, work unit/department, and hospital function. Three questions regarding perceptions of the work environment were rated on a 4-point Likert scale: 1) strongly disagree, 2) disagree, 3) agree, and 4) strongly agree. The generated PICASN (42 items) were rated on a 5-point Likert scale: 1) strongly disagree, 2) disagree, 3) neither agree nor disagree, 4) agree, and 5) strongly agree.

The collected data were analyzed using Jamovi (version 2.3.21) (https://www.jamovi.org/) to assess the psychometric properties of the instrument with EFA using maximum likelihood with Promax rotation. Sampling adequacy was assessed using the Kaiser-Meyer-Olkin (KMO) test, with values >0.6 considered good (Li & Lopez, 2007). Bartlett’s sphericity test was used to examine the null hypothesis that the variables are not correlated in the population (p <0.05) (Tabachnick & Fidell, 2007).

In this study, Bartlett’s test of sphericity was highly significant (p <0.001), indicating that the dataset was suitable for the reduction technique. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was 0.93, demonstrating that factor analysis was appropriate for these data.

As a rule of thumb to assess the significance of factor loadings, factor loadings of 0.3 to 0.4 are minimally accepted (Hair et al., 2010). Factor loadings below 0.30 are typically ignored; a better cut-point would be 0.40 (Brown, 2009). The number of factors with eigenvalues greater than 1.0 were considered, and factor loading coefficients of 0.40 and greater were selected as the criterion for retention. The total score and the mean factor score (MFS) of the PICASN were calculated. The significance level was set at p <0.05.

Step 4. Reliability Measure

The reliability of the instrument was assessed using Cronbach’s alpha coefficient. I-R correlation values were calculated when items with a value of 0.6 or higher were employed. The ideal Cronbach’s alpha coefficient value is at least 0.70 (Polit & Beck, 2010).

Ethical Considerations

The ethical approval for this international study was obtained from the Ethics Committee of Tokushima University Hospital, Japan, (approval number 3610-3). The consent form was provided through the link before the survey questions and was only accessible if participants agreed to participate.

Demographic Characteristics of the Participants

Survey questionnaires were distributed to 797 novice nurses. Complete data from 203 participants were included in the analysis, resulting in a final response rate of 25.5%. Table 1 displays the demographic characteristics of the study participants. There were 20 (9.9%) males and 183 (90.1%) females. The majority were 23-24 years (53.7%), with a mean age of 24.1 ± 4.59 years. Eighty-four (41.4%) participants worked at public hospitals, and 75 (36.9%) were at private hospitals. The areas of assignment included mostly the Department of Surgery with 79 (38.9%) and the Department of Internal Medicine with 60 (29.6%). Regarding the questions related to the work environment and turnover, most had been scolded by a senior nurse. Moreover, the majority of the participants felt that the atmosphere in their department was good and expressed a desire to continue working in their current hospitals.

Validity Measure Results

It is noted that the examination of EFA, CVI, and item-rest (I-R) correlation was conducted concurrently with the EFA to determine the number of factors. The initial I-CVI and S-CVI of the ten nursing experts were 0.84 and 0.69, respectively. The mean value and standard deviation of the 42-item PICASN can be seen in Table 2. However, upon excluding ten items (Q7, Q10, Q19, Q20, Q26, Q33, Q35, Q37, Q38, and Q39), the I-CVI and S-CVI improved to 0.94 and 0.88, respectively. The overall S-CVI reached an acceptable level of 0.88. Furthermore, based on an I-R correlation threshold of less than 0.6, five additional items (Q1, Q2, Q16, Q40, and Q42) were removed. Items with an I-R correlation of 0.6 or higher indicated high consistency and congruence between individual item measurements and the overall scale. Consequently, the final analysis retained 27 items with two extracted factors, which explained 80% of the variance.

Reliability Measure Results

The reliability measures for the two-factor structure of the PICASN (perception dimension, 27 items) demonstrated strong internal consistency. Specifically, the Basic Competencies as a Novice nurse (Factor 1), consisting of 15 items, had an excellent reliability of 0.94. Similarly, the Relationship Building Skills within the Healthcare Team (Factor 2), comprising 12 items, demonstrated a high internal consistency reliability of 0.91. The overall scale reliability was exceptionally high at 0.95, as indicated in Table 3.

Implications for Nursing Education and Practice

The PICASN, designed for assessing and enhancing the interpersonal skills of novice nurses, plays a pivotal role in nursing education and practice. In educational settings, PICASN allows for establishing baseline assessments, enabling instructors to evaluate students’ initial competency levels and monitor progress over time. This dynamic feedback loop facilitates curriculum adjustments, ensuring that teaching strategies align with the evolving needs of novice nurses. The tool empowers novice nurses by fostering self-awareness and offering targeted insights into areas requiring improvement. In practice, PICASN provides experienced nurses and nurse managers valuable insights into team dynamics, guiding interventions for continuous quality improvement. The tool serves as a common language for discussing and addressing interpersonal skills, fostering enhanced communication and collaboration among healthcare teams. Ultimately, PICASN contributes to a positive work environment, aligning with best practices in nursing education and supporting the delivery of high-quality, patient-centered care.

Limitations of the Study and Recommendations

The study focused on developing and validating the PICASN within the Japanese cultural context, potentially limiting its generalizability to diverse cultural settings. Future research should investigate the cross-cultural applicability of the tool to ensure its relevance and effectiveness in assessing the interpersonal skills and competencies of novice nurses across various healthcare environments. Increasing the number of subjects through additional surveys and conducting confirmatory factor analysis will be necessary. The incongruence between perception and behavior should also be clarified. Conducting longitudinal studies to assess the predictive validity of the PICASN would offer valuable insights into its effectiveness in tracking the development of novice nurses’ interpersonal skills over time. Long-term observations can contribute to refining the tool and providing evidence of its utility in educational and practice settings.