Skip to Main Content

Telemental Health: Outcomes

At this time, there is only one published framework adapted for psychiatric- mental health nursing telehealth practice by Finley (2022) which adapts and combines Media Richness Theory (MRT) and Interpersonal Relations Theory (IRT); meanwhile, there are no original theories specific to psychiatric mental health nursing telehealth practice (Nagel & Penner, 2016).

Theoretical Frameworks & Existing Models

Expand All

Examples of Adapted Theoretical Frameworks in Nursing Telehealth

Examples of Adapted Theoretical Frameworks in Nursing Telehealth

  • Benner’s Novice to Expert Theory for advanced practice registered nurse (APRN) education telehealth competency development (Dzioba, et al., 2022)
  • King’s Conceptual Framework and Theory of Goal Attainment in nursing telehealth education and practice (Fronczek, 2022; Fronczek, 2019; Fronczek & Rouhana, 2018)
  • Normalization Process utilized for telehealth treatment for pregnant women with opioid use disorder (Abebe, Goldsby, & Renbarger, 2020; Wagner, Melin, & Wood, 2021)
  • Adapting existing theoretical frameworks for nursing telehealth (Fronczek, 2019)
  • Systematic review of theories and framework adaption in telehealth nursing (Nagel & Penner, 2016)
  • Telehealth competencies for nursing education and practice using ‘The Four P’s of Telehealth’ (Rutledge, et al., 2021)

Example of Existing Telehealth Models (not specific to nursing or psychiatric practice)

Unified Theory of Acceptance and Use of Technology (UTAUT)
UTAUT is a model that assesses the interactions between variables that influence technology acceptance, usefulness, ease of use, and behavioral intent
(Venkatesh, et al., 2003)

 

References

Abebe, S. Y., Goldsby, E. A., & Renbarger, K. M. (2020). Telehealth for pregnant women with opioid use disorder: A theory-based approach. Journal of Psychosocial Nursing and Mental Health Services, 58(12), 13-20.

Dzioba, C., LaManna, J., Perry, C. K., Toerber-Clark, J., Boehning, A., O’Rourke, J., & Rutledge, C. (2022). Telehealth competencies: Leveled for continuous advanced practice nurse development. Nurse Educator, 10-1097.

Finley, B. A. (2022). Virtual healing: Exploring how psychiatric mental health nurse practitioners experience therapeutic alliance while using telepsychiatry. (Publication No. 29326289). [Doctoral dissertation, The University of Arizona]. The University of Arizona ProQuest Dissertations Publishing.

Fronczek, A. E. (2022). Ushering in a new era for King’s Conceptual System and Theory of Goal Attainment. Nursing Science Quarterly, 35(1), 89-91.

Fronczek, A. E. (2019). Nursing theory in virtual care. Nursing Science Quarterly, 32(1), 35-38.

Fronczek, A. E., & Rouhana, N. A. (2018). Attaining mutual goals in telehealth encounters: Utilizing King’s framework for telenursing practice. Nursing Science Quarterly, 31(3), 233-236.

Nagel, D. A., & Penner, J. L. (2016). Conceptualizing telehealth in nursing practice: advancing a conceptual model to fill a virtual gap. Journal of Holistic Nursing, 34(1), 91-104.

Rutledge, C. M., O’Rourke, J., Mason, A. M., Chike-Harris, K., Behnke, L., Melhado, L., … & Gustin, T. (2021). Telehealth competencies for nursing education and practice: The four P’s of telehealth. Nurse Educator, 46(5), 300.

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 425-478.

Wagner, N. R., Melin, T. T., & Wood, F. (2021). Implementation of telehealth: Normalization Process Theory and NoMAD-Ex. The Journal for Nurse Practitioners, 17(9), 1141-1143.

Further Readings

van Dyk, L. (2014). A review of telehealth service implementation frameworks. International Journal of Environmental Research and Public Health, 11(2), 1279-1298.

Wade, V., Gray, L., & Carati, C. (2017). Theoretical frameworks in telemedicine research. Journal of Telemedicine and Telecare, 23(1), 181-187.

Psychometrically Validated Telehealth Survey Instruments

Expand All

Instruments, Focus, & Application

Psychometrically Validated Telehealth Survey Instruments

Focus & Application
Patient Assessment of Communication During Telemedicine (PACT) Patient-focused measure adapted from an existing, validated patient satisfaction questionnaire focusing on patient communication, provider competence, interpersonal skills, and service convenience (Agha, et al., 2009)
Patient Experience Questionnaire (PEQ) Patient self-reported measure that assesses experiential variables of communication, emotions, short-term outcomes, staff relationships, and use barriers (Steine, Finest, & Laerum, 2001)
System Usability Scale (SUS) One of the oldest and widely used scales that focuses on technological usability, quality, ease of use, use complexity, and user confidence for both provider and patient (Brooke, 1996)
Technology Acceptance Model (TAM) One of the most widely used measures, it focuses on usability, attitude, and behavioral intent when using telehealth for both provider and patient (Holden & Karsh, 2010)
Telehealth Satisfaction Scale (TeSS) Assesses the quality, length of time to access, usability, privacy, comfort and attitude to telehealth (Morgan, et al., 2014)
Telenursing Questionnaire Recently validated Likert questionnaire that takes an estimated 7 minutes to complete focusing on nurses’ acceptance, usefulness, and appropriateness of care (Marco-Franco, et al., 2022)
Telenursing Interaction and Satisfaction Questionnaire (TISQ) One of the only nurse-focused measures and qualitative surveys- this focuses on interpersonal interaction, affect, decision-making, user competence and satisfaction experienced (Mattisson, et al., 2019)
Telemedicine Perception Questionnaire (TMPQ) Measure focusing on communication, privacy/confidentiality, time & cost savings, equipment trust, and accessibility (Demiris, Speedie, & Finkelstein, 2000)
Telemedicine Satisfaction and Usefulness Questionnaire (TSUQ) Measuring perceived usefulness, effective, usability, and behavioral intent when using telehealth in comparison to in-person care and is also validated in Spanish (Bakken, et al., 2006)
Telemedicine Satisfaction Questionnaire (TSQ) One of the first widely-used scales for telehealth, the TSQ focuses on technological quality, satisfaction, time, accessibility, and convenience while integrating user experience variables such as interpersonal interactions and communication (Yip, et al., 2003)
Telehealth Usability Questionnaire (TUQ) Commonly used and measures usefulness, learnability, interface quality, reliability, satisfaction, effectiveness, and ease of use (Parmanto, et al., 2016)
Unified Theory of Acceptance and Use of Technology (UTAUT) measure applied to telehealth UTAUT has a complimentary questionnaire measured with a Likert scale regarding technology usefulness, ease of use, and behavioral intent- this ‘unified’ measure combines aspects of the SUS, TAM, and TISQ questionnaires (van Dyk, 2014)
Other Variables/Outcomes Beyond telehealth-specific measures, other outcomes can be measured, including therapeutic alliance, no shows, assessment accuracy, symptom management/outcomes, medication adherence, treatment completion, care quality, number of services provided/relative value units, cultural competency, wait times, session length, distance to service (miles from patient and provider), mental health stigma, value and cost, resource utilization, access to care, and patient safety (Shore, et al., 2014).

References

Agha, Z., Schapira, R. M., Laud, P. W., McNutt, G., & Roter, D. L. (2009). Patient satisfaction with physician–patient communication during telemedicine. Telemedicine and e-Health, 15(9), 830-839.

Bakken, S., Grullon-Figueroa, L., Izquierdo, R., Lee, N. J., Morin, P., Palmas, W., … & Starren, J. (2006). Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. Journal of the American Medical Informatics Association, 13(6), 660-667.

Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7.

Demiris G, Speedie S, Finkelstein S. A questionnaire for the assessment of patients’ impressions of the risks and benefits of home telecare. J Telemed Telecare, 2000; 6:278–284.

Holden R.J., Karsh B.T. The technology acceptance model: its past and its future in health care. J Biomed Inform. 2010;43:159–172.

Marco-Franco, J. E., Reis-Santos, M., Barrachina-Martínez, I., González-de-Julián, S., & Camaño-Puig, R. (2022). Validation of a New Telenursing Questionnaire: Testing the Test. Mathematics, 10(14), 2463.

Mattisson M., Johnson C., Borjeson S., Arestedt K., Lindberg M. Development and content validation of the Telenursing Interaction and Satisfaction Questionnaire (TISQ). Health Expect. 2019;22:1213–1222.

Morgan D.G., Kosteniuk J., Stewart N., O’Connell M.E., Karunanayake C., Beever R. The telehealth satisfaction scale: reliability, validity, and satisfaction with telehealth in a rural memory clinic population. Telemed J E Health. 2014;20:997–1003.

Parmanto B., Lewis A.N., Jr., Graham K.M., Bertolet M.H. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil. 2016;8:3–10.

Shore, J. H., Mishkind, M. C., Bernard, J., Doarn, C. R., Bell, I., Jr., Bhatla, R., . . . & Eppolito, A. (2014). A lexicon of assessment and outcome measures for telemental health. Telemedicine and e-Health, 20(3), 282-292.

Steine, S., Finset, A., & Laerum, E. (2001). A new, brief questionnaire (PEQ) developed in primary health care for measuring patients’ experience of interaction, emotion and consultation outcome. Family Practice, 18(4), 410-418.

Yip M.P., Chang A.M., Chan J., MacKenzie A.E. Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: A preliminary study. J Telemed Telecare. 2003;9:46–50.

Further Readings

Hajesmaeel-Gohari, S., & Bahaadinbeigy, K. (2021). The most used questionnaires for evaluating telemedicine services. BMC Medical Informatics and Decision Making, 21(1), 1-11.

Weaver, M. S., Lukowski, J., Wichman, B., Navaneethan, H., Fisher, A. L., & Neumann, M. L. (2021). Human connection and technology connectivity: A systematic review of available telehealth survey instruments. Journal of Pain and Symptom Management, 61(5), 1042-1051.

Checklists

Expand All

Checklists, Background, & Application

Checklists

Background & Application
Ten-Question Self-Assessment for Psychiatric Mental TeleMental Health Providers This self-assessment for providers was created by APNA Board of Directors (Adams et al., 2018) and published in JAPNA. This 10 question self-assessment can be utilized to ensure that a provider is ready to provide TMH services within guidelines and best practices. This also has a complimentary patient readiness questionnaire.
Nursing Telehealth Competency A Delphi study with nursing experts resulting in 52 nursing competencies for best-practice telehealth provision (van Houwelingen, et al., 2016)
Teaching Interpersonal Skills in Telehealth Checklist (TIPS-TC) 12-item checklist for assessing observable communication behaviors that foster interpersonal skills in telehealth (Henry, et al., 2022)

References

Adams, S. M., Rice, M. J., Jones, S. L., Herzog, E., Mackenzie, L. J., & Oleck, L. G. (2018). Telemental health: Standards, reimbursement, and interstate practice. Journal of the American Psychiatric Nurses Association, 24(4), 295-305.

Henry, B. W., Billingsly, D., Block, D. E., & Ehrmann, J. (2022). Development of the teaching interpersonal skills for telehealth checklist. Evaluation & the Health Professions, 45(3), 260-269.

van Houwelingen, C. T., Moerman, A. H., Ettema, R. G., Kort, H. S., & Ten Cate, O. (2016). Competencies required for nursing telehealth activities: A Delphi-study. Nurse Education Today, 39, 50-62

 

APNA Telemental Health Toolkit Menu

National Policy Considerations

  • State Licensure & Collaborative Agreements
  • Reimbursement
  • Prescribing
  • Case Management
  • Therapy

View

Equity, Inclusion, & Access

  • Rural Populations
  • Older Adults
  • Children & Adolescents
  • People with Disabilities and/or LEP
  • LGBTQIA+
  • Substance Use

View

Outcome Measures

  • Theoretical Frameworks & Existing Models
  • Psychometrically Validated Telehealth Survey Instruments
  • Checklists

View

Logistical Concerns

  • Planning
  • Platform
  • Building a Program
  • Safety/Crisis Information
  • Technology Issues
  • Taking Care of Yourself

View

Education

  • Precepting/Tele-precepting
  • Simulation
  • Frameworks/Competencies

View

Your dynamic community of Psychiatric-Mental Health Nurses.

Connect with fellow psychiatric-mental health nurses, be the first to know about evidence-based resources and programs, and participate in exclusive opportunities to advance the care and treatment of those with mental health & substance use needs.

JOIN TODAY MEMBER COMMUNITY

Footer Cta