Telemental Health: Outcomes

At this time, there is only one published framework adapted for psychiatric- mental health nursing telehealth practice by Finley (2022) which adapts and combines Media Richness Theory (MRT) and Interpersonal Relations Theory (IRT); meanwhile, there are no original theories specific to psychiatric mental health nursing telehealth practice (Nagel & Penner, 2016).

Theoretical Frameworks & Existing Models

Expand All

Examples of Adapted Theoretical Frameworks in Nursing Telehealth

Benner’s Novice to Expert Theory for advanced practice registered nurse (APRN) education telehealth competency development (Dzioba, et al., 2022)
King’s Conceptual Framework and Theory of Goal Attainment in nursing telehealth education and practice (Fronczek, 2022; Fronczek, 2019; Fronczek & Rouhana, 2018)
Normalization Process utilized for telehealth treatment for pregnant women with opioid use disorder (Abebe, Goldsby, & Renbarger, 2020; Wagner, Melin, & Wood, 2021)
Adapting existing theoretical frameworks for nursing telehealth (Fronczek, 2019)
Systematic review of theories and framework adaption in telehealth nursing (Nagel & Penner, 2016)
Telehealth competencies for nursing education and practice using ‘The Four P’s of Telehealth’ (Rutledge, et al., 2021)

Example of Existing Telehealth Models (not specific to nursing or psychiatric practice)

Unified Theory of Acceptance and Use of Technology (UTAUT)
UTAUT is a model that assesses the interactions between variables that influence technology acceptance, usefulness, ease of use, and behavioral intent
(Venkatesh, et al., 2003)

References

Abebe, S. Y., Goldsby, E. A., & Renbarger, K. M. (2020). Telehealth for pregnant women with opioid use disorder: A theory-based approach. Journal of Psychosocial Nursing and Mental Health Services, 58(12), 13-20.

Dzioba, C., LaManna, J., Perry, C. K., Toerber-Clark, J., Boehning, A., O’Rourke, J., & Rutledge, C. (2022). Telehealth competencies: Leveled for continuous advanced practice nurse development. Nurse Educator, 10-1097.

Finley, B. A. (2022). Virtual healing: Exploring how psychiatric mental health nurse practitioners experience therapeutic alliance while using telepsychiatry. (Publication No. 29326289). [Doctoral dissertation, The University of Arizona]. The University of Arizona ProQuest Dissertations Publishing.

Fronczek, A. E. (2022). Ushering in a new era for King’s Conceptual System and Theory of Goal Attainment. Nursing Science Quarterly, 35(1), 89-91.

Fronczek, A. E. (2019). Nursing theory in virtual care. Nursing Science Quarterly, 32(1), 35-38.

Fronczek, A. E., & Rouhana, N. A. (2018). Attaining mutual goals in telehealth encounters: Utilizing King’s framework for telenursing practice. Nursing Science Quarterly, 31(3), 233-236.

Nagel, D. A., & Penner, J. L. (2016). Conceptualizing telehealth in nursing practice: advancing a conceptual model to fill a virtual gap. Journal of Holistic Nursing, 34(1), 91-104.

Rutledge, C. M., O’Rourke, J., Mason, A. M., Chike-Harris, K., Behnke, L., Melhado, L., … & Gustin, T. (2021). Telehealth competencies for nursing education and practice: The four P’s of telehealth. Nurse Educator, 46(5), 300.

Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 425-478.

Wagner, N. R., Melin, T. T., & Wood, F. (2021). Implementation of telehealth: Normalization Process Theory and NoMAD-Ex. The Journal for Nurse Practitioners, 17(9), 1141-1143.

Psychometrically Validated Telehealth Survey Instruments

Expand All

Instruments, Focus, & Application

Psychometrically Validated Telehealth Survey Instruments	Focus & Application
Patient Assessment of Communication During Telemedicine (PACT)	Patient-focused measure adapted from an existing, validated patient satisfaction questionnaire focusing on patient communication, provider competence, interpersonal skills, and service convenience (Agha, et al., 2009)
Patient Experience Questionnaire (PEQ)	Patient self-reported measure that assesses experiential variables of communication, emotions, short-term outcomes, staff relationships, and use barriers (Steine, Finest, & Laerum, 2001)
System Usability Scale (SUS)	One of the oldest and widely used scales that focuses on technological usability, quality, ease of use, use complexity, and user confidence for both provider and patient (Brooke, 1996)
Technology Acceptance Model (TAM)	One of the most widely used measures, it focuses on usability, attitude, and behavioral intent when using telehealth for both provider and patient (Holden & Karsh, 2010)
Telehealth Satisfaction Scale (TeSS)	Assesses the quality, length of time to access, usability, privacy, comfort and attitude to telehealth (Morgan, et al., 2014)
Telenursing Questionnaire	Recently validated Likert questionnaire that takes an estimated 7 minutes to complete focusing on nurses’ acceptance, usefulness, and appropriateness of care (Marco-Franco, et al., 2022)
Telenursing Interaction and Satisfaction Questionnaire (TISQ)	One of the only nurse-focused measures and qualitative surveys- this focuses on interpersonal interaction, affect, decision-making, user competence and satisfaction experienced (Mattisson, et al., 2019)
Telemedicine Perception Questionnaire (TMPQ)	Measure focusing on communication, privacy/confidentiality, time & cost savings, equipment trust, and accessibility (Demiris, Speedie, & Finkelstein, 2000)
Telemedicine Satisfaction and Usefulness Questionnaire (TSUQ)	Measuring perceived usefulness, effective, usability, and behavioral intent when using telehealth in comparison to in-person care and is also validated in Spanish (Bakken, et al., 2006)
Telemedicine Satisfaction Questionnaire (TSQ)	One of the first widely-used scales for telehealth, the TSQ focuses on technological quality, satisfaction, time, accessibility, and convenience while integrating user experience variables such as interpersonal interactions and communication (Yip, et al., 2003)
Telehealth Usability Questionnaire (TUQ)	Commonly used and measures usefulness, learnability, interface quality, reliability, satisfaction, effectiveness, and ease of use (Parmanto, et al., 2016)
Unified Theory of Acceptance and Use of Technology (UTAUT) measure applied to telehealth	UTAUT has a complimentary questionnaire measured with a Likert scale regarding technology usefulness, ease of use, and behavioral intent- this ‘unified’ measure combines aspects of the SUS, TAM, and TISQ questionnaires (van Dyk, 2014)
Other Variables/Outcomes	Beyond telehealth-specific measures, other outcomes can be measured, including therapeutic alliance, no shows, assessment accuracy, symptom management/outcomes, medication adherence, treatment completion, care quality, number of services provided/relative value units, cultural competency, wait times, session length, distance to service (miles from patient and provider), mental health stigma, value and cost, resource utilization, access to care, and patient safety (Shore, et al., 2014).

References

Agha, Z., Schapira, R. M., Laud, P. W., McNutt, G., & Roter, D. L. (2009). Patient satisfaction with physician–patient communication during telemedicine. Telemedicine and e-Health, 15(9), 830-839.

Bakken, S., Grullon-Figueroa, L., Izquierdo, R., Lee, N. J., Morin, P., Palmas, W., … & Starren, J. (2006). Development, validation, and use of English and Spanish versions of the telemedicine satisfaction and usefulness questionnaire. Journal of the American Medical Informatics Association, 13(6), 660-667.

Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7.

Demiris G, Speedie S, Finkelstein S. A questionnaire for the assessment of patients’ impressions of the risks and benefits of home telecare. J Telemed Telecare, 2000; 6:278–284.

Holden R.J., Karsh B.T. The technology acceptance model: its past and its future in health care. J Biomed Inform. 2010;43:159–172.

Marco-Franco, J. E., Reis-Santos, M., Barrachina-Martínez, I., González-de-Julián, S., & Camaño-Puig, R. (2022). Validation of a New Telenursing Questionnaire: Testing the Test. Mathematics, 10(14), 2463.

Mattisson M., Johnson C., Borjeson S., Arestedt K., Lindberg M. Development and content validation of the Telenursing Interaction and Satisfaction Questionnaire (TISQ). Health Expect. 2019;22:1213–1222.

Morgan D.G., Kosteniuk J., Stewart N., O’Connell M.E., Karunanayake C., Beever R. The telehealth satisfaction scale: reliability, validity, and satisfaction with telehealth in a rural memory clinic population. Telemed J E Health. 2014;20:997–1003.

Parmanto B., Lewis A.N., Jr., Graham K.M., Bertolet M.H. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil. 2016;8:3–10.

Shore, J. H., Mishkind, M. C., Bernard, J., Doarn, C. R., Bell, I., Jr., Bhatla, R., . . . & Eppolito, A. (2014). A lexicon of assessment and outcome measures for telemental health. Telemedicine and e-Health, 20(3), 282-292.

Steine, S., Finset, A., & Laerum, E. (2001). A new, brief questionnaire (PEQ) developed in primary health care for measuring patients’ experience of interaction, emotion and consultation outcome. Family Practice, 18(4), 410-418.

Yip M.P., Chang A.M., Chan J., MacKenzie A.E. Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: A preliminary study. J Telemed Telecare. 2003;9:46–50.

Checklists

Expand All

Checklists, Background, & Application

Checklists	Background & Application
Ten-Question Self-Assessment for Psychiatric Mental TeleMental Health Providers	This self-assessment for providers was created by APNA Board of Directors (Adams et al., 2018) and published in JAPNA. This 10 question self-assessment can be utilized to ensure that a provider is ready to provide TMH services within guidelines and best practices. This also has a complimentary patient readiness questionnaire.
Nursing Telehealth Competency	A Delphi study with nursing experts resulting in 52 nursing competencies for best-practice telehealth provision (van Houwelingen, et al., 2016)
Teaching Interpersonal Skills in Telehealth Checklist (TIPS-TC)	12-item checklist for assessing observable communication behaviors that foster interpersonal skills in telehealth (Henry, et al., 2022)

References

Adams, S. M., Rice, M. J., Jones, S. L., Herzog, E., Mackenzie, L. J., & Oleck, L. G. (2018). Telemental health: Standards, reimbursement, and interstate practice. Journal of the American Psychiatric Nurses Association, 24(4), 295-305.

Henry, B. W., Billingsly, D., Block, D. E., & Ehrmann, J. (2022). Development of the teaching interpersonal skills for telehealth checklist. Evaluation & the Health Professions, 45(3), 260-269.

van Houwelingen, C. T., Moerman, A. H., Ettema, R. G., Kort, H. S., & Ten Cate, O. (2016). Competencies required for nursing telehealth activities: A Delphi-study. Nurse Education Today, 39, 50-62

What can we help you find?

About APNA

Membership

Continuing Education

Resources Curated for You

Telemental Health: Outcomes

Telemental Health Toolkit

Theoretical Frameworks & Existing Models

Examples of Adapted Theoretical Frameworks in Nursing Telehealth

Example of Existing Telehealth Models (not specific to nursing or psychiatric practice)

References

Further Readings

Psychometrically Validated Telehealth Survey Instruments

Instruments, Focus, & Application

References

Further Readings

Checklists

Checklists, Background, & Application

References

APNA Telemental Health Toolkit Menu

National Policy Considerations

Equity, Inclusion, & Access

Outcome Measures

Logistical Concerns

Education

Additional Telemental Health Resources for Psychiatric-Mental Health Nurses

APNA Telemental Health Toolkit

National Survey on Telehealth Use is 2021: Disparities between Audio and Video

Providers Clinical Support System: Telehealth for Opioid Use Disorder Toolkit

APNA Continuing Education: Telemental Health

Telemental Health Background: Changes & Trends

Telemental Health Background: Practice Considerations

Telemental Health Background: Evidence of Efficacy

Telemental Health Articles in the Journal of the American Psychiatric Nurses Association

Epstein, Becker, Green Free Telemental Health Laws App

National Organization of Nurse Practitioner Faculty Supports Telehealth in Nurse Practitioner Education

American Academy of Child and Adolescent Psychiatry Telepsychiatry Resources

American Telemedicine Association Research, Case Studies, & White Papers

American Telemedicine Association Practice Guidelines

American Psychological Association Guidelines for the Practice of Telepsychology

American Psychiatric Association Telepsychiatry Toolkit

CMS Medicare Telemedicine Health Care Provider Fact Sheet

SAMHSA Evidence-based Resource Guide: Telehealth for the Treatment of Serious Mental Illness and Substance Use Disorders

CMS General Provider Telehealth and Telemedicine ToolKit

HHS Best Practice Guide to Telehealth for Mental Health Care

How to Prepare for a Video Appointment with Your Mental Health Clinician

Your dynamic community of Psychiatric-Mental Health Nurses.