Clinician Acceptance of Order Sets for Pain Management: A Survey in Two Urban Hospitals

• Abstract
• Background and Significance
• Objectives
• Methods
• Participants and Setting
• Study Design
• Statistical Analyses
• Results
• Effort Expectancy
• Performance Expectancy
• Social Influence
• Facilitating Conditions
• Logistic Regression Model
• Discussion
• Findings
• Limitations
• Conclusion
• Clinical Relevance Statement
• Multiple Choice Questions
• References

Abstract

Background Order sets are a clinical decision support (CDS) tool in computerized provider order entry systems. Order set use has been associated with improved quality of care. Particularly related to opioids and pain management, order sets have been shown to standardize and reduce the prescription of opioids. However, clinician-level barriers often limit the uptake of this CDS modality.

Objective To identify the barriers to order sets adoption, we surveyed clinicians on their training, knowledge, and perceptions related to order sets for pain management.

Methods We distributed a cross-sectional survey between October 2020 and April 2021 to clinicians eligible to place orders at two campuses of a major academic medical center. Survey questions were adapted from the widely used framework of Unified Theory of Acceptance and Use of Technology. We hypothesize that performance expectancy (PE) and facilitating conditions (FC) are associated with order set use. Survey responses were analyzed using logistic regression.

Results The intention to use order sets for pain management was associated with PE to existing order sets, social influence (SI) by leadership and peers, and FC for electronic health record (EHR) training and function integration. Intention to use did not significantly differ by gender or clinician role. Moderate differences were observed in the perception of the effort of, and FC for, order set use across gender and roles of clinicians, particularly emergency medicine and internal medicine departments.

Conclusion This study attempts to identify barriers to the adoption of order sets for pain management and suggests future directions in designing and implementing CDS systems that can improve order sets adoption by clinicians. Study findings imply the importance of order set effectiveness, peer influence, and EHR integration in determining the acceptability of the order sets.

Background and Significance

Clinical decision support (CDS) systems present clinicians, patients, and other stakeholders (including administrative staff and other members of the care team) with relevant clinical knowledge and patient information to improve health and health care delivery.[1] Examples of CDS include computerized alerts, diagnostic support, visualized summaries, and, in recent years, predictive analytics functions.[2] [3] [4] [5] One key area of CDS is within the computerized provider order entry (CPOE) system. Adopted by over 95% of the U.S. non-federal acute hospitals,[6] CPOE plays a central role in the clinical workflow. Since many potential adverse drug events occur because of errors during order placement, an important role of CDS for CPOE is to reduce errors and support safe and efficient order placement.[7] A classic CDS for order entry in CPOE is an order set. Order sets present multiple orders for a particular clinical purpose as a set, with appropriate default settings, for clinicians to select.[8] Providing an alternative to placing standalone orders one by one, order sets often serve as an electronic health record (EHR)-embedded guideline for best practices. They are expected to result in improved safety, effectiveness, and efficiency while reducing care variations.[9] [10] [11] Thus, the creation of order sets has been widely accepted as a requirement for a successful CPOE implementation.[8] [12] [13]

Condition- or population-specific order sets have been associated with improved health care quality in the ambulatory setting,[14] surgical intensive care units,[15] chronic obstructive pulmonary disease units,[16] and emergency departments.[17] [18] For instance, the adoption of standardized order sets in sepsis and septic shock has resulted in a significant reduction in morbidity, mortality, and use of health care resources[19] and led to improved antibiotic therapy and greater fluid administration.[20] A study conducted to evaluate the impact of the implementation of a standardized order set demonstrated that there was a statistically significant reduction in the length of stay and 100-day readmissions for pediatric asthma patients.[16] Particularly related to pain management, studies conducted in the emergency departments at two separate health systems by Delgado et al and Montoy et al concluded that default settings in the EHR order sets can influence prescription choices where a lower number of opioid tablet set in the default setting led to a lower number of actually prescribed pills.[21] [22] Furthermore, a study conducted by Netley et al that evaluated the implementation of ALTO (alternative to opioids) order sets within an EHR found that they not only encouraged reduced prescription of opioids but also helped in optimizing dosing of nonopioid analgesics.[23] For better pain management, in addition to order sets, various tools, such as interactive pain management systems,[24] visual analytics for opioid documentation,[25] and pain information models,[26] have also been applied.

Despite order sets' common presence in CPOE and reported benefits, order set usage or acceptance reported by previous studies varies by clinical specialties, potentially limiting the improvement in patient care through CDS.[8] [12] [13] [27] Asaro et al reported that while CPOE order sets were preferred by physicians over the paper version, the shortcomings in the operational activities of the system coupled with cultural and organizational challenges limited the effectiveness of the CPOE implementation.[28] Previous studies have reported that performance and peer influence were the most critical factors in determining the intention to use EHR or health care applications.[29] [30] In particular, processes, training, and efficacy have been named as barriers to CPOE adoption.[31] Similarly, Vandenberg et al found that autonomy, familiarity, and learning curve were barriers in an order set designed to improve geriatric prescribing in the emergency department.[18]

We hypothesize that demonstrated benefits of order sets, and an environment that facilitate their use, may be associated with order sets' acceptability and intention to use by clinicians. To test our hypothesis, we used the Unified Theory of Acceptance and Use of Technology (UTAUT) framework.[32] UTAUT is a well-established and widely used theoretical framework for understanding technology acceptance and has been applied to explain CDS adoption.[33] Among the several theoretical models that have been used to assess the acceptability and intention to use of any new technological product, including theory of reasoned action (TRA), technology acceptance models (TAM), motivational model, theory of planned behavior (TPB), combined TAM and TPB, model of personal computer utilization, innovation diffusion theory, and social cognitive theory, UTAUT is one of the most well-established and validated frameworks in the health care domain.[32] In UTAUT, performance expectancy (PE), effort expectancy (EE), social influence (SI), and facilitating conditions (FC) are the key determining factors of behavioral intention (BI) and actual behavior. Gender, age, experience, and voluntariness/willingness to use serve as modifying factors affecting user behavior.[34] Related to PE and EE in UTAUT is the Physician Order Entry User Satisfaction and Usage Survey,[35] which was focused on an early effort of POE and found that satisfaction was correlated with its ability to improve productivity. The FC in UTAUT refers to environments and support systems that enable the technology use. The concept is also covered in the Computer Self-Efficacy Scale that assesses individual skills and knowledge related to using and learning computers.[36]

Objectives

To understand factors that affect the use of order sets by clinicians in an urban academic medical center, a cross-sectional survey based on UTAUT constructs was conducted among clinicians in two hospital campuses of an urban academic medical center, between October 2020 and April 2021. We particularly focused on the use of order sets for safe pain management. Opioid misuse is a national crisis that significantly affects public health and social welfare in the United States.[37] Opioid prescription in inpatient and emergency department settings is a common and yet less recognized gateway for opioid-naïve patients to develop dependence.[38] [39] [40] Recent studies have demonstrated the potential of EHR-based interventions in the emergency department to mitigate opioid prescriptions by using default settings to nudge clinicians to limit prescriptions.[21] In addition, previous studies have reported that many care variations exist in opioid prescribing among clinicians.[41] [42] [43] Naturally, with the CDS functions of order sets through their controlled content and default settings, the use of order sets may play a role in facilitating safety and quality care in the context of opioid prescribing. However, few studies have examined clinician perception of order sets for opioid prescriptions.[44] This study investigates what factors may affect clinicians' decision to adopt the order sets into their clinical work, thus improving care safety and quality in pain management and opioid prescription. Our objective is to assess barriers and facilitators to the use of order sets by clinicians at a large urban academic medical center. We hypothesized that PE and FC as defined in the UTAUT framework might determine clinicians' intention of use.

Methods

Participants and Setting

We recruited clinicians including physicians, physician assistants, and nurse practitioners at two campuses of an academic medical center. There were two inclusion criteria. First, the participant had to have a clinical role to use the CPOE functions in the EHR, and second, the eligible participant had to have used, or tried to use, order sets to place an order at least once within a 1-year period before the survey. In the study site, order sets are built in the Epic EHR system, which was newly implemented, transitioning from Allscripts Sunrise EHR, one month prior to the survey initiation. Participants were recruited from emergency medicine, internal medicine, surgery, and other divisions in medicine. Recruitment venues included the listservs of departmental residents and faculty/staff. Participants received a $10 gift card as an incentive.

Study Design

The survey was a cross-sectional, online survey in English and accessible via a secure website. The survey was designed to take 5 to 10 minutes to complete. Surveys were created using survey software Qualtrics (Provo, Utah, United States) and distributed using an electronic link. All survey questions are listed in [Table 1]. The questions were designed to elicit responses demonstrating clinicians' characteristics and BI toward order sets in the EHR, particularly for pain management. Clinician characteristics included department affiliation, clinical role (physician, physician assistant, nurse practitioner, and other), levels of experience (attending physician and resident), and gender. The survey also included questions designed to capture EE (perceived ease of use), PE (perceived value of using order sets in terms of easing workload and providing CDS), SI, FC (level of IT training and literacy provided by the institution[45] [46] and past experience in designing or modifying order sets), and BI (a user's intention to use new technology). We also asked about desired functions in the order set particularly related to pain management. Response options include multiple choices for characteristics questions and 7-point Likert scales for construct questions. [Fig. 1] displays the questions and the constructs they represent based on UTAUT. This study was approved by Weill Cornell Medicine Institutional Review Board, and all participants provided written informed consent.

Statistical Analyses

We computed descriptive statistics from the survey, including the count and percentage of each survey answer by constructs. For analysis, we categorized departments into three groups: emergency medicine, general internal medicine, and other groups. Participants in the other groups included surgery, obstetrics and gynecology, and other divisions in medicine. Responses with missing affiliation information were also included in the other groups. Participants' clinical roles were classified as attending physicians or non-attending clinicians that include residents, nurse practitioners, physician assistants, and other providers. Since responses to the UTAUT questions were in the 7-point Likert scales, we transformed descriptive responses to be numeric scores with the range of 1 to 7, namely setting 1 as “strongly disagree,” 2 as “disagree,” and 7 as “strongly agree.” After generating the transformed score for each question, we averaged scores for all questions in each construct for each participant and used it as the construct's overall score for this participant. For each construct, to test clinician characteristics and other construct differences between participants with high and low construct score groups, we defined those with scores higher than 4 as the high construct score group and those no higher than 4 as the low construct score group. To compare participants' characteristics and construct scores for different groups of participants, we conducted Wilcoxon rank-sum tests for continuous variables and chi-square or Fisher's exact tests for categorical variables, at an α of 0.05.

Further, we converted answers to BI questions to be a binary BI variable, which is 1 (intend to use) if participants somewhat agreed/agreed/strongly agreed to either questions 16 or 17, and 0 (not intend to use) if otherwise. We also tested the differences in clinician characteristics and the four constructs between the binary BI variables, intend to use and not intend to use groups, with Wilcoxon rank-sum tests for continuous variables and chi-square or Fisher's exact tests for categorical variables, at a significant level of 0.05. To further test our hypothesis, we used the binary BI as the dependent variable, and answered questions given in [Table 1] regarding clinicians' department, clinical roles, and UTAUT constructs as independent variables in a logistic regression model. Forward stepwise logistic regression was used to choose the final model. All analyses were conducted using R 3.6.0.

Results

The survey was distributed to two listservs of at least 630 clinicians through emails containing web links to the survey. Out of 127 surveys started, 124 (97.6%) finished the survey, and eight surveys were removed from the analysis because more than 50% of their answers were missing. Among the 116 remaining respondents, the gender distribution was 54 (46.6%) males versus 62 (53.4%) females and 37 (31.9%) attending physicians versus 79 (68.1%) non-attending clinicians. Participants from internal medicine, emergency medicine, and other departments accounted for 45.7% (N = 53), 44.0% (N = 51), and 10.3% (N = 12), respectively.

[Table 2] shows the descriptive statistics for UTAUT constructs. Most respondents agreed that they had received sufficient training for the use of the EHR at their current institution (94.8%), felt that the current EHR was easy to use (90.5%), and agreed that the order sets in the EHR enabled the easier placement of orders (88.8%). The majority were interested in having an order set for pain management (88.8%) and were interested in using an order set with CDS functions for alternative pain management to opioids (90.5%). However, only 40.5% of the respondents were aware of how to suggest changes to order sets at their current institution, which may be related to the new EHR implementation but underscored the need to communicate these processes to front-line staff. Only 15.5% answered yes when asked whether order sets exist for pain management in the institution.

[Supplementary Table S1] (available in the online version) shows Spearman's rank correlation among all UTAUT constructs. BI was positively correlated with SI construct 1 and 2, FC construct 2, and EE construct 1. [Supplementary Tables S1] to [S5] (available in the online version) present the descriptive statistics and statistical analysis by four constructs, EE, PE, SI, and FC.

Effort Expectancy

[Supplementary Table S2] (available in the online version) shows that most of the respondents' EE scores were higher than 4 (N = 97, 83.6%) or in the high EE score group. More than half (50.5%) of the respondents in the high EE score group were males, while only a little more than one-quarter (26.3%) of the respondents in the low EE score group were males; however, there were more females in the low EE score group than those in the high EE score group. Among three department categories, internal medicine respondents (63.2%) were the largest sub-group in the low EE group and emergency medicine respondents (46.4%) were the largest sub-group in the high EE group. The PE, SI, and FC scores in the low EE score group were lower than those in the high EE score group, and particularly, the difference in PE was statistically significant (p-value = 0.041).

Performance Expectancy

As shown in [Supplementary Table S3] (available in the online version), there were more (N = 93, 80.2%) respondents in the high PE score group than those in the low PE score group (N = 23, 19.8%). Both in the low PE score group and high PE score group, there were more females than males, with 52.2% females in the low PE score group and 53.8% females in the high PE score group. Among the three departments, internal medicine respondents held the highest proportion in the low PE score group (65.2%) and emergency medicine respondents held the highest proportion in the high PE group (47.3%). Similar to EE, non-attending clinicians represented a higher proportion than attending physicians in both the low PE score group and the high PE score group, 82.6 and 64.5%, respectively. For other construct scores, similar to the statistics by EE score, respondents in the low PE score group had lower EE, SI, and FC scores than those in the high PE score group. The differences in SI and FC constructs were statistically significant (p-values < 0.001).

Social Influence

As in [Supplementary Table S4] (available in the online version), there were more respondents in the high SI score group (N = 100, 86.2%) than those in the low SI score group (N = 16, 13.8%). Similar to the PE score group, there were more females than males in both the low SI score group and the high SI score group. There was a statistically significant difference in the department distribution between low and high SI score groups (p-value = 0.002) among the emergency medicine, internal medicine, and other departments, with internal medicine having the largest proportion in the low SI score group and emergency medicine the largest proportion in the high SI score group. Regarding the respondents' clinical roles, similar to PE, more non-attending clinicians were in the low and high SI score groups (87.5 and 65.0%). The high SI score group had statistically higher EE, PE, and FC construct scores than the low SI group (p-values = 0.005, < 0.001, and < 0.001, respectively).

Facilitating Conditions

As shown in [Supplementary Table S5] (available in the online version), although 60.3% of the respondents (N = 70) were in the high FC score group, this percentage was lower than that of high EE, PE, and SI score groups. There were more females in the low FC score group (60.9%) but more males in the high FC group (51.4%). Within the low FC score group, internal medicine had the highest percentage of respondents (73.9%), and within the high FC score group, emergency medicine had the highest percentage of respondents (64.3%). Statistically, the difference among department distributions between the two groups was significant (p-value < 0.001). Furthermore, non-attending clinicians had a statistically higher percentage than the attending physicians in the low FC score group than in the high FC score group (p-value < 0.001). Respondents who were in the high FC group were more likely to be in high EE, PE, and SI score groups as well, and statistically significant differences were observed in PE and SI constructs (p-values < 0.001).

Logistic Regression Model

The outcome for the logistic regression model was defined using the binary BI variable into intend to use group versus not intend to use group. We tested the differences in clinician characteristics and construct scores as shown in [Table 3]. The majority (92.2%, N = 107) of respondents were in the intend to use group, showing an interest in having an order set for pain management or having an order set with decision support for alternative pain management to opioids. In the not intend to use group, there were more males than females, 55.6 versus 44.4%, while in the intend to use group, there were more females than males, 54.2 versus 45.8%. Within either group, respondents in internal medicine took up a higher percentage (44.4 and 45.8%) than those in emergency medicine (33.3 and 44.9%) or other (22.2 and 9.3%) departments. The respondents' average EE, PE, SI, and FC construct scores were all higher in the intend to use group than those in the not intend to use group, but the only difference in SI score was statistically significant (p-value = 0.004).

The results of the logistic regression model are displayed in [Table 4]. Questions related to PE (p-value = 0.013), SI (p-value = 0.015), and FC (p-values = 0.047 and 0.034) had statistically significant impacts on the binary BI variable, controlling for departments, and clinical roles. A point increase in the PE construct 1, a question about the performance of the current order sets in EHR (Q9), increased the odds of BI by a factor of 3.32. A point increase in the SI construct 2, a question regarding peers' use of order sets (Q21), increased the odds of BI by a factor of 5.25. A point increase in the FC construct 1, a question related to EHR training (Q7), and a point increase in the FC construct 3, a question on the integration of pain management (Q14), decreased the odds of BI by a factor of 0.16 and 0.33, respectively. In terms of departments, the odds ratios of BI from emergency medicine and other departments over internal medicine were 0.08 and 0.04, respectively, relatively small, which indicated that compared with clinicians in internal medicine, clinicians in these departments were likely to have much lower intention to use order sets. Variance inflation factors between variable pairs were all less than 10, removing concerns for multicollinearity among variables in the model.

Discussion

Findings

Based on our survey result, the clinicians at two campuses of an urban academic medical center had a strong interest in order sets for pain management. A majority (88.8%) were interested in having an order set for pain management; 90.5% were interested in using an order set with CDS functions for alternative pain management to opioids. In total, 88.8% agreed that order sets make order placement easier, and over half (55.2%) of clinicians agreed that it requires less work to use order sets compared with standalone orders in the current EHR. While the former is based on PE, the latter is based on EE.

Three out of four major constructs, PE, SI, and FC, have statistically significant impacts on clinicians' adoption intention, which are consistent with previous studies on factors impacting clinicians' technology adoption intention by using the UTAUT framework.[34] [47] [48] [49] Our study found that SI had the strongest impact (odds ratio = 5.25, p-value = 0.015) on clinicians' intention to use order sets in EHR among the four constructs in our UTAUT model. SI has been found to be a strong factor that impacts physicians' EHR adoption in extant research via various research methods,[29] [34] [50] [51] [52] although the impact level varied across studies. Some studies found that SI was less impactful than PE, EE, and FC,[47] which is different from our results. This study showed that the PE, order sets make order placement easier, has the second strongest impact on clinicians' intention to use the order sets (odds ratio = 3.32, p-value = 0.013). On the contrary to previous studies, we found that the EE does not have a statistically significant impact on clinicians' intention to use the order sets. This may suggest that high-performing order sets would be more acceptable, which should be considered in the design of the future order sets in EHRs. Contrary to our expectation, the FC, FC1, and FC3, were negatively associated with intention to use, particularly FC3 that is related to EHR training. Since the question was asked about order sets, it is possible that those who were more experienced with the EHR had other preferences such as quick-lists or order panels (i.e., answered based on HER-specific nomenclature but still might or might not prefer content that could provide CDS), although we are not able to verify these preferences. It is also possible that some unavailable controlled variables or a limited sample size may have biased our model result. We also found trends across gender and clinician types. While not statistically significant, female clinicians were consistently more likely to be in the lower UTAUT construct groups (EE, SI, and FC). On the contrary, more female clinicians answered positively to intention to use. Similar trends were observed with non-attending clinicians, who were more likely to report low PE, FC, and SI.

Limitations

Our study has several limitations. The primary one is that the majority of survey respondents were clinicians concentrated in internal medicine or emergency medicine in two urban hospitals in one urban area. Reduced sample size for analysis due to missing values, moreover, limited the number of respondents from other departments such as surgery. In addition, only hospital sites in an academic medical center were studied, which might limit generalizability to other sorts of medical organizations. Further survey distribution to other settings or specialties, such as non-academic centers and surgery department, is needed to be representative of the general clinician population. The newly implemented EHR at the time of survey initiation may have also caused difficulty in assessing whether perceptions assessed in the survey were formed from the legacy EHR or the new EHR system. While we considered broad factors in developing our research questions and questionnaires, future studies could assess respondents' attitudes in terms of their specific user preferences as well as their perceptions on having educational initiatives for the same. Qualitative analysis on open-ended free-text responses could provide additional insights.

Conclusion

The present study findings highlight the importance of health system performance and the technological culture in understanding the barriers to order set acceptance. Under the UTAUT framework, we find that the acceptance of order set use for pain management may be steered by its ease to use as a CDS, peer usage, organizational endorsement, and better integration with other functions in EHR, ensuring that order set design is configured to improve quality metrics and having organizational leadership support to raise awareness may increase acceptance. We further found that perceptions differed across gender and clinician types although not with statistical significance. Ensuring that clinicians across experience levels receive the same EHR training and similar clinical working environment may provide more support to order set use. Future work may also investigate detailed clinician characteristics to determine acceptance.

Clinical Relevance Statement

The study sheds light on factors to be considered when incorporating order sets, a CDS tool in the EHR, specifically for pain management. Identified factors represented the importance of PE, SI, and FC in using order sets. Insights into barriers of order set use are crucial to understand in implementing system-level changes to ensure its efficacy in patient care.

Multiple Choice Questions

1. When implementing order sets, which of the following did the survey find to useful to pay attention to?

   • Patient characteristics

   • Hospital size

   • EHR vendor communication

   • Social influence

   Correct Answer: The correct answer is option d. The results showed that clinicians' intention to use order sets was associated with social influence by leadership and peers.

2. What technology acceptance framework may one consider in studying the perceptions of order sets as a form of clinical decision support?

   • Unified Theory of Acceptance and Use of Technology

   • Theory of reasoned action

   • Theory of planned behavior

   • All of the above

   Correct Answer: The correct answer is option a. Unified Theory of Acceptance and Use of Technology framework is a well-established, validated theoretical framework for understanding technology acceptance in the health care domain.[32]

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

An IRB approval was obtained from Weill Cornell Medicine IRB to conduct this project.

• References

• 1 Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform 2016; 1 (Suppl. 01) S103-S116
  PubMedGoogle Scholar
• 2 Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007; 14 (02) 141-145
  CrossrefPubMedGoogle Scholar
• 3 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
  Google Scholar
• 4 Nanji KC, Garabedian PM, Shaikh SD. et al. Development of a perioperative medication-related clinical decision support tool to prevent medication errors: an analysis of user feedback. Appl Clin Inform 2021; 12 (05) 984-995
  Article in Thieme ConnectPubMedGoogle Scholar
• 5 Harle CA, Dilulio J, Downs SM. et al. Decision-centered design of patient information visualizations to support chronic pain care. Appl Clin Inform 2019; 10 (04) 719-728
  Article in Thieme ConnectPubMedGoogle Scholar
• 6 Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing-2016. Am J Health Syst Pharm 2017; 74 (17) 1336-1352
  CrossrefPubMedGoogle Scholar
• 7 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163 (12) 1409-1416
  CrossrefPubMedGoogle Scholar
• 8 Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc 2003; 10 (04) 322-329
  CrossrefPubMedGoogle Scholar
• 9 Avansino J, Leu MG. Effects of CPOE on provider cognitive workload: a randomized crossover trial. Pediatrics 2012; 130 (03) e547-e552
  CrossrefPubMedGoogle Scholar
• 10 Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag 2009; 38 (03) 26-37
  PubMedGoogle Scholar
• 11 Ghahramani N, Lendel I, Haque R, Sawruk K. User satisfaction with computerized order entry system and its effect on workplace level of stress. J Med Syst 2009; 33 (03) 199-205
  CrossrefPubMedGoogle Scholar
• 12 Wright A, Feblowitz JC, Pang JE. et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
  CrossrefPubMedGoogle Scholar
• 13 Zhang Y, Padman R, Levin JE. Paving the COWpath: data-driven design of pediatric order sets. J Am Med Inform Assoc 2014; 21 (e2): e304-e311
  CrossrefPubMedGoogle Scholar
• 14 Ancker JS, Kern LM, Edwards A. et al; HITEC Investigators. Associations between healthcare quality and use of electronic health record functions in ambulatory care. J Am Med Inform Assoc 2015; 22 (04) 864-871
  CrossrefPubMedGoogle Scholar
• 15 Hildreth AN, Enniss T, Martin RS. et al. Surgical intensive care unit mobility is increased after institution of a computerized mobility order set and intensive care unit mobility protocol: a prospective cohort analysis. Am Surg 2010; 76 (08) 818-822
  CrossrefPubMedGoogle Scholar
• 16 Gellert GA, Davenport CM, Minard CG, Castano C, Bruner K, Hobbs D. Reducing pediatric asthma hospital length of stay through evidence-based quality improvement and deployment of computerized provider order entry. J Asthma 2020; 57 (02) 123-135
  CrossrefPubMedGoogle Scholar
• 17 Best JT, Frith K, Anderson F, Rapp CG, Rioux L, Ciccarello C. Implementation of an evidence-based order set to impact initial antibiotic time intervals in adult febrile neutropenia. Oncol Nurs Forum 2011; 38 (06) 661-668
  CrossrefPubMedGoogle Scholar
• 18 Vandenberg AE, Vaughan CP, Stevens M. et al. Improving geriatric prescribing in the ED: a qualitative study of facilitators and barriers to clinical decision support tool use. Int J Qual Health Care 2017; 29 (01) 117-123
  PubMedGoogle Scholar
• 19 Rivers EP, Coba V, Rudis M. Standardized order sets for the treatment of severe sepsis and septic shock. Expert Rev Anti Infect Ther 2009; 7 (09) 1075-1079
  CrossrefPubMedGoogle Scholar
• 20 Thiel SW, Asghar MF, Micek ST, Reichley RM, Doherty JA, Kollef MH. Hospital-wide impact of a standardized order set for the management of bacteremic severe sepsis. Crit Care Med 2009; 37 (03) 819-824
  CrossrefPubMedGoogle Scholar
• 21 Delgado MK, Shofer FS, Patel MS. et al. Association between electronic medical record implementation of default opioid prescription quantities and prescribing behavior in two emergency departments. J Gen Intern Med 2018; 33 (04) 409-411
  CrossrefPubMedGoogle Scholar
• 22 Montoy JCC, Coralic Z, Herring AA, Clattenburg EJ, Raven MC. Association of default electronic medical record settings with health care professional patterns of opioid prescribing in emergency departments: a randomized quality improvement study. JAMA Intern Med 2020; 180 (04) 487-493
  CrossrefPubMedGoogle Scholar
• 23 Netley J, Armstrong W, Meeks S. Implementation of order sets for opioid alternatives in community hospital emergency departments. Am J Health Syst Pharm 2020; 77 (15) 1258-1264
  CrossrefPubMedGoogle Scholar
• 24 Aldekhyyel RN, Bakker CJ, Pitt MB, Melton GB. The impact of patient interactive systems on the management of pain in an inpatient hospital setting: a systematic review. Appl Clin Inform 2019; 10 (04) 580-596
  Article in Thieme ConnectPubMedGoogle Scholar
• 25 Dolan JE, Lonsdale H, Ahumada LM. et al. Quality initiative using theory of change and visual analytics to improve controlled substance documentation discrepancies in the operating room. Appl Clin Inform 2019; 10 (03) 543-551
  Article in Thieme ConnectPubMedGoogle Scholar
• 26 Westra BL, Johnson SG, Ali S. et al. Validation and refinement of a pain information model from EHR flowsheet data. Appl Clin Inform 2018; 9 (01) 185-198
  Article in Thieme ConnectPubMedGoogle Scholar
• 27 McAlearney AS, Chisolm D, Veneris S, Rich D, Kelleher K. Utilization of evidence-based computerized order sets in pediatrics. Int J Med Inform 2006; 75 (07) 501-512
  CrossrefPubMedGoogle Scholar
• 28 Asaro PV, Sheldahl AL, Char DM. Physician perspective on computerized order-sets with embedded guideline information in a commercial emergency department information system. AMIA Annu Symp Proc 2005; 2005: 6-10
  PubMedGoogle Scholar
• 29 Hao H, Padman R, Sun B, Telang R. Quantifying the impact of social influence on the information technology implementation process by physicians: a hierarchical Bayesian learning approach. Inf Syst Res 2018; 29 (01) 25-41
  CrossrefPubMedGoogle Scholar
• 30 Zhang Y, Liu C, Luo S. et al. Factors influencing patients' intentions to use diabetes management apps based on an extended unified theory of acceptance and use of technology model: web-based survey. J Med Internet Res 2019; 21 (08) e15023
  CrossrefPubMedGoogle Scholar
• 31 Kruse CS, Goetz K. Summary and frequency of barriers to adoption of CPOE in the U.S. J Med Syst 2015; 39 (02) 15
  CrossrefPubMedGoogle Scholar
• 32 Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. Manage Inf Syst Q 2003; 27 (03) 425-478
  CrossrefPubMedGoogle Scholar
• 33 Liu S, Reese TJ, Kawamoto K, Del Fiol G, Weir C. A systematic review of theoretical constructs in CDS literature. BMC Med Inform Decis Mak 2021; 21 (01) 102
  CrossrefPubMedGoogle Scholar
• 34 Kijsanayotin B, Pannarunothai S, Speedie SM. Factors influencing health information technology adoption in Thailand's community health centers: applying the UTAUT model. Int J Med Inform 2009; 78 (06) 404-416
  CrossrefPubMedGoogle Scholar
• 35 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc 1996; 3 (01) 42-55
  CrossrefPubMedGoogle Scholar
• 36 Compeau DR, Higgins CA. Computer self-efficacy—development of a measure and initial test. Manage Inf Syst Q 1995; 19 (02) 189-211
  CrossrefPubMedGoogle Scholar
• 37 Murthy VH. Ending the opioid epidemic—a call to action. N Engl J Med 2016; 375 (25) 2413-2415
  CrossrefPubMedGoogle Scholar
• 38 Calcaterra SL, Yamashita TE, Min SJ, Keniston A, Frank JW, Binswanger IA. Opioid prescribing at hospital discharge contributes to chronic opioid use. J Gen Intern Med 2016; 31 (05) 478-485
  CrossrefPubMedGoogle Scholar
• 39 Jena AB, Goldman D, Karaca-Mandic P. Hospital prescribing of opioids to medicare beneficiaries. JAMA Intern Med 2016; 176 (07) 990-997
  CrossrefPubMedGoogle Scholar
• 40 Clarke H, Soneji N, Ko DT, Yun L, Wijeysundera DN. Rates and risk factors for prolonged opioid use after major surgery: population based cohort study. BMJ 2014; 348: g1251
  CrossrefPubMedGoogle Scholar
• 41 Foy R, Leaman B, McCrorie C. et al. Prescribed opioids in primary care: cross-sectional and longitudinal analyses of influence of patient and practice characteristics. BMJ Open 2016; 6 (05) e010276
  CrossrefPubMedGoogle Scholar
• 42 Bogen DL, Whalen BL, Kair LR, Vining M, King BA. Wide variation found in care of opioid-exposed newborns. Acad Pediatr 2017; 17 (04) 374-380
  CrossrefPubMedGoogle Scholar
• 43 Carey EP, Nolan C, Kerns RD, Ho PM, Frank JW. Association between facility-level utilization of non-pharmacologic chronic pain treatment and subsequent initiation of long-term opioid therapy. J Gen Intern Med 2018; 33 (Suppl. 01) 38-45
  CrossrefPubMedGoogle Scholar
• 44 Isaacs AN, Knight KL, Nisly SA. Analysis of a standardized perioperative pain management order set in highly opioid-tolerant patients. J Patient Saf 2019; 15 (02) 105-110
  CrossrefPubMedGoogle Scholar
• 45 Hoonakker PLT, Carayon P, Walker JM. Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Appl Clin Inform 2010; 1 (03) 268-285
  Article in Thieme ConnectPubMedGoogle Scholar
• 46 Beam KS, Cardoso M, Sweeney M, Binney G, Weingart SN. Examining perceptions of computerized physician order entry in a neonatal intensive care unit. Appl Clin Inform 2017; 8 (02) 337-347
  Article in Thieme ConnectPubMedGoogle Scholar
• 47 Kalavani A, Kazerani M, Shekofteh M. Acceptance of evidence based medicine (EBM) databases by Iranian medical residents using unified theory of acceptance and use of technology (UTAUT). Health Policy Technol 2018; 7 (03) 287-292
  CrossrefPubMedGoogle Scholar
• 48 Hsiao J-L, Chen R-F. Critical factors influencing physicians' intention to use computerized clinical practice guidelines: an integrative model of activity theory and the technology acceptance model. BMC Med Inform Decis Mak 2016; 16 (01) 3
  CrossrefPubMedGoogle Scholar
• 49 Liu L, Miguel Cruz A, Rios Rincon A, Buttar V, Ranson Q, Goertzen D. What factors determine therapists' acceptance of new technologies for rehabilitation—a study using the Unified Theory of Acceptance and Use of Technology (UTAUT). Disabil Rehabil 2015; 37 (05) 447-455
  CrossrefPubMedGoogle Scholar
• 50 Hao H, Padman R, Sun B, Telang R. Modeling social learning on consumers' long-term usage of a mobile technology: a Bayesian estimation of a Bayesian learning model. Electron Commerce Res 2019; 19 (01) 1-21
  CrossrefPubMedGoogle Scholar
• 51 Hao H, Padman R. An empirical study of opinion leader effects on mobile technology implementation by physicians in an American community health system. Health Informatics J 2018; 24 (03) 323-333
  CrossrefPubMedGoogle Scholar
• 52 Hao H, Padman R, Telang R. An empirical study of opinion leader effects on mobile information technology adoption in healthcare. AMIA Annu Symp Proc 2011; 2011: 537-542
  PubMedGoogle Scholar

Address for correspondence

Publication History

Received: 24 November 2021

Accepted: 18 February 2022

Article published online:
27 April 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

• References

• 1 Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform 2016; 1 (Suppl. 01) S103-S116
  PubMedGoogle Scholar
• 2 Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007; 14 (02) 141-145
  CrossrefPubMedGoogle Scholar
• 3 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
  Google Scholar
• 4 Nanji KC, Garabedian PM, Shaikh SD. et al. Development of a perioperative medication-related clinical decision support tool to prevent medication errors: an analysis of user feedback. Appl Clin Inform 2021; 12 (05) 984-995
  Article in Thieme ConnectPubMedGoogle Scholar
• 5 Harle CA, Dilulio J, Downs SM. et al. Decision-centered design of patient information visualizations to support chronic pain care. Appl Clin Inform 2019; 10 (04) 719-728
  Article in Thieme ConnectPubMedGoogle Scholar
• 6 Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing-2016. Am J Health Syst Pharm 2017; 74 (17) 1336-1352
  CrossrefPubMedGoogle Scholar
• 7 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163 (12) 1409-1416
  CrossrefPubMedGoogle Scholar
• 8 Payne TH, Hoey PJ, Nichol P, Lovis C. Preparation and use of preconstructed orders, order sets, and order menus in a computerized provider order entry system. J Am Med Inform Assoc 2003; 10 (04) 322-329
  CrossrefPubMedGoogle Scholar
• 9 Avansino J, Leu MG. Effects of CPOE on provider cognitive workload: a randomized crossover trial. Pediatrics 2012; 130 (03) e547-e552
  CrossrefPubMedGoogle Scholar
• 10 Jamal A, McKenzie K, Clark M. The impact of health information technology on the quality of medical and health care: a systematic review. Health Inf Manag 2009; 38 (03) 26-37
  PubMedGoogle Scholar
• 11 Ghahramani N, Lendel I, Haque R, Sawruk K. User satisfaction with computerized order entry system and its effect on workplace level of stress. J Med Syst 2009; 33 (03) 199-205
  CrossrefPubMedGoogle Scholar
• 12 Wright A, Feblowitz JC, Pang JE. et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
  CrossrefPubMedGoogle Scholar
• 13 Zhang Y, Padman R, Levin JE. Paving the COWpath: data-driven design of pediatric order sets. J Am Med Inform Assoc 2014; 21 (e2): e304-e311
  CrossrefPubMedGoogle Scholar
• 14 Ancker JS, Kern LM, Edwards A. et al; HITEC Investigators. Associations between healthcare quality and use of electronic health record functions in ambulatory care. J Am Med Inform Assoc 2015; 22 (04) 864-871
  CrossrefPubMedGoogle Scholar
• 15 Hildreth AN, Enniss T, Martin RS. et al. Surgical intensive care unit mobility is increased after institution of a computerized mobility order set and intensive care unit mobility protocol: a prospective cohort analysis. Am Surg 2010; 76 (08) 818-822
  CrossrefPubMedGoogle Scholar
• 16 Gellert GA, Davenport CM, Minard CG, Castano C, Bruner K, Hobbs D. Reducing pediatric asthma hospital length of stay through evidence-based quality improvement and deployment of computerized provider order entry. J Asthma 2020; 57 (02) 123-135
  CrossrefPubMedGoogle Scholar
• 17 Best JT, Frith K, Anderson F, Rapp CG, Rioux L, Ciccarello C. Implementation of an evidence-based order set to impact initial antibiotic time intervals in adult febrile neutropenia. Oncol Nurs Forum 2011; 38 (06) 661-668
  CrossrefPubMedGoogle Scholar
• 18 Vandenberg AE, Vaughan CP, Stevens M. et al. Improving geriatric prescribing in the ED: a qualitative study of facilitators and barriers to clinical decision support tool use. Int J Qual Health Care 2017; 29 (01) 117-123
  PubMedGoogle Scholar
• 19 Rivers EP, Coba V, Rudis M. Standardized order sets for the treatment of severe sepsis and septic shock. Expert Rev Anti Infect Ther 2009; 7 (09) 1075-1079
  CrossrefPubMedGoogle Scholar
• 20 Thiel SW, Asghar MF, Micek ST, Reichley RM, Doherty JA, Kollef MH. Hospital-wide impact of a standardized order set for the management of bacteremic severe sepsis. Crit Care Med 2009; 37 (03) 819-824
  CrossrefPubMedGoogle Scholar
• 21 Delgado MK, Shofer FS, Patel MS. et al. Association between electronic medical record implementation of default opioid prescription quantities and prescribing behavior in two emergency departments. J Gen Intern Med 2018; 33 (04) 409-411
  CrossrefPubMedGoogle Scholar
• 22 Montoy JCC, Coralic Z, Herring AA, Clattenburg EJ, Raven MC. Association of default electronic medical record settings with health care professional patterns of opioid prescribing in emergency departments: a randomized quality improvement study. JAMA Intern Med 2020; 180 (04) 487-493
  CrossrefPubMedGoogle Scholar
• 23 Netley J, Armstrong W, Meeks S. Implementation of order sets for opioid alternatives in community hospital emergency departments. Am J Health Syst Pharm 2020; 77 (15) 1258-1264
  CrossrefPubMedGoogle Scholar
• 24 Aldekhyyel RN, Bakker CJ, Pitt MB, Melton GB. The impact of patient interactive systems on the management of pain in an inpatient hospital setting: a systematic review. Appl Clin Inform 2019; 10 (04) 580-596
  Article in Thieme ConnectPubMedGoogle Scholar
• 25 Dolan JE, Lonsdale H, Ahumada LM. et al. Quality initiative using theory of change and visual analytics to improve controlled substance documentation discrepancies in the operating room. Appl Clin Inform 2019; 10 (03) 543-551
  Article in Thieme ConnectPubMedGoogle Scholar
• 26 Westra BL, Johnson SG, Ali S. et al. Validation and refinement of a pain information model from EHR flowsheet data. Appl Clin Inform 2018; 9 (01) 185-198
  Article in Thieme ConnectPubMedGoogle Scholar
• 27 McAlearney AS, Chisolm D, Veneris S, Rich D, Kelleher K. Utilization of evidence-based computerized order sets in pediatrics. Int J Med Inform 2006; 75 (07) 501-512
  CrossrefPubMedGoogle Scholar
• 28 Asaro PV, Sheldahl AL, Char DM. Physician perspective on computerized order-sets with embedded guideline information in a commercial emergency department information system. AMIA Annu Symp Proc 2005; 2005: 6-10
  PubMedGoogle Scholar
• 29 Hao H, Padman R, Sun B, Telang R. Quantifying the impact of social influence on the information technology implementation process by physicians: a hierarchical Bayesian learning approach. Inf Syst Res 2018; 29 (01) 25-41
  CrossrefPubMedGoogle Scholar
• 30 Zhang Y, Liu C, Luo S. et al. Factors influencing patients' intentions to use diabetes management apps based on an extended unified theory of acceptance and use of technology model: web-based survey. J Med Internet Res 2019; 21 (08) e15023
  CrossrefPubMedGoogle Scholar
• 31 Kruse CS, Goetz K. Summary and frequency of barriers to adoption of CPOE in the U.S. J Med Syst 2015; 39 (02) 15
  CrossrefPubMedGoogle Scholar
• 32 Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. Manage Inf Syst Q 2003; 27 (03) 425-478
  CrossrefPubMedGoogle Scholar
• 33 Liu S, Reese TJ, Kawamoto K, Del Fiol G, Weir C. A systematic review of theoretical constructs in CDS literature. BMC Med Inform Decis Mak 2021; 21 (01) 102
  CrossrefPubMedGoogle Scholar
• 34 Kijsanayotin B, Pannarunothai S, Speedie SM. Factors influencing health information technology adoption in Thailand's community health centers: applying the UTAUT model. Int J Med Inform 2009; 78 (06) 404-416
  CrossrefPubMedGoogle Scholar
• 35 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc 1996; 3 (01) 42-55
  CrossrefPubMedGoogle Scholar
• 36 Compeau DR, Higgins CA. Computer self-efficacy—development of a measure and initial test. Manage Inf Syst Q 1995; 19 (02) 189-211
  CrossrefPubMedGoogle Scholar
• 37 Murthy VH. Ending the opioid epidemic—a call to action. N Engl J Med 2016; 375 (25) 2413-2415
  CrossrefPubMedGoogle Scholar
• 38 Calcaterra SL, Yamashita TE, Min SJ, Keniston A, Frank JW, Binswanger IA. Opioid prescribing at hospital discharge contributes to chronic opioid use. J Gen Intern Med 2016; 31 (05) 478-485
  CrossrefPubMedGoogle Scholar
• 39 Jena AB, Goldman D, Karaca-Mandic P. Hospital prescribing of opioids to medicare beneficiaries. JAMA Intern Med 2016; 176 (07) 990-997
  CrossrefPubMedGoogle Scholar
• 40 Clarke H, Soneji N, Ko DT, Yun L, Wijeysundera DN. Rates and risk factors for prolonged opioid use after major surgery: population based cohort study. BMJ 2014; 348: g1251
  CrossrefPubMedGoogle Scholar
• 41 Foy R, Leaman B, McCrorie C. et al. Prescribed opioids in primary care: cross-sectional and longitudinal analyses of influence of patient and practice characteristics. BMJ Open 2016; 6 (05) e010276
  CrossrefPubMedGoogle Scholar
• 42 Bogen DL, Whalen BL, Kair LR, Vining M, King BA. Wide variation found in care of opioid-exposed newborns. Acad Pediatr 2017; 17 (04) 374-380
  CrossrefPubMedGoogle Scholar
• 43 Carey EP, Nolan C, Kerns RD, Ho PM, Frank JW. Association between facility-level utilization of non-pharmacologic chronic pain treatment and subsequent initiation of long-term opioid therapy. J Gen Intern Med 2018; 33 (Suppl. 01) 38-45
  CrossrefPubMedGoogle Scholar
• 44 Isaacs AN, Knight KL, Nisly SA. Analysis of a standardized perioperative pain management order set in highly opioid-tolerant patients. J Patient Saf 2019; 15 (02) 105-110
  CrossrefPubMedGoogle Scholar
• 45 Hoonakker PLT, Carayon P, Walker JM. Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Appl Clin Inform 2010; 1 (03) 268-285
  Article in Thieme ConnectPubMedGoogle Scholar
• 46 Beam KS, Cardoso M, Sweeney M, Binney G, Weingart SN. Examining perceptions of computerized physician order entry in a neonatal intensive care unit. Appl Clin Inform 2017; 8 (02) 337-347
  Article in Thieme ConnectPubMedGoogle Scholar
• 47 Kalavani A, Kazerani M, Shekofteh M. Acceptance of evidence based medicine (EBM) databases by Iranian medical residents using unified theory of acceptance and use of technology (UTAUT). Health Policy Technol 2018; 7 (03) 287-292
  CrossrefPubMedGoogle Scholar
• 48 Hsiao J-L, Chen R-F. Critical factors influencing physicians' intention to use computerized clinical practice guidelines: an integrative model of activity theory and the technology acceptance model. BMC Med Inform Decis Mak 2016; 16 (01) 3
  CrossrefPubMedGoogle Scholar
• 49 Liu L, Miguel Cruz A, Rios Rincon A, Buttar V, Ranson Q, Goertzen D. What factors determine therapists' acceptance of new technologies for rehabilitation—a study using the Unified Theory of Acceptance and Use of Technology (UTAUT). Disabil Rehabil 2015; 37 (05) 447-455
  CrossrefPubMedGoogle Scholar
• 50 Hao H, Padman R, Sun B, Telang R. Modeling social learning on consumers' long-term usage of a mobile technology: a Bayesian estimation of a Bayesian learning model. Electron Commerce Res 2019; 19 (01) 1-21
  CrossrefPubMedGoogle Scholar
• 51 Hao H, Padman R. An empirical study of opinion leader effects on mobile technology implementation by physicians in an American community health system. Health Informatics J 2018; 24 (03) 323-333
  CrossrefPubMedGoogle Scholar
• 52 Hao H, Padman R, Telang R. An empirical study of opinion leader effects on mobile information technology adoption in healthcare. AMIA Annu Symp Proc 2011; 2011: 537-542
  PubMedGoogle Scholar

• Supplementary Material