Investigation the success rate of hospital information system (HIS): Development of a questionnaire and case study

Abstract

Introduction and objectives

The implantation of hospital information systems (HISs) has grown dramatically in recent years. Understanding the success rate of HIS is key in health organizations. In this study, a validated questionnaire for HISs evaluation based on the Information System Success Model (ISSM) has been provided. In addition, the HIS success rate was determined.

Materials and methods

The current study was conducted at one of the largest teaching hospitals in eastern Iran. The 44-items questionnaire was developed for data gathering. The questionnaire covered ISSM dimensions, which include analysis of quality system, quality of information, quality service, system use, usefulness, satisfaction, and net benefits. Content validity, constructs validity, and reliability of the ISSM questionnaire was measured. HIS success rate has been determined and categorized based on users’ perspective as follows: appropriate (75% ≤ HIS success rate), moderate (50% ≤ HIS success rate < 75%), low coverage (25% ≤ HIS success rate < 50%), and poor (coverage rate < 25%).

Results

In total, 253 users participated in the study. The ISSM questionnaire was validated by an expert panel with CVI: 85.12% and CVR: 88.22%. The overall Cronbach's alpha value of the instrument was determined as 92.2%. Nine factors with eigenvalues greater than 1.00 were identified, jointly accounting for 66.91% of the total variance. The value of KMO was.866 showed that the sample size was adequate for factor analysis. The highly significant Bartlett's test (p < 0.000) indicated that variables were correlated and the factor analysis was appropriate. Our results demonstrated that the total mean of HIS success was “moderate” base on the users’ point of view.

Conclusions

The findings of the current study, provide valuable scientific evidence for key affecting factors on hospital EHR in Iran as a developing country.

Resumen

Introducción y objetivos

La implantación de los sistemas de información hospitalaria (HIS) ha crecido drásticamente en los últimos años. Comprender la tasa de éxito de un HIS es clave para las organizaciones sanitarias. En este estudio se aportó un cuestionario validado para la evaluación de HIS, basado en el Information System Success Model (ISSM). Además, se calculó la tasa de éxito del HIS.

Materiales y métodos

El estudio actual se llevó a cabo en uno de los mayores hospitales docentes del este de Irán. Se desarrolló un cuestionario de 44 ítems para recabar los datos. Dicho cuestionario cubrió las dimensiones del ISSM, que incluyen el análisis del sistema de calidad, la calidad de la información, la calidad del servicio, así como el uso, utilidad, satisfacción y beneficios netos del sistema. Se midieron la validez del contenido, la validez de los constructos y la fiabilidad del cuestionario ISSM. Se determinó y categorizó la tasa de éxito del HIS sobre la base de la perspectiva de los usuarios, de la manera siguiente: adecuada (75% ≤ tasa de éxito de HIS), moderada (50% ≤ tasa de éxito de HIS < 75%), baja cobertura (25% ≤ tasa de éxito de HIS < 50%), y mala (tasa de cobertura < 25%).

Resultados

En total participaron 253 usuarios en el estudio. El cuestionario ISSM fue validado por un panel de expertos, donde CVI = 85,12% y CVR = 88,22%. El valor general de alfa de Cronbach del instrumento fue del 92,2%. Se identificaron nueve factores con Eigen valores superiores a 1, que representaron en conjunto el 66,91% de la varianza total. El valor de KMO fue de 0,866, lo cual reflejó que el tamaño de la muestra fue adecuado para el análisis factorial. La prueba de Bartlett altamente significativa (p < 0,000) indicó que las variables estaban correlacionadas, y que el análisis factorial era adecuado. Nuestros resultados demostraron que la media total del éxito de HIS era «moderada», basándose en el punto de vista del usuario.

Conclusiones

Los hallazgos del presente estudio aportan evidencia científica valiosa para los factores clave que afectan a los registros sanitarios electrónicos (EHR) en Irán, como país en desarrollo.

Introduction

In the past several decades, hospital information systems (HISs) have played an important role in health care settings. The HISs has been widely adopted in hospitals. Evaluating the success rate of health information systems to help decision-makers obtain information about the impact of these systems, therefore, system evaluation becomes an important topic to all organizations that aim to implement any health information systems.¹

System success refers to the adoption and usage of an information system by a large number of users on a continued basis.² Success in health information systems refers to “the usefulness of the information system for users and healthcare managers”.³

Information System Success Model (ISSM) is one of the well known and valid models to assess the success rate of information systems. ISSM was developed by DeLone and Mclean in 1990 and was subsequently updated in 2003. ISSM is encompassed of seven dimensions including system quality, information quality, service quality system use, intention to use, satisfaction, and net-benefits.⁴

ISSM was adopted and modified in many of health care studies for HIS evaluation. For example in the studies by Nunes et al.,⁵ Ojo,⁶ Petter,⁷ Ribière,⁸ and Yu et al.⁹ HIS was evaluated using original or a modified version of ISSM. The original version of ISSM is a conceptual framework and does not provide a questionnaire for evaluating the HISs.¹⁰ As well, ISSM is a causal mole. The overall system success depends on the interaction of seven dimensions of ISSM.⁴ In plenty of studies, the most important influencing factors on HISs success were determined based on ISSM.^5,11,12 In several studies, the relation between the dimensions of ISSM was examined or ISSM was modified and validated.^6,8,9,13,14 For example in a study by Nunes et al.⁵ influencing factors on satisfaction with the HIS in Brazil was assessed. As far as we know, few studies have focused on suggesting a validated questionnaire for HISs evaluation and investigating of HIS success rate.

The aim of the current study was to create a validated questionnaire for HISs evaluation based on the ISSM model. Secondary aim was to determine and categorize the HIS success rate in a large teaching hospital based on the users’ perspective.

Methods

Instrument

In the current study, a self-administrated structured questionnaire was developed based on ISSM for data gathering. To ensure validity, questions assessing each dimension were adapted from the validated questionnaires of the previous studies.

The questionnaire was divided into two sections. The first section was included sociodemographic information of the participant users. The second section was comprised of 44 questions that covered seven dimensions of ISSM dimensions. A 5-point Likert scale ranging from “completely disagree” to “completely satisfy” was mainly used. The seven dimensions and number of questions in each dimension were as follows: system quality (n = 8)^12,15–19 information quality (n = 11),^{12,17,20–22} service quality (n = 8),^23–26 system use (n = 1),¹⁵ intention to use (n = 2),¹⁵ satisfaction (n = 3), and net benefits (n = 11).^{10,12,20,25,27–30}Appendix A demonstrates the questionnaire. The dimension of system quality consisted of three evaluation measures including availability, reliability, and usability. The dimension of system quality, information quality, and service quality encompassed several evaluation measures.

Content validity ISSM questionnaire was measured using a Content Validity Index (CVI) and Content Validity Ratio (CVR) by an expert panel (ten medical informatics specialists).

Construct validity was carried out using factor analysis. Factor analysis was performed based on the principal component analysis (PCA) technique to explore important and appropriate factors within each dimension. The Kaiser-Meyer-Olkin (KMO) was applied to measure sampling adequacy, and Bartlett's Test of Sphericity was used to assess whether factor analysis is appropriate. Varimax rotation was performed to facilitate the identification of uncorrelated factors. The number of factors to be retained was based on eigenvalues > 1, with factor loading serving as an indicator of the degree to which each item was associated with the factor. For each of the remained factors, the number of items making a significant contribution was determined by considering a factor loading of the threshold of 0.400 or greater. The reliability of the questionnaire was measured by Cronbach's alpha.

Study settings

The study conducted in a large teaching hospital. The case hospital is the largest teaching hospital in eastern Iran, which has been in operation for more than 70 years. The hospital consists of 29 specialized and sub-specialized departments such as gynecology, midwifery, poisoning, cardiac surgery, ICU, and CCU.

The HIS of the case hospital covers subsystems for different services including admission, discharge, and transfer (ADT), outpatient clinics, pharmacy, nursing, laboratory, radiology, accounting, and insurance.

Participants

All HIS users who worked in administrative and financial departments, clinical departments, laboratory, pharmacy, radiology, social security, nutrition, and physiotherapy were invited to participate in the current study. Given, the high number of users in the clinical departments (n = 1450), 10 percent of them were considered as sample size.

Data collection

The current study was a part of the doctorate thesis (research plan code: 951350) and supported by Mashhad University of Medical Sciences. The study was reviewed and approved by the ethical committee of Mashhad University of Medical Sciences. The researches met all HISs’ users in person and asked them to participate in the study. Users who agreed to participate in this study completed the questionnaires. Participation in the study was voluntary. Participants were free to discontinue completing the questionnaire at any time.

Measurement of HIS success rate

HIS success rate in the case hospital was calculated in three steps. First, the rate of HIS success based on the point of view of each user was determined for evaluation measures, dimensions, and total dimensions of the ISSM based questionnaire. In the second step, the mean and confidence interval of the HIS success rate was calculated for evaluation measures, dimensions, and total dimensions of the ISSM based questionnaire by all groups of users. These groups were financial and administrative departments, clinical departments, laboratory, pharmacy, radiology, and nutrition and physiotherapy. For more details about the measurement of the HIS success rate, see our previous paper.³¹

In the third step, the mean of the HIS success rate was categorized as follows: 1 – Appropriate (75% ≤ HIS success rate), 2 – Moderate (50% ≤ HIS success rate < 75%), 3 – Low coverage (25% ≤ HIS success rate < 50%), and 4 – Poor (coverage rate < 25%).

Statistical analysis

Sociodemographic variables were summarized using frequencies and proportions. Missing data were removed from the analysis. As well, negative statements were reversed so that higher values indicated more score.

To compare the mean of HIS success rate among users’ characteristics the normality of data in seven dimensions of ISSM was assessed. The Shapiro–Wilk test was performed to assess the normality of data. Mean comparisons of the HIS success rate were made among seven user groups and other users’ characteristics using ANOVA for normal variables and Kruskal–Wallis test for non-normal variables. The Tukey Honestly Significant Difference (HSD) test for normal variables and the Mann–Whitney test for non-normal variables was performed to identify significant inter-group differences. Statistical significance for all of the analyses was defined as p ≤ 0.05. Data analysis was performed using SPSS, version 21 statistical software.

Results

Participants

A total of 300 questionnaires were distributed to all of the target users. Finally, 253 valid questionnaires were collected. (Response rate %84.3). Response rates were very high across the departments in the case hospital. Table 1 shows the characteristics of participants.

Table 1.

Participants’ characteristics.

Participants’ characteristics	Sub groups	Frequency (%)
Departments	Administrative and financial	44 (17.4)
	Clinical	148 (58.5)
	Laboratory	28 (11.1)
	Pharmacy	5 (2)
	Radiology	13 (5.1)
	Social security	10 (4)
	Nutrition – physiotherapy	5 (2)
Sex	Male	94 (37.2)
	Female	159 (62.8)
Age (years)	20–29	54 (21)
	30–39	123 (47)
	40–49	72 (28)
	=>50	4 (1.6)
Education level	High School Diploma	15 (5.8)
	Associate's	22 (8.6)
	Bachelor's	192 (74.7)
	Master's	24 (9.3)
Work experience (years)	<4	54 (24.1)
	5–9	39 (17.4)
	10–14	52 (23.2)
	15–19	46 (20.5)
	20–24	21 (9.4)
	≥25	12 (5.4)
ICDL certification	No	30 (11.9)
	Yes	215 (85)
Computer experience (years)	≤1	23 (9.1)
	2	23 (9.1)
	3	11 (4.3)
	4	26 (10.3)
	≥5	170 (67.2)
HIS experience (years)	≤1	10 (4)
	2	7 (2.8)
	3	14 (5.5)
	4	11 (4.3)
	≥5	211 (83.4)

Reliability and validity

The ISSM questionnaire was validated by an expert panel with CVI: 85.12% and CVR: 88.22%. The overall Cronbach's alpha value of the instrument was determined as 92.2%, demonstrating high reliability. The value of Cronbach's alpha was very high among all dimensions of the questionnaire and the range of value of Cronbach's alpha was 70.2% to 93%. Since the dimension of “system use” had only one question, Cronbach's alpha for this dimension was not measured. Nine factors with eigenvalues greater than 1.00 were identified, jointly accounting for 66.91% of the total variance. The value of KMO was.866 showed that the sample size was adequate for factor analysis. The highly significant Bartlett's test (p < 0.000) indicated that variables were correlated and the factor analysis was appropriate. “The range” of factor loadings was 0.496 to 0.909. Out of a total of 44 questions, the factor loadings of three questions were lower than 0.3 that were excluded from the questionnaire. The questions were as follows: “interacting with the HIS does not require a lot of your mental effort”, “privileges required to access the HIS restrict accessibility to necessary patient information for daily tasks”, and “you have received appropriate levels of training; so that you can employ HIS features and procedures for your daily tasks”. These questions belonged to dimensions of system quality, information system, and service quality, respectively. The factors 1–2–3 were labeled availability, reliability, and usability. These three factors considered as a dimension of system quality. The factors of 4–9 were labeled according to the ISSM dimensions that were as follows: information quality, service quality, intention to use, system use, satisfaction, and net benefits. Table 2 shows the value of Cronbach's alpha for each factor and the factor loadings of questions.

Table 2.

Results of factor analysis.

Dimensions	Factors	(Cronbach's alpha %)	Questions (factor loading)
System quality	1	(70.2)	Availability1 (0.724), availability2 (0.635).
	2	(79.4)	Reliability1 (0.852), reliability2 (0.835).
	3	(74.7)	Usability1 (0.743), usability2 (0.792), usability3 (0.759), usability4 (-0.042)*.
Information quality	4	(72.3)	Security1 (0.645), security2 (-0.006)*, ease of understanding1 (0.703), ease of understanding2 (0.627), ease of understanding3 (0.515), ease of understanding4 (0.689), ease of understanding5 (0.549), completeness1 (0.7), completeness2 (0.695), acurracy1 (0.509), adaptability1 (0.065).
Service quality	5	(89.2)	Responsiveness1 (0.645), responsiveness2 (0.734), assurance1 (0.752), assurance2 (0.724), empathy1 (0.761), empathy2 (0.715), empathy3 (0.496), education1 (0.64), education2 (-0.002)*.
Intention to use	6	(90.6)	Intention to use 1 (0.89), intention to use 2 (0.909).
System use	7		use1 (0.766).
Satisfaction	8	(77.8)	satisfaction1 (0.5), satisfaction2 (0.427).
Net benefits	9	(93.0)	Net benefits1 (0.802), net benefits2 (0.768), net benefits3 (0.781), net benefits4 (0.703), net benefits5 (0.722), net benefits6 (0.715), net benefits7 (0.738), net benefits8 (0.639), net benefits9 (0.73), net benefits10 (0.695), net benefits11 (0.71).

Note: * The excluded questions from the questionnaire (factor loadings lower than 0.3).

HIS success rate

On average, the “total HIS success rate” was 63.83% (CI: 62.65%, 65.02%) that were categorized in the “moderate” group. The range of the HIS success rate was 51.15% to 73.86%. The case HIS acquired highest rate in system quality dimension. The dimension of information quality had the lowest rates of HIS success. Table 3 presents the mean success rates for all HIS evaluation dimensions based on the users’ point of view. Fig. 1 shows the total HIS success rate in departments of the case hospital.

Table 3.

Mean of hospital information system (HIS) success rate based on Information System Success Model (ISSM).

Dimensions	Evaluation measures	Mean (95% confidence interval)	HIS success categories*
System quality	Availability	58.93 (56.57, 61.29)	Moderate
	Reliability	67.00 (64.76, 69.22)	Moderate
	Usability	72.83 (71.04, 74.62)	Moderate
	Total	73.86 (72.35, 75.37)	Moderate
Information quality	Security	21.74 (20.99, 22.49)	Poor
	Ease of understanding	79.59 (78.13, 81.04)	Appropriate
	Completeness	42.58 (41.29, 43.87)	Low
	Accuracy	22.79 (22.11, 23.48)	Poor
	Compatibility	47.04 (44.26, 49.81)	Low
	Total	51.15 (50.05, 52.24)	Moderate
Service quality	Responsiveness	59.80 (57.65, 61.96)	Moderate
	Assurance	61.15 (58.97, 63.32)	Moderate
	Empathy	63.29 (61.33, 65.26)	Moderate
	Education	55.42 (53.03, 57.80)	Moderate
	Total	54.42 (52.73, 56.11)	Moderate
Intention to use		68.18 (65.72, 70.64)	Moderate
System use		69.73 (65.83, 73.62)	Moderate
Satisfaction		63.05 (60.83, 65.26)	Moderate
Net benefits		64.48 (62.29, 66.67)	Moderate
Total HIS success		63.83 (62.65, 65.02)	Moderate

Note: Appropriate (75% ≤ HIS success rate), Moderate (50% ≤ HIS success rate < 75%), Low coverage (25% ≤ HIS success rate < 50%), and Poor (coverage rate < 25%).

Figure 1.

Total HIS success rate.

System quality

Based on users’ opinion the dimension of system quality attained the highest mean of HIS success. Fig. 2 shows the mean of HIS success in the factors of system quality dimension. A significant difference between “usability” and the variable of gender was observed (p < 0.046). The “usability” in the female was higher than men. The findings of this study showed that there is a significant difference in the mean of HIS success in the “usability” evaluation measure between users in the various departments (p < 0.011). The “usability” based on the opinion of users in financial and administrative departments was higher than users in clinical and radiology departments. There was a significant difference between the ICDL certificate and system reliability (p < 0.014). Users with ICDL certificate compare to users without ICDL agreed that HIS had lower reliability.

Figure 2.

The mean success rate on the dimension of system quality.

Information quality

The dimension of “system quality” the lowest mean of HIS success. Fig. 3 shows the mean of HIS success in various departments in this dimension. Our results showed that there was a reverse relation between the mean of evaluation measure of “completeness” and “education level” (p-value: 0.004). A significant difference was observed between the evaluation measure of “compatibility” and “departments” (p-value: 0.24). The HIS has more compatibility with other computer systems in the financial and administrative departments compare to clinical departments. There was a significant difference between the “ICDL certificate” and “system reliability” (p < 0.003). The users with ICDL certificate compare to users without ICDL believed that the HIS had more compatibility with other systems.

Figure 3.

The mean success rate on the dimension of information quality.

Service quality

The mean HIS success rate of the “service quality” dimension was ranked 6th. Fig. 4 shows the mean of HIS success in this dimension. A significant relation was observed between the mean of HIS success rate of responsiveness and the variable of “departments” (p-value: 0.019). The mean of HIS success rate of responsiveness in the laboratory department had a significant difference with the financial and administrative department (p-value: 0.009), clinical departments (p < 0.001), and nutrition and physiotherapy (p-value: 0.032). Half of the participants in the present study belonged to these three departments. The mean of HIS success rate of responsiveness in the laboratory department was lower than the mentioned departments. There was a positive relationship between the mean of HIS success rate of “responsiveness” and the variables of “age” (p < 0.001), “education level” (p-value: 0.008), and “work experience” (p-value: 0.014). As well, there was a significant difference the mean of HIS success rate with the variables of “age” (p-value: 0.008), “education level” (p-value: 0.014), and “work experience” (p-value: 0.019). The results showed that users younger than 40 years old needed more training courses than older users (p-value: 0.010).

Figure 4.

The mean success rate on the dimension of service quality.

The mean HIS success in the evaluation measure of “responsiveness” was significantly associated with users’ education level (p-value: 0.008). There was an important difference between the mean of “responsiveness” between users with Master's and High School Diploma (p-value: 0.050) as well as an Associate's degree (p-value: 0.045).

The results showed that there was a significant relationship between the mean of HIS success in the evaluation measure of “assurance” and “work experience”. The mean of assurance in the users with higher “work experience” was larger than users with lower work experience.

Intention to use and system use

The mean of HIS success rate of “intention to use” and “system use” dimensions was ranked 2nd and 3nd, respectively. There was a significant difference between “intention to use” and users’ characteristics. Given, the high mean rate of HIS success in the dimension of “system use”. But, a significant difference between the mean of “system use” in financial and administrative departments and other departments was observed. The mean rate of “system use” in these departments was higher than the others. The lowest mean rate of “system use” belonged to clinical departments. The mean rate of “system use” among users of financial and administrative departments, social security, laboratory, pharmacy, radiology, nutrition and physiotherapy and, clinical departments were 96.13%, 94.00%, 87.14%, 76.00%, 73.85%, 72.00%, and 56.88%, respectively. There was a significant difference between “education level” and “system use” (p < 0.011).

There was a significant relation between “system use” and education level ‘(p-value: 0.016). The rate of system use was lower in the Diploma users in compression with Associate's users (p-value: 0.040) as well as Master's users (p-value: 0.05). As well, a positive relationship between “system use” and Master's user and Bachelor's users was observed. Reserves significant relation was observed between “system use” and age. “System use” in older users was lower than younger users.

Satisfaction

Based on users’ experience the mean success rate of the “satisfaction” dimension was 63.05% (CI: 60.83%, 65.26%). The mean HIS success in the evaluation measure of “satisfaction” was not significantly associated with departments in the case study. A positive relationship between satisfaction and education level was observed. There was an important difference between the mean of satisfaction between users with High School Diploma and Associate's degree (p-value: 0.05) as well as a Master's degree (p-value: 0.024). Our results showed that there was a reverse relation between the mean of evaluation measure of “satisfaction” and “ICDL certification” (p-value: 0.012). The users with ICDL certificate comparison with users without ICDL had a higher level of satisfaction.

Net benefits

The “net-benefits” dimension acquired a moderate success rate on users’ viewpoints. A significant difference between the mean success rate of “net-benefits” and users’ characteristics was not observed.

Discussion

In the current study, an ISSM based questionnaire was developed for hospital information systems. The questionnaire had a high degree of validity and reliability. As well, the HIS success rate based on users’ perspective in seven dimensions of ISSM was assessed and categorized. The most important results of the current study will be discussed further in the following paragraphs.

Based on the results of factor analysis nine factors were identified in the ISSM questionnaire. Six factors completely confirmed the original ISSM dimensions. But, three independent factors including availability, reliability, and usability were identified that was considered as a dimension of system quality. The factor of availability refers to “access to the computer resource and infrastructure where and when it is needed”. Reliability is “the dependability and consistency of access and uptime of systems”.²⁴ Usability refers to “the capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfill the specified rang of tasks, within the specified range of environmental scenarios”.³² The current study was conducted in a large teaching hospital in a developing country. Hospitals in developing countries face technological and economic constraints. From our results, it is possible that the factors of availability, reliability, and usability can get different success rates in a hospital. Our results showed that although the case hospital have not sufficient computer resources and computer equipment was subject to frequent unexpected or inconvenient downtimes, the HIS usability had approximately an appropriate success rate. These are important findings in the understanding of the influencing factors on system quality in a developing country. It seems the success rate of availability, reliability, and usability factors may be measured separately. As well, the impact on HIS success of three factors was confirmed in many of the previous studies. The findings of the current study were directly in line with previous findings. For example, the results of a study by Khalifa and Alswailem showed the unavailability of laptop computers and mobile computers to data entry and information retrieval processes at the point of care, system failures, unexpected slowness of HIS, and poor interface are most important influencing factors of HIS success. They believed that this factor can reduce the chance of HIS acceptance by users and success implementation.³³

The results of the present of the study showed there was a positive relation between HIS success and users’ characteristics. For example, a significant difference between the success rate of usability and gender as well as users’ department was observed. Also, our results revealed that there was a reverse relation between evaluation measure of completeness and education level. Users with university degrees highlighted that the HIS did not cover the department's workflow and did not offer the information and functions that users need. A similar result was reported by Amin et al.³⁴ The results of this study showed that there are significant differences between different types of users in the quality of HIS interface, quality of HIS function, and quality of HIS performance.³⁴ Nadri et al.³⁵ and Phichitchaisopa et al.³⁶ found out that there is a significant difference between usability and gender. We suggest that HIS developer should pay attention to the important roles of users in HIS success, and the associated characteristics of users when delivering HIS.

System compatibility¹⁷ and completeness^21,37 are two critical characteristics to effect an individual health care provider's performance and HIS success rate. The participants did not satisfied with the completeness and system compatibility of the case HIS. A significant difference between system compatibility and departments was observed. The results showed that the financial and administrative departments’ workflow has a higher level of HIS coverage in comparison with clinical and laboratory departments in HIS. The results demonstrate two things. First, the case HIS focuses on financial and administrative workflow. Second, HIS has low compatibility with clinical software and equipment. This result ties well with previous studies. The findings of the study by Asadi et al. revealed that information systems in Iran are mainly focused on financial objectives.³⁸ Moghaddasi et al. stated that although more than 80 percent of government hospitals in Iran have HIS, Existing HISs encompasses functions including administrative, financial, admissions, discharge, transfers, para clinical data, and very limited clinical function.³⁹

HIS education acquired the lowest HIS success based on the users’ perspectives. Our results revealed that although the case HIS had been implemented for 20 years, users need to holding HIS training courses. These results confirmed the importance of holding continuous training courses for HIS. A similar pattern of results was obtained in a study by Ajami et al. In this study a hospital information system was evaluated 3 years after implementation. They found out that users still need HIS education.² The results of a study by Zikos et al. highlighted that many challenges in HIS implementation were directly related to the lack of skilled IT experts and insufficiently trained users.⁴⁰

In the present study, a reverse relation between HIS education and the variables of age, work experience, and computer experience was observed. The young users with low work experience and computer experience need more HIS education to compare to other users. These are explanations of the reasons for our results. HIS in the case hospital had been implemented 20 years ago. As well, the HIS does not have a built-in user manual. It seems new users with compression older users received lower education training courses and resources. They may be face problems in using the system. Overall these findings are supported by findings reported by Aldosari et al. They found out the variables of gender, age, education level, years of work experience, years of computer experience, and formal computer training were the important factors that had a positive impact on the system acceptance.⁴¹ As well, Ajami et al. state that non-trained users fear system changes because they fear would not be able to use the HIS and lose their job.² So, the hospital should hold regular training courses to educate users on how to use the HIS.

In the study by Jebraeily et al. the quality of IT service was measured in four university hospitals in Iran. The results of this study showed that among all influencing factors of service quality, users had the lowest satisfaction with the responsiveness. Jebraeily et al. believed that the IT staffs in hospitals are responsible for many tasks and many problems are referred to them.⁴² They stated the high workload led to the low level of responsiveness of IT staff.⁴² A similar result was observed

In the current study. The mean of HIS success in the evaluation measure of “responsiveness” was lower than 60%. As well, responsiveness had a significant difference in laboratory department comparison with other departments. Laboratory subsystems in the case HIS has been changed dramatically the three months ago that led to a remarkable increase number of laboratory user's requests. An important finding of these results is the perceived importance of the enhancement of HIS support when the system has major changes. Our results provide evidence of the existence of a positive relation between responsiveness and education level. The rate of IT staffs’ responsiveness in the users with Maters’ degree was higher than other users. In a study by Ahmadian et al. was reported that there was a positive relationship between education level and hardware challenges.⁴³ From this evidence, it is clear that there is an important relationship between education, hardware challenges, and IT staffs responsiveness.

Conclusion

Our results indicated the seven of ISSM were found to be applicable to measure the HIS success rate in hospitals. As well, the developed questionnaire provide objective scientific evidence for HIS success rate that can help to judge by comparison HISs success rate in various hospitals.

Conflict of interests

The authors declare that they have no conflict of interest.

Acknowledgments

The current study was a part of the PhD project (research plan code: 951350) and supported by medical informatics department at Mashhad University of Medical Sciences.