Validation of satisfaction questionnaire for services provided in the labour rooms of government health facilities

Abstract

Introduction and objectives

Using the assessment of satisfaction of patients of labour room services, the caregiver or policymakers can identify the gaps in the implemented programmes and health policies. This study was aimed to design a valid and reliable satisfaction questionnaire that will help in identifying the aspects of gaps that need improvement.

Methods

A facility and community-based observational cross-sectional study was conducted in the district of western Uttar Pradesh (India) between January 2019 and August 2020. Validation of the structured questionnaire with 34 dichotomous questions categorized under five subscales was performed by evaluating its validity and reliability. A total of 380 beneficiaries were selected from 48 government health facilities.

Results

The calculated Content validity index was calculated to be 9.5 which was adequate as per the guidelines. The reliability analysis of the questionnaire showed that the internal consistency was high with an overall Cronbach's alpha of 0.710. The variation in Cronbach's alpha on the elimination of any question from the questionnaire ranged from 0.676 to 0.767. The mean patient satisfaction score in the total surveyed population was 24.39 ± 4.684 (total score = 34) and there was a greater variation in the satisfaction score of infrastructure when compared with other subscales.

Conclusion

The findings from this study support the reliability and validity of the patient satisfaction questionnaire as it is capable of evaluating the satisfaction in terms of delivery services provided in labour rooms as a whole.

Resumen

Introducción y objetivos

Al evaluar la satisfacción de las usuarias con relación a la atención en las sala de partos, los gestores clínicos pueden identificar los puntos de mejora en los programas y políticas implementados. Este estudio tuvo como objetivo diseñar un cuestionario válido y fiable para evaluar la satisfacción de las usuarias que podría ayudar a identificar los aspectos que necesitan mejorar.

Métodos

Se realizó un estudio observacional de corte transversal en instalaciones y comunidades en el distrito de Uttar Pradesh occidental (India) entre enero de 2019 y agosto de 2020. Se llevó a cabo la validación del cuestionario estructurado con 34 preguntas dicotómicas categorizadas en 5 subescalas mediante la evaluación de su validez y confiabilidad. Se seleccionó a un total de 380 usuarias de 48 establecimientos de salud gubernamentales.

Resultados

El índice de validez de contenido se calculó en 9,5 y fue considerado adecuado según las guías existentes. El análisis de confiabilidad del cuestionario mostró que la consistencia interna era alta, con un alfa de Cronbach general de 0,710. La variación en el alfa de Cronbach al eliminar cualquier pregunta del cuestionario osciló entre 0,676 y 0,767. La media de la puntuación de satisfacción de los pacientes en la población total encuestada fue de 24,39 ± 4,684 y hubo una mayor variación en la puntuación de satisfacción de la infraestructura en comparación con otras subescalas.

Conclusión

Los hallazgos de este estudio apoyan la confiabilidad y validez del cuestionario de satisfacción de las usuarias, ya que es capaz de evaluar la satisfacción en cuanto a los servicios brindados en su conjunto en las salas de partos.

Introduction

A review analysis conducted by World Health Organization (WHO) indicated that despite the availability of evidence-based clinical practice guidelines, there was no guidance on expected standards of care and appropriate measures of quality care. In 2016, WHO proposed Standards for improving the quality of maternal and newborn care in health facilities and it stated that, health care must be efficient, equitable, safe, timely, effective and people-centred.¹

During 2013 in India, Operational Guidelines for Quality Assurance in Public Health were laid down which defined the standards of quality of care to be provided in terms of measurable elements and checkpoints.² The framework for assessing the quality of care was based on the ‘Donabedian model’ and classified quality of care in terms of three aspects – structure, process and outcome. It was in 2016 that further categorization of the quality of care model was done and standard of care was measured in terms of eight areas of concern which were – patient rights, service provision, infection control, inputs, support services, clinical services, quality management and outcome.^3–6

Many guidelines have been laid yet little heed has been paid to assess the satisfaction of the beneficiaries i.e. mothers and neonates. Are the beneficiaries receiving the services or are they satisfied with the services being provided? These are the questions that need answers if we want to assess the standard of care being provided.

Assessment of beneficiaries satisfaction allows the caregiver to investigate the extent to which their services meets the need of the beneficiaries. If a questionnaire assesses satisfaction about various components of the services provided, it will enable the caregiver or policymakers to identify the aspects of services where patients are less satisfied and there is a scope of potential improvement.

Baker et al.⁷ argued that the patient satisfaction scale must fulfil three requirements: it must be reliable, valid and show transferability. Hence, this study aims to describe the validation of a new questionnaire which shall assess the satisfaction of beneficiaries who have availed delivery service in the labour rooms of government health facilities. Provided that the questionnaire is valid and reliable, the questionnaire shall be used in identifying gaps in terms of labour room services.

Material and methods

Design and study population

A community and facility-based cross-sectional study was conducted in a district of Uttar Pradesh, state between January 2019 and August 2020. The study was conducted in all the delivery point where at least 10 deliveries in a month were being conducted. A total of 52 health facilities were selected. To calculate the sample size for beneficiaries to be surveyed, the prevalence of institutional delivery in the study district was used which was 57.2%. By using the value of Z_1−α/2 as 1.96 at 5% type I error, the sample size was calculated using the following formula:

The formula gave the value of 377 beneficiaries and to achieve a minimum sample size of 377, eight beneficiaries per facility were surveyed from 52 health facilities. Hence 416 beneficiaries who delivered their newborn through normal vaginal delivery, were to be surveyed to assess their satisfaction in terms of delivery services provided in the labour rooms. The survey of the selected beneficiaries was taken by two Post Graduate students of Community Medicine who were briefed about the study.

Development of the questionnaire

An Extensive literature review was performed to identify and study the content available which would aid in assessing the beneficiary satisfaction in terms of the services provided in labour rooms. A systematic search was conducted using the following search terms: (Delivery OR Birth OR labour) AND (Satisfaction OR Perception OR Evaluation). The questionnaire was developed by assessing the guidelines available in various literature (WHO-Standards for Improving Quality of Maternal and Newborn Care in Health Facilities, Labour Room Quality Improvement Initiative (LaQshya), Maternal and Newborn Health Toolkit, Guidelines for Standardization of Labour Rooms At Delivery Points, DAKSHTA).^1,3–6 Domains of structure, process and outcome were identified through the literature search. A group discussion of two staff nurses along and a lady medical officer was conducted to draft questions under the identified domains. Initial questionnaire consisted of 32 questions which was then opinionated by two experts – Professors from Department of Community Medicine and Obstetrics & Gynaecology, who contributed their knowledge on practical aspects. Two questions were broken down into four, making a total of 34 questions in the final draft.

The questionnaire was divided into four sections among which the first section included data on the socio-demographic profile of the beneficiary. The Second section collected data of mother and neonate including parity, weight of the baby at birth and gender. Third section collected data on time and day of hospital admission while the fourth section contained 34 dichotomous questions which were categorized under subscales of health facility infrastructure, health personnel behaviour, services provided by health personnel, expenses faced and outcome of the delivery in terms of maternal and newborn health. Equal weightage was given to all 34 questions and response score of 1 & 0 was imparted to the dichotomous questions where a score of 1 represented a satisfying response while 0 represented an unsatisfied response. Greater the score, greater was the satisfaction of the beneficiaries. The questionnaire was translated into the mother tongue i.e. in Hindi. To ensure that both the questionnaires in Hindi and English were conceptually equivalent, forward translation and back translation methods were implemented.

Methodology

According to the list provided by the Chief Medical Officer of the district, health facilities that were conducting vaginal deliveries included one medical college, one district hospital, seven Community Heath Centres (CHCs), five Primary Health Centres (PHCs) and 52 sub-centres making a total of 66 health facilities. Out of these, 52 health facilities were chosen for the study as they conducted more than 10 vaginal deliveries in the past six months. The facilities chosen for the study purpose comprised of one medical college, one district hospital, seven CHSc, five PHCs and 38 sub- centres. Among the 52 facilities, 48 were observed as in the remaining facilities deliveries were not being conducted.

Based on the calculation of sample size, eight beneficiaries per facility were surveyed. For selecting the beneficiaries, a detailed list of all mothers along with their contact number, address and accompanying Accredited Social Health Activist (ASHA), who delivered at the facility in the past six months from the time of data collection was obtained from the facility in-charge. From the list obtained, eight beneficiaries who delivered most recently were selected to prevent recall bias. To provide an unbiased response the beneficiary was surveyed at her house itself. To do so, the beneficiary was first contacted telephonically and verbal consent was taken. Then with the help of ASHA, the house of the selected beneficiary was located and the survey was conducted. If the beneficiary was unavailable at her house for two occasions then the next beneficiary from the list was selected.

Data analysis

Face validity

Face validity involves scrutiny of all the items in the questionnaire to establish that they are a valid measure of the concept. It evaluates the appearance of the questionnaire regarding feasibility, readability, consistency of style, formatting and the clarity of the language used.^7,8 Face validity of the questionnaire was assessed by two public health experts with the help of an evaluation form. The questionnaire was assessed for clarity, ease of comprehension and layout.

Content validity

The content validity depicts how well an instrument measures the construct that it is intend to measure. It is typically achieved by a thorough analysis of the instrument by experts familiar with the construct of interest or experts on the research subject.⁹ The content validity was illustrated in the form of the Content Validity Index (CVI) by two public health experts who were invited to be a part of the review panel to review the questionnaire for content validity. The questionnaire along with the review form with detailed instruction was provided to them in person. Each question was assessed in terms of its relevance with the research question and graded accordingly from 1 to 4 where, ‘1’ represented the question being not relevant while ‘4’ represented high relevance to the research question. The relevance rating was recoded as 1 (relevance scale of 3 or 4) and 0 (relevance scale of 1 or 2). The cumulated level of agreement among the experts was averaged and assigned a numerical value known as the content validity index (CVI). As recommended by Sangoseni, a value of at least 0.78 was considered acceptable when two experts were reviewing the questionnaire for content validity.¹⁰

Construct validity

To test construct validity, the correlation was assessed between scores on each specific subscale and overall satisfaction score through Pearson's correlation test.¹¹

Reliability

Internal consistency was tested by the calculation of Cronbach's alpha, whose values between 0.70 and 0.90 were considered to be acceptable.¹² As the questionnaire had a dichotomous response, the internal consistency reliability should have been tested according to Kuder and Richardson (K R 20) coefficient. We considered the values of Cronbach's alpha, as it as a generalization of coefficient introduced by Kuder Richardson, while KR20 is simply a special case of alpha and it has the same interpretation as alpha.¹³

Results

Based on the sample size calculation, eight beneficiaries per health facility were to be surveyed. However, among the 48 health facilities observed, 380 beneficiaries were surveyed. Table 1 presents the socio-economic, demographic and pregnancy-related characteristics of the beneficiaries. The mean age of the beneficiaries surveyed was 24.8 with 51.8% below the age of 25 years. Literacy level was low as one-fifth of the beneficiaries were illiterate while 43% were educated up to class eighth. More than four-fifth of the beneficiaries belonged to the socially vulnerable class among which 45% belonged to Other Backward Castes (OBC) while 42.1% belonged to Scheduled Castes and Scheduled Tribes (SC/ST). Most women (99.5%) were housewives with the majority (46%) living as a nuclear family (Table 1).

Table 1.

Socio- demographic profile of the beneficiaries (N = 380).

Variables	Categories	Frequency n (%)
		(N = 380)
Age	Mean (S.D.)	24.8 ± 2.8
	<25 years	197 (51.8)
	≥25 years	183 (48.2)
Education	Illiterate	80 (21.1)
	Up-to 8th grade	164 (43.0)
	High school	61 (16.1)
	Intermediate	66 (17.4)
	Graduate/postgraduate	9 (2.4)
Caste	General	49 (12.9)
	Other backward caste	171 (45.0)
	Schedule cast/tribes	160 (42.1)
Occupation	Housewife	378 (99.5)
	Earns salary	2 (0.5)
Family type	Nuclear	175 (46.1)
	Joint	111 (29.2)
	Three generation	94 (24.7)
Socio-Economic Status (SES)	Class I	1 (0.3)
	Class II	32 (8.4)
	Class III	149 (39.2)
	Class IV	191 (50.3)
	Class V	7 (1.8)
Parity	Primi parous	80 (21.1)
	Multiparous	279 (73.4)
	Grand multiparous	21 (5.5)
Gender of baby	Male	172 (45.3)
	Female	208 (54.7)
Weight of baby	<2.5 kg	152 (40.0)
	≥2.5 kg	228 (60.0)
Transport used to reach the facility	Government aided vehicle	56 (14.7)
	Personal vehicle	324 (85.3)
Day of admission at the facility	Monday–Friday	354 (93.2)
	Saturday–Sunday	26 (6.8)
Time of admission at the facility	10 a.m. to 4 p.m.	353 (92.9)
	4 p.m. to 10 a.m.	27 (7.1)
Person who conducted the delivery	A.N.M.	293 (77.1)
	Staff nurse	77 (20.3)
	Doctor	10 (2.6)

The panel of experts reviewed each of the 34 questions and reported that the questions were easily comprehensible. The language of the questions was clear and would be acceptable to the target audience. The experts suggested that the layout and appearance of the questionnaire was acceptable. After assessing the face validity, all 34 questions were forwarded for evaluating the content validity.

The panel of experts reviewed the 34 questions on beneficiary satisfaction and rated them on a relevance scale from 1 to 4. Based on the reviews from expert panel, the calculated Content validity index was 9.5 which was adequate as per the guidelines according to Davis.

To test the construct validity of the subscales, scores were correlated with the overall satisfaction scores. Scores of the overall satisfaction showed a significant positive correlation with the scores from all subscales (Table 2). Increase in scores of subscales will lead ultimately lead to a greater overall score implying a greater satisfaction.

Table 2.

Correlation between subscales and overall satisfaction.

Subscale	r	p value
Infrastructure	0.883	0.001
Health personnel behaviour	0.470	0.001
Services provided by health personnel	0.832	0.001
Expenses incurred	0.512	0.001
Outcome of the delivery	0.316	0.001

The reliability analysis of the questionnaire showed that the internal consistency was high with an overall Cronbach's alpha of 0.710 (Table 3). When the questions under infrastructure subscale were evaluated, an acceptable Cronbach's alpha of 0.760 was reported while for the rest of the subscales the Cronbach's alpha did not fall in the acceptable range i.e. it ranged from 0.212 to 0.417 (Table 3). The variation in Cronbach's alpha on the elimination of any question from the questionnaire ranged from 0.676 to 0.767 (Tables 4 and 5). The mean of patient satisfaction score in the total surveyed population was 24.39 ± 4.684 (total score = 34) and there was a greater variation in the satisfaction score of infrastructure when compared with other subscales (Table 3).

Table 3.

Reliability analysis for the subscales.

Subscales	Mean	S.D.	α
Infrastructure	5.58	2.339	0.760
Health personnel behaviour	4.23	0.832	0.282
Expenses incurred	2.40	0.995	0.471
Outcome of the delivery	1.78	0.450	0.212
Total survey	24.39	4.648	0.710

Table 4.

Analysis of internal consistency per variable of the beneficiary satisfaction questionnaire.

S.no.	Variable	α on removing the question
D1	Proper privacy was maintained during examination	.695
D2	Proper privacy was maintained during delivery	.704
D3	Proper privacy was maintained during breastfeeding	.686
D4	Delivery was conducted in a labour table	.709
D5	Labour table was clean	.692
D6	Labour room premises were clean	.682
D7	Was any working source of light present in the labour room	.685
D8	Was any working source of Fan/air condition present in the labour room	.685
D9	Was drinking water available at the health facility	.676
E1	I was attended by a healthcare provider at the health facility promptly	.700
E2	I was verbally abused by the healthcare provider at the health facility	.707
E3	I was physically abused by the healthcare provider at the health facility	.710
E4	I was allowed to be accompanied by a family member inside the labour room	.710
E5	I was neglected/abandoned by the healthcare provider during delivery	.705
F1	I was informed prior to every vaginal examination	.706
F2	I was informed about the status of my labour after examination	.721
F3	Enema was administered prior to the delivery	.713
F4	I was given the choice of position for delivery	.714
F5	Episiotomy was given while performing delivery	.718
F6	I was informed about all necessary procedures performed during labour & childbirth	.707
F7	Immediate skin to skin contact was made with the baby when the baby was delivered	.702
F8	I was informed about the status of my baby soon after birth	.714
F9	I was informed about the gender of my baby soon after birth	.699
F10	I was informed about the weight of my baby soon after birth	.710
F11	I was informed to breastfeed my newborn baby soon after delivery	.690
F12	Assisted to breastfeed the newborn soon after delivery	.692
F13	Received sanitary pads after delivery	.689
F14	Did the newborn receive three vaccine before discharge from the facility	.678
G1	I was charged for the ambulance service	.711
G2	I was made to buy drugs and consumables from outside the facility	.702
G3	I was asked for money to avail services at the health facility	.691
G4	Received monetary incentives for institutional birth	.690
H1	Were you in good health at the time of discharge from the facility	.696
H2	Was the newborn in good health at the time of discharge from the facility	.697

Table 5.

Descriptive data (mean and standard deviation) for the total participants.

Variable	Mean	S.D.
	(N = 380)
Proper privacy was maintained during examination	.75	.432
Proper privacy was maintained during delivery	.92	.270
Proper privacy was maintained during breastfeeding	.61	.489
Delivery was conducted in a labour table	.93	.257
Labour table was clean	.61	.488
Labour room premises were clean	.48	.500
Was any working source of light present in the labour room	.51	.500
Was any working source of Fan/air condition present in the labour room	.39	.488
Was drinking water available at the health facility	.37	.484
I was attended by a healthcare provider at the health facility promptly	.56	.498
I was verbally abused by the healthcare provider at the health facility	.94	.234
I was physically abused by the healthcare provider at the health facility	.98	.135
I was allowed to be accompanied by a family member inside the labour room	.87	.338
I was neglected/abandoned by the healthcare provider during delivery	.89	.317
I was informed prior to every vaginal examination	.13	.333
I was informed about the status of my labour after examination	.61	.489
Enema was administered prior to the delivery	.96	.207
I was given the choice of position for delivery	.05	.224
Episiotomy was given while performing delivery	.93	.253
I was informed about all necessary procedures performed during labour & childbirth	.11	.314
Immediate skin to skin contact was made with the baby when the baby was delivered	.55	.498
I was informed about the status of my baby soon after birth	.95	.224
I was informed about the gender of my baby soon after birth	.52	.500
I was informed about the weight of my baby soon after birth	.68	.468
I was informed to breastfeed my newborn baby soon after delivery	.62	.485
Assisted to breastfeed the newborn soon after delivery	.26	.437
Received sanitary pads after delivery	.36	.480
Did the newborn receive three vaccine before discharge from the facility	.26	.441
I was charged for the ambulance service	.14	.347
I was made to buy drugs and consumables from outside the facility	.60	.491
I was asked for money to avail services at the health facility	.87	.336
Received monetary incentives for institutional birth	.79	.408
Were you in good health at the time of discharge from the facility	.97	.160
Was the newborn in good health at the time of discharge from the facility	.81	.394

Discussion

The main aim of this study was to test the validity and reliability of the beneficiary satisfaction questionnaire. The results show that the questionnaire satisfy the criteria of Baker for an adequate questionnaire and the questionnaire was internally reliable and construct valid when tested on the beneficiaries. Scores of each subscale correlated significantly with the overall satisfaction suggesting that each subscale measures some aspect of patient satisfaction. Analysis revealed that each of the questions were internally reliable suggesting that the questions were related and correctly interpreted the satisfaction of beneficiaries. The questionnaire also fulfils the criteria of content validation as the Content Validity Index fulfilled the acceptable level (Fig. 1).

Figure 1.

Drafting of the questionnaire.

Though the overall reliability of the questionnaire was acceptable but individual reliability of subscales was unacceptable owing to a low Cronbach's alpha. A low value of alpha could be due to varied number of reasons like less number of questions, poor inter-relatedness or heterogenous constructs. But as the overall satisfaction showed a significant positive correlation with all the subscales, it is possible the low alpha value was due to less number of questions in the subscales. As none of the parameters when removed changed alpha significantly, all the parameters were chosen to be kept in the final questionnaire. Construct validity supported the fact that the questionnaire will measure the satisfaction of the beneficiaries.

The findings from this study support the reliability and validity of the patient satisfaction questionnaire although, further research could assess other aspects of reliability and validity. The questionnaire is capable of evaluating the satisfaction in terms of delivery services provided in labour rooms as a whole however, modifications need to be made to evaluate the satisfaction at subscale level by adding few parameters to improve the reliability of individual subscales.

The designed questionnaire covers all the aspects of the childbirth process – the input, the process and the outcome. The questionnaire has been designed in the mother tongue of the people being surveyed and care has been taken to ensure that various technical/medical terms have been incorporated in the local dialect in the questionnaire. Being in local language and dialect, in case of shortage of human resource for carrying on a survey the beneficiary, the questionnaire can be self-administered. The questionnaire was tested over a large sample size throughput the district. Thus any geographical bias that could have crept in the sample has been minimized.

The questionnaire used has been designed on the basis of inputs of doctors, healthcare professionals, paramedical staff etc. However a more comprehensive and well-rounded questionnaire would have been one where the perspective of the beneficiaries is taken into account. The beneficiaries of the private hospitals can be included which can then lead to a more representative sample.

Conclusion

To assess the effectiveness of any system, any criteria needs to focus on not only the inputs, the processes but also the outcomes. A crucial thing to note is that by including processes this includes not only the tangibles but also the intangibles and by including the outcomes it is able to focus on the total experience.

This study intends to fulfil the above criteria and aims to focus on assessing the quality of healthcare from all the above three aspects – inputs, processes and outcomes by creating a validated tool for assessing beneficiary satisfaction. There have been studies which overemphasize on the infrastructure component and thus miss out on the processes which are equally important. Similarly studies tend to focus more on the output than the outcomes. Including the outcomes would help to ascertain the quality of healthcare being provided and to incorporate the experience of the beneficiaries. These can then serve as an important feedback for improving the inputs and processes.

Conflicts of interest

The authors declare that they have no conflict of interest.