A validity and reliability study of Parkinson Disease Sleep Scale2 (PDSS-2) in Parkinson disease patient with sleep disorder

Abstract

Objectives

This study aims to measure the level of validity and reliability of PDSS-2 in Parkinson's disease with sleep disorders.

Methods

The study was conducted on Parkinson's patients with sleep disorders that the inclusion criteria are women or men aged >40 years, diagnosed with Parkinson's disease. The PDSS-2 score is used to assess sleep disorders. Assessment is done twice with an interval of examination 10 min. Then measure the Pearson correlation test between two examinations (validity test) and reliability by calculating the Cronbach's alpha PDSS-2 value, and inter-rater reliability to determine internal consistency.

Results

The study subjects were 15 PD patients, range age between 51 and 75 years (63.0 ± 8.95). Consist of 11 men and four women. The validity test value on the two examinations showed 9 of 15 items (60% validity) and 7 of 15 items (46.6% validity). The second reliability test was declared reliable with each value of Cronbach alpha 0.833 and 0.644 (r table value = 0.60), with inter-rater reliability value is 0.706.

Conclusion

The PDSS-2 score is reliable in assessing sleep disorders in Parkinson's disease with internal consistency 0.706, and the validity value is 80%.

Introduction

Parkinson's disease (PD) is the second most common neurodegenerative disorder in the world. In China, approximately 48–89% of Chinese patients with PD are affected by sleep disorders. In recent decades, there have been significant advances in our understanding of the relationship between sleep disorders and PD, yet many questions remain unanswered.¹

Sleep disorder is a non-motor symptom often found in Parkinson's disease, with a prevalence of about 40–90%. It can occur at an advanced stage of Parkinson's disease also precede motor symptoms.² Sleep disorders often occur in PD patients but often undetected due to historical identification incomplete due to focus on motor manifestations, as well as lack of self-reporting. Sleep disorders have a severe impact on health, general well-being, and quality of life. Sleep problems in PD have many potential causes including the direct effects of PD itself, side effects of anti-Parkinson's drugs, sleep disorder during the day, age-related, and other comorbidities.³

Clinical evaluation not accurate enough to diagnose sleep disorders. Some assessment tools have been developed to help detect and manage sleep problems in PD patients. Polysomnography (PSG) is considered a gold standard to diagnose and evaluate the severity of nocturnal sleep disorders. The symptoms of nocturnal psychosis and nocturia are PDSS items that have been reported to correlate with several PSG parameters in patients with PD.⁴

Several scales have been proposed to assess sleep disorders in PD patients. More recently, the Movement Disorder Society has assigned a special division to examine some sleep disturbance assessment scales and evaluate their application in PD patients.⁵ Only a few scales are appropriate for the PD population. Specifically, Parkinson's Disease Sleep Scale (PDSS) has been proposed as a sleep disorder screening tool and measures the severity of sleep disorders in PD. However, PDSS has been found inadequate as a screening tool for specific sleep disorders in PD, such as sleep apnea, REM behavior disorder (RBD), and restless leg syndrome (RLS). Based on these limitations and the need for sleep disorder measurement tools in PD, a modified version of PDSS (PDSS-2) was established and published in 2011. PDSS-2 was developed to assess nocturnal disorders.⁶

Based on the background, the researcher will measure a validity and reliability study of the PDSS-2 scale to apply well in PD patients in Indonesia. Cross-cultural adaptation aims to assess whether the scale can be understood by patients in their countries, as well as the absence of the results of the validity and reliability study of the scale (PDSS-2) conducted in Indonesia behind this study. Besides, previous similar studies used the test–retest method, but this study has used the inter-rater reliability method.⁷

Method

This study was an observational study by a cross-sectional design with a simple random sampling method to calculate the level of agreement between two examiners (inter-rater reliability) in assessing patient understanding in filling out research instruments. This research was conducted at Neurologic outpatient clinic RSUP Dr. Wahidin Sudirohusodo, and the Education Network Hospital in Makassar. It used the Indonesian version of the PDSS-2 questionnaire. The study population was PD patients who visited the neurologic outpatient clinic in those hospitals. The inclusion criteria are male or female who has been diagnosed with PP, can understand in the Indonesian language and agree to participate in this research by signing an informed-consent, while the exclusion criteria are patients with comorbid conditions that can hinder the assessment of PD adequately (such as blindness, amputation both limbs, state of severe systemic disorders), diagnosed with dementia, and experiencing psychiatric disorders that cause inability to understand and complete a self-assessment questionnaire. The number of samples needed in this study uses a small sample size based on the calculation of outpatient sample visits in several teaching hospitals, which is 15 samples.

After the data obtained, a validity test uses the r count criteria based on the r table, and the reliability test uses Cronbach alpha (α) statistical study on the questionnaire items of each examination. An analysis of level agreement between items of two examinations based on the Kappa Cohen statistics is calculated using SPSS 20.0 computer program. The Kappa Cohen agreement coefficient interpretation uses Landis and Koch's instructions.

Results

In May 2019, a PDSS-2 validity and reliability study was conducted in Parkinson's disease patient with sleep disorders. It was conducted at Wahidin Sudirohusodo Hospital and Network Education Hospital in Makassar. The number of subjects was 15 peoples with PD who fulfill the inclusion criteria.

The study used an Indonesian version of the PDSS-2 questionnaire. The instrument filling was guided and filled out by researchers and colleagues by conducting interviews with subjects. A total of 15 questions evaluated the subject's sleep habits so that the quality of sleep in the last week was known. The questionnaire was filled in 2 times, the duration of the examination about 30 min, at one time, and 10 min of interval.

The results of this questionnaire obtained a total PDSS-2 score with a maximum value of 60 points, and a higher score means more sleep disturbance at night. Furthermore, a cutoff value of ≥18 distinguishes poor sleep quality (≥18) and good (<18) in clinical practice.

Validity test extent of the accuracy and measuring instrument (questionnaire) in carrying out its measurement function. The measurement criteria used are r count greater than the r table (0.482). The validity test of PDSS-2 items in two examinations is shown in Table 1.

Table 1.

PDSS-2 validity test.

Analysis factors	Corrected item total correlation		r table	Conclusion
	1^st Exam.	2^nd Exam.		1^st Exam.	2^nd Exam.
Item 1	0.664	0.598	0.482	Valid	Valid
Item 2	0.321	0.479	0. 482	Not valid	Not valid
Item 3	0.331	0.468	0. 482	Not valid	Not valid
Item 4	0.846	0.800	0. 482	Valid	Valid
Item 5	0.840	0.750	0. 482	Valid	Valid
Item 6	0.668	0.593	0. 482	Valid	Valid
Item 7	0.507	0.507	0. 482	Not valid	Valid
Item 8	0.459	0.148	0. 482	Not valid	Not valid
Item 9	0.643	0.582	0. 482	Valid	Valid
Item 10	0.841	0.295	0. 482	Valid	Not valid
Item 11	0.875	0.546	0. 482	Valid	Valid
Item 12	0.474	0.438	0. 482	Valid	Not valid
Item 13	−0.132	0.050	0. 482	Not valid	Not valid
Item 14	0.212	0.097	0. 482	Not Valid	Not valid
Item 15	0.514	0.001	0. 482	Valid	Not valid

Based on the table, explained that from a total of 15 questionnaire items, there were seven items invalid on the results of the first examination and there were eight items were invalid on the results of the second examination, with r count values smaller than r tables (0.482).

The reliability test was carried out to measure the questionnaire, which is an indicator of the variable. The statistical test used is Cronbach alpha (α), declared reliable if it has a Cronbach alpha of more than 0.60 (>0.60). The PDSS-2 item reliability test at two checks is shown in Table 2.

Table 2.

PDSS-2 reliability test.

Analysis factor	Cronbach's alpha		Critical value	Conclusion
	1^st Exam.	2^nd Exam.
Items questioner	0.833	0.644	0.600	Reliable

Based on the description table above show that each examination result has a Cronbach alpha >0.60. Thus the variable is declared reliable on two examinations.

Furthermore, an agreement (association) test was conducted between two examinations of each PDSS-2 item stated in the Kappa coefficient using Landis and Koch's instructions. Based on the interpretation of the PDSS-2 Kappa coefficient per item, the significance value and the level of an agreement are varied. The results of the agreement test are shown in Table 3.

Table 3.

Agreement test between two examinations of PDSS-2 items.

Analysis factors	p value (p)	Kappa value (k)
Item 1	0.000	0.556
Item 2	0.001	0.464
Item 3	0.001	0.528
Item 4	0.035	0.298
Item 5	0.004	0.371
Item 6	0.194	0.224
Item 7	0.186	0.196
Item 8	0.000	0.659
Item 9	0.023	0.318
Item 10	0.023	0.250
Item 11	0.717	0.046
Item 12	0.032	0.423
Item 13	0.015	0.360
Item 14	0.001	0.586
Item 15	0.612	0.063

Based on the cross-tabulation, it shows the Kappa coefficient value is 0.706, with a significance of 0.004 (Table 4). Based on the interpretation of the Kappa agreement coefficient, PDSS-2 has a good agreement value (0.61 < K < 0.80) (Fig. 1).

Table 4.

Cross tabulation of sleep quality based on PDSS-2 in two examinations.

	2^nd examination		Total
	Good	Poor
1^stexamination
Good	4	0	4
Poor	2	9	11
Total	6	9	15
	k = 0.706		p = 0.004

Fig. 1

Discussion

The results showed that Kappa coefficient value or level of agreement various between two examinations. It can be affected by several factors, such as instrument factors, subject factors, and examiners who make an assessment. In this study, the PDSS-2 firstly translated into the Indonesian language.

Instrument factors contain some questions about symptoms that some of the items are almost the same can affect the subject factors of the accumulation show different results due to the process of subject adaptation, which is indicated boring of the questions made repeatedly. Instrument blur in this study can be caused by this instrument being translated by non-medical linguists. These things can cause narrative differences in perceptions between the two examinations. Also, the number of research subjects is considered to influence the results of the validity and reliability study. The more research subjects, the level of validity can be illustrated.

Complete PDSS-2 questionnaire based on the examiner, it means filling out the questionnaire is done by interviewing the subject or guardian or his bedmate by the examiner and the questionnaire filled out by the examiner. Besides, the examiner factor also has a role in the level of agreement by the examiner's ability to provide an understanding of the questions to the subject so that they can provide accurate answers. Other blur information relating to examiners such as communication skills, time limitation, and physical fatigue.

Based on the cross-tabulation showed the sleep quality of excellent and poor PD patients with a Kappa coefficient 0.706 (p < 0.004) and interpretation of the good agreement. However, the Kappa coefficient study on instrument items, the results are varied. It also seen in the different validity values between two examinations and the higher reliability values at the first examination. If a correction made to an item that is declared invalid by eliminating or by replacing the problem, then it can increase the reliability value. However, it was not done in this study because this research was research on converting language to Indonesian, but it was not done in this study.

Various attempts have been made to reduce blur as optimal as possible. The blur of the examiner is reduced by repeated explanation and discussion of the difficulties encountered during the examination. Limitation of the number of samples examined so that the examiner does not experience fatigue. Explaining the purpose and objectives of the study to the sample will also reduce the blur. Instrument blur can be reduced by translating the PDSS-2 scale into the Indonesian language by Indonesian and English experts, then re-translating the PDSS-2 scale in Indonesian into the original language, which is English by different translators, and compared to the PDSS-2 scale original. If there are differences, the words in the Indonesian version of the PDSS-2 scale can be corrected so that the statements in the Indonesian version of the PDSS-2 scale resemble the original language without reducing meaning or misinterpretation in the Indonesian language.

Conclusion

The PDSS-2 Scale was declared valid and reliable to use as an instrument for assessing sleep disorders in PD patients with sleep disorders in Indonesia.

Conflict of interest

The authors declare no conflict of interest.

Acknowledgments

Praise the authors pray to Allah SWT for the blessings, gifts, and guidance so that the author can complete this study. On this occasion, allow the authors thankfully and maximum appreciation to our colleagues in the Department of Neurology whom great patience and attention have provided encouragement, guidance, and advice as long as the authors carry out this study.