Validation of Neuromyotype: a smart keyboard for the evaluation of spinal muscular atrophy patients

Abstract

Introduction

Spinal muscular atrophy 5q (SMA) is a genetic neurodegenerative disease that affects alpha motor neurons producing progressive weakness. New outcome measures are currently required to accurately characterise the disease progression and the efficacy of new available treatments. The objective of this work is to preliminarily validate a new intelligent keyboard (Neuromyotype) measuring typing strength and speed in patients with SMA.

Material and methods

Twenty two SMA patients older than 15 years, and 26 healthy controls were included. Three measurements were obtained with the keyboard (maximum strength, execution time of a random typing task, execution time of a sequential typing task) together with the time to complete the Nine-Hole Peg Test (9HPT). Patients were also administered motor (Hammersmith Functional Motor Scale Expanded, HFMSE; Revised Upper Limb module, RULM), and functional scales (Egen Klassification, EK2; and the revised version of Amyotrophic Lateral Sclerosis Functional Rating Scale, ALSFRS-R). The viability and construct validity of the Neuromyotype were analysed, measuring the discriminative power between patients and controls (using ROC curves and the Bangdiwala's B statistic), between the different functional types of SMA (walker, sitter and non-sitter) and their correlation with the rest of motor scales.

Results

Neuromyotype measurements could be performed in all patients, unlike the rest of the scales. Its administration was quick and easy. The 3 variables on the keyboard discriminated very well between patients and controls, with strength (ROC = 0.963) being the one that best differentiates from the 3, equaling 9HPT (ROC = 0.966). They also showed a good ability to differentiate by functional type (especially non-sitters from sitters and walkers), with sequential time (B = 0.83) being the tool that best discriminates between the three groups above the rest of motor scales. All motor and functional scales showed strong or very strong correlations with each other (rs = 0.71–0.99), with strength correlating better with motor scales and timed variables with functional scales.

Conclusion

This study shows the feasibility and validity of Neuromyotype for the evaluation of adolescent and adult patients with SMA. Data obtained with this tool could be of great clinical relevance, saving time and resources compared to the rest of the scales.

Resumen

Introducción

La atrofia muscular espinal 5q (AME) es una enfermedad genética neurodegenerativa que afecta a las motoneuronas alfa produciendo debilidad progresiva. Actualmente se precisan nuevas herramientas que permitan medir y caracterizar con precisión la progresión de la enfermedad y la eficacia de los nuevos tratamientos. El objetivo del presente trabajo es realizar la validación preliminar de un nuevo teclado inteligente (Neuromyotype) capaz de medir la fuerza y velocidad de tecleo en pacientes con AME.

Material y métodos

Se incluyeron 22 pacientes mayores de 15 años con AME y 26 controles sanos. En ellos se realizó tres mediciones con el teclado (fuerza máxima, tiempo de ejecución de una tarea aleatoria de tecleo, tiempo de ejecución de una tarea secuencial de tecleo) y el tiempo para completar el nine hole peg test (9HPT). A los pacientes se les administró además unas escalas motoras (Hammersmith Functional Motor Scale Expanded [HFMSE]; Revised Upper Limb module [RULM]), y funcionales (Egen Klassification [EK2]; y la versión revisada de Amyotrophic Lateral Sclerosis Functional Rating Scale [ALSFRS-R]). Se analizó la viabilidad y validez de constructo de Neuromyotype, midiendo el poder discriminativo entre pacientes y controles (mediante curvas ROC y el estadístico de Bangdiwala) y entre los distintos tipos funcionales de AME (walker, sitter y non-sitter) y su correlación con el resto de escalas motoras.

Resultados

Las mediciones del teclado se pudieron realizar en todos los pacientes, al contrario que el resto de las escalas. Su administración fue rápida y sencilla. Las 3 variables del teclado discriminaron muy bien entre pacientes y controles, siendo la fuerza (ROC = 0.963) la que mejor diferencia de las 3, igualando al 9HPT (ROC = 0.966). También mostraron una buena capacidad para diferenciar por tipo funcional (especialmente a los non-sitter de los sitter y walker), siendo el tiempo secuencial (B = 0.83) la herramienta que mejor discriminaba entre los tres grupos por encima del resto de escalas motoras. Todas las escalas motoras y funcionales mostraron correlaciones fuertes o muy fuertes entre sí (rs = 0.71–0.99), con la fuerza correlacionando mejor con escalas motoras y las variables de tiempo con las escalas funcionales.

Conclusión

Este estudio demuestra la viabilidad y validez de Neuromyotype para la valoración de pacientes adolescentes y adultos con AME. Los datos obtenidos con esta herramienta podrían ser de gran relevancia clínica, ahorrando tiempo y recursos en comparación con el resto de las escalas.

Introduction

Spinal muscular atrophy (SMA) linked to chromosome 5q is a genetic neurodegenerative disease that leads to a progressive loss of muscle strength due to a gradual destruction of the lower motor neurons. This condition follows an autosomal recessive inheritance pattern; its diagnosis is based on the detection of a biallelic mutation of the SMN1 gene, which is responsible for producing the survival of motor neuron (SMN) protein. According to the maximum motor milestone achieved and the age of symptom onset, SMA is classified into types 1-4; according to current functional status, patients are classified as walkers, sitters, and non-sitters.¹

Since 2016, 3 disease-modifying therapies have been developed: nusinersen (or Spinraza™), an anti-sense oligonucleotide administered by lumbar puncture that favours the production of the SMN protein by the SMN2 gene²; onasemnogene abeparvovec (Zolgensma™), the first gene therapy, which uses the adeno-associated virus serotype 9 (AAV9) vector to deliver copies of the SMN1 gene to the cells of the treated children³; and lastly, risdiplan (Evrysdi™), a nanomolecule administered orally that presents similar action to that of nusinersen.³

One significant characteristic of SMA is its great clinical heterogeneity (both in age of onset and in the degree of functional disability). This has led to the use or development of many different scales, most of which are motor scales based on a series of tasks that patients perform in the gym.⁴ However, the scales used to date present certain limitations, such as the need for human and material resources that are not available at all centres, or their design and validation for paediatric patients, limiting their usefulness in the adult population.⁵ Furthermore, most of the scales may only be used in certain subgroups of patients, frequently leaving aside those patients with poorer functional status.^1,5

Lastly, most of the scales are unable to detect changes over periods shorter than 2 years, particularly in adults.⁴

Therefore, there is a need for new tools to accurately measure and characterise the natural history of SMA and/or the efficacy of these new treatments, especially in adult patients.^1,5 These tools should ideally be applicable to the greatest possible number of patients, and be simple, quick to administer, and reliable. Two devices have recently been developed, MyoGrip® and MyoPinch®, which measure grip and pinch strength, respectively. These devices have been tested in longitudinal trials with patients older than 6 years,^6–8 showing their construct validity and sensitivity to detect changes; the results suggest that the measurement of hand strength is a useful marker to assess progression or treatment response.

The aim of our study is to perform a preliminary validation of a new smart keyboard (Neuromyotype) that assesses strength and typing speed in patients with SMA.

Methods

Study design

This is a cross-sectional study performed between May 2020 and June 2021.

We consecutively recruited patients older than 15 years and with a genetic diagnosis of SMA who visited the outpatient clinics at Hospital la Fe. We excluded those who were unable to press the key with the index finger.

We also recruited age- and sex-matched controls older than 15 years, excluding those with diseases affecting the hands that may prevent or hinder performance of the study tasks.

Variables

On the day of the visit, we collected demographic variables (age and sex) and administered the Nine-Hole Peg Test (9HPT) to both patients and controls. We also recorded the type of SMA (1-3) and the functional status of the patients (walker, sitter, non-sitter) and administered a series of functional and motor scales, validated for these patients and used in everyday clinical practice.⁵ Functional scales use a series of items to measure patients’ functional status in different domains (motor, respiratory, bulbar function, etc.). In this study, we administered the Amyotrophic Lateral Sclerosis Functional Rating Scale Revised (ALSFRS-R), with scores ranging from 0 to 48, and the Egen Klassifikation 2 (EK2) scale. The motor scales used included the Hammersmith Functional Motor Scale Expanded (HFMSE), which scores overall motor function on a scale from 0 to 66, and the Revised Upper Limb Module (RULM), which measures the motor function of the upper limb on a scale from 0 to 37.

Lastly, we gathered the keyboard variables: strength and execution time. Strength refers to the maximum strength in grammes used by the subject to press key 14, which was illuminated for 5 seconds. The keyboard is equipped with a sensor that records strength from 50 to 2950 grammes. We first measured the dominant hand and subsequently the non-dominant hand. Finally, we calculated the mean for both hands.

Execution time was defined as the time taken by the individual to press a sequence of keys that were illuminated in a sequential (sequential time) or random (random time) order. The key was illuminated for a maximum duration of 4 seconds. Once this time had elapsed, the next key was activated, even if the user was not able to press the key with sufficient strength (Fig. 1). Therefore, if the user is not able to press with enough strength, the test continues to the end. As a result, this test provides the total time in seconds needed to complete the sequence, with the maximum time being 64 000 milliseconds. As a final result, we calculated the mean for both hands.

Figure 1.

A patient performing the sequential test.

After the keyboard task was completed, results were automatically recorded in a computer file, with one file per participant.

Ethics committee

This research project was approved by the bioethics committee of Hospital la Fe (2018/0382). All participants signed informed consent forms.

Statistical analysis

First, we performed a descriptive analysis of the results by calculating the mean (standard deviation), median (percentiles 25 and 75), and absolute and relative frequencies, and generated box-and-whisker plots. Next, we conducted a linear regression analysis to analyse the influence of age and sex on the values obtained by patients and controls in the keyboard task.

To evaluate the construct validity of keyboard parameters, we analysed divergent and convergent validity. First, we analysed the capacity of the different variables to discriminate between patients and controls, using a logistic regression analysis considering as predictive variables the scales under study and sex (as this variable seemed to have an influence on keyboard task performance in controls). For each resulting model, we calculated the corresponding receiver operating characteristic (ROC) curve. Accuracy was classified as moderate when the area under the ROC curve (AUC) was between 0.70 and 0.80, good when the AUC was between 0.80 and 0.90, and excellent when it was greater than 0.90.⁹

To analyse the capacity to discriminate between functional groups (walker, sitter, and non-sitter), we performed ordinal models for each variable. We then compared the prediction of the models with the actual classification using Bangdiwala’s B statistic and agreement chart. Concordance was quantified as moderate when the result of Bangdiwala’s B statistic was between 0.50 and 0.69, strong when the result was between 0.70 and 0.89, and very strong when it was greater than 0.90.10

Convergent construct validity was evaluated by creating a correlation matrix between keyboard variables and the remaining scales, using the Spearman rho. The strength of correlation was quantified as moderate when correlation value was between 0.50 and 0.69, strong between 0.70 and 0.89, and very strong for correlations greater than 0.90.

Statistical significance was set at P < .05. All statistical and graphical analyses were performed using the R software (version 4.0.3).

Results

Our study included a total of 22 patients and 26 age- and sex-matched controls, with both groups presenting a slight male predominance and a mean age of approximately 40 years (Table 1). Most patients presented type 3 SMA (73%), and were non-walkers (55%). The results of the motor scales per functional type are presented in Suplementary Table 1. All patients were able to perform the keyboard task with at least one hand. Although the execution time was not systematically measured, it was less than 5 minutes in all patients. Due to functional limitations, one patient could not perform the 9HPT, and 3 were unable to complete the HFMSE.

Table 1.

Clinical and demographic characteristics of patients and controls. SMA: spinal muscular atrophy.

Variable		Patients	Controls
Age	Mean (SD)	40.23 (16.73)	41.87 (10.01)
	Median (p25-p75)	40.72 (27.37-53.16)	42.48 (33.11-48.22)
Sex	Women	8 (36.36%)	11 (42.31%)
	Men	14 (63.63%)	15 (57.69%)
Type of SMA	2a	2 (9.09%)
	2b	4 (18.18%)
	3a	6 (27.27%)
	3b	10 (45.45%)
Functional type	Non-sitter	5 (22.73%)
	Sitter	7 (31.82%)
	Walker	10 (45.45%)

Divergent validity

The 3 keyboard variables and the 9HPT presented considerable differences between patients and controls (Fig. 2A). We did not find age or sex to influence the values obtained by patients, but we did observe sex to have an impact in controls; therefore, ROC curves were adjusted for sex. All the variables analysed presented excellent discriminant ability (> 0.9, Fig. 2B). Strength (AUC = 0.963) was the keyboard variable that best discriminated patients from controls, with very similar performance to the 9HPT (AUC = 0.966). This variable may have presented even higher discriminant ability were it not for the ceiling effect (2950 grammes) that we observed in most controls and also in one patient. Both time variables (random [AUC = 0.906] and sequential times [(AUC = 0.92]) presented very similar results, but somewhat poorer results than strength.

Figure 2.

Box-and-whisker plot (A) and ROC curves (B) of the keyboard variables and 9HPT, for patients and controls. 9HPT: Nine-Hole Peg Test.

The box-and-whisker plots in Fig. 3 present the results of the keyboard variables and the remaining scales, according to functional type. We may observe that all 4 variables moderately or strongly discriminate between the 3 subgroups, although all upper limb variables (keyboard, 9HPT, RULM, and EK2) discriminated better between non-walkers (sitters vs non-sitters), whereas the HFMSE and ALSFRS-R (which also measure lower limb function) better discriminated between walkers and sitters. In the time variables, all but one of the non-sitter patients needed the maximum time (64 000 milliseconds). This effect was not observed in the 9HPT, as this is a timed test with no time limit, in which non-sitters with poorer functional status were excluded due to their inability to perform the test. Despite this, in general terms, the sequential time (Bangdiwala’s weighted agreement coefficient = 0.83) obtained with the keyboard is the measurement best discriminating between the 3 functional groups, of all tests performed.

Figure 3.

Box-and-whisker plot (A) of the keyboard variables and traditional scale scores per functional type. 9HPT: Nine-Hole Peg Test; ALSFRS-R: Amyotrophic Lateral Sclerosis Functional Rating Scale Revised; BW: Bangdiwala’s weighted coefficient; EK2: Egen Klassifikation 2 scale; HFMSE: Hammersmith Functional Motor Scale Expanded; RUML: Revised Upper Limb Module.

Convergent validity

Fig. 4 shows the correlation matrix of the scales used in this study. All keyboard variables show a strong or very strong correlation with each other and with the remaining scales. The variable strength is better correlated with motor scales, whereas time variables are better correlated with functional scales.

Figure 4.

Correlation matrix of the scales administered. 9HPT: Nine-Hole Peg Test. ALS-UL: mean score of the 3 ALSFRS-R items corresponding to the upper limbs. ALSFRS-R: Amyotrophic Lateral Sclerosis Functional Rating Scale Revised; EK2: Egen Klassifikation 2 scale; HFMSE: Hammersmith Functional Motor Scale Expanded; RULM: Revised Upper Limb Module.

Discussion

This pilot study confirms the viability and validity of Neuromyotype in a heterogeneous sample of adult patients with SMA.

To date, 2 main types of scales have been used to study adults with SMA (Table 2): motor scales (HFMSE and RULM) and functional scales (ALSFRS-R and EK2).⁵ The main advantage of functional scales is their simplicity and quick administration (5-10 minutes); however, few studies have evaluated their reliability and sensitivity to changes.^5,10 Furthermore, the EK2 was designed exclusively for non-walkers.^5,10 Motor scales (particularly the HFMSE), despite being widely used, present a ceiling effect in patients with better functional status and a floor effect in patients with poorer functional status,⁵ and require long administration times and extensive, specific training.

Table 2.

Comparison of the main characteristics of traditional tools vs new digital tools.

	RULM	HFMSE	ALSFRS-R	EK2	9HPT	New digital tools
Applicability to all functional types	✓	x	✓	x	x	✓
Human and material resources	x	x	✓	✓	–	✓
Automatic/self-administration	x	x	✓	✓	x	✓
Application time	> 30 min	> 30 min	5-10 min	5-10 min	5-10 min	< 5 min
Limitations	Ceiling effect in walkers	Floor effect in non-walkers	Less sensitivity in non-walkers	Ceiling effect in walkers	Long, difficult, and frustrating task for non-sitters	Validation and further longitudinal studies are pending

9HPT: Nine-Hole Peg Test; ALSFRS-R: Amyotrophic Lateral Sclerosis Functional Rating Scale Revised; EK2: Egen Klassifikation 2 scale; HFMSE: Hammersmith Functional Motor Scale Expanded; RUML: Revised Upper Limb Module.

Thus, there is a need for new tools for reliable, sensitive measurement of the progression of adult patients with SMA, especially those with poorer performance. These tools should ideally be applicable to all functional types, simple and quick to perform, and present high sensitivity to changes.^1,11,12

Today, several studies support the importance of measuring strength in the follow-up of patients with motor neuron diseases, especially for adult patients with SMA.^13–15 Furthermore, previous studies in these patients suggest that strength measurement is a better tool than motor scales to detect changes in these patients.^8,16 Movement speed and accuracy, measured with such tools as the 9HPT, are also frequently used in the assessment of patients with neuromuscular disorders, and to assess fatigability in SMA.¹⁷

Viability

Our results confirm the simplicity and quick administration of the Neuromyotype test, confirming its clinical viability. Thus, the entire patient sample, including 5 non-sitters with poor functional status, could perform the tasks with at least one hand in a short period of time. In contrast, not all patients could complete the remaining motor scales, which also presented much longer execution times (Table 2).

Furthermore, the keyboard is easy to transport and store, and offers the possibility of self-administration, with results being sent electronically to a central device, thus avoiding unnecessary travel to the hospital. Table 2 includes the advantages and limitations of new digital tools, such as Neuromyotype, compared with traditional scales.

Divergent validity

The variables obtained with the keyboard, especially strength, performed similarly to the 9HPT in discriminating between patients and controls. The ceiling effect for strength in controls prevented us from obtaining even better discriminant ability. Furthermore, keyboard variables also provided good discriminant ability for distinguishing between functional types, and especially for differentiating non-sitters from sitters and walkers. This was unsurprising, as all scales involving the upper limbs (9HPT, RULM, and EK2) also discriminate better between non-walkers (sitters vs non-sitters), whereas those scales also measuring lower limb function (HFMSE and ALSFRS-R) discriminate between walkers and sitters. Despite the presence of a ceiling effect in the execution times of patients with poorer functional status, the sequential time obtained with Neuromyotype is, among all scales, the variable best discriminating between the 3 functional types.

Convergent validity

We also observed a very strong correlation between the Neuromyotype variables, and especially between sequential and random times, which suggests they measure the same phenomenon. Furthermore, both time variables present stronger correlations with functional scales than with motor scales, whereas strength is better correlated with motor scales. This suggests that strength and time variables report on complementary aspects of patients’ motor function.

Usefulness of digital tools

We have already mentioned the limitations of the current measurement tools to assess adult patients with SMA, especially those with a poorer functional status. The Medical Research Council strength scale and conventional dynamometers present some limitations, particularly the high inter- and intra-rater variability. This has led to the development of new digital tools that limit the evaluator effect and may provide several advantages (Table 3). An example is the set of devices known as Myotools®, which were designed to measure such variables as grip strength, pinch strength, and typing speed.^6–8 However, Neuromyotype may present certain advantages over these devices. Firstly, the finger flexion analysed with Neuromyotype may be more representative of the everyday activities of non-sitter patients (joystick controls, touchscreens, and keyboards) than the pinch movement. Furthermore, this movement requires better functional status and may be more strongly influenced by deformities or retractions of the hand, which are frequent in these patients. Secondly, Neuromyotype uses a single device to measure typing strength and speed, whereas Myotools® requires the combination of 2 devices. Thirdly, Neuromyotype is connected to a computer or tablet, automatically recording the results obtained. This precludes the possibility of transcription errors and facilitates the electronic acquisition of measurements.

Table 3.

Data from some functional and motor scales used in patients with SMA. Table modified from Finkel et al.⁵

	HFMSE	MFM	RULM	Myotools®	Neuromyotype
Walker	•	•	•	•	•
Sitter	•	•	•	•	•
Non-sitter	X	•	•	•	•
Construct validity	•	•	•	•	•
Action mechanism	•	•	•	•	•
Normative range	•	•	•	•	•
Multi-centre study	•	•	•	•	X
Inter- and intra-rater reliability	•	•	•	•	X
Published studies	•	•	•	•	X
Response to change	•	•	In progress	•	X
Clinical relevance	•	•	•	•	X
MCID	X	X	X	X	X

HFMSE: Hammersmith Functional Motor Scale Expanded; MCID: minimum clinically important difference; MFM: Motor Function Measurement; RUML: Revised Upper Limb Module.

Strengths

This pilot study has determined the feasibility and validity of a smart keyboard (Neuromyotype) in a large and heterogeneous cohort of adult patients with SMA. Data on typing strength and speed seem to provide supplementary information on functional performance in these patients. Thus, Neuromyotype seems especially useful in patients with limited functional performance, which cannot be evaluated with traditional scales.

Limitations

Several limitations were observed in the keyboard that may be improved in future versions. The main limitation is the ceiling effect in strength and performance time, which may be resolved by changing the predefined parameters. As this is a cross-sectional pilot study, we did not analyse the ability of keyboard variables to detect changes and did not evaluate test-retest reliability. Despite this, both are assumed to be high, considering previous experiences with Myotools and the fact that the entire measurement process is automatic and does not depend on an external evaluator or observer. Therefore, the next version of Neuromyotype should eliminate the ceiling effects and enable the automatic and remote transmission of data to a central database, thus enabling electronic measurements. Before implementing Neuromyotype in clinical practice, future longitudinal and multicentre studies should analyse the viability of these measurements and analyse the instrument’s reliability and sensitivity to change. There is also a need to compare it with other similar devices that are currently available, and to study its usefulness in other neuromuscular disorders.

Finally, as this is a pilot study in a rare disease, the sample size is relatively small considering the heterogeneity of the disease; therefore, the results should be confirmed with studies including larger numbers of cases and age groups.

Conclusions

Our study demonstrates the viability and validity of Neuromyotype, a device enabling integrated measurement of movement strength and speed in the assessment of adolescent and adult patients with SMA. The data obtained with this tool may be highly clinically relevant, saving time and resources in comparison with the remaining scales. Future studies should complement the validation of the keyboard in order for it to be incorporated into research and clinical practice.

Conflicts of interest

JFVC has received fees as a speaker and lecturer from Biogen and Roche, which have also paid for registrations to congresses.

Funding

JFVC is funded by a research contract with the Instituto de Salud Carlos III (JR19/00030). This research project has been financed by REDIT-La Fe 2017 and UCIE IIS La Fe grants (INNCON-2020-6/Agencia Valenciana de Innovación). Neuromyotype has been registered as a Utility Model: U202132029 Device for the assessment of a psychomotor response (OEPM, 18/10/2021).

Ethical liabilities: This project has been approved by the ethics committee of the Instituto de Investigación Sanitaria La Fe (2018/0382).

Acknowledgements

We would like to thank all patients with SMA and controls who have voluntarily participated in this project.