Neuroprosthetic devices based on brain-machine interface technology hold promise for the restoration of body mobility in patients suffering from devastating motor deficits caused by brain injury, neurologic diseases and limb loss. During the last decade, considerable progress has been achieved in this multidisciplinary research, mainly in the brain-machine interface that enacts upper-limb functionality. However, a considerable number of problems need to be resolved before fully functional limb neuroprostheses can be built. To move towards developing neuroprosthetic devices for humans, brain-machine interface research has to address a number of issues related to improving the quality of neuronal recordings, achieving stable, long-term performance, and extending the brain-machine interface approach to a broad range of motor and sensory functions. Here, we review the future steps that are part of the strategic plan of the Duke University Center for Neuroengineering, and its partners, the Brazilian National Institute of Brain-Machine Interfaces and the École Polytechnique Fédérale de Lausanne (EPFL) Center for Neuroprosthetics, to bring this new technology to clinical fruition.
Millions of people worldwide suffer from sensorimotor deficits caused by neurologic injuries, diseases or limb loss. According to recent data reported in Medical News Today, five million people in the USA alone currently suffer from some type of severe body paralysis.1 Currently, there is no cure for such devastating cases of paralysis, for example complete spinal cord injury (SCI).2 Meanwhile, treatment is only partially effective in less severe cases.3 Neural prosthetic devices based on brain-machine interfaces (BMIs) hold promise to restore both partial and full body mobility in paralyzed patients.4–10 BMIs bypass the site of the neural lesion and connect the remaining healthy motor areas of the brain, particularly the motor cortex, directly to assistive and prosthetic devices that can take the shape of, for example, robotic limbs or a full body exoskeleton. The main idea behind BMIs is to employ the activity of healthy motor brain areas, which in many cases of paralysis remain capable of generating motor commands despite being disconnected from the body effectors,11 to control artificial tools that restore the patient's mobility.
During the last decade, the field of BMIs has experienced an explosive development.7,9 Hence, it has generated high expectations among neuroscientists, physicians and patients alike, regarding its potential clinical applications. A number of BMI systems have been studied in rodents12 and nonhuman primates.13–17 BMI technology also entered human clinical research where both non-invasive EEG-based systems5,18,19 and invasive BMIs based on brain implants20–22 have been tested. Notwithstanding the success of these pioneering experiments, a number of issues need to be resolved before a fully functional practical neuroprosthetic for long-term use can be built.7 These include: implant biocompatibility issues;23 increasing the number of neural channels of the recording system; improving BMI decoding algorithms; building fully implantable systems; sensorizing neuroprosthetic limbs; and extending the BMI approach to a broader range of motor control tasks, especially tasks that require lower limb control: bipedal walking24 and upright posture control.25
The Duke University Center for Neuroengineering (DUCN) has been at the forefront of BMI research on cortical prosthetic devices for motor rehabilitation since this field emerged about 12 years ago. At the DUCN, we have developed pioneering BMI systems that enact a wide range of motor functions, from arm reaching and grasping 13,17,26 to bipedal locomotion24,27 in a variety of artificial actuators. DUCN researchers were also the first to incorporate artificial somatic sensation in BMIs.28,29 Here, we review the most recent findings of the BMI initiative at the DUCN and discuss its perspectives and strategic plan for the near- and long-term future of BMI research.
BMI COMPONENTSThe essential components of a BMI system are well captured by the BMI that enacts reaching and grasping.7,9,13,17,26 In this BMI design, a rhesus monkey controls a robotic arm with its motor cortical activity, while visual and/or somatosensory feedback signals from the robot are delivered back to the brain as either natural visual stimuli or, in the case of artificial tactile information, intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1).7,28,29 In these studies, we implant multielectrode arrays in multiple cortical areas of the rhesus monkey's brain.30 The present generation of chronic multielectrode implants is capable of recording the extracellular electrical activity of hundreds of cortical cells.13,24,26,30 As a result of recent technological developments, this benchmark number is expected to rise to several thousands of simultaneously recorded neurons in the next decade. Recording such large-scale neuronal ensemble activity is crucially important for BMI accuracy.9 Concurrent neuronal ensemble activity is processed by BMI decoding algorithms which translate myriad neuronal spikes into continuous signals that drive the robotic arm's movements, according to the voluntary motor intentions of the subject. The BMI setup also includes the data acquisition system, the computer cluster running multiple decoding models in real time, the robot arm, the visual display and a sensory feedback loop from the actuator to the brain. Below we discuss these key BMI components in more detail.
LARGE SCALE NEURONAL RECORDINGSOur work on BMIs has clearly demonstrated that a large number of recording channels is needed for accurate extraction of motor intentions from the brain.9,13,17,31,32 Shifting from the classic focus on single neurons, today, more and more evidence accumulates in favor of the notion that distributed ensembles of neurons define the true physiologic unit of the mammalian central nervous system.9 During the last two decades, advanced electrophysiological methods have allowed recording from progressively larger samples of single neurons in behaving animals.30,33–35 This methodology has equipped neuroscientists with better tools to study the neurophysiologic principles that define the operation of the cortex,9 and has also made the idea of BMIs practical. Today, the most advanced BMIs developed at the DUCN utilize simultaneous recordings of the extracellular electrical activity of hundreds of individual neurons.30
Neuron-dropping analysis, a technique developed at the DUCN to measure the dependence of parameter extraction accuracy on the number of simultaneously recorded neurons,17 shows that BMI accuracy increases with the size of the neuronal ensemble. Typically, the best extractions of motor parameters are obtained when the activity of populations of neurons from several cortical areas is recorded simultaneously.13,24 Importantly, the number of required neurons increases as more motor parameters are simultaneously extracted.24
Large neural ensembles confer redundancy of control and, hence, reliability.9 In our current research on BMIs that enact arm and leg movements, we can obtain a peak signal-to-noise ratio (SNR) of 10dB using approximately 100 task related neurons.24,36 This corresponds to a 5 bits/s bandwidth. SNR can be further improved via time averaging, but this slows down the movements. Realistic and useful movements should have a SNR of about 20 to 30dB. Indeed, we consider this benchmark as the lower bound that one should aim in order to build a neuroprosthetic device for clinical use. Thus, to estimate the number of neurons needed to obtain this level of accuracy, we assume that noise decreases as the inverse of the square root of the number of neurons—an assumption that holds for both Gaussian and Poisson noise sources. Under this assumption, a tenfold increase in the sample of neurons recorded simultaneously would be needed to achieve the 10dB of improvement needed. Therefore, to obtain a control signal with 20dB fidelity (the DUCN short-term goal), recordings involving 1000 neurons are needed; to obtain 40dB (the DUCN long-term goal), 100,000 neurons are needed. In addition, it should be emphasized that commensurately more neurons are required for prosthetic limbs with many degrees of freedom (DOFs). As practical neuroprosthetics for paralyzed subjects will have to cope with at least 4–10 DOF, demands for large samples of recorded neurons will remain a central bottleneck for the development of clinical applications of BMIs for the foreseeable future. Thus, only when a new generation of high-channel count recordings becomes available can practical neuroprosthetics for clinical use be implemented with a reasonable expectation for clinical success. At the DUCN, we expect to conclude the development of such new technology and, as a consequence, launch the testing phase of our first clinical application of a BMI for restoring full body mobility by the summer of 2014.
To achieve this goal, we have been developing advanced sensors that sample large-scale extracellular electrical activity from the brain. In our most recent design of multielectrode implants, electrodes are arranged into subsets sitting inside guiding tubes.37 The electrodes within these subsets have different lengths. The guiding tubes are spaced at 0.5–2 mm, and each of them is independently movable. This new three-dimensional (3D) design, named ‘the multi-electrode recording cube’ not only improves the size of the potential neuronal sample recorded simultaneously per probe, but also enhances the implantation capacity of the microelectrodes in the brain tissue as each electrode subset penetrates individually, minimizing cortical dimpling. Currently, we are expanding this new cube design to develop the next generation of implants that will increase the number of potential active recording sites per cube to about 1500. Also, as the size of each implanted microelectrode is of vital importance when so many electrodes are inserted into richly vascularized brain tissue, it is imperative to produce as little tissue displacement as possible during the surgical implantation of these recording devices. This issue will be resolved by removing the structural elements after implantation, thereby freeing the microelectrodes made of smaller diameter microwires than that required to pierce and penetrate the brain tissue. Not only will small diameter microwires minimize the mechanical distortion of nervous tissue, it will also help to minimize microglial and other immune responses to the foreign material. Our approach will allow implantation of small-diameter wires and thereby avoid failures associated with large-diameter implants. Fine electrodes will be guided into the brain with a strong, stiff tungsten central shaft. Later on, the shaft will be removed, leaving the electrodes in the brain. Each single electrode shaft will carry 10–20 recording microwires, staggered to cover the targeted nervous tissue. To refer properly to this new technology and to distinguish it from previous approaches, we have coined the term ‘very-large scale multichannel brain implants' (VLS-MBI).
DECODING ALGORITHMSLarge-scale neuronal activity recorded by our very large-scale multichannel brain implants will be processed by real-time BMI encoding algorithms that extract behavioral parameters, for example kinematic parameters of many DOF limb movements. At the DUCN, we employ an integrated BMI suite that combines the neurophysiologic recording and stimulation hardware, as well as the computer cluster that runs behavioral tasks and BMI decoders.24,29 We have incorporated into this BMI suite several neuronal decoders: the unscented Kalman filter,36 the Wiener filter, artificial neural networks, and discrete state Bayesian approaches.38 In particular, the unscented Kalman filter makes Bayesian inferences of the repetitive patterns of the movements performed during arm reaching tasks, as such patterns will occur frequently in the tasks that a practical neuroprosthetic limb has to perform. For example, a prosthetic limb that aids in feeding moves back and forth between the food item and the user's mouth. The unscented Kalman filter exploits non-linear models of neural tuning and prior knowledge about movement patterns. In addition, it keeps a short history of the state variables—in this case, the desired limb movements. This algorithm also captures complex patterns in the desired movements. At the DUCN, the unscented Kalman filter was tested in a BMI that enacted arm reaching movements and achieved significantly better accuracy as compared to the Kalman filter, the Wiener filter, the population vector method, and the stochastic state point process filter.38
In addition to the unscented Kalman filter, we employ a multiple-model-switching paradigm.24 In this approach, separate submodels are trained to decode particular behavioral states (e.g. the grasping phase of the reach and grasp movement versus the reaching phase, or walking forward versus walking backward), and the state predictor model serves to detect the state and select the appropriate submodel. The simplest switching model works as a combination of three linear decoding models: a model for predicting state 1 (e.g. reaching), a model for state 2 (e.g. grasping) and the paradigm predictor model (the switch). These models are arranged in two layers with the paradigm predictor model controlling a toggle between the two kinematic submodels, which are then shunted to the final output of the switching model. When it is determined that the monkey is performing in state 1, one submodel is used to produce the output, and when state 2 is detected, the other submodel is used.
These algorithmic tools allow us to test different neural control modes. In the most straightforward implementation, we predict the kinematics of each joint of the limb and convert them into the actuator configuration. This control model is of interest for several reasons. First, this mode of operation is similar to the operation of the nervous system in controlling natural movements. Second, the number of degrees of freedom that can be achieved by a BMI is of great interest to those working in this field. At the same time, practical BMI applications will be inconvenient to use if the user is required to control each degree of freedom independently. To overcome this problem, we intend to implement in our clinical applications a mode of BMI operation in which the control over an external actuator is shared between the subject's brain activity and robotic controls. This mode of operation is known as shared control of a BMI.39 During shared control, the subject's brain is primarily in charge of high-order control of movements (when to initiate movement and where to move), whereas the low-level coordination of the movement is performed by an autonomous controller. Such sharing optimizes BMI performance: the user can control it through their voluntary intentions, whereas the robotic controller assures accuracy and stability.
BRAIN-MACHINE-BRAIN INTERFACEThe DUCN research team was the first to add an artificial somatosensory feedback loop to a BMI for arm reaching. In our initial studies, we used spatiotemporal patterns of ICMS to instruct primate arm reaches. In a study conducted in owl monkeys,40 we addressed two issues that are critical for using ICMS as an artificial sensory channel in neural prostheses: (i) whether such artificial sensation can be evoked by multichannel ICMS; and (ii) whether ICMS is suitable for long-term usage in a BMI-like application. We explored the first issue by testing the capacity of owl monkeys to discriminate multichannel ICMS of increasing complexity. We investigated the second issue by testing the long-term efficacy of ICMS over many months. We discovered that owl monkeys could learn to discriminate spatiotemporal patterns of ICMS delivered directly to their S1 and guide their reaching movements based on this discrimination. Owl monkeys were implanted with multielectrode arrays in several cortical areas, and S1 implants were employed to deliver spatiotemporal patterns of ICMS. The behavioral task performed by the owl monkeys progressed from a simple requirement of detecting the presence of ICMS, to the requirement of discriminating spatiotemporal patterns created using four electrode pairs. The monkeys successfully learned these tasks. Interestingly, they learned new microstimulation patterns more rapidly compared with initial training. This result suggests that the monkey brain may have mimicked the operation of this intra-cortical microstimulation paradigm as if it were a new sensory channel.
In a study conducted in rhesus monkeys, we employed ICMS to cue monkeys that performed BMI reaching tasks.29 The behavioral tasks consisted of acquiring visual targets with a computer cursor. The monkeys first performed the behavioral tasks manually, using a hand-held joystick, and later controlled the cursor movements directly with the electrical activity of a sample of cortical neurons. Manual performance data were used to train linear decoding models that extracted cursor position from the modulations of populations of simultaneously recorded cortical cells. Once the model parameters were calculated, the mode of operation was switched to brain control during which the joystick was disconnected from the cursor and cursor position became directly controlled by the signals extracted from the animal's brain. ICMS of S1 was then used to cue monkeys to which direction they had to move their arms during execution of a target choice task. In this experiment, monkeys had to choose among two visually identical targets based on an instruction, in the format of spatiotemporal pattern of ICMS, delivered directly to the animals' S1. The monkeys were implanted with multiple microwire arrays in several cortical areas. The dorsal premotor cortex (PMd) and primary motor cortex (M1) arrays were used to extract motor commands, while the primary somatosensory cortex (S1) array was employed as the main target for ICMS. The electrodes chosen for ICMS yielded recorded S1 neurons with clear receptive fields located on the ventral aspect of the second, third and fourth digits and palm pads. Biphasic current pulses were injected into S1 through these electrode pairs synchronously at 30 to 60 Hz. We simultaneously recorded the electrical activity of 50–200 cortical neurons. The monkeys learned to perform in BMI control with and without using the joystick.
Moreover, in our recent study,28 ICMS served as an artificial sense of active touch as it conveyed to the monkeys the properties of virtual objects that the actuator (computer cursor or a virtual image of a monkey arm) touched. Two monkeys learned to operate this new apparatus, without any need to move their own limbs or use visual feedback to solve the task.
Thus, as a result of converging principles of motor and sensory neuroprosthetics, we have demonstrated the feasibility of moving from a BMI to a brain-machine-brain interface in which artificial actuators and brain tissue are connected bi-directionally, without any meaningful interference or constraint imposed by the physical limits of the subject's body.
POSITION SENSEHaving created a paradigm for introducing an artificial sense of touch to our BMIs, the next goal of the DUCN is to develop neuroprosthetic limbs that incorporate an artificial sense of position. Position sense is of great importance for clinical applications because ideal prosthetic limbs should be perceived as natural extensions of the users' bodies. Normally, such positional signals are provided by the afferents of muscles, joints and skin. Afferent information ascends to the brain where it is eventually transformed into representations encoding limb position in different coordinate frames, such as body-centered and external space-centered coordinates.41,42 The complexity of spatial processing in the brain makes the task of creating an artificial position sense particularly difficult. Theoretically, artificial proprioception could be implemented in a straightforward way by applying microstimulation to the somatosensory cortex or thalamus that reproduces individual joint angles of an artificial arm. The subject would then experience sensation of many DOFs of the artificial arm and would be able to perceive its spatial orientation. However, practical realization of such position sense would be problematic because of numerous uncertainties in choosing the transformation from the stimulation patterns to the joint angles. Given the complexity of cortical processing of proprioceptive information,43 it would be impossible to implement such transformation as a precise mapping from the arm joints to the brain somatosensory map. It is also questionable if the user would be able to transform the stimulation patterns designed to mimic individual joints into a coherent position sense. Additionally, certain centrally generated components of normal position sense, such as corollary discharges,44 would be difficult to incorporate in such implementation.
Because of these foreseen difficulties in the straightforward implementation of an artificial position sense, we chose a simpler and more feasible approach in which stimulation of S1 is not initially coupled to the orientation of the limb position, but instead represents 3D spatial locations to which the subject is required to reach. In this approach, the subject starts with learning how ICMS of cortical somatotopic representations of the body is mapped to a 3D space. Once the learning of such mapping is perfected, the same ICMS pattern is used to represent the spatial location of the endpoint of a prosthetic limb, fulfilling the goal of providing a limb neuroprosthetic with a position sense. This experimental design bears similarity to the studies on sensory substitution in which visual information was conveyed by the stimulation of skin surfaces,45–47 with the difference that, instead of using peripheral receptors as the entry point for the artificial sensory channel, we opted to deliver the information directly to the somatosensory cortex or thalamus. We expect that similarly to sensory substitution using peripheral stimulation, training with stimulation of the somatosensory areas of the brain would eventually give rise to an artificial position sense that represents external space. Furthermore, we expect that as such position sense becomes coupled to the position of a virtual image of a monkey arm (monkey avatar), monkeys will develop an artificial proprioception sense capable of guiding their movements thereafter.
OPTOGENETICSOur initial approach to sensorizing neuroprosthetic limbs was based on ICMS, which involves delivering small pulses of current through microelectrodes directly into the sensory areas of the brain.28,29,40 Although this technique proved to be efficient in bi-directional BMIs, it has relatively low spatial resolution48 and produces electrical artifacts that saturate neural recording channels,49 causing problems for extracting neural activity both during and after this period, as it typically occludes 5–10 ms of neural data per pulse. ICMS acts on both fibers and neurons, and neurons in the stimulated area may become inhibited instead of being excited.50 A further problem is that ICMS pulses have to be precisely balanced, otherwise an unbalanced current injection could cause damage to the neural tissue stimulated by this procedure.
To explore an alternative method for sensorizing neuroprosthetics, we have started to design a cortical multichannel stimulator based on optogenetics—a new technique for the stimulation of neurons. Optogenetics is based on genetically modified ion channels that respond directly to light.51 These light-gated ion channels, such as Channelrhodopsin-2 (Chr-2), allow precise, millisecond control of specific neurons.52,53 This technique eliminates most of the key problems associated with ICMS: there is no associated electrical artifact to interfere with the electrophysiological recordings, nor any tissue damage from the current injection. It also allows for finer control of the spatial pattern of activation. We expect that multichannel optogenetic stimulation combined with multichannel neuronal recordings will allow us to develop an artificial position sense in monkeys and to study the neuronal mechanisms involved in it. Bi-directional BMIs based on such a design would be superior to current designs in both the specificity and long-term performance of the sensory loop and the quality of neuronal recordings. We are currently building a prototype optogenetic implant that will simultaneously sample the activity of cortical neuronal ensembles and deliver complex stimulation patterns through multichannel optogenetic stimulation of cortical sensory areas. Our optogenetic arrays consist of both recording electrodes and fibers for optical stimulation. This design uses a novel light delivery system in which light pulses from a laser are directed onto a Digital Micromirror Device. These DMD chips consist of over 150,000 individually movable micrometer size mirrors. Depending on the alignment of these mirrors, they will either allow the light to bypass the chip or reflect it into the components of a multicore fiberoptic, therefore allowing incredibly sophisticated patterns of activation. The multicore fibers consist of 30,000 individual strands. A number of these strands will be attached to movable electrode shafts allowing them to penetrate deep into the cortical columnar structure, thus negating the problem of light scatter by the tissue and allow very specific targeting of individual neurons.
After the Chr-2 gene is delivered to the somatosensory and motor cortex of our monkeys through adeno-associated virus vector (AAV) injections performed using implanted cannulae, combined optical-stimulation/recording grids will be inserted in the cortex through the same cannulae. We expect that implanted animals will learn to use the spatiotemporal optogenetic stimulation delivered to their S1 as BMI feedback. We will integrate this artificial sensory channel into our real-time BMI system, in which monkeys control the movements of primate virtual bodies, known as avatars, directly by brain activity while receiving sensory input through spatiotemporal optogenetic stimulation.
BIMANUAL BMISo far, BMIs for arm movements involved only a single actuator that typically enacted reaching and grasping movements. Future BMIs will have to generate bimanual functionality. In their truest sense, bimanual movements are temporally and spatially coordinated movements of both upper limbs. At the DUCN, we are designing a BMI capable of extracting the motor commands to enact such coordination. The starting point in developing a bimanual BMI is to understand the underlying cortical processes. Bimanual processes activate different subsets of neurons than unimanual tasks.54–59 As previous BMI research has focused on unimanual tasks, a key component of transitioning to successful bimanual control will be recognizing and adapting to the differences in the cortical representation of this behavior. Unimanual movement initiation and coordination have often been shown to be correlated to neural activity in the contralateral hemisphere of the motor cortex. However, in tasks involving both arms, activated cortical areas are less clearly defined and have greater interhemispheric interactions. Interhemispheric connections of premotor regions have been the subject of several investigations, and several findings point to the fact that these regions exhibit much greater interhemispheric connectivity than M1.59 At least five cortical regions show activation related to inter-limb coordination: dorsal premotor cortex (PMd); cingulate motor area (CMA); supplementary motor area (SMA); posterior parietal cortex (PPC); and M1.60 We will implant these areas to build the first bimanual BMI.
In our bimanual BMI experiment, monkeys have to move two avatar hands within an initial target that appears in the center of the screen. When this task is learned, the monkey will learn a similar behavior, except that two peripheral targets will appear, and the monkeys will have to guide each avatar hand to the separate targets. We expect that monkeys will eventually learn to control such bimanual movements through a BMI, without engaging any overt movements of their limbs.
BIPEDAL LOCOMOTIONPrevious BMI studies focused predominantly on the behavioral tasks in which an artificial actuator enacted upper extremity movements, such as reaching and grasping. Except for a few studies,24,61,62 virtually no attempts have been made to translate BMI technology to tasks enacting motor functionality of lower extremities. Yet, deficits or the complete loss of the ability to walk present a considerable problem. Such deficits commonly result from spinal cord injury,63–65 neurologic diseases66,67 and limb loss.68 Surveys of paraplegic patients showed that they prioritized walking and trunk stability among the most desired motor functions they would like to be restored.69,70 Quadriplegic patients prioritized arm and hand function.69 Thus, developing neural prosthetic devices for restoration of leg mobility is as important as developing neuroprosthetics for resuming arm and hand movements, and for some categories of patients it is their top priority. Such neuroprosthetic devices would clearly have a major impact on the community of patients suffering from leg paralysis.
In our treadmill locomotion setup, rhesus macaques walked bipedally on a custom modified treadmill.24 We tracked movements of the right legs of the monkeys using a wireless, video-based tracking system developed in our laboratory.71 We then added a second tracking system to the setup and tracked movements of both legs. Leg kinematics and electromyographies (EMGs) of leg muscles were reconstructed from neuronal ensemble activity in real time using the Wiener filter.13,17,26,72 While the individual neuronal firing rates were highly variable from step to step at the millisecond scale, combining the activity of many neurons using a BMI decoder produced accurate predictions of leg movements. Extraction of neural information performed separately for different cortical areas showed that walking parameters could be predicted using neuronal activity recorded in either M1 or S1 contralateral to the right leg, as well as ipsilateral M1. In these experiments, neuronal ensemble activity was simultaneously recorded from all 384 channels.
Using the locomotion setup, we also demonstrated real-time BMI control of bipedal walking in a robot.27 We sent the predictions of monkey leg kinematics to the Advanced Telecommunications Research Institute (ATR), Kyoto, Japan, where our collaborators set their humanoid 51-degree of freedom robot (manufactured by SARCOS, Salt Lake City, UT, USA) to reproduce monkey locomotion patterns. The monkey received visual feedback of the robot's movements on a video screen.
POSTURE AND BALANCEBMI for restoration of lower limb function is fundamentally different from the BMI for upper limbs because it has to not only replicate basic gait and stepping functions, but also adapt to postural control. In addition to developing a BMI for bipedal locomotion, we have developed a proof of concept BMI for postural control.25 Using this experimental setup, in which monkeys stood on a platform that produced abrupt horizontal movements and evoked postural perturbations, we demonstrated recently that cortical neurons modulate their firing rate in relation to changes in posture. We then applied BMI decoders to extract information about these changes from cortical ensemble activity. The platform was driven either rhythmically, allowing the animal to anticipate the upcoming perturbation, or with a random time delay between movements so that the animal was unable to predict when the next movement would come. Similarly to our treadmill experiments, single unit analysis of the neural activity showed that a vast majority of cortical neurons (mostly in M1 and S1 representations of the legs) were highly modulated in association with compensatory postural reactions. These modulations were directionally tuned: neuronal rates changed differently depending on the direction of platform displacement. The BMI decoder based on a linear model predicted postural disturbances with good signal-to-noise ratios. Importantly, the decoder yielded different results depending on anticipated versus unanticipated platform displacements. Model parameters were different in these two cases, and anticipated displacements were predicted with higher accuracy than anticipated. These results show the feasibility of a BMI for restoration of upright posture. The concrete implementation of such a BMI should be based on shared control rather than relying solely on cortical modulations for postural control.
INTEGRATING FUNCTIONAL ELECTRICAL STIMULATION (FES) TO THE BMI PARADIGMThe issue of directly actuating the movements of paralyzed legs has been addressed in many clinical studies in spinal cord injury patients. Robotic orthoses73–75 and functional electrical stimulation (FES) devices76–79 have been introduced as therapies for leg paralysis. FES devices for the lower extremities date back to the 1960s when the first FES application to restore standing was developed.80 In this study, paraplegic subjects were able to stand after FES was applied to their quadriceps and gluteus maximus muscles. Also in the 1960s, FES of the common peroneal nerve was used to correct foot drop during the swing phase of the gait.81 Foot drop stimulators have been commercialized.82,83 This technique is used in a Parastep® system developed by Sigmedics, Inc, Fairborn, OH, USA.84,85 At present, a variety of FES methods have been developed that allow stimulation of multiple muscle groups with surface or intramuscular electrodes. Importantly, FES systems have many positive therapeutic effects on patients with incomplete spinal cord injury (SCI).79 Unfortunately, current FES systems allow for only crude movement such as short-distance ambulation or short-term standing, require support from a walker or similar tool, and are primarily offered as a way to avoid medical complications of prolonged limb inactivity rather than as an alternative to locomotion restoration. Additionally, such systems require manual actuation by the user, resulting in a limited and arguably non-intuitive way to restore leg function.
Building on our success in BMIs that enact bipedal locomotion and postural control, we are currently developing a cortically driven FES system for the restoration of walking. This novel BMI will be tested in a monkey model of bipedal walking. Our objective is to build a BMI-driven FES system that produces bipedal locomotion patterns by converting cortical ensemble activity into stimulation patterns that drive leg muscles. We suggest that BMI control over bipedal locomotion can be established by recording large-scale cortical activity from the sensorimotor cortex, extracting locomotion patterns from the raw cortical signals and converting them into trains of FES applied to multiple leg muscles.
WHOLE-BODY NEUROPROSTHETICAs follows from our results on BMIs that enact leg movements, BMIs for the whole body are likely to become a real possibility in the near future. We propose the development of a whole-body BMI in which neuronal ensemble activity recorded from multiple cortical areas in rhesus monkeys controls the actuators that enact movements of both upper and lower extremities.37 This BMI will be first implemented in a virtual environment (monkey avatar) and then using a whole-body exoskeleton. In these experiments we will also examine the plasticity of neuronal ensemble properties caused by their involvement in the whole-body BMI control and the ability of cortical ensembles to adapt to represent novel external actuators.
Furthermore, we will also explore the ability of an animal to navigate a virtual environment using both physical and neural control that engages both the upper and lower limbs. The first phase of these experiments will be to train the animals to walk in a bipedal manner on a treadmill while assisting the navigation with a hand steering control. We have already built a virtual environment needed for the monkey to navigate using 3D visualization software. Within this environment, the monkey's body is represented by a life-like avatar. This representation is viewed in the third person by the monkey and employs real-world inverse kinematics to move, allowing the avatar's limbs to move in close relation to the experimental animal.
Initially, the direction that the avatar is facing will be dictated by the monkey moving a handlebar with its hands. As the animal moves the handlebar left or right, the avatar will rotate in the corresponding direction. The avatar's legs will mimic the exact motion of the monkey's legs on the treadmill. The simplest task will be for the animal to simply move the avatar forward to an object that represents a reward, a virtual fruit. Virtual fruits will appear at different angular positions relative to the monkey, which will let us measure the neuronal representation of navigation direction and modulations in cortical arm representation related to the steering. The monkey will have to make several steps while steering in the required direction to approach a virtual reward and to obtain an actual reward. The next set of experiments will allow the animal to control the virtual BMI in a manner similar to how we anticipate that the eventual application will be used: with no active movement of the subject's body parts. The animals will use the neural control of the environment to obtain rewards when they are seated in a monkey chair. We expect that the monkey will be able to generate periodic neural modulations associated with individual steps of the avatar even though it does not perform any actual steps with its own legs.
Finally, we will use the algorithms developed in these experiments to control a full-body monkey exoskeleton in a non-human primate which has been subjected to a spinal cord anesthetic block to produce a temporary and reversible state of quadriplegia. This exoskeleton will encase the monkey's arms and legs. It will be attached to the monkey using bracelets molded in the shape of the monkey's limbs. A full body exoskeleton prototype will be utilized. The basic design and controller will be based on the humanoid robot, Computational Brain (CB).27,86 The exoskeleton will provide full-sensory feedback to the BMI set up—joints position/velocity/torque, ground contacts and orientations. In BMI mode, the exoskeleton will guide the monkey's limbs with smooth motions while at the same time monitoring its range of motions to ensure it is within the safety limits. This demonstration will provide the first prototype of a neural prosthetic device that would allow paralyzed people to walk again.
CONCLUSIONSBMI technology offers a revolutionary treatment for paralysis. Recent studies suggest that BMIs have the potential to restore mobility to both upper and lower extremities and to enable a range of motor tasks, from arm reaching and grasping, to bipedal locomotion and balance. Moreover, it is feasible to enhance BMIs with an artificial somatosensory feedback either through ICMS or optogenetic stimulation. We envision that multidisciplinary BMI research will lead to the creation of whole-body neural prosthetic devices aimed at restoring full, essential mobility functions to paralyzed patients.