Augmentation of Sensorimotor Functions with Neural Prostheses

History of Neural Prostheses

Many would agree that modern Neuroscience started with the pioneering discoveries of Ramon Cajal (Finger 1994, Ramón y Cajal 1995) and Camillo Golgi (Golgi 1995) on the structure and connectivity of brain neurons, recognized by the Nobel prize in 1906. These founding fathers of Neuroscience principally disagreed in their views on the morphology and function of individual neurons. Cajal described single neurons as morphological entities. Golgi argued that neurons are not separated anatomically and do not work individually. He insisted that they are fused into a network. Although Golgi’s views were initially rejected, they turned to be valid at the end, as ion-conducting gap junctions were discovered between many types of brain neurons (Lewis and Rinzel 2003, Connors and Long 2004), the complexity that makes it difficult to describe brain networks as circuits composed of individual neurons similarly to the way electrical circuits are composed of transistors. The discovery of ephaptic coupling (Anastassiou, Perin et al. 2011), that is coupling by electrical fields produced by neurons, adds more complexity to brain network operations.
While many reports can be found in the literature of very specific response properties of single neurons, such as grandmother cells (Gross 2002) and Jennifer Aniston cells (Quiroga, Reddy et al. 2005), there is also a sizeable literature claiming that information is represented in the brain by large populations of neurons located in multiple neural regions (Houk and Wise 1995, Nicolelis and Lebedev 2009).
In motor neurophysiology, the studies of Apostolos Georgopoulos have been particularly influential to advance the population encoding ideas (Georgopoulos, Schwartz et al. 1986, Georgopoulos, Kettner et al. 1988, Georgopoulos 1994). Georgopoulus proposed a population vector model to explain how neuronal ensembles represent motor parameters. He also assessed the size of neuronal population needed to produce an accurate representation. However, Georgopoulus did not record from many neurons simultaneously; his conclusions were based on the analyses of neural data collected in a serial manner over many days. With this approach, information could not be extracted from a sufficiently large neuronal ensemble in real time.
A decisive development was achieved in the 90s by Miguel Nicolelis, John Chapin and several other researchers who pioneered techniques for multielectrode recordings (Nicolelis, Lin et al. 1993, Wilson and McNaughton 1993, Nicolelis, Ghazanfar et al. 1998). These recording methods form the foundation of modern invasive NPs. 
NPs are artificial systems for bidirectional communication with the brain (Figure 1). The main goal of NPs is the development of medical applications, such as neurally controlled limb prostheses for paralyzed patients. Additionally, NP approach finds applications in such areas as computer gaming (Mason, Bohringer et al. 2004, Finke, Lenhardt et al. 2009, Martisius and Damasevicius 2016), safety systems that monitor drivers’ state of wakefulness (Picot, Charbonnier et al. 2008, Liu, Chiang et al. 2013, Garces Correa, Orosco et al. 2014) and even education (Marchesi and Riccò 2013). Some view NP as a technology for augmenting brain functions (Maguire and McGee 1999, Farah and Wolpe 2004, Madan 2014, Zehr 2015). NPs are interchangeably referred to as brain-machine interfaces (BMIs) (Lebedev, Crist et al. 2008, Sakurai 2014), brain-computer interfaces (BCIs) (Allison, Wolpaw et al. 2007, Serruya 2015) and even biohybrids (Zehr 2015, Vassanelli and Mahmud 2016). Since NP interfere with the mind, they bring about many ethical issues (Attiah and Farah 2014, Glannon 2014, Schleim 2014, Hildt 2015, Kyriazis 2015).
Historically, the venture into reading out the brain content started in the 60-70s when the idea of biofeedback gained popularity (Dahl 1962, Smith and Ansell 1965, Kamiya 1971, Sterman 1973, Kaplan 1975, Suter 1977). Biofeedback of neural activity is often called neurofeedback. Such neurofeedback, usually provided by visual or auditory signals, gives subjects an ability to monitor and volitionally modify their own brain activity. For example, subjects can learn to modify their electroencephalographic (EEG) rhythms (Evans and Abarbanel 1999). 
The first demonstration that can be described as an NP dates to 1963, when Gray Walter demonstrated neural control of an external device by human subjects. Although Walter himself did not publish these results one can learn about them from the writings of Daniel Dennett who attended Walter’s lecture (Dennett 1991). Per Dennett, Walter recorded from motor cortical activity in human patients using implanted electrodes. The patients were instructed to press a button to advance a slide projector, the task that Walter used to investigate motor cortical readiness potentials that developed several hundred milliseconds prior to movement onset. Next, Walter electrically disconnected the button and switched to using the readiness potential as the trigger to advance the slides. The subjects were surprised that in this mode of operation the machinery detected their motor intentions before they initiated the hand movements. 
The goal of building NPs was explicitly formulated in the late 60s by Karl Frank, the chief of the laboratory of neural control at National Institutes of Health (NIH). He stated that the  laboratory would be developing systems that link the brain to external devices and computers (Frank 1968). Frank was also involved in collaborative research on NPs that restore vision to the blind. The NIH laboratory conducted their experiments in monkeys. They simultaneously recorded from 3-8 neurons in the motor cortex while monkeys flexed or extended their wrists. In offline analyses, the parameters of wrist movements were reconstructed from the neuronal data using multiple linear regression (Humphrey, Schmidt et al. 1970). This work continued for a decade and culminated in a demonstration of real-time decoding of cortical signals (Schmidt 1980): recordings were conducted using 12 electrodes chronically implanted in the motor cortex, and the implanted monkey learned to control one-dimensional movements of a cursor on an LED display with its cortical activity.
In the late 60s, Eberhard Fetz trained monkeys to volitionally control the activity of single neurons in their motor cortex (Fetz 1969). Fetz interpreted these findings in terms of neurofeedback. In these experiments, monkeys learned to modulate the discharge rate of their neurons, provided they had a visual or auditory indicator of those discharges. 
Around the same time, Michael Craggs used baboons with spinal cord transections to test the possibility of restoring motor function to paraplegic subjects (Craggs 1975). Prior to the spinal cord injury, the baboons were trained on a leg movement task Craggs discovered that, even after the motor cortical representation of the leg was disconnected from the spinal cord, it continued to generate task-related activity recorded with epidural electrodes. Accordingly, Craggs suggested that this type of recordings could be used for functional recovery of patients with leg paralysis.
While these scientists developed NPs for extraction of motor commands from the brain, other researcher groups started developing NPs for the delivery of sensory information to the brain. They electrically stimulated peripheral sensory nerves (Collins, Nulsen et al. 1960, Hensel and Boman 1960) and sensory areas of the brain (Libet, Alberts et al. 1964, Brindley and Lewin 1968, Brindley and Lewin 1968) to evoke artificial, but still recognizable sensations. Among these developments, a cochlear implant has achieved a spectacular success, with hundreds of thousands deaf people implanted with this device (Djourno and Eyriès 1957, Simmons, Mongeon et al. 1964, House 1976, Wilson and Dorman 2008). Some progress has been made in the development of visual NPs: the groups led by Giles Brindley(Brindley and Lewin 1968, Brindley and Lewin 1968, Brindley 1970) and William Dobell (Dobelle and Mladejovsky 1974, Dobelle, Mladejovsky et al. 1974, Dobelle, Mladejovsky et al. 1976, Dobelle, Quest et al. 1979) electrically stimulated visual cortex in blind patients. The stimulation evoked perceptions of light flashes, called phosphenes, and combinations of phosphenes produced by multi-channel stimulation could be matched to visual objects. The studies on visual NPs continue nowadays (Christie, Ashmont et al. 2016, Lewis, Ayton et al. 2016). There is also ongoing work on vestibular NPs (Shkel and Zeng 2006, Golub, Phillips et al. 2011).
As noted above, a significant breakthrough in the development of NPs occurred in the mid-90s, with the advent of chronic multielectrode implants (Buzsaki, Bickford et al. 1989, Nicolelis, Lin et al. 1993, Nicolelis, Lin et al. 1993, Nicolelis, Baccala et al. 1995). In 1999, Chapin, Nicolelis and their colleagues published a landmark study, where rats learned to control a simple robotic manipulator with their cortical ensembles (Chapin, Moxon et al. 1999). Following this success, Nicolelis commenced a series of NP studies in monkeys, including both New World and Old World species. A landmark study published in 2000, where owl monkeys controlled movements of a robotic arm with their cortical activity (Wessberg, Stambaugh et al. 2000). This work led to a series of monkey studies on NPs enabling arm movements (Serruya, Hatsopoulos et al. 2002, Taylor, Tillery et al. 2002, Carmena, Lebedev et al. 2003). 
Invasive NP research has been also conducted in humans. The multielectrode implant, called the Utah probe, is approved for human trials. This is a silicon-based matrix of needle electrodes in a 10x10 arrangement (Campbell, Jones et al. 1991, Nordhausen, Maynard et al. 1996, Maynard, Nordhausen et al. 1997). The other electrode approved for human recordings is the neurotrophic electrode developed by Philip Kennedy (Kennedy 1989, Kennedy, Bakay et al. 1992, Kennedy, Mirra et al. 1992). The electrode contains nerve growth factor that promotes neurite growth into the glass cone to which recording microwires are connected. Kennedy and his colleagues reported that this recording method allowed severely paralyzed patients to operate several types of NPs that restored their communication with the outside world (Guenther, Brumberg et al. 2009, Brumberg, Nieto-Castanon et al. 2010).
Noninvasive NPs, i.e. the devices with recording sensors that do not penetrate the body, experienced their own impressive development. These systems are not described in detail here. In brief, Jacques Vidal pioneered this research the 70s by decoding EEG evoked responses (Vidal 1973). In 1988, the first report was published where human subjects controlled a robot with their EEG (Bozinovski, Sestakov et al. 1988). In that study, subjects issued binary commands by closing and opening their eyes. This maneuver started and stoped an alpha recorded with EEG sensors placed over the occipital cortex.  These studies towards the development of an NP for disabled patients culminated in the publication by Niels Birbaumer of a pivotal study on an EEG-based spelling device for locked-in patients (Birbaumer, Ghanayim et al. 1999). The device utilized slow cortical potentials. 

Figure 1. Schematics of a neural prosthesis (NP) controlled by a rhesus monkey. A monkey is implanted with multielectrode arrays placed in multiple cortical areas. These cortical implants are used for both recording of neuronal ensemble activity and stimulating cortical tissue with electrical pulses. The monkey is seated in front of a computer screen that displays a virtual hand and several targets. The monkey explores the targets with the virtual hand. This exploration is performed either manually, using a joystick, or through the NP. In the NP mode of operation, cortical ensemble activity is processed by a decoder to generate kinematic parameters of the virtual arm. Each time the virtual arm touches a screen target, a pattern of electrical stimulation is applied to the somatosensory cortex. The stimulation mimics artificial textures of the targets. Thus, this NP enables bidirectional communication with the brain: motor commands are extracted from cortical activity simultaneously with the delivery of somatosensory feedback back to the brain. Aided with this NP, the monkey actively explores the targets, finds the one associated with a particular artificial texture, and receives a reward for the correct identification of that texture.

Types of Neural Prostheses 

Several classifications have been proposed to describe different types of NPs. NPs can be classified by function into: (1) motor NPs, (2) sensory NPs, (3) sensorimotor NPs, (4) cognitive NPs, and (5) brain-nets. Motor NPs generate movements, for example movements of artificial limbs (Wessberg, Stambaugh et al. 2000, Carmena, Lebedev et al. 2003, Velliste, Perel et al. 2008, Collinger, Wodlinger et al. 2013) or movements of a motorized wheelchair (Xu, So et al. 2014, Rajangam, Tseng et al. 2016). Sensory NPs evoke sensations using electrical (Romo, Hernández et al. 1998) or optogenetic (Jarvis and Schultz 2015, Kwon, Lee et al. 2015) stimulation of nervous tissue. Sensorimotor NPs, also called bidirectional, simultaneously produce movements and evoke sensations (O'Doherty, Lebedev et al. 2009, O'Doherty, Lebedev et al. 2011, Bensmaia and Miller 2014). Cognitive NPs (Andersen, Burdick et al. 2004, Andersen, Hwang et al. 2010) reproduce higher-order functions, notably  attention (Astrand, Wardak et al. 2014, Ordikhani-Seyedlar, Lebedev et al. 2016), memory (Berger, Hampson et al. 2011, Deadwyler, Hampson et al. 2013, Madan 2014, Song, Harway et al. 2014), and decision making (Musallam, Corneil et al. 2004). Brain-nets are NPs incorporating several interconnected brains (Ramakrishnan, Ifft et al. 2015). 
The other useful classification of NPs is the classification into invasive (Chapin, Moxon et al. 1999) and noninvasive (Birbaumer, Ghanayim et al. 1999, Waldert 2016) NPs. Invasive NPs provide better-quality neural recordings, but they carry risks to patients, including risk of tissue damage by invasive surgical procedures and electrode insertion, and the possibility of infection, particularly when recording cables pierce the skull and skin. Noninvasive NPs do not have such risks, but often suffer from low spatial and temporal resolution of the recorded neural signals.
NPs can be also classified by their operation principle into endogenous and exogenous devices. Endogenous NPs mimic “free will”:  users are free to choose the type and timing of actions. For example, in a motor imagery NP, users imagine moving their body parts to generate NP output (Obermaier, Neuper et al. 2001, Pfurtscheller and Neuper 2006). Exogenous NPs require an external stimulus to operate, and that stimulus paces the actions. The stimulus evokes a neuronal response, and the user task consists of volitionally controlling that response (Sellers, Krusienski et al. 2006, Lee, Sie et al. 2010). A popular exogenous NP utilizes P300 evoked potentials, which increase when the user attends to the stimulus (Donchin, Spencer et al. 2000, Finke, Lenhardt et al. 2009, Brunner, Ritaccio et al. 2011). 

Neural Representation of Information

Although we still have a rather poor understanding of how the brain represents and processes information, the term “neuronal encoding” is commonly used to describe the properties of neuronal discharges. Usually, what is meant by neuronal encoding is the correlation of neuronal discharge rate to a behavioral parameter or an external stimulus. For example, discharge rates of neurons in motor areas clearly correlate with limb kinematics, and the rates of neurons in visual areas correlate with the features of visual stimuli. Such a correlation is often referred to as “neuronal tuning”. 
Neurons tuned to a certain behavioral parameter could be used by a BMI designed to extract the same parameter. In neurophysiology, such neurons are called task-related neurons. Even the best task-related neurons represent behavioral parameters in a noisy way, which hinders BMI decoding. Decoding accuracy can be improved by extracting information from many neurons simultaneously (Fetz 1992, Nicolelis and Lebedev 2009, Lebedev 2014). Combining contributions from many neurons improves the signal to noise ratio because noisy inputs from different neurons cancel each other, unless this is a common noise.
The pioneering work on neuronal tuning was conducted by Edward Evarts who developed the technique of single-unit recordings from the brain of awake, behaving monkeys (Evarts, Bental et al. 1962, Evarts 1964, Evarts 1966). Evarts usually recorded from one neuron at a time using a sharp-tipped electrode that he inserted in monkey motor cortex. Monkey were trained to perform motor tasks. Evarts found that neuronal discharge rates reflected movement onsets and the force of muscle contraction.
Apostolos Georgopoulos used Evarts’ recording methods to explore the relationship between the discharges of motor cortical neurons and the direction of arm movement. He described that relationship as a broad tuning curve that could be fitted with a cosine function of movement direction angle (Georgopoulos, Schwartz et al. 1986, Georgopoulos, Kettner et al. 1988, Kettner, Schwartz et al. 1988, Schwartz, Kettner et al. 1988). 
With the advent of multielectrode recordings, investigators started to get more insights on the neuronal population encoding. It was experimentally that decoding accuracy improves with the neuronal population size (Blazquez, Fujii et al. 2002, Musallam, Corneil et al. 2004, Batista, Yu et al. 2008, Lebedev 2014, Montijn, Vinck et al. 2014). It was also discovered that the physiology of neuronal populations is governed by certain principles (Nicolelis and Lebedev 2009). Among these principles, single neuron insufficiency principle explains that each individual neuron encodes only a small amount of information. Mass effect principle states that a certain number of neurons (a mass) is required for the represented amount of information to stabilize. After the neuronal mass is reached, adding more neurons changes the information content very little, and many more neurons needed to be recorded for extracting new information. Distributed encoding principle asserts that many brain areas encode and process the same information; there is no localized processing in the brain. Multiplexing principle describes the property of individual neurons to represent many variables simultaneously. The conservation of firing principle states that the average discharge frequency of the neurons in the ensemble remains approximately constant, even when the brain state changes. A somewhat similar principle is called free energy principle (Friston, Kilner et al. 2006, Friston 2009, Friston 2010, Tozzi, Zare et al. 2016). The context principle states that ensemble activity patterns critically depend on the behavioral context: neuronal responses to the same stimulus could differ dramatically in different contexts. And, finally, the plasticity principle highlights the capacity of neuronal ensembles to adapt to new conditions and behaviors.

Recording Methods

Currently, microwire implants are the most popular method of neuronal ensemble recordings (Nicolelis, Ghazanfar et al. 1997, Kralik, Dimitrov et al. 2001, Schwarz, Lebedev et al. 2014). Microwires in such an implant can be individually movable or fixed. This method is suitable for recording neuronal activity in both the cortex and subcortical areas. The Utah array composed of silicon electrodes is another popular recording method (Campbell, Jones et al. 1991). As mentioned above, the Utah array has been approved for human trials.
New recording methods are constantly being developed. The main goals here are increasing the number of recording channels, minimizing tissue damage and increasing recording longevity. These goals are achieved using novel floating (Gualtierotti and Bailey 1968, Musallam, Bak et al. 2007, Neves, Orban et al. 2007, Spieth, Brett et al. 2011) and flexible (Takeuchi, Suzuki et al. 2003, Kozai and Kipke 2009, Hassler, Guy et al. 2011, Agorelius, Tsanakalis et al. 2015, Agorelius, Tsanakalis et al. 2015) implants. One promising method, called sinusoidal probe, uses thin, flexible electrodes with reduced motion relative to the nervous tissue (Sohal, Jackson et al. 2014).
Several microelectrodes designs improve the yield and quality of neuronal recordings. NeuroNexus microelectrodes increase the number of recording channels by having multiple contacts along the shaft (Najafi, Wise et al. 1985, Anderson, Najafi et al. 1989, Weiland and Anderson 2000, Vetter, Williams et al. 2004). Twisted bundles of four electrodes, called tetrodes, have enhanced capacity to discriminate single units (Recce and O’keefe 1989, Wilson and McNaughton 1993, Jog, Connolly et al. 2002, Santos, Opris et al. 2012).
Recently, a principally new recording method has been introduced, called neural dust (Seo, Carmena et al. 2015, Seo, Neely et al. 2016). Neural dust is composed of small (10-100 microns) sensors that detect bioelectrical potentials. The sensors communicate with an external transducer though an ultrasonic link. Each sensor has a piezoelectric element that reflects the ultrasound sent from the transducer, and the reflected signal changes depending on the electrical potential detected by the sensor.
Another electrode type, called endovascular electrode, penetrates the brain through the blood vessels. Endovascular nano-electrodes can penetrate into the brain capillaries without breaking the blood-brain barrier (Llinas, Walton et al. 2005). Additionally, larger endovascular electrodes can be placed into cerebral arteries to record neural signals similar to EEG recordings (Boniface and Antoun 1997). A multichannel endovascular probe, called stentrode, was recently developed (Oxley, Opie et al. 2016). The stentrode was introduced into the sheep brain and retained good recording quality for 190 days.
Optical recordings are another method that can be employed for sampling signals from neuronal populations. These methods utilize fluorescent markers that are sensitive to voltage (Tasaki, Watanabe et al. 1968, Patrick, Valeur et al. 1971, Grinvald, Frostig et al. 1988, Grinvald and Hildesheim 2004) or intracellular calcium (Smetters, Majewska et al. 1999, Stosiek, Garaschuk et al. 2003, Grewe, Langer et al. 2010).
Electrocorticographic (ECoG) recordings represent a minimally invasive method for recording cortical activity (Crone, Sinai et al. 2006, Leuthardt, Miller et al. 2006, Miller, Shenoy et al. 2007, Hill, Gupta et al. 2012). High-density ECoG grids offer a significantly improved resolution of recordings (Wang, Degenhart et al. 2009, Viventi, Kim et al. 2011, Bleichner, Freudenburg et al. 2016). 
Multichannel neuronal recordings usually require cables to connect the electrodes to external recording equipment (Chapin, Moxon et al. 1999, Wessberg, Stambaugh et al. 2000, Serruya, Hatsopoulos et al. 2002, Taylor, Tillery et al. 2002, Carmena, Lebedev et al. 2003). More recently, a variety of wireless recording methods has been developed (Obeid, Nicolelis et al. 2004, Morizio, Irazoqui et al. 2005, Borghi, Bonfanti et al. 2007, Chestek, Gilja et al. 2009, Harrison, Kier et al. 2009, Kim, Bhandari et al. 2009, Zippo, Romanelli et al. 2015).

Decoding Algorithms

Mathematical algorithms for decoding information from neuronal activity can be generally described as multiple-input, multiple-output (MIMO) models (Kim, Sanchez et al. 2006). The characteristics of a decoder are often set up using a training recording session. During that session, subjects either perform overt limb movements or passively the movements of an external actuator while imagining that they control those movements. The training session is needed to measure the relationship between the neuronal discharges and the behavioral parameters of interests. For example, if a subject moves the arm in different directions for some time, the decoder could be trained to extract arm kinematics from the neuronal activity. After the decoder is trained, the mode of operation can be switched to brain control, during which the decoder output controls an external device. Additionally, adaptive decoders can be used to adjust an ongoing brain control (Taylor, Tillery et al. 2002, Carmena, Lebedev et al. 2003, Ganguly and Carmena 2009, Li, O'Doherty et al. 2011, Orsborn, Dangi et al. 2012, Dangi, Gowda et al. 2014).
A great variety of neural decoders have been developed over the years (Schwartz, Taylor et al. 2001, Li 2014, Agorelius, Tsanakalis et al. 2015). The simplest, but also very effective algorithm is the linear model that represents output signals as weighted sums of neuronal firing rates (Wessberg, Stambaugh et al. 2000, Taylor, Tillery et al. 2002, Carmena, Lebedev et al. 2003, Wessberg and Nicolelis 2004) (Figure 2). For example, Georgopoulos’ population vector algorithm is a linear model that computes a vector some of unit vectors pointing in the neurons’ preferred directions and weighted by the neuronal frequencies of discharge (Georgopoulos, Schwartz et al. 1986, Georgopoulos, Kettner et al. 1988, Georgopoulos, Lurito et al. 1989). The population vector algorithm, however, is not optimal because it does not minimize the decoding error. A better method, the Wiener filter (Figure 2), is an optimal linear algorithm that minimizes the error using a well-known multiple linear regression approach (Haykin 2014). To calculate a parameter of interest at time t, the Wiener filter measures neuronal rates at multiple time points preceding t, called taps, lags or bins, and assigns separate weight for each neuron, and for each tap.
Kalman Filter (Kalman 1960, Kalman and Bucy 1961) is another popular decoding algorithm that has been employed in many NPs (Serruya, Shaikhouni et al. 2003, Patil, Carmena et al. 2004, Wu, Shaikhouni et al. 2004, Kim, Sanchez et al. 2006, Li, O'Doherty et al. 2009, Okorokova, Lebedev et al. 2015). The filter separates variables into the state variables (for example, arm position and velocity) and observed variables (neuronal discharge rates). The relationship between the state variables and neuronal rates is described by function called the tuning model, and the dynamical properties of the state variables are described by the state model. The Kalman filter uses both models to update the state variables data in discrete steps, for example every 50-100 ms. Each update consists of several steps. First, an estimation of new state is generated from the previous state using the state model. Next, an expectation of neuronal rates is derived from the estimated state and the tuning model. That expectation is compared with the observed neuronal discharges, and, based on this comparison, the state estimation is corrected. An improved filter, the unscented Kalman filter, accounts for nonlinear relationship between neuronal rates and state variables, and outperforms the classical Kalman filter (Li, O'Doherty et al. 2009). It has been suggested that the brain itself uses computations like the Kalman filter (Wolpert and Ghahramani 2000).
Several adaptive decoders have been developed to improve stability of decoding. One such algorithm let monkeys to control a virtual arm with their cortical activity for 29 days without the need to run training sessions (Li, O'Doherty et al. 2011). An alternative approach is to fix the decoder settings and allow the nervous systems to plastically adapt to improve the performance (Ganguly and Carmena 2009).
Decoders have been recently introduced that track the distance between the cursor and target of movement, and adjust their parameters to minimize that distance (Kowalski, He et al. 2013, Suminski, Fagg et al. 2013, Shanechi, Orsborn et al. 2014).
Reinforcement learning is yet another adaptive algorithm employed in NPs (DiGiovanna, Mahmoudi et al. 2009). This algorithm updates its parameters based on the success or errors of the behavioral trials. In one implementation of reinforcement learning, an error signal was extracted from the brain itself, namely from the activity of nucleus accumbens, making this NP a self-sufficient, unsupervised learning system (Mahmoudi, Pohlmeyer et al. 2013). 

Figure 2. Decoding neuronal ensemble activity using a linear decoder (Wiener filter). For a time of interest, t, neuronal firing rates are measured in a time window preceding t. The window is split into several bins, also called taps; firing rates are measured within each bin. A behavioral parameter of interest (for example, arm coordinate) is then represented as a weighted sum of neuronal rates for different bins. The weights are calculated using the well-known multiple linear regression methods.

Neurally Controlled Prosthetic Arms

The development of neurally controlled prostheses of the upper limbs has been one the main directions of NP research (Carmena, Lebedev et al. 2003, Hochberg, Bacher et al. 2012, Collinger, Wodlinger et al. 2013). Such interest to the upper limb function is understandable because of the role that arm movements have in our motor repertoire. The first demonstration of the control of a robotic arm by a primate was performed using owl monkeys as an experimental model (Wessberg, Stambaugh et al. 2000). That was an open-loop brain control because monkeys did not receive any sensory feedback from the robot (Wessberg, Stambaugh et al. 2000). While the monkeys performed a reaching task with a joystick, activity of their motor cortical neuronal ensembles was recorded, decoded with a Wiener filter and sent to the robot. The robot reproduced the joystick movements with some errors, which the monkeys obviously could not correct.
Brain control in a closed-loop mode (i.e. with a vision of the robot or cursor, or other type of sensory feedback) was first demonstrated in rhesus monkeys (Serruya, Hatsopoulos et al. 2002, Taylor, Tillery et al. 2002, Carmena, Lebedev et al. 2003). Jose Carmena and his colleagues chronically implanted rhesus monkeys with mulielectrode arrays in multiple cortical areas, and trained the animals to control the movements of a robotic arm with the recorded cortical activity (Carmena, Lebedev et al. 2003). The robotic arm performed reaching and grasping movements. Monkeys started with controlling the robot arm manually, using a two-dimensional joystick that could be also squeezed to generate grip force of the robot. The monkeys did not have vision of the robot, but received visual feedback from it on a computer screen. The robot position was indicated by a computer cursor, and the grip force was indicated by the cursor size. Reach targets showed up on the screen, as well. While the monkeys performed the task manually, three Wiener filters were trained to extract X and Y components of joystick velocity and the grip force from cortical ensemble activity. Next, the joystick was electrically disconnected from the robot, and the Wiener filters’ outputs controlled the robot instead. The monkeys continued to assist themselves by moving the joystick for some time during this brain control mode. The joystick was then physically removed from the setup, after which the performance accuracy initially dropped but then improved.
A somewhat similar experiment was conducted by Dawn Taylor and her colleagues at the laboratory of Andrew Schwartz (Taylor, Tillery et al. 2002). In that study, monkey wore stereoscopic glasses that displayed a cursor in a three-dimensional space. The cursor position was controlled by the motor cortical activity using a population vector decoder. A coadaptive algorithm was employed to improve the decoding. The coadaptation consisted of comparing the cursor trajectories to the ideal trajectories connecting the starting position with the target, and adjusting the population vector weights to bring the trajectories closer to the ideal ones. In the next study of the Schwartz laboratory, monkeys controlled a robotic hand that grasped pieces of food and brought them to the monkey’s mouth (Velliste, Perel et al. 2008). 
John Donoghue and his colleagues demonstrated real-time cortical control of a computer cursor and robotic hand in human patients (Hochberg, Serruya et al. 2006). Paralyzed human subjects received Utah probes in the motor cortex, which allowed to record several tens single-units. Several years later the same group demonstrated a neuroprosthetic arm that picked up a coffee bottle and brought it to patient’s mouth (Hochberg, Bacher et al. 2012). In that experiment, the performance was assisted by shared control, where some operations were handled by a robotic controller instead of the patient. The neural part of the control was handled by the Kalman filter.
Recently, Andrew Schwartz and his colleagues recorded several hundreds of single units in the human motor cortex (Collinger, Wodlinger et al. 2013). With this improved recording quality, patients learned to control a seven degrees of freedom robotic arm that reached toward knobs, grasped them, and turned in different directions.
Peter Ifft and his colleagues reported a further achievement in NPs for arm control (Ifft, Shokur et al. 2013). In those experiments, monkeys controlled two virtual arms simultaneously that performed center-out reaching movements towards two separate targets for each arm. Approximately five hundred neurons were recorded in multiple cortical areas, and an unscented Kalman filter was used for decoding.

Neural Prostheses for Restoration of Locomotion

Invasive NPs for the control of lower limbs have remained underdeveloped for some time. That was because most neurophysiological studies in nonhuman primates have traditionally focused on the upper limb tasks, whereas the control of the lower limbs remained virtually neglected. Only several years ago, NPs have started to develop for restoration of legged locomotion (Cheng, Fitzsimmons et al. 2007, Bouyarmane, Vaillant et al. 2014), and currently we are witnessing a rapid rise in such NPs.
Nathan Fitzsimmons and his colleagues recorded from sensorimotor cortical ensembles in monkey trained to walk bipedally on a treadmill (Fitzsimmons, Lebedev et al. 2009). While the monkeys performed the walking, movements of their lower limbs were monitored using a video tracking system (Peikon, Fitzsimmons et al. 2009). Using these recordings as a training data, multiple Wiener filters were set to reproduce the lower limbs kinematics from the cortical recordings. The Wiener filters also reproduced EMGs of the lower limb muscles. Both forward walking and backward walking were decoded. Next, the researchers sent the decoded kinematic parameters of monkey walking to Kyoto, Japan, where a humanoid robot reproduced the monkey gait at the laboratory of Mitsuo Kawato (Cheng, Fitzsimmons et al. 2007, Kawato 2008). 
These findings highlight the fact that invasive cortical recordings can provide highly efficient signals for the control of devices that restore walking, for example for exoskeletons, such as ExoAtlet (Figure 3). Exoskeletons controlled by invasive NPs, such as cortical microelectrode recordings and ECoG, most certainly will emerge in the near future because both the recording methods (Leuthardt, Miller et al. 2006, Collinger, Wodlinger et al. 2013, Schwarz, Lebedev et al. 2014) and exoskeleton technologies (Farris, Quintero et al. 2012, Frolov, Biriukova et al. 2013, Lisi, Noda et al. 2014, Wall, Borg et al. 2015, Onose, Cârdei et al. 2016) already exist.
One interesting strategy in NPs for locomotion is using cortical signals as a control signal to an electrical stimulator to the spinal cord that evokes walking automatism. The feasibility of such a system was recently demonstrated in a study conducted in rhesus monkeys with partial spinal cord injuries (Capogrosso, Milekovic et al. 2016). In that study, monkeys with spinal cord lesions attempted to walk quadrupedally but experienced deficits in the leg ipsilateral to the lesion site. The researchers alleviated this deficit by decoding the step cycle from motor cortical activity and triggering the spinal cord stimulation at the appropriate phases of the cycle. The stimulation induced near-normal stepping movements in the impaired leg.
Currently, the wheelchair still remains the main means of locomotion for paralyzed patients. Here, invasive NP technology could come handy, particularly for severely paralyzed patients who cannot use their upper limbs to control the wheelchair. Although noninvasive NPs for wheelchair control already exist (Moore 2003, Craig and Nguyen 2007, Galán, Nuttin et al. 2008), invasive NPs could offer much better information transfer rate, reaction time and versatility. A pioneering study of an invasive NP for wheelchair control was conducted by Rajangam and her colleagues who demonstrated that rhesus monkeys could navigate while seating on top of a motorized wheelchair and steering it with their motor cortical activity (Rajangam, Tseng et al. 2016). For this purpose, two Wiener filters were trained; one controlled the linear velocity of the wheelchair (i.e., back and forth movements), and the other controlled rotational velocity (i.e., wheelchair turns). 

Figure 3. The ExoAtlet. This is an electrically actuated exoskeleton that assists patients suffering from leg paralysis. The ExoAtlet allows to set the stepping parameters and enacts several bipedal states, such as standing, walking on different surfaces and stepping over obstacles. Reproduced with permission from Ekaterina Bereziy, exoatlet.ru.

Controlling Patient’s Own Muscles

An alternative to using robotic devices is the possibility to reanimate patient’s paralyzed body using functional electrical stimulation (FES) of the muscles. Several FES-based NPs have been already developed.
Efficient NPs of this type should be able to decode muscle-like patterns from the brain activity. The feasibility of such decoding was demonstrated using simultaneous recordings of cortical activity and arm EMGs in monkeys (Morrow and Miller 2003, Santucci, Kralik et al. 2005, Pohlmeyer, Solla et al. 2007, Fitzsimmons, Lebedev et al. 2009). In these experiments, linear decoders successfully reconstructed EMG patterns from cortical activity. Furthermore, experiments in humans showed that multi-channel FES of hand muscles evoked  a variety of movements that approximated normal hand movement(Seifert and Fuglevand 2002, Johnson and Fuglevand 2011).
The first demonstration of a FES-based NP involved EEG recordings (Pfurtscheller, Müller et al. 2003, Pfurtscheller, Rupp et al. 2005). Aided by this NP, a patient learned to control an FES device that animated the paralyzed hand. The patient was able to grip and translate objects. Invasive FES-based NPs were demonstrated in monkeys (Moritz, Perlmutter et al. 2008, Pohlmeyer, Oby et al. 2009, Ethier, Oby et al. 2012). In these experiments, monkey arms were temporarily paralyzed by local anesthetics applied to the nerves. Neuronal activity was recorded in the motor cortex and converted into FES patterns. Monkeys were able to perform motor tasks by putting their arms into action with the FES. And finally, an invasive, FES-based NP restored mobility to the hand of a human patient with a complete spinal cord injury (Bouton, Shaikhouni et al. 2016).

Artificial Somatosensory Sensations

Sensory NPs transmit information from the outside world to the brain (Dobelle 1994, Lebedev and Nicolelis 2006, Nicolelis and Lebedev 2009, Rothschild 2010, Lebedev, Tate et al. 2011, Bensmaia and Miller 2014). Such NPs are intended for people with sensory disabilities. Given the large number of sensory modalities, one can imagine a variety of NPs that restore sight, hearing, tactile sensations, etc. by sending the appropriate information to the corresponding sensory areas of the brain. Sensory NPs could be interfaced to different levels of sensory hierarchy: to the peripheral nerves, spinal cord, thalamus and cortex. Ideally, sensory NPs should account not only for the bottom-up flow of information from the peripheral receptors to the brain, but also for the top-down, anticipatory communications that are known to play an essential role in sensory processing (Lebedev, Denton et al. 1994, Siegel, Körding et al. 2000, Ghazanfar, Krupa et al. 2001, Krupa, Wiest et al. 2004, Gilbert and Sigman 2007, Pais-Vieira, Lebedev et al. 2013, Pezzulo, D'Ausilio et al. 2016). 
Several NPs have been proposed for restoring somatosensory sensations. This work is rooted in the pioneering discoveries made in the beginning of the 20th century on the effects of electrical stimulation of the brain. In 1909, Harvey Cushing discovered that electrical stimulation of the human cortex can evoke somatosensory percepts without inducing limb movements (Cushing 1909). The sensory effects of electrical stimulation were subsequently studied in great detail by Wilder Penfield (Penfield and Boldrey 1937). Penfield’s patients reported sensations of numbness, tingling, and rarely pain after their somatosensory cortex was electrically stimulated with surface electrodes.
Modern stimulation methods are based on microstimulation, that is injection of small currents into the brain tissues using thin electrodes (Bartlett and Doty 1980, Fitzsimmons, Drake et al. 2007, Kim, Callier et al. 2015). Romo and his colleagues employed microstimulation of monkey primary somatosensory cortex to induce sensations comparable by those evoked by vibrotactile stimulation of the hands (Romo, Hernández et al. 1998). Fitzsimmons and his colleagues pioneered the usage of chronic cortical implants for the same purpose (Fitzsimmons, Drake et al. 2007). In those experiments, owl monkey learned progressively more complex discrimination tasks, starting from detecting the mere presence of microstimulation, then discriminating different temporal patterns of microstimulation, and finally discriminating spatiotemporal patterns delivered to the somatosensory cortex through several pairs of electrodes. 
Bidirectional NPs (Figure 1), opened a new chapter in the development of sensory NPs. These systems simultaneously extract motor commands from the brain motor areas and deliver artificial sensory feedback to the sensory areas. O’Doherty and his colleagues implemented bidirectional NPs in monkeys (O'Doherty, Lebedev et al. 2009, O'Doherty, Lebedev et al. 2011). In these experiments, monkeys controlled a virtual arm shown on a computer screen with their motor cortical activity. The monkeys’ task was to search through an array of screen targets with the virtual hand. The targets were visually identical, but they were associated with different artificial tactile sensations produced by microstimulation of the primary somatosensory cortex. The microstimulation started when the monkey placed the virtual hand over a target. Using such a bidirectional NP, monkeys were able to quickly search through up to three targets displayed on the screen.

Brain-Nets

Brain-nets represent a futuristic development in NPs. These are NPs that incorporate several brains that work like a super-brain, and potentially could even work as a global brain (Kyriazis 2015). The brains included in a brain-net can perform cooperative tasks and exchange information with each other. A pioneering brain-net experiment was conducted in rats (Pais-Vieira, Lebedev et al. 2013). One rat performed a motor task and acted as a transmitter because its brain activity, after moderate processing with a sigmoidal transfer function, was passed to another rat, called receiver. Microstimulation was applied to the receiver’s sensorimotor cortex to deliver the information.
Brain-nets can connect different species, for example they can connect the human brain to animal brain. In one such experiment neural information was transmitted from a human operating an EEG-based NP to an anesthetized rat (Yoo, Kim et al. 2013). The transmitted command triggered an ultrasound stimulator that activated the rat motor cortex and evoked tail movements. In another experiment, information was transferred from the human brain to cockroach brain (Li and Zhang 2016).
Information exchange was also carried out between two human brains. Grau and his colleagues conducted a study, where one subject operated an EEG-based NP, while the second received messages in the form of transcranial magnetic stimulation (TMS) of the visual cortex that evoked phosphenes (Grau, Ginhoux et al. 2014). In a similar experiment, Rao and his colleagues had one subjects operate an EEG-based NP. Whereas TMS was applied to the second subject’s motor cortex, and evoked hand movements (Rao, Stocco et al. 2014). In an even more advanced demonstration, the same group enabled humans to read the mind of the other humans using an interactive question and answer game (Stocco, Prat et al. 2015).
Yet another type of a brain-net, called brain plus the brain interface, was implemented in rhesus monkeys (Ramakrishnan, Ifft et al. 2015). That interface assisted collaboration between the subjects. Several monkeys (two or three) contributed their cortical signals, which resulted in a better control of a single virtual arm.

Conclusion

Overall, we have seen a significant progress in the field of invasive NPs. Improvements in neural recordings methods allow sampling signals of better quality from a larger number of channels. The high channel count translates into improved neural decoding and more accurate control of external devices. Sensory and bidirectional NPs have been developed with the goal of assisting patients with sensory disabilities. Moreover, brain-nets connect the nervous systems of several participants into a higher-order circuit. These trends in invasive NPs will be translated in the future into multiple benefits for the humanity