A new Nature Neuroscience study decodes speech and body language from the same patch of cortex. The hard part is no longer reading the signal. It is deciding what the signal is allowed to move.
The study
On September 14, a team led by Edward Chang at the University of California, San Francisco published a paper in Nature Neuroscience with a plain title: simultaneous speech and gesture decoding for multimodal communication in paralysis [1]. The National Institutes of Health, which helped fund the work, describes it as the first brain-computer interface to decode speech and upper-body gestures at the same time. Earlier systems handled one mode or the other [1].
The setup is worth stating carefully, because the details are where the honest story lives. Three people with severe paralysis took part, and results were reported for two. One had a stroke and the other has ALS. Each received an implant of 253 electrodes lying on the surface of the sensorimotor cortex [1][2]. They tried to speak up to ten phrases and perform up to ten gestures, such as waving or clapping, separately and together. Machine learning models decoded the activity and drove a digital avatar, which moved its body and showed the speech as text on screen [2][3].
The scientific surprise is that the combined signal is not a simple sum of the two parts. The NIH release says there was overlap, but the patterns differed, and decoders trained on simultaneous attempts did better on mixed expressions than decoders trained on isolated ones [1]. Fast Company adds that some electrodes responded to speech only, some to gestures only, and some to both [3]. Samantha Brosler, a study author, gave the human reason this matters. Saying maybe while nodding means something very different from saying maybe while shaking your head [2].
What the numbers do and do not say
Do not let the headline do your thinking. The results are proof of concept. For the stroke participant, accuracy for both speech and gestures reached 100 percent across three rounds of conversation tests. For the participant with ALS, it averaged about 85 percent for gestures and 75 percent for phrases across five rounds, a gap the researchers tentatively relate to the different causes of paralysis [2]. The NIH release reports no accuracy figures or long-term outcomes at all [1].
Outside scientists were measured. Daniel Rubin called it an important step toward naturalistic systems. Christian Herff pointed out that the level of speech decoding is below what has been reported before, and that the system decoded isolated gestures and sentences rather than continuous conversation [2]. Brosler said herself that we are still a long way from seamlessly controlling an artificial body [3]. The system was wired to external processing, and the team plans to test a fully implantable wireless version next [1]. Science News, surveying the broader field, notes that changing neural signals over time are a real challenge and not yet solved [4].
So here is the sober reading. This is a small study with real limits and a real direction. The direction is what interests me, because it shows how quickly the engineering question gets answered once the signal is understood.
The line that should make you stop
In the Fast Company interview there is a sentence that most readers will skim past. Asked about future uses such as robotic limbs, Brosler said the decoder itself does not necessarily need to change depending on how its outputs are used [3].
Think about what that means. The system reads intent from a region of cortex. What happens next, whether it animates an avatar, types text, speaks through a voice, or moves a prosthetic, is a choice made downstream. The researchers describe the long-term goal as a whole-body interface that avoids limiting people to controlling one function at a time [3]. That is a generous ambition, and it is also a description of a general-purpose input device wired to a human nervous system.
I spent my career watching what happens when a general-purpose input channel meets a growing list of things it can control. The channel is never the problem. The problem is that the list of controlled things grows, and nobody writes down who agreed to each addition. A keyboard that can only type is boring. A keyboard that can also open doors, move money and drive machinery needs rules about which keystrokes do what, for whom, and who can switch each of those off.
Permission belongs at the output
Consent in neurotech is usually framed at the signal. Did the person agree to have their brain activity recorded and decoded? That question matters, and international standards say so. UNESCO’s recommendation on the ethics of neurotechnology, adopted in November 2025, requires prior, free and informed consent for collecting and processing neural data and asks states to treat it as sensitive personal data. It is soft law and not binding, and the commentary I read doubts whether states will make it enforceable [5]. At the national level, the picture is thinner. Colorado, California and Montana have passed neural data privacy laws, but the reporting available notes they target consumer devices and does not say whether they reach implanted clinical devices [6].
All of that is about the data. This study points at a different gap, which is the action. Once a decoder can drive more than one output, consent has to attach to each output, not only to the recording. Agreeing to animate an avatar for a research study is not agreeing that the same decoder may drive a prosthetic limb, trigger a smart home, or authorize a purchase. A person whose only channel to the world is this signal cannot easily say no on the fly. The default has to be that a new output requires a new decision, made deliberately by the person, and recorded.
That sounds bureaucratic until you remember the alternative. In every other domain of computing, we learned that capabilities accumulate silently when nobody owns the list. Scripts gain permissions. Integrations get added for convenience. Years later no one can say what a given credential can reach. A neural interface that grows outputs the same way would be far harder to audit, because the thing being asked to approve the change is the same signal the system is reading.
Attempted, imagined, and unspoken
There is a second detail that deserves attention. Fast Company reports that participants chose strategies they could sustain for fatigue reasons. One imagined gestures, and another silently attempted speech [3]. The decoder therefore reads activity that does not produce any visible movement.
That is the whole point of the technology for someone who cannot move. It is also why the boundary between what a person means to do and what a person is merely thinking becomes a design question. The NIH release adds a caution from earlier research: attempting speech and gesture together can reduce speech capability [1]. Trying to do both at once costs something. Engineers will tune the system to reduce that cost, and tuning decisions are choices about what counts as intent.
UNESCO’s text says neurotechnology should never be used for undue influence or manipulation that compromises autonomy [5]. I would add a corollary from the security side. Any system that interprets intent will sometimes misread it, and the consequences of a misreading grow with the power of the output. A false positive on an avatar wave is a minor embarrassment. A false positive on a motorized limb or a payment is something else. The more capable the output, the higher the bar the signal should have to clear, and the more important it is that the person can stop an action quickly.
The off switch problem
Which brings me to the control nobody likes to discuss. How does a person with severe paralysis revoke an output? If the decoder reads the only channel they have, then stopping it requires sending a stop signal through that very channel, on a system that may be misreading them at the time.
This is not a hypothetical detail. Science News reports that Casey Harrell, who has ALS, uses his speech interface ten to fourteen hours a day and has communicated nearly two million words, according to a Nature Medicine report from June [4]. Researcher David Brandman estimates he is about five years from offering an implant to a patient with ALS [4]. Technology that people live inside for most of their waking hours cannot rely on a stop control that only works when the decoder is working well. It needs a stop method with its own channel, one that fails safe, such as a physical control a caregiver can operate, or a trained fallback that requires less precision than the primary decoder.
This is ordinary safety engineering. Industrial machines carry emergency stops precisely because the operator’s judgment cannot be the only barrier. When the machine is a person’s own body or voice, the engineering is harder and the ethics are higher, and that is no reason to leave it for later.
What to ask before the architecture sets
The researchers are doing careful, humane work, and nothing here is a criticism of the study. The criticism is of the timeline. Wireless implantable versions are next, and devices that leave the lab tend to harden their architecture around the first decisions made. Decisions about which outputs a decoder may drive, who authorizes new ones, and how a stop works are far easier to make now than after thousands of units exist.
I would put four questions to any team building a multi-output interface. Which outputs is the decoder wired to today, and who decided? Who can add an output, and does that person have to be the patient? What is the stop mechanism, and does it work when the decoder fails? And what is logged about each action the decoder triggers, so that the person, not only the manufacturer, can review it?
The first generation of these systems restores a voice and a wave. That is the right place to start and a good reason to be optimistic. The second generation will connect to things, because that is what generations do. The responsible thing is to treat the connection as the decision that needs consent, and to write the rules while the list of outputs is still short enough to read in one sitting.
About the Author
Len Noe is a Solutions Architect at BeyondTrust, Transhuman, Podcaster, International Cyber Security Speaker, Author, Technical Evangelist, and Biohacker with 11 implanted microchips. A former blackhat and ex-1% motorcycle club member with more than 30 years in technology, he has presented in over 70 countries and is featured in the documentary I Am Machine, which premiered at DEF CON 2025.
References
- National Institutes of Health. “Neuroprosthesis for paralysis enables simultaneous speech and body language” (news release on Brosler et al., Nature Neuroscience, DOI 10.1038/s41593-026-02446-2). NIH, September 14, 2026. nih.gov
- Smithsonian Magazine. “This Brain Implant Allows People With Paralysis to Speak and Gesture at the Same Time via a Mind-Controlled Virtual Avatar.” Smithsonian Magazine, October 6, 2026. smithsonianmag.com
- Jesus Diaz. “This AI-powered brain implant hints at the future of human interaction.” Fast Company, October 9, 2026. fastcompany.com
- Laura Sanders. “Brain implants are helping people who’ve lost their voice.” Science News, October 15, 2026. sciencenews.org
- “Neurorights and neural data: the UNESCO Recommendation on the ethics of neurotechnology, the first global standard protecting the mind.” Law and Technology (secondary commentary on the Recommendation adopted November 11, 2025). lawandtechnology.eu
- Kate Ruder. “States Pass Privacy Laws To Protect Brain Data Collected by Devices.” KFF Health News, July 23, 2025. kffhealthnews.org