Monkey mindpong - playing game with mind

[intotheblack]

Elon Musk's brain-chip company -  ‘Neuralink’ released a video of  Pager the macaque playing video games with his mind

https://www.google.de/amp/s/www.bbc.co.uk/news/amp/technology-56688812

Edited April 11, 2021 by intotheblack

[Florian]

brain-chimp company

[Rilles]

Just now, Florian said:

brain-chimp company

Pure chimpery - Leo Gura

[Forestluv]

Amazing new technology. In a way, it might be easier for a chimp to learn it than a human. Chimps just do it. That chimp is in a meditative “zone”. They don’t have extra mind noise like “Am I doing it right? I can never learn this. I’d rather be watching YouTube videos”.

Also, notice at the end how the monkey isn’t moving his hand at all. He isn’t feinting any arm/hand movements as if it was a cursor. This means the neuralink has gone prior to the brain sending neural signals from the brain to motorneurons. They are able to detect the brains intention prior to the motor output. And there are no longer any motor signals sent, which is amazing.

I’m curious how this would be subjectively experienced. Would someone simply need to imagine the pong paddle moving? Or would it be like imagining I was moving a controller to move the paddle? By the look of the monkey, it seems like he simply imagines the computer pixels moving. That would be like imaging text being created on your screen and it appears. 

[Forestluv]

A good video that explains recent neuralink advancements

 

[Loving Radiance]

That stuff is insane and precisely why I want to work in neuroscience.

[Willie]

11 hours ago, intotheblack said:

Elon Musk's brain-chip company -  ‘Neuralink’ released a video of  Pager the macaque playing video games with his mind

https://www.google.de/amp/s/www.bbc.co.uk/news/amp/technology-56688812

I misread this and thought the macaque’s name was Prager, and it was hilarious.

[intotheblack]

12 minutes ago, Willie said:

I misread this and thought the macaque’s name was Prager, and it was hilarious

Ha.  I thought the name was just Paige when I first heard it. 

Edited April 11, 2021 by intotheblack

[intotheblack]

3 hours ago, Forestluv said:

A good video that explains recent neuralink advancements

 

Cool! 
 

4 hours ago, Forestluv said:

Amazing new technology. In a way, it might be easier for a chimp to learn it than a human. Chimps just do it. That chimp is in a meditative “zone”. They don’t have extra mind noise like “Am I doing it right? I can never learn this. I’d rather be watching YouTube videos”.

Also, notice at the end how the monkey isn’t moving his hand at all. He isn’t feinting any arm/hand movements as if it was a cursor. This means the neuralink has gone prior to the brain sending neural signals from the brain to motorneurons. They are able to detect the brains intention prior to the motor output. And there are no longer any motor signals sent, which is amazing.

I’m curious how this would be subjectively experienced. Would someone simply need to imagine the pong paddle moving? Or would it be like imagining I was moving a controller to move the paddle? By the look of the monkey, it seems like he simply imagines the computer pixels moving. That would be like imaging text being created on your screen and it appears. 

Yeah i wonder how that would work on a human, they could just change their thoughts to try and trick the computer. 
but then again, it’s about movement.  We don’t think about moving our body it just happens.  
I’m trying to understand is it moving by vision and the monkey is looking at where he wants the blocks to move?  It’s really crazy.  
imagine in the future we can do things just by thinking/looking at something, like telekinesis 

Edited April 11, 2021 by intotheblack

[Forestluv]

1 hour ago, intotheblack said:

We don’t think about moving our body it just happens.  
I’m trying to understand is it moving by vision and the monkey is looking at where he wants the blocks to move?  It’s really crazy.  
imagine in the future we can do things just by thinking/looking at something, like telekinesis 

Based on the steps shown in the video, it doesn't look like it's through the visual system of the brain (although that is inter-connected). Here is my impression of their process.

Step 1: Teach the monkey how to use the controller so the dot moves onto the box. Every time the dot and square meet, the monkey gets a little banana smoothie. 

Step 2: Use an algorithm to analyze brain activity as the monkey moves the controller. The algo would need to filter out all the background neural activity (such as brain activity driving swallowing, breathing etc). The algo learns which activity specifically means "press up", "move right" etc. This is termed "decoding".

Step 3: test the decoding accuracy by having the new link be: monkey brain intention to move controller up => algo decodes this as intention to move controller up => computer algo moves paddle up. This takes out the controller step. Yet the monkey is still using an unplugged controller. 

Step 4: Remove brain activity / motor neuron activity. Once the algo can decode intentions, muscle movement is extraneous. So they taught the monkey that the brain no longer needs to send the impulses down the spinal cord to muscles.

I'm very impressed the neuralink scientists have gone prior to muscle movement and they did it so cleanly. They cleanly removed all the steps that connect the thalamus, spinal cord and motor neurons. It's simply the intention to move the paddle in the video game. And they did this in six weeks. . . Six weeks!!! 

I'm super curious what this would feel like to a human. Would it feel like imagining moving or would it feel like imagining what you want manifested. For example, if someone wanted to type text - would they imagine typing on a keyboard or simply think the words? At somepoint, would it be automatic, like typing? I just think what words I want to appear on the screen and my fingers move in a way to make that happen. I'm not thinking about moving my fingers in anyway, I'm not thinking "press the "b" button, press the "k" button. I'm just thinking about the words I want to appear. and they 'magically' appear on the screen without me having to think about moving my fingers. So I wonder after a while, if it becomes automatic for the person. You just intend it and it appears.

The implications of this would be extraordinary. This would go way beyond simply moving a cursor on a computer screen. 

 

[Loving Radiance]

19 hours ago, Forestluv said:

Step 4: Remove brain activity / motor neuron activity. Once the algo can decode intentions, muscle movement is extraneous. So they taught the monkey that the brain no longer needs to send the impulses down the spinal cord to muscles.

That is unclear to me. Did they physically disconnect the neurons that control hand movement? If so, I think that the fact of physically disconnecting the neurons was essential for the monkey to recognize that the courser moves just from intention alone. It would be interesting to see what happens when Neuralink is removed and the neurons reconnected.

Quote

if someone wanted to type text - would they imagine typing on a keyboard or simply think the words? At somepoint, would it be automatic, like typing? I just think what words I want to appear on the screen and my fingers move in a way to make that happen. I'm not thinking about moving my fingers in anyway, I'm not thinking "press the "b" button, press the "k" button. I'm just thinking about the words I want to appear. and they 'magically' appear on the screen without me having to think about moving my fingers. So I wonder after a while, if it becomes automatic for the person. You just intend it and it appears.

I intuit that the algorithm recognizing the movement would be possible after one makes it muscle memory / automatic.

Quote

The implications of this would be extraordinary. This would go way beyond simply moving a cursor on a computer screen. 

Just imagine playing games or creating art just from thought. I can imagine training the Neuralink for years and it allows for cross-combination of intentions allowing more expression of creativity. Radical degrees of freedom allow for radical creativity beyond what you can currently imagine.

Edited April 12, 2021 by Loving Radiance

[Forestluv]

2 hours ago, Loving Radiance said:

That is unclear to me. Did they physically disconnect the neurons that control hand movement? If so, I think that the fact of physically disconnecting the neurons was essential for the monkey to recognize that the courser moves just from intention alone. It would be interesting to see what happens when Neuralink is removed and the neurons reconnected.

The generalized pathway is 1) motor cortex in brain -> 2) spinal cord neurons -> 3) motor neurons -> 4) muscle -> 5) controller -> video game

They didn't physically "remove" anything. By "remove", I mean take out steps 2-4. They were able to decode electrical brain activity in the motor cortex. For example: five 80mHZ 'blips' means 'move hand to the right'. The algo is able to decode the message sent to the motor neurons, so steps 2-4 is unnecessary. Neurons in the spinal cord, motor neurons and muscle no longer need to fire.

The new pathway is 1) motor cortex in brain -> 2) neuralink sensor -> 3) neural link decoder -> 4) video game.

All the motor neurons are still there, they just aren't firing because there is no need to. Over time, the monkey learned that he didn't need to send signals to his muscles. Yet if they disable the neuralink and put back the controller, the monkey would realize he needs to start moving his hand again to play the game and get the banana smoothie. 

Imagine that a person was conditioned to believe that to blink their eyes, they must pull their eyelids up and down with their fingers. Then one day, they realize all they have to do is think about blinking - the fingers are unnecessary. They could still use their finger if they want, yet why bother? It's extra motion that is unneeded. 

2 hours ago, Loving Radiance said:

Just imagine playing games or creating art just from thought. I can imagine training the Neuralink for years and it allows for cross-combination of intentions allowing more expression of creativity. Radical degrees of freedom allow for radical creativity beyond what you can currently imagine.

The neuralink output goes to a computer, so someone could play video games or create digital art with their imagination. Yet they couldn't create traditional art with regular paintbrushes and canvas. For that, they would need to add in robotic arms to pick up an move the painbrushes. For digital art, the neural link can potentially have very high resolution, yet for traditional art, it would be hard for a robotic arm to compete with the fine motor capacity of a hand. I can see things like neuralink controlling robotic arms to prepare food for a meal, which is pretty crude. Yet creating detailed art via a robotic arm would be very hard to do. 

Yet, I see digital art via neural link being amazing. 

 

[Loving Radiance]

17 hours ago, Forestluv said:

Over time, the monkey learned that he didn't need to send signals to his muscles.

Excuse my geekness, how does that look like on a cellular level in the brain; how is it registered that hand movement and action on the screen aren't correlated anymore?

[Forestluv]

3 hours ago, Loving Radiance said:

Excuse my geekness, how does that look like on a cellular level in the brain; how is it registered that hand movement and action on the screen aren't correlated anymore?

That involves "synaptic plasticity". The structure of synapses is such that they are "plastic" and can be "remolded". In terms of learning, it is called "long term potentiation". 

At a cellular level, synaptic modelers like neurotrophic factors, AMPA/NMDA pathways and glial cells can alter the structure of synapses and when/what neurons fire. 

For example, in the image below notice how NMDA/AMPA receptor activity (red and green receptors) alters synaptic structure. Three forms of restructuring are shown on the right: The presynaptic neuron can produce more neurotransmitter (red dots, top right image), or new synaptic connections can form (middle right) or the postsynaptic membrane can produce more receptors (bottom right). These three examples all show increases in synaptic strength. Synaptic remoldeling also involves decreases in synaptic strength. As well, AMPA/NMDA signaling is common in synaptic remodeling, yet there are also many other remodeling factors. 

[Loving Radiance]

@Forestluv Is it known from past research how an animal recognizes that certain neurons like spinal cord neurons aren't needed anymore even when the desired outcome is the same? How would that look like on an intercellular collective level?

For example, assuming that a Neuralink had been operated onto my brain without my knowledge. Every action of me typing on the keyboard stays the same before and after the operation, at least subjectively. So, how do I recognize that physically typing is decoupled from the letter appearing on the screen? Is it known how the intercellular interplay looks like for that recognition to happen? Or am I just being naive here by believing that this was already investigated/studied in similar scenarios?

[Forestluv]

2 hours ago, Loving Radiance said:

@Forestluv Is it known from past research how an animal recognizes that certain neurons like spinal cord neurons aren't needed anymore even when the desired outcome is the same? How would that look like on an intercellular collective level?

I'm not quite sure on a detailed level. This sounds like the reverse of associative learning. Perhaps a new area of research called "dissociative learning". In which two things that were coupled together become uncoupled. 

It could also be related to breaking automatic habitual patterns.  

Yet there is a change of neuronal communication such that certain neurons are no longer exciting other neurons to fire. 

2 hours ago, Loving Radiance said:

@Forestluv 

For example, assuming that a Neuralink had been operated onto my brain without my knowledge. Every action of me typing on the keyboard stays the same before and after the operation, at least subjectively. So, how do I recognize that physically typing is decoupled from the letter appearing on the screen? Is it known how the intercellular interplay looks like for that recognition to happen? Or am I just being naive here by believing that this was already investigated/studied in similar scenarios?

It seems like you mean how would I become conscious that the two things are not linked? From an awakening perspective, there would be a realization that it happens independently of each other. Yet if someone was deeply ingrained that the two were coupled, they might not even consider the possibility. 

For example, people would just assume that moving fingers on the keyboard and the words on the screen are coupled. That's the way we are taught and the two seem exactly coupled. Noone would even question that they are actually uncoupled. So even if there was a "glitch" it would be filtered out.

In your example, lets say that the neural link is 99,999% accurate. So, pressing the button and word appearing has a nanosecond difference, yet noone would even consider this. it's not perceived and any nanosecond delay is consistent with the signal traveling for keyboard to monitor. Let's say that there is a substantial perceptible Neuralink error every 1 in 100,000 keystrokes. About once every six months there is a neural link "glitch" in which the signal is not sent directly from the Neuralink implant to the monitor rapidly. For example, about once every six months one of the following glitches happens: 1) you press a key and there is a 1 second delay in the appearance, 2) you press a key and nothing happens, 3) a letter appears for which you didn't press the key. This happens to everyone, we wouldn't think twice about it. Just a glitch in our phone, just press the key again or I had fat finger. This could happen everyday and no one would pause and think "wait a minute. I just noticed a "k" appeared and I didn't press the "k" key. Maybe I'm linked directly to the monitor and don't know it. Let me try doing it without the keyboard. Very few people would realize it.

Yet interestingly, with 7 billion people in the world - all it takes is for someone to figure it out. And then they could rapidly spread that realization throughout the world. The neuralink network would need to quickly respond by disabling the Neuralink and enabling the keyboard. 

In the case of the monkey, it would be like using the controller and believing you are moving the video ball via the joystick (yet it's actually through the Neuralink). If the monkey was able to recognize this and think "what a second, do I even need to use the joystick". To maintain the charade, Neuralink would need to rapidly disable the Neuralink and reactivate the controller. So when the monkey tries to make the video ball with his mind it doesn't work. He sees he can't do it via his mind and returns to the controller. Neuralink can no disable the controller and re-enable the Nerualink. This could even re-enforce the belief in the controller. If someone asked "How do you know you need the controller", the mind would think "I tried it without the controller and couldn't do it"

Overall, it is an extreme example of correlation implying causation.

 

[Loving Radiance]

11 hours ago, Forestluv said:

Overall, it is an extreme example of correlation implying causation.

Of course. Good to keep that in mind.

Thanks for the explanation. That thought experiment opened up for me to think about Neuralink and in general the concept of correlation & causation in a new way.

[integral]

It can read but can it write? 

 

[Loving Radiance]

@integral  Do you mean can Neuralink influence the brain chemistry?

[integral]

@Loving Radiance Yes, right now its reading brain activity. In the future i expect it will be able to apply that brain pattern to force that behavior. Imagine recording brain activity for psychedelics or meditation, the clicking a button on your phone to trigger that state.