Susan Schneider, philosopher and director of the Center for the Future of Mind, AI & Society, recently highlighted the risk of ethical confusion: prematurely assuming a chatbot is conscious could lead to all sorts of problems.
The problem is that chatbots are great mimics… and so they’re asserting consciousness and people believe them.
For instance, in situations in which we have to balance the moral value of an AI versus that of a human, we might in some cases balance them equally, for we have decided that they are both conscious. In other cases, we might even sacrifice a human to save two AIs.
[And] if we allow someone who built the AI to say that their product is conscious and it ends up harming someone, they could simply throw their hands up and exclaim: “It made up its own mind–I am not responsible.” Accepting claims of consciousness could shield individuals and companies from legal and/or ethical responsibility for the impact of the technologies they develop.
These issues will arise whether or not AI is or can be conscious.
I wonder how to weight ethical confusion, as a risk? As I said yesterday humans are pretty self-centred, and we’re not going to treat AIs, chickens or workers in sweatshops any better just because we are co-sentients.
Schneider highlighted another risk back in 2017 that on the face of it appear more far-fetched but personally I give more weight. What if silicon can never be conscious? Therefore as we start using brain implants, at what point do humans stop being conscious?
machine consciousness could impact the viability of brain-implant technologies, like those to be developed by Elon Musk’s new company, Neuralink. If AI cannot be conscious, then the parts of the brain responsible for consciousness could not be replaced with chips without causing a loss of consciousness. And, in a similar vein, a person couldn’t upload their brain to a computer to avoid death because that upload wouldn’t be a conscious being.
It’s a slippery slope: let’s say you have a computer chip running a large language model, and some of it is offloaded to a clump of brain tissue. Is that conscious? Instinctively we’d say no. btw hybrid computer chips/brain tissue were built back in 2023 and they can do the audio processing that underpins speech recognition.
But on the other end of things, let’s say you have a human brain with a the very smallest possible implant: if you buy extended cognition, you might call always-on AirPods a minimum viable brain prosthetic, especially if they can sense and respond to brainwaves. So is that “cognitive hybrid” conscious? Yes, we’d instinctively say, it’s just a person with AirPods.
I mean, forget AirPods, I’m 100% sure that even Noland Armagh moving a cursor with a brain-computer interface (NPR) is conscious.
How far can we go? A brain-computer “interface” is just an interface, like a mouse or multitouch, even though it’s inside the skull. Subjectively there’s no difference between raising my arm to catch a ball or “thinking” the cursor to the top of the screen, right? Or “knowing” the date (by thinking) and “knowing” the time (by unthinkingly glancing at the status bar on my ever-present phone).
If these don’t delete consciousness then the conscious “self” is located elsewhere in the brain maybe. Smaller and smaller…
But… there’s a threshold somewhere, we’ve just talked about both ends… so as we load an individual with brain implants to control computers… to speak… control a powered chair… augment memory… is there a line beyond which they are no longer conscious, and we’re granting personhood (ethically, legally) to someone/something that is no longer a person?
Do we declare some legal limit, an arbitrary Karman line of being a p-zombie?
Or the other way round, a Karman line over which a large language model is declared conscious?
(The Karman line is the conventional and imaginary boundary of space, 100km/62 miles straight up.)
It’s a nonsense.
Yet we’ll need answers, for all those pragmatic questions above.
I suspect that we’ll end up with a pragmatic hodgepodge, hammered out one precedent-setting legal decision at a time, in the same way that we assign personhood to corporations because it’s convenient and kinda feels right (in folk understanding Amazon has about the same amount of personhood as an ant), and copyright which is kinda ownership and kinda about incentivising developing ideas and kinda this fair use thing… it’s all a fudge.
But ideally it wouldn’t be a fudge (much).
What this really exposes for me is that we’re going to need a more sophisticated way to think about consciousness…
Back in 2022, OpenAI co-founder Ilya Sutskever tweeted it may be that today’s large neural networks are slightly conscious. That “slightly” is incredibly load-bearing. What on earth does it mean.
If you enjoyed this post, please consider sharing it by email or on social media. Here’s the link. Thanks, —Matt.
‘Yes, we’ll see them together some Saturday afternoon then,’ she said. ‘I won’t have any hand in your not going to Cathedral on Sunday morning. I suppose we must be getting back. What time was it when you looked at your watch just now?’ "In China and some other countries it is not considered necessary to give the girls any education; but in Japan it is not so. The girls are educated here, though not so much as the boys; and of late years they have established schools where they receive what we call the higher branches of instruction. Every year new schools for girls are opened; and a great many of the Japanese who formerly would not be seen in public with their wives have adopted the Western idea, and bring their wives into society. The marriage laws have been arranged so as to allow the different classes to marry among[Pg 258] each other, and the government is doing all it can to improve the condition of the women. They were better off before than the women of any other Eastern country; and if things go on as they are now going, they will be still better in a few years. The world moves. "Frank and Fred." She whispered something to herself in horrified dismay; but then she looked at me with her eyes very blue and said "You'll see him about it, won't you? You must help unravel this tangle, Richard; and if you do I'll--I'll dance at your wedding; yours and--somebody's we know!" Her eyes began forewith. Lawrence laughed silently. He seemed to be intensely amused about something. He took a flat brown paper parcel from his pocket. making a notable addition to American literature. I did truly. "Surely," said the minister, "surely." There might have been men who would have remembered that Mrs. Lawton was a tough woman, even for a mining town, and who would in the names of their own wives have refused to let her cross the threshold of their homes. But he saw that she was ill, and he did not so much as hesitate. "I feel awful sorry for you sir," said the Lieutenant, much moved. "And if I had it in my power you should go. But I have got my orders, and I must obey them. I musn't allow anybody not actually be longing to the army to pass on across the river on the train." "Throw a piece o' that fat pine on the fire. Shorty," said the Deacon, "and let's see what I've got." "Further admonitions," continued the Lieutenant, "had the same result, and I was about to call a guard to put him under arrest, when I happened to notice a pair of field-glasses that the prisoner had picked up, and was evidently intending to appropriate to his own use, and not account for them. This was confirmed by his approaching me in a menacing manner, insolently demanding their return, and threatening me in a loud voice if I did not give them up, which I properly refused to do, and ordered a Sergeant who had come up to seize and buck-and-gag him. The Sergeant, against whom I shall appear later, did not obey my orders, but seemed to abet his companion's gross insubordination. The scene finally culminated, in the presence of a number of enlisted men, in the prisoner's wrenching the field-glasses away from me by main force, and would have struck me had not the Sergeant prevented this. It was such an act as in any other army in the world would have subjected the offender to instant execution. It was only possible in—" "Don't soft-soap me," the old woman snapped. "I'm too old for it and I'm too tough for it. I want to look at some facts, and I want you to look at them, too." She paused, and nobody said a word. "I want to start with a simple statement. We're in trouble." RE: Fruyling's World "MACDONALD'S GATE" "Read me some of it." "Well, I want something better than that." HoME大香蕉第一时间
ENTER NUMBET 0016www.efyoft.com.cn lfstem.com.cn gxwd.net.cn hyuemp.com.cn lbirti.com.cn oomipo.com.cn www.mv48.org.cn tcchain.com.cn www.reyuu.net.cn www.qiandasc.com.cn
Susan Schneider, philosopher and director of the Center for the Future of Mind, AI & Society, recently highlighted the risk of ethical confusion:
The problem is that chatbots are great mimics… and so they’re asserting consciousness and people believe them.
These issues will arise whether or not AI is or can be conscious.
I wonder how to weight ethical confusion, as a risk? As I said yesterday humans are pretty self-centred, and we’re not going to treat AIs, chickens or workers in sweatshops any better just because we are co-sentients.
Schneider highlighted another risk back in 2017 that on the face of it appear more far-fetched but personally I give more weight. What if silicon can never be conscious? Therefore as we start using brain implants, at what point do humans stop being conscious?
(I highlighted the same quote when I talked about AI sentience and Susan Schneider in 2023).
It’s a slippery slope: let’s say you have a computer chip running a large language model, and some of it is offloaded to a clump of brain tissue. Is that conscious? Instinctively we’d say no. btw hybrid computer chips/brain tissue were built back in 2023 and they can do the audio processing that underpins speech recognition.
But on the other end of things, let’s say you have a human brain with a the very smallest possible implant: if you buy extended cognition, you might call always-on AirPods a minimum viable brain prosthetic, especially if they can sense and respond to brainwaves. So is that “cognitive hybrid” conscious? Yes, we’d instinctively say, it’s just a person with AirPods.
I mean, forget AirPods, I’m 100% sure that even Noland Armagh moving a cursor with a brain-computer interface (NPR) is conscious.
How far can we go? A brain-computer “interface” is just an interface, like a mouse or multitouch, even though it’s inside the skull. Subjectively there’s no difference between raising my arm to catch a ball or “thinking” the cursor to the top of the screen, right? Or “knowing” the date (by thinking) and “knowing” the time (by unthinkingly glancing at the status bar on my ever-present phone).
If these don’t delete consciousness then the conscious “self” is located elsewhere in the brain maybe. Smaller and smaller…
But… there’s a threshold somewhere, we’ve just talked about both ends… so as we load an individual with brain implants to control computers… to speak… control a powered chair… augment memory… is there a line beyond which they are no longer conscious, and we’re granting personhood (ethically, legally) to someone/something that is no longer a person?
Do we declare some legal limit, an arbitrary Karman line of being a p-zombie?
Or the other way round, a Karman line over which a large language model is declared conscious?
(The Karman line is the conventional and imaginary boundary of space, 100km/62 miles straight up.)
It’s a nonsense.
Yet we’ll need answers, for all those pragmatic questions above.
I suspect that we’ll end up with a pragmatic hodgepodge, hammered out one precedent-setting legal decision at a time, in the same way that we assign personhood to corporations because it’s convenient and kinda feels right (in folk understanding Amazon has about the same amount of personhood as an ant), and copyright which is kinda ownership and kinda about incentivising developing ideas and kinda this fair use thing… it’s all a fudge.
But ideally it wouldn’t be a fudge (much).
What this really exposes for me is that we’re going to need a more sophisticated way to think about consciousness…
Back in 2022, OpenAI co-founder Ilya Sutskever tweeted That “slightly” is incredibly load-bearing. What on earth does it mean.