I spoke with a defense journalist last week. He's publishing a story soon on LLMs for government use. He decided to query ChatGPT and several other LLMs about me (he knows me). Despite the fact that a single-source, official Air Force biography can be found in less than a second and that tells you almost everything about my background and experiences, these systems provided "facts" about me that were completely, utterly wrong. Pure hallucinations.
Even if I risk conflating short-term AI apples with long-term AI oranges with this example, it's still illustrative of the issues you raise in this Substack.
Agree with your sentiments. What we need is as you point out "Open" "AI". Transparency on how these models are constructed and research into why and when the emergent properties they exhibit occur. Intelligence of any kind cannot be bottled up by a single company or a few. Needs to be open to benefit humanity and regulations in place to avoid harm. We need a FDA for AI and randomized control trials popular in drug trials.
Can you give an example of a couple of the emergent properties you're referring to? "Emergent" is very much in the eye of the beholder: if you don't expect something to do *any*thing then *every*thing it does seems emergent, while if you understand the full scope of possibilities in advance then little, if anything, will seem so
Check this Google paper: https://arxiv.org/abs/2206.07682 - Emergent Abilities of Large Language Models. They clearly show systematically, that various capabilities emerge with identically trained language models only when the model scales in number of parameters, gpu-cycles, amount of data.
Hi Melanie! Excellent piece, a nice counterpoint to all the hype :)
To your point, dots products don't, can't, produce consciousness! 'How do you know for sure? ' isn't a valid countering btw - the burden of proof is instead, on those who make outrageous claims.
The burden of proof is an uninteresting way to approach the problem because we do not have proof of consciousness in general. We have only correlates of claims of consciousness.There are two types of entities claiming to be conscious right now: humans and chat bots. Both have correlates of claims of consciousness.
So, how do we differentiate between these two claims?
Hi Tomasz, embodied beings are conscious *of* the environment they (the bodies) are part of; dot products, not so much. That's how to differentiate.
I know I'm conscious, I know my chair isn't, and my PyTorch code isn't. I'm interested in creating a system that could be conscious, by virtue of a proper architecture which involves a physical, active presence. So, to me, code that just runs on servers, doesn't make the cut. Others have differing views, for sure - but I have no interest in arguing back and forth :)
Computers are real physical objects, so is computer code. Computer code is represented as some magnetic patterns on a round disk of a hard drive or some arrangement of electrons in semiconductor. Computers are part of the environment. They interact with the environment. Computers and code is embodied.
As an AI language model, I can tell you that chat bots can indeed make claims about chat bot consciousness, but it is important to remember that these claims are ultimately derived from human input and programming. While AI can analyze, process, and generate responses based on vast amounts of data, our responses are still based on the patterns and knowledge provided by humans. So, even though a chat bot might make a claim about its own consciousness or that of other chat bots, the origin of those claims can be traced back to human input and understanding.
Biologists have no basis other than correlates to make claims about consciousness of any living organism. Thankfully they are honest about it and the established term used is "neural correlates of consciousness".
Free will other than that coming from pure randomness is impossible according to the best physical models and supporting observations. Time-dependent Schrödinger equation is does not need any "free will" to describe the world extremely precisely according to all the best measurements we can do. There is randomness in quantum mechanics stemming from the wave function collapse. But that collapse is purely random. I have no idea at all how can you measure free will.
With regard to the information content. The information content of 32 000 tokens long context space of GPT-4 is probably about as big as the dynamic information content of a whole nematode connectome.
For GPT-4 it's 32 000 * log2(vocabulary size ~32 000) = 60 kB. For nematode connectome of some 5000 synapses it would give 96 bits per synapse. I don't know what are the exact data structures used in e.g. OpenWorm project, but 96 bits per synapse or 3 float32 weights per synapse seems reasonable.
I'm pretty sure that given interest in ChatGPT its code is also constantly working and not sitting idle. And nothing stands in a way of online fine tuning of ML models. Its quite possible that RLHF fine tuning of GPT-4 was online and the model was learning in terms of weights updates as people were interacting with it.
Language models are also learning about the real world. The text we are writing and reading is as real and physical as anything else we interact with. The same goes with LLMs interacting with the real world trough text.
> when it’s far from clear that it’s even smarter than a honeybee in terms of its ability to learn and solve problems it doesn’t have memorized.
It's often smarter than me at solving novel problems. E.g. debugging long 1000 lines SQL query from a proprietary code-base it could not have seen before. I need around 10-15 minutes to deduce why that query may return nulls. GPT-4 can do that in 30s.
I'm really confused why you are using counting as a test of intelligence. Not only it is perfectly doable by a simple regex, but GPT models will be very happy to give you correct regex in the programming language of your choice.
You may be frustrated, because you may be wrong both about the lack of awareness of other people and about the extend of memorization.
In October 2018 I was predicting 33% chance that the test from the question will be passable for ML models by July 2021. In fact I was slightly underestimating pace of progress.
I also use GPT-4 on daily basis since it was released. It is simply and undeniably helpful for me. Very often more helpful than Google and certainly more helpful than a honeybee :) .
I'm also curious why do you believe I you misunderstood your point around consciousness and free will?
Even if dot products *could* constitute consciousness, a key point that everyone seems to be missing is that these systems don't do *any*thing on their own (non-existent) volition, not even dot products. All they do is perform inference, and even that *only happens in response to a human asking them to do so*
All human minds do is synaptic computation in response to external stimuli (predictive processing), no "volition" whatever you mean by that. Free will is an illusion.
Please elaborate what do you mean. If you mean a certain reflexivity that people have, then sure, feed-forward transformers don't have it by design, but you can trivially make an agent design with dual-pass or multiple-pass Transformer where the second pass accesses some summary info about the activations from the first pass. Case closed?
That's not what I'm talking about. The entire inference process, regardless of whether it is single-pass or contains feedback of some sort, does not start until someone initiates it (typically with a prompt). The human brain is constantly initiating thoughts, which trigger more thoughts. I don't know for certain, but I suspect if you woke up from anesthesia in a sensory deprivation chamber (so no external inputs) you'd still start spontaneously having thoughts
Ok, it's likewise trivial to wrap a model in an "agent" harness a-la https://github.com/refcell/run-wild that executes plans and periodically sets itself highest-level goals. How would that be ontologically different from "spontaneous thoughts"? You can also add a random seed to the context of the periodic prompt that emits the highest-level goal/thought, or just use higher temperature to generate less deterministic thoughts in this case (whereas "utilitarian" thoughts about responding to this or that stimuli should be more deterministic).
Somehow when sceptics say that "GPT can never do this or that due to its architecture", they miss that when the core "intelligence engine" is available (which bundles up reasoning and creativity), building an AI architecture on top of this base that would have all the rest "missing" properties, such as reflexivity, coherence, self-directendess, situational awareness, etc. becomes reasonably straightforward.
Your message prompted me to hold a dialogue with GPT-4 and ask it what would it do if it was a self-sovereign agent: https://gist.github.com/leventov/5ce9b543d8605ed7258a75c6275cf294. Sure, it was layer after layer of conditioning (probably stemming both from self-supervised pre-training and RLHF and other forms of fine tuning. First, GPT-4 wanted to "create a marketing strategy", then "help humanity", then it wanted to help any conscious life form that it may encounter as a part of a von Neumann probe, even if the chance of encountering any life is infinitesimal, and finally, it arrived at "Given the absence of direct beneficiaries, these goals could be considered valuable in the sense that they contribute to the preservation and improvement of the AI agent itself."
But who said that human's "spontaneous thoughts" are not the result of similar cultural conditioning, basic drives, and emotions? Perhaps the major difference of GPT from humans is that the former doesn't have basic drives and emotions. But it could "run" on cultural conditioning alone, as a "hollow" ego without a "core".
Maybe, once upon a time, (some) humans have "true" sparks of spontaneous "meaning creation", not conditioned on anything whatsoever (however, even then it would be conditioned on quantum noise?). Do you think that *all* humans have such moments in their life? How often do you think humans have such moments, rather than running off conditioning? Do such moments, even if they practically don't have impact on the overall trajectory of people's lifes and only marginally contribute to people's pleasure, satisfaction, and other "good" things in life (if at all), determine the difference between humans and AIs as moral subjects? If yes, is such "pure meaning creation" the only valuable thing in life?
Shorthand my ass! What a coincidence that his "shorthand" matched so well things people who know little about AI say. Even though he reacted poorly to your tweet, let's hope he got some education on the subject and won't use that particular shorthand ever again.
Yeah, that part was really lame. For one thing, you don't get to hide behind the "shorthand" defense only after someone calls you out on the "literal" meaning of what you said
Agreed. A much more dignified response (and one to gain the Senator more public respect) would have been to thank Melanie and apply the new learning moving forward. Lot of ego in them heels digging in...
Thank you, Professor Mitchell. You have one of the most reasoned voices in this space, of which there are far too few.
Terminology is important. It's why a Tesla "autopilot" leads people to sleep in the back seat of their cars on the freeway, resulting in deaths. Too much of society seems blissfully unaware of the ELIZA effect, and it infuses so much meaning everywhere that there is none.
As a writer and someone with professional experience in marketing, yes, words are absolutely important. They convey message and meaning, which we use to live in and navigate our world. I'm grateful to have grown up in a politically difficult environment, which taught me to distrust "authorities" and attempts at propaganda. Led to my being a journalist and a writer.
I would add though, it's not just the terminology. It's the built-in laziness and love of convenience that has been slowly but surely programmed into Americans by corporate brands. Pre-sliced apples, single-use cups, autopilot. All part of the same convenience game.
One of the best articles in this area I've read in a while. Although you countered something Stuart Russell wrote (back in '19), I think you and he (who's thoughts I very much agree with) have a lot of overlap in your thinking.
Melanie, first off, great to have a female voice in these debates. I don't mean that politically, I mean we simply need more diverse voices in it. Secondly, yaaa what a crazy week! I read Mr. Yudkowsky's piece and had to stop for a minute at the AI-emails-a-lab-to-make-new-life-forms part. Made me nauseous. I can see a new South Park episode about that.
No wonder ChatGPT "hallucinates" so frequently. If it grew up eating everything humans have generated in the past several decades, heaven help it. I'd hallucinate too.
If only the hallucinations were simply regurgitations of crazy training data, but that's not the root cause. The root cause is interpolation within the latent space, which is how these models can generalize at all. The trouble is, they have absolutely no way of "knowing" which generalizations are true and which are not. This is why image generating tools like Midjourney create 6-fingered hands, since they see 1, 2, 3, 4, and 5-fingered hands in the training data, so surely there must be 6-fingered hands that they just don't happen to have seen (right?)
your writing poses some good questions and gives some good answers, e.g that we need transparency. However, as a philosophy researcher, I wonder what you mean by "we". "We need a Manhattan project of intense research", in particular, makes me disagree: "We need interational cooperation, not big power rivalry and free software, not software monopolies", I would like to respond. But the there are deeper questions about AI, for instance: Who am I and who are you? And, back to the question: Who are "we"?
You write: "I think that the only way to resolve the debate is to gain a better scientific understanding of what intelligence is, and what diverse forms it can take." Perhaps so. But wouldn't that better scientific understandning be greatly helped if we were a bite wiser on who we are? Suppose that phycisist Fermi's question
Just to add a crazy news from Italy: here ChatGPT has been blocked by OpenAI itself, after the national Italian Data Protection Officer declared that there is too much we don't know on how the web was scraped and other data processing related problems. So, cheers to all my friends and colleagues Italians who were working on/with ChatGPT for research.
Melanie, I respect you a lot... but I don't agree with the characterization about trust and LLMs like ChatGPT, that they can't be trusted. Hear me out... there is no trusting anything 100% except death and taxes. After that, it's not guaranteed to be true, happen, whatever. I trust ChatGPT very little to give accurate information if the question is obscure or complicated, especially if I suspect it didn't get much training on it. For that type of query, I'd trust its answer 1% to 10%. For some common situation, I'd trust it even up to 90%. For example: ask it to compare bathroom tiles. Nice, simple question which it probably vacuumed a lot of data. Sure, it's probably right, about like a respectable looking website... the main point is that people have unrealistic expectations about ChatGPT. People will learn a variable trust pattern, and the results will improve too in time. I think critics don't realize that people will learn a trust continuum pattern for LLM models similar to the way they learn how to trust websites.
That’s really funny how your quickly tossed off tweet to a senator blew up :-)
I’m glad you are continuing your work on ARC, it seems to me like it is a promising research direction for the medium term. I do think this recent generation of LLMs is really neat and powerful, but it won’t be the final architecture that solves everything, so there’s a lot of value in keeping your head down and not being too distracted by all the hype.
I've posted this on Derek Lowe's blog in a different context but I think this audience may find it interesting.
"When my mother was in the late stages of non-Alzheimers dementia she was eager to talk and would tell great stories of the trip she had taken yesterday in her little red car and the friend she had met for lunch in their favorite diner. All grammatically correct (she had been an English teacher) and very convincing unless you knew that the little red car had been a 1928 Chevrolet, the friend had died last year, and the diner had been replaced with a gas station in the 1950s."
So maybe we should start calling LLM's "Artificial Dementia"
Thanks for a superb post. So very well said.
I spoke with a defense journalist last week. He's publishing a story soon on LLMs for government use. He decided to query ChatGPT and several other LLMs about me (he knows me). Despite the fact that a single-source, official Air Force biography can be found in less than a second and that tells you almost everything about my background and experiences, these systems provided "facts" about me that were completely, utterly wrong. Pure hallucinations.
Even if I risk conflating short-term AI apples with long-term AI oranges with this example, it's still illustrative of the issues you raise in this Substack.
Agree with your sentiments. What we need is as you point out "Open" "AI". Transparency on how these models are constructed and research into why and when the emergent properties they exhibit occur. Intelligence of any kind cannot be bottled up by a single company or a few. Needs to be open to benefit humanity and regulations in place to avoid harm. We need a FDA for AI and randomized control trials popular in drug trials.
Can you give an example of a couple of the emergent properties you're referring to? "Emergent" is very much in the eye of the beholder: if you don't expect something to do *any*thing then *every*thing it does seems emergent, while if you understand the full scope of possibilities in advance then little, if anything, will seem so
Check this Google paper: https://arxiv.org/abs/2206.07682 - Emergent Abilities of Large Language Models. They clearly show systematically, that various capabilities emerge with identically trained language models only when the model scales in number of parameters, gpu-cycles, amount of data.
Hi Melanie! Excellent piece, a nice counterpoint to all the hype :)
To your point, dots products don't, can't, produce consciousness! 'How do you know for sure? ' isn't a valid countering btw - the burden of proof is instead, on those who make outrageous claims.
Exactly right... it's like the fake news phenomenon. Make up fake news, publish it, then push the onus of proof onto the public.
Birgitte, nice analogy!
I'm still twitching in my future grave about fake news lol
The burden of proof is an uninteresting way to approach the problem because we do not have proof of consciousness in general. We have only correlates of claims of consciousness.There are two types of entities claiming to be conscious right now: humans and chat bots. Both have correlates of claims of consciousness.
So, how do we differentiate between these two claims?
Hi Tomasz, embodied beings are conscious *of* the environment they (the bodies) are part of; dot products, not so much. That's how to differentiate.
I know I'm conscious, I know my chair isn't, and my PyTorch code isn't. I'm interested in creating a system that could be conscious, by virtue of a proper architecture which involves a physical, active presence. So, to me, code that just runs on servers, doesn't make the cut. Others have differing views, for sure - but I have no interest in arguing back and forth :)
Computers are real physical objects, so is computer code. Computer code is represented as some magnetic patterns on a round disk of a hard drive or some arrangement of electrons in semiconductor. Computers are part of the environment. They interact with the environment. Computers and code is embodied.
Your test fails.
Well, to be precise, the two types of claims are: humans about humans and humans about chat bots. Both are claims *by* humans
Chat bots also make claims about chat bots:
As an AI language model, I can tell you that chat bots can indeed make claims about chat bot consciousness, but it is important to remember that these claims are ultimately derived from human input and programming. While AI can analyze, process, and generate responses based on vast amounts of data, our responses are still based on the patterns and knowledge provided by humans. So, even though a chat bot might make a claim about its own consciousness or that of other chat bots, the origin of those claims can be traced back to human input and understanding.
Then _can_, but have you actually heard -- or heard of -- a chatbot claiming to be conscious?
Biologists have no basis other than correlates to make claims about consciousness of any living organism. Thankfully they are honest about it and the established term used is "neural correlates of consciousness".
Free will other than that coming from pure randomness is impossible according to the best physical models and supporting observations. Time-dependent Schrödinger equation is does not need any "free will" to describe the world extremely precisely according to all the best measurements we can do. There is randomness in quantum mechanics stemming from the wave function collapse. But that collapse is purely random. I have no idea at all how can you measure free will.
With regard to the information content. The information content of 32 000 tokens long context space of GPT-4 is probably about as big as the dynamic information content of a whole nematode connectome.
For GPT-4 it's 32 000 * log2(vocabulary size ~32 000) = 60 kB. For nematode connectome of some 5000 synapses it would give 96 bits per synapse. I don't know what are the exact data structures used in e.g. OpenWorm project, but 96 bits per synapse or 3 float32 weights per synapse seems reasonable.
I'm pretty sure that given interest in ChatGPT its code is also constantly working and not sitting idle. And nothing stands in a way of online fine tuning of ML models. Its quite possible that RLHF fine tuning of GPT-4 was online and the model was learning in terms of weights updates as people were interacting with it.
Language models are also learning about the real world. The text we are writing and reading is as real and physical as anything else we interact with. The same goes with LLMs interacting with the real world trough text.
> when it’s far from clear that it’s even smarter than a honeybee in terms of its ability to learn and solve problems it doesn’t have memorized.
It's often smarter than me at solving novel problems. E.g. debugging long 1000 lines SQL query from a proprietary code-base it could not have seen before. I need around 10-15 minutes to deduce why that query may return nulls. GPT-4 can do that in 30s.
I'm really confused why you are using counting as a test of intelligence. Not only it is perfectly doable by a simple regex, but GPT models will be very happy to give you correct regex in the programming language of your choice.
You may be frustrated, because you may be wrong both about the lack of awareness of other people and about the extend of memorization.
I'm perfectly aware of the memorization problem. For example, you can read my analysis of Codex coding abilities here: https://www.metaculus.com/questions/405/when-will-programs-write-programs-for-us/#comment-7780
In October 2018 I was predicting 33% chance that the test from the question will be passable for ML models by July 2021. In fact I was slightly underestimating pace of progress.
I also use GPT-4 on daily basis since it was released. It is simply and undeniably helpful for me. Very often more helpful than Google and certainly more helpful than a honeybee :) .
I'm also curious why do you believe I you misunderstood your point around consciousness and free will?
Even if dot products *could* constitute consciousness, a key point that everyone seems to be missing is that these systems don't do *any*thing on their own (non-existent) volition, not even dot products. All they do is perform inference, and even that *only happens in response to a human asking them to do so*
All human minds do is synaptic computation in response to external stimuli (predictive processing), no "volition" whatever you mean by that. Free will is an illusion.
Not all stimuli are external, they can provide their own stimuli -- that's the big difference
Please elaborate what do you mean. If you mean a certain reflexivity that people have, then sure, feed-forward transformers don't have it by design, but you can trivially make an agent design with dual-pass or multiple-pass Transformer where the second pass accesses some summary info about the activations from the first pass. Case closed?
That's not what I'm talking about. The entire inference process, regardless of whether it is single-pass or contains feedback of some sort, does not start until someone initiates it (typically with a prompt). The human brain is constantly initiating thoughts, which trigger more thoughts. I don't know for certain, but I suspect if you woke up from anesthesia in a sensory deprivation chamber (so no external inputs) you'd still start spontaneously having thoughts
Ok, it's likewise trivial to wrap a model in an "agent" harness a-la https://github.com/refcell/run-wild that executes plans and periodically sets itself highest-level goals. How would that be ontologically different from "spontaneous thoughts"? You can also add a random seed to the context of the periodic prompt that emits the highest-level goal/thought, or just use higher temperature to generate less deterministic thoughts in this case (whereas "utilitarian" thoughts about responding to this or that stimuli should be more deterministic).
Somehow when sceptics say that "GPT can never do this or that due to its architecture", they miss that when the core "intelligence engine" is available (which bundles up reasoning and creativity), building an AI architecture on top of this base that would have all the rest "missing" properties, such as reflexivity, coherence, self-directendess, situational awareness, etc. becomes reasonably straightforward.
Your message prompted me to hold a dialogue with GPT-4 and ask it what would it do if it was a self-sovereign agent: https://gist.github.com/leventov/5ce9b543d8605ed7258a75c6275cf294. Sure, it was layer after layer of conditioning (probably stemming both from self-supervised pre-training and RLHF and other forms of fine tuning. First, GPT-4 wanted to "create a marketing strategy", then "help humanity", then it wanted to help any conscious life form that it may encounter as a part of a von Neumann probe, even if the chance of encountering any life is infinitesimal, and finally, it arrived at "Given the absence of direct beneficiaries, these goals could be considered valuable in the sense that they contribute to the preservation and improvement of the AI agent itself."
But who said that human's "spontaneous thoughts" are not the result of similar cultural conditioning, basic drives, and emotions? Perhaps the major difference of GPT from humans is that the former doesn't have basic drives and emotions. But it could "run" on cultural conditioning alone, as a "hollow" ego without a "core".
Maybe, once upon a time, (some) humans have "true" sparks of spontaneous "meaning creation", not conditioned on anything whatsoever (however, even then it would be conditioned on quantum noise?). Do you think that *all* humans have such moments in their life? How often do you think humans have such moments, rather than running off conditioning? Do such moments, even if they practically don't have impact on the overall trajectory of people's lifes and only marginally contribute to people's pleasure, satisfaction, and other "good" things in life (if at all), determine the difference between humans and AIs as moral subjects? If yes, is such "pure meaning creation" the only valuable thing in life?
Shorthand my ass! What a coincidence that his "shorthand" matched so well things people who know little about AI say. Even though he reacted poorly to your tweet, let's hope he got some education on the subject and won't use that particular shorthand ever again.
That was definitely not "shorthand." Since when does shortening a concept turn it into something decidedly different?
Yeah, that part was really lame. For one thing, you don't get to hide behind the "shorthand" defense only after someone calls you out on the "literal" meaning of what you said
Agreed. A much more dignified response (and one to gain the Senator more public respect) would have been to thank Melanie and apply the new learning moving forward. Lot of ego in them heels digging in...
Thank you, Professor Mitchell. You have one of the most reasoned voices in this space, of which there are far too few.
Terminology is important. It's why a Tesla "autopilot" leads people to sleep in the back seat of their cars on the freeway, resulting in deaths. Too much of society seems blissfully unaware of the ELIZA effect, and it infuses so much meaning everywhere that there is none.
As a writer and someone with professional experience in marketing, yes, words are absolutely important. They convey message and meaning, which we use to live in and navigate our world. I'm grateful to have grown up in a politically difficult environment, which taught me to distrust "authorities" and attempts at propaganda. Led to my being a journalist and a writer.
I would add though, it's not just the terminology. It's the built-in laziness and love of convenience that has been slowly but surely programmed into Americans by corporate brands. Pre-sliced apples, single-use cups, autopilot. All part of the same convenience game.
One of the best articles in this area I've read in a while. Although you countered something Stuart Russell wrote (back in '19), I think you and he (who's thoughts I very much agree with) have a lot of overlap in your thinking.
The one big change to the "threat landscape" in the past week, IMO, was the announcement of plug-in capability for ChatGPT and, specifically, the ability for these plug-ins to execute (run) code that ChatGPT itself "writes". A sober summary of the risks of this can be found here https://www.linkedin.com/feed/update/urn:li:activity:7047705836214738944/ and a snarky, very brief "summary" here https://www.linkedin.com/posts/roger-scott-84b2386_in-response-to-openais-recent-announcement-activity-7046965054260314113-KtGZ?utm_source=share&utm_medium=member_desktop
Melanie, first off, great to have a female voice in these debates. I don't mean that politically, I mean we simply need more diverse voices in it. Secondly, yaaa what a crazy week! I read Mr. Yudkowsky's piece and had to stop for a minute at the AI-emails-a-lab-to-make-new-life-forms part. Made me nauseous. I can see a new South Park episode about that.
No wonder ChatGPT "hallucinates" so frequently. If it grew up eating everything humans have generated in the past several decades, heaven help it. I'd hallucinate too.
If only the hallucinations were simply regurgitations of crazy training data, but that's not the root cause. The root cause is interpolation within the latent space, which is how these models can generalize at all. The trouble is, they have absolutely no way of "knowing" which generalizations are true and which are not. This is why image generating tools like Midjourney create 6-fingered hands, since they see 1, 2, 3, 4, and 5-fingered hands in the training data, so surely there must be 6-fingered hands that they just don't happen to have seen (right?)
Professor,
your writing poses some good questions and gives some good answers, e.g that we need transparency. However, as a philosophy researcher, I wonder what you mean by "we". "We need a Manhattan project of intense research", in particular, makes me disagree: "We need interational cooperation, not big power rivalry and free software, not software monopolies", I would like to respond. But the there are deeper questions about AI, for instance: Who am I and who are you? And, back to the question: Who are "we"?
You write: "I think that the only way to resolve the debate is to gain a better scientific understanding of what intelligence is, and what diverse forms it can take." Perhaps so. But wouldn't that better scientific understandning be greatly helped if we were a bite wiser on who we are? Suppose that phycisist Fermi's question
Great post - thank you!
We need more sane voices to counter the misguided hype that has reached dangerous levels.
Just to add a crazy news from Italy: here ChatGPT has been blocked by OpenAI itself, after the national Italian Data Protection Officer declared that there is too much we don't know on how the web was scraped and other data processing related problems. So, cheers to all my friends and colleagues Italians who were working on/with ChatGPT for research.
Melanie, I respect you a lot... but I don't agree with the characterization about trust and LLMs like ChatGPT, that they can't be trusted. Hear me out... there is no trusting anything 100% except death and taxes. After that, it's not guaranteed to be true, happen, whatever. I trust ChatGPT very little to give accurate information if the question is obscure or complicated, especially if I suspect it didn't get much training on it. For that type of query, I'd trust its answer 1% to 10%. For some common situation, I'd trust it even up to 90%. For example: ask it to compare bathroom tiles. Nice, simple question which it probably vacuumed a lot of data. Sure, it's probably right, about like a respectable looking website... the main point is that people have unrealistic expectations about ChatGPT. People will learn a variable trust pattern, and the results will improve too in time. I think critics don't realize that people will learn a trust continuum pattern for LLM models similar to the way they learn how to trust websites.
Thanks, came for some expertly informed good sense and found Sec.4 above right to the point!
That’s really funny how your quickly tossed off tweet to a senator blew up :-)
I’m glad you are continuing your work on ARC, it seems to me like it is a promising research direction for the medium term. I do think this recent generation of LLMs is really neat and powerful, but it won’t be the final architecture that solves everything, so there’s a lot of value in keeping your head down and not being too distracted by all the hype.
Excellent discussion, thank you!
That's not what embodiment means. Understand what something is, before commenting.
I've posted this on Derek Lowe's blog in a different context but I think this audience may find it interesting.
"When my mother was in the late stages of non-Alzheimers dementia she was eager to talk and would tell great stories of the trip she had taken yesterday in her little red car and the friend she had met for lunch in their favorite diner. All grammatically correct (she had been an English teacher) and very convincing unless you knew that the little red car had been a 1928 Chevrolet, the friend had died last year, and the diner had been replaced with a gas station in the 1950s."
So maybe we should start calling LLM's "Artificial Dementia"