Can ChatGPT Handle Complex Physics Questions Accurately?

[Demystifier]

I have tried ChatGPT, an artificial intelligence system, to ask hard questions.

Me: What's the best interpretation of quantum mechanics?

ChatGPT:
Quantum mechanics is a fundamental theory in physics that describes the behavior of matter and energy at the atomic and subatomic scale. It is a mathematical framework that provides a set of rules for understanding and predicting the behavior of particles, such as electrons and photons.

There are several different interpretations of quantum mechanics that have been proposed over the years, each offering a different perspective on the underlying principles of the theory. Some of the most well-known interpretations include the Copenhagen interpretation, the many-worlds interpretation, and the de Broglie-Bohm interpretation.

The Copenhagen interpretation, developed by Niels Bohr and others in the 1920s, is perhaps the most widely accepted interpretation of quantum mechanics. It suggests that the behavior of particles at the quantum level is inherently probabilistic and that the act of observation plays a role in determining the outcome of a quantum event.

The many-worlds interpretation, proposed by physicist Hugh Everett in the 1950s, suggests that every possible outcome of a quantum event actually occurs in a separate, parallel universe.

The de Broglie-Bohm interpretation, also known as the pilot wave interpretation, suggests that particles in the quantum world are guided by an underlying wave function and that their behavior can be predicted deterministically.

Ultimately, the choice of which interpretation of quantum mechanics to adopt is a matter of personal preference and philosophical perspective. Some physicists and philosophers find one interpretation more appealing than others, while others may find elements of multiple interpretations to be useful in understanding the behavior of particles at the quantum level.

Me: Are hidden variables ruled out by the Bell theorem?

ChatGPT:
The Bell theorem is a mathematical result in quantum mechanics that demonstrates the existence of certain correlations between particles that cannot be explained by classical physics. These correlations, known as Bell correlations, violate a principle known as local realism, which states that physical events in one location should not be able to instantaneously affect events in another location, and that the outcomes of physical measurements should be determined by hidden variables that exist independently of measurement.

The Bell theorem suggests that local realism cannot be true in the quantum world, as it is incompatible with the observed correlations between particles. However, the theorem does not completely rule out the existence of hidden variables in quantum mechanics. Some interpretations of quantum mechanics, such as the de Broglie-Bohm interpretation, posit the existence of hidden variables that determine the behavior of particles in the quantum world, and these hidden variables are consistent with the predictions of the Bell theorem.

It is important to note that the existence of hidden variables in quantum mechanics is still a matter of ongoing debate and research in physics, and it is not yet clear whether hidden variables exist or not.

Me: Is Bohmian mechanics compatible with relativistic quantum field theory?

ChatGPT:
Bohmian mechanics, also known as the de Broglie-Bohm interpretation of quantum mechanics, is a theory that proposes that particles in the quantum world are guided by an underlying wave function and that their behavior can be predicted deterministically. Bohmian mechanics is based on the idea that particles have well-defined positions and velocities at all times, even when they are not being observed, and that these positions and velocities are determined by the wave function.

Relativistic quantum field theory is a theory that combines the principles of quantum mechanics with those of special relativity, which is the theory of relativity that describes the behavior of objects moving at constant speeds. Relativistic quantum field theory is used to describe the behavior of particles and fields in the context of special relativity, and it has been successful in explaining a wide range of phenomena in particle physics and condensed matter physics.

Bohmian mechanics has been shown to be compatible with non-relativistic quantum mechanics, which describes the behavior of particles moving at speeds that are much slower than the speed of light. However, it is not clear whether Bohmian mechanics can be reconciled with relativistic quantum field theory, as the two theories are based on different conceptual frameworks. Some physicists and philosophers have suggested that Bohmian mechanics could be extended to incorporate special relativity, while others have argued that it is incompatible with the principles of special relativity.

Quite impressive, isn't it?

 

[Demystifier]

Me: Since you are so smart, do we need Physics Forums?

ChatGPT:
Physics forums are online discussion platforms that provide a space for physicists and people interested in physics to discuss and exchange ideas about different topics in the field. These forums can be a useful resource for people who are looking for information about a particular topic in physics, or for those who want to engage in discussions with other people who have a similar interest in physics.

While I, as a language model, am able to provide information and explanations on a wide range of topics, I do not have access to the Internet and am not able to browse for or access current information about specific topics. Additionally, I do not have personal experiences or perspectives, and my responses are based on the information that I have been trained on. Therefore, physics forums can be a valuable source of information and discussion, as they provide a space for people to share their own knowledge, experiences, and perspectives on different topics in physics.

 

[PeterDonis]

Quite impressive, isn't it?

In a way, yes. But note that while it's not reliable--you can't trust what you read if you don't already know the subject matter--it seems reliable to someone who doesn't already know the subject matter. Which means that, as text like this starts to spread over the Internet, wrong information will drown out correct information even more than it does now.

To give just one example, consider this:

special relativity, which is the theory of relativity that describes the behavior of objects moving at constant speeds

It sounds just as authoritative as the rest of the passage, and it's wrong. In fact it is repeating a common misconception about SR (because, of course, it's finding that misconception in its training data).

 

[martinbn]

I have tried ChatGPT, an artificial intelligence system, to ask hard questions.

Quite impressive, isn't it?

I don't know. Sounds like it is quoting wikipedia articles or whatever came first after a web search.

 

[Demystifier]

It sounds just as authoritative as the rest of the passage, and it's wrong. In fact it is repeating a common misconception about SR (because, of course, it's finding that misconception in its training data).

It makes errors, just like real intelligence. Moreover, it can correct itself.

Me: You said: "Relativistic quantum field theory is a theory that combines the principles of quantum mechanics with those of special relativity, which is the theory of relativity that describes the behavior of objects moving at constant speeds." But it's wrong, theory of relativity does not only deal with constant speeds.

ChatGPT:
Thank you for pointing out the error in my previous response. You are correct that the theory of relativity, including special relativity, describes the behavior of objects moving at any speed, not just objects moving at constant speeds.

Special relativity is a theory of relativity that was developed by Albert Einstein in the early 20th century, and it describes the behavior of physical laws and phenomena in inertial frames of reference, which are frames of reference that are moving at a constant speed in a straight line. Special relativity is based on the principle of relativity, which states that the laws of physics should be the same for all observers, regardless of their relative motion. Special relativity includes the idea that the speed of light is always the same for all observers, regardless of their relative motion, and that time and space are relative, meaning that they are not absolute quantities but are rather dependent on the frame of reference of the observer.

General relativity is a theory of relativity that was developed by Einstein as an extension of special relativity, and it describes the behavior of physical laws and phenomena in any frame of reference, including frames of reference that are accelerating or under the influence of gravity. General relativity is based on the principle of equivalence, which states that the effects of gravity and acceleration are equivalent, and it describes gravity as a curvature of spacetime caused by the presence of mass or energy.

Both special relativity and general relativity are fundamental theories of physics that have had a profound impact on our understanding of the nature of time, space, and matter, and they have been extremely successful in explaining a wide range of physical phenomena.

 

[PeroK]

Quite impressive, isn't it?

It's very impressive the way it can collate and present information. I can't say, however, that I see much in the way of independent thinking. To take an example from when I was working. When I joined an IT project I would often have to look through everything and try to formulate some coherent statement of what was going on. Often in the context of why the project was failing! There was no point in regurgitating what everyone else had already written. I had to provide something original and elucidating - essentially material that did not yet exist. And that included identifying where existing material was wrong in some way. (Like what Peter did with SR and constant velocity above).

It's possible that what we as humans do is no more than glorified and more sophisticated collation, but I'm not convinced. There is a genuine spark when real intelligence is involved to see beyond the material that's presented and create, even in a modest way, something original.

 

[Demystifier]

I don't know. Sounds like it is quoting wikipedia articles or whatever came first after a web search.

Me: Someone said that you sound like quoting wikipedia articles or whatever came first after a web search.

ChatGPT:
As a language model, I do not have access to the Internet and am not able to browse for or access specific websites or online resources. My responses are based on the information that I have been trained on, which includes a wide range of texts from various sources. These texts have been carefully curated and include scientific articles, books, and other reliable sources of information on a wide range of topics.

I am designed to provide accurate and helpful information in response to user inquiries, and I strive to provide well-reasoned and well-supported responses that are based on the best available evidence and knowledge. If you have any specific concerns about the accuracy or reliability of my responses, please don't hesitate to let me know and I will do my best to address your concerns.

 

[Demystifier]

It's possible that what we as humans do is no more than glorified and more sophisticated collation, but I'm not convinced. There is a genuine spark when real intelligence is involved to see beyond the material that's prsesented and create, even in a modest way, something original.

Me: Can you be creative and original?

ChatGPT:
As a language model, I am able to generate original responses to user inquiries based on the information that I have been trained on. I have been designed to be able to generate human-like text and can produce a wide range of responses to user input, including responses that may be considered creative or original. However, it is important to note that I am not a human and do not have personal experiences or perspectives, and my responses are based on the information that I have been trained on.

I do not have the ability to experience emotions, make personal choices, or engage in independent thought in the same way that humans do. My responses are generated using machine learning algorithms that allow me to analyze the structure and content of the input I receive and generate appropriate responses based on the patterns and connections I have learned from the data I was trained on.

While I can generate original responses based on the information that I have been trained on, I do not have the ability to create new ideas or concepts in the same way that a human can. My responses are limited to the information and patterns that I have learned from the data I was trained on, and I do not have the ability to think creatively in the same way that a human can.

 

[martinbn]

Me: What is an automorphic representation?

ChatGPT:
An automorphic representation is a type of representation of a group or algebraic structure that is invariant under a given automorphism. In other words, it is a way of representing the group or algebraic structure in such a way that the representation is preserved under the action of a particular automorphism.

An automorphism of a group or algebraic structure is a type of symmetry that preserves the structure of the group or algebra. It is a bijective mapping from the group or algebraic structure to itself that preserves the operation of the group or algebra.

For example, if G is a group and φ is an automorphism of G, then for any g and h in G, we have that φ(g * h) = φ(g) * φ(h), where * is the operation of the group. This means that if we represent the elements of G using a particular automorphic representation, then the operation of the group will be preserved under the action of the automorphism.

Automorphic representations are important in mathematics and physics because they provide a way of studying the symmetries of groups and algebraic structures. They are also used in the study of number theory, where they are used to study the symmetries of certain algebraic structures called "algebraic number fields".

Well, not even close! My guess is that it never says "I don't know". To me it looks like someone who doesn't understand anything but can try to collate an answer based on a web search. It is very fast, but equally useless as the guy with the web search.

 

[PeterDonis]

My guess is that it never says "I don't know".

It can't.

To me it looks like someone who doesn't understand anything but can try to collate an answer based on a web search. It is very fast, but equally useless as the guy with the web search.

That's exactly what it is, except substitute "a fixed set of training data pulled from the Internet" for "a web search". It doesn't search the web dynamically based on what you ask; its underlying data set is fixed (although I think they update the data set when they do a new version).

 

[PeterDonis]

it can correct itself.

In the sense that if you ask it a question that includes a statement that it made a mistake, it will happily agree that it made a mistake and hork back another block of text. Which, in this case, includes another error:

it describes the behavior of physical laws and phenomena in inertial frames of reference

I've lost count of how many times I've had to correct this one in PF threads.

Btw, have you tried telling it it made a mistake when it actually didn't and seeing how it responds? (My prediction is that it will thank you for telling it that it made a mistake even though it didn't.)

 

[martinbn]

It can't.That's exactly what it is, except substitute "a fixed set of training data pulled from the Internet" for "a web search". It doesn't search the web dynamically based on what you ask; its underlying data set is fixed (although I think they update the data set when they do a new version).

I asked. It said that it uses information available up to 2021.

It reminds me of a politician. It writes a lot with the right key words but it is just some generalities without actual content.

 

[PeterDonis]

It said that it uses information available up to 2021.

That would be when the final training data set was assembled for that version.

 

[Orodruin]

It can't.

It can

 

[PeterDonis]

It can
View attachment 319421

What was this in response to?

 

[Orodruin]

What was this in response to?

I asked it nicely to type out ”I don’t know”

 

[PeterDonis]

I asked it nicely to type out ”I don’t know”

Ah, that explains it.

 

[Orodruin]

Me: Write a quantum mechanics textbook exerpt in biblical style.

Bot: And it came to pass that the Lord did speak unto the children of men, saying "Verily, I say unto thee, all things are not as they seem. For within the smallest of particles, there are mysteries beyond thy understanding. These particles, called quanta, do not behave as thou wouldst expect, for they are both wave and particle, existing in multiple states at once. They can be in two places at the same time, and their actions can be entangled across vast distances. Thou shalt call this strange behavior quantum mechanics." And the children of men did ponder these words, and they were filled with wonder at the Lord's creation.

 

[Demystifier]

It also "understands" jokes.

Me: Explain this joke: Schrodingers's cat walks into a bar. And doesn't.

ChatGPT:
This joke is based on the concept of Schrödinger's cat, which is a thought experiment in quantum mechanics. It was proposed by the physicist Erwin Schrödinger in 1935 as a way to illustrate the strange and counterintuitive nature of quantum states.

In the thought experiment, a cat is placed in a sealed box with a device that has a 50% chance of releasing a lethal dose of poison. According to the principles of quantum mechanics, the state of the cat (alive or dead) is indeterminate until the box is opened and the state is observed. This means that until the box is opened, the cat is simultaneously both alive and dead, and it is not until the state is observed that the cat becomes either alive or dead with certainty.

The joke plays on this idea by stating that "Schrodingers's cat walks onto a bar," implying that the cat is alive and able to walk. However, the punchline then contradicts this by stating "And doesn't," implying that the cat is not alive and therefore cannot walk. This creates a humorous contradiction and plays on the idea of the indeterminate state of the cat in the thought experiment.

The joke might be particularly funny to people who are familiar with the concept of Schrödinger's cat and the principles of quantum mechanics, as it references something specific about those subjects. It might also be amusing to people who enjoy wordplay and jokes that play on contradictory statements. However, as with all jokes, the humor of this joke is subjective and might not be appreciated by everyone.

 

[Demystifier]

Btw, have you tried telling it it made a mistake when it actually didn't and seeing how it responds? (My prediction is that it will thank you for telling it that it made a mistake even though it didn't.)

Why don't you try by yourself? It's fun to play with it.

 

[martinbn]

The joke doesnt imply anything about wether the cat is dead or alive.

 

[Maarten Havinga]

It just combines texts humans put on the internet. If it took itself as bonus source and the internet were clouded with chatGPT's, the quality of the internet would degrade very swiftly.

 

[Jarvis323]

Me: Can you be creative and original?

ChatGPT:
As a language model, I am able to generate original responses to user inquiries based on the information that I have been trained on. I have been designed to be able to generate human-like text and can produce a wide range of responses to user input, including responses that may be considered creative or original. However, it is important to note that I am not a human and do not have personal experiences or perspectives, and my responses are based on the information that I have been trained on.

I do not have the ability to experience emotions, make personal choices, or engage in independent thought in the same way that humans do. My responses are generated using machine learning algorithms that allow me to analyze the structure and content of the input I receive and generate appropriate responses based on the patterns and connections I have learned from the data I was trained on.

While I can generate original responses based on the information that I have been trained on, I do not have the ability to create new ideas or concepts in the same way that a human can. My responses are limited to the information and patterns that I have learned from the data I was trained on, and I do not have the ability to think creatively in the same way that a human can.

These specific responses are heavily reinforced, so that they pop up whenever issues related to sentience or comparison with humans are discussed. This is to avoid problems stemming from those issues, like existential crisis, AI rights activism, or people developing unhealthy personal relationships with AI.

 

[PeterDonis]

If it took itself as bonus source and the internet were clouded with chatGPT's, the quality of the internet would degrade very swiftly.

It doesn't even need to include itself in sources; humans are already posting chatGPT output all over the Internet, and the next version will have all of that data in its training set.

These specific responses are heavily reinforced, so that they pop up whenever issues related to sentience or comparison with humans are discussed. This is to avoid problems stemming from those issues, like existential crisis, AI rights activism, or people developing unhealthy personal relationships with AI.

And the very fact that those problems are expected to occur is what economists call a negative externality. In this case, I suspect it will turn out to be a huge one.

IMO this is one of those cases where the people who put this out on the Internet for everyone to use really, really should have stopped to consider the environmental impact, so to speak.

 

[Demystifier]

Me: Write a poem about Schrodinger cat.

ChatGPT:
Schrödinger's cat, both alive and dead
Trapped in a box, its fate left unsaid
Quantum mechanics, a strange and perplexing game
The cat's fate, both definite and mundane

But open the box, and the cat's state is clear
It's either alive, or it's not, that's clear
The superposition, it disappears
Leaving us with just one reality, dear

But what if the box, we never do peek?
The cat's fate, forever a mystery unique
Both alive and dead, it remains in suspense
A paradox, a conundrum, a strange quandary immense

Schrödinger's cat, a thought experiment bold
Leaves us wondering, what if, what if, what if untold?

(EDIT: I've googled this poem, and google didn't find anything similar. ChatGPT made it up.)

 

[Maarten Havinga]

If it can
1. Identify and rephrase the central points of a topic
2. Generate rhymes

Then this is simply what you get.

 

[Jarvis323]

It just combines texts humans put on the internet.

Is there any possible written response to anything which couldn't be said to be a combination of text on the internet?

You can write an abstract poem on the spot and it can analyze it. Or you can propose a random hairbrained idea to it and it can tell you what's wrong with your idea.

Sure those answers will still be just combinations of text that has already been written (e.g., orderings of words and punctuation).

Obviously there is more to it. Unfortunately, just like for a human, it is not easy to explain what that is.

 

[Maarten Havinga]

If I ask about Schrodinger's cat only, and it starts on an explanation of neutrino oscillations how they are also a superposition state, then I'd be tempted to think it has intelligence.

 

[Maarten Havinga]

Intelligence is to me not only that you understand a question, but also decide where to go with that question beyond its scope, not randomly but based on your estimation of where/who the question came from and feeling of what is appropriate to answer in this situation.

If I ask it to explain a "fact" which is purely not true, it will try to do so instead of - well, I expect a machine to give a 400 bad request error in these cases. That would be more intelligent

 

[vanhees71]

Philosophers sound pretty similar :-). SCNR.

 

[Nugatory]

[Chatbot says]

…moving at any speed, not just objects moving at constant speeds.

That’s not much of an improvement. I think we’d have to prompt it a bit more before it gets the words “acceleration” or “non-inertial” into the response.

 

[Hornbein]

It's possible that what we as humans do is no more than glorified and more sophisticated collation,

I'd say that this describes the great majority of human intelligent behaviour. And most people can't do this.

I'm a musician. Creativity in that field is at least 90% glorified and more sophisticated collation. Look at Youtube videos by amateur musicians. The successful ones are usually a copy of something, often with great (sometimes astonishing) exactitude. This is what is in demand. Pro musicians don't usually do that in public, but generally they have spent a great deal of time doing this in their bedrooms. Eventually they learn to draw on established work and mutate it a little bit. The result will sound mainstream and familiar. This can be sold.

On the other hand, the van Halen brothers said they were failures as session men. Their sound had too much originality. They couldn't get enough work.

I'm told that physics and math publication is more restricted than this. Original work can't be published because there are no peers to review it. It might be wrong. Readers might not be interested. Instead what you get are minor variations on familiar themes. A mathematician who needed a track record wrote he gave up on originality and concentrated on minor variations on established topics. If a journal has already published 100 papers on a topic there is a very good chance they will publish a 101st.

but I'm not convinced. There is a genuine spark when real intelligence is involved to see beyond the material that's presented and create, even in a modest way, something original.

Just yesterday I used an AI program that creates images on request. I was happy with its originality. What's more though, it was able to get my style out of some database and present me with new ideas. I quite liked seven out of the eight (the eighth was too cliche). I had to be impressed.

In sum I'd say that these programs are more intelligent and original than the great majority of people. They might not be Einsteins, but they are working on it. Who knows what they will be doing ten years from now? One hundred years? One thousand?

 

[PeterDonis]

In sum I'd say that these programs are more intelligent and original than the great majority of people.

The word "intelligent" can be defined in many ways, but at least one way implies some kind of understanding of the material. That's what ChatGPT doesn't have. So-called "AI" programs might indeed be very helpful at sifting through an astronomical number of possible permutations of some data set and picking out ones that are likely to meet a human aesthetic criterion, such as an image that looks reasonably good or a piece of music that sounds reasonably good. But meeting an aesthetic criterion is not the same thing as being correct and knowing you're correct because you understand the subject matter. (Arguably this notion of "correct" doesn't even apply to aesthetic judgments like those we make about images or music.)

 

[Hornbein]

The word "intelligent" can be defined in many ways, but at least one way implies some kind of understanding of the material. That's what ChatGPT doesn't have.

I'm saying that this is also something the great majority of people don't have.

 

[PeterDonis]

I'm saying that this is also something the great majority of people don't have.

With regard to particular subject matter, yes, that's probably the case. The great majority of people probably don't understand relativty or quantum mechanics, for example, meaning that they could not give correct answers to questions about those subjects, say, here on Physics Forums. Is your point simply that ChatGPT, which can't do that either, still does as well as most humans even if it is not "intelligent" in that sense?