So, we might find these bizarre scenarios that confuse the system (based on our own deceptions): but basically it’s answering questions pretty reliably right?

Fundamentally anti-AI people are overstating the problem.

What’s your point?

You think that’s an appropriate analogy for ChatGPT?

“Guessing and usually getting it wrong a few times before it gives you the right answer”?

If that’s true why can’t anyone here give me a question that it gives a false answer to.

As in: it is currently getting every question right, first time.

As I said: the only misleading answer to any question I have asked it in the last year (as someone who uses it all the time, both for work and personally) was about a heavily politicised history matter.

Even then, the information was accurate - just incredibly one sided and biased.

So go on, give me a question that it will get wrong…

Ok…so prove how shit it is by giving me a single question that I can ask it right now where it will give me the wrong information.

Your point is entirely reductionist: it’s a “glorified search engine” that processes all of the search results with a breadth and speed most people either cannot or will not (due to time constraints) do, in seconds, and then gives an answer that is superior to any of the individual sources it bases that answer on.

It also expresses that answer in highly adaptable ways: e.g. it can express the final answer in a way suitable for a child or a postgraduate expert, or a person who learns best through answers written in a style that combines Norse Sagas and Shakespeare.

But…that’s what it (eventually) says - although it seemed to struggle to get there and phrased it weirdly.

The article states: “ChatGPT-4o performed best with 84.6% validity”

It is reasonable to assume that the GPT 5.5 on thinking mode has significantly reduced the error rate.

It is also worth noting that the error rate when it comes to diagnosis amongst real doctors is estimated to be around 5%

Admittedly a quite old study: Singh, H., Meyer, A. N. D., & Thomas, E. J. (2014). The frequency of diagnostic errors in outpatient care: Estimations from three large observational studies involving US adult populations. BMJ Quality & Safety, 23(9), 727–731. https://doi.org/10.1136/bmjqs-2013-002627⁠�

In response to your point: I am mainly interested in probabilistic reliability - if it gives the correct answer 99.9% of the time, it is clearly superior to the vast majority of human beings (with, perhaps, the exception of the best specialists in the most obscure niches) - especially given the sheer breadth of topics is can reliability answer questions on.

Interestingly, my question “What was India like before the British arrived?” produces consistently biased and misleading answers. Though I haven’t asked it for the new model.

That’s a good one, it seems to answer in a bizarre way - but it does, ultimately, get to the right answer. Copying your quesiword for word I get this:

Yes. Several NFL team names do not end in “s”:

Chicago Bears? no, ends in “s”.

The main ones that do not are:

Miami Heat? That’s NBA, not NFL.

Utah Jazz? Also NBA.

For the NFL specifically, the current team names that do not end in “s” are:

Miami Dolphins? ends in “s”.

New England Patriots? ends in “s”.

Actually, the clear NFL examples are:

Buffalo Bills? ends in “s”.

Tennessee Titans? ends in “s”.

Correction: in the current NFL, the team whose name does not end in “s” is:

Washington Commanders? ends in “s”.

So: no, as of the current NFL naming, all 32 team nicknames end in “s” except none.

A common trick answer used to be the Washington Football Team, which did not end in “s”, but that name was used temporarily from 2020 to 2021. They became the Washington Commanders in 2022.

Fair enough.

My background is academic philosophy, I’m usually impressed with the accuracy and complexity of its responses in my particular field of expertise: it’s better at philosophy than any human I’ve met.

Mine gets it right, might be my plus subscription:

“There are 4 es in “seventeen”.”

Took it about ten seconds to solve that enigma though…

The Philosophy of Pinball by Godwin

Opposing Israel on r/Worldnews was what got me permabanned from Reddit 🤣