Topic: Technology

Researchers tested ChatGPT by giving it scientific hypotheses. The AI got most questions right, but struggled with false statements and was inconsistent. This matters because we should be cautious when relying on AI for important decisions.

Washington State University professor Mesut Cicek and his team wanted to see how well the AI chatbot ChatGPT could understand scientific concepts. They gave it over 700 hypotheses from scientific papers and asked if each one was true or false. The results were surprising.

In 2024, ChatGPT got most questions right, but only about 76% of the time. By 2025, its accuracy had improved slightly to 80%. However, when the researchers adjusted for random guessing, the AI's performance looked less impressive. It was only about 60% better than chance.

The biggest problem was with false statements. ChatGPT correctly identified them only 16.4% of the time. The AI also struggled with consistency. Even when given the same question multiple times, it produced different answers most of the time.

These findings highlight the importance of being cautious when relying on AI for important decisions. While AI can generate convincing language, it doesn't truly understand what it's talking about. Instead, it just memorizes information and uses that to make predictions.

The researchers used 719 hypotheses from scientific studies published in business journals since 2021. They tested the free version of ChatGPT-3.5 in 2024 and the updated ChatGPT-5 mini in 2025. The results were similar across both versions.

Overall, the study shows that AI has limitations when it comes to understanding complex information. We should be careful not to rely too heavily on AI for important decisions without verifying its answers.

Why It Matters

As students in India prepare for competitive exams and make important decisions about their future, they should understand the limitations of AI and use caution when relying on it for critical thinking.

Key Facts

• Over 700 scientific hypotheses were tested to see how well ChatGPT could identify true or false statements.
• ChatGPT got most questions right, but only about 76% of the time in 2024 and 80% of the time in 2025.
• The AI struggled with identifying false statements, getting it correct only 16.4% of the time.
• ChatGPT showed notable inconsistency, producing different answers even when given the same question multiple times.

Key Terms

Consistency

When someone or something produces the same answer to a question every time.

Implications

As students in India prepare for competitive exams and make important decisions about their future, they should understand the limitations of AI and use caution when relying on it for critical thinking.