AI chatbot 'ChatGPT' fails to clear UPSC CSE Prelims 2022 by 30%

Riya Shukla

Mar 14,2023
2 min read
0
515 Views

ChatGPT is designed to generate human-like writing by predicting upcoming word sequences. Unlike most chatbots, ChatGPT cannot search the internet. Instead, it generates text using word relationships predicted by its internal processes.

"ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness," as per Sam Altman, chief executive at OpenAI.

This time, OpenAI's AI chatbot ChatGPT has failed to clear the Indian Civil Services Examination, one of the toughest in the world, according to media reports on Saturday. To check its proficiency, Bengaluru-based Analytics India Magazine subjected it to clear the Civil service exams with questions ranging from topics such as Geography, Economy, History, Ecology, General Science, and current affairs.

Although ChatGPT is limited to knowledge up to September 2021, it was expected to perform better on current events questions. The chatbot is designed to generate human-like writing by predicting upcoming word sequences, and it cannot search the internet like most chatbots. Instead, it generates text using word relationships predicted by its internal processes.

The Magazine asked ChatGPT all the 100 questions from Question Paper 1 (Set A) from UPSC Prelims 2022, an IANS report said. "Only 54 of them were correctly answered by ChatGPT," it reported. Even as ChatGPT's knowledge is limited to September 2021, questions on current events were not answered properly. Meanwhile, ChatGPT also provided wrong answers for non-time-specific topics such as Economy and Geography.

Launched in November 2022, the chatbot has gained immense popularity. It also cleared several exams in the US, including the US Medical Licensing Exam (USMLE) and other MBA exams. It also managed to clear Google Coding Interview for Level 3 Engineers.

To check its proficiency, Bengaluru-based Analytics India Magazine subjected it to clear the Civil service exams with questions ranging from topics such as Geography, Economy, History, Ecology, General Science and current affairs.

To assess its proficiency, Bengaluru-based Analytics India Magazine tested ChatGPT's ability to clear the UPSC exam by asking it all 100 questions from Question Paper 1 (Set A) from UPSC Prelims 2022, covering topics such as Geography, Economy, History, Ecology, General Science and current affairs. However, ChatGPT was only able to correctly answer 54 of the questions, and it provided wrong answers for non-time-specific topics such as Economy and Geography.

Conclusion

OpenAI's CEO, Sam Altman, has commented that ChatGPT's capabilities are limited, and it may create a misleading impression of greatness in some areas. Additionally, reports suggest that ChatGPT also performed poorly in an examination designed for sixth graders in Singapore.