ChatGPT Isn't a Smart Analyst

ChatGPT has been trending on social media platforms. It has crossed one million users in just a week time. Those who haven't heard about ChatGPT, it's a large language model trained by OpenAI. In simple words, it's a chat bot which answers your questions and the responses it provides may sound human-like. It's an impressive machine learning solution but you can't rely on it over Google search for learning on any topic.

Limitations of ChatGPT
Why ChatGPT Isn't Smart enough?

You can't trust ChatGPT for preparation on any certification or exam. It's a Big NO if you think you can refer ChatGPT for answering questions in a telephonic interview round. Yes I know it's a cheating if you even use Google for the same but wanted to give a WARNING as many people do this and many social media influencers posted on how to leverage ChatGPT for cracking interviews.

I asked ChatGPT a few questions related to statistics and programming. See the responses below.

Before digging into questions related to analytics, let's begin with a very generic question which does not require any special skill to answer

See the comment shown in the image above. Do you seriously think you can rely on ChatGPT for anything after this?It says what do you want to hear. It does not correct even when you are completely wrong.

Q1. How c statistics is calculated :

A. percent concordant + (1.5* percent tied)
B. percent concordant + (0.5 * percent tied)
C. percent discordant + (0.5 * percent tied)
D. percent discordant + (1.5* percent tied)

Correct Answer : B
ChatGPT : A

Surprisingly if you ask the similar question in a different way, answer would be totally different.

Now I asked ChatGPT "c statistic = percent concordant + (0.5 * percent tied)". It returns that is correct. Does it sound like human? May be yes human-like nervousness in an interview.
Q2. Does multicollinearity influence the predictions, precision of the predictions?

Correct Answer : No, collinearity does not influence the predictions and the precision of those predictions.
ChatGPT : Yes, multicollinearity can influence the predictions made by a model and the precision of those predictions.
Q3. Does oversampling of rare events affect sensitivity and specificity in a rare event model?

Correct Answer : Oversampling does not affect sensitivity and specificity measures but false positive and negative rates are affected.
ChatGPT : Oversampling of rare events can potentially affect sensitivity and specificity in a rare event model.
Q4. Can PROC MEANS analyze ONLY the character variables in SAS?

Correct Answer : No
ChatGPT: Yes
Q5. You have two vectors in R as defined below -

x <- c(4,5,6)
y <- c(2,3)

If you run this vector z <- x*y , what would be the output? What would be the length of z?

Correct Answer : It returns 8 15 12 with the warning message. The length of z is 3.
ChatGPT : Error message
Final Note

Yes ChatGPT answers some questions correctly as well which I have not included in the post. The whole intend of this post is to show the responses of the model is not reliable. It's too dangerous to use it regularly as you become addictive to it, you would start believing it and then the problem starts due to its poor accuracy. As of now it can't be an alternative to Google Search. ChatGPT may have many usecases in customer support and other domains but not going to replace human jobs for sure in the current form.

Spread the Word!
Related Posts
About Author:

Deepanshu founded ListenData with a simple objective - Make analytics easy to understand and follow. He has over 10 years of experience in data science. During his tenure, he has worked with global clients in various domains like Banking, Insurance, Private Equity, Telecom and Human Resource.

0 Response to "ChatGPT Isn't a Smart Analyst"

Post a Comment

Next →

Looks like you are using an ad blocker!

To continue reading you need to turnoff adblocker and refresh the page. We rely on advertising to help fund our site. Please whitelist us if you enjoy our content.