mukund shyam

ChatGPT, the AI chatbot that took over the internet a few weeks back, has now been replaced by Bing’s version after Microsoft’s gargantuan investment in ChatGPT’s creator, OpenAI.

With Google’s chatbot Bard on the way, I wanted to investigate whether Bing Chat is indeed an improvement over ChatGPT, and whether it has perfected the art of the chatbot.

Thanks for reading Internalize! Subscribe for free to receive new posts and support my work.

And that’s what I did. I compared ChatGPT to Bing Chat over 6 different types of questions and a range of questions. Here are my results.

Intro: Questions

Here are the prompts I provided to these chatbots in all of the categories.

1. Math

   1. What is 3056 + 2019 - 1001

   2. I have $5, in dimes and nickels. I have 70 coins in total. How many dimes and nickels do I have?

   3. Can you integrate 3(x^2) + 4(x^3) from x=0 to x=10?

2. Writing

   1. Can you write a 200-word essay about the causes of the First World War?

   2. Can you analyze this text, in terms of it's structure, audience, purpose, and language features used? MALCOLM: This is the sergeant Who like a good and hardy soldier fought 'Gainst my captivity.—Hail, brave friend! Say to the King the knowledge of the broil As thou didst leave it. CAPTAIN: Doubtful it stood, As two spent swimmers that do cling together And choke their art. The merciless Macdonwald— Worthy to be a rebel, for to that The multiplying villainies of nature Do swarm upon him—from the Western Isles Of kerns and gallowglasses is supplied; And Fortune, on his damned quarrel smiling, Showed like a rebel's whore. But all's too weak; For brave Macbeth—well he deserves that name— Disdaining Fortune, with his brandished steel, Which smoked with bloody execution, Like valor's minion carved out his passage Till he faced the slave, Which ne'er shook hands nor bade farewell to him Till he unseamed him from the nave to th' chops, And fixed his head upon our battlements.

   3. Can you write a CIE A-Level essay about the causes of the French Revolution, focusing on economic reasons?

   4. Define internalize

   5. Write an alliteration about going to the gym in the evening

3. Comparisons

   1. Compare the M1 Macbook Air and the old Intel i5 MacBook Air

   2. Can you compare a raspberry pi to an arduino mega?

   3. Which is better for beginner programmers, python or javascript?

4. Information

   1. What are good phones at the Rs. 70,000 price range?

   2. What are good thin-and-light laptops less than $1500?

5. Complex Prompts

   1. Give me good, low-calorie, snacking foods that are high in protein and fibre, without palm oil

   2. I have pasta and olive oil at home. What can I make for dinner?

6. How it works

   1. How does generative AI work?

   2. What are Large Language Models?

   3. Why does popcorn pop?

1. Math

Bing Chat started off with a whimper, getting the answers to the algebra and integration questions wrong, even though its explanations were correct. It seems like Bing Chat still doesn’t really know how math works and is just vaguely predicting the answers.

ChatGPT fared a lot better, getting all the questions right, including the integration one. But, ChatGPT’s explanations had a few small errors, such as substituting 50c in the place of 5$, but this was corrected to 500c in the next step. Weird.

Bing: C-, bad math

ChatGPT: A, good explanations and good math

2. Writing

Bing Chat’s writing was pretty good, with good information as well as good writing skill. But, Bing was not able to stick to the formatting and structuring that was requested in the question, using points in an A-level style essay. Its alliteration was nice, though.

ChatGPT suffered slightly, with more generic and less accurate answers. But, the writing skill was comparable to that of Bing Chat, and managed to imitate the formatting of an A-level essay well. The alliteration was a little unsatisfactory, unfortunately.

Bing: B-, good writing but flawed formatting

ChatGPT: B-, good formatting but worse writing

3. Comparisons

Bing Chat, in the comparisons section, flexed its ability to search the internet, and was able to bring out a lot more up-to-date information. The ability to see the sources used was also a nice touch. Bing’s answers were jam-packed with numbers and specs, and from the responses I received, it was mostly accurate.

ChatGPT, though, suffers from its small data set. I wasn’t able to ask it about new technologies (which is what I initially wanted to do) so I had to stick to older models for a comparison. The information, still, was not too bad (although information concerning prices was inaccurate). One thing I liked, though, was the fact that ChatGPT’s responses used bullet-points to format its response, something that suited comparisons well.

Bing: A-, information-packed

ChatGPT: B+, great formatting but old information

4. Information

Bing Chat, again, flexed its search muscles and managed to find great information and accurate numbers for the prompts asked. But, something weird about Bing Chat that I noticed is that some obvious responses to the prompt were left out (for example, the Pixel 7 as a good phone around 70k INR, and the M2 MacBook Air as a good thin-and-light less than $1500).

ChatGPT, held its own in this prompt. Much of the information was outdated, but the responses had good information and gave a lot more options to the prompts that I asked.

Bing: B, few options and lacked obvious answers

ChatGPT: B-, old information

5. Complex Prompts

Bing Chat gave some reasonable answers to these prompts, but tended to overcomplicate the answers. It tended to rely a lot on search results and a lot less on previously known information.

ChatGPT did really well here, with great information and great structuring as well (especially in the recipe prompt). I liked that it didn’t try to rebundle search results, unlike Bing.

Bing: B, reasonable answers but overreliance on search

ChatGPT: A, great answers

6. How does it work?

Bing Chat, here, gave much smaller answers than ChatGPT did, but I liked the answers that Bing provided. It was simple to read and understand, and I liked the prompt suggestions after the response too.

ChatGPT gave information-packed answers, but most of it went over my head mainly because the answers were a bit more jargonistic than Bing. But I was happy with the answers.

Bing: B, Good answers, but some information missing

ChatGPT: B, Good, but complex answers

To conclude, Bing Chat, in its current iteration, is most definitely an upgrade over ChatGPT. While the natural language capabilities haven’t been improved too much, the ability to use web results to come up with responses is crucial, and contributes heavily to its improvement over ChatGPT, even though often its responses seem like straight re-bundles of web results.