ChatGPT and Solving Math Problems

In December 2022 OpenAI announced the public availability of ChatGPT — a more conversational AI.

I wanted to investigate how well it can now solve common mathematics problems — not basic computational ones that google can solve such as “how many miles is 15 km” but actually show/tell me how to solve a problem.

Pythagorean Theorem

First, I asked it a pretty guided question involving the pythagorean theorem:

“How do I use the pythagorean theorem to solve for the hypotenuse when the other two sides are 9 and 16?”

Then, I removed some scaffolding hints from the previous question:

Pretty impressive — code included.

Next, I wanted to see if it could be used to help people know how to solve broader cases:

“How can I find the missing side of a non-right triangle?” and…

Here I was impressed because ChatGPT not only used the law of cosines but showed an example of it, etc.

Logic Puzzles

Ok, so when it comes to a common mathematical (high school level) problems, it does pretty well! But what about more complex puzzles?

My kids have been into discussing questions from a “historical connections” book published by the AIMS education foundation and so I put one of the puzzles from the book by Benjamin Banneker into Chat.

Suppose a ladder 60 feet long is placed in a Street so as to reach a window on one Side 37 feet high, and without moving it at bottom, will reach another window on the other side of the Street which is 23 feet high, how far across is the street?

The result:

My own scratch work (sorry limited paper and writing utensils as my desk!) shows this is wrong… Chat GPT thought the window heights were to be subtracted instead of computed separately then added. (right?) Still, I was very impressed it was able to conceptualize the ‘building’ and ‘window heights’ and convert that to mathematical data. It would be a great “my favorite no” kind of response to get kids to think a lot.

However, I also selected ‘thumbs down’ on my Chat GPT instance and gave the feedback that the solution didn’t take this into account. I’ll keep hitting refresh until it gets it right, and thus help train the model better. That’s the beauty of AI and machine learning… they don’t tire of learning from mistakes!