A Chat About ChatGPT

Continuing along with the artificial intelligence (AI) theme, here are a couple of observations about ChatGPT. I talked about it in a recent blog entry. As I pointed out in that post, ChatGPT is intended to make “content creation faster, easier, and more efficient. It uses natural language processing (NLP) technology to generate optimized content tailored to your website’s needs.”¹ It is the darling of its age for AI. It is a sensation much like the Atari back was in the 1970s. Everyone wants to jump on the bandwagon.

But is it really the sensation it seems to be? There is no doubt that it is helping to usher in the age if AI, and there is not doubt it represents AI in its current stage. But, to my way of thinking, it is really no better to the future of AI than the Atari game Pong² was in its day.

No doubt it is breaking ceilings and moving frontiers. Some of its abilities have been extolled in social media and other ways. It is able to write content for such things as marketing materials and perhaps even draft valuation reports for business valuations. Its progeny, still in development in some cases, can provide analytics for financial statements that can be used for multiple internal and maybe external reports and analyses. 

But it can also stub its toe in its current form. It is true that it uses what is available publicly on the internet to develop materials, but it cannot reach behind pay walls nor access other materials that are generally not available to the public. Its use of language is generally good, but it can still be stilted. And it can demonstrate biases that it picks up on its own or is led to them by the users or generators.

Recently The Athletic, a subscription-based sports website, tried its own experiment to see how ChatGPT would perform at fantasy baseball. As it turns out, not too well.  The headline from The Athletic blared, “ChatGPT Sucks at Fantasy Baseball.” Here is an example of an answer (selection) ChatGPT delivered: “Does ChatGPT think Josh Bell will hit over 23 home runs? The AI refused to answer this question, instead going the middle-school essay route of ‘it’s impossible to say for certain … but there is a chance.’” The Athletic gave that response an F-.

Oday Merhi, a senior financial analyst at our own Business Valuation Resources, did his own testing on ChatGPT. He asked the program to talk to him as though ChatGPT was Shannon Pratt. As part of that discussion, Oday asked the following question: “Hello, Shannon, what are the methods you use to value a business?”  Shannon answered, in part, “Great question. There are several methods I use to determine the value of a business, and the choice of method(s) depends on the nature of the business, the purpose of the valuation, and the availability of information."

In summary, I look at ChatGPT as a beginning of the AI era—the AI Atari phase, you might say. If you want to learn more about this I recently did a webinar on this very subject.