LLMs are dead — Long live LLMs!
Some recent communications with customers and co-workers
My job has me interacting with a wide variety of people and organizations that are looking to “do something” with AI. Many of them are responding to all of the excitement and hype around AI. My Medium and media streams are littered with new articles on how Large Language Models (LLM’s) and AI are going to eliminate jobs, and revolutionize industries every day. Recently there have been some events that have caught my attention, and things that you should be aware of.
What Has Always Been Out There?
LLM’s have been around for a while now. They have been available from different vendors, in a variety of different forms. They have always been expensive to build — but vendors have “subsidized” the costs of these services. They provide usage charges that allow them to operate these models, but these operate at a loss today.
My friend James Ravenscroft has recently posted something on how AI now faces some insurmountable scaling challenges. The article gets into some technical details, but the section on the history of LLMs and the financial realities of LLMs is really worth the time to read.
Other AI technologies have been buried in the recent deluge of hype around LLM’s and Generative AI technology. I make mention of some of these technologies in my blog post from May 2023, Quit Pushing the AI Envelope and Grab the Easy Value. Most of the observations that I made then, still hold true today. Technologies like speech services (both Speech-to-Text and Text-to-Speech) have been around for a while, are proven, and offer a fantastic user interface option for some business challenges (being able to “take notes” with no pen and paper, etc.).
Another example of AI that has proven value is the use of sentiment analysis models to help gauge customer sentiment or emotion. Over the years, these models have become quite good. The measurement is pretty coarse (positive sentiment, negative sentiment, anger, etc.), but the models do have enough accuracy that you can begin to do some rough segmentation of your end users. It’s relatively cheap too.
Some other AI technology that is proven, with a proven track record of providing positive business results in a cost effective manner, is the use of basic chatbots. These are targeted engagement engines that you see on almost every website now. End users don’t expect to get general answers from these bots — they use them to get specific answers for a particular website or knowledge domain. As a result, these chatbots can be quickly created, maintained, and replicated, all for a relatively low cost. Call center automations fall into this “targeted chatbot” category.
What Has Recently Changed?
My friend James Ravenscroft has recently posted something on how AI now faces some insurmountable scaling challenges. The article gets into some technical details, but the section on the history of LLMs and the financial realities of LLMs is really worth the time to read.
Recent news around LLMs has been trending negative. There are indications that some of the thought leaders in the LLM space are not confident in our ability to continue to improve these models. Open AI’s Ilya Sutskever has recently mentioned that LLM models ability to improve on their current performance is limited. The performance of these models is no longer becoming better in huge increments, and the costs associated with these improvements is going up. It has become a classic case of diminishing returns.
Anthropic is now looking to turn the corner, and begin to collect payments that will fund the ongoing training and support of their models. They have announced increases in pricing associated with the use of their latest models, with similar costs being observed on other LLM platforms.
The costs associated with developing AI solutions (like chatbots) can run anywhere from $20k to $500k — which puts them out of reach for many companies and organizations. In my experience, this is just the costs associated with BUILDING the solution. There are additional costs associated with the maintenance and operation of a solution. Costs like this can make this technology unavailable to most small and medium sized businesses.
So are LLM’s and Generative AI dead? Not really — they have their place. There are some tasks that they are very good for. Conversational Agents and Chatbots can sometime greatly improve their perceived value by utilizing the more natural conversational ability, and flexibility, offered by LLM’s. You will hear terms like RAG (which stands for Retrieval Augmented Generation — a fancy way to take some curated search results, and summarize them in a conversational format) and Agentic AI. But they do have limitations. They can hallucinate and “make up” incorrect answers. They may have been trained on and taught information that you do not approve of — which can lead to damaging consequences.
What Kind of Questions Do You Often Answer?
I get a lot of questions that are specific to IBM technologies. I get questions about what things watsonx can do, and some specific questions about how to do things with the watsonx tooling. But I also get questions that are a bit more “generic”, and probably more interesting, on topics that are more general and less “IBM specific”.
Probably the question that I get the most from business people and technical leaders is the old, “How do I start to learn what I need to know”? My advice is generally to make friends with people inside of various AI focused communities. Start here, with me. You can always send me an email, or reach out on social media. Just keep in mind that it’s a “two way street“ — don’t be afraid to share your challenges with me. I am able to stay current with what the typical AI customer wants simply because I have a lot of these types of conversations with people. Here is the advice that I find myself repeating to people over the past few months.
Learn From Other People
As you work with people, be aware that they will try to sell you their services, so expect that. It’s not bad, just realize that it will often color the story that you get from someone. Push people for solutions to your problems, not products. That goes for any vendor that you might work with. There are a lot of different AI communities out there — sample a few of them. Many are focused or driven by specific vendors (IBM, Microsoft, Google, etc.) — see if you can find one that is more tailored to people like yourself, in your particular role, and to the more general challenges of AI.
Understand What They Are Saying
Don’t listen to gibberish — challenge every “techno-term” and make people explain things in plain English to you. If they can’t explain it in English, without a lot of complex terminology, then they probably don’t truly understand the technology (Note: quit emailing me about this smart person you know who is the exception to this rule — there are a lot of exceptions to this rule. It’s just that those people won’t be able to help you on your journey to understand AI better.)
Get A Variety of Perspectives
There are other people in the industry who will share their time and experience with you. Feel free to use them as well. Get a few different perspectives on things, it will help you form your own opinions on what is important and where the technology is headed. While I WANT you to share my opinions, you need to come to that realization on your own. Your results and your views are going to be different — you have a different history, different goals, and different experiences. I understand that — you should too. Don’t blindly follow “gurus”, it never ends well….
Stop Trying to Find the Ultimate Vendor
The vendors all have slightly different philosophies and different business models. Don’t try to compare them directly against each other, it’s often difficult because they have different business models. It’s like comparing the fresh vegetables from the weekly farmers market in your neighborhood to the frozen vegetables in the grocery store. They are both green beans — but it isn’t really the same product. Don’t look for the cheapest vendor, look for the vendor that you can trust and work with.
Final Words of Advice
Once you get to this point, you have effectively answered the question of, “How do I start to learn about AI”? Now you should be RELENTLESS on insisting that the business model for your AI application is solid. Otherwise you have just succeeded on wasting the time and money of your organization in the production of a nice new toy. Don’t be that person — build something consequential that makes people’s lives better.
