Everyone is talking about ChatGPT at the moment, and for good reason. Whether you’ve been reading articles impressively written by the AI itself or hearing about how it will transform [insert industry of your choice], the internet is abuzz with predictions about the impact of Open AI’s latest Large Language Model (LLM). One area it is speculated that ChatGPT could transform is market research and vendor discovery. In this blog, we’ll look at exactly what this would mean and why ChatGPT might not actually be up to the task.
A powerful chatbot with a penchant for lies
So, what is ChatGPT? We have all seen ChatGPT’s impressive outputs. But, before we launch into specific examples, let’s briefly look at how this technology works.
In its own words:
“ChatGPT-3 is a variation of GPT-3 (Generative Pre-trained Transformer 3) that has been specifically trained for conversational tasks such as chatbots and dialogue systems. It has been fine-tuned to generate more natural and human-like responses in conversational contexts.”
This definition raises an important point about what ChatGPT is designed for. Specifically, ChatGPT is an LLM designed to sound like a human when it gives you answers to specific user-asked questions. In other words, it is designed as a conversational Google-like experience. There is an important distinction to emphasise here. When answering questions, ChatGPT is primarily designed to sound like a human, not answer the questions factually accurately. This doesn’t mean ChatGPT won’t often answer correctly. But it does mean that it can very convincingly justify statements which are just outright false. This distinction will be important to keep in mind later, when we start thinking about applying ChatGPT tasks such as market and vendor discovery.
As ChatGPT gains widespread visibility, businesses are keen to start testing the limits of this technology and exploring how ChatGPT can add value to their processes. One example is using ChatGPT for market and vendor discovery. Let’s take a closer look at this application.
A market and vendor discovery assistant: The good
Vendor discovery involves finding new and innovative suppliers in your supply chain, while market discovery involves mapping out and understanding your market to uncover new opportunities and trends.
ChatGPT as a trampoline
A number of ways have been proposed for how ChatGPT might transform market and vendor discovery. Some suggested strategies include this example of how ChatGPT can be used as a great springboard to start your vendor discovery process, coming back with a list of companies in a certain area. Take this example of non-alcoholic gin companies.
ChatGPT as translator
AI is also great at translating one kind of information into another. Take this example where ChatGPT reformats information in format A into format B. This can greatly speed up research processes.
Will ChatGPT transform industries? Probably. ChatGPT is an undeniably cool bit of tech. For both these tasks, ChatGPT can be a pretty useful tool to start off the market and vendor discovery process and help to automate simple jobs. But can we take the role of ChatGPT further? Can we successfully conduct vendor or market discovery using this tool? Probably not. Or at least, probably not very well.
If we consider its design purpose, this shouldn’t be surprising. ChatGPT is designed to be a highly fluent, conversational chatbot (which it is incredibly good at). But it is not designed to conduct market research, to find correct and up-to-date company data, or many other tasks essential for a successful market and vendor discovery process. And assuming that it can perform these tasks can lead to serious problems for businesses and derail your strategies.
To illustrate some of the challenges ChatGPT faces when applied to the task of market and vendor discovery, we’ll now go through three ways ChatGPT falls short. As a point of comparison, we show how Forestreet’s specialist AI-driven market and vendor discovery platform has been designed to meet these challenges.
Using ChatGPT for market and vendor discovery: The bad and the ugly
-
ChatGPT struggles with facts (and often constructs fluent lies)
One serious problem with using ChatGPT for research is its concerning tendency to lie. And these lies can be very convincing, very detailed and very hard to detect. Indeed, OpenAI have published a warning about incorrect output, stating: “ChatGPT sometimes writes plausible sounding but incorrect or nonsensical answers.” This doesn’t sound ideal for a market and vendor discovery tool.
Here’s an example of ChatGPT getting it just plain wrong (Argentina have won 3 world cups).
And another fun example taken from Reddit, where ChatGPT insists (in fluent Danish) that it cannot speak Danish.
These examples are certainly funny. But they pose a serious problem if you’re trying to implement this technology into your business processes. Indeed, the fact that ChatGPT can give you correct information is more of a coincidence than its function. While for simple requests, ChatGPT can provide accurate and comprehensive replies, this propensity to hallucinate answers is worrying and could potentially derail your inquiries. It’s also worth keeping in mind ChatGPT’s currently limited knowledge base which, at least for the moment, only includes data up to 2021; making it ill-equipped to generate insights into a rapidly changing supplier market.
Comprehensive data enrichment and real-time information
This contrasts with Forestreet’s ability to collect and present comprehensive data-points such as size, funding, revenue and location for all companies in a study. Forestreet works in real-time, pulling current company information along with news and events that occurred today.
-
No workflow and all the legwork
Relatedly, ChatGPT is just not that easy to use for vendor discovery. Yes, it’s great for providing you with a convincing sounding answer, list or a neat summary to some question (as long as you are warned that this answer may just be totally false).
But when you’re doing market and vendor discovery, you want to know that your results are comprehensive and feel confident that you’re not missing out a main player. You also want to make sure there aren’t too many errors creeping into your results. If you’re going to be spending large amounts of time and money acting on this information, you need to be sure it is correct. This means you’ll need to do legwork checking ChatGPT’s outputs yourself. Yes, ChatGPT can bring back a list of company names which can be a great starting point to your search. But you are then on your own to Google the companies and confirm that ChatGPT’s list is correct (which from our trials, it was quite often not).
Returning to vendor discovery with non-alcoholic gin
Let’s go back to the example of non-alcoholic gin companies. At first glance this looks like a very useful output. But when we start breaking down the list a bit further, we can spot several errors creeping in.
Suddenly, this list seems a lot less useful, and highlights how important it is to manually check ChatGPT’s outputs. Ultimately, the effort that would go in to creating a comprehensive market overview, cross-referencing it (to ensure it is up-to-date and correct) and then transforming into a presentation or report would not be dissimilar to the amount of work that Google research requires, something which was always an option before ChatGPT.
There is a knack to learning how to engage with ChatGPT in the first place and a new skill needs to be honed: prompt generation. Users will need to learn how to generate effective prompts to get useful outputs from ChatGPT. Or, in other words, they will need to learn to be an ‘AI whisperer’.
Forestreet, on the other hand, is specially designed to be intuitive to use, with the user interface guiding the user seamlessly through the research and discovery process. It provides a workflow, allowing users to easily review each company within the platform rather than just being presented with a list. The platform also allows for easy collaboration with co-workers using comments and tags.
-
Lack of control (and lack of spontaneity)
Building on this last problem, as it is not designed as a vendor discovery tool, ChatGPT’s interface is challenging to navigate. Using ChatGPT’s interface it can be hard to find the information you’re searching for and easily refine your discovery process. There is also less opportunity to surface interesting information you had not thought about before starting the discovery process.
AI platforms can be specialised for vendor discovery
Forestreet will let you remain in control of your research and guide the search in useful directions for your business. Our platform also offers more than answers to specific questions. Within the platform you can explore additional features including powerful analysis tools, real-time news and events and ESG scores. In many ways, our platform can do the questioning for you, letting you iterate and learn as you go and adapting agilely to market fluctuations and changing business needs.
Turns out, ChatGPT is not a great market and vendor discovery tool. If we return to the initial design purpose of ChatGPT, this doesn’t seem very surprising. It’s like digging with a hammer. Yes, it will kind of help, and might be better than digging with just your hands. But, as the hammer wasn’t really designed for this task, you’re going to find you have some problems.
If you want to use AI to supercharge your market and vendor discovery process, there are other options. At Forestreet, we have built an AI tool designed specifically for market research and vendor discovery: a specialist platform, not a generic question and answer machine. If you want to learn more about how Forestreet can supercharge your market and vendor discovery, book a demo here.
In summary
ChatGPT is designed to sound like a human – which it’s really good at. It’s also useful for translating data and providing a starting point for research, which can help streamline and automate your processes. But we should not overestimate ChatGPT’s abilities and keep in mind what the tool was designed to do. If you’re looking to supercharge your vendor and market discovery processes, AI might well be the answer. But it is more likely to be in the form of specialist platforms, than a general chatbot, however compelling and all-powerful that chatbot might seem.
