On 30th November 2022, OpenAI released an extraordinary new artificial intelligence chatbot called ChatGPT, and the internet hasn’t stopped talking about it since. People have found many remarkable uses for it, from coming up with fitness plans to writing full songs with it. Its uncannily accurate text output has also inevitably caught the attention of content marketers who have fallen over themselves with excitement at the prospect of an A.I. that can write anything from product descriptions to full blog posts entirely automatically, in seconds.

As someone who worked in the field of A.I. between 2016 and 2022 - and, specifically, creative artificial intelligence - I find myself cautiously excited by ChatGPT myself, tempered by concern that the excessive hype surrounding it points to a dangerous misunderstanding of its capabilities that will invariably harm misinformed enthusiasts. In this blog I aim to shed some light onto what ChatGPT genuinely can and can’t do, based on my experience with A.I. and my own dabbling with the chatbot itself. As my main domain is marketing I will speak primarily to this vertical, but my comments apply equally to anybody hoping to use ChatGPT for other popular purposes (such as writing code).

Let’s begin.

What is ChatGPT, exactly?

Basically, it’s a very clever chatbot. Its interface presents much like most chat interfaces where you input messages in natural language and receive responses, just like a conversation. Unlike most chatbots, its ability to understand what you’re saying without needing to word the question in a specific way - a typical problem of most artificial intelligence chat systems to date - makes it incredibly accessible to even the most non-technical of users.

Delving deeper, this is ChatGPT… sorry, Assistant’s own explanation of what it is:

To expand on this, ChatGPT’s model was trained on a large dataset of text pulled from the internet. It was fed billions of words from various curated sources, including websites such as Wikipedia, books and articles. From this data it learned patterns which allow it to predict, with somewhat unprecedented accuracy, which words follow each other in a contextual space. Or to dramatically oversimplify, you can think of ChatGPT as some kind of autocomplete on steroids.

In practice this gives it the ability to produce astonishingly human-like responses to a wide variety of questions, from the kinds of thing you might expect the likes of Google and Amazon Alexa to handle:

To more technical examples like a recipe based on specific ingredients, like this exotic concoction:

It can also write very convincing prose and even basic poetry:

You can even tell it to tell the same story from a different perspective:

It can handle translations:

And even write code:

As you may have noticed, it is able to understand context very well. When I asked it to convert the story to a poem, it knew what I was referring to. The example of telling the same story from the bear’s perspective is an even better example of this.

On the surface it looks to be extremely intelligent, and this kind of result has had mouths dropping across the web with some even proclaiming this to be the birth of Artificial General Intelligence, the mythical milestone whereby A.I. achieves a level of intelligence akin to or exceeding human beings. This is a fundamental misunderstanding of how ChatGPT works, but for what it is ChatGPT is a very strong example of what such so-called large language models can do and how they can be used. But they have limitations, as I will cover next.

The limitations of ChatGPT

There are some specific hard limitations that you need to know about before you use ChatGPT, including:

It doesn’t have access to the internet

The dataset that it was trained on includes data up to 2021 but nothing beyond that. This means it cannot comment on anything beyond that year, and doesn’t have access to the latest news or current events.

It is heavily constrained

When ChatGPT first launched, it was more or less open to anything. Which, of course, led to people doing all kinds of nefarious things with it such as asking it to write code to hack a bank. Because of this, and following OpenAI’s core principle of ethical A.I., there are some things that ChatGPT simply won’t do for you because those things are illegal or immoral.

It has biases

Given that the corpus of text it was trained on is from the predominantly white, Western, male landscape of the internet up to 2021, it inevitably contains biases that could get you into trouble if you regurgitated them. Although OpenAI is actively enforcing guardrails against such things, nothing is ever going to be 100% safe.

It doesn’t care about truth

This is the most important limitation of all and one that is most widely misunderstood. Because the function of ChatGPT is essentially auto-completion of text, it does not have any understanding of what it is saying, nor does it have any ability to care if what it says is truthful or not. This will lead it to very confidently spewing out incorrect information on a regular basis, such as the following example where it gives me incorrect information:

The FA Cup final on 18 May 2019 was between Manchester City and Watford, with Man City winning 6-0.

It also struggles with mathematical problems occasionally, such as the following example where it trips over a mathematical riddle (riddles being another area that it struggles with generally, incidentally):

Eventually, after further prompting, it gets the right answer, although its first “right answer” still makes a logical error by stating we are both 42:

Until finally:

Understanding these limitations is key to knowing how to use ChatGPT properly. As you can see, its responses can be extremely convincing if a little generic, and it cannot be trusted in any way to tell the truth. This means that you should only rely on its output if you already know the answer to whatever you’re asking of it, or if you are not relying on its output in any direct way. If you’re using it to write code, you will quickly run into bugs in whatever code it produces and will need to rely on actual coding knowledge to fix them. If you’re asking it to write an essay, don’t expect anything more sophisticated than high-school level 5-paragraph assignments. If you’re using it to write product descriptions, be aware that it will happily write detailed descriptions for a literal non-existent thingamajig:

And so, with that understanding, how can ChatGPT be used for marketing?

How ChatGPT can be used in marketing

Rather than trying to figure it out on my own, I thought I would ask ChatGPT itself how it can help with marketing. Here’s what it said:

An impressive list for sure, but how much of this is actually true? The best way to establish that is to ask it to elaborate on each point, which is exactly what I did. I will not post up every answer it gave me as that would make for a somewhat tedious blog post, and some of the tasks required multiple levels of digging before arriving at a sensible answer. So below I will summarise the capabilities based on my discovery process, highlighting ChatGPT responses only where they were particularly interesting or relevant.

Developing marketing strategies

The chatbot claims to be able to help define marketing goals and objectives, so I started with this. I asked it “Help me define my marketing goals” and it responded by asking me to explain what my company did and who my target audience was. I obliged, and it responded with a pretty solid if rather generic list of marketing goals including “increasing brand awareness”, “reducing client churn” and “improving our website’s conversion rate”. It also told me that these goals should be SMART (Specific, Measurable, Attainable, Relevant and Time-bound), which is also pretty good advice. I riffed with it on a few more specific goals, giving it different target audiences and it consistently came back with solid answers for all of them. It wouldn’t come up with specific goals for my business because it didn’t have access to data on my business, but it did provide some helpful examples and ideas to work from.

Creating marketing plans and campaigns

I then asked it to create a marketing plan for me and it responded with a pretty detailed outline of what a marketing plan should consist of, along with a caveat that it would need to know a lot more about my business in order to actually fill it out for me. Wanting to see how far it could actually go with this, I pushed it to do a SWOT analysis of our agency, and reached its limits as it finally responded:

As with the marketing strategy task, it is very helpful in coming up with examples and ideas as a starting point, but will stop short of actually writing your marketing plan for you.

Identifying target audiences

This task followed much the same pattern, in that it was able to provide example audiences based on my specific criteria. For example, I asked it to provide target audiences for e-commerce clients, and it listed some sensible options such as e-commerce business owners, e-commerce managers and data analysts, with short descriptions of what each of these people do in a work sense. It was equally able to get it to write up personas for each of these that I could use in a marketing plan. Perhaps more directly useful was when I asked it to give me a list of job titles for these audiences that I could use in a LinkedIn search, which it happily provided:

Analyse market data

I was curious about this one due to its limitation of not being web-connected with a 2021 cutoff. I asked it for a list of the top 5 growth marketing agencies in the world and, despite its obviously glaring omission of Growthmode in that list, the response was a mixed bag of well-known growth agencies and platforms such as SimilarWeb. When I asked it for data on one of the companies in the list, it responded that the company was private and therefore it couldn’t help me, suggesting instead that I contact them directly (generally not the “done thing” when conducting competitor research… at least not overtly). I then asked it for stats on the growth marketing industry and it gave me a cop-out answer along the lines of it not being able to reliably source numbers for the vertical, instead giving me vague, unsourced forecasts about general digital marketing spending up to 2020. As I suspected, using ChatGPT for market research will definitely be somewhat limited for this reason, although it can be helpful in providing general narrative on a sector as it did for growth marketing:

Creating marketing materials

Again, I was very skeptical of this one as ChatGPT is only able to produce text. It claimed it could help produce brochures, so when I asked it to create one for me it gave me a general overview of the structure of a brochure, including descriptions of what the cover should have on it (your logo, tagline and an “eye-catching image or graphic”), and then an outline of some of the sections it should contain. As a general guide for somebody who has never done a deck before it could be quite helpful, but certainly fell short of the expectation set in the original task list. When I asked it where I could find templates for brochures it did point me in the direction of Canva and Adobe Spark as well as marketplaces like Envato Elements and Graphic River, which is more helpful. Although notably, Adobe Spark has recently rebranded to Adobe Express which again highlights the out-of-date nature of the dataset.

Researching and identifying new marketing opportunities

I was a little skeptical of this one as well based on my experiences with the first couple of tasks, and was sure it would regurgitate much of what it gave me when exploring marketing strategies. When I asked it to research and identify new marketing opportunities for my business, it spat out a fairly general list of marketing activities I could undertake, such as influencer marketing, social media marketing and building partnerships. It also suggested I should keep an eye on new social media platforms and technologies that may provide opportunities to reach my target market in new ways. All pretty generic stuff really. I asked it to elaborate on how to keep an eye on emerging technologies and it listed a bunch of fairly solid ways to do so including following industry experts on social media, monitoring tech news and attending industry events. All pretty obvious stuff I suppose but sometimes the most obvious stuff gets forgotten.

Email marketing

Here I felt I was venturing into one of the more popular areas of ChatGPT’s potential. Its ability to write convincingly makes it an ideal candidate for churning out email copy and, sure enough, it did a pretty solid job on the first ask:

Interestingly it remembered that we have a specialisation in e-commerce (amongst other things), and brought that in without me needing to prompt it to. This again shows the power of its context memory, which is one of its most powerful features when you use it to iterate the results. For example, while this email is decent, it’s much too long and far too formal for my liking. So I asked it to change that:

This is far closer to something I might actually use. It also claimed to be able to help with creating email lists, and when I asked it to do so it reverted back to providing a list of ways in which I could build out my email list, predictably being unable to create one itself. This is ethically a very good thing on balance, and the suggestions it did provide were genuinely helpful on the whole. On the subject of tracking campaign performance it was a similar story, giving me a list of metrics to track (such as Open Rate, CTR and Unsubscribes), with explanations of what each metric meant. Again, quite helpful if you’re new to this.

Content marketing

This is the most controversial use-case of ChatGPT so far, as it has the ability to write out entire blog posts automatically. For example, here is a snippet of the blog post it wrote for me on the subject of growth marketing:

It wrote a full 500 words on the subject, and would have produced more with additional nudging. As you can see, it reads quite well. If you have spent any time at all browsing the web for articles on subject matter like this you will have encountered many similar articles written in a similar way. This points to the very nature of what ChatGPT is, as it is trained, don’t forget, on a snapshot of the web. Its output can therefore be considered a sort of average of all the web’s content at that point in time, so most of its writing will therefore inevitably come out as familiar and formulaic.

There is a problem with this. The inevitable outcome of A.I. writing being so convincing is that the web will become flooded with generic A.I.-generated content by people looking to game the search engine algorithms. In fact, this very problem is behind Google’s recent Helpful Content update, which is aimed at detecting A.I. content and demoting it in favour of content that provides value. A somewhat nebulous definition of course, but the question of whether A.I.-generated content is detectable has already been answered. Passing my ChatGPT-generated blog post into one of the many emerging A.I. content detector tools, it came out as 99% certain to have been written by an A.I. By contrast, pasting in the first few paragraphs of this blog post that I am writing came out as 100% human. Because it is.

For this reason, ChatGPT should only be used in content marketing as a tool to help with inspiration. It can generate outlines and précis of blog posts for you, but the writing itself still needs to be carried out by a human, if you care about SEO. And speaking of SEO…

Search engine optimisation (SEO)

The penultimate task that ChatGPT claims to help with is in fact SEO. Here it claims to be able to optimize my website and online content, which feels like a claim that needs clarification as it is unable to access the web. As it turns out, the way in which it can help is if you copy and paste the text from your website into ChatGPT and ask it to rewrite it. To test this, I gave it the hero headline from our own website:

Its explanation is quite a useful touch and the amended copy isn’t half bad either. Amusingly, ChatGPT’s rewrite also scored as more human than my original copy according to the A.I. content detector tool. If you haven’t yet written any copy for your website, it can of course help with that too:

This type of thing is where ChatGPT excels in my experience. Writing tight copy for things like website headings is a difficult job and having a tool that can brainstorm with you is extremely valuable.

Pay-per-click (PPC) advertising

Its final task that it can help with is PPC ads. Here it claims to be able to help create and manage PPC campaigns, which again feels like false advertising given its lack of web connectivity. But, once you understand its general capabilities and approach, you can still extract a lot of value out of it to help you with PPC. For example, it will helpfully list out all of the steps you need to consider when setting up a PPC campaign including bid management, A/B testing, keyword research and ad copy. It can directly help with some of these, such as writing or optimising copy for ads and suggesting keywords for your niche. For tasks like producing reports (which it claims to do), it ends up defaulting back to step-by-step outlines of what should go into a report rather than it producing one for you. This is no surprise given the earlier experiences with marketing strategies and plans.

So in the end, what is it good for?

Having gone through a fairly comprehensive exploration of ChatGPT’s capabilities in relation to marketing, I still feel I have barely scratched the surface of what it can do in this area alone, never mind in the wider context outside of marketing. I haven’t touched on its ability to write code and how that can help with setting up Google Tag Manager, nor have I mentioned its ability to write macros for tools like Excel which can really help you level up your efficiency in some of the more tedious areas of digital marketing.

Overall, ChatGPT works very well as a tool for creating boilerplates, brainstorming, optimising texts and giving detailed instructions on tasks you may be less familiar with. It cannot be fully trusted with its output which is why anything it produces should be checked by a human before being published, and it does have some limitations which constrain it in certain aspects, but when used correctly it has the potential to be a powerful time-saving tool and an almost must-have in any forward-thinking marketer’s arsenal.

Its potential for misuse either through direct nefariousness or inadvertent ignorance is going to be a huge challenge for the sector as a whole. Content marketing itself will become a massively over-saturated medium where quality and originality will start to matter more than ever. But ChatGPT’s ability to save time on some of the more mundane aspects of digital marketing should mean, in theory, that people’s time will be freed up so that they can focus on quality over quantity. OpenAI’s commitment to ensuring its tool is not used for undesirable outcomes is encouraging, and while I am, as always, cautiously optimistic about where we are headed with A.I., I am slightly concerned that the hype surrounding ChatGPT will lead to an exponential explosion in A.I. breakthroughs over the next 12 months that could wreak havoc on the digital marketing landscape. For now though, you would be a fool to ignore the benefits of ChatGPT. Give it a try, it won’t bite.

To learn more about what A.I. tools can do for you, contact us.