ChatGPT is here: Should humans be worried?
ChatGPT is here: Should humans be worried?
(Not yet! But...)
Hi readers,
Yes, I am on my winter break, but something interesting came along that I wanted to share with you. I am talking about ChatGPT, a new generation of chatbots, powered by the latest advancements in natural language processing and machine learning. I managed to get an account and have been running tests every day: you will find some screenshots in this newsletter. I find it fascinating and scary at the same time and I can already see the implications for the Future of Work.
And in case you are wondering - yes, I did write this newsletter myself. You don’t think a chatbot could ever replace me, now do you? 😉
Meeting ChatGPT
I am sure you’ve seen examples of ChatGPT output since it arrived two weeks ago. Social media is flooded with them, including debates if this will be the end of student essays (no) or why ChatGPT doesn’t know what the fastest marine mammal is (the killer whale). People are equally stunned by how good and how bad it is. Version 3.5 of this chatbot is capable of having coherent conversations with humans on a wide range of topics. It can even provide useful information and advice, like recipes and interior design suggestions. But what I want to explore today is: is ChatGPT a threat to human workers? Should we be worried about the rise of artificial intelligence?
You know that I am always looking for new tools and technologies to experiment with. So when I heard about ChatGPT and understood it was a natural language processing tool that uses machine learning algorithms to write complex answers in real time, I knew I had to give it a try.
I did think people’s initial reactions were overblown so I wasn't sure what to expect. I have tried other AI writing apps before, but none were as advanced and sophisticated as this one. It is really easy to get started with ChatGPT: You enter a question and the tool writes a response using its knowledge base. It is almost like having a conversation with a very smart (and often frustrating) person.
It’s also addictive: once you have read the answer, you can ask follow up questions and ChatGPT will provide more details until it gets stuck. And it gets stuck a lot. Sometimes it replies that you ask too many questions and should slow down, and then it simply stops responding. I have not yet discovered if this is because of server issues or because they limit everyone’s daily usage. It happens less frequently in the European morning, so my guess is that they are overwhelmed by the popularity. Anyway.
Write me a story, please
I decided to put ChatGPT to the test and see how well it could assist me with some writing. I asked it to write a “Day at work in 2050” and this is what it produced:
I think the story is boring and not very futuristic, but it’s clear that ChatGPT understood my question and answered it appropriately. It wrote the text in first person ( I asked it to do so in a question leading up to this one) and it included technologies, like teleporting, that we don’t use today. The writing has a nice flow to it, and the story describes a day in the life from morning until evening. And it churns out text at surprising speed. So far so good.
The writing style is convincing: if I had told you I wrote this story you probably would have believed me. So I thought I needed to level it up by engaging the bot in a conversation.
Let’s talk about you
The reason you can have a real conversation with ChatGPT is because it’s persistent. And that makes all the difference. Once you state the topic of conversation in the first question, ChatGPT continues to answer questions in the spirit of that topic. That is very convenient, as it allows you to dig deep into a topic without constantly having to repeat it in your question. It’s almost like having a conversation with a real human being: on occasion I found myself using “please” and “thank you”, just as you would in a regular conversation.
It’s also repetitive, as you can see in the example above. When you reach a certain point in the conversation it keeps repeating similar phrases, almost as if it has reached the “end” of the knowledge base, and doesn’t have anything new to add. The interesting thing is that you can share your frustration, and ask it to come up with a better answer. Which an actual human probably wouldn’t accept - they would just tell you to stop asking so many questions!
Now write me a Wikipedia entry
Next, I asked it to write my Wikipedia entry. I had seen that other people did that and I thought it would be fun. Even though ChatGPT’s knowledge base was cut off at 2021 (as it frequently reminds you), I assumed it should be able to come up with enough information about me to write a reasonable bio. This is what it wrote:
It’s clear that ChatGPT knows what Wikipedia is, and can apply that knowledge to come up with a coherent text in the correct style. So far so good: it understood the question, it can write and knows what is required for a Wikipedia entry. Impressive.
The problem: none of it is true. I guess I am not yet famous enough to be in the knowledge base. I am not an artist, and I don’t share my name with someone who is. But the real-world information (cities, schools, museum) makes it sound real and even convincing. (Go ahead, try to find my artist namesake and if you do, let me know!) I repeated this question while giving more information about myself (where I worked, what I do) and on the next attempts I became a successful fashion CEO, a politician and an intelligence analyst. All great choices, but totally made up.
Since I tried this last week, I’ve noticed that ChatGPT now refuses to write Wikipedia entries for people that are not in its knowledge base. But give it a try anyway and see if it works for you.
You don’t know what ChatGPT doesn’t know
While I was initially impressed by ChatGPT's ability to quickly generate high-quality text, I also encountered limitations and challenges that made me pause. I even started to question the sincerity of the tool as well as its knowledge base.
For one, ChatGPT does not always understand the nuances of language and context. I found that it sometimes generated responses that were off-topic or irrelevant to the question I asked. This makes it difficult to use ChatGPT as a reliable source of ideas and inspiration. For example, when I asked for a Top 10 list of the greatest humans, the reply included a certain German who started World War II. When I asked why, ChatGPT replied that while his impact was largely negative, he is considered a most influential figure because he had a significant impact on the world. Okay then.
Which brings us to the main problem of ChatGPT: even though a lot of it is great, you don’t know what isn’t. It makes up stuff. A lot. Social media is full of examples of these mistakes. Unless you are knowledgeable on the topic of conversation, you don’t know what is and isn’t true. A lot of it is, but the devil is in the details. Take this list for instance, and see if you can spot the invention that is definitely not American:
It also uses sources without telling you which ones. Sometimes it even plagiarizes without you knowing it. When I asked it for a definition of the metaverse, it gave me the Gartner definition (I ran the answer through a search engine) without referencing the source. That’s a big problem.
It’s also unethical: we don’t know what the knowledge base contains, but ChatGPT refers to “books, articles and websites”. But texts that are publicly available are not free for all. For example, I write my newsletter for free, and it is in the public domain, but that does not mean that anyone can reuse my writing without properly attributing it to me. And even when you quote my writing (please do), it’s still mine. I wouldn’t like my texts to be used to educate a chatbot that then spits out free variations.
And sometimes it knows things, but it claims not to know them. I guessed that it would probably have the complete text of the Bible in its knowledge base, but when I asked about the Bible as book, it answered it wasn’t able to access books. When I rephrased the question, it gave me the correct answer. I discovered it can quote all the Bible verses. I tried some other famous titles, and sometimes that works. For instance, it can quote from “Hamlet” by Shakespeare but not from “For Whom the Bell Tolls” by Hemingway.
Another issue I encountered with ChatGPT was its lack of flexibility and customization. The tool offers a limited set of options for controlling the output, such as the length and style of the generated text. It can’t count: whenever I specified the output in number of words or characters, the text never had the correct length. What it wrote as “creative” or “funny” text, I did not always found creative or funny. This made it difficult to fine-tune the output to match my specific needs and preferences.
Despite these limitations, I still think that ChatGPT is a useful tool for brainstorming and generating rough drafts or outlines. However, I would not recommend relying on it as a sole source of ideas or inspiration. Instead, I would suggest using it together with other tools and techniques, such as old-fashioned brainstorming and research (with references!!), to generate a more diverse and well-rounded set of ideas.
HR chatbots beware
I mostly enjoyed ChatGPT because of its contextual understanding and persistence. I have used a lot of HR chatbots, and they are all pretty straightforward: you ask a question and it provides an answer. But if the answer doesn’t help you, you have to rephrase your question and start all over again. It’s hard to have a continued conversation.
Most chatbots I have tested use workflows on the backend. The chatbot validates the input, reviews a set of potential answers and decides which one is the most appropriate. Because ChatGPT is persistent, and that delivers an almost human experience. Once you start a conversation, you can continue without having to remind it of the topic. And when it provides an answer, you can give it feedback why you don’t like the response, or why you think the response is incorrect, and it will review the answer and provide you with an updated one. I even got it to incorporate some of my feedback in a next iteration of the question. Impressive, but also dangerous: who says that my suggestion is correct?
I realize that that behavior is unwanted in HR: just imagine someone debating with a chatbot until they receive the answer they want. When it comes to salary raises, you can see how that could be a problem. But aside from that, the ChatGPT interaction is so much better than any chatbot I have ever used. The persistent conversation, the fact that it has an almost human voice, the fact that you can ask it to explain something that you did not understand without having to repeat the question, it all adds up to a very nice experience. And the HR knowledge base is so much smaller than the one ChatGPT uses, that I could see how this would take the HR chatbot world by storm.
So in that sense, ChatGPT is setting the golden standard for chatbots. I hear a new version (4.0) will be released next year, and it’s supposed to be even better. The interaction quality is something all HR chatbot vendors - and customers - should be worried about. After all, isn’t the human experience the 2023 trend everyone in HR is aiming for?
So what’s the verdict? Should humans be worried?
I don't think we should panic just yet and assume that ChatGPT is going to replace humans in the short run. It’s too often wrong, which means it can’t be trusted at all. We don’t know what its knowledge base consists of and if these are reliable sources.
And even though ChatGPT is a better AI, it can't do everything: while it can write creatively, it doesn't think creatively or come up with new ideas by putting together two concepts that don’t belong. It can't adapt to changing circumstances, or solve complex problems. In other words, it’s good at what it is programmed to do, but it’s not going to replace the human mind anytime soon. And if you’re not convinced, take some time to read up on the ELIZA effect.
But, as I always say in my keynotes: we better learn how to work with bots, while remembering that it’s still a bot. It’s not a human and it should not be treated as such. OpenAI has delivered an impressive chatbot, and that comes with great responsibility. Its mission of “developing friendly AI in a way that benefits humanity as a whole” is in danger if they aren’t more transparent about their approach.
Despite the issues I encountered, my experience with ChatGPT was positive. If you are knowledgeable on a topic and you explain what you need, it can quickly write an outline that helps you structure your thinking process. It can even write it in a certain style (funny, academic, etc) and rewrite it as many times as you need. It exceeded my expectations in terms of its capabilities and ease of use, and I will definitely use it again in the future.
Just not for writing this newsletter. That is going to remain uniquely me. Promise.
PS: Penicillin was discovered in 1928 by Scottish bacteriologist Sir Alexander Fleming while working in London.
Get in touch
My newsletter is free: please think of me when you need an engaging and interactive Future of Work keynote!
Or support me by buying my book How to Select Your Next Payroll.
I also provide advice. You can connect with me here:
What can I do to make the newsletter better for you?
Did you enjoy this newsletter?