How AI chatbots work, in plain English

AI chatbots can draft emails, summarise reports and answer questions in natural language. The technology behind most of them is the large language model, and its core idea is easier to grasp than it sounds.

A short history

Computers have been processing language for decades. Early systems relied on rules written by hand, and later ones on statistics about which words tend to appear together. Neural networks, loosely inspired by the brain and built from many simple connected units, gradually took over during the 2010s as computers became powerful enough to train them on large datasets.

A turning point came in 2017, when researchers at Google described a design called the transformer. It made it practical to train much larger models on much more text. Most of today’s chatbots are built on that design. Chatbots became a mass-market product after ChatGPT was released to the public at the end of 2022, and several companies now offer competing models.

Predicting the next word

A language model is trained on a very large amount of text. During training it repeatedly sees a passage with the next piece hidden and tries to predict it. Each time it is wrong, its internal settings, called parameters, are adjusted slightly. After enough examples, the model becomes very good at continuing text in a plausible way.

The pieces it predicts are called tokens: whole words, parts of words or punctuation.

How it works, step by step

When you send a message, a simplified version of what happens looks like this:

1. Your text is split into tokens, and each token is turned into a list of numbers.
2. Those numbers pass through many layers of the network. In each layer, a mechanism called attention lets every token take account of the others, so that a word like “bank” can be read differently in a sentence about rivers than in one about money.
3. At the end, the model produces a score for every possible next token.
4. One token is chosen, usually with a little randomness, which is why the same question can get slightly different answers.
5. The chosen token is added to the text, and the whole process repeats until the reply is complete.

The amount of text a model can consider at once is called its context window. Anything outside it, such as the start of a very long conversation, may effectively be forgotten.

From prediction to conversation

A model that only continues text is not yet a helpful assistant. Developers further train it on examples of good answers and use feedback from people to prefer responses that are helpful, accurate and safe. The result can follow instructions, hold a conversation and decline some requests.

Why they make mistakes

• Plausible is not the same as true. The model generates what is likely to come next, which is usually right for well-known facts and less reliable for obscure ones.
• Training has a cut-off. A model knows little about events after its training data was collected, unless it is connected to search or other tools.
• Confident tone. Errors can be written as fluently as correct answers, so the tone is not a guide to accuracy.

Invented details, such as a made-up reference or a quotation nobody said, are often called hallucinations. They are a direct side effect of generating plausible text, which is why they are hard to eliminate completely.

Common misconceptions

• It is not looking things up. Unless a chatbot is connected to a search tool, it is not consulting a database. Its knowledge is stored indirectly, spread across its parameters.
• It does not usually learn from your conversation as you talk. The model’s parameters stay fixed while you chat. Some services may use conversations to train future versions, depending on their policies and your settings.
• Fluent writing is not proof of understanding. Researchers continue to debate how much these systems genuinely reason and how much they reproduce patterns from their training.

Using them well

Treat a chatbot like a fast, well-read assistant whose work you check. Ask it to show sources where possible, verify important facts elsewhere, and avoid sharing private information you would not want stored.

Some practical habits help:

• Give context. Saying who the text is for and what you need it for usually produces a better answer.
• Ask for the reasoning or the sources, then check them yourself.
• Use them for drafts, outlines and explanations, and keep the final judgement for yourself, especially on health, legal or financial questions.

What is still uncertain

Many questions remain open. Researchers are still working out what happens inside these models in detail, which makes their behaviour hard to predict fully. There are active debates about the use of copyrighted material in training data, about the energy and water used by the data centres that run them, and about their effects on jobs and education. Governments, including the European Union with its AI Act, have begun writing rules for the technology, and how those rules work in practice is still being settled.