Question answering
Answering from documents and knowledge bases with retrieval.
Technology / Generative and Language
Large Language Models
Large language models are the technology behind today's AI assistants, copilots and many agents. Understanding how they work, and where they fail, helps organisations choose models wisely and design applications that stay reliable.
What is Large Language Models?
Short answer
A large language model (LLM) is a deep neural network, usually based on the transformer architecture, trained on very large amounts of text to predict the next token in a sequence, which enables it to understand instructions, answer questions, summarise, translate, reason over text and generate language and code.
LLMs are first pre-trained on broad text data, then refined through instruction tuning and alignment techniques so they follow requests helpfully. They process input within a context window, a limit on how much text they can consider at once, which has grown substantially in recent models.
Organisations can use LLMs through commercial APIs or host open-weight models themselves. They can be adapted through prompting, retrieval-augmented generation that supplies relevant documents, or fine-tuning on specific data. Choosing among these depends on quality requirements, data sensitivity, cost, latency and the need for control.
Related concepts
- Transformers
- RAG
- Context window
- Fine-tuning
- Generative AI
- Agentic AI
- NLP
How it works
How does Large Language Models work?
-
Tokenisation
Text is split into tokens, pieces of words the model processes as numbers.
-
Pre-training
The model learns language patterns by predicting next tokens across huge datasets.
-
Alignment
Instruction tuning and feedback-based training make the model follow requests and behave more safely.
-
Inference with context
At use time the model reads the prompt, instructions and retrieved content within its context window.
-
Generation
It produces output token by token, guided by probabilities and settings such as temperature.
Applications
Where is Large Language Models useful?
Common applications across industries. Suitability always depends on the specific data, process and constraints.
Summarisation
Condensing reports, calls, meetings and case histories.
Extraction
Pulling structured data from contracts, emails and forms.
Classification and routing
Categorising requests and messages with explanations.
Code
Writing, reviewing and explaining software code.
Reasoning for agents
Planning steps and choosing tools in agentic systems.
Limitations
Limitations and when not to use it
Every technology has boundaries. Knowing them early avoids expensive disappointment.
- LLMs can hallucinate, producing confident but false statements.
- Knowledge is limited to training data unless retrieval is used.
- Output can vary and be sensitive to prompt wording.
- Long or complex reasoning can still fail in subtle ways.
- Costs and latency grow with model size and context length.
Common misconceptions
Bigger models are always better.
Smaller models often match larger ones on specific tasks at lower cost and latency.
LLMs remember everything you tell them.
Models do not learn from conversations by default; memory must be designed explicitly.
Longer context replaces retrieval.
Long context helps, but retrieval remains more efficient and precise for large or changing knowledge bases.
Comparison
Ways to adapt an LLM to your needs
Most applications combine prompting and retrieval; fine-tuning is used selectively.
| Approach | What it changes | Best for | Trade-offs |
|---|---|---|---|
| Prompt design | Instructions and examples given at run time | Format, tone, task definition | Quick to change; limited depth |
| Retrieval-augmented generation | Relevant knowledge supplied at run time | Company knowledge, current facts | Needs good data pipelines and retrieval |
| Fine-tuning | Model weights, using task examples | Specialised formats, style, narrow tasks | Needs data and retraining as needs change |
| Tool use | Model can call functions and systems | Live data, calculations, actions | Requires permissions and security design |
Decision guide
How to evaluate Large Language Models for your organisation
Questions worth answering before investing. They form the basis of our feasibility and proof-of-concept work.
| Question to ask | Why it matters |
|---|---|
| Which specific tasks will the LLM perform? | Models vary by task, so evaluate on your tasks rather than general benchmarks. |
| What data sensitivity and residency apply? | Determines whether commercial APIs, private deployments or open-weight models fit. |
| What latency and volume are expected? | Affects model size, hosting and cost. |
| How will hallucination risk be managed? | Grounding, validation and review design depend on consequences of errors. |
| How easily can we switch models? | Model capabilities and commercial terms change quickly; avoid tight coupling. |
AARHIT services
Services that apply Large Language Models
How we put this technology to work for organisations.
Hybrid Intelligence
Generative AI and Large Language Models
Build grounded generative AI applications using large language models, retrieval, evaluation and guardrails.
Explore service ResearchAI Model Evaluation and Optimization
Measure model quality, bias, robustness, latency and cost, then tune models and prompts against clear criteria.
Explore service Technology & EngineeringCloud AI Infrastructure
Design compute, storage, networking and GPU capacity for training and inference with cost and security controls.
Explore service Hybrid IntelligenceConversational AI and Virtual Assistants
Create assistants for customers or staff that answer from approved knowledge and hand over to people when needed.
Explore serviceRelated technologies
Industries
FAQ
Large Language Models FAQs
What is a large language model?
A large language model is a neural network trained on vast text data to predict the next word or token, enabling it to understand and generate language, answer questions, summarise and write code.
What is the difference between open-weight and commercial LLMs?
Commercial models are accessed through provider APIs and are often the most capable. Open-weight models can be downloaded and hosted privately, giving more control over data and cost, with capability varying by model.
What causes LLM hallucinations?
LLMs generate statistically likely text rather than retrieving verified facts, so when information is missing or ambiguous they may produce plausible but incorrect content.
What is a context window?
The context window is the maximum amount of text, measured in tokens, that a model can consider at one time, including instructions, conversation history and supplied documents.
How do we choose the right LLM?
Build an evaluation set from your real tasks, test candidate models for quality, latency and cost, and consider data handling, hosting and contractual requirements.
Next step
Exploring Large Language Models?
Tell us what you want to achieve. We will help you judge whether this technology is the right fit and how to test it with your own data.