LLM or Business as Usual?

Hardly a day goes by without new headlines about Artificial Intelligence (AI). Many of our clients are currently asking: Can we benefit from Large Language Models (LLMs)? These language-based models are only one subset of AI – but currently the most visible one.

Process acceleration, knowledge management, automation – the potential seems enormous. The temptation to “do something with AI” is strong. But a new technology alone is not progress. It only becomes progress when it addresses a concrete problem – and solves it better than before.

LLMs are not classic software programs. They do not operate deterministically. There is no fixed programmed sequence, no rule-based chain of “If A, then B.” Instead, they generate responses based on probabilities learned from patterns in massive datasets. Like someone who has read thousands of books and developed an excellent sense of language – without ever formally learning grammar rules.

An LLM does not possess real-world knowledge or opinions. It does not “know” what is right or wrong; it responds based on what is likely to fit the context. That’s why a model can provide an answer that sounds grammatically perfect and plausible but is factually completely wrong. This is called a “hallucination.” And no: that’s not a bug – it’s a feature. It’s inherent to the technology.

LLMs are best suited for tasks involving understanding, structuring, or creatively combining natural language – not for delivering exact facts.

If you treat LLMs like traditional software, you will inevitably be disappointed. But if you understand their flexibility and quirks, wonderful possibilities open up.

A common wish in AI projects: “We have so much internal data; the LLM should just make it usable.” Sounds logical. But it doesn’t work that easily. LLMs don’t automatically become better helpers with more data; they improve through the quality and structure of the information. The key is how the data is prepared, linked, and provided in context.

Anyone who wants to make internal knowledge usable through LLMs must prepare it so that a language model can consume it effectively. This is often the real challenge.

When used correctly, LLMs can massively accelerate internal processes, for example:

They help make access to internal knowledge from manuals, wikis, and other sources easier.

They relieve teams of standard tasks (e.g., in support, sales, or daily project work).

They serve as interactive interfaces for complex systems.

But: LLMs are not a reliable source of truth. They generate, summarize, and formulate. They do not judge in a human sense. They do not fact-check. Therefore, they should not make final decisions involving compliance, security, legal, or financial obligations.

Even so-called (multi-)agent systems – architectures where LLMs autonomously plan tasks, use tools, and apply control logic – do not fundamentally change this. They expand the range of applications, but not necessarily the reliability of answers. Their “decisions” are not based on true understanding. This makes them helpful for clearly defined tasks but not trustworthy for safety-critical or legal responsibilities.

LLMs are trained on language. Other data types like images, videos, or sensor values often require different model types. Not every business question is automatically a case for a classic LLM.

There are several ways to run a Large Language Model internally or integrate it into existing systems – and technological development is advancing rapidly. Current common options include:

Self-Hosting: Maximum control over data and infrastructure – ideal for high data protection requirements. However, it requires powerful hardware, technical expertise, and ongoing maintenance.

Cloud-Based Solutions: Quick to deploy, scalable, and low-maintenance. But data control is limited since the model and infrastructure are operated externally.

Hybrid Approaches: Your data stays internal while model processing happens via cloud services – often a good compromise between control and efficiency.

Which option fits best depends heavily on data protection requirements, technical infrastructure, and of course, the use case. In general: The more sensitive the data, the more effort is required – both technically and organizationally.

The cost question is complex: infrastructure, usage frequency, data maintenance and update cycles, model upkeep and training, as well as organizational changes and internal skill development all play a role. A realistic cost picture therefore includes not only financial aspects but also time and organizational factors.