Context budget

On every conversation, your assistant’s language model receives the same starting context: the main instruction, the system instructions from the advanced settings, and all connected knowledge sources. This context is measured in tokens, and each conversation type has a recommended size. The wizard shows you when you approach or exceed that size. Saving always works. The icon is a hint, not a block.

​

What a token is

Language models do not read text in words but in tokens: fragments of a few characters. As a rule of thumb, 1,000 tokens correspond to about 650 German words. Long compound words need more tokens than short ones. The wizard estimates tokens from the word count: German text with a factor of 1.5, English text with 1.3. The estimate is deliberately slightly conservative so that the icon appears before things actually get tight. A deviation of a few percent from the model’s exact count is normal.

​

What counts towards the context

​

Standard and Realtime Conversation

You choose the conversation type under Advanced settings. The two types work with different models and therefore have different budgets. With Realtime Conversation the size is much smaller because your instructions and the conversation share the same window: the larger the starting context, the less room remains for the conversation itself.

​

What happens above the recommended size

Nothing breaks and nothing is switched off. As the context grows, however, three things change, gradually and not at a fixed boundary:

• Accuracy. The more text the model has to keep in view at once, the more likely it overlooks individual rules or facts, especially from the middle of long instructions. It then answers confidently but inaccurately. This behaviour is often called “hallucination”.
• Response time. A larger context takes slightly longer per answer. On the phone this is more noticeable than in chat.
• With Realtime Conversation additionally: the less room remains for the conversation, the earlier the model can lose sight of earlier parts of the call.

These effects are the reason for the recommendation. An assistant with 45,000 tokens works, but a compact assistant answers faster and more reliably.

​

The icon in the wizard

The icon appears in the header of the main instruction under Behavior & Knowledge:

• No icon: the context is within the recommended size.
• Yellow: you are approaching the recommended size.
• Red: the recommended size is exceeded.

Hover over the icon or tap it. The card shows the share of the budget, the current tokens and the split between main instruction and knowledge base. The numbers update while you type and when you connect or disconnect knowledge sources. Switching the conversation type changes the budget immediately.

​

How to stay within the budget

• Keep instructions short. The main instruction describes role, flow and boundaries. Facts such as prices, opening hours or product lists belong in the knowledge base.
• Connect sources deliberately. Connect only the sources the respective assistant needs. Several small, focused sources are better than one large collection.
• Remove duplicate content. The same information in the instruction and in a knowledge source uses tokens twice and can contradict itself.
• Do not paste large data sets. Long catalogues or customer lists do not belong in the context. Use tools that look data up on demand instead.
• Keep Realtime Conversation compact. If the size is not enough, switch to Standard Conversation or shorten the knowledge sources.
• Test after changes. Start a test conversation and check that the assistant follows all rules.

​

Frequently asked questions