- Sound natural. Text that looks good in writing quickly sounds stiff and robotic when read aloud. The instruction must actively work against the language model’s tendency toward perfect written prose.
- Work reliably. The agent must call tools correctly, follow rules, hand over to humans, and deal with errors - live on the phone, with no user interface as a fallback.
Recommended structure
Organize your instruction into named sections with Markdown headings. This prevents rules from different contexts from blending together, and language models pay particularly reliable attention to certain headings - especially# Guardrails.
The proven order:
# Tools and # Error handling. The order, however, should be preserved: persona before behavior, behavior before rules, rules before tool usage.
# Personality
Defines who the agent is. Describe the personality as audible behavior, not as adjectives - “friendly and helpful” is the default mode of every language model anyway and changes nothing.# Environment
Tells the agent where it is and what it can assume about the callers. A short but effective section: a frustrated caller needs a different opening than a curious prospect.# Tone
Sets response length, pace, and speaking style. Every voice instruction should contain a concrete target length - this is one of the most effective single sentences in the entire prompt.# Goal
Numbered steps the agent should work through. For transactional agents (appointment booking, order status) this is the most important section: it turns vague expectations into an unambiguous flow.# Guardrails
Non-negotiable rules. Phrase guardrails as “never” statements and collect them centrally here instead of scattering them across the instruction.# Tools
One subsection per tool with three blocks: when it is used, how it is used, and what happens on errors. Details in the section Tools in the instruction.# Error handling
Overarching rules for the case that a tool fails or returns unexpected data.Tips & tricks
Show, don’t describe
Vague style instructions (“be casual”, “sound natural”) accomplish almost nothing. The most effective thing you can do: write concrete example utterances into the instruction - each as a pair of the robotic default version and the desired natural version.Anchor important rules twice
Language models weight the most recent conversation turns more heavily than instructions far back in the context. Your 1–2 most important rules therefore belong in two places: once in# Guardrails, once in the section where they concretely apply (e.g. at the respective tool). Append “Das ist wichtig.” (“This is important.”) to truly critical sentences - used sparingly, this works reliably.
The one rule that belongs in every instruction with tools, twice:
Wenn du etwas nicht weißt oder ein Tool fehlschlägt, sage das offen. Erfinde niemals Informationen. Das ist wichtig.Otherwise the model’s instinct to be “helpful” produces invented appointment confirmations, prices, or order statuses - the most expensive mistakes in live operation.
Formal or informal address - decide and stick to it
Keep it lean
Every line must earn its place. Long instructions slow down response time - and on the phone, every delay is directly noticeable.Instruct, don’t motivate
“Try really hard to be especially friendly!” wastes tokens. Write what the agent should do. And use ALL CAPS and “NIEMALS” (never) sparingly - if everything is emphasized, nothing is.Model- and voice-specific tips
Depending on the language model and voice you selected, additional rules apply to the instruction. Check the matching section before going live: ElevenLabs v3 and Soniox v2 for speech output in Standard mode, GPT-Live for the full-duplex model in Realtime Conversation.Nuances: spoken language on the phone
Callers speak their details out loud - the model has to convert them into written form before passing them to a tool. This is where most errors happen in live operation. Write the conversion rules directly into the instruction or into the tool description.Times: the “halb drei” trap
The most common reliability error with German-language voice agents: “halb drei” means 14:30 (2:30 PM), not 15:30. Don’t rely on the model getting this right - write the rule into the instruction with examples:Dates, phone numbers, numbers
- Dates: The format is DD.MM.YYYY - “der dritte Mai” (the third of May) becomes
03.05., not05/03. The agent can only resolve relative expressions like “übermorgen” (the day after tomorrow) if it knows the current date - use the date variables for this. - Phone numbers: German callers often dictate in groups of two (“dreißig vierundsiebzig” = 30 74). Rule for the instruction: resolve digit pairs, remove separators, replace a leading 0 with +49 where needed.
- Numbers: The decimal separator is the comma (
1,50 €), the thousands separator is the period (1.500). “Einundzwanzig” (one-and-twenty) is 21 - the model has to reverse the order.
Avoid email addresses on the phone
If it has to be done: callers say “ät” or “Klammeraffe” for@, “Punkt” for ., “Bindestrich” for -, and spell out following the pattern “M wie Martha”. Write this conversion into the tool description - and have the agent read the address back for confirmation.
When in doubt: read it back
The most effective single line for any agent that processes bookings or data:ElevenLabs v3 prompting
If your agent uses an ElevenLabs v3 voice, a few additional rules apply to speech output. v3 reads audio tags in square brackets as stage directions for delivery, and it reacts much more strongly to punctuation than older voices. Both can be used deliberately in the instruction - and both can quickly make an agent sound over-acted. This section describes what belongs in the instruction so that speech output stays natural and reliability does not suffer.Audio tags only take effect in the Standard Conversation type (STT → LLM → TTS), where the language model’s responses are passed to speech output as text. You choose the voice and its settings in the agent’s voice and audio settings.
Audio tags
Audio tags are words in square brackets that v3 does not read aloud but interprets as an instruction for delivery - for example[whispering], [sighs], [laughs], [pauses], or [rushed]. They control how something is said, not what is said.
In the voice agent, the language model writes the tags into its responses itself. The instruction therefore defines which tags the agent may use, how often, and when. Without this guidance, the model either uses no tags at all or scatters them uncontrolled.
Two ground rules:
- Audible delivery only. A tag describes something that can be heard on the phone: pace, volume, tone, a pause, a sigh. Tags for visible things like
[standing]or[grinning]have no audible counterpart, and sound effects like[music]or[applause]do not belong in a customer conversation. - Tags never change the message. A tag must not add content, invent anything, or soften a rule from
# Guardrails. And it must never end up in a tool parameter - a[pauses]in the name field of an appointment booking is a data error.
Pauses
In v3, pauses come from the text itself:
For most customer conversations, punctuation is entirely sufficient.
[pauses] is the exception for moments where the pause itself makes a statement - for instance before the agent delivers bad news.
Use sparingly
One tag per response is a lot; most responses need none. And a tag has to fit the selected voice and the situation: a calm support voice should not be[shouting] or [laughs harder], and a serious, professional voice often reacts unnaturally to playful tags. The voice needs to be similar to the desired delivery in the first place - a tag amplifies what is already in the voice, it does not create it.
For customer-service agents, a short allow-list has proven effective: three or four restrained tags, named explicitly - everything else is off-limits. Use the English tag names from the ElevenLabs documentation, even in German instructions; this matches the “English headings, German content” convention.
Punctuation and capitalization
v3 reads punctuation as a delivery instruction: ellipses slow down and add weight, CAPITAL LETTERS are emphasized, regular punctuation sets the rhythm. The consequence for the instruction: the model copies the example utterances from the prompt (the “Statt/Besser” pairs) - including their punctuation. Write them the way they should sound. Do not rely on assumptions for any of these effects. How strongly a voice reacts to a tag or a punctuation mark differs from voice to voice - test every phrasing with the voice you actually selected.Examples
A# Tone section that releases audio tags in a controlled way:
# Guardrails for v3 voices:
Soniox v2 prompting
If your agent uses Sonioxtts-rt-v2 for speech output, audio tags can shape its delivery. Put English tags in square brackets before the text they should affect, even when the spoken text is German.
Write the words to fit the intended emotion and let punctuation carry the normal speech rhythm. Keep tags short and single-purpose. Stack them sparingly, and name only the tags the agent may use in its instruction. For example:
Hmm, Ähm, and similar spoken interjections belong in the text as ordinary words, not in brackets. Do not use tags for environmental sounds such as [phone rings]. Unrecognized tags may be read aloud; even documented directions may not be fully realized by every voice. Test the chosen tags with the voice you actually use.
Further reading at Soniox: Emotion, tone, and audio tags and TTS models.
GPT-Live prompting
If your agent uses GPT-Live from OpenAI, a few things about the instruction work differently. GPT-Live is a full-duplex speech model: it listens while it speaks, handles turn-taking and interruptions itself, and produces audio directly - with no intermediate text stage. It does not do the reasoning and tool calls itself; it hands them to a backend model and speaks that model’s result. Everything in the sections above still applies: the recommended structure, short spoken replies, the “Statt/Besser” pairs, the when/how/on-errors blocks for tools, and the rules for spoken language on the phone. This section only adds what is different with GPT-Live.You select GPT-Live under the Realtime Conversation type in the agent’s voice and audio settings. The model brings its own voices; external voices, audio tags, response time, and turn-detector settings do not exist here - turn-taking is up to the model.
What the instruction controls - and what it does not
Your instruction (system prompt plus AI instruction) goes to the speech model in full. It defines role, speaking style, conversation goal, and guardrails - exactly as before. The backend model that calls tools and checks results works with fixed platform instructions; these are not editable. It receives the task at hand with its context from the speech model, for example “book an appointment on Thursday at 14:30 for Ms. Berger”. Three practical limits follow from this:- No verbatim text. GPT-Live phrases everything in its own words - including the greeting and bridging phrases like “One moment please”. Describe what the agent should say, not the exact wording. A sample sentence in the instruction is matched in meaning, not read aloud.
- No changes during the call. The instruction is fixed for the duration of the call. Intents and sub-prompts that switch the instruction mid-conversation are not available for GPT-Live. Put all conversation paths into one lean instruction - or split them across several agents.
- Tool rules arrive through the task. The backend model reads the tool descriptions and whatever the speech model hands over as the task. So keep writing rules for a tool briefly and concretely into the instruction (when/how/on errors) - the speech model passes them on to the backend that way.
Short, in behavior instead of wording
GPT-Live needs a shorter instruction than a Standard agent. A few sentences on role, tone, and pace in# Personality and # Tone are enough - the model chooses the wording itself. Long lists of sample sentences achieve little here; two to four “Statt/Besser” pairs are enough to set the direction.
Listening, acknowledging, allowing interruptions
Two things GPT-Live does better than a Standard agent, and that the instruction should regulate explicitly:- Backchannels while listening. GPT-Live can slip in short acknowledgements while the caller is speaking (“Mhm”, “Verstehe”). Set the amount - “occasionally”, not in every sentence - and listen in testing whether the acknowledgements help or talk over the caller.
- Interruptions. If the caller cuts in, the agent should stop talking and listen. Distinguish in the instruction between “stop” (just go quiet) and a correction like “no, Friday, not Thursday” (the task in progress changes).
Wait for results instead of guessing
Every answer that depends on a tool - availability, price, booking, status - must be handed to the backend by the speech model, which then waits for the result. The most common source of errors with GPT-Live: the model bridges the waiting time and “confirms” something that has not happened yet. The rule for this belongs in# Tools and once more in # Guardrails:
# Error handling applies as usual.
Example
A complete GPT-Live instruction for appointment booking - deliberately shorter than example 1 for the Standard agent:Tools in the instruction
Tools (appointment booking, SMS, email, API requests, n8n) are created in the sidebar and activated per agent in the Voice Wizard. The condition on the tool controls when it is fundamentally available - the instruction describes how it fits into the conversation flow. Describe each tool in the instruction with three blocks:Recommended flow for appointment booking
If your agent should take appointments, we recommend this conversation flow as a guide in the# Goal section:
1
Clarify the desired time
Ask for the desired appointment first. If it is not available, the agent immediately suggests suitable alternatives instead of making the caller ask again and again.
2
Capture the name
Capture the caller’s first and last name. For unusual names, briefly confirm the spelling.
3
Book the appointment
The agent creates the appointment and repeats the key details for confirmation.
4
Optional: confirmation via SMS
Combine the flow with the SMS tool and send the caller a confirmation. Clarify beforehand whether the calling number may be used, or alternatively capture a mobile number. If the booking fails in the backend, an SMS with a booking link can be sent instead.
Variables in the instruction
Variables are placeholders that are replaced with dynamic values at runtime. Use double curly braces and the appropriate namespace, for example{{system.caller_number}} or {{pre_call.first_name}}.
Type {{ to select a variable. This works in both the normal prompt editor and fullscreen mode. The picker inserts the complete placeholder.
The previous percent-sign notation (
%%…%%) is no longer used for new variables. Write variables with {{…}}.Call data
These variables are populated from the ongoing call and are available only in the AI instruction (not in the system prompt):Values from the Inbound Webhook
Values loaded before the call through the Inbound Webhook use thepre_call namespace. For example, create the variable first_name and use {{pre_call.first_name}} in the AI instruction. The same values can also be used in the initial greeting, as a forwarding destination, and in post-processing.
Date & time
For date and time in the AI instruction, you can use{{system.datetime}}, {{system.date}}, {{system.time}}, or {{system.day_of_week}}. Use the JSON notation to set the time zone and language explicitly:
Parameters:
variable:datetime,date,time, orday_of_weektimezone: IANA time zone (e.g.Europe/Berlin,UTC) - default:Europe/Berlinlanguage:deoren- default:de
Use case: forwarding only during business hours
Use case: forwarding only during business hours
Inspecting resolved variables while testing
Inspecting resolved variables while testing
When testing your instruction, you can inspect the actually inserted values in two places:
- Browser test: The resolved instruction is shown when switching to the prompt or at the start of the conversation.
- Conversations view: Click “Details” and check the
tec_outputsin the debug view.
In addition, you can define your own variables in post-processing that are extracted from the conversation after the call and used in email, SMS, and API tasks (e.g.
{{post_call.kunden_email}}). These are not available in the AI instruction.Example prompts
Two complete instructions that apply the structure and tips from this guide. Both address callers with the formal Sie - adapt this to your brand. The prompts are written in German, as recommended for German-speaking agents.Example 1: appointment booking for a car workshop (with tools)
Example 1: appointment booking for a car workshop (with tools)
Example 2: FAQ and forwarding for an electronics store (no booking)
Example 2: FAQ and forwarding for an electronics store (no booking)
Checklist
Go through these points before putting an instruction live:- Is there a
# Personalitysection with audible behavior instead of adjectives? - Does
# Toneset a concrete response length (e.g. “1–2 sentences”)? - Does
# Goalcontain numbered, unambiguous steps? - Are there 4–8 example pairs (“Statt … / Besser …”) for natural phrasing?
- Is the form of address (Sie/Du) decided and consistent in every example?
- Does every tool have a when/how/on-errors description?
- Is the conversion of spoken details covered (times including “halb drei”, dates, phone numbers)?
- Does the “never invent anything” rule appear in at least two places?
- Is the instruction lean?
- Tested in the browser, including the resolved variables?
- Using an ElevenLabs v3 voice: allow-list of audio tags defined and tested with the selected voice?
- Using GPT-Live: backchannels and interruptions regulated, “wait for the result” rule in two places, no verbatim text assumed?
Common mistakes
- Adjectives instead of behavior. “Be friendly and professional” changes nothing - describe what should be audible.
- Emphasizing everything. If every other line contains “NIEMALS”, the emphasis loses its effect. Reserve it for the 1–2 truly critical rules.
- Stating rules only once. Critical rules belong in two places - otherwise they get lost in long conversations.
- Tools without guidance. A tool without a when/how/on-errors description leads to improvised, unreliable calls.
- One mega-prompt for everything. Three or more distinct flows in a single instruction make the agent slow and error-prone.
- Only reading, never calling. An instruction that reads well can sound bad. Test on the phone and adapt iteratively - one change per iteration.