Autonomous Silence Handling

Technical documentation for the autonomous_silence_handling feature.

Overview

Autonomous Silence Handling is a feature that allows the AI assistant to intelligently detect and respond to prolonged silence during a call. When enabled, the assistant will:

1. Detect when the caller has stopped speaking
2. Prompt the caller to check if they're still on the line
3. After multiple attempts without response, gracefully end the call

This prevents calls from hanging indefinitely when callers walk away, get disconnected, or put the phone down.

Configuration

Assistant Schema

{
  "autonomous_silence_handling": true
}

Automatic Tool Addition

When autonomous_silence_handling is enabled, the system automatically adds the end_call tool to the assistant's toolset if not already present. This ensures the assistant can hang up after detecting prolonged silence.

How It Works

Timing Configuration

These are defaults, not constants. Since August 2026 all three are set per assistant under speech_config.silence — see the Assistants API. Leaving a field out keeps the platform default, so an assistant configured before these settings existed behaves exactly as it did.

Silence handling works on both pipeline and realtime assistants.

State Machine

[User Speaking] → [User Away] → [Silence Check 1] → [Silence Check 2] → [Silence Check 3] → [Hang Up]
       ↑              │                │                   │                   │
       └──────────────┴────────────────┴───────────────────┴───────────────────┘
                              (User speaks → reset timer)

Event Flow

1. User State: "away" - Silence timer starts
2. 3 seconds pass - First silence check triggered (the first_check_seconds default)
3. Assistant speaks - "Are you still there?" (contextual)
4. 12 more seconds - Second silence check
5. Assistant speaks - "I'm still here if you need me"
6. 12 more seconds - Third and final check
7. Assistant speaks - Brief goodbye + calls end_call tool

Contextual Prompts

If you write no sentences of your own, the assistant is asked to formulate the check itself. It receives different instructions based on the attempt number:

Attempt 1:

IMPORTANT: No speech detected from caller. This is attempt 1 of 3.
Based on context: if they said they need to look something up, say you'll wait.
Otherwise, ask a short question to check if they are still on the line.

Attempt 2:

IMPORTANT: Still no speech from caller. This is attempt 2 of 3.
Ask one more time if they are still there.
If no response after this, you will need to end the call.

Attempt 3 (Final):

IMPORTANT: No speech detected from caller for a long time.
This is attempt 3 of 3.
You must now say a brief goodbye and immediately use end_call to hang up.

Fixed Sentences Instead of Prompts

speech_config.silence.messages replaces the model-written prompts above with lines you wrote yourself, spoken verbatim. The order is deterministic — attempt n speaks messages[(n - 1) % messages.length], so two sentences over three attempts repeat the first. silence.final_message is the line for the last attempt, spoken instead of the model's goodbye; the platform then hangs up itself rather than asking the model to call end_call. Up to five messages, 300 characters each.

Leave both empty and the contextual prompts above are what happens.

Key Behaviors

1. No check while assistant speaking: Silence timer pauses when assistant is speaking
2. Reset on user speech: Any detected user speech resets the timer and attempt counter
3. Graceful termination: Assistant is instructed to say goodbye before hanging up
4. Context-aware: First prompt considers conversation context (e.g., "let me look that up")

Use Cases

Good For

• Reception/Front Desk: Callers may walk away to check something
• Customer Support: Long hold times while customers search for info
• Appointment Booking: Callers checking their calendars

Consider Disabling For

• Emergency Lines: May need to stay connected even during silence
• Listening Services: Where silence is expected and acceptable
• IVR/Menu Systems: Where users navigate with keypresses, not speech

Interaction with Other Features

With voicemail_detection

Both can be enabled simultaneously. Voicemail detection triggers first (on initial greeting), while silence handling activates during the conversation.

With max_duration_seconds

If max_duration_seconds is set, the call will end at that time regardless of silence handling state.

With end_call Tool

If the assistant already has end_call in its tools, no duplicate is added. The silence handling instructions simply tell the assistant to use the existing tool.

Webhook Events

When a call ends due to silence handling:

{
  "message": {
    "type": "end-of-call-report",
    "call": {
      "status": "ended"
    },
    "summary": "Call ended after no response from caller..."
  }
}

The status-update webhook will show status: "ended" (not a special silence-related status).

Troubleshooting

Assistant Not Detecting Silence

• Verify autonomous_silence_handling: true is in the assistant configuration
• Check the call transcript to see if silence detection prompts appear
• Ensure STT (Speech-to-Text) is properly configured to detect when user stops speaking

Assistant Ending Calls Too Quickly

• The initial delay is 3 seconds by default; raise speech_config.silence.first_check_seconds if your callers routinely need longer to answer
• Review the call transcript to see timing of silence prompts
• The timings are not fixed: first_check_seconds, interval_seconds and max_attempts are set per assistant

Assistant Not Hanging Up After 3 Attempts

• The end_call tool is automatically added when silence handling is enabled
• Review the end-of-call-report to see if the assistant attempted to end the call
• The LLM may occasionally not follow instructions perfectly - this is expected behavior with AI assistants