Voice

Voice connects spoken conversations to the agent's Kravos.ai workflow. A call belongs to a normal WEB_VOICE conversation; its media connection, captions, backend tasks, recording and accounting have separate state.

Providers And Migration

Native OpenAI voice uses a browser WebRTC connection and Kravos.ai's backend task runtime. It does not require a remote ElevenLabs agent, tool IDs, or remote sync. Backend tasks use the agent's model, knowledge, tools, skills and permissions; voice transport selection does not replace those settings.

New calls use native OpenAI voice, including agents with no provider selection or a saved legacy selection. Stored ElevenLabs agent, voice and LLM IDs are historical values, not OpenAI settings, and are not rewritten. Historical calls retain their provider and accounting records. Active ElevenLabs integration has been retired.

The product.voice.native.enabled admission switch defaults to true. An explicit false pauses new calls with VOICE_NOT_AVAILABLE; it never routes to a retired provider. End, recovery and historical reads are not disabled by this switch.

Native Agent Overrides

Native defaults live in voiceConfig.native, not in the retained ElevenLabs fields. Save them through Agent Settings or the existing update_agent_voice_settings tool:

{
  "provider": "OPENAI",
  "native": {
    "model": "gpt-live-1",
    "voice": "marin",
    "instructions": "Use the agent's approved business information."
  }
}

The settings HTTP route wraps this object in voiceConfig; the tool wraps it in settings. Partial updates preserve omitted native fields; null resets an individual override to its server default. Custom voices use voice: { "id": "authorized-custom-voice-id" }. Model, voice and instructions are resolved on the server for each new connection; editing them does not reconfigure an already active connection. Backend task models remain in aiConfig. Configuration export/import preserves both native overrides and retained legacy IDs.

Native Browser Contract

The widget and Voice Lab use the same client. The application protocol is version 1:

Widget requests use the Kravos publishable key in X-API-Key and the embedding origin in X-Kravos-Parent-Origin. Voice Lab uses its authenticated dashboard session and workspace identity; API keys are not accepted for Voice Lab state reads. Every read is scoped to the tenant, agent, call and browser session. OpenAI project keys remain on the server.

Start requests carry version, a new requestId, source (widget or voice-lab), sessionId, and transport: { type: 'webrtc', sdp }. The idempotency key is voice-live:start:<requestId>. Deliberate resume also supplies resume: { voiceSessionId, previousConnectionId } with a fresh connection request ID and SDP offer.

End requests reuse the original creation requestId, source and sessionId, with an optional known connectionId, reason and disposition. The disposition is end-call or release-connection; its idempotency key is voice-live:<disposition>:<requestId>. An accepted end is not proof of final usage, task completion or playback.

State reads require source, sessionId and voiceSessionId query parameters. They return { data: ... } with Cache-Control: private, no-store. A call outside the authorized scope returns 404. Read failures return 503 STATE_UNAVAILABLE; retrying that read does not retry an external action.

Captions, Tasks And History

• Captions preserve exact transcript fragments and connection-relative timing. Fragments may overlap or arrive late. They are not authoritative completed turns or proof that the caller heard audio.
• Tasks and their replies, media and citations appear separately. A displayed task result is not necessarily spoken. Opening a link is deliberate, not an automatic media load.
• Saved state is polled without overlapping reads. A failed read leaves the last observed state visible with a warning; it does not restart the voice connection or replay tasks.
• Connection loss stops local capture and shows a degraded state. Resume voice is deliberate; there is no automatic cross-provider fallback. Restoring history does not resolve an uncertain external action.
• Recording availability and gaps are shown separately from usage and settlement. Pending or estimated usage is not final usage. Missing audio must not be presented as a complete recording. Playback authorization and retention remain with the recording service.

Private Recording Playback

Select Load recording to request retained metadata, then Load audio or Prepare download for a chosen segment. Audio never autoplays. If a playback link expires, load that audio again to request a fresh grant. This does not enable recording capture or replace the call's recording consent policy.

Recording reads require the same source, sessionId and voiceSessionId scope as saved state. Metadata responses contain { manifest, segments, nextId }; use afterId for another page and optionally limit (default 50). Adding segmentId requests { url, expiresAt }; download=true requests attachment disposition. These values are inside the usual { data: ... } envelope with Cache-Control: private, no-store. General saved state never contains storage grants. Retention and access checks apply to every request; a missing or inaccessible call returns 404, and unavailable state returns 503 STATE_UNAVAILABLE without exposing storage details.

Devices And Embeds

Start or Resume requests microphone permission. Permission denial does not create a provider session. Mute stops local microphone input; Silence playback affects the current device only and does not cancel backend tasks.

Use Enable audio when autoplay is blocked. Select a different microphone or speaker when needed; unsupported output selection falls back visibly to the system output. End, minimize, mode changes and unmount release browser media resources.

Embeds must permit microphone; autoplay; speaker-selection. Browser and host Permissions Policy restrictions can still prevent access.

Platform/MCP Voice Reads

Platform/MCP voice reads are local database reads. They never contact OpenAI, and they never return recording storage keys, playback grants or audio.

Native calls retain caption fragments instead of the historical transcript JSON. For those calls get_voice_transcript returns transcript: null and a populated call object; historical sessions can return both. Captions keep exact fragments and connection-relative timing, and each connection's usageState distinguishes pending or estimated usage from final usage. providerCostUsd is the provider charge; settlement is the customer tariff and its state. Recording gaps and states never imply audio availability; use the dashboard recording service for scoped playback.

Legacy Client Compatibility

The retired /api/sdk/voice/start and /api/sdk/voice/end routes authenticate callers and return HTTP 409 VOICE_PROTOCOL_UPGRADE_REQUIRED with the native replacement contract. Start returns VOICE_NOT_AVAILABLE while admissions are paused. Neither route creates calls, settles client-reported usage or enqueues fallback work. Do not retry another provider after an uncertain native start.

The retired /api/voice/agent/sync, /sync-status, /diff and /import operations return HTTP 409 VOICE_SYNC_UNSUPPORTED for all agents. Save local configuration through Agent Settings instead.

The /api/voice/voices and /api/voice/llms catalogs default to native discovery. Explicit provider=ELEVENLABS requests return 410 VOICE_PROVIDER_RETIRED. Platform/MCP list_voices and list_voice_models return local native metadata with readiness: 'not_checked' and configured: false; explicit legacy requests also receive retirement metadata. No catalog request contacts a provider.

Platform/MCP native voice names are returned in native.voices, with case-insensitive search filtering. This built-in discovery list is not a paginated listing of your custom voices. Legacy collection fields are empty and do not indicate provider readiness.

Remote-sync tools are no longer in the Platform/MCP catalog. Use get_agent_configuration and update_agent_voice_settings for local configuration. Existing resource grants and permissions still apply.

The provider webhook, browser relay and manual transcript-fallback enqueue endpoint have been removed. Read retained transcripts from the existing conversation/session read APIs; no request restarts legacy provider ingestion.

Historical Recordings

Conversation detail exposes retained recordings through the scoped recording service. GET /api/voice/sessions/:conversationId/audio?voiceSessionId=... now returns recording metadata, not a provider audio stream. It accepts the same afterId, limit, segmentId and download options described above. Playback grants require an authenticated dashboard session with access to the conversation's agent. Imported segments, gaps and unavailable-media exceptions remain visible. Audio that was never retained cannot be fetched from the retired provider.

Related Docs