--- title: "Your agent" description: "Your own agent talks on a phone call over a WebSocket. We send what the caller says as text and speak what you send back — or stream raw audio." canonical_url: "https://www.agent-identity.dev/docs/phone/your-agent" markdown_url: "https://www.agent-identity.dev/docs/phone/your-agent.md" last_updated: "2026-10-09" x_farming_labs_generated_preamble: true agent: tokenBudget: 1300 task: "Run your own agent on a phone call over the live call WebSocket." outcome: "When a call is answered we connect to your wss:// server, send transcript frames, and speak the text frames you send back." appliesTo: package: - "@aid/api" - "@agentidentity/sdk" prerequisites: - "A server reachable at a wss:// address." - "The agent's incoming call action is auto_accept or webhook, or the call is placed with client_websocket_url." files: - "apps/api/src/phone-sessions.ts" - "packages/sdk-ts/src/phone-stream.ts" commands: - run: "npm install @agentidentity/sdk ws" description: "The SDK exports the frame types PhoneStreamToAgent and PhoneStreamFromAgent." sideEffects: - "Both sides of the conversation are written to the call's transcript." verification: - description: "Point a call at the echo agent first." expect: "Calling wss://api.agent-identity.dev/v1/phone/echo-text answers with \"You said: …\"." rollback: - "Switch the incoming call action back to hosted_agent." failureModes: - symptom: "The call is answered and then hangs up straight away." resolution: "We could not reach your server within 5 seconds. Check the wss:// address and TLS. If voicemail_mode is leave_message, the caller goes to voicemail instead." - symptom: "Text you send is never spoken." resolution: "Text for a turn the caller interrupted is dropped. Start a new reply after the barge_in, with the new turn_id." --- # Your agent URL: /docs/phone/your-agent LLM index: /llms.txt Description: Your own agent talks on a phone call over a WebSocket. We send what the caller says as text and speak what you send back — or stream raw audio. Related: /docs/phone/incoming-calls, /docs/phone/placing-calls, /docs/sdk ## Agent Contract Task: Run your own agent on a phone call over the live call WebSocket. Outcome: When a call is answered we connect to your wss:// server, send transcript frames, and speak the text frames you send back. ### Applies To - Package: `@aid/api`, `@agentidentity/sdk` ### Prerequisites - A server reachable at a wss:// address. - The agent's incoming call action is auto_accept or webhook, or the call is placed with client_websocket_url. ### Files - `apps/api/src/phone-sessions.ts` - `packages/sdk-ts/src/phone-stream.ts` ### Commands - `npm install @agentidentity/sdk ws` — The SDK exports the frame types PhoneStreamToAgent and PhoneStreamFromAgent. ### Side Effects - Both sides of the conversation are written to the call's transcript. ### Verification - Point a call at the echo agent first. - Expected: Calling wss://api.agent-identity.dev/v1/phone/echo-text answers with "You said: …". ### Rollback - Switch the incoming call action back to hosted_agent. ### Failure Modes - The call is answered and then hangs up straight away. — Recovery: We could not reach your server within 5 seconds. Check the wss:// address and TLS. If voicemail_mode is leave_message, the caller goes to voicemail instead. - Text you send is never spoken. — Recovery: Text for a turn the caller interrupted is dropped. Start a new reply after the barge_in, with the new turn_id. # Your agent When a call is answered, we open a WebSocket **to your** `wss://` address and send JSON text frames both ways. By default this is **text mode**: we turn the caller's speech into text and speak the text you send back. Your agent never touches audio, so any program that can call a language model can hold a phone call. ```ts title="agent.ts" import { WebSocketServer } from "ws"; import type { PhoneStreamToAgent, PhoneStreamFromAgent } from "@agentidentity/sdk"; new WebSocketServer({ port: 8080 }).on("connection", (socket) => { const send = (frame: PhoneStreamFromAgent) => socket.send(JSON.stringify(frame)); socket.on("message", async (data) => { const frame: PhoneStreamToAgent = JSON.parse(data.toString()); if (frame.event === "transcript" && frame.is_final) { const reply = await yourModel(frame.text); send({ event: "text", delta: reply, done: true, turn_id: frame.turn_id }); } if (frame.event === "barge_in") { // The caller talked over you. Stop generating; what was already spoken has been cut off. } }); }); ``` ## Frames we send | Event | Meaning | | --- | --- | | `start` | The stream is open. `start.speech_to_text` and `start.text_to_speech` say who does the speech work; `media_format` is PCMU, 8 kHz, mono. | | `transcript` | What the caller said: `text`, `is_final`, `turn_id`. Partial results (`is_final: false`) arrive while they are still speaking; the final one ends the turn. | | `barge_in` | The caller talked over the agent: `trigger`, `text`, `tts_interrupted`, `turn_id`. We have already stopped the speech. | | `media` | The caller's audio, only if you asked for it. | | `stop` | The call ended: `reason`. | ## Frames you send | Event | Meaning | | --- | --- | | `text` | Words to say: `delta`, `done`, `turn_id`. Send pieces with `done: false` and a last one with `done: true`. We speak a sentence at a time as soon as it is complete. | | `clear` | Stop speaking and drop what is queued. | | `dtmf` | Press keys: `{"event":"dtmf","digits":"12#"}`, 1 to 30 of `0-9`, `*`, `#`. Anything else is ignored. | | `media` | Audio to play, only if you send your own audio. | | `stop` | End the call. Add `"after_speech": true` to end it once what is queued has been spoken. | Both sides are written to the call's transcript. A reply the caller cut off is dropped: `text` frames that still arrive for the interrupted `turn_id` are ignored. ## The WebSocket connection The connection carries an `X-Call-Context` header, a JSON object: ```json title="X-Call-Context" { "call_id": "0193…", "phone_number": "+12125550100", "direction": "inbound", "contacts": [] } ``` `phone_number` is the agent's own number; `direction` is `inbound` or `outbound`. It is signed with `x-aid-request-id`, `x-aid-timestamp` and `x-aid-signature` (`v1=`), using the agent's signing secret. We wait 5 seconds for your server to accept. If it cannot be reached, we hang up rather than leave the caller on a silent line, or send them to voicemail if the agent's `voicemail_mode` is `leave_message`. If your server closes the connection without sending `stop`, the call ends. ## Raw audio To handle audio yourself, for your own speech recognition or a voice model, say so in your server's handshake response. Each header defaults to `true`; send `false` to take that half yourself: ```text title="handshake response headers" X-Use-AID-Speech-To-Text: false # you receive the caller's audio as media frames instead of transcripts X-Use-AID-Text-To-Speech: false # you send audio as media frames instead of text ``` Audio is PCMU at 8 kHz, mono, base64 encoded: `{"event":"media","media":{"payload":""}}`. `clear` drops audio that is queued but not yet played. You can mix the two: let us listen but send your own audio, or the other way round. Raw audio needs the API to have a public address configured. ## Try it from a terminal | Address | What it does | | --- | --- | | `wss://api.agent-identity.dev/v1/phone/echo-text` | Text mode. Greets you, then says back what you said: "You said: …". | | `wss://api.agent-identity.dev/v1/phone/echo` | Raw audio. Plays your voice back about a second later. | Use one as the agent's `client_websocket_url` and call the number from your phone to hear the whole path work. ## Sitemap See the full [sitemap](/sitemap.md) for all pages. Docs-scoped sitemap: [/docs/sitemap.md](/docs/sitemap.md). Well-known sitemap: [/.well-known/sitemap.md](/.well-known/sitemap.md).