> ## Documentation Index
> Fetch the complete documentation index at: https://roomstechnologygmbh-serhat-sdk-29-update-sdk-docs-to-includ.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LiveKit Plugin (Node.js)

> Integrate Deepslate voice AI with the LiveKit Agents Node framework for real-time voice applications

Use the `@deepslate-labs/livekit` package to add a `RealtimeModel` implementation to the [LiveKit Agents](https://github.com/livekit/agents) **Node.js / TypeScript** framework, so you can integrate with the Deepslate unified voice AI infrastructure.

<Note>
  Using the **Python** LiveKit framework instead? See the [LiveKit Plugin (Python)](livekit) page for the `deepslate-livekit` package. The two plugins share the same configuration model and feature set; this page is written against the Node framework's API (the realtime classes live under the `llm` namespace, and audio frames come from `@livekit/rtc-node`).
</Note>

<Note>
  This plugin lives in the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks). We welcome contributions — feel free to open issues or pull requests there.
</Note>

## Prerequisites

* A Deepslate account with API credentials
* Node.js 18+
* LiveKit server and API credentials
* (Optional) ElevenLabs API key for server-side TTS

## Installation

```bash theme={null}
npm install @deepslate-labs/livekit
```

The plugin declares the LiveKit framework packages as **peer dependencies** — install them alongside it:

```bash theme={null}
npm install @livekit/agents @livekit/rtc-node
```

You don't need to install `@deepslate-labs/core` separately. It's pulled in automatically.

## Environment Variables

Set up your credentials as environment variables:

| Variable | Required | Description |
| - | - | - |
| `DEEPSLATE_VENDOR_ID` | Yes | Your Deepslate vendor ID |
| `DEEPSLATE_ORGANIZATION_ID` | Yes | Your Deepslate organization ID |
| `DEEPSLATE_API_KEY` | Yes | Your Deepslate API key |
| `DEEPSLATE_MODEL` | No | Opal model to run on. Defaults to the platform default model |
| `ELEVENLABS_API_KEY` | No | ElevenLabs API key for server-side TTS |
| `ELEVENLABS_VOICE_ID` | No | ElevenLabs voice ID |
| `ELEVENLABS_MODEL_ID` | No | ElevenLabs model (e.g., `eleven_turbo_v2`) |

<Warning>
  Never expose your Deepslate or ElevenLabs API keys to clients. This plugin is for **server-side use** with LiveKit Agents.
</Warning>

## Quick Start

```ts theme={null}
import { fileURLToPath } from "node:url";
import { type JobContext, ServerOptions, cli, defineAgent, voice } from "@livekit/agents";
import {
  RealtimeModel,
  DeepslateRealtimeSession,
  elevenLabsConfigFromEnv,
} from "@deepslate-labs/livekit";

class Assistant extends voice.Agent {
  constructor() {
    super({ instructions: "You are a helpful voice AI assistant." });
  }

  async onEnter(): Promise<void> {
    const session = this.getActivityOrThrow().realtimeLLMSession as DeepslateRealtimeSession;
    await session.speakDirect("Hello! How can I help you today?");
  }
}

export default defineAgent({
  entry: async (ctx: JobContext) => {
    await ctx.connect();

    const session = new voice.AgentSession({
      llm: new RealtimeModel({
        ttsConfig: elevenLabsConfigFromEnv(),
      }),
    });

    await session.start({
      agent: new Assistant(),
      room: ctx.room,
    });
  },
});

cli.runApp(new ServerOptions({ agent: fileURLToPath(import.meta.url) }));
```

## Configuration Reference

<AccordionGroup>
  <Accordion title="RealtimeModel Options">
    The `RealtimeModel` constructor takes a single options object (`RealtimeModelOptions`):

    | Field | Type | Default | Description |
    | - | - | - | - |
    | `vendorId` | `string` | env: `DEEPSLATE_VENDOR_ID` | Deepslate vendor ID |
    | `organizationId` | `string` | env: `DEEPSLATE_ORGANIZATION_ID` | Deepslate organization ID |
    | `apiKey` | `string` | env: `DEEPSLATE_API_KEY` | Deepslate API key |
    | `baseUrl` | `string` | `"https://app.deepslate.eu"` | Base URL for the Deepslate API |
    | `model` | `DeepslateModelId` | env: `DEEPSLATE_MODEL`, else platform default | Opal model to run on, see [Model Selection](#model-selection) |
    | `systemPrompt` | `string` | `"You are a helpful assistant."` | Default system prompt |
    | `temperature` | `number` | `1.0` | Sampling temperature (0.0–2.0) |
    | `generateReplyTimeout` | `number` | `30.0` | Timeout in seconds for `generateReply` (0 = no timeout) |
    | `vad` | `VadConfig` | defaults | Voice activity detection tuning (see below) |
    | `ttsConfig` | `TtsConfig` | `undefined` | TTS configuration (enables audio output). Use a hosted or ElevenLabs config (see below). |
    | `wsUrl` | `string` | `undefined` | Direct WebSocket URL override — useful for local development |
  </Accordion>

  <Accordion title="VAD Configuration">
    Voice Activity Detection is handled **server-side** by Deepslate. You tune it via the `vad` object on `RealtimeModel` — no client-side VAD pipeline is needed.

    | Field | Default | Description |
    | - | - | - |
    | `confidenceThreshold` | `0.5` | Minimum confidence score to classify audio as speech (0.0–1.0) |
    | `minVolume` | `0.01` | Minimum audio volume to consider (0.0–1.0) |
    | `startDurationMs` | `200` | Consecutive speech duration required to start a turn (ms) |
    | `stopDurationMs` | `500` | Silence duration required to end a turn (ms) |
    | `backbufferDurationMs` | `1000` | Audio buffered before the detection window (ms) |

    ```ts theme={null}
    import { RealtimeModel } from "@deepslate-labs/livekit";

    const model = new RealtimeModel({
      vad: {
        confidenceThreshold: 0.5,
        minVolume: 0.01,
        startDurationMs: 200,
        stopDurationMs: 500,
        backbufferDurationMs: 1000,
      },
    });
    ```

    **Tuning tips:**

    * **Noisy environments:** increase `confidenceThreshold` (0.6–0.8) and `minVolume` (0.02–0.05)
    * **Lower latency:** decrease `startDurationMs` (100–150) and `stopDurationMs` (200–300)
    * **Natural pacing:** slightly increase `stopDurationMs` (600–800)
  </Accordion>

  <Accordion title="HostedTtsConfig">
    Use a voice cloned and hosted within Deepslate — no external TTS provider credentials required. Pass it as `ttsConfig` to enable audio output.

    | Field | Type | Default | Description |
    | - | - | - | - |
    | `provider` | `"hosted"` | required | Selects the Deepslate-hosted TTS provider |
    | `voiceId` | `string` | required | The ID of the hosted (cloned) voice to use for synthesis |
    | `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |

    **`HostedTtsMode` values:**

    | Value | Description |
    | - | - |
    | `HIGH_QUALITY` | Best output quality with still relatively low latency. Recommended for most use cases (default). |
    | `LOW_LATENCY` | Low latency generation mode that takes next to no time to complete. Output quality may be significantly reduced. |

    ```ts theme={null}
    import { RealtimeModel, HostedTtsMode } from "@deepslate-labs/livekit";

    // Default — high quality
    const model = new RealtimeModel({
      ttsConfig: {
        provider: "hosted",
        voiceId: "c3dfa73f-a1ab-4aad-b48a-0e9b9fe4a69f",
      },
    });

    // Explicit low latency mode
    const fastModel = new RealtimeModel({
      ttsConfig: {
        provider: "hosted",
        voiceId: "c3dfa73f-a1ab-4aad-b48a-0e9b9fe4a69f",
        mode: HostedTtsMode.LOW_LATENCY,
      },
    });
    ```
  </Accordion>

  <Accordion title="ElevenLabsTtsConfig">
    Configure server-side text-to-speech with ElevenLabs. Pass it as `ttsConfig` to enable audio output and automatic interruption handling.

    | Field | Type | Description |
    | - | - | - |
    | `provider` | `"eleven_labs"` | Selects the ElevenLabs TTS provider |
    | `apiKey` | `string` | ElevenLabs API key (env: `ELEVENLABS_API_KEY`) |
    | `voiceId` | `string` | Voice ID (env: `ELEVENLABS_VOICE_ID`) |
    | `modelId` | `string` | Model ID, e.g., `eleven_turbo_v2` (env: `ELEVENLABS_MODEL_ID`) |
    | `location` | `ElevenLabsLocation` | API endpoint region — `US` (default), `EU`, or `INDIA` |
    | `voiceSettings` | `ElevenLabsVoiceSettings` | Fine-grained voice control (see below) |

    Use `elevenLabsConfigFromEnv()` to build a config from environment variables.

    **`ElevenLabsVoiceSettings`** — fine-grained control over the synthesized voice:

    | Field | Type | Description |
    | - | - | - |
    | `stability` | `number` | Voice consistency (0.0–1.0); higher = more stable |
    | `similarityBoost` | `number` | Clarity and similarity to the original voice (0.0–1.0) |
    | `style` | `number` | Style exaggeration (0.0–1.0) |
    | `useSpeakerBoost` | `boolean` | Boost similarity to the original speaker |
    | `speed` | `number` | Speaking speed multiplier |

    ```ts theme={null}
    import { RealtimeModel, ElevenLabsLocation, elevenLabsConfigFromEnv } from "@deepslate-labs/livekit";

    // Load from environment variables
    const model = new RealtimeModel({ ttsConfig: elevenLabsConfigFromEnv() });

    // Or configure manually, with overrides
    const tunedModel = new RealtimeModel({
      ttsConfig: {
        provider: "eleven_labs",
        apiKey: "your_elevenlabs_key",
        voiceId: "21m00Tcm4TlvDq8ikWAM",
        modelId: "eleven_turbo_v2",
        location: ElevenLabsLocation.EU,
        voiceSettings: { stability: 0.7, similarityBoost: 0.85, speed: 1.1 },
      },
    });
    ```

    <Tip>
      When using server-side TTS (ElevenLabs or hosted), automatic interruption handling (context truncation) is enabled. The server tracks exactly what was spoken before the interruption, keeping the model's context accurate. Without server-side TTS you can use LiveKit's standard TTS integration, but this interruption context tracking will not be available.
    </Tip>
  </Accordion>
</AccordionGroup>

## Model Selection

Connections use the platform's default Opal model unless you pin one with `model`:

```ts theme={null}
import { DeepslateModel, RealtimeModel } from "@deepslate-labs/livekit";

// opal-v2.1 is the current platform default.
// See Available Models for other model ids.
const llm = new RealtimeModel({ model: DeepslateModel.OPAL_V2_1 });

// Equivalent, using the model id as a plain string:
// const llm = new RealtimeModel({ model: "opal-v2.1" });
```

See [Available Models](opal#available-models) for the current list and how to request access to preview models.

## Features

<CardGroup cols={2}>
  <Card title="Real-time Voice Streaming" icon="waveform-lines">
    Low-latency bidirectional audio streaming for natural conversations
  </Card>

  <Card title="Server-side VAD" icon="microphone">
    Voice activity detection handled server-side for reliable, configurable speech detection
  </Card>

  <Card title="Function Tools" icon="wrench">
    Define and use function tools with LiveKit's `llm.tool()` helper
  </Card>

  <Card title="Flexible TTS" icon="volume-high">
    Server-side TTS via Deepslate-hosted (cloned) voices or ElevenLabs
  </Card>

  <Card title="Low Latency Mode" icon="bolt">
    Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality
  </Card>

  <Card title="Direct Speech" icon="comment-dots">
    Speak text directly via TTS without routing through the LLM
  </Card>

  <Card title="Conversation Queries" icon="magnifying-glass">
    Run one-shot side-channel inference without affecting the main conversation
  </Card>

  <Card title="Chat History Export" icon="clock-rotate-left">
    Export the full conversation history on demand
  </Card>

  <Card title="Live Configuration" icon="sliders">
    Update the system prompt mid-session without reconnecting
  </Card>
</CardGroup>

## Function Tools

Use LiveKit's `llm.tool()` helper to expose tools to the model. Tool parameters are described with a [zod](https://zod.dev/) schema:

```ts theme={null}
import { llm, voice } from "@livekit/agents";
import { z } from "zod";

const lookupWeather = llm.tool({
  description: "Get the current weather for a given location.",
  parameters: z.object({
    location: z.string().describe("The city or location to look up weather for."),
  }),
  execute: async ({ location }) => `It's sunny and 22°C in ${location}.`,
});

const agent = new voice.Agent({
  instructions: "You are a helpful assistant.",
  tools: { lookupWeather },
});
```

## The Deepslate Session

For Deepslate-specific capabilities (direct speech, conversation queries, history export, live configuration), use the `DeepslateRealtimeSession` that the `AgentSession` created for the active agent. Don't call `model.session()` for this, since it creates a new session that isn't connected to the running call.

```ts theme={null}
import { DeepslateRealtimeSession } from "@deepslate-labs/livekit";

// agentSession is the voice.AgentSession you started
const session = agentSession.currentAgent.getActivityOrThrow()
  .realtimeLLMSession as DeepslateRealtimeSession;
```

Inside an agent method such as `onEnter()`, use `this.getActivityOrThrow().realtimeLLMSession` instead.

The session is an event emitter — subscribe with `session.on(...)`.

## Sending a Welcome Message

To greet the user, speak directly the moment the agent becomes active. Subclass `voice.Agent`, override `onEnter()`, and call `speakDirect()` on the realtime session that the `AgentSession` created for you, reachable via `getActivityOrThrow().realtimeLLMSession`. `speakDirect()` buffers the utterance until the session is ready, so no fixed delay or event handling is needed:

```ts theme={null}
import { voice } from "@livekit/agents";
import { RealtimeModel, DeepslateRealtimeSession, elevenLabsConfigFromEnv } from "@deepslate-labs/livekit";

class Assistant extends voice.Agent {
  constructor() {
    super({ instructions: "You are a helpful voice AI assistant." });
  }

  async onEnter(): Promise<void> {
    const session = this.getActivityOrThrow().realtimeLLMSession as DeepslateRealtimeSession;
    await session.speakDirect(
      "Please note that this call is handled by an AI and may be recorded.",
      /* includeInHistory */ true,
      /* uninterruptable */ true,
    );
  }
}

const session = new voice.AgentSession({
  llm: new RealtimeModel({ ttsConfig: elevenLabsConfigFromEnv() }),
});
await session.start({ agent: new Assistant(), room: ctx.room });
```

## Direct Speech

`speakDirect()` synthesizes and plays audio directly — bypassing the LLM entirely. This is useful for scripted prompts, confirmations, or fallback messages.

```ts theme={null}
await session.speakDirect(
  "Welcome back! How can I help you today?",
  true,  // includeInHistory — record as an assistant turn (default: true)
  false, // uninterruptable — allow the user to barge in (default: false)
);
```

Passing `includeInHistory: false` speaks the text without adding it to the conversation context — ideal for system-level announcements.

Passing `uninterruptable: true` makes the utterance play to completion: overlapping user speech is ignored until playback finishes.

## Conversation Queries

`queryConversation()` runs a one-shot inference call on a side channel, separate from the main conversational turn. The result is returned as a string and does **not** affect the conversation history or trigger any audio.

```ts theme={null}
const summary = await session.queryConversation(
  "Summarize the conversation so far in one sentence.",
);
console.log(summary); // e.g., "The user asked about weather in Berlin."
```

You can also pass a second `instructions` argument to further constrain the model's output format.

## Chat History Export

Export the full conversation history at any point during a session. The result is delivered via the `"chat_history_exported"` event:

```ts theme={null}
// Listen for the export result
session.on("chat_history_exported", (messages) => {
  for (const msg of messages) {
    console.log(msg.role, msg.content);
  }
});

// Request the export
await session.exportChatHistory(
  false, // awaitPending — set true to wait for any in-flight operations first
  false, // excludeAudio — set true to omit audio blobs (transcripts only)
);
```

## Live Configuration

Update the system prompt mid-session without reconnecting:

```ts theme={null}
await session.updateInstructions(
  "You are now a concise assistant. Keep replies under two sentences.",
);
```

Changes take effect on the next model turn.

## Contributing

This plugin is open source. Visit the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks) to:

* Report issues
* Submit pull requests
* Request features

## Next Steps

<CardGroup cols={2}>
  <Card title="LiveKit Plugin (Python)" icon="microphone-lines" href="livekit">
    The Python edition of this plugin
  </Card>

  <Card title="WebSocket API" icon="server" href="websocket">
    Low-level WebSocket access for custom integrations
  </Card>

  <Card title="API Reference" icon="code" href="api-reference/realtime">
    Full message schemas and configuration options
  </Card>

  <Card title="LiveKit Agents Docs" icon="book" href="https://docs.livekit.io/agents/">
    LiveKit Agents framework documentation
  </Card>

  <Card title="GitHub Repository" icon="github" href="https://github.com/deepslate-labs/deepslate-sdks">
    Source code, issues, and contributions
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.