> For the complete documentation index, see [llms.txt](https://ai.ortusbooks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai.ortusbooks.com/readme/release-history/3.6.0.md).

# 3.6.0

BoxLang AI Module 3.6.0 (unreleased). Cartesia provider, streaming text-to-speech with aiSpeakStream(), Cloudflare Workers AI provider, MariaDB vector memory, an HTTP streaming demo, and a round of sp

**Released:** Unreleased (in development)

BoxLang AI 3.6.0 is a voice release. It adds **streaming text-to-speech** so you can hand synthesized audio to a caller, a browser or a phone bridge while it is still being generated, a new **Cartesia** provider for fast text-to-speech and speech-to-text, and a **Cloudflare Workers AI** provider for chat, streaming, tool calling and embeddings.

It also adds **MariaDB vector memory**, which runs semantic search inside MariaDB 11.7+ using its native vector type and index.

It also fixes a batch of speech provider bugs found while verifying every speech service end to end. ElevenLabs, Grok, Mistral and Gemini each had at least one request that silently ignored your settings or returned audio that would not play.

> This page is updated as new items land on `development`.

***

## ✨ New Features

### 🔊 Streaming Text-to-Speech: `aiSpeakStream()`

`aiSpeak()` waits for the complete audio file. `aiSpeakStream()` calls your function with audio **as the provider generates it**, so the listener hears the first words long before the full answer is synthesized. It is built for voice agents that restream audio to a customer.

| BIF or method                                      | Description                                                   |
| -------------------------------------------------- | ------------------------------------------------------------- |
| `aiSpeakStream( text, callback, params, options )` | Stream audio chunks to a callback and return a summary struct |
| `aiSpeak().text( "..." ).stream( callback )`       | The same thing through the fluent builder                     |

```javascript
summary = aiSpeakStream(
    "Thanks for calling BoxLang support.",
    ( event ) => {
        if ( event.type == "audio" ) {
            socket.send( event.data )   // binary audio, ready to forward
        }
    },
    { voice: "female" },
    { provider: "cartesia", outputFormat: "pcm" }
)
// { provider, model, audioFormat, sampleRate, chunks, bytes, completed }
```

The callback receives one struct per event, discriminated by `type`:

| Event        | Shape                                                                   |
| ------------ | ----------------------------------------------------------------------- |
| `audio`      | `{ type, data, format, sampleRate, sequence }` where `data` is binary   |
| `timestamps` | `{ type, words, start, end }` (Cartesia over SSE with `add_timestamps`) |
| `done`       | `{ type, chunks, bytes }`, always last                                  |

**Stop early (barge-in):** return an explicit `false` from the callback. The provider connection is closed immediately and the summary reports `completed: false`.

**Providers:** Cartesia, ElevenLabs, OpenAI, Mistral and Gemini. Grok only streams over a WebSocket, which is not implemented yet, so it does not declare the capability.

**Formats for voice agents:** `mp3` plays in a browser `<audio>` tag as it arrives, `pcm` is raw 16-bit audio with no decoding for agents, and `mulaw` or `alaw` are 8 kHz telephony formats.

Under the hood this adds the `IAiSpeechStreamService` interface and the `speechStream` capability. `aiSpeakStream()` requires both the interface and the declared capability on the provider. The timeout is an **idle timeout**: a long stream is fine as long as chunks keep arriving.

**Documentation**: [Streaming Text-to-Speech](/main-components/audio/streaming-speech.md), [aiSpeakStream reference](/advanced/reference/built-in-functions/aispeakstream.md)

***

### 🎙️ Cartesia Provider

Cartesia is a low-latency voice provider. Sonic generates speech and Ink-Whisper transcribes it, through the same BIFs you already use.

| Capability               | Details                                                                           |
| ------------------------ | --------------------------------------------------------------------------------- |
| Text-to-speech           | Sonic, output `mp3`, `wav`, `pcm`, `mulaw`, `alaw`                                |
| Streaming text-to-speech | Bytes for `mp3` and `wav`, SSE with word timestamps for `pcm`, `mulaw` and `alaw` |
| Speech-to-text           | Ink-Whisper through `aiTranscribe()`                                              |
| Translation              | Not supported, `translate()` throws `UnsupportedCapability`                       |

```javascript
// Set CARTESIA_API_KEY, or pass the key explicitly
aiSpeak(
    text   : "Great news! Your order has shipped.",
    params : { voice: "male", emotion: "happy", speed: 1.1 },
    options: { provider: "cartesia", outputFormat: "wav", outputFile: "/tmp/shipped.wav" }
)

text = aiTranscribe( audio: "/tmp/shipped.wav", options: { provider: "cartesia" } )

service = aiService( "cartesia", apiKey )
```

`flac` and `opus` output throw `InvalidArgument` instead of quietly returning the wrong format. Set `params.sample_rate` to change the rate of raw formats.

**Documentation**: [Audio overview](/main-components/audio.md), [Text-to-Speech](/main-components/audio/text-to-speech.md)

***

### ☁️ Cloudflare Workers AI Provider

Chat, streaming, tool calling and embeddings through the OpenAI compatible endpoints of Workers AI.

| Setting                  | Details                                                                     |
| ------------------------ | --------------------------------------------------------------------------- |
| Provider name            | `cloudflare`                                                                |
| API key                  | `CLOUDFLARE_API_KEY`                                                        |
| Account ID               | The `accountId` option, or the `CLOUDFLARE_ACCOUNT_ID` environment variable |
| Default chat model       | `@cf/openai/gpt-oss-20b`                                                    |
| Default embeddings model | `@cf/baai/bge-base-en-v1.5`                                                 |
| Capabilities             | `chat`, `stream`, `embeddings`                                              |

```javascript
answer = aiChat(
    "Explain edge inference in one sentence.",
    {},
    { provider: "cloudflare", accountId: "your-account-id" }
)

vector = aiEmbed(
    "BoxLang",
    {},
    { provider: "cloudflare", accountId: "your-account-id" }
)
```

**Documentation**: [Provider Setup](/getting-started/installation/provider-setup.md)

***

### 🗄️ MariaDB Vector Memory

`MariaDBVectorMemory` stores embeddings in MariaDB 11.7+ and ranks them inside the database with the native `VECTOR(n)` type, an HNSW `VECTOR INDEX` and the `VEC_DISTANCE_COSINE` / `VEC_DISTANCE_EUCLIDEAN` functions. Unlike `MysqlVectorMemory`, only the top matches come back to BoxLang instead of every row.

| Setting            | Details                                                       |
| ------------------ | ------------------------------------------------------------- |
| Memory type        | `mariadb` (or `MariaDBVectorMemory`)                          |
| Requires           | MariaDB 11.7+ and the `bx-mariadb` module                     |
| `datasource`       | Required                                                      |
| `distanceFunction` | `COSINE` (default) or `L2`. MariaDB has no native dot product |
| `dimensions`       | Must match the embedding model, default `1536`                |
| `m`                | HNSW M, `3` to `200`, default `8`                             |
| `autoCreate`       | Create the table if missing, default `true`                   |
| Multi-tenant       | `userId` and `conversationId` use dedicated columns           |

```javascript
memory = aiMemory(
    memory         : "mariadb",
    userId         : "alice",
    conversationId : "support-1",
    config         : {
        datasource        : "myMariaDS",
        collection        : "ai_memory",
        distanceFunction  : "COSINE",
        embeddingProvider : "openai",
        embeddingModel    : "text-embedding-3-small"
    }
)

memory.add( "Invoices are due within 30 days" )
memory.getRelevant( "When do I have to pay?", 3 )
```

MariaDB allows **one vector index per table**, fixed to one metric and one dimension size, so collections that share a table share both. Configuring an existing table with a different `distanceFunction` throws `MariaDBVectorMemory.DistanceMismatch`. It also works as the `vectorProvider` of `HybridMemory`.

**Documentation**: [MariaDBVectorMemory](/main-components/memory/vector-memory.md#mariadbvectormemory), [Multi-Tenant Memory](/main-components/memory/multi-tenant-memory.md)

***

### 🔒 MCP Client Controls

Attaching an MCP server used to mean trusting its address completely. You can now decide what a client may reach. Every option is off by default, so nothing changes unless you ask.

| Option                          | Description                                                                                                                                      |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| `withUrlGuard( fn )`            | Called with the URL before every request. Return `false` or throw to refuse it. Nothing is sent                                                  |
| `withRedirects( false )`        | A redirect becomes a failed response and the new address is never called                                                                         |
| `withMaxResponseBytes( n )`     | Refuse an answer longer than `n` characters                                                                                                      |
| `withHandshake()`               | Send the MCP `initialize` handshake once and send back the `Mcp-Session-Id`                                                                      |
| `withHttpVersion( "HTTP/1.1" )` | Choose the HTTP version. The default stays `HTTP/2`, which on a plain http address starts with an upgrade request that some servers answer badly |
| `listTools( cursor )`           | Read the next page of a server that pages its tools, with the `nextCursor` of the previous answer                                                |

`withMCPServer( server, config )` and the `mcpServers` argument of `aiAgent()` and `aiModel()` accept the same options as `urlGuard`, `redirects`, `maxResponseBytes`, `handshake` and `httpVersion`, plus two more:

* **`prefix`:** tools are exposed as `prefix__name`, so two servers with a tool called `search` cannot clash. The server still receives the original name.
* **`toolFilter`:** an array of tool names. Only these are exposed to the agent.

```javascript
agent = aiAgent( name: "DocsAgent" )
    .withMCPServer(
        "https://boxlang.example.com/mcp",
        {
            prefix          : "boxlang",
            toolFilter      : [ "searchDocumentation", "getPage" ],
            urlGuard        : ( url ) => !isPrivateAddress( url ),
            redirects       : false,
            maxResponseBytes: 200000
        }
    )
```

**Documentation**: [Controlling What a Client Can Reach](/model-context-protocol-mcp/client.md#controlling-what-a-client-can-reach), [MCP Server Options](/main-components/agents/tools-and-mcp.md#mcp-server-options)

***

### 🌐 HTTP Streaming Speech Demo

`examples/http-streaming-speech` in the bx-ai repository is a runnable MiniServer demo that answers the question "how does the user actually hear it?". It has two modes:

* **`<audio>` mode:** `speak.bxm` streams raw mp3 to an `<audio>` tag, so playback starts after the first chunk.
* **Voice agent mode:** `speak-events.bxm` streams NDJSON events with base64 PCM and word timestamps to a Web Audio player with word highlighting and barge-in.

The same demo is in `bx-ai-intro` as `examples/speech-stream-http`, and the BX AI Explorer has streaming samples. No browser plugin is needed.

***

## 🪲 Fixed

* **MCP client** `withTimeout( ms )` is documented in milliseconds but the value was passed to `bx:http`, whose timeout is in seconds, so `withTimeout( 10000 )` waited about 2.8 hours. It is now converted to seconds, rounded up, at least one. Code that relied on the long wait needs a larger value.
* **MCP client** read the first `data:` line of an event stream as the answer, so a server that sent a notification (progress, log) before the answer made the call return the notification. Messages that are not a JSON-RPC response are now skipped.
* **MysqlVectorMemory** placed metadata filter keys directly into SQL. Filter keys are now validated (letters, numbers and underscores), and an invalid key throws `MysqlVectorMemory.InvalidFilterKey`.
* **ElevenLabs** ignored the requested voice, model, output format and speed, so `voice`, `.female()` and `outputFormat` had no effect. They are now honored, `wav` output is a real wav, and the speech-to-text upload uses the documented `file` field. The default voice is the premade Sarah voice, because Rachel is a library voice that free plan keys cannot use over the API.
* **Grok** text-to-speech now sends the documented xAI body (`text`, `voice_id`, `language`, `output_format`). The default voice is `eve`, and `male()` and `female()` map to `rex` and `eve`.
* **Mistral** text-to-speech now decodes the JSON `audio_data` response, and a UUID voice is sent as `voice_id`.
* **Mistral** failed with `Voice 'Charlotte' not found`, because `Charlotte` is not a valid Mistral voice, and OpenAI's default voice (`ash`) also leaked into Mistral requests. With no voice requested, Mistral now uses the first preset voice from `GET /v1/audio/voices`, looked up once and cached. If that lookup fails, the request is still sent and Mistral's own error is shown.
* **Gemini** text-to-speech returned headerless PCM labelled as wav. It is now wrapped in a RIFF header so the file plays.
* **Gemini** defaulted to `gemini-2.5-flash-preview-tts`, which is no longer listed in the Gemini docs. The default is now `gemini-3.8-flash-tts`, and errors include the HTTP status, the model and the response body.
* **Gemini** streaming text-to-speech failed with `Failed to parse JSON [DONE]`, because the Interactions stream ends with a non-JSON `[DONE]` sentinel. Speech streaming now ignores it, for every provider that streams over SSE.

***

## 📖 Documentation

* [Streaming Text-to-Speech](/main-components/audio/streaming-speech.md)
* [aiSpeakStream reference](/advanced/reference/built-in-functions/aispeakstream.md)
* [Audio overview](/main-components/audio.md)
* [Text-to-Speech](/main-components/audio/text-to-speech.md)
* [Provider Setup](/getting-started/installation/provider-setup.md)
* [Vector Memory Systems](/main-components/memory/vector-memory.md)
* [Multi-Tenant Memory](/main-components/memory/multi-tenant-memory.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://ai.ortusbooks.com/readme/release-history/3.6.0.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.