For the complete documentation index, see llms.txt. This page is also available as Markdown.

3.6.0

BoxLang AI Module 3.6.0. Cartesia provider, streaming text-to-speech with aiSpeakStream(), Cloudflare Workers AI provider, MariaDB vector memory, an MCP client for servers that run as a child process

Released: 2026-10-20

BoxLang AI 3.6.0 is a voice and MCP release. It adds streaming text-to-speech so you can hand synthesized audio to a caller, a browser or a phone bridge while it is still being generated, a new Cartesia provider for fast text-to-speech and speech-to-text, and a Cloudflare Workers AI provider for chat, streaming, tool calling and embeddings.

It also adds MariaDB vector memory, which runs semantic search inside MariaDB 11.7+ using its native vector type and index.

It also lets agents use MCP servers that run as a child process over stdin and stdout, such as bxplaywright mcp, with the same MCP() BIF, the same MCPResponse and the same agent integration as HTTP servers.

It also fixes a batch of speech provider bugs found while verifying every speech service end to end. ElevenLabs, Grok, Mistral and Gemini each had at least one request that silently ignored your settings or returned audio that would not play.


⚠️ Migration

MCPClient was renamed to MCPHttpClient. The MCP() BIF, withMCPServer() and the mcpServers argument are not affected, and MCP() now returns an IMCPClient. Only code that created the class directly needs a change:

// Before
client = new bxModules.bxai.models.mcp.MCPClient( "http://localhost:3000" )

// After
client = MCP( "http://localhost:3000" )

The named argument MCP( baseURL: url ) still works (the new name is mcpServer). The client events (onMCPClientRequest, onMCPClientResponse, onMCPClientError) now carry the IMCPClient in client, and have a new transport key. See Events.


✨ New Features

🔊 Streaming Text-to-Speech: aiSpeakStream()

aiSpeak() waits for the complete audio file. aiSpeakStream() calls your function with audio as the provider generates it, so the listener hears the first words long before the full answer is synthesized. It is built for voice agents that restream audio to a customer.

BIF or method
Description

aiSpeakStream( text, callback, params, options )

Stream audio chunks to a callback and return a summary struct

aiSpeak().text( "..." ).stream( callback )

The same thing through the fluent builder

The callback receives one struct per event, discriminated by type:

Event
Shape

audio

{ type, data, format, sampleRate, sequence } where data is binary

timestamps

{ type, words, start, end } (Cartesia over SSE with add_timestamps)

done

{ type, chunks, bytes }, always last

Stop early (barge-in): return an explicit false from the callback. The provider connection is closed immediately and the summary reports completed: false.

Providers: Cartesia, ElevenLabs, OpenAI, Mistral and Gemini. Grok only streams over a WebSocket, which is not implemented yet, so it does not declare the capability.

Formats for voice agents: mp3 plays in a browser <audio> tag as it arrives, pcm is raw 16-bit audio with no decoding for agents, and mulaw or alaw are 8 kHz telephony formats.

Under the hood this adds the IAiSpeechStreamService interface and the speechStream capability. aiSpeakStream() requires both the interface and the declared capability on the provider. The timeout is an idle timeout: a long stream is fine as long as chunks keep arriving.


🎙️ Cartesia Provider

Cartesia is a low-latency voice provider. Sonic generates speech and Ink-Whisper transcribes it, through the same BIFs you already use.

Capability
Details

Text-to-speech

Sonic, output mp3, wav, pcm, mulaw, alaw

Streaming text-to-speech

Bytes for mp3 and wav, SSE with word timestamps for pcm, mulaw and alaw

Speech-to-text

Ink-Whisper through aiTranscribe()

Translation

Not supported, translate() throws UnsupportedCapability

flac and opus output throw InvalidArgument instead of quietly returning the wrong format. Set params.sample_rate to change the rate of raw formats.


☁️ Cloudflare Workers AI Provider

Chat, streaming, tool calling and embeddings through the OpenAI compatible endpoints of Workers AI.

Setting
Details

Provider name

cloudflare

API key

CLOUDFLARE_API_KEY

Account ID

The accountId option, or the CLOUDFLARE_ACCOUNT_ID environment variable

Default chat model

@cf/openai/gpt-oss-20b

Default embeddings model

@cf/baai/bge-base-en-v1.5

Capabilities

chat, stream, embeddings

Documentation: Provider Setup


🗄️ MariaDB Vector Memory

MariaDBVectorMemory stores embeddings in MariaDB 11.7+ and ranks them inside the database with the native VECTOR(n) type, an HNSW VECTOR INDEX and the VEC_DISTANCE_COSINE / VEC_DISTANCE_EUCLIDEAN functions. Unlike MysqlVectorMemory, only the top matches come back to BoxLang instead of every row.

Setting
Details

Memory type

mariadb (or MariaDBVectorMemory)

Requires

MariaDB 11.7+ and the bx-mariadb module

datasource

Required

distanceFunction

COSINE (default) or L2. MariaDB has no native dot product

dimensions

Must match the embedding model, default 1536

m

HNSW M, 3 to 200, default 8

autoCreate

Create the table if missing, default true

Multi-tenant

userId and conversationId use dedicated columns

MariaDB allows one vector index per table, fixed to one metric and one dimension size, so collections that share a table share both. Configuring an existing table with a different distanceFunction throws MariaDBVectorMemory.DistanceMismatch. It also works as the vectorProvider of HybridMemory.


🔒 MCP Client Controls

Attaching an MCP server used to mean trusting its address completely. You can now decide what a client may reach. Every option is off by default, so nothing changes unless you ask.

Option
Description

withUrlGuard( fn )

Called with the URL before every request. Return false or throw to refuse it. Nothing is sent

withRedirects( false )

A redirect becomes a failed response and the new address is never called

withMaxResponseBytes( n )

Refuse an answer longer than n characters

withHandshake()

Send the MCP initialize handshake once and send back the Mcp-Session-Id

withHttpVersion( "HTTP/1.1" )

Choose the HTTP version. The default stays HTTP/2, which on a plain http address starts with an upgrade request that some servers answer badly

listTools( cursor )

Read the next page of a server that pages its tools, with the nextCursor of the previous answer

withMCPServer( server, config ) and the mcpServers argument of aiAgent() and aiModel() accept the same options as urlGuard, redirects, maxResponseBytes, handshake and httpVersion, plus two more:

  • prefix: tools are exposed as prefix__name, so two servers with a tool called search cannot clash. The server still receives the original name.

  • toolFilter: an array of tool names. Only these are exposed to the agent.


🖥️ MCP Client over Stdio

Many MCP servers are programs, not web services: bxplaywright mcp, npx -y @modelcontextprotocol/server-github. BoxLang AI can now launch one as a child process and talk to it over stdin and stdout, as the stdio transport of the MCP specification asks. Pass a struct with a command to MCP() instead of a URL:

It has the same methods as the HTTP client (listTools, send, listResources, readResource, listPrompts, getPrompt, getCapabilities, stats and callbacks) and returns the same MCPResponse, so tools, agents and models need no change. Agents take the struct anywhere they take a URL:

Option
Default
Description

command

required

The executable followed by its arguments, as an array. Never a shell string

env

{}

Environment variables added to the child process

cwd

current directory

Working directory of the child process

inheritEnv

true

false starts the child with only env

maxRestarts

2

How many times in a row a dead server is started again

commandGuard

none

( command ) => boolean, called before every launch

timeout

30000

Milliseconds to wait for each answer

What you get:

  • Lazy launch and handshake: the process starts on the first call, then initialize and notifications/initialized are done for you, with protocol version negotiation (2025-06-18, 2025-03-26, 2024-11-05). Threads that make their first calls together wait for the handshake. The timeout covers the launch too, so give a program that downloads itself on its first run (npx -y ...) a larger one, for example 120000.

  • Concurrency and timeouts: one reader thread matches answers to callers by JSON-RPC id, so threads can share a client. A request that gets no answer fails, and the server is told with notifications/cancelled.

  • Logging: the stderr of the server goes to the ai log and its last lines are kept (getStderrTail()). Lines on stdout that are not JSON-RPC are skipped.

  • Restarts and shutdown: a dead server is started again on the next call, up to maxRestarts in a row. A call that was in flight is never repeated. close() closes stdin, waits, terminates, then kills the process and the processes it started, and live clients are also closed when the module unloads and when the JVM shuts down. Close the clients you create: a client that is never closed keeps its process until then.

  • Security: no shell is used. The module setting mcp.stdio.allowedCommands limits the executables (empty, the default, allows any): a command without a directory (npx, npx.cmd) is matched by name, a command with a directory must be listed with that exact path. commandGuard can refuse a command. A refused command throws MCPCommandRefused, also from withMCPServer(). Options a stdio server does not have (the HTTP-only ones such as token and urlGuard, or a typo such as commandGaurd) throw InvalidArgument instead of being ignored.

New for agents and models: closeMCPServers(), which closes the clients they built themselves.

Behind this, the MCP clients now share an IMCPClient interface and a BaseMCPClient base class. MCPHttpClient and MCPStdioClient only implement the transport.


🌐 HTTP Streaming Speech Demo

examples/http-streaming-speech in the bx-ai repository is a runnable MiniServer demo that answers the question "how does the user actually hear it?". It has two modes:

  • <audio> mode: speak.bxm streams raw mp3 to an <audio> tag, so playback starts after the first chunk.

  • Voice agent mode: speak-events.bxm streams NDJSON events with base64 PCM and word timestamps to a Web Audio player with word highlighting and barge-in.

The same demo is in bx-ai-intro as examples/speech-stream-http, and the BX AI Explorer has streaming samples. No browser plugin is needed.


🪲 Fixed

  • MCP stdio server (MCPRequestProcessor::processStdio()) never ended. When stdin closed, the failed read was answered with an error and tried again in a loop that wrote -32600 errors to stdout forever. A closed stdin now stops the transport, which is how a client asks a stdio server to stop.

  • Example 09-mcp-stdio.bxs printed its start banner to stdout, which is the protocol stream. It now goes to stderr.

  • MCP client withTimeout( ms ) is documented in milliseconds but the value was passed to bx:http, whose timeout is in seconds, so withTimeout( 10000 ) waited about 2.8 hours. It is now converted to seconds, rounded up, at least one. Code that relied on the long wait needs a larger value.

  • MCP client read the first data: line of an event stream as the answer, so a server that sent a notification (progress, log) before the answer made the call return the notification. Messages that are not a JSON-RPC response are now skipped.

  • MysqlVectorMemory placed metadata filter keys directly into SQL. Filter keys are now validated (letters, numbers and underscores), and an invalid key throws MysqlVectorMemory.InvalidFilterKey.

  • ElevenLabs ignored the requested voice, model, output format and speed, so voice, .female() and outputFormat had no effect. They are now honored, wav output is a real wav, and the speech-to-text upload uses the documented file field. The default voice is the premade Sarah voice, because Rachel is a library voice that free plan keys cannot use over the API.

  • Grok text-to-speech now sends the documented xAI body (text, voice_id, language, output_format). The default voice is eve, and male() and female() map to rex and eve.

  • Mistral text-to-speech now decodes the JSON audio_data response, and a UUID voice is sent as voice_id.

  • Mistral failed with Voice 'Charlotte' not found, because Charlotte is not a valid Mistral voice, and OpenAI's default voice (ash) also leaked into Mistral requests. With no voice requested, Mistral now uses the first preset voice from GET /v1/audio/voices, looked up once and cached. If that lookup fails, the request is still sent and Mistral's own error is shown.

  • Gemini text-to-speech returned headerless PCM labelled as wav. It is now wrapped in a RIFF header so the file plays.

  • Gemini defaulted to gemini-2.5-flash-preview-tts, which is no longer listed in the Gemini docs. The default is now gemini-3.8-flash-tts, and errors include the HTTP status, the model and the response body.

  • Gemini streaming text-to-speech failed with Failed to parse JSON [DONE], because the Interactions stream ends with a non-JSON [DONE] sentinel. Speech streaming now ignores it, for every provider that streams over SSE.


📖 Documentation

Last updated