3.6.0
BoxLang AI Module 3.6.0. Cartesia provider, streaming text-to-speech with aiSpeakStream(), Cloudflare Workers AI provider, MariaDB vector memory, an MCP client for servers that run as a child process
Released: 2026-10-20
BoxLang AI 3.6.0 is a voice and MCP release. It adds streaming text-to-speech so you can hand synthesized audio to a caller, a browser or a phone bridge while it is still being generated, a new Cartesia provider for fast text-to-speech and speech-to-text, and a Cloudflare Workers AI provider for chat, streaming, tool calling and embeddings.
It also adds MariaDB vector memory, which runs semantic search inside MariaDB 11.7+ using its native vector type and index.
It also lets agents use MCP servers that run as a child process over stdin and stdout, such as bxplaywright mcp, with the same MCP() BIF, the same MCPResponse and the same agent integration as HTTP servers.
It also fixes a batch of speech provider bugs found while verifying every speech service end to end. ElevenLabs, Grok, Mistral and Gemini each had at least one request that silently ignored your settings or returned audio that would not play.
⚠️ Migration
MCPClient was renamed to MCPHttpClient. The MCP() BIF, withMCPServer() and the mcpServers argument are not affected, and MCP() now returns an IMCPClient. Only code that created the class directly needs a change:
// Before
client = new bxModules.bxai.models.mcp.MCPClient( "http://localhost:3000" )
// After
client = MCP( "http://localhost:3000" )The named argument MCP( baseURL: url ) still works (the new name is mcpServer). The client events (onMCPClientRequest, onMCPClientResponse, onMCPClientError) now carry the IMCPClient in client, and have a new transport key. See Events.
✨ New Features
🔊 Streaming Text-to-Speech: aiSpeakStream()
aiSpeak() waits for the complete audio file. aiSpeakStream() calls your function with audio as the provider generates it, so the listener hears the first words long before the full answer is synthesized. It is built for voice agents that restream audio to a customer.
aiSpeakStream( text, callback, params, options )
Stream audio chunks to a callback and return a summary struct
aiSpeak().text( "..." ).stream( callback )
The same thing through the fluent builder
The callback receives one struct per event, discriminated by type:
audio
{ type, data, format, sampleRate, sequence } where data is binary
timestamps
{ type, words, start, end } (Cartesia over SSE with add_timestamps)
done
{ type, chunks, bytes }, always last
Stop early (barge-in): return an explicit false from the callback. The provider connection is closed immediately and the summary reports completed: false.
Providers: Cartesia, ElevenLabs, OpenAI, Mistral and Gemini. Grok only streams over a WebSocket, which is not implemented yet, so it does not declare the capability.
Formats for voice agents: mp3 plays in a browser <audio> tag as it arrives, pcm is raw 16-bit audio with no decoding for agents, and mulaw or alaw are 8 kHz telephony formats.
Under the hood this adds the IAiSpeechStreamService interface and the speechStream capability. aiSpeakStream() requires both the interface and the declared capability on the provider. The timeout is an idle timeout: a long stream is fine as long as chunks keep arriving.
Documentation: Streaming Text-to-Speech, aiSpeakStream reference
🎙️ Cartesia Provider
Cartesia is a low-latency voice provider. Sonic generates speech and Ink-Whisper transcribes it, through the same BIFs you already use.
Text-to-speech
Sonic, output mp3, wav, pcm, mulaw, alaw
Streaming text-to-speech
Bytes for mp3 and wav, SSE with word timestamps for pcm, mulaw and alaw
Speech-to-text
Ink-Whisper through aiTranscribe()
Translation
Not supported, translate() throws UnsupportedCapability
flac and opus output throw InvalidArgument instead of quietly returning the wrong format. Set params.sample_rate to change the rate of raw formats.
Documentation: Audio overview, Text-to-Speech
☁️ Cloudflare Workers AI Provider
Chat, streaming, tool calling and embeddings through the OpenAI compatible endpoints of Workers AI.
Provider name
cloudflare
API key
CLOUDFLARE_API_KEY
Account ID
The accountId option, or the CLOUDFLARE_ACCOUNT_ID environment variable
Default chat model
@cf/openai/gpt-oss-20b
Default embeddings model
@cf/baai/bge-base-en-v1.5
Capabilities
chat, stream, embeddings
Documentation: Provider Setup
🗄️ MariaDB Vector Memory
MariaDBVectorMemory stores embeddings in MariaDB 11.7+ and ranks them inside the database with the native VECTOR(n) type, an HNSW VECTOR INDEX and the VEC_DISTANCE_COSINE / VEC_DISTANCE_EUCLIDEAN functions. Unlike MysqlVectorMemory, only the top matches come back to BoxLang instead of every row.
Memory type
mariadb (or MariaDBVectorMemory)
Requires
MariaDB 11.7+ and the bx-mariadb module
datasource
Required
distanceFunction
COSINE (default) or L2. MariaDB has no native dot product
dimensions
Must match the embedding model, default 1536
m
HNSW M, 3 to 200, default 8
autoCreate
Create the table if missing, default true
Multi-tenant
userId and conversationId use dedicated columns
MariaDB allows one vector index per table, fixed to one metric and one dimension size, so collections that share a table share both. Configuring an existing table with a different distanceFunction throws MariaDBVectorMemory.DistanceMismatch. It also works as the vectorProvider of HybridMemory.
Documentation: MariaDBVectorMemory, Multi-Tenant Memory
🔒 MCP Client Controls
Attaching an MCP server used to mean trusting its address completely. You can now decide what a client may reach. Every option is off by default, so nothing changes unless you ask.
withUrlGuard( fn )
Called with the URL before every request. Return false or throw to refuse it. Nothing is sent
withRedirects( false )
A redirect becomes a failed response and the new address is never called
withMaxResponseBytes( n )
Refuse an answer longer than n characters
withHandshake()
Send the MCP initialize handshake once and send back the Mcp-Session-Id
withHttpVersion( "HTTP/1.1" )
Choose the HTTP version. The default stays HTTP/2, which on a plain http address starts with an upgrade request that some servers answer badly
listTools( cursor )
Read the next page of a server that pages its tools, with the nextCursor of the previous answer
withMCPServer( server, config ) and the mcpServers argument of aiAgent() and aiModel() accept the same options as urlGuard, redirects, maxResponseBytes, handshake and httpVersion, plus two more:
prefix: tools are exposed asprefix__name, so two servers with a tool calledsearchcannot clash. The server still receives the original name.toolFilter: an array of tool names. Only these are exposed to the agent.
Documentation: Controlling What a Client Can Reach, MCP Server Options
🖥️ MCP Client over Stdio
Many MCP servers are programs, not web services: bxplaywright mcp, npx -y @modelcontextprotocol/server-github. BoxLang AI can now launch one as a child process and talk to it over stdin and stdout, as the stdio transport of the MCP specification asks. Pass a struct with a command to MCP() instead of a URL:
It has the same methods as the HTTP client (listTools, send, listResources, readResource, listPrompts, getPrompt, getCapabilities, stats and callbacks) and returns the same MCPResponse, so tools, agents and models need no change. Agents take the struct anywhere they take a URL:
command
required
The executable followed by its arguments, as an array. Never a shell string
env
{}
Environment variables added to the child process
cwd
current directory
Working directory of the child process
inheritEnv
true
false starts the child with only env
maxRestarts
2
How many times in a row a dead server is started again
commandGuard
none
( command ) => boolean, called before every launch
timeout
30000
Milliseconds to wait for each answer
What you get:
Lazy launch and handshake: the process starts on the first call, then
initializeandnotifications/initializedare done for you, with protocol version negotiation (2025-06-18,2025-03-26,2024-11-05). Threads that make their first calls together wait for the handshake. Thetimeoutcovers the launch too, so give a program that downloads itself on its first run (npx -y ...) a larger one, for example120000.Concurrency and timeouts: one reader thread matches answers to callers by JSON-RPC id, so threads can share a client. A request that gets no answer fails, and the server is told with
notifications/cancelled.Logging: the stderr of the server goes to the
ailog and its last lines are kept (getStderrTail()). Lines on stdout that are not JSON-RPC are skipped.Restarts and shutdown: a dead server is started again on the next call, up to
maxRestartsin a row. A call that was in flight is never repeated.close()closes stdin, waits, terminates, then kills the process and the processes it started, and live clients are also closed when the module unloads and when the JVM shuts down. Close the clients you create: a client that is never closed keeps its process until then.Security: no shell is used. The module setting
mcp.stdio.allowedCommandslimits the executables (empty, the default, allows any): a command without a directory (npx,npx.cmd) is matched by name, a command with a directory must be listed with that exact path.commandGuardcan refuse a command. A refused command throwsMCPCommandRefused, also fromwithMCPServer(). Options a stdio server does not have (the HTTP-only ones such astokenandurlGuard, or a typo such ascommandGaurd) throwInvalidArgumentinstead of being ignored.
New for agents and models: closeMCPServers(), which closes the clients they built themselves.
Behind this, the MCP clients now share an IMCPClient interface and a BaseMCPClient base class. MCPHttpClient and MCPStdioClient only implement the transport.
Documentation: MCP Stdio Client, MCP BIF reference, Stdio servers for agents
🌐 HTTP Streaming Speech Demo
examples/http-streaming-speech in the bx-ai repository is a runnable MiniServer demo that answers the question "how does the user actually hear it?". It has two modes:
<audio>mode:speak.bxmstreams raw mp3 to an<audio>tag, so playback starts after the first chunk.Voice agent mode:
speak-events.bxmstreams NDJSON events with base64 PCM and word timestamps to a Web Audio player with word highlighting and barge-in.
The same demo is in bx-ai-intro as examples/speech-stream-http, and the BX AI Explorer has streaming samples. No browser plugin is needed.
🪲 Fixed
MCP stdio server (
MCPRequestProcessor::processStdio()) never ended. When stdin closed, the failed read was answered with an error and tried again in a loop that wrote-32600errors to stdout forever. A closed stdin now stops the transport, which is how a client asks a stdio server to stop.Example
09-mcp-stdio.bxsprinted its start banner to stdout, which is the protocol stream. It now goes to stderr.MCP client
withTimeout( ms )is documented in milliseconds but the value was passed tobx:http, whose timeout is in seconds, sowithTimeout( 10000 )waited about 2.8 hours. It is now converted to seconds, rounded up, at least one. Code that relied on the long wait needs a larger value.MCP client read the first
data:line of an event stream as the answer, so a server that sent a notification (progress, log) before the answer made the call return the notification. Messages that are not a JSON-RPC response are now skipped.MysqlVectorMemory placed metadata filter keys directly into SQL. Filter keys are now validated (letters, numbers and underscores), and an invalid key throws
MysqlVectorMemory.InvalidFilterKey.ElevenLabs ignored the requested voice, model, output format and speed, so
voice,.female()andoutputFormathad no effect. They are now honored,wavoutput is a real wav, and the speech-to-text upload uses the documentedfilefield. The default voice is the premade Sarah voice, because Rachel is a library voice that free plan keys cannot use over the API.Grok text-to-speech now sends the documented xAI body (
text,voice_id,language,output_format). The default voice iseve, andmale()andfemale()map torexandeve.Mistral text-to-speech now decodes the JSON
audio_dataresponse, and a UUID voice is sent asvoice_id.Mistral failed with
Voice 'Charlotte' not found, becauseCharlotteis not a valid Mistral voice, and OpenAI's default voice (ash) also leaked into Mistral requests. With no voice requested, Mistral now uses the first preset voice fromGET /v1/audio/voices, looked up once and cached. If that lookup fails, the request is still sent and Mistral's own error is shown.Gemini text-to-speech returned headerless PCM labelled as wav. It is now wrapped in a RIFF header so the file plays.
Gemini defaulted to
gemini-2.5-flash-preview-tts, which is no longer listed in the Gemini docs. The default is nowgemini-3.8-flash-tts, and errors include the HTTP status, the model and the response body.Gemini streaming text-to-speech failed with
Failed to parse JSON [DONE], because the Interactions stream ends with a non-JSON[DONE]sentinel. Speech streaming now ignores it, for every provider that streams over SSE.
📖 Documentation
MCP Stdio Client, new
MCP BIF reference, rewritten against the real API. It used
callTool()andgetResource(), which do not exist (the methods aresend()andreadResource()), and treated the answer oflistTools()as an array instead of anMCPResponseMCP Server Transports, now with the rules for a stdio server and the correct
processStdio()nameThe
bx-ai-mcpskill in ortus-boxlang/skills covers MCP clients and servers for AI coding agents
Last updated