# 3.1.0

**Released:** April 2026

BoxLang AI 3.1.0 introduces full **audio support** (text-to-speech, speech-to-text, audio translation), the **ElevenLabs** provider, **async runnables** on all pipeline objects, the new **`aiParallel()`** BIF for concurrent multi-model execution, and additional **memory events**.

***

## ✨ New Features

### 🎤 Audio BIFs

Three new global Built-in Functions expose AI audio capabilities through the same familiar interface as all other bx-ai BIFs.

| BIF                                      | Description                                  |
| ---------------------------------------- | -------------------------------------------- |
| `aiSpeak( text, params, options )`       | Convert text to natural speech audio         |
| `aiTranscribe( audio, params, options )` | Transcribe audio file/URL/binary to text     |
| `aiTranslate( audio, params, options )`  | Translate any-language audio to English text |

**Documentation**: [Audio — Speech & Transcription](/main-components/audio.md)

```javascript
// Text-to-speech — save to file
audio = aiSpeak( "Welcome to BoxLang AI!" )
audio.saveToFile( "/audio/welcome.mp3" )

// Speech-to-text — get plain text
text = aiTranscribe( "/recordings/meeting.mp3" )

// Audio translation — any language → English
english = aiTranslate( "/recordings/french-meeting.mp3" )
```

***

### 🔌 New Interfaces: `IAiSpeechService` & `IAiTranscriptionService`

Providers that support TTS and/or STT now implement one or both of these interfaces:

* **`IAiSpeechService`** — exposes `speak( speechRequest )` → `AiSpeechResponse`
* **`IAiTranscriptionService`** — exposes `transcribe( transcriptionRequest )` and `translate( transcriptionRequest )` → `AiTranscriptionResponse`

These interfaces are used internally by the `aiSpeak()`, `aiTranscribe()`, and `aiTranslate()` BIFs and are useful when extending the module with custom providers.

***

### 📦 Response Classes

#### `AiSpeechResponse`

| Method                           | Returns | Description                                                 |
| -------------------------------- | ------- | ----------------------------------------------------------- |
| `saveToFile( filePath )`         | string  | Saves audio binary to the given path; returns absolute path |
| `getBase64()`                    | string  | Base64-encoded audio data                                   |
| `getMimeType()`                  | string  | MIME type (e.g. `audio/mpeg`)                               |
| `toDataURI()`                    | string  | Ready-to-use `data:audio/mpeg;base64,...` URI               |
| `hasAudio()`                     | boolean | `true` if audio data is present and non-empty               |
| `getSize()`                      | numeric | Size in bytes                                               |
| `getAudioFormat()`               | string  | Format string: `mp3`, `wav`, `flac`, etc.                   |
| `toStruct()`                     | struct  | Metadata struct (no binary)                                 |
| `toJSON()`                       | string  | JSON-serialized metadata                                    |
| `getMetadataValue( key )`        | any     | Read a metadata value                                       |
| `setMetadataValue( key, value )` | this    | Write a metadata value (chainable)                          |

#### `AiTranscriptionResponse`

| Method                   | Returns | Description                                    |
| ------------------------ | ------- | ---------------------------------------------- |
| `getText()`              | string  | Transcribed / translated text                  |
| `hasText()`              | boolean | `true` if text is non-empty                    |
| `getWordCount()`         | numeric | Number of words                                |
| `getFormattedDuration()` | string  | Human-readable duration, e.g. `"1:23"`         |
| `hasSegments()`          | boolean | `true` if segment data is present              |
| `hasWords()`             | boolean | `true` if word-level timestamp data is present |
| `getSegments()`          | array   | Array of segment structs with timestamps       |
| `getWords()`             | array   | Array of word structs with timestamps          |
| `toStruct()`             | struct  | Full struct representation                     |
| `toJSON()`               | string  | JSON-serialized response                       |
| `toString()`             | string  | Alias for `getText()`                          |

#### `AiBaseResponse` (shared base)

Both response classes extend `AiBaseResponse`, which provides:

* `model` — model used for the operation
* `provider` — provider name (string)
* `metadata` — raw provider response metadata struct
* `timestamp` — datetime when response was created
* `toJSON()` — serialize to JSON
* `getMetadataValue( key )` — read from metadata bag
* `setMetadataValue( key, value )` — write to metadata bag (chainable)

***

### 🤖 ElevenLabs Provider

A new `elevenlabs` provider brings high-quality, multilingual neural voice synthesis.

**Supported operations:** Text-to-speech only (STT/translation not supported)

**Authentication:** Set `ELEVENLABS_API_KEY` in your environment or pass `apiKey` in options. ElevenLabs uses the `xi-api-key` header (handled automatically by the module).

**Default model:** `eleven_multilingual_v2`

**Voice selection:** Pass a `voice_id` from your ElevenLabs voice library in `params`.

```javascript
audio = aiSpeak(
    "Bonjour, bienvenue dans BoxLang AI.",
    { voice_id: "21m00Tcm4TlvDq8ikWAM" },  // Rachel voice ID
    { provider: "elevenlabs" }
)
audio.saveToFile( "/audio/bonjour.mp3" )
```

**Module configuration:**

```javascript
// config/boxlang.json
{
    "modules": {
        "bxai": {
            "settings": {
                "providers": {
                    "elevenlabs": {
                        "params": {
                            "model": "eleven_multilingual_v2"
                        }
                    }
                }
            }
        }
    }
}
```

***

### ⚡ Async Runnables — `runAsync()`

All `IAiRunnable` objects now expose a `runAsync()` method that dispatches execution to the `io-tasks` virtual thread pool and returns a **`BoxFuture`**.

**Affected types:** `AiModel`, `AiAgent`, `AiRunnableSequence`, `AiTransformRunnable`, `AiRunnableParallel`

```javascript
model = aiModel( "openai" )

// Non-blocking execution
future = model.runAsync( "Summarize this document..." )

// Block and retrieve result when you need it
result = future.get()
println( result )

// Or use a callback
future.then( result -> {
    println( "Done: #result#" )
})
```

This is especially powerful when running multiple independent tasks simultaneously:

```javascript
tasks = [
    aiModel( "openai" ).runAsync( "Summarize topic A" ),
    aiModel( "openai" ).runAsync( "Summarize topic B" ),
    aiModel( "openai" ).runAsync( "Summarize topic C" )
]

// All three run concurrently; collect results as they complete
summaries = tasks.map( f => f.get() )
```

***

### 🔀 Parallel Pipelines — `aiParallel()`

The new `aiParallel()` BIF creates an `AiRunnableParallel` that fans-out a single input to multiple named runnables **concurrently** and collects results into a named struct.

```javascript
aiParallel( runnables )
```

| Parameter   | Type   | Description                                                                     |
| ----------- | ------ | ------------------------------------------------------------------------------- |
| `runnables` | struct | Named struct where each key is a result name and each value is an `IAiRunnable` |

**Returns:** `AiRunnableParallel` — implements `IAiRunnable`, so it can be used in any pipeline.

```javascript
parallel = aiParallel({
    openai:  aiModel( "openai",  { params: { model: "gpt-4o-mini" } } ),
    claude:  aiModel( "claude",  { params: { model: "claude-3-haiku-20240307" } } ),
    mistral: aiModel( "mistral", { params: { model: "mistral-small-latest" } } )
})

results = parallel.run( "What is the capital of France?" )

println( results.openai  )  // "Paris"
println( results.claude  )  // "Paris"
println( results.mistral )  // "The capital of France is Paris."
```

Because `AiRunnableParallel` implements `IAiRunnable`, it composes into larger pipelines:

```javascript
pipeline = aiParallel({ fast: aiModel( "groq" ), smart: aiModel( "openai" ) })
    .transform( results => "Fast: #results.fast#\nSmart: #results.smart#" )

combined = pipeline.run( "Explain quantum entanglement in one sentence" )
println( combined )
```

***

### 🎤 Audio Agent Tools

Three new tools are auto-registered in the global AI Tool Registry at module startup, making audio capabilities immediately available to any agent:

| Tool Key          | Description                                                                                                     |
| ----------------- | --------------------------------------------------------------------------------------------------------------- |
| `speak@bxai`      | Convert text to speech; returns the saved file path (auto-generates a temp file if no `outputFile` is supplied) |
| `transcribe@bxai` | Transcribe a local audio file or URL to plain text                                                              |
| `translate@bxai`  | Translate any-language audio to English text                                                                    |

Opt-in by referencing the tool keys in your agent definition — audio tools are **not** injected automatically:

```javascript
agent = aiAgent(
    name: "voice-assistant",
    aiModel: "openai",
    tools: [ "speak@bxai", "transcribe@bxai", "translate@bxai" ]
)
```

***

### 🗂️ FileSystem Agent Tools

A new `FileSystemTools` class (`models/tools/filesystem/FileSystemTools.bx`) ships 19 `@mcpTool`-annotated methods that cover the full filesystem lifecycle.

> ⚠️ **Security-first design**: `FileSystemTools` is **NOT auto-registered**. You must opt in explicitly via `aiToolRegistry().scanClass()` so agents never receive filesystem access unless you grant it.

**Path-guard constructor**: Pass `allowedPaths: [...]` to canonicalize and validate every path argument at invocation time, blocking directory-traversal attacks before any file operation runs.

```javascript
// Register with path restrictions
aiToolRegistry().scanClass(
    new FileSystemTools( allowedPaths: [ "/app/data", "/app/uploads" ] )
)

// Assign specific tools to an agent
agent = aiAgent(
    name: "file-manager",
    aiModel: "openai",
    tools: [
        "readFile@bxai",
        "writeFile@bxai",
        "listDirectory@bxai"
    ]
)
```

**Available tool keys:**

| Tool Key                      | Description                                     |
| ----------------------------- | ----------------------------------------------- |
| `readFile@bxai`               | Read a file's contents                          |
| `readMultipleFiles@bxai`      | Read multiple files at once                     |
| `writeFile@bxai`              | Write content to a file                         |
| `appendFile@bxai`             | Append content to a file                        |
| `editFile@bxai`               | Apply targeted edits to a file                  |
| `fileMetadata@bxai`           | Get file size, timestamps, permissions          |
| `pathExists@bxai`             | Check whether a path exists                     |
| `deleteFile@bxai`             | Delete a file                                   |
| `moveFile@bxai`               | Move or rename a file                           |
| `copyFile@bxai`               | Copy a file                                     |
| `searchFiles@bxai`            | Search for files by pattern or content          |
| `listAllowedDirectories@bxai` | List directories the agent is allowed to access |
| `listDirectory@bxai`          | List directory contents                         |
| `directoryTree@bxai`          | Recursive directory tree                        |
| `createDirectory@bxai`        | Create a directory                              |
| `deleteDirectory@bxai`        | Delete a directory                              |
| `zipFiles@bxai`               | Compress files into a zip archive               |
| `unzipFile@bxai`              | Extract a zip archive                           |
| `checkZipFile@bxai`           | Inspect a zip archive's contents                |

***

### 🧠 New Memory Events

Two additional interception points are now fired by the memory subsystem.

| Event               | When Fired                                          | Available Data                        |
| ------------------- | --------------------------------------------------- | ------------------------------------- |
| `onHybridMemoryAdd` | When a message is added to a HybridMemory instance  | `memory`, `message`                   |
| `onVectorSearch`    | When a semantic search runs against a vector memory | `memory`, `query`, `limit`, `results` |

```javascript
BoxRegisterInterceptor( "onVectorSearch", event => {
    println( "Vector search: query='#event.query#' found #event.results.len()# results" )
})
```

***

### 🔊 Audio Module Configuration

A new `audio` section in module settings provides defaults for all audio operations:

```javascript
// config/boxlang.json
{
    "modules": {
        "bxai": {
            "settings": {
                "audio": {
                    "defaultVoice": "alloy",
                    "defaultOutputFormat": "mp3",
                    "defaultSpeechModel": "tts-1",
                    "defaultTranscriptionModel": "whisper-1"
                }
            }
        }
    }
}
```

### 📡 New Audio Events

Six new interception points cover all audio operations:

| #  | Event                   | When Fired                                      |
| -- | ----------------------- | ----------------------------------------------- |
| 34 | `beforeAISpeech`        | Before a text-to-speech request is sent         |
| 35 | `afterAISpeech`         | After a text-to-speech response is received     |
| 36 | `beforeAITranscription` | Before a speech-to-text request is sent         |
| 37 | `afterAITranscription`  | After a speech-to-text response is received     |
| 38 | `beforeAITranslation`   | Before an audio translation request is sent     |
| 39 | `afterAITranslation`    | After an audio translation response is received |

***

## 🐛 Bug Fixes

1. **Streaming `onAITokenCount` never fired**: `chatStream()` across all providers never announced the `onAITokenCount` event, making all streaming calls invisible to usage tracking, billing interceptors, and monitoring. The non-streaming `chat()` path was unaffected.
2. **`AiModel.stream()` missing middleware**: Agent and model middleware were not injected into `chatRequest` during streaming, unlike the `run()` path — now consistent.
3. **Closure scoping in tool calls**: `DockerModelRunnerService`, `OpenAIService.chat()`, `OpenAIService.chatStream()`, and `CohereService.chat()` all had `ArgumentsScope` resolution failures inside `.each()` / `.map()` closures used for tool calling; fixed by capturing outer variables into local vars before the closure.
4. **`onAITokenCount` standardization**: Event data shape was inconsistent across providers. Standardized and added the missing event fire to `BedrockService`, `ClaudeService`, `CohereService`, and `GeminiService`.
5. **`MCPServer.scan()` / `scanClass()`**: Several path-resolution cases and permutation combinations were not handled correctly; now reliable across dot-notation, absolute, relative, and instance inputs.
6. **`aiAgent()`** **`skills` / `availableSkills` normalization**: Both parameters now accept a single skill instance *or* an array — the BIF normalizes to an array internally, removing the requirement to wrap a single skill in `[]`.
7. **`ModuleConfig.bx` startup order**: Module now listens to `onRuntimeStart()` instead of an earlier hook, ensuring caches and other services are fully initialized before skills and global tool registry setup runs.
8. **Flight recorder tape directory**: Tapes were written to an invalid/incorrect directory path; corrected.
9. **Docker Service interface compatibility**: Resolved breakage caused by the capability-interface refactor introduced in 3.0.0.
10. **Memory count off-by-one**: `WindowMemory.count()` returned `maxMessages + 1` instead of actual message count when at capacity
11. **Tool result serialization**: Nested arrays in tool return values were double-serialized as JSON strings
12. **Stream callback null chunk**: Streaming responses from Gemini occasionally emitted a `null` chunk on the final SSE event; now handled gracefully
13. **AWS Bedrock signature**: Signed URL included a trailing slash in the path that caused `SignatureDoesNotMatch` errors on certain endpoints
14. **ElevenLabs MIME detection**: `AiSpeechResponse.getMimeType()` returned `application/octet-stream` for ElevenLabs PCM audio; corrected to `audio/pcm`
15. **Groq transcription model default**: Module was sending `whisper-1` instead of `whisper-large-v3` when no model was specified for Groq
16. **aiParallel empty struct**: Calling `aiParallel({}).run("input")` threw a `NullPointerException`; now returns an empty struct
17. **SessionMemory web-only check**: `SessionMemory` threw a misleading error when called outside a web context; error message now clearly states it requires an active HTTP session
18. **PDF loader binary mode**: `PDFLoader` opened files in text mode on Windows, corrupting multi-byte characters; now always opens in binary mode
19. **HybridMemory deduplication**: Combining `WindowMemory` recent messages with vector search results could produce duplicate messages when the same message ranked highly in both sources
20. **aiTranscribe URL redirect**: HTTP redirects (301/302) from audio file CDNs were not followed; now follows up to 5 redirects before throwing

***

## 📖 Documentation

* [Audio — Speech & Transcription](/main-components/audio.md)
* [Text-to-Speech](/main-components/audio/text-to-speech.md)
* [Speech-to-Text](/main-components/audio/speech-to-text.md)
* [Audio Translation](/main-components/audio/audio-translation.md)
* [Parallel Pipelines](/main-components/pipelines/advanced.md)
* [Events Reference](/advanced/events.md)
* [aiSpeak BIF Reference](/advanced/reference/built-in-functions/aispeak.md)
* [aiTranscribe BIF Reference](/advanced/reference/built-in-functions/aitranscribe.md)
* [aiTranslate BIF Reference](/advanced/reference/built-in-functions/aitranslate.md)
* [aiParallel BIF Reference](/advanced/reference/built-in-functions/aiparallel.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ai.ortusbooks.com/readme/release-history/3.1.0.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.