> For the complete documentation index, see [llms.txt](https://ai.ortusbooks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai.ortusbooks.com/introduction/release-history/2.1.0.md).

# 2.1.0

BoxLang AI Module v2.1.0 Release Notes - Multi-Tenant Tracking, OpenSearch Vector Memory, Bedrock Streaming, and Provider Configuration

This release focuses on enterprise features including multi-tenant usage tracking, OpenSearch vector memory support, AWS Bedrock streaming, and enhanced provider configuration capabilities.

## 🎉 Major Features

### Multi-Tenant Usage Tracking

Provider-agnostic request tagging for per-tenant billing and usage tracking.

**Track Usage by Tenant:**

```javascript
// Tag requests with tenant information
result = aiChat(
    messages: "Generate report",
    options: {
        tenantId: "customer-123",
        usageMetadata: {
            costCenter: "marketing",
            projectId: "campaign-2024",
            userId: "user@example.com"
        }
    }
)

// Works with all providers: OpenAI, Bedrock, Ollama, DeepSeek, etc.
```

**Intercept Token Usage Events:**

```javascript
// Register interceptor for billing
BoxRegisterInterceptor( {
    class: "TenantBillingInterceptor",
    points: {
        onAITokenCount: function( event, data ) {
            var usage = data.usage // { prompt_tokens, completion_tokens, total_tokens }
            var tenantId = data.tenantId
            var metadata = data.usageMetadata

            // Record usage for billing
            queryExecute(
                "INSERT INTO tenant_usage (tenant_id, tokens, cost_center, timestamp) VALUES (:tid, :tokens, :cc, :ts)",
                {
                    tid: tenantId,
                    tokens: usage.total_tokens,
                    cc: metadata.costCenter,
                    ts: now()
                }
            )
        }
    }
} )
```

### Predefined Provider Configuration

Define multiple providers with default parameters in module configuration.

**Module Configuration:**

```javascript
// In ModuleConfig.bx or application settings
settings = {
    providers: {
        "openai": {
            params: {
                model: "gpt-4"
            },
            options: {
                apiKey: "my-openai-api-key"
            }
        },
        "ollama": {
            params: {
                model: "qwen2.5:0.5b-instruct"
            },
            options: {
                baseUrl: "http://my-ollama-server:11434/"
            }
        }
    }
}
```

**Using Predefined Providers:**

```javascript
// Use predefined configuration
model = aiModel( provider: "openai" ) // Automatically uses gpt-4 and API key

// Seed services with additional options
model = aiModel(
    provider: "openai",
    options: { logRequestToConsole: true }
)
```

### Provider-Specific Options

Generic `providerOptions` struct for provider-specific configuration.

**AWS Bedrock Inference Profiles:**

```javascript
result = aiChat(
    provider: "bedrock",
    messages: "Analyze this data",
    options: {
        providerOptions: {
            inferenceProfileArn: "arn:aws:bedrock:us-east-1:123456789012:inference-profile/my-profile"
        }
    }
)
```

**Access Provider Options in Custom Services:**

```javascript
// In custom service implementation
function invoke( required request ) {
    var profileArn = arguments.request.getProviderOption(
        key: "inferenceProfileArn",
        defaultValue: ""
    )

    if ( profileArn.len() ) {
        // Use inference profile
    }
}
```

### OpenSearch Vector Memory Provider

Full integration with OpenSearch k-NN for semantic search and RAG applications.

**Create OpenSearch Memory:**

```javascript
// Basic configuration
memory = aiMemory(
    provider: "opensearch",
    options: {
        url: "https://search-domain.us-east-1.es.amazonaws.com",
        indexName: "conversation-memory",
        embeddingProvider: "openai",
        vectorDimension: 1536
    }
)

// AWS OpenSearch with authentication
memory = aiMemory(
    provider: "opensearch",
    options: {
        url: "https://search-domain.us-east-1.es.amazonaws.com",
        indexName: "knowledge-base",
        embeddingProvider: "openai",
        vectorDimension: 1536,
        awsRegion: "us-east-1", // Enables SigV4 authentication
        username: "admin",
        password: "MyPassword123!"
    }
)

// Custom HNSW configuration
memory = aiMemory(
    provider: "opensearch",
    options: {
        url: "http://localhost:9200",
        indexName: "vectors",
        embeddingProvider: "ollama",
        vectorDimension: 768,
        spaceType: "cosinesimilarity", // or "l2", "innerproduct"
        hnswM: 16,
        hnswEfConstruction: 100,
        hnswEfSearch: 100
    }
)
```

**Multi-Tenant Isolation:**

```javascript
// Store with tenant context
memory.add(
    messages: messages,
    userId: "user-123",
    conversationId: "conv-456"
)

// Query with tenant filtering
results = memory.query(
    query: "What did we discuss about pricing?",
    userId: "user-123",
    conversationId: "conv-456",
    limit: 5
)
```

### OpenAI-Compatible Embedding Support

Use custom OpenAI-compatible embedding services with vector memory providers.

**Custom Embedding Endpoint:**

```javascript
// Use Ollama for embeddings
memory = aiMemory(
    provider: "pinecone",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:11434/v1" // Ollama endpoint
        }
    }
)

// Use LM Studio for embeddings
memory = aiMemory(
    provider: "qdrant",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:1234/v1" // LM Studio endpoint
        }
    }
)
```

### AWS Bedrock Streaming Support

Full streaming support for AWS Bedrock provider with all model families.

**Stream Bedrock Responses:**

```javascript
// Stream with Claude on Bedrock
aiChatStream(
    provider: "bedrock",
    messages: "Write a long story",
    params: { model: "anthropic.claude-3-sonnet-20240229-v1:0" },
    onChunk: function( chunk ) {
        writeOutput( chunk.content )
    }
)

// Supported model families: Claude, Titan, Llama, Mistral
```

### Custom Ollama Base URLs

Configure custom base URLs for Ollama chat and embeddings endpoints.

**Custom Ollama Server:**

```javascript
// Configure in module settings
settings = {
    providers: {
        "ollama": {
            options: {
                baseUrl: "http://my-ollama-server:11434/"
            }
        }
    }
}

// Or pass directly to BIF
result = aiChat(
    provider: "ollama",
    messages: "Hello!",
    options: { baseUrl: "http://custom-server:11434/" }
)
```

## 🎁 New Features

* **New Event**: `onMissingAiProvider` - Handle cases where requested provider is not found
* **aiModel() Options**: New `options` struct parameter to seed services with configuration
* **Request Merge Control**: `AiBaseRequest.mergeServiceParams()` and `mergeServiceHeaders()` now accept `override` boolean argument
* **Ollama Embeddings**: Support for `nomic-embed-text` model for local embeddings

## 🔧 Enhancements

* **Provider Defaults**: All AI provider services inherit default chat and embedding parameters from `IAiService` interface
* **Service Configuration**: `IAiService.configure()` now accepts generic `options` argument instead of just `apiKey`
* **Request Clarity**: `AiRequest` class renamed to `AiChatRequest` for clarity and multi-modality support
* **Error Handling**: Added more `AiError` exception handling for service JSON errors
* **Docker Retry Logic**: Increased Model Runner retry time to 5 seconds with 10 max retries for large model loading

## 🐛 Bug Fixes

* **Event Names**: Corrected chat request event names to `onAIChatRequest`, `onAIChatRequestCreate`, and `onAIChatResponse`
* **Headers Passthrough**: `aiChat`, `aiChatStream` now correctly pass headers to `AiChatRequest`
* **Request Building**: BIFs now use `aiChatRequest()` to build requests instead of manual construction
* **MCP Spec Compliance**: Prompts now return `arguments` key instead of `args` per MCP specification
* **Model Setting**: `AiChatRequest` now correctly sets model from params
* **API Key Passthrough**: API key now properly passed to service in `aiChat()`, `aiChatStream()` BIFs
* **Character Function**: Fixed typo from `chr()` to `char()` in SSE formatting
* **Model Retrieval**: `AiModel.getModel()` now correctly returns model name when using predefined providers
* **OpenSearch URL**: Fixed parameter conflict by using `requestUrl` for HTTP requests instead of `url`

## 📚 Documentation Updates

* OpenSearch vector memory provider documentation
* Multi-tenant usage tracking examples
* Provider-specific options documentation
* Custom embedding endpoint configuration
* Bedrock streaming examples

## 🚀 Upgrade Notes

This release is backward compatible. New features are opt-in:

1. **Multi-tenant tracking** - Add `tenantId` and `usageMetadata` options to requests
2. **Predefined providers** - Configure providers in module settings for cleaner code
3. **OpenSearch** - Use for scalable vector search in production
4. **Custom embeddings** - Point to self-hosted embedding services
5. **Bedrock streaming** - Enable real-time streaming for Bedrock models

## 🙏 Thank You

Thank you to all contributors and users who continue to make BoxLang AI better!


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://ai.ortusbooks.com/introduction/release-history/2.1.0.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
