> For the complete documentation index, see [llms.txt](https://ai.ortusbooks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ai.ortusbooks.com/main-components/chatting/service-chatting.md).

# Service-Level Chatting

Take full control of AI interactions with service-level chatting in BoxLang, ideal for advanced use cases requiring custom configurations and multiple providers.

Take full control of AI interactions by working directly with service objects. Perfect for advanced scenarios requiring custom configuration, multiple providers, or direct API access.

## 🏗️ Service Architecture

```mermaid
graph TB
    subgraph "Your Application"
        CODE[Your Code]
    end

    subgraph "Service Layer"
        SVC1[aiService OpenAI]
        SVC2[aiService Claude]
        SVC3[aiService Ollama]
    end

    subgraph "Request Building"
        REQ[aiChatRequest]
    end

    subgraph "AI Providers"
        API1[OpenAI API]
        API2[Claude API]
        API3[Ollama Local]
    end

    CODE --> REQ
    REQ --> SVC1
    REQ --> SVC2
    REQ --> SVC3

    SVC1 --> API1
    SVC2 --> API2
    SVC3 --> API3

    style CODE fill:#4CAF50
    style SVC1 fill:#2196F3
    style SVC2 fill:#9C27B0
    style SVC3 fill:#FF9800
    style REQ fill:#FFC107
```

**Benefits:**

* Direct control over service configuration
* Multiple providers in one application
* Custom timeouts and endpoints
* Reusable service instances

## Creating Services

### 🔄 Service Lifecycle

```mermaid
sequenceDiagram
    participant App as Your App
    participant Svc as Service Instance
    participant Req as Chat Request
    participant API as Provider API

    App->>Svc: aiService("openai")
    activate Svc
    Note over Svc: Configure defaults<br/>Set API key<br/>Set endpoints

    App->>Req: aiChatRequest("message")
    App->>Req: setParams({...})

    App->>Svc: invoke(request)
    Svc->>Svc: Merge request params<br/>with service defaults
    Svc->>API: HTTP POST
    API-->>Svc: Response
    Svc->>Svc: Parse & transform
    Svc-->>App: Result

    Note over Svc: Service stays alive<br/>for reuse

    App->>Svc: invoke(request2)
    Note over Svc: Reuse same config
    deactivate Svc
```

### Basic Service Creation

```java
// Default API key from config or environment
service = aiService( "openai" )

// Custom API key (simple string)
service = aiService( "openai", "sk-your-key-here" )

// Configuration options struct (v2.1.0+)
service = aiService( "openai", {
    apiKey: "sk-your-key-here",
    timeout: 120
})

// Custom base URL for Ollama or OpenAI-compatible services
service = aiService( "ollama", {
    baseURL: "http://localhost:11434"
})

// Different providers
openai = aiService( "openai" )
claude = aiService( "claude" )
gemini = aiService( "gemini" )
mistral = aiService( "mistral" )
ollama = aiService( "ollama" )
```

### Service Configuration

```java
// Method 1: Configure at creation (v2.1.0+ - Recommended)
service = aiService( "openai", {
    apiKey: "sk-your-key",
    baseURL: "https://api.openai.com/v1",
    timeout: 60,
    headers: {
        "X-Custom-Header": "value"
    },
    logRequest: true,
    logResponse: false
})

// Set default parameters
service.defaults( {
    model: "gpt-4",
    temperature: 0.7,
    max_tokens: 1000
} )

// Method 2: Reconfigure after creation
service = aiService( "openai" )
service.configure({
    apiKey: "new-key",
    timeout: 90
})
```

## Building Chat Requests

Use `aiChatRequest()` for detailed request control:

### Basic Request

```java
request = aiChatRequest( "Hello, world!" )
response = service.invoke( request )
```

### With Messages Array

```java
request = aiChatRequest()
    .setMessages( [
        { role: "system", content: "You are helpful" },
        { role: "user", content: "Explain AI" }
    ] )

response = service.invoke( request )
```

### With Parameters

```java
request = aiChatRequest( "Write a story" )
    .setParams( {
        model: "gpt-4",
        temperature: 0.9,
        max_tokens: 500,
        top_p: 0.95
    } )

response = service.invoke( request )
```

### Complete Request

```java
request = aiChatRequest()
    .setMessages( [
        { role: "user", content: "Hello" }
    ] )
    .setParams( {
        model: "gpt-4",
        temperature: 0.7
    } )
    .setHeaders( {
        "X-Custom-Header": "value"
    } )
    .setOptions( {
        returnFormat: "raw",
        logRequest: true
    } )

response = service.invoke( request )
```

## Service Operations

### Invoke (Synchronous)

```java
service = aiService( "openai" )
request = aiChatRequest( "What is BoxLang?" )

response = service.invoke( request )
println( response )
```

### Invoke Stream

```java
service = aiService( "openai" )
request = aiChatRequest( "Tell me a story" )
    .setStream( true )

service.invokeStream( request, ( chunk ) => {
    content = chunk.choices?.first()?.delta?.content ?: ""
    print( content )
} )
```

## Custom Headers

Add authentication, tracking, or custom headers:

```java
request = aiChatRequest( "Hello" )
    .setHeaders( {
        "Authorization": "Bearer custom-token",
        "X-Request-ID": createUUID(),
        "X-User-ID": "user123"
    } )

response = service.invoke( request )
```

## Multiple Services

Manage multiple providers simultaneously:

```java
class {
    property name="openai";
    property name="claude";
    property name="local";

    function init() {
        variables.openai = aiService( "openai" )
        variables.claude = aiService( "claude" )
        variables.local = aiService( "ollama" )
        return this
    }

    function askAll( required string question ) {
        request = aiChatRequest( arguments.question )

        return {
            openai: variables.openai.invoke( request ),
            claude: variables.claude.invoke( request ),
            local: variables.local.invoke( request )
        }
    }

    function askBest( required string question ) {
        // Use fastest or preferred provider
        try {
            return variables.openai.invoke( aiChatRequest( arguments.question ) )
        } catch( any e ) {
            // Fallback to Claude
            return variables.claude.invoke( aiChatRequest( arguments.question ) )
        }
    }
}
```

## Advanced Patterns

### Retry Logic

```java
function invokeWithRetry( required service, required request, maxRetries = 3 ) {
    for( var i = 1; i <= maxRetries; i++ ) {
        try {
            return arguments.service.invoke( arguments.request )
        } catch( any e ) {
            if( i == maxRetries ) {
                throw( e )
            }
            sleep( 1000 * i )  // Exponential backoff
        }
    }
}

// Usage
response = invokeWithRetry( service, request )
```

### Request Queue

```java
class {
    property name="service";
    property name="queue" type="array";

    function init( required service ) {
        variables.service = arguments.service
        variables.queue = []
        return this
    }

    function enqueue( required request ) {
        variables.queue.append( arguments.request )
    }

    function process() {
        results = []
        for( request in variables.queue ) {
            results.append( variables.service.invoke( request ) )
        }
        variables.queue = []
        return results
    }
}
```

### Load Balancer

```java
class {
    property name="services" type="array";
    property name="currentIndex" type="numeric" default="1";

    function init( required array providers ) {
        variables.services = arguments.providers.map( p => aiService( p ) )
        return this
    }

    function invoke( required request ) {
        service = variables.services[ variables.currentIndex ]
        variables.currentIndex = variables.currentIndex < variables.services.len()
            ? variables.currentIndex + 1
            : 1

        return service.invoke( arguments.request )
    }
}

// Usage
balancer = new LoadBalancer( [ "openai", "claude", "gemini" ] )
response1 = balancer.invoke( request1 )  // Uses OpenAI
response2 = balancer.invoke( request2 )  // Uses Claude
response3 = balancer.invoke( request3 )  // Uses Gemini
```

## Provider-Specific Features

### OpenAI

```java
service = aiService( "openai" )
request = aiChatRequest( "Hello" )
    .setParams( {
        model: "gpt-4",
        response_format: { type: "json_object" },
        seed: 12345,  // Deterministic responses
        user: "user123"  // Track usage
    } )
```

### Claude

```java
service = aiService( "claude" )
request = aiChatRequest()
    .setMessages( [
        { role: "user", content: "Long document analysis..." }
    ] )
    .setParams( {
        model: "claude-3-opus-20240229",
        max_tokens: 4096  // Claude requires this
    } )
```

### Ollama

```java
service = aiService( "ollama" )
    .setChatURL( "http://localhost:11434" )

request = aiChatRequest( "Hello" )
    .setParams( {
        model: "llama3.2",
        temperature: 0.7
    } )
```

## Request Options

### Return Formats

```java
// Single string
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "single" } )

// All messages
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "all" } )

// Raw API response
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "raw" } )
```

### Logging

```java
request = aiChatRequest( "Hello" )
    .setOptions( {
        logRequest: true,
        logResponse: true,
        logRequestToConsole: true
    } )
```

## Practical Examples

### Cost Tracker

```java
class {
    property name="service";
    property name="totalTokens" default="0";

    function init( required service ) {
        variables.service = arguments.service
        return this
    }

    function invoke( required request ) {
        arguments.request.setOptions( { returnFormat: "raw" } )
        response = variables.service.invoke( arguments.request )

        variables.totalTokens += response.usage.total_tokens

        return response.choices[1].message.content
    }

    function getCost( costPerToken = 0.00002 ) {
        return variables.totalTokens * arguments.costPerToken
    }
}
```

### Response Cache

```java
class {
    property name="service";
    property name="cache" type="struct";

    function init( required service ) {
        variables.service = arguments.service
        variables.cache = {}
        return this
    }

    function invoke( required request ) {
        cacheKey = hash( serializeJSON( arguments.request ) )

        if( structKeyExists( variables.cache, cacheKey ) ) {
            return variables.cache[ cacheKey ]
        }

        response = variables.service.invoke( arguments.request )
        variables.cache[ cacheKey ] = response

        return response
    }
}
```

### A/B Testing

```java
function abTest( required string question, modelA, modelB ) {
    serviceA = aiService( "openai" )
    serviceB = aiService( "claude" )

    requestA = aiChatRequest( arguments.question )
        .setParams( { model: arguments.modelA } )
        .setOptions( { returnFormat: "raw" } )

    requestB = aiChatRequest( arguments.question )
        .setParams( { model: arguments.modelB } )
        .setOptions( { returnFormat: "raw" } )

    responseA = serviceA.invoke( requestA )
    responseB = serviceB.invoke( requestB )

    return {
        modelA: {
            response: responseA.choices[1].message.content,
            tokens: responseA.usage.total_tokens,
            model: responseA.model
        },
        modelB: {
            response: responseB.choices[1].message.content,
            tokens: responseB.usage.total_tokens,
            model: responseB.model
        }
    }
}
```

## Best Practices

1. **Reuse Service Objects**: Create once, use many times
2. **Handle Errors**: Wrap invoke() in try/catch
3. **Set Timeouts**: Prevent hanging requests
4. **Use Raw Format**: For detailed debugging and cost tracking
5. **Cache Responses**: Save money on repeated questions
6. **Implement Retries**: Handle transient failures
7. **Monitor Usage**: Track tokens and costs

## Next Steps

* [**Pipeline Overview**](/main-components/pipelines.md) - Learn about AI pipelines
* [**Working with Models**](/advanced/models.md) - Services in pipelines
* [**Basic Chatting**](/main-components/chatting/basic-chatting.md) - Back to basics


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://ai.ortusbooks.com/main-components/chatting/service-chatting.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.