# Service-Level Chatting

Take full control of AI interactions by working directly with service objects. Perfect for advanced scenarios requiring custom configuration, multiple providers, or direct API access.

## 🏗️ Service Architecture

{% @mermaid/diagram content="graph TB
subgraph "Your Application"
CODE\[Your Code]
end

```
subgraph "Service Layer"
    SVC1[aiService OpenAI]
    SVC2[aiService Claude]
    SVC3[aiService Ollama]
end

subgraph "Request Building"
    REQ[aiChatRequest]
end

subgraph "AI Providers"
    API1[OpenAI API]
    API2[Claude API]
    API3[Ollama Local]
end

CODE --> REQ
REQ --> SVC1
REQ --> SVC2
REQ --> SVC3

SVC1 --> API1
SVC2 --> API2
SVC3 --> API3

style CODE fill:#4CAF50
style SVC1 fill:#2196F3
style SVC2 fill:#9C27B0
style SVC3 fill:#FF9800
style REQ fill:#FFC107" %}
```

**Benefits:**

* Direct control over service configuration
* Multiple providers in one application
* Custom timeouts and endpoints
* Reusable service instances

## 📋 Table of Contents

* [Creating Services](#creating-services)
* [Building Chat Requests](#building-chat-requests)
* [Direct Service Invocation](#direct-service-invocation)
* [Multiple Providers](#multiple-providers)
* [Custom Configuration](#custom-configuration)
* [When to Use Services](#when-to-use-services)
* [Best Practices](#best-practices)

***

## Creating Services

### 🔄 Service Lifecycle

{% @mermaid/diagram content="sequenceDiagram
participant App as Your App
participant Svc as Service Instance
participant Req as Chat Request
participant API as Provider API

```
App->>Svc: aiService("openai")
activate Svc
Note over Svc: Configure defaults<br/>Set API key<br/>Set endpoints

App->>Req: aiChatRequest("message")
App->>Req: setParams({...})

App->>Svc: invoke(request)
Svc->>Svc: Merge request params<br/>with service defaults
Svc->>API: HTTP POST
API-->>Svc: Response
Svc->>Svc: Parse & transform
Svc-->>App: Result

Note over Svc: Service stays alive<br/>for reuse

App->>Svc: invoke(request2)
Note over Svc: Reuse same config
deactivate Svc" %}
```

### Basic Service Creation

```java
// Default API key from config or environment
service = aiService( "openai" )

// Custom API key (simple string)
service = aiService( "openai", "sk-your-key-here" )

// Configuration options struct (v2.1.0+)
service = aiService( "openai", {
    apiKey: "sk-your-key-here",
    timeout: 120
})

// Custom base URL for Ollama or OpenAI-compatible services
service = aiService( "ollama", {
    baseURL: "http://localhost:11434"
})

// Different providers
openai = aiService( "openai" )
claude = aiService( "claude" )
gemini = aiService( "gemini" )
mistral = aiService( "mistral" )
ollama = aiService( "ollama" )
```

### Service Configuration

```java
// Method 1: Configure at creation (v2.1.0+ - Recommended)
service = aiService( "openai", {
    apiKey: "sk-your-key",
    baseURL: "https://api.openai.com/v1",
    timeout: 60,
    headers: {
        "X-Custom-Header": "value"
    },
    logRequest: true,
    logResponse: false
})

// Set default parameters
service.defaults( {
    model: "gpt-4",
    temperature: 0.7,
    max_tokens: 1000
} )

// Method 2: Reconfigure after creation
service = aiService( "openai" )
service.configure({
    apiKey: "new-key",
    timeout: 90
})
```

## Building Chat Requests

Use `aiChatRequest()` for detailed request control:

### Basic Request

```java
request = aiChatRequest( "Hello, world!" )
response = service.invoke( request )
```

### With Messages Array

```java
request = aiChatRequest()
    .setMessages( [
        { role: "system", content: "You are helpful" },
        { role: "user", content: "Explain AI" }
    ] )

response = service.invoke( request )
```

### With Parameters

```java
request = aiChatRequest( "Write a story" )
    .setParams( {
        model: "gpt-4",
        temperature: 0.9,
        max_tokens: 500,
        top_p: 0.95
    } )

response = service.invoke( request )
```

### Complete Request

```java
request = aiChatRequest()
    .setMessages( [
        { role: "user", content: "Hello" }
    ] )
    .setParams( {
        model: "gpt-4",
        temperature: 0.7
    } )
    .setHeaders( {
        "X-Custom-Header": "value"
    } )
    .setOptions( {
        returnFormat: "raw",
        logRequest: true
    } )

response = service.invoke( request )
```

## Service Operations

### Invoke (Synchronous)

```java
service = aiService( "openai" )
request = aiChatRequest( "What is BoxLang?" )

response = service.invoke( request )
println( response )
```

### Invoke Stream

```java
service = aiService( "openai" )
request = aiChatRequest( "Tell me a story" )
    .setStream( true )

service.invokeStream( request, ( chunk ) => {
    content = chunk.choices?.first()?.delta?.content ?: ""
    print( content )
} )
```

## Custom Headers

Add authentication, tracking, or custom headers:

```java
request = aiChatRequest( "Hello" )
    .setHeaders( {
        "Authorization": "Bearer custom-token",
        "X-Request-ID": createUUID(),
        "X-User-ID": "user123"
    } )

response = service.invoke( request )
```

## Multiple Services

Manage multiple providers simultaneously:

```java
class {
    property name="openai";
    property name="claude";
    property name="local";

    function init() {
        variables.openai = aiService( "openai" )
        variables.claude = aiService( "claude" )
        variables.local = aiService( "ollama" )
        return this
    }

    function askAll( required string question ) {
        request = aiChatRequest( arguments.question )

        return {
            openai: variables.openai.invoke( request ),
            claude: variables.claude.invoke( request ),
            local: variables.local.invoke( request )
        }
    }

    function askBest( required string question ) {
        // Use fastest or preferred provider
        try {
            return variables.openai.invoke( aiChatRequest( arguments.question ) )
        } catch( any e ) {
            // Fallback to Claude
            return variables.claude.invoke( aiChatRequest( arguments.question ) )
        }
    }
}
```

## Advanced Patterns

### Retry Logic

```java
function invokeWithRetry( required service, required request, maxRetries = 3 ) {
    for( var i = 1; i <= maxRetries; i++ ) {
        try {
            return arguments.service.invoke( arguments.request )
        } catch( any e ) {
            if( i == maxRetries ) {
                throw( e )
            }
            sleep( 1000 * i )  // Exponential backoff
        }
    }
}

// Usage
response = invokeWithRetry( service, request )
```

### Request Queue

```java
class {
    property name="service";
    property name="queue" type="array";

    function init( required service ) {
        variables.service = arguments.service
        variables.queue = []
        return this
    }

    function enqueue( required request ) {
        variables.queue.append( arguments.request )
    }

    function process() {
        results = []
        for( request in variables.queue ) {
            results.append( variables.service.invoke( request ) )
        }
        variables.queue = []
        return results
    }
}
```

### Load Balancer

```java
class {
    property name="services" type="array";
    property name="currentIndex" type="numeric" default="1";

    function init( required array providers ) {
        variables.services = arguments.providers.map( p => aiService( p ) )
        return this
    }

    function invoke( required request ) {
        service = variables.services[ variables.currentIndex ]
        variables.currentIndex = variables.currentIndex < variables.services.len()
            ? variables.currentIndex + 1
            : 1

        return service.invoke( arguments.request )
    }
}

// Usage
balancer = new LoadBalancer( [ "openai", "claude", "gemini" ] )
response1 = balancer.invoke( request1 )  // Uses OpenAI
response2 = balancer.invoke( request2 )  // Uses Claude
response3 = balancer.invoke( request3 )  // Uses Gemini
```

## Provider-Specific Features

### OpenAI

```java
service = aiService( "openai" )
request = aiChatRequest( "Hello" )
    .setParams( {
        model: "gpt-4",
        response_format: { type: "json_object" },
        seed: 12345,  // Deterministic responses
        user: "user123"  // Track usage
    } )
```

### Claude

```java
service = aiService( "claude" )
request = aiChatRequest()
    .setMessages( [
        { role: "user", content: "Long document analysis..." }
    ] )
    .setParams( {
        model: "claude-3-opus-20240229",
        max_tokens: 4096  // Claude requires this
    } )
```

### Ollama

```java
service = aiService( "ollama" )
    .setChatURL( "http://localhost:11434" )

request = aiChatRequest( "Hello" )
    .setParams( {
        model: "llama3.2",
        temperature: 0.7
    } )
```

## Request Options

### Return Formats

```java
// Single string
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "single" } )

// All messages
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "all" } )

// Raw API response
request = aiChatRequest( "Hello" )
    .setOptions( { returnFormat: "raw" } )
```

### Logging

```java
request = aiChatRequest( "Hello" )
    .setOptions( {
        logRequest: true,
        logResponse: true,
        logRequestToConsole: true
    } )
```

## Practical Examples

### Cost Tracker

```java
class {
    property name="service";
    property name="totalTokens" default="0";

    function init( required service ) {
        variables.service = arguments.service
        return this
    }

    function invoke( required request ) {
        arguments.request.setOptions( { returnFormat: "raw" } )
        response = variables.service.invoke( arguments.request )

        variables.totalTokens += response.usage.total_tokens

        return response.choices[1].message.content
    }

    function getCost( costPerToken = 0.00002 ) {
        return variables.totalTokens * arguments.costPerToken
    }
}
```

### Response Cache

```java
class {
    property name="service";
    property name="cache" type="struct";

    function init( required service ) {
        variables.service = arguments.service
        variables.cache = {}
        return this
    }

    function invoke( required request ) {
        cacheKey = hash( serializeJSON( arguments.request ) )

        if( structKeyExists( variables.cache, cacheKey ) ) {
            return variables.cache[ cacheKey ]
        }

        response = variables.service.invoke( arguments.request )
        variables.cache[ cacheKey ] = response

        return response
    }
}
```

### A/B Testing

```java
function abTest( required string question, modelA, modelB ) {
    serviceA = aiService( "openai" )
    serviceB = aiService( "claude" )

    requestA = aiChatRequest( arguments.question )
        .setParams( { model: arguments.modelA } )
        .setOptions( { returnFormat: "raw" } )

    requestB = aiChatRequest( arguments.question )
        .setParams( { model: arguments.modelB } )
        .setOptions( { returnFormat: "raw" } )

    responseA = serviceA.invoke( requestA )
    responseB = serviceB.invoke( requestB )

    return {
        modelA: {
            response: responseA.choices[1].message.content,
            tokens: responseA.usage.total_tokens,
            model: responseA.model
        },
        modelB: {
            response: responseB.choices[1].message.content,
            tokens: responseB.usage.total_tokens,
            model: responseB.model
        }
    }
}
```

## Best Practices

1. **Reuse Service Objects**: Create once, use many times
2. **Handle Errors**: Wrap invoke() in try/catch
3. **Set Timeouts**: Prevent hanging requests
4. **Use Raw Format**: For detailed debugging and cost tracking
5. **Cache Responses**: Save money on repeated questions
6. **Implement Retries**: Handle transient failures
7. **Monitor Usage**: Track tokens and costs

## Next Steps

* [**Pipeline Overview**](broken://pages/ELUyp1BGQ6Iez4WHVERc) - Learn about AI pipelines
* [**Working with Models**](/advanced/models.md) - Services in pipelines
* [**Basic Chatting**](/main-components/chatting/basic-chatting.md) - Back to basics


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://ai.ortusbooks.com/main-components/chatting/service-chatting.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.