AI SDK Integration

Inject chaos into AI SDK v6 applications

Cruel provides comprehensive chaos testing support for Vercel AI SDK v6, enabling you to simulate real-world AI failures in your applications.

Installation

npm

npm install cruel

Quick Start

cruelModel

import { openai } from '@ai-sdk/openai'
import { generateText } from 'ai'
import { cruelModel } from 'cruel/ai-sdk'

const model = cruelModel(openai('gpt-4'), {
  rateLimit: 0.2,
  overloaded: 0.1,
  delay: [100, 500],
})

const result = await generateText({
  model,
  prompt: 'Explain quantum computing',
})

Key Features

Comprehensive Model Support

Cruel supports all AI SDK v6 model types:

• Language Models - cruelModel() for text generation
• Embedding Models - cruelEmbeddingModel() for embeddings
• Image Models - cruelImageModel() for image generation
• Speech Models - cruelSpeechModel() for text-to-speech
• Transcription Models - cruelTranscriptionModel() for speech-to-text
• Video Models - cruelVideoModel() for video generation

Realistic AI Failures

Simulate production-grade AI failures:

API Errors

Rate limits, quota exceeded, invalid API keys, model unavailability

Stream Issues

Stream cuts, slow tokens, corrupt chunks, partial responses

Content Filters

Content moderation, context length errors, empty responses

Tool Failures

Tool execution failures, tool timeouts, delayed responses

Built-in Presets

Five ready-to-use chaos presets:

import { cruelModel, presets } from 'cruel/ai-sdk'

// Light chaos for development
const dev = cruelModel(model, presets.realistic)

// Medium chaos for staging
const staging = cruelModel(model, presets.unstable)

// Heavy chaos for stress testing
const stress = cruelModel(model, presets.nightmare)

See Presets for full details.

Integration Methods

Cruel offers three ways to integrate chaos into your AI SDK applications:

1. cruelModel - Direct Model Wrapping

   Wrap individual model instances for fine-grained control:

   const model = cruelModel(openai('gpt-4'), { rateLimit: 0.1 })
   

   Best for: Testing specific models, per-model configuration

2. cruelProvider - Provider-Level Chaos

   Apply chaos to all models from a provider:

   const provider = cruelProvider(openai, { delay: [100, 300] })
   const gpt4 = provider('gpt-4')
   const gpt35 = provider('gpt-3.5-turbo')
   

   Best for: Consistent chaos across multiple models, provider-wide testing

3. cruelMiddleware - AI SDK Middleware

   Use AI SDK’s middleware system:

   const model = openai('gpt-4', {
     experimental_middleware: cruelMiddleware({ streamCut: 0.05 })
   })
   

   Best for: Integrating with existing middleware, composable chaos

Chaos Events

Monitor chaos injections in real-time:

const model = cruelModel(openai('gpt-4'), {
  rateLimit: 0.2,
  streamCut: 0.1,
  onChaos: (event) => {
    console.log(`Chaos injected: ${event.type}`, event)
  },
})

Event types include:

• rateLimit - Rate limit exceeded
• overloaded - Model overloaded
• streamCut - Stream interrupted
• delay - Response delayed
• contentFilter - Content filtered
• toolFailure - Tool execution failed

See the ChaosEvent type for all events.

Error Handling

Cruel throws AI SDK-compatible errors that your retry logic can handle:

import { CruelAPIError } from 'cruel/ai-sdk'

try {
  const result = await generateText({
    model: cruelModel(openai('gpt-4'), { rateLimit: 0.3 }),
    prompt: 'Hello',
    maxRetries: 3, // AI SDK will retry on retryable errors
  })
} catch (error) {
  if (error instanceof CruelAPIError) {
    console.log('Status:', error.statusCode)
    console.log('Retryable:', error.isRetryable)
    console.log('Message:', error.message)
  }
}

Streaming Support

Full support for AI SDK streaming:

import { streamText } from 'ai'
import { cruelModel } from 'cruel/ai-sdk'

const model = cruelModel(openai('gpt-4'), {
  slowTokens: [20, 80],    // Delay between tokens
  streamCut: 0.1,          // 10% chance of stream interruption
  corruptChunks: 0.05,     // 5% chance of corrupt chunks
})

const result = streamText({
  model,
  prompt: 'Write a story',
})

for await (const chunk of result.textStream) {
  process.stdout.write(chunk)
}

Next Steps

Model Wrapping

Learn about cruelModel and model-specific options

Provider Wrapping

Explore provider-level chaos injection

Middleware

Use AI SDK middleware for chaos

Tool Chaos

Inject failures into tool execution

Presets

Use built-in chaos presets