Skip to Content
Engineering11 Documentation 🔥
BackendServer API PackagesAI

AI API

Production-ready OpenAI and LangChain wrapper with structured output validation, observability, and rate limiting.

What It Does

The AI API provides a sophisticated layer over OpenAI and LangChain that enables type-safe, observable AI text generation with automatic retries, progress tracking, and usage quota enforcement. It transforms complex AI prompt engineering into simple, validated TypeScript workflows with built-in monitoring and error handling.

Key Capabilities

CapabilityDescription
Structured Output GenerationDefine prompts with Zod schemas for type-safe, validated AI responses
ObservabilityFull Langfuse integration for tracing, token usage tracking, and execution metadata
ResilienceExponential backoff retries, configurable timeouts, and graceful error handling
Progress TrackingStage-based progress updates with percentage and status messages
Rate LimitingPer-user quota enforcement with Foundation access control integration
Text CompletionSimple text generation with token approximation and throttling handling
Content ModerationOpenAI moderation API integration for policy violation checks
Batch OperationsParallel prompt execution with transaction-safe rate limiting

Configuration

Default Configuration

  • Model: gpt-4o-mini
  • Temperature: 0.1
  • Max length: 200 characters (~50 tokens)
  • Retry attempts: 3
  • Base delay: 1000ms

Environment Variables

  • OPENAI_API_KEY - OpenAI API key (stored in secrets manager)
  • LANGFUSE_PUBLIC_KEY - Langfuse public key
  • LANGFUSE_SECRET_KEY - Langfuse secret (in secrets manager)
  • LANGFUSE_HOST - Optional custom Langfuse host

Error Types

  • USE_LIMIT_EXCEEDED - User quota exceeded
  • COMPLETION_PROVIDER_UNKNOWN_ERROR - Unknown provider error
  • COMPLETION_PROVIDER_HTTP_ERROR - HTTP errors
  • COMPLETION_PROVIDER_THROTTLE - Rate limit (429) errors

Exports

Services

  • TextCompletionService - Basic text completion
  • UseLimitedTextCompletionService - Rate-limited completion
  • TemplateExecutorService - Structured prompt execution
  • TextModerationService - Content moderation
  • OpenAIService - OpenAI client wrapper

Classes and Types

  • E11PromptInput - Base class for prompt definitions
  • TextCompletionConfig - Configuration interface
  • PromptTemplateStringType - Type-safe prompt templates

Functions

  • executeFromPromptInput() - Standalone executor
  • executeFromChain() - Chain execution
  • buildLangchainTraceHandler() - Langfuse handler

Constants

  • Error type constants
  • Default configuration values

Common Use Cases

  • Structured data extraction: Parse unstructured text into typed objects
  • Content generation: Generate marketing copy, product descriptions, summaries
  • Analysis workflows: Analyze documents, code, user feedback with structured output
  • Hiring automation: Resume analysis with multi-section structured reports
  • Content moderation: Screen user-generated content for policy violations
  • Multi-step workflows: Chain multiple AI operations with progress tracking
  • User-facing AI features: Build AI features with rate limiting and quota management

What Customers Don’t Have to Build

  • Structured output validation with Zod schemas
  • LLM call observability and tracing
  • Token usage tracking and cost monitoring
  • Exponential backoff retry logic
  • Progress tracking for long-running operations
  • User quota and rate limiting systems
  • Content moderation integration
  • Batch prompt execution with ordering
  • Timeout and cancellation management
  • LangChain integration and chain execution
  • OpenAI API error handling
  • Secrets management for API keys
  • Model configuration management
  • Trace context propagation
  • Execution metadata capture
Last updated on
Dynamic LinkCrypto