Integration Overview

Jetty provides extensive integrations with AI models, data processing tools, and external services to create powerful workflow capabilities. This comprehensive integration ecosystem enables you to build sophisticated AI/ML pipelines with minimal configuration.

Overview

Jetty's integration architecture is built on three core principles:

1. Unified Interface: Consistent configuration patterns across all integrations
2. Secure Credentials: Multi-layered secrets management with environment variable fallbacks
3. Extensible Design: Easy addition of new integrations through the step library system

AI Model Integrations

Google Gemini Integration

Jetty provides native integration with Google's Gemini models through the Google AI SDK.

Supported Models

• gemini-2.0-flash-001 (default) - Latest high-performance model
• gemini-pro - Production-ready balanced model
• gemini-pro-vision - Multimodal model with image understanding

Configuration

Environment Variables:

GEMINI_API_KEY=your_google_ai_api_key

Workflow Configuration:

{
  "step_configs": {
    "analyze_text": {
      "activity": "gemini_prompt",
      "model": "gemini-2.0-flash-001",
      "prompt": "Analyze the sentiment of this text: {{init_params.text}}",
      "temperature": 0.7,
      "api_key_secret": "GEMINI_API_KEY"
    }
  }
}

Available Gemini Steps

GeminiPrompt (gemini_prompt)

{
  "activity": "gemini_prompt",
  "model": "gemini-2.0-flash-001",
  "prompt": "Your prompt here",
  "api_key": "optional_direct_api_key",
  "api_key_secret": "GEMINI_API_KEY"
}

GeminiTextReader (gemini_text_reader)

{
  "activity": "gemini_text_reader",
  "text_path": "previous_step.outputs.text",
  "prompt": "Summarize this text in 100 words",
  "model": "gemini-pro"
}

GeminiJSONReader (gemini_json_reader)

{
  "activity": "gemini_json_reader",
  "json_path": "data_step.outputs.json_data",
  "prompt": "Extract key insights from this JSON data",
  "model": "gemini-pro"
}

LiteLLM Multi-Provider Integration

LiteLLM provides a unified interface to 100+ language model providers including OpenAI, Anthropic, Azure, and more.

Supported Providers

OpenAI Models:

• gpt-4, gpt-4-turbo, gpt-4o
• gpt-3.5-turbo, gpt-3.5-turbo-16k
• dall-e-3, dall-e-2 (image generation)

Anthropic Models:

• claude-3-opus-20240229
• claude-3-sonnet-20240229
• claude-3-haiku-20240307

Google Models (via LiteLLM):

• gemini-pro, gemini-pro-vision
• palm-2-chat-bison, palm-2-codechat-bison

Azure OpenAI:

• All OpenAI models through Azure endpoints

Local/Open Source Models:

• Ollama models (ollama/llama2, ollama/codellama)
• Together AI models
• Replicate models

Configuration

Environment Variables:

# OpenAI
OPENAI_API_KEY=your_openai_api_key

# Anthropic
ANTHROPIC_API_KEY=your_anthropic_api_key

# Azure OpenAI
AZURE_API_KEY=your_azure_api_key
AZURE_API_BASE=https://your-resource.openai.azure.com/
AZURE_API_VERSION=2023-12-01-preview

# Google AI
GOOGLE_API_KEY=your_google_api_key

# LiteLLM (fallback)
LITELLM_API_KEY=your_provider_api_key

Available LiteLLM Steps

LiteLLMChat (litellm_chat)

{
  "activity": "litellm_chat",
  "model": "gpt-4",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant" },
    { "role": "user", "content": "{{init_params.question}}" }
  ],
  "temperature": 0.7,
  "max_tokens": 1000,
  "api_key_secret": "OPENAI_API_KEY"
}

Cross-Provider Comparison Example:

{
  "steps": [
    "openai_analysis",
    "claude_analysis",
    "gemini_analysis",
    "compare_results"
  ],
  "step_configs": {
    "openai_analysis": {
      "activity": "litellm_chat",
      "model": "gpt-4",
      "prompt": "{{init_params.question}}",
      "api_key_secret": "OPENAI_API_KEY"
    },
    "claude_analysis": {
      "activity": "litellm_chat",
      "model": "claude-3-sonnet-20240229",
      "prompt": "{{init_params.question}}",
      "api_key_secret": "ANTHROPIC_API_KEY"
    },
    "gemini_analysis": {
      "activity": "litellm_chat",
      "model": "gemini-pro",
      "prompt": "{{init_params.question}}",
      "api_key_secret": "GOOGLE_API_KEY"
    },
    "compare_results": {
      "activity": "litellm_chat",
      "model": "gpt-4",
      "messages": [
        {
          "role": "user",
          "content": "Compare these three analyses:\nOpenAI: {{openai_analysis.outputs.text}}\nClaude: {{claude_analysis.outputs.text}}\nGemini: {{gemini_analysis.outputs.text}}"
        }
      ]
    }
  }
}

Replicate Model Integration

Access thousands of open-source models through Replicate's cloud API.

Configuration

Environment Variables:

REPLICATE_API_TOKEN=your_replicate_token

Popular Models:

{
  "step_configs": {
    "generate_image": {
      "activity": "replicate_prediction",
      "model": "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
      "input": {
        "prompt": "{{init_params.image_prompt}}",
        "num_outputs": 1,
        "guidance_scale": 7.5
      }
    },
    "upscale_image": {
      "activity": "replicate_prediction",
      "model": "nightmareai/real-esrgan:42fed1c4974146d4d2414e2be2c5277c7fcf05fcc3a73abf41610695738c1d7b",
      "input": {
        "image": "{{generate_image.outputs.image_url}}"
      }
    }
  }
}

Data Processing Integrations

Pandas Data Processing

Comprehensive data manipulation and analysis using pandas DataFrames.

Available Operations

Data Loading and Validation:

{
  "activity": "pandas_processor",
  "operations": ["load_csv", "validate_schema", "clean_nulls"],
  "input_file": "{{init_params.data_file}}",
  "validation_rules": {
    "required_columns": ["id", "timestamp", "value"],
    "data_types": { "id": "int64", "value": "float64" }
  }
}

Data Transformation:

{
  "activity": "pandas_transformer",
  "operations": [
    "normalize_columns",
    "calculate_rolling_averages",
    "detect_outliers",
    "engineer_features"
  ],
  "parameters": {
    "rolling_window": 7,
    "outlier_threshold": 3.0,
    "normalization_method": "z_score"
  }
}

Statistical Analysis:

{
  "activity": "pandas_analyzer",
  "analysis_types": [
    "descriptive_statistics",
    "correlation_analysis",
    "trend_analysis",
    "distribution_analysis"
  ],
  "output_format": "json"
}

File Processing Tools

Document Processing:

{
  "activity": "document_processor",
  "input_files": "{{init_params.documents}}",
  "operations": ["extract_text", "parse_metadata", "split_pages"],
  "supported_formats": ["pdf", "docx", "txt", "md"]
}

Image Processing:

{
  "activity": "image_processor",
  "input_image": "{{previous_step.outputs.image_path}}",
  "operations": ["resize", "enhance", "extract_metadata"],
  "parameters": {
    "resize_dimensions": [1024, 768],
    "enhancement_level": "auto"
  }
}

Evaluation and Testing Integrations

Verdict LLM-as-Judge Framework

Advanced evaluation system for AI model outputs and workflow results.

Evaluation Types

Categorical Evaluation:

{
  "activity": "verdict_judge",
  "items": ["{{step1.outputs.text}}", "{{step2.outputs.text}}"],
  "prompt": "Classify the quality of this response",
  "judge_type": "categorical",
  "categories": ["excellent", "good", "fair", "poor"],
  "model": "gpt-4",
  "with_explanation": true
}

Scale-Based Evaluation:

{
  "activity": "verdict_judge",
  "item_path": "analysis_step.outputs.result",
  "prompt": "Rate the accuracy of this analysis on a scale of 1-10",
  "judge_type": "scale",
  "scale_range": [1, 10],
  "with_explanation": true,
  "model": "claude-3-sonnet-20240229"
}

Judge-Then-Verify Pipeline:

{
  "activity": "verdict_pipeline",
  "items": "{{responses_to_evaluate}}",
  "pipeline_type": "judge_verify",
  "judge_prompt": "Evaluate the technical accuracy of this response",
  "verify_prompt": "Review the previous judgment for accuracy and completeness",
  "categories": ["accurate", "mostly_accurate", "inaccurate"],
  "model": "gpt-4"
}

AgentBench Evaluation

Specialized framework for evaluating AI agent capabilities.

{
  "activity": "agentbench_evaluator",
  "agent_responses": "{{agent_step.outputs.responses}}",
  "test_suite": "coding_tasks",
  "evaluation_criteria": [
    "correctness",
    "efficiency",
    "code_quality",
    "problem_solving"
  ],
  "difficulty_levels": ["easy", "medium", "hard"]
}

Harbor Terminal Bench Evaluation

{
  "activity": "harbor_terminal_bench",
  "agent_path": "init_params.agent",
  "model_path": "init_params.model",
  "dataset_path": "init_params.dataset",
  "task_name_path": "init_params.task_name"
}

Secrets and Configuration Management

Multi-Layer Secrets Management

Jetty implements a hierarchical approach to secrets management:

1. Direct Configuration (highest priority)
2. Secrets Manager (organization-scoped)
3. Environment Variables (fallback)

Secrets Manager Integration

AWS Secrets Manager:

# Secrets are automatically scoped by organization
secret_name = f"org-{org_id}-OPENAI_API_KEY"

Configuration Pattern:

Environment Variable Patterns

Development Environment (.env file):

# Workflow Engine Configuration
WORKFLOW_ENGINE_URL=localhost:7233
TASK_QUEUE_NAME=mise-tasks

# Storage Configuration
STORAGE_TYPE=local
STORAGE_PATH=./local-storage

# AI Model API Keys
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...
GEMINI_API_KEY=AIza...
REPLICATE_API_TOKEN=r8_...

# Cloud Storage (optional)
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_REGION=us-east-1
STORAGE_BUCKET=my-workflow-bucket

# Application Configuration
PORT=8000
LOG_LEVEL=INFO
MISE_AUTHOR=your-name@company.com

Production Environment:

# Workflow Engine Configuration
WORKFLOW_ENGINE_URL=workflow-engine.production:7233
TASK_QUEUE_NAME=mise-production-tasks

# Storage Configuration
STORAGE_TYPE=s3
STORAGE_BUCKET=production-workflows
AWS_REGION=us-east-1

# Secrets (use secrets manager in production)
# API keys should be stored in AWS Secrets Manager
# and referenced via api_key_secret in configurations

# Application Configuration
PORT=8080
LOG_LEVEL=WARN
ENABLE_METRICS=true

Development and Testing Configuration

Local Development Setup

Docker Compose Configuration:

version: '3.8'
services:
  mise-api:
    build: .
    ports:
      - '8000:8000'
    environment:
      - WORKFLOW_ENGINE_URL=workflow-engine:7233
      - STORAGE_TYPE=local
      - STORAGE_PATH=/app/storage
    volumes:
      - ./storage:/app/storage
      - ./.env:/app/.env
    depends_on:
      - workflow-engine
      - database

  workflow-engine:
    # Durable execution engine
    ports:
      - '7233:7233'
      - '8080:8080' # Web UI
    depends_on:
      - database

  database:
    # Relational database for workflow state
    volumes:
      - db_data:/var/lib/data

volumes:
  db_data:

Testing Configuration

Unit Test Environment:

# conftest.py
import pytest
import tempfile
import os
from mise.storage import factory

@pytest.fixture
def test_storage():
    """Provide isolated storage for tests."""
    with tempfile.TemporaryDirectory() as temp_dir:
        os.environ["STORAGE_TYPE"] = "local"
        os.environ["STORAGE_PATH"] = temp_dir
        yield factory.get_storage_provider()

@pytest.fixture
def mock_api_keys():
    """Provide mock API keys for testing."""
    test_keys = {
        "OPENAI_API_KEY": "sk-test-key-123",
        "ANTHROPIC_API_KEY": "sk-ant-test-key-123",
        "GEMINI_API_KEY": "test-gemini-key-123"
    }

    for key, value in test_keys.items():
        os.environ[key] = value

    yield test_keys

    for key in test_keys:
        os.environ.pop(key, None)

Integration Test Workflows:

Performance and Optimization

Caching Strategies

Model Response Caching:

Data Processing Optimization:

Resource Management

Memory-Efficient Processing:

Rate Limiting and Throttling:

{
  "step_configs": {
    "api_calls": {
      "activity": "litellm_chat",
      "model": "gpt-4",
      "rate_limit": {
        "requests_per_minute": 60,
        "tokens_per_minute": 40000
      },
      "retry_policy": {
        "max_attempts": 3,
        "backoff_multiplier": 2
      }
    }
  }
}

Security Best Practices

API Key Security

1. Never hardcode API keys in workflow configurations
2. Use secrets management for production deployments
3. Rotate keys regularly and update secrets manager
4. Monitor API usage for unusual patterns
5. Use least-privilege access for cloud storage and services

Data Protection

Encryption at Rest:

{
  "step_configs": {
    "secure_processing": {
      "activity": "data_processor",
      "encryption_enabled": true,
      "encryption_key_secret": "DATA_ENCRYPTION_KEY",
      "secure_temporary_storage": true
    }
  }
}

Data Anonymization:

{
  "step_configs": {
    "anonymize_data": {
      "activity": "data_anonymizer",
      "pii_fields": ["email", "phone", "ssn"],
      "anonymization_method": "hashing",
      "preserve_format": true
    }
  }
}

Monitoring and Observability

Integration Metrics

API Usage Tracking:

# Automatically tracked metrics
- Total API calls per provider
- Token usage per model
- Error rates by integration
- Response times and latency
- Cost tracking per organization

Custom Metrics Collection:

{
  "step_configs": {
    "monitored_analysis": {
      "activity": "litellm_chat",
      "model": "gpt-4",
      "metrics": {
        "track_tokens": true,
        "track_cost": true,
        "track_latency": true,
        "custom_labels": {
          "use_case": "content_analysis",
          "department": "marketing"
        }
      }
    }
  }
}

Integration Health Checks

{
  "name": "monitoring/integration-health-check",
  "description": "Verify all integrations are working correctly",
  "steps": ["check_openai", "check_anthropic", "check_gemini", "check_storage"],
  "step_configs": {
    "check_openai": {
      "activity": "litellm_chat",
      "model": "gpt-3.5-turbo",
      "prompt": "Hello, this is a health check",
      "max_tokens": 10,
      "timeout": 30
    },
    "check_storage": {
      "activity": "storage_health_check",
      "operations": ["write", "read", "delete"],
      "test_file_size": "1KB"
    }
  }
}

Troubleshooting Common Issues

API Key Issues

Problem: API key not found error

# Check environment variables
echo $OPENAI_API_KEY

# Verify secrets manager access
aws secretsmanager get-secret-value --secret-id org-myorg-OPENAI_API_KEY

# Test API key validity
curl -H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models

Model Access Issues

Problem: Model not available or rate limited

{
  "step_configs": {
    "robust_analysis": {
      "activity": "litellm_chat",
      "model": "gpt-4",
      "fallback_models": ["gpt-3.5-turbo", "claude-3-haiku-20240307"],
      "retry_policy": {
        "max_attempts": 3,
        "backoff_strategy": "exponential"
      }
    }
  }
}

Storage Configuration Issues

Problem: Storage backend not accessible

# Test storage connectivity
from mise.storage import factory

# Test local storage
config = factory.StorageConfig(storage_type="local", storage_path="/tmp/test")
storage = factory.create_storage_provider(config)

# Test write/read operations
await storage.write_async("test.txt", b"Hello World")
data = await storage.read_async("test.txt")
print(data.decode())

Next Steps

• Explore the Step Library to understand how integrations work within Jetty workflows
• Check out Guides to see integration patterns in complete workflows
• Learn about Architecture to understand the technical implementation of integrations

---

Ready to build workflows with these integrations? Explore the Step Library to start creating your first integrated workflow.