Replicate Integration

Replicate steps provide access to specialized AI models hosted on Replicate's platform, including cutting-edge image generation, video generation, custom models, and community-contributed solutions. Perfect for accessing the latest AI research models and specialized use cases.

Available Steps (8)

`replicate_text2image`

Generate high-quality images from text prompts using state-of-the-art models.

Activity Name: replicate_text2image

Use Cases: Marketing visuals, creative content, product mockups, artistic generation

`replicate_text2video`

Generate videos from text prompts or input images using video generation models.

Activity Name: replicate_text2video

Use Cases: Social media content, product demos, animated explanations, creative video

`replicate_text_stream`

Stream text generation from hosted language models with real-time output.

Activity Name: replicate_text_stream

Use Cases: Interactive chat, real-time content generation, streaming responses

`replicate_extract_embeddings_url`

Extract image embeddings using CLIP and other vision models for similarity analysis.

Activity Name: replicate_extract_embeddings_url

Use Cases: Image search, visual similarity, content-based recommendation

`replicate_segment`

Perform image segmentation and object detection with specialized models.

Activity Name: replicate_segment

Use Cases: Object detection, image analysis, automated masking, content extraction

`replicate_brand_compliance`

Check images against brand guidelines using custom compliance models.

Activity Name: replicate_brand_compliance

Use Cases: Brand monitoring, content moderation, marketing asset validation

`replicate_modify_image`

Transform and edit images using AI-powered modification models.

Activity Name: replicate_modify_image

Use Cases: Image editing, style transfer, content modification, automated retouching

`replicate_multi_image_modify_image`

Transform images using multiple input images as context.

Activity Name: replicate_multi_image_modify_image

Use Cases: Style mixing, image combination, multi-reference generation

Configuration

Authentication

All Replicate steps use unified API token management:

{
  "api_token_secret": "REPLICATE_API_TOKEN"
}

Authentication Patterns:

Direct: "api_token": "r8_..."
Secrets Manager: "api_token_secret": "REPLICATE_API_TOKEN" (recommended)
Environment: Falls back to REPLICATE_API_TOKEN environment variable

Model Selection

Replicate uses owner/model naming convention:

{
  "model": "black-forest-labs/flux-schnell"
}

Popular Models:

Image Generation: black-forest-labs/flux-schnell, stability-ai/sdxl
Text Generation: anthropic/claude-3.5-haiku, meta/llama-2-70b-chat
Image Analysis: salesforce/clip-vit-large-patch14

Step Documentation

replicate_text2image

Generate high-quality images from text prompts using advanced diffusion models.

Configuration

{
  "activity": "replicate_text2image",
  "model": "black-forest-labs/flux-schnell",
  "prompt": "A futuristic cityscape at sunset"
}

Parameters

prompt (string, required) - Text description of desired image
model (string, default: black-forest-labs/flux-schnell) - Image generation model
go_fast (boolean, default: true) - Enable fast generation mode
num_outputs (int, default: 1) - Number of images to generate
aspect_ratio (string, default: 1:1) - Image aspect ratio
output_format (string, default: webp) - Image format (webp, jpg, png)
num_inference_steps (int, default: 4) - Generation quality steps

Advanced Parameters

guidance_scale (float, default: 3.0) - Prompt adherence strength
prompt_strength (float, default: 0.8) - Prompt influence level
megapixels (string, default: 1) - Output resolution setting

Example

{
  "name": "create_hero_image",
  "activity": "replicate_text2image",
  "config": {
    "model": "black-forest-labs/flux-schnell",
    "prompt": "Professional product photography of wireless earbuds on clean white background, studio lighting",
    "aspect_ratio": "16:9",
    "output_format": "jpg",
    "guidance_scale": 3.5,
    "num_inference_steps": 8
  }
}

replicate_text2video

Generate videos from text prompts using video generation models like Seedance.

Configuration

{
  "activity": "replicate_text2video",
  "model": "bytedance/seedance-1-pro",
  "prompt": "A cat walking through a garden"
}

Parameters

prompt (string, required) - Text description of desired video
model (string, default: bytedance/seedance-1-pro) - Video generation model
duration (int, default: 5) - Video duration in seconds
resolution (string, default: 1080p) - Output resolution
aspect_ratio (string, default: 16:9) - Video aspect ratio
fps (int, default: 24) - Frames per second
camera_fixed (bool, default: false) - Lock camera position
output_format (string, default: mp4) - Output video format
seed (int, optional) - Random seed for reproducibility
image_path (string, optional) - Starting image for image-to-video generation

Example: Text-to-Video

{
  "name": "create_product_video",
  "activity": "replicate_text2video",
  "config": {
    "model": "bytedance/seedance-1-pro",
    "prompt": "A sleek smartphone rotating slowly on a white background with soft studio lighting",
    "duration": 5,
    "resolution": "1080p",
    "aspect_ratio": "16:9",
    "fps": 24
  }
}

Example: Image-to-Video

{
  "name": "animate_image",
  "activity": "replicate_text2video",
  "config": {
    "model": "bytedance/seedance-1-pro",
    "prompt": "The character starts walking forward with natural movement",
    "image_path": "generate_image.outputs.images[0].path",
    "duration": 3,
    "camera_fixed": true
  }
}

Output

{
  "outputs": {
    "videos": [
      {
        "path": "collection/flow/0001/video_0.mp4",
        "content_type": "video/mp4",
        "extension": "mp4"
      }
    ]
  }
}

replicate_text_stream

Stream text generation with real-time output from hosted language models.

Configuration

{
  "activity": "replicate_text_stream",
  "model": "anthropic/claude-3.5-haiku",
  "prompt": "Explain quantum computing"
}

Parameters

prompt (string, required) - Input text prompt
model (string, default: anthropic/claude-3.5-haiku) - Language model
max_tokens (int, default: 8192) - Maximum response length
temperature (float, default: 1.0) - Response randomness
system_prompt (string) - System instruction for model behavior
top_p (float, default: 1.0) - Nucleus sampling parameter
top_k (int, default: 50) - Top-k sampling parameter

Example

{
  "name": "generate_article",
  "activity": "replicate_text_stream",
  "config": {
    "model": "meta/llama-2-70b-chat",
    "prompt": "Write a comprehensive guide about sustainable energy solutions",
    "max_tokens": 4000,
    "temperature": 0.7,
    "system_prompt": "You are an expert in environmental science and renewable energy."
  }
}

replicate_extract_embeddings_url

Extract visual embeddings from images using CLIP and other vision models.

Configuration

{
  "activity": "replicate_extract_embeddings_url",
  "model": "krthr/clip-embeddings",
  "image_path": "previous_step.outputs.images[0].path"
}

Parameters

image_path (string, required) - Path to image file
model (string, default: CLIP model) - Embedding extraction model

Output

embeddings (array) - Numerical vector representation of image content

Example

{
  "name": "extract_features",
  "activity": "replicate_extract_embeddings_url",
  "config": {
    "image_path": "uploaded_image.outputs.image_path",
    "model": "krthr/clip-embeddings:1c0371070cb827ec3c7f2f28adcdde54b50dcd239aa6faea0bc98b174ef03fb4"
  }
}

replicate_segment

Perform image segmentation and object detection using specialized models.

Configuration

{
  "activity": "replicate_segment",
  "model": "segmentation_model",
  "image_path": "previous_step.outputs.images[0].path"
}

Parameters

image_path (string) - Path to input image
model (string, required) - Segmentation model identifier
output_json (boolean, default: true) - Include JSON metadata

Output

images (array) - Segmented image results
json (array) - Segmentation metadata and coordinates

Example

{
  "name": "segment_objects",
  "activity": "replicate_segment",
  "config": {
    "image_path": "product_photo.outputs.image_path",
    "model": "segment-anything-model",
    "output_json": true
  }
}

replicate_brand_compliance

Analyze images for brand guideline compliance using custom models.

Configuration

{
  "activity": "replicate_brand_compliance",
  "model": "brand_compliance_model",
  "images_path": "previous_step.outputs.images"
}

Parameters

images_path (string) - Path to images for analysis
model (string, required) - Brand compliance model
input_params (array) - Additional model-specific parameters

Output

prediction_* (mixed) - Compliance analysis results per image
embeddings_* (array) - Feature embeddings for each image

Example

{
  "name": "check_brand_compliance",
  "activity": "replicate_brand_compliance",
  "config": {
    "images_path": "marketing_assets.outputs.images",
    "model": "custom-brand-compliance-model",
    "input_params": ["brand_guidelines", "color_palette"]
  }
}

replicate_modify_image

Transform and edit images using AI-powered modification models.

Configuration

{
  "activity": "replicate_modify_image",
  "model": "black-forest-labs/flux-kontext-pro",
  "image_path": "previous_step.outputs.images[0].path",
  "prompt": "Add sunset lighting to this scene"
}

Parameters

image_path (string, required) - Path to input image
prompt (string, required) - Modification instruction
model (string, default: black-forest-labs/flux-kontext-pro) - Modification model
aspect_ratio (string, default: match_input_image) - Output aspect ratio
output_format (string, default: jpg) - Output image format
safety_tolerance (int, default: 2) - Content safety level
prompt_upsampling (boolean, default: false) - Enhance prompt quality

Example

{
  "name": "enhance_product_image",
  "activity": "replicate_modify_image",
  "config": {
    "image_path": "original_photo.outputs.images[0].path",
    "prompt": "Remove background and add professional studio lighting",
    "model": "black-forest-labs/flux-kontext-pro",
    "output_format": "png",
    "safety_tolerance": 3
  }
}

Advanced Workflows

Image Generation Pipeline

{
  "steps": [
    {
      "name": "generate_base_image",
      "activity": "replicate_text2image",
      "config": {
        "model": "black-forest-labs/flux-schnell",
        "prompt": "Modern office workspace, clean and professional"
      }
    },
    {
      "name": "modify_lighting",
      "activity": "replicate_modify_image",
      "config": {
        "image_path": "generate_base_image.outputs.images[0].path",
        "prompt": "Add warm, natural lighting from large windows"
      }
    },
    {
      "name": "extract_features",
      "activity": "replicate_extract_embeddings_url",
      "config": {
        "image_path": "modify_lighting.outputs.images[1].path"
      }
    }
  ]
}

Content Analysis Workflow

{
  "steps": [
    {
      "name": "segment_image",
      "activity": "replicate_segment",
      "config": {
        "image_path": "uploaded_content.outputs.image_path",
        "model": "segment-anything-model"
      }
    },
    {
      "name": "check_compliance",
      "activity": "replicate_brand_compliance",
      "config": {
        "images_path": "segment_image.outputs.images",
        "model": "brand-guidelines-checker"
      }
    }
  ]
}

{
  "steps": [
    {
      "name": "generate_description",
      "activity": "replicate_text_stream",
      "config": {
        "model": "meta/llama-2-70b-chat",
        "prompt": "Describe a futuristic vehicle design in detail"
      }
    },
    {
      "name": "create_visualization",
      "activity": "replicate_text2image",
      "config": {
        "prompt": "{{generate_description.outputs.text}}",
        "model": "black-forest-labs/flux-schnell"
      }
    }
  ]
}

Error Handling

Common Issues

Authentication Error: Verify REPLICATE_API_TOKEN configuration
Model Not Found: Check model owner/name format and availability
Input Format Error: Ensure image paths and formats are supported
Rate Limiting: Replicate has generous limits, but implement retry logic

Best Practices

Use appropriate models for specific tasks (Flux for quality, SDXL for speed)
Monitor model availability as community models may be deprecated
Cache expensive operations like embeddings extraction
Implement fallback models for critical workflows

Performance Tips

Model Selection

Speed Priority: flux-schnell, lightweight CLIP models
Quality Priority: flux-pro, high-parameter models
Cost Optimization: Compare pricing across similar models

Batch Operations

Process multiple images in sequence using step chaining
Use appropriate batch_size for bulk operations
Consider model warm-up time for first requests

Resource Management

Monitor GPU usage for compute-intensive models
Use streaming for long text generation tasks
Cache model outputs when possible

Model Categories

Image Generation

Flux Models: Latest high-quality generation (flux-schnell, flux-pro)
Stable Diffusion: Various SDXL and SD variants
Specialized: Style-specific and fine-tuned models

Text Generation

LLaMA: Meta's open source models (llama-2-70b-chat)
Claude: Anthropic models hosted on Replicate
Specialized: Code generation, creative writing models

Computer Vision

CLIP: Image-text understanding and embeddings
Segmentation: SAM (Segment Anything Model) variants
Detection: Object detection and classification models

LiteLLM Multi-Provider - Alternative for standard models with fallback support
Gemini Integration - Google's native models with file processing
Step Library Overview - Complete step documentation and usage patterns

Integration Examples

View complete workflow examples in the Flow Library:

Creative content generation pipelines
Image analysis and processing workflows
Multi-modal AI interactions
Custom model integration patterns

Available Steps (8)​

replicate_text2image​

replicate_text2video​

replicate_text_stream​

replicate_extract_embeddings_url​

replicate_segment​

replicate_brand_compliance​

replicate_modify_image​

replicate_multi_image_modify_image​

Configuration​

Authentication​

Model Selection​

Step Documentation​

replicate_text2image​

Configuration​

Parameters​

Advanced Parameters​

Example​

replicate_text2video​

Configuration​

Parameters​

Example: Text-to-Video​

Example: Image-to-Video​

Output​

replicate_text_stream​

Configuration​

Parameters​

Example​

replicate_extract_embeddings_url​

Configuration​

Parameters​

Output​

Example​

replicate_segment​

Configuration​

Parameters​

Output​

Example​

replicate_brand_compliance​

Configuration​

Parameters​

Output​

Example​

replicate_modify_image​

Configuration​

Parameters​

Example​

Advanced Workflows​

Image Generation Pipeline​

Content Analysis Workflow​

Multi-Modal Generation​

Error Handling​

Common Issues​

Best Practices​

Performance Tips​

Model Selection​

Batch Operations​

Resource Management​

Model Categories​

Image Generation​

Text Generation​

Computer Vision​

Related Steps​

Integration Examples​

Available Steps (8)

`replicate_text2image`

`replicate_text2video`

`replicate_text_stream`

`replicate_extract_embeddings_url`

`replicate_segment`

`replicate_brand_compliance`

`replicate_modify_image`

`replicate_multi_image_modify_image`

Configuration

Authentication

Model Selection

Step Documentation

replicate_text2image

Configuration

Parameters

Advanced Parameters

Example

replicate_text2video

Configuration

Parameters

Example: Text-to-Video

Example: Image-to-Video

Output

replicate_text_stream

Configuration

Parameters

Example

replicate_extract_embeddings_url

Configuration

Parameters

Output

Example

replicate_segment

Configuration

Parameters

Output

Example

replicate_brand_compliance

Configuration

Parameters

Output

Example

replicate_modify_image

Configuration

Parameters

Example

Advanced Workflows

Image Generation Pipeline

Content Analysis Workflow

Multi-Modal Generation

Error Handling

Common Issues

Best Practices

Performance Tips

Model Selection

Batch Operations

Resource Management

Model Categories

Image Generation

Text Generation

Computer Vision

Related Steps

Integration Examples