Skip to main content

AWS Bedrock

AWS Bedrock is supported as a provider for any LiteLLM-backed activity (litellm_chat, litellm_vision, litellm_embeddings, …), the OpenAI-compatible passthrough at /v1/chat/completions, and the runbook activity. This page shows how to configure credentials on a Jetty collection and how to route requests to Bedrock from each surface.

Configure Bedrock credentials​

Bedrock credentials live on the collection as environment variables. Set them once and every workflow run inherits them.

Two authentication styles are supported:

The simplest setup. Create a long-lived Bedrock API key in the AWS console and store the bearer token on the collection along with the region.

TOKEN="$(cat ~/.config/jetty/token)"
cat <<'BODY' | curl -s -X PATCH \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "https://flows-api.jetty.io/api/v1/collections/{COLLECTION}/environment" \
  --data-binary @-
{
  "environment_variables": {
    "AWS_BEARER_TOKEN_BEDROCK": "ABSK...",
    "AWS_REGION": "us-east-1"
  }
}
BODY

LiteLLM picks up AWS_BEARER_TOKEN_BEDROCK automatically when the request is routed to a Bedrock model — no further wiring needed.

IAM access keys​

If you'd rather use IAM credentials, set the standard AWS variables:

{
  "environment_variables": {
    "AWS_ACCESS_KEY_ID": "AKIA...",
    "AWS_SECRET_ACCESS_KEY": "...",
    "AWS_REGION": "us-east-1"
  }
}

Optionally include AWS_SESSION_TOKEN for temporary STS credentials.

Verify what's configured​

TOKEN="$(cat ~/.config/jetty/token)"
curl -s -H "Authorization: Bearer $TOKEN" \
  "https://flows-api.jetty.io/api/v1/collections/{COLLECTION}/environment" \
  | jq '.environment_variables | keys'

Model identifiers​

Bedrock exposes models as inference profiles. Use the cross-region inference-profile ID, not the bare foundation-model ID, so requests can route across AWS regions automatically.

FamilyInference profile ID
Claude Sonnet 4.6us.anthropic.claude-sonnet-4-6
Claude Opus 4.7us.anthropic.claude-opus-4-7
Claude Haiku 4.5us.anthropic.claude-haiku-4-5
Llama 3.1 70Bus.meta.llama3-1-70b-instruct-v1:0
Mistral Largeus.mistral.mistral-large-2407-v1:0

Use the eu. or apac. prefix for the European and Asia-Pacific inference profile pools; the prefix must match the region in AWS_REGION.

Route a request to Bedrock​

There are three ways to tell Jetty / LiteLLM that a request should hit Bedrock. Pick whichever is most ergonomic — they all produce the same call.

1. model_provider: "bedrock" parameter​

Recommended for litellm_chat and other LiteLLM activities. The model field stays clean and provider-agnostic.

{
  "activity": "litellm_chat",
  "model_path": "init_params.model",
  "model_provider_path": "init_params.model_provider",
  "prompt_path": "init_params.prompt",
  "max_tokens": 256
}
{
  "init_params": {
    "model": "us.anthropic.claude-sonnet-4-6",
    "model_provider": "bedrock",
    "prompt": "Reply with the single word PONG and nothing else."
  }
}

2. bedrock/ model prefix​

LiteLLM convention. Works anywhere a model field is accepted.

{
  "model": "bedrock/us.anthropic.claude-sonnet-4-6"
}

3. Bare inference-profile ID​

If only Bedrock credentials are configured on the collection, a bare us.* / eu.* / apac.* inference-profile ID resolves to Bedrock automatically.

{
  "model": "us.anthropic.claude-sonnet-4-6"
}

Use Bedrock from each surface​

litellm_chat activity​

{
  "init_params": {
    "model": "us.anthropic.claude-sonnet-4-6",
    "model_provider": "bedrock",
    "prompt": "Summarize the attached document in 3 bullets."
  },
  "step_configs": {
    "chat": {
      "activity": "litellm_chat",
      "max_tokens": 512,
      "model_path": "init_params.model",
      "model_provider_path": "init_params.model_provider",
      "prompt_path": "init_params.prompt"
    }
  },
  "steps": ["chat"]
}

Run it:

TOKEN="$(cat ~/.config/jetty/token)"
curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  -F 'init_params={"model":"us.anthropic.claude-sonnet-4-6","model_provider":"bedrock","prompt":"Hello!"}' \
  "https://flows-api.jetty.io/api/v1/run-sync/{COLLECTION}/{TASK}" \
  | jq '.trajectory.steps.chat.outputs.text'

The same model_provider parameter works on every LiteLLM activity (litellm_vision, litellm_embeddings, litellm_function_call, litellm_batch).

Chat Completions passthrough​

The OpenAI-compatible endpoint at /v1/chat/completions accepts Bedrock models with no jetty block — just authenticate against a collection that has Bedrock credentials.

TOKEN="$(cat ~/.config/jetty/token)"
curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  "https://flows-api.jetty.io/v1/chat/completions" \
  -d '{
    "model": "bedrock/us.anthropic.claude-sonnet-4-6",
    "messages": [{"role":"user","content":"Reply with PONG only."}],
    "max_tokens": 32
  }'

Streaming works the same way — set "stream": true and read the SSE response. See Chat Completions API for the full reference.

Runbook activity​

The runbook activity runs an agent (Claude Code, Codex, Gemini CLI) inside a sandbox. To run the agent against Bedrock, set model_provider on the step:

{
  "init_params": {
    "agent": "claude-code",
    "model": "us.anthropic.claude-sonnet-4-6",
    "model_provider": "bedrock",
    "snapshot": "python312-uv",
    "instruction": "..."
  },
  "step_configs": {
    "run": {
      "activity": "runbook",
      "agent_path": "init_params.agent",
      "model_path": "init_params.model",
      "model_provider_path": "init_params.model_provider",
      "snapshot_path": "init_params.snapshot",
      "instruction_path": "init_params.instruction"
    }
  },
  "steps": ["run"]
}

The agent inside the sandbox uses the collection's Bedrock credentials transparently; no extra environment wiring is required inside the runbook itself.

Troubleshooting​

SymptomLikely causeFix
AccessDeniedException with no detailThe model isn't enabled in your AWS account, or your IAM principal lacks bedrock:InvokeModel for itRequest access to the model in the Bedrock console under Model access, then retry
ValidationException: Invocation of model ID … with on-demand throughput isn't supportedYou used a bare foundation-model ID instead of an inference profileSwitch to the us. / eu. / apac. inference-profile ID for that model
An error occurred (ExpiredTokenException)The bearer token (or STS session token) has expiredRotate AWS_BEARER_TOKEN_BEDROCK (or refresh AWS_SESSION_TOKEN) on the collection
Calls succeed against the wrong regionAWS_REGION doesn't match the inference-profile prefixSet AWS_REGION to a region in the same pool (e.g. any us-* region for us.* profiles)
Long synchronous runs return Cloudflare 524Cloudflare 100s edge timeoutThe workflow keeps running server-side — switch to /api/v1/run (async) and poll the trajectory