Skip to content

AWS Bedrock

What it is

AWS Bedrock is a fully managed service from Amazon Web Services that makes foundational models (FMs) available through an API. It provides a single interface to access models from leading AI providers including Amazon, Anthropic, AI21 Labs, Cohere, Meta, Mistral AI, and Stability AI. In July 2026, it is a primary enterprise gateway for deploying models like Claude 4.8 Opus, Gemma 3, and Llama 4 Maverick, now featuring native support for the NVIDIA Rubin architecture and MCP 3.0 tool integration.

What problem it solves

It simplifies the process of building and scaling generative AI applications by removing the need to manage underlying infrastructure. It provides a unified API for multiple models, along with tools for fine-tuning, RAG (Knowledge Bases for Amazon Bedrock), and agentic workflows (Agents for Amazon Bedrock). It addresses enterprise concerns regarding data privacy, security, and high-performance execution on next-generation hardware.

Where it fits in the stack

Provider / Infrastructure. It serves as an enterprise-grade gateway and orchestration layer for high-performance LLMs, often paired with Docker for specialized containerized deployments.

Typical use cases

  • Enterprise AI Applications: Building secure, scalable AI solutions within the AWS ecosystem.
  • Retrieval-Augmented Generation (RAG): Using "Knowledge Bases for Amazon Bedrock" to connect models to proprietary S3-hosted data.
  • Agentic Workflows: Deploying autonomous agents that leverage the MCP 3.0 Task Protocol to execute multi-step tasks across AWS resources.
  • Hardware-Accelerated Inference: Utilizing NVIDIA Rubin GPUs for ultra-low latency inference of frontier models.

Strengths

  • Enterprise-Grade Security: Strong data privacy and compliance features (HIPAA, GDPR, etc.). Data is not used to train the underlying foundation models.
  • Model Variety: Access to Gemma 3, Claude 4.8, and Llama 4 through a single API.
  • NVIDIA Rubin Support: Optimized for the latest GPU architectures to provide superior price-performance.
  • AWS Integration: Seamless integration with S3, Lambda, IAM, and MCP 3.0 servers.
  • Managed RAG: Built-in support for automated vectorization and retrieval via Knowledge Bases.

Limitations

  • AWS Ecosystem Lock-in: Deeply tied to AWS; moving to another provider requires significant re-engineering (unless using LiteLLM).
  • Configuration Complexity: AWS's extensive IAM and VPC requirements can be daunting for smaller teams.
  • Regional Availability: Newest models (e.g., Gemma 3) and Rubin-based instances may not be available in all regions simultaneously.
  • API Latency: Managed service overhead can be slightly higher than direct-to-metal self-hosting via vLLM.

When to use it

  • When building enterprise-scale AI applications requiring high security, compliance, and AWS-native scalability.
  • If your organization is already standardized on the AWS ecosystem.
  • When you need a managed RAG or agent framework that integrates natively with cloud resources via MCP 3.0.
  • For multi-model applications requiring a unified billing and security model.

When not to use it

  • For simple, low-volume projects where a direct API (OpenAI/Anthropic) is faster to implement.
  • If you require a provider-agnostic solution (consider LiteLLM).
  • When you need the absolute lowest latency possible for real-time applications (consider vLLM on EC2).

Getting started

1. Prerequisites

  • An AWS account with Bedrock model access enabled.
  • AWS CLI configured with appropriate credentials.
  • Python 3.9+ and boto3.

2. Installation

pip install boto3

3. Hello-world task (Python)

import boto3
import json

bedrock = boto3.client(service_name='bedrock-runtime', region_name='us-east-1')

# Note: Model IDs follow the July 2026 technical context.
prompt = "Explain the benefit of MCP 3.0 in one sentence."
body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 100,
    "messages": [{"role": "user", "content": prompt}]
})

response = bedrock.invoke_model(
    body=body,
    modelId='anthropic.claude-4-8-opus-20260528-v1:0'
)

response_body = json.loads(response.get('body').read())
print(response_body['content'][0]['text'])

CLI examples

Commonly used commands for inspecting model availability and performing quick tests.

# List available foundation models (including Gemma 3)
aws bedrock list-foundation-models --region us-east-1

# Get details for a specific model
aws bedrock get-foundation-model --model-identifier google.gemma-3-27b-it-v1:0

# Invoke a model via CLI and save output
aws bedrock-runtime invoke-model \
  --model-id anthropic.claude-4-8-opus-20260528-v1:0 \
  --body '{"anthropic_version": "bedrock-2023-05-31", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello Bedrock!"}]}' \
  output.txt

# List Knowledge Bases for RAG
aws bedrock-agent list-knowledge-bases

API examples

Using the boto3 SDK for streaming responses and MCP 3.0 tool integration.

Streaming Response

import boto3
import json

client = boto3.client(service_name='bedrock-runtime')

def stream_response(prompt):
    body = json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 512,
        "messages": [{"role": "user", "content": prompt}]
    })

    response = client.invoke_model_with_response_stream(
        modelId='anthropic.claude-4-8-opus-20260528-v1:0',
        body=body
    )

    for event in response.get('body'):
        chunk = json.loads(event.get('chunk').get('bytes'))
        if chunk['type'] == 'content_block_delta':
            print(chunk['delta']['text'], end='', flush=True)

stream_response("Write a short poem about the NVIDIA Rubin architecture.")

Knowledge Base Retrieval

import boto3

agent_client = boto3.client(service_name='bedrock-agent-runtime')

def retrieve_from_kb(kb_id, query):
    response = agent_client.retrieve(
        knowledgeBaseId=kb_id,
        retrievalQuery={'text': query}
    )
    return response['results']

Sources / references

Contribution Metadata

  • Last reviewed: 2026-07-21
  • Confidence: high