Samuel Liedtke

Using Anthropic Claude with llm via AWS Bedrock

July 3, 2024 • Anthropic, llm, AWS

Unfortunately, the Claude API is not available to individuals. Here is how to access Claude 3 Sonnet and Haiku with `llm` via AWS Bedrock and the `llm-bedrock-anthropic` plugin.

Until very recently, my default model for using the llm CLI has been GPT-3.5. The main reason being that it's cheap and usually gave good enough results.

When Anthropic recently released their impressive new model Clauce 3.5 Sonnet, I was curious to try it out with llm. I also learned that apparently, their less capable model Claude Haiku is both better and cheaper than GPT-3.5.

To use the llm-claude-3 plugin, I needed access to the Claude API. Unfortunately, the Claude API is currently not available for individual use. (You need an organisation and a tax ID in order to sign up for the Build plan.)

After presenting my recent blog post in a lightning talk on the Django User Group meeting in Berlin last week, one of the attendees pointed out to me that I can also use the Claude models via AWS Bedrock.

Luckily, there already is a plugin for llm to use the Anthropic models via the Bedrock API.

llm install llm-bedrock-anthropic

Using it requires to set up an AWS profile locally via the aws CLI, see instruction below.

  1. Install AWS CLI using the Mac installer
aws --version
# aws-cli/2.17.5 Python/3.11.8 Darwin/22.6.0 exe/x86_64
  1. Create new user in AWS IAM
  2. Attach a security policy

Here are the permissions that I used:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream",
                "bedrock:ListFoundationModels",
                "bedrock:GetFoundationModel"
            ],
            "Resource": "*"
        }
    ]
}
  1. Create an access key for user
  2. Configure profile locally
aws configure --profile bedrock-api
# AWS Access Key ID [****************]: 
# AWS Secret Access Key [****************: 
# Default region name [us-east-1]: us-east-1
# Default output format [json]:
  1. Set default region and profile (e.g. in .zshrc)
export AWS_DEFAULT_REGION=us-east-1
export AWS_PROFILE=bedrock-api
  1. Request access to the models from the AWS Bedrock console.

Now it should be ready to use:

# list available models
aws bedrock list-foundation-models

# use claude haiku (can also use alias `bh`)
llm -m anthropic.claude-3-haiku-20240307-v1:0 -s \
  "ten names for a pet octopus"

Unfortunately, the latest model Claude 3.5 Sonnet was not yet available for me to request. Hopefully it will be available soon.

In the meantime, I can already use Claude Haiku instead of GPT-3.5. Nice!

cat accessing-anthropic-claude-via-api-individual.md | \
  llm -m bh -s "write a haiku about this blogpost"
# Anthropic Claude shines 
# AWS Bedrock unlocks access 
# LLM power grows

-- Update July 11, 2024 --

Today, I was able to request access to Claude 3.5 Sonnet on Amazon Bedrock! Also, llm-bedrock-anthropic added support for it as well. I think I'll be using it a lot!

Recent posts