Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'Anthropic' object has no attribute 'count_tokens' #314

Open
moodys-onkar opened this issue Dec 23, 2024 · 3 comments
Open

Comments

@moodys-onkar
Copy link

moodys-onkar commented Dec 23, 2024

Getting this error when using anthropic models from bedrock.py module of langchain.

The error comes from this module.

Screenshot 2024-12-23 at 1 35 50 PM

Looking at the anthropic code, the api for count_tokens has been changed and it needs to be updated in the utils.py for bedrock.
import anthropic

client = anthropic.Anthropic()

response = client.messages.count_tokens(
model="claude-3-5-sonnet-20241022",
system="You are a scientist",
messages=[{
"role": "user",
"content": "Hello, Claude"
}],
)

print(response.json())

@langcarl langcarl bot added the investigate label Dec 23, 2024
@ming1in
Copy link

ming1in commented Jan 2, 2025

is there a version of the anthropic py lib that is compatible with langchain without this issue?

@michaelnchin
Copy link
Contributor

michaelnchin commented Jan 4, 2025

The count_tokens() and get_tokenizer() were removed/changed in anthropic v0.39.0: https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md?plain=1#L99

I tried downgrading to v0.38.0, but anthropic threw the same error noted in #297, which requires the package to be upgraded for resolution. So we probably do need to update the code to use the new client.messages.count_tokens API.

@michaelnchin
Copy link
Contributor

Took a look into fixing this, but there isn't a clear-cut resolution path at the moment.

The built-in token counting functionality in Anthropic Python SDK has been removed entirely in favor of an server-side implementation in the Messages API. This means that users are now required to provide a separate Anthropic API key. And while the count_tokens API itself is stated free to use, any requests to it will still fail with a BadRequestError if the Anthropic account does not have sufficient funds.

A possible future solution may lie with the Anthropic SDK's Bedrock client, which is able to interface with the base Messages API using only AWS credentials. However, Bedrock does not yet support the count_tokens API: https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/lib/bedrock/_client.py#L62

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants