Skip to main content

Anyscale

Anyscale is a platform to run, fine tune and scale LLMs via production-ready APIs. Anyscale Endpoints serve many open-source models in a cost-effective way.

Anyscale also provides an example how to setup LangChain with Anyscale for advanced chat agents.

Installation and Setup​

  • Get an Anyscale Service URL, route and API key and set them as environment variables (ANYSCALE_SERVICE_URL,ANYSCALE_SERVICE_ROUTE, ANYSCALE_SERVICE_TOKEN).
  • Please see the Anyscale docs for more details.

We have to install the openai package:

pip install openai

LLM​

See a usage example.

from langchain_community.llms.anyscale import Anyscale
API Reference:Anyscale

Chat Models​

See a usage example.

from langchain_community.chat_models.anyscale import ChatAnyscale
API Reference:ChatAnyscale

Embeddings​

See a usage example.

from langchain_community.embeddings import AnyscaleEmbeddings
API Reference:AnyscaleEmbeddings

Was this page helpful?