bedrock-proxy-endpoint

Spin up your own custom OpenAI API server endpoint for easy AWS Bedrock LLM text inference (using standard baseUrl, and apiKey params)

A service that lets you spin up your own custom OpenAI API compatible server endpoint that proxies calls to AWS Bedrock for LLM text inference, using the standard baseUrl and apiKey params. It is aimed at keeping applications platform-agnostic and avoiding the need to format inference calls for the Bedrock SDK or reconcile per-model configuration differences. It is useful for getting existing OpenAI API compatible applications working with AWS Bedrock. The endpoint can be run from source with Node.js, or via Docker using a pre-built image from GitHub Container Registry (GHCR) or a local build, with configuration supplied through environment variables and an optional docker-compose setup.

Installation

  1. Clone the repository:

    git clone https://github.com/jparkerweb/bedrock-proxy-endpoint.git

  2. Navigate to the project directory:

    cd bedrock-proxy-endpoint

  3. Install the dependencies:

    npm ci