AI APIs are useful when you want to build applications powered by large language models. The problem is that using multiple AI providers can quickly become complicated.
Each provider gives you a different API key. Each has different models, rate limits, and free-tier limits. If one provider reaches its limit, you may need to manually switch your application to another one.
FreeLLMAPI aims to simplify this.
FreeLLMAPI is an open-source, self-hosted, OpenAI-compatible API proxy that lets you connect API keys from multiple AI providers and access them through one unified API endpoint.
Instead of managing every provider separately inside your application, you can manage them from one place.
GitHub Repository:
https://github.com/tashfeenahmed/freellmapi
In this guide, you will learn what FreeLLMAPI is, how it works, how to install it, how to add your AI provider API keys, and how to use the unified API in your own projects.
What Is FreeLLMAPI?
FreeLLMAPI acts as a layer between your application and different AI providers.
Normally, your application might connect directly to providers such as Gemini, Groq, Mistral, or other LLM APIs.
With FreeLLMAPI, your application connects to a single local endpoint instead.
FreeLLMAPI then decides which configured provider and model should handle the request.
The basic flow looks like this:
Your Application
↓
FreeLLMAPI
↓
Available AI Provider
↓
AI Model
↓
Response
This means your application only needs to communicate with FreeLLMAPI.
You can manage the actual provider API keys separately through the FreeLLMAPI dashboard.
One important thing to understand is that FreeLLMAPI does not magically create unlimited free AI usage.
You still need to obtain API keys from supported providers and follow their individual free-tier limits and terms.
FreeLLMAPI helps you organize and route access to those providers.
What Can FreeLLMAPI Do?
FreeLLMAPI provides several useful features.
One Unified API
You can connect multiple AI providers but access them through a single OpenAI-compatible API endpoint.
This makes integration much easier.
Automatic Failover
If one provider returns certain errors, hits a rate limit, or times out, FreeLLMAPI can attempt to route the request through another available model in your configured fallback chain.
This can make experimentation more reliable when working with limited free tiers.
Smart Routing
You can use the “auto” model option and allow the router to choose an available model.
You can also specify a particular supported model when needed.
Rate-Limit Tracking
FreeLLMAPI tracks usage limits for configured keys and models, including metrics such as requests and tokens.
This helps the router avoid keys that have already reached their configured limits.
Sticky Sessions
Multi-turn conversations can stay connected to the same model for a period of time instead of constantly switching between models.
This helps maintain consistency during longer conversations.
Encrypted API Key Storage
Provider API keys stored by FreeLLMAPI are encrypted at rest using AES-256-GCM.
The keys are decrypted in memory when they are required for a request.
Admin Dashboard
FreeLLMAPI includes a web dashboard.
You can use it to:
Add and manage provider API keys.
Configure the fallback chain.
View analytics.
Check usage.
Test prompts through the playground.
Manage your FreeLLMAPI unified API key.
How to Install FreeLLMAPI
Now let’s set up FreeLLMAPI on your computer.
This guide uses the Node.js installation method.
Step 1: Install the Requirements
Before installing FreeLLMAPI, make sure you have:
Node.js 20 or newer
npm
Git
You can verify Node.js by opening Terminal, Command Prompt, or PowerShell and running:
node –version
Then check npm:
npm –version
And Git:
git –version
If these commands return version numbers, you should be ready to continue.
Step 2: Download FreeLLMAPI
Open Terminal or PowerShell.
Clone the FreeLLMAPI repository:
git clone https://github.com/tashfeenahmed/freellmapi.git
Now enter the project directory:
cd freellmapi
You should now be inside the downloaded FreeLLMAPI project.
Step 3: Install Dependencies
Run:
npm install
npm will download and install the packages required by FreeLLMAPI.
Depending on your internet connection and computer, this may take a few minutes.
Wait until the installation finishes before continuing.
Step 4: Create Your Environment File
FreeLLMAPI uses an environment file for important configuration.
The repository includes an example environment file.
On macOS or Linux, you can copy it using:
cp .env.example .env
On Windows, you can manually duplicate the “.env.example” file and rename the copy to:
.env
The .env file will contain your local configuration.
Step 5: Generate an Encryption Key
FreeLLMAPI requires an ENCRYPTION_KEY.
This key is important because it is used to protect the provider API keys stored by the application.
You can generate a random 32-byte encryption key using Node.js.
Run:
node -e “console.log(require(‘crypto’).randomBytes(32).toString(‘hex’))”
You will receive a long random string.
Copy it.
Now open your .env file and set:
ENCRYPTION_KEY=your_generated_key
Replace “your_generated_key” with the value you generated.
Keep this encryption key private.
Do not upload your .env file or encryption key to a public GitHub repository.
For real provider keys, you should explicitly configure ENCRYPTION_KEY rather than relying on development-mode fallbacks.
Step 6: Start FreeLLMAPI
Once everything is configured, run:
npm run dev
This starts the FreeLLMAPI server and development dashboard.
Keep the terminal window running while using FreeLLMAPI.
Now open your browser and visit:
http://localhost:5173
You should see the FreeLLMAPI dashboard.
The API server runs separately and is available on port 3001.
Your API base URL will normally be:
http://localhost:3001/v1
How to Add Your AI Provider API Keys
Installing FreeLLMAPI does not automatically give you API access to every provider.
You need to obtain API keys from the AI providers you want to use.
Depending on the currently supported provider catalog, you can connect available providers that offer compatible API access and free tiers.
The exact free limits can change over time.
Always check the provider’s current pricing, free-tier limits, and terms before using its API.
Once you have an API key, open the FreeLLMAPI dashboard.
Go to the Keys section.
Select the provider.
Add your API key.
Save it.
Repeat this process for the providers you want to use.
You do not necessarily need to configure every provider.
For example, you could start with two or three providers.
As you add more provider keys, FreeLLMAPI has more available options when routing requests.
Configure the Fallback Chain
One of the useful features of FreeLLMAPI is its fallback system.
Imagine your preferred model reaches its rate limit.
Without a fallback system, your application might simply return an error.
With FreeLLMAPI, another configured model can potentially handle the request.
Inside the dashboard, find the Fallback Chain settings.
You can reorder the available models based on your preferences.
The models near the top of your fallback chain will generally receive higher priority according to the routing configuration.
If the selected provider cannot process the request because of a supported failure condition, the router can move to another available option.
This allows you to combine multiple free tiers while keeping one API interface in your application.
Get Your Unified FreeLLMAPI API Key
Your applications do not need direct access to all your provider API keys.
Instead, FreeLLMAPI gives you a unified API key.
You can find this key from the Keys page in the dashboard.
It will look similar to:
freellmapi-your-unified-key
Copy this key.
You will use it when connecting your applications to your local FreeLLMAPI server.
Your application therefore needs two main pieces of information:
Base URL:
http://localhost:3001/v1
API Key:
freellmapi-your-unified-key
Instead of configuring Gemini, Groq, Mistral, and other providers individually inside every application, you can point compatible clients toward this single API.
How to Use FreeLLMAPI With Python
FreeLLMAPI is compatible with the OpenAI API format.
This means you can use the OpenAI Python SDK while pointing it toward your FreeLLMAPI server.
First, install the OpenAI Python package:
pip install openai
Create a Python file.
For example:
app.py
Add the following code:
from openai import OpenAI
client = OpenAI(
base_url=”http://localhost:3001/v1″,
api_key=”freellmapi-your-unified-key”,
)
response = client.chat.completions.create(
model=”auto”,
messages=[
{
“role”: “user”,
“content”: “Explain artificial intelligence in simple words.”
}
],
)
print(response.choices[0].message.content)
Replace:
freellmapi-your-unified-key
with your actual unified API key.
Now run:
python app.py
Your request will go to your local FreeLLMAPI server.
The router will then select an available configured provider.
The response will be returned using the OpenAI-compatible response format.
What Does model=”auto” Mean?
In the previous example, we used:
model=”auto”
This tells FreeLLMAPI to let its router choose the model.
This can be useful when you care more about getting a successful response than using one specific model.
You can also request a specific supported model.
For example:
model=”gemini-2.5-flash”
The exact available models depend on the current FreeLLMAPI catalog and the providers you have configured.
For beginners, using:
model=”auto”
is usually the easiest place to start.
Using FreeLLMAPI With curl
You can also test your API directly from the command line.
For example:
curl http://localhost:3001/v1/chat/completions
-H “Authorization: Bearer freellmapi-your-unified-key”
-H “Content-Type: application/json”
-d ‘{
“model”: “auto”,
“messages”: [
{
“role”: “user”,
“content”: “Explain machine learning in one sentence.”
}
]
}’
Replace the API key with your actual unified FreeLLMAPI key.
If everything is configured correctly, you should receive a JSON response containing the AI-generated answer.
Using Streaming Responses
FreeLLMAPI also supports streaming chat responses.
Streaming means the response arrives piece by piece instead of waiting for the entire answer to finish.
Here is a Python example:
from openai import OpenAI
client = OpenAI(
base_url=”http://localhost:3001/v1″,
api_key=”freellmapi-your-unified-key”,
)
stream = client.chat.completions.create(
model=”auto”,
messages=[
{
“role”: “user”,
“content”: “Write a short story about artificial intelligence.”
}
],
stream=True,
)
for chunk in stream:
print(
chunk.choices[0].delta.content or “”,
end=””,
flush=True
)
This is useful when building AI chat interfaces where you want text to appear as the model generates it.
Using FreeLLMAPI in Your Existing AI Application
If your application already uses an OpenAI-compatible client, switching to FreeLLMAPI may require only a few configuration changes.
Instead of connecting to the original API endpoint, configure:
Base URL:
http://localhost:3001/v1
Then use your unified FreeLLMAPI API key.
Your application sends requests to FreeLLMAPI.
FreeLLMAPI handles communication with your configured providers.
This makes it useful for:
AI experiments
Personal projects
Local AI applications
Prototypes
Testing different models
Learning how LLM APIs work
Development tools
AI chat applications
Understanding Automatic Failover
Suppose you have configured multiple providers.
Your application sends a request using:
model=”auto”
FreeLLMAPI selects an available option.
If that provider returns a rate-limit error, certain server errors, or times out, the router can skip it temporarily and try another model from your fallback chain.
The process might look like:
Your App
↓
FreeLLMAPI
↓
Provider A
↓
Rate Limit
↓
Provider B
↓
Successful Response
↓
Your App
This happens behind the same OpenAI-compatible API interface.
Your application does not need separate fallback logic for every provider.
Viewing Analytics
FreeLLMAPI includes analytics for monitoring your API usage.
Depending on the available dashboard features, you can inspect information such as:
Number of requests.
Request latency.
Token usage.
Success rates.
Provider usage.
Model usage.
This can help you understand which providers your applications are using and how your requests are being routed.
Production Setup
The development command:
npm run dev
is useful when testing FreeLLMAPI locally.
For a production build, the project’s documentation provides the following approach:
npm run build
Then start the built server:
node server/dist/index.js
The server and dashboard are then served on port 3001.
You can access the application locally through:
http://localhost:3001
For beginners, starting with local development mode is easier.
Only move to a production setup once you understand how the application works and how to secure it properly.
Important Security Considerations
FreeLLMAPI is designed as a local-first, single-user tool.
You should treat your provider API keys as sensitive information.
Never publish your .env file.
Never commit API keys to GitHub.
Never share your ENCRYPTION_KEY publicly.
Do not expose your FreeLLMAPI instance directly to the public internet without understanding the security implications.
Keep your unified API key private.
FreeLLMAPI’s own documentation positions the project primarily for personal, self-hosted use rather than as a public multi-user API service.
Each AI provider also has its own terms and usage restrictions.
You remain responsible for following those terms when using provider API keys through FreeLLMAPI.
What FreeLLMAPI Does Not Currently Support
FreeLLMAPI focuses mainly on text-based chat completion workflows.
According to the project’s current documentation, some API capabilities are not yet supported, including:
Embeddings.
Image generation.
Audio and speech APIs.
Vision and multimodal inputs.
Legacy text completion endpoints.
Moderation endpoints.
Multiple completions using n greater than 1.
Multi-user billing and multi-tenant authentication.
The main supported workflow is the OpenAI-compatible chat completions API.
For many text-based AI projects, this is enough to build chatbots, assistants, text generators, and other LLM-powered applications.
Common Setup Problems
The Dashboard Does Not Open
Make sure:
npm run dev
is still running.
Then check:
http://localhost:5173
Also check your terminal for error messages.
The API Request Fails
Make sure your API server is running.
Your base URL should normally be:
http://localhost:3001/v1
Also verify that your unified FreeLLMAPI API key is correct.
No AI Model Responds
Check whether you have added at least one valid provider API key.
Then verify that the provider key is active and has available quota.
Free tiers can change.
A provider may reduce its limits, remove a free tier, or change its API policies.
You Get a Rate-Limit Error
Your provider may have reached its free-tier limit.
If you have multiple providers configured correctly, FreeLLMAPI may attempt to use another available provider depending on your routing and fallback configuration.
Check the dashboard to see your provider status and usage.
Final Setup Overview
Your complete FreeLLMAPI setup works like this:
- Install Node.js 20+, npm, and Git.
- Clone the FreeLLMAPI GitHub repository.
- Run npm install.
- Create your .env file.
- Generate and configure an ENCRYPTION_KEY.
- Start FreeLLMAPI using npm run dev.
- Open the dashboard at localhost:5173.
- Obtain API keys from the AI providers you want to use.
- Add those provider keys to FreeLLMAPI.
- Configure your fallback chain.
- Copy your unified FreeLLMAPI API key.
- Point your OpenAI-compatible application to localhost:3001/v1.
- Use model=”auto” for automatic routing or select a supported model manually.
After this setup, your application can communicate with multiple configured AI providers through one OpenAI-compatible API interface.
FreeLLMAPI handles routing, provider availability, rate tracking, and supported failover behavior behind the scenes.
For developers experimenting with multiple free-tier AI APIs, this provides a practical way to manage them from one place without building a separate integration for every provider.
Thanks a lot