How to give your team AI access without handing out API keys

Why shared API keys go wrong

• One runaway script spends the month. A shared key has no per-person limit, so one loop or one curious intern can burn the whole budget overnight.
• Nobody knows who sent what. With one key for everyone, the provider bill is a single number. You cannot see which team, tool or person drove it.
• You cannot revoke one person. When someone leaves, the only fix is rotating the key — and breaking every tool that uses it.
• Keys leak. Shared keys end up in chat threads, spreadsheets and code repositories, where anyone can copy them.
• Credits are stuck in personal accounts. Cloud credits or company accounts tied to one person’s login become a problem the day that person moves on.

The pattern: one gateway, many keys

LiteLLM is an open-source proxy that speaks the OpenAI API and forwards requests to more than a hundred providers. Your people never see a provider key — they get a virtual key that the gateway checks, limits and records.

Set it up, step by step

1. 01 Pick a home for the gateway

   A small Linux server with Docker is enough to start — the gateway is light; Postgres stores keys and spend. Give it a domain with TLS, and keep the admin interface off the public internet (VPN, tailnet or an IP allow-list).

2. 02 Create the two secrets that matter

   The master key is the admin key — it creates every other key and must start with sk-. The salt key encrypts provider credentials stored in the database; set it once and never change it after you add models. Keep both in your secret manager, not in the repository.

   umask 077
   {
     echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 24)"
     echo "LITELLM_SALT_KEY=$(openssl rand -hex 32)"
     echo "POSTGRES_PASSWORD=$(openssl rand -hex 16)"
   } > .env
   
   # then add your provider keys — they live only on this server
   echo 'OPENAI_API_KEY=...'    >> .env
   echo 'ANTHROPIC_API_KEY=...' >> .env
   echo 'DEEPSEEK_API_KEY=...'  >> .env

3. 03 Describe your models

   Each entry maps the name your people call to a real provider model. Using cloud credits? Point a model at a specific Google Cloud project — one project per department or course keeps budgets separate.

   model_list:
     - model_name: gpt-4o-mini
       litellm_params:
         model: openai/gpt-4o-mini
         api_key: os.environ/OPENAI_API_KEY
   
     - model_name: claude-sonnet
       litellm_params:
         model: anthropic/claude-sonnet-4-5
         api_key: os.environ/ANTHROPIC_API_KEY
   
     # Google Cloud credits: one GCP project per team or course
     - model_name: gemini-teaching
       litellm_params:
         model: vertex_ai/gemini-2.5-pro
         vertex_project: "faculty-course-a"
         vertex_location: "asia-southeast1"   # a region where the model is offered
         vertex_credentials: os.environ/VERTEX_SA_COURSE_A
   
     - model_name: deepseek-chat
       litellm_params:
         model: deepseek/deepseek-chat
         api_key: os.environ/DEEPSEEK_API_KEY
   
   litellm_settings:
     drop_params: true                # ignore params a provider doesn't support
     turn_off_message_logging: true   # no prompts in logging callbacks
   
   general_settings:
     master_key: os.environ/LITELLM_MASTER_KEY
     database_url: os.environ/DATABASE_URL

   Model names change often — use the current model ids from each provider’s console. The names on the left are yours; keep them stable so tools never need editing.

4. 04 Run it

   LiteLLM ships a Docker image. Pin a release tag in production so upgrades and rollbacks are deliberate, and check the liveness endpoint once it starts.

   services:
     litellm:
       image: ghcr.io/berriai/litellm:main-stable   # pin a release tag in production
       command: ["--config", "/app/config.yaml", "--port", "4000"]
       volumes:
         - ./config.yaml:/app/config.yaml:ro
       env_file: .env
       environment:
         DATABASE_URL: postgresql://llmproxy:${POSTGRES_PASSWORD}@db:5432/litellm
       ports:
         - "127.0.0.1:4000:4000"   # expose through your TLS proxy, not directly
       depends_on:
         db:
           condition: service_healthy
       restart: unless-stopped
   
     db:
       image: postgres:16
       environment:
         POSTGRES_DB: litellm
         POSTGRES_USER: llmproxy
         POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
       volumes:
         - litellm_pg:/var/lib/postgresql/data
       healthcheck:
         test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
         interval: 5s
         retries: 10
       restart: unless-stopped
   
   volumes:
     litellm_pg:

   docker compose up -d
   curl -s http://127.0.0.1:4000/health/liveliness

5. 05 Create a team for each group, with a budget

   Teams carry the models a group may use and a spend cap that resets on a schedule. Requests are rejected once the team crosses its budget.

   curl -s https://llm.example.com/team/new \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "team_alias": "finance",
       "models": ["gpt-4o-mini", "claude-sonnet"],
       "max_budget": 200,
       "budget_duration": "1mo"
     }'

6. 06 Issue a key per person

   Each virtual key names its owner, belongs to a team, lists the models it may call, and has its own budget, rate limit and expiry. Hand out this key — never the provider key.

   curl -s https://llm.example.com/key/generate \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "key_alias": "priya-laptop",
       "user_id": "[email protected]",
       "team_id": "<team_id from the previous step>",
       "models": ["gpt-4o-mini", "claude-sonnet"],
       "max_budget": 20,
       "budget_duration": "30d",
       "rpm_limit": 60,
       "duration": "90d"
     }'

7. 07 Point your tools at the gateway

   The gateway speaks the OpenAI API, so most tools only need a base URL and the person’s virtual key — SDKs, chat apps such as Open WebUI or LibreChat, and coding agents that accept an OpenAI-compatible endpoint.

   export OPENAI_BASE_URL=https://llm.example.com/v1
   export OPENAI_API_KEY=sk-...   # the person's virtual key

   from openai import OpenAI
   
   client = OpenAI()  # reads OPENAI_BASE_URL and OPENAI_API_KEY
   reply = client.chat.completions.create(
       model="claude-sonnet",  # the name from your config, not the provider's
       messages=[{"role": "user", "content": "Summarise this policy in 3 bullets."}],
   )
   print(reply.choices[0].message.content)

8. 08 Watch spend, cap it, revoke it

   Every call is recorded against the key, the person and the team. Block a key the moment someone leaves or a device is lost — nobody else is affected.

   # spend and limits for one key
   curl -s "https://llm.example.com/key/info?key=$KEY" \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY"
   
   # switch a key off (and back on)
   curl -s https://llm.example.com/key/block \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" -d "{\"key\": \"$KEY\"}"
   curl -s https://llm.example.com/key/unblock \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" -d "{\"key\": \"$KEY\"}"

Before you hand out the first key

• The master key never leaves your ops team — not in a laptop’s shell history, not in a shared doc.
• Every key gets a budget and an expiry. There are no unlimited keys.
• Budgets are hard limits: LiteLLM estimates a request’s cost before sending it and rejects it if the budget would be crossed. Turn on fail-closed budget enforcement so requests stop if spend can’t be checked.
• The admin UI and management endpoints sit behind a VPN, tailnet or IP allow-list.
• Prompts with personal data stay out of third-party logging (turn_off_message_logging).
• Run Redis once you have more than one gateway instance, so rate limits and budgets are shared.
• Pin the image version, back up Postgres, and keep config.yaml in version control.
• Review spend by team every week for the first month — it shows you who actually uses what.