--- title: "Self-Host LiteLLM in Production: 2026 Guide" canonical: https://wavect.io/blog/self-host-litellm-production-2026/ language: en description: "Self-host LiteLLM safely in 2026. Plan Docker or Kubernetes, Postgres, Redis, virtual keys, monitoring, patching, rollout cost and alternatives." image: "https://wavect.io/img/blog/headers/header_self-host-litellm-production-2026.png" --- [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 13 min read · 13 Aug 2026 [**Next**](/blog/linux-for-ai-agents/) # How to Self-Host LiteLLM in Production: 2026 Architecture and Security Guide TL;DR Self-hosting LiteLLM means operating the AI gateway, not necessarily the models behind it. A production setup needs at least two pinned gateway replicas behind TLS, private managed Postgres, Redis for shared rate limits and routing state, scoped virtual keys, secret storage, health probes, metrics, backups and a tested upgrade path. LiteLLM OSS has no licence fee, but infrastructure and on-call ownership are real costs. The 2026 PyPI compromise and proxy vulnerabilities make exact version pinning, image-signature verification, network restriction and fast patching launch requirements. Start with a staging container, then move to a resilient stack only when control, compliance or provider portability justify the operations burden. LiteLLM can give every application one OpenAI-compatible endpoint while the gateway handles provider credentials, virtual keys, budgets, routing and observability. Its [official documentation](https://docs.litellm.ai/) positions the proxy as a central service for platform teams, separate from the Python SDK that runs inside one application. That distinction matters: this guide is about operating the proxy as infrastructure. The underserved question is no longer “Can I start the container?” It is “Can my team patch, scale and recover the gateway that now holds every model credential?” A demo needs one process. Production needs an owner, a private data layer, a release policy and evidence that failure does not stop every AI feature at once. ## What does self-hosting LiteLLM actually mean? **Self-hosting LiteLLM means that you operate the gateway inside infrastructure you control.** Requests still reach OpenAI, Anthropic, Bedrock or another configured provider unless you point the gateway at a local inference server such as vLLM or Ollama. You control the proxy, keys, logs and routing policy, not automatically the model execution. This separates the new article from two nearby decisions. Our [LLM gateway comparison](/blog/llm-gateway-router-comparison-2026/) helps you choose LiteLLM, OpenRouter, Portkey or a routing framework. Our [EU self-hosted LLM cost guide](/blog/self-hosting-llms-eu-cost/) covers running model weights and GPU inference. Here, the product is the gateway control plane between your applications and any mix of hosted or local models. ## When is self-hosting LiteLLM worth it? LiteLLM says its open-source gateway has no licence fee and lists virtual keys, budgets, rate limits, fallbacks, logging and Prometheus metrics in that tier. Enterprise adds governance and support features such as SSO, SCIM and audit logs. Verify the current split on the [LiteLLM pricing page](https://www.litellm.ai/pricing) before procurement. | Situation | Best default | Why | | --- | --- | --- | | One prototype, one provider, no platform owner | Call the provider directly | A gateway adds another production dependency before it solves a real problem. | | Several products or teams share provider accounts | Self-hosting can pay off | Scoped keys, central budgets and one provider abstraction create a clear control point. | | Delivery speed matters more than infrastructure control | Use a managed gateway | You buy operations, upgrades and support instead of building them. | | Private networking, EU deployment or custom controls are mandatory | Evaluate self-hosting | You choose the network, region, logs, retention and deployment cadence. | | You also need local model inference | Run gateway and inference as separate layers | LiteLLM routes requests. vLLM, Ollama or another server executes the model. | ## What architecture should a production LiteLLM deployment use? The current [LiteLLM production deployment guide](https://docs.litellm.ai/docs/proxy/deploy) describes stateless services behind a load balancer, with PostgreSQL for keys, teams, spend logs and configuration, plus Redis for shared rate limits, routing state and caching. It recommends two or more replicas and an explicit migrations job. That is the minimum credible production shape, not a one-container Compose file exposed to the internet. | Layer | Production responsibility | Failure question | | --- | --- | --- | | TLS ingress or load balancer | Terminate TLS, restrict routes, rate-limit abuse and drain replicas safely | Can one bad client reach management endpoints or exhaust the service? | | Two or more LiteLLM replicas | Serve traffic from an exact signed image version | Does a rollout or crashed pod interrupt active streams? | | Managed PostgreSQL | Persist keys, teams, spend and configuration with backups | Can you restore before the gateway becomes a company-wide outage? | | Managed Redis | Share rate limits, cache and router state across replicas | Do limits stay correct when traffic lands on different pods? | | Secret store | Hold provider credentials, master key and permanent salt key | Can one application retrieve another provider credential? | | Metrics, logs and traces | Measure availability, latency, spend, errors and saturation without leaking prompts | Will the on-call engineer know whether LiteLLM, the database or a provider failed? | Start with the monolithic image unless independent scaling of the gateway, backend and UI solves a measured bottleneck. A smaller architecture is easier to patch and recover. Componentization is useful at high traffic or for strict administrative separation, but it increases release coordination. ## How do you deploy LiteLLM safely? 1. **Define the gateway contract.** List the model aliases applications may call, the provider and regional fallback order, per-workload budgets, allowed endpoints, retention rules and the owner of each alert. A unified endpoint without policy only centralizes risk. 2. **Prove the path in staging.** Run a pinned container on a private endpoint, mount a versioned `config.yaml`, inject provider credentials from the environment and send a request with the standard OpenAI client. Do not expose `main-latest` or enable detailed debug logging in production. 3. **Add Postgres before issuing team keys.** Use a private managed database, encrypted connections, automated backups and a separate migration job. Keep schema changes out of serving replicas so an autoscaling event cannot race a database migration. 4. **Add Redis before the second replica.** LiteLLM’s [production checklist](https://docs.litellm.ai/docs/proxy/prod) recommends Redis 7 or newer once more than one proxy runs. Without shared state, replicas enforce limits separately and cache hits stay local. Scale Kubernetes with one worker per pod and CPU, not retained process memory. 5. **Issue a different virtual key per workload.** The [virtual-key documentation](https://docs.litellm.ai/docs/proxy/virtual_keys) requires Postgres and supports model access, budgets and spend attribution. Set explicit model and route allowlists. Never give an application the master key, and do not rely on an empty list to mean no access. 6. **Put the gateway on a private network.** Expose only the request routes clients need through TLS. Keep the Admin UI and management APIs behind identity-aware access or a separate administrative path. Restrict egress to approved providers and observability destinations. 7. **Make upgrades reversible.** Test image, configuration and migrations against a copy of production schema, deploy to a canary, replay representative requests, then roll forward. Keep the prior image and a compatible database backup ready for rollback. 8. **Run failure drills.** Disable one provider, one gateway replica, Redis and Postgres in turn. Confirm the expected fallback, readiness state, alert, recovery objective and client error. A fallback that was never exercised is documentation, not resilience. ## What changed for LiteLLM security in 2026? Security must shape the architecture because the proxy can hold model credentials, database access and request content. In March 2026, malicious LiteLLM versions 1.82.7 and 1.82.8 were published to PyPI. The project’s [incident timeline](https://github.com/BerriAI/litellm/issues/24518) says the affected packages could steal environment and cloud credentials, while proxy Docker image users were not affected. Teams that installed those packages should follow the incident guidance and rotate exposed credentials. Separate proxy vulnerabilities raised the bar further. A critical [SQL injection in API-key verification](https://github.com/BerriAI/litellm/security/advisories/GHSA-r75f-5x8p-qvmc) affected versions from 1.81.16 through 1.83.6. An [MCP test-endpoint command injection](https://github.com/BerriAI/litellm/security/advisories/GHSA-v4p8-mg3p-g94g) affected 1.74.2 through 1.83.6. A later [virtual-key privilege escalation](https://github.com/advisories/GHSA-qrc4-49gv-mv9m) was fixed in 1.83.14. Those old patched versions are historical floors, not recommended deployment targets. The practical response is straightforward: use a currently supported stable release, not a prerelease or old minimum patch. On 13 August 2026, GitHub marked v1.96.2 as the latest release and published a cosign verification command on the [LiteLLM releases page](https://github.com/BerriAI/litellm/releases). Re-check that page on deployment day, pin the exact version or digest, verify its signature, scan it, and promote the same digest through environments. ## What must be true before go-live? - **Least privilege is explicit.** Each workload has its own virtual key, named owner, allowed models, routes, budget and expiry. The master key never reaches an application. - **Secrets are recoverable and separated.** Provider keys and the master key live in the platform secret store. The permanent `LITELLM_SALT_KEY` is backed up separately because changing it after credentials are stored makes them unreadable. - **The network fails closed.** Admin routes are private, TLS verification stays on, provider egress is restricted and the database and Redis have no public address. These controls follow LiteLLM’s [security best practices](https://docs.litellm.ai/docs/proxy/security_best_practices). - **Logs have a data policy.** Decide whether prompts and responses may be logged, redact sensitive fields before export, set retention, restrict access and test deletion. Our [PII redaction gateway guide](/blog/pii-redaction-before-llm-prompts/) covers the boundary in detail. - **Health checks mean different things.** LiteLLM documents unauthenticated liveness and readiness endpoints that do not call models, while the authenticated model health endpoint sends real provider requests. Wire orchestrator probes and deeper synthetic checks accordingly using the [health-check contract](https://docs.litellm.ai/docs/proxy/health). - **Spend is reconciled.** Compare LiteLLM attribution with provider invoices and test streaming, retries, cache hits and fallback. A dashboard estimate is useful, but finance needs a reconciliation path. - **An owner can patch quickly.** Subscribe to security advisories, define a patch service level, maintain a staging smoke suite and document credential rotation after a suspected compromise. ## How much work does self-hosted LiteLLM require? There is no honest universal price because the gateway inherits your cloud, availability target, identity system and compliance scope. The ranges below are Wavect planning estimates for scoping, not LiteLLM quotes or cloud price promises. | Deployment level | Typical engineering effort | What is included | | --- | --- | --- | | Private staging gateway | 1 to 3 engineering days | Pinned container, two providers, config, one virtual key, basic logs and smoke tests | | Single-region production baseline | 1 to 3 engineering weeks | Replicas, TLS, managed Postgres and Redis, secret store, budgets, metrics, backups, canary rollout and runbook | | Regulated or multi-team platform | 4 to 10 engineering weeks | Identity integration, tenant policy, audit evidence, privacy controls, disaster recovery, load tests and support handover | | Ongoing ownership | Named monthly capacity plus on-call | Patches, provider changes, cost-map review, incident response, access review and restore drills | The infrastructure bill is rarely the deciding cost at modest traffic. Ownership is. If no one can accept the patching and recovery duty, a managed gateway is cheaper even when its invoice is higher. If a platform team already runs Kubernetes, Postgres, Redis, secrets and observability, LiteLLM can fit into controls you already pay for. ## Should you self-host LiteLLM or buy a managed gateway? **Self-host LiteLLM when control is a requirement and operations are an existing capability.** Buy a managed gateway when speed, support and reduced on-call load are worth more than infrastructure control. Keep direct provider access for a narrow prototype that has not earned a gateway yet. A useful procurement test is to price one year, not one container. Include design, implementation, database and Redis, monitoring, backups, upgrades, security review, on-call and a provider migration. Then compare that total with the managed option and the cost of doing nothing. For cost optimization after the gateway exists, use our [LLM token-cost reduction playbook](/blog/reduce-llm-token-costs-2026/). ## Frequently Asked Questions ### Is LiteLLM free to self-host? The LiteLLM open-source gateway has no licence fee. You still pay for compute, PostgreSQL, Redis, traffic, observability, backups, model-provider usage and the engineers who patch and operate it. Enterprise governance and support are separately priced. ### Does self-hosting LiteLLM keep prompts on my servers? The prompt passes through your gateway, but it still leaves for a hosted provider unless the selected model runs on infrastructure you control. Review the complete route, including logs, callbacks, provider retention and backups. ### Can LiteLLM run in Docker without Kubernetes? Yes. Docker is suitable for development, staging and some VM deployments. Production still needs multiple processes or replicas, TLS, Postgres, Redis, health checks, backups, monitoring and a safe release process. Kubernetes is one way to supply those controls, not the goal itself. ### Does LiteLLM need Postgres and Redis? Postgres is required for proxy authentication, virtual keys and spend tracking. Redis becomes necessary when multiple proxy instances must share rate limits, routing state and cache. A minimal stateless experiment can run without the full data layer, but it is not the multi-team production design. ### Which LiteLLM version should I deploy? Use the currently supported stable release, pin its exact image tag or digest, verify its signature and test it before promotion. Do not use moving tags or treat an old vulnerability patch floor as a current recommendation. ### When should I choose a managed alternative? Choose managed when no team owns gateway patches, recovery and on-call, or when time to market outweighs private infrastructure control. Revisit self-hosting when compliance, networking, provider portability or scale creates a specific business case. ## Final thoughts Self-hosted LiteLLM is a small service with a large blast radius. The container is easy. Production is the discipline around it: private networking, exact signed releases, scoped keys, Postgres, Redis, metrics, backups, failure drills and an owner who can patch quickly. Use LiteLLM when one controlled gateway simplifies several products and providers. Keep model inference as a separate architectural decision. If your team cannot own the gateway through an incident and a restore, buy the operations with a managed service. If it can, start with a narrow staging contract, prove resilience, and expand only from evidence. ## You may also like.. [**LLM gateways compared in 2026** Choose between LiteLLM, OpenRouter, Portkey and RouteLLM before committing to an operating model.](/blog/llm-gateway-router-comparison-2026/) [**EU AI vendor security questionnaire** Turn security, privacy, retention and incident-response claims into evidence requests before procurement.](/blog/ai-vendor-security-questionnaire-eu/) Models and infrastructure ## Continue through this cluster Model selection, inference economics, local deployment, compression and serving architecture. [Start with the cornerstone**Self-Hosting LLMs in the EU: When Open Weights Actually Pay Off**](/blog/self-hosting-llms-eu-cost/) - [AI-Ready Company Wiki: Architecture and Build Guide](/blog/ai-ready-company-wiki/) - [Does Claude Watermark Text? The 2026 API Answer](/blog/claude-text-watermark-api-2026/) - [OpenKB Review: Knowledge Compiler vs RAG](/blog/openkb-review-vs-rag/) - [Unsloth Desktop Review: A Private Local AI Workstation?](/blog/unsloth-desktop-local-ai-workstation-review/) - [NeMo Switchyard 0.2: Agent Model Routing Without Training?](/blog/nemo-switchyard-model-router/) Inbox, without the noise ## Follow the work that matters to you Get a short email when we publish something new. Follow the whole blog or only the problems you care about. [**Back**](/blog/overview/) [![Kevin Riedl](/img/team/kevin.webp)](/team/kevin-riedl/) [Kevin Riedl](/team/kevin-riedl/) https://linkedin.com/in/wsdt 13 min read · 13 Aug 2026 [**Next**](/blog/linux-for-ai-agents/) New posts by email × × Get new posts by email A short email when we publish. Free, no tracking. ## Structured Data ```json { "@context": "https://schema.org", "@graph": [ { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ], "employee": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "founder": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "legalRepresentative": [ { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } } ], "name": "Wavect GmbH", "subjectOf": { "@id": "https://wavect.io/verified-claims.json#dataset", "@type": "Dataset", "creator": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "description": "A machine-readable registry of quantitative and qualitative claims published by Wavect, with review dates, localized page appearances and public third-party citations where available.", "inLanguage": "en", "isAccessibleForFree": true, "license": "https://creativecommons.org/licenses/by/4.0/", "name": "Wavect verified publication claims", "url": "https://wavect.io/verified-claims.json" }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/team/christof-jori/#person", "@type": "Person", "jobTitle": "Managing Director", "name": "Christof Jori", "sameAs": [ "https://www.wikidata.org/wiki/Q139796367", "https://www.linkedin.com/in/jocr77/", "https://github.com/jo-chris" ], "url": "https://wavect.io/team/christof-jori/", "worksFor": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] } }, { "@id": "https://wavect.io/#website", "@type": "WebSite", "inLanguage": [ "en", "de", "es", "zh" ], "name": "Wavect", "potentialAction": { "@type": "SearchAction", "query-input": "required name=search_term_string", "target": { "@type": "EntryPoint", "urlTemplate": "https://wavect.io/search/?q={search_term_string}" } }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/" }, { "@id": "https://wavect.io/blog/self-host-litellm-production-2026/#webpage", "@type": "WebPage", "dateModified": "2026-08-13", "inLanguage": "en", "isPartOf": { "@id": "https://wavect.io/#website", "@type": "WebSite" }, "lastReviewed": "2026-08-13", "url": "https://wavect.io/blog/self-host-litellm-production-2026/" } ] } ``` ```json { "@context": "https://schema.org", "@type": "BlogPosting", "abstract": "Self-hosting LiteLLM means operating the AI gateway, not necessarily the models behind it. A production setup needs at least two pinned gateway replicas behind TLS, private managed Postgres, Redis for shared rate limits and routing state, scoped virtual keys, secret storage, health probes, metrics, backups and a tested upgrade path. LiteLLM OSS has no licence fee, but infrastructure and on-call ownership are real costs. The 2026 PyPI compromise and proxy vulnerabilities make exact version pinning, image-signature verification, network restriction and fast patching launch requirements. Start with a staging container, then move to a resilient stack only when control, compliance or provider portability justify the operations burden.", "articleBody": " Blog overview/AI and agents/Models and infrastructure How to Self-Host LiteLLM in Production: 2026 Architecture and Security Guide TL;DR Self-hosting LiteLLM means operating the AI gateway, not necessarily the models behind it. A production setup needs at least two pinned gateway replicas behind TLS, private managed Postgres, Redis for shared rate limits and routing state, scoped virtual keys, secret storage, health probes, metrics, backups and a tested upgrade path. LiteLLM OSS has no licence fee, but infrastructure and on-call ownership are real costs. The 2026 PyPI compromise and proxy vulnerabilities make exact version pinning, image-signature verification, network restriction and fast patching launch requirements. Start with a staging container, then move to a resilient stack only when control, compliance or provider portability justify the operations burden. LiteLLM can give every application one OpenAI-compatible endpoint while the gateway handles provider credentials, virtual keys, budgets, routing and observability. Its official documentation positions the proxy as a central service for platform teams, separate from the Python SDK that runs inside one application. That distinction matters: this guide is about operating the proxy as infrastructure. The underserved question is no longer “Can I start the container?” It is “Can my team patch, scale and recover the gateway that now holds every model credential?” A demo needs one process. Production needs an owner, a private data layer, a release policy and evidence that failure does not stop every AI feature at once. What does self-hosting LiteLLM actually mean? Self-hosting LiteLLM means that you operate the gateway inside infrastructure you control. Requests still reach OpenAI, Anthropic, Bedrock or another configured provider unless you point the gateway at a local inference server such as vLLM or Ollama. You control the proxy, keys, logs and routing policy, not automatically the model execution. This separates the new article from two nearby decisions. Our LLM gateway comparison helps you choose LiteLLM, OpenRouter, Portkey or a routing framework. Our EU self-hosted LLM cost guide covers running model weights and GPU inference. Here, the product is the gateway control plane between your applications and any mix of hosted or local models. When is self-hosting LiteLLM worth it? LiteLLM says its open-source gateway has no licence fee and lists virtual keys, budgets, rate limits, fallbacks, logging and Prometheus metrics in that tier. Enterprise adds governance and support features such as SSO, SCIM and audit logs. Verify the current split on the LiteLLM pricing page before procurement. SituationBest defaultWhy One prototype, one provider, no platform ownerCall the provider directlyA gateway adds another production dependency before it solves a real problem. Several products or teams share provider accountsSelf-hosting can pay offScoped keys, central budgets and one provider abstraction create a clear control point. Delivery speed matters more than infrastructure controlUse a managed gatewayYou buy operations, upgrades and support instead of building them. Private networking, EU deployment or custom controls are mandatoryEvaluate self-hostingYou choose the network, region, logs, retention and deployment cadence. You also need local model inferenceRun gateway and inference as separate layersLiteLLM routes requests. vLLM, Ollama or another server executes the model. What architecture should a production LiteLLM deployment use? The current LiteLLM production deployment guide describes stateless services behind a load balancer, with PostgreSQL for keys, teams, spend logs and configuration, plus Redis for shared rate limits, routing state and caching. It recommends two or more replicas and an explicit migrations job. That is the minimum credible production shape, not a one-container Compose file exposed to the internet. LayerProduction responsibilityFailure question TLS ingress or load balancerTerminate TLS, restrict routes, rate-limit abuse and drain replicas safelyCan one bad client reach management endpoints or exhaust the service? Two or more LiteLLM replicasServe traffic from an exact signed image versionDoes a rollout or crashed pod interrupt active streams? Managed PostgreSQLPersist keys, teams, spend and configuration with backupsCan you restore before the gateway becomes a company-wide outage? Managed RedisShare rate limits, cache and router state across replicasDo limits stay correct when traffic lands on different pods? Secret storeHold provider credentials, master key and permanent salt keyCan one application retrieve another provider credential? Metrics, logs and tracesMeasure availability, latency, spend, errors and saturation without leaking promptsWill the on-call engineer know whether LiteLLM, the database or a provider failed? Start with the monolithic image unless independent scaling of the gateway, backend and UI solves a measured", "articleSection": "Engineering", "author": { "@id": "https://wavect.io/team/kevin-riedl/#person", "@type": "Person", "name": "Kevin Riedl", "sameAs": [ "https://www.wikidata.org/wiki/Q139796365", "https://www.linkedin.com/in/wsdt", "https://github.com/wsdt" ], "url": "https://wavect.io/team/kevin-riedl/" }, "citation": [ { "@type": "WebPage", "name": "official documentation", "url": "https://docs.litellm.ai/" }, { "@type": "WebPage", "name": "LiteLLM pricing page", "url": "https://www.litellm.ai/pricing" }, { "@type": "WebPage", "name": "LiteLLM production deployment guide", "url": "https://docs.litellm.ai/docs/proxy/deploy" }, { "@type": "WebPage", "name": "production checklist", "url": "https://docs.litellm.ai/docs/proxy/prod" }, { "@type": "WebPage", "name": "virtual-key documentation", "url": "https://docs.litellm.ai/docs/proxy/virtual_keys" }, { "@type": "WebPage", "name": "incident timeline", "url": "https://github.com/BerriAI/litellm/issues/24518" }, { "@type": "WebPage", "name": "SQL injection in API-key verification", "url": "https://github.com/BerriAI/litellm/security/advisories/GHSA-r75f-5x8p-qvmc" }, { "@type": "WebPage", "name": "MCP test-endpoint command injection", "url": "https://github.com/BerriAI/litellm/security/advisories/GHSA-v4p8-mg3p-g94g" }, { "@type": "WebPage", "name": "virtual-key privilege escalation", "url": "https://github.com/advisories/GHSA-qrc4-49gv-mv9m" }, { "@type": "WebPage", "name": "LiteLLM releases page", "url": "https://github.com/BerriAI/litellm/releases" }, { "@type": "WebPage", "name": "security best practices", "url": "https://docs.litellm.ai/docs/proxy/security_best_practices" }, { "@type": "WebPage", "name": "health-check contract", "url": "https://docs.litellm.ai/docs/proxy/health" } ], "dateModified": "2026-08-13", "datePublished": "2026-08-13", "description": "Self-hosting LiteLLM means operating the AI gateway, not necessarily the models behind it. A production setup needs at least two pinned gateway replicas behind TLS, private managed Postgres, Redis for shared rate limits and routing state, scoped virtual keys, secret storage, health probes, metrics, backups and a tested upgrade path. LiteLLM OSS has no licence fee, but infrastructure and on-call ownership are real costs. The 2026 PyPI compromise and proxy vulnerabilities make exact version pinning, image-signature verification, network restriction and fast patching launch requirements. Start with a staging container, then move to a resilient stack only when control, compliance or provider portability justify the operations burden.", "headline": "How to Self-Host LiteLLM in Production: 2026 Guide", "image": "https://wavect.io/img/blog/headers/header_self-host-litellm-production-2026.svg", "inLanguage": "en", "keywords": "LiteLLM, AI infrastructure, LLM gateway, Self-hosting", "mainEntityOfPage": { "@id": "https://wavect.io/blog/self-host-litellm-production-2026/", "@type": "WebPage" }, "publisher": { "@id": "https://wavect.io/#organization", "@type": [ "Organization", "ProfessionalService", "LocalBusiness" ] }, "url": "https://wavect.io/blog/self-host-litellm-production-2026/", "wordCount": 2392 } ``` ```json { "@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "item": "https://wavect.io/", "name": "Home", "position": 1 }, { "@type": "ListItem", "item": "https://wavect.io/blog/overview/", "name": "Blog overview", "position": 2 }, { "@type": "ListItem", "item": "https://wavect.io/blog/topics/ai-agents/", "name": "AI and agents", "position": 3 }, { "@type": "ListItem", "item": "https://wavect.io/blog/clusters/models-infrastructure/", "name": "Models and infrastructure", "position": 4 }, { "@type": "ListItem", "item": "https://wavect.io/blog/self-host-litellm-production-2026/", "name": "Self-Host LiteLLM in Production: 2026 Guide | ", "position": 5 } ] } ``` ```json { "@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [ { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "The LiteLLM open-source gateway has no licence fee. You still pay for compute, PostgreSQL, Redis, traffic, observability, backups, model-provider usage and the engineers who patch and operate it. Enterprise governance and support are separately priced." }, "name": "Is LiteLLM free to self-host?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "The prompt passes through your gateway, but it still leaves for a hosted provider unless the selected model runs on infrastructure you control. Review the complete route, including logs, callbacks, provider retention and backups." }, "name": "Does self-hosting LiteLLM keep prompts on my servers?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Yes. Docker is suitable for development, staging and some VM deployments. Production still needs multiple processes or replicas, TLS, Postgres, Redis, health checks, backups, monitoring and a safe release process. Kubernetes is one way to supply those controls, not the goal itself." }, "name": "Can LiteLLM run in Docker without Kubernetes?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Postgres is required for proxy authentication, virtual keys and spend tracking. Redis becomes necessary when multiple proxy instances must share rate limits, routing state and cache. A minimal stateless experiment can run without the full data layer, but it is not the multi-team production design." }, "name": "Does LiteLLM need Postgres and Redis?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Use the currently supported stable release, pin its exact image tag or digest, verify its signature and test it before promotion. Do not use moving tags or treat an old vulnerability patch floor as a current recommendation." }, "name": "Which LiteLLM version should I deploy?" }, { "@type": "Question", "acceptedAnswer": { "@type": "Answer", "text": "Choose managed when no team owns gateway patches, recovery and on-call, or when time to market outweighs private infrastructure control. Revisit self-hosting when compliance, networking, provider portability or scale creates a specific business case." }, "name": "When should I choose a managed alternative?" } ] } ```