VernLLM

The production reliability layer for LLM applications

Open Source

Developer Tools

GitHub

OpenAI Day

Visit WebsiteSee on Product HuntGithub ⧉Npm ⧉Vercel ⧉Twitter ⧉

Hey hunters! VernLLM is a lightweight resilience layer for OpenAI compatible LLM chat completion APIs. It wraps your existing LLM client and adds production reliability features like retries, timeouts, circuit breakers, caching, structured outputs, schema validation, usage tracking, and observability through one typed interface. I built VernLLM because while calling an LLM API is simple, running LLM applications reliably in production is not. Developers end up rebuilding the same infrastructure around every project: retry logic for transient failures, timeout handling, provider fallbacks, response validation, caching, and usage monitoring. The goal was to create a small, dependency light layer that keeps the simplicity of OpenAI's API style while adding the reliability features needed for real applications. During development, the focus evolved from solving individual reliability problems into building a complete reliability foundation that stays lightweight and flexible. Instead of locking developers into one provider or framework, VernLLM works across OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together AI, Fireworks, Ollama, Anthropic, Gemini, AWS Bedrock, and custom HTTP endpoints. Key advantages: • Production reliability without a heavy framework • Automatic retries with exponential backoff and jitter • Timeout protection and circuit breaking • Structured outputs with provider native JSON Schema support • Typed schema validation for safer AI responses • Caching with custom adapters • Usage tracking and observability hooks • Zero bundled dependencies and TypeScript first design VernLLM is open source, MIT licensed, and tested with 110+ unit and integration tests covering core reliability features and provider adapters. If you are building with OpenAI or other LLM providers, I would love to hear what reliability challenges you have faced moving from prototype to production!