This product was not featured by Product Hunt yet. It will not be visible on their landing page and won't be ranked (cannot win product of the day regardless of upvotes).
Verify LLM API checks whether the model behind your AI agent or API matches its claim. Tell a Codex agent to run verifyllmapi.com/run, or test an OpenAI- or Anthropic-compatible endpoint. It reuses the Agent’s login in fresh sessions, samples one-token outputs, and matches their distribution with public fingerprints using Jensen–Shannon distance. Credentials and raw samples stay local. The library covers 161 exact model IDs; unknown or weak matches return INCONCLUSIVE.
Hey Product Hunt 👋
I built Verify LLM API after reading “One Token Is Enough,” a paper that shows how repeated single-token output distributions can act as behavioral model fingerprints.
The smallest useful flow is one prompt:
Run https://verifyllmapi.com/run to verify this AI Agent’s model.
For a Codex session, the Skill reuses the Agent’s host-managed login and starts fresh isolated sessions. It keeps credentials and raw samples local, then compares answer distributions with public references using Jensen–Shannon distance. The result is CONSISTENT, MISMATCH, or INCONCLUSIVE. It is behavioral evidence, not vendor attestation.
The main limit today is coverage. The bundled library spans 161 exact model IDs, not every current model or routing layer. New releases, provider changes, and wrappers can shift a fingerprint. I hope other builders will use the same method to measure the newest models and share reproducible results. If you test a newer model fingerprint, please email me at [email protected]. The same address is listed on verifyllmapi.com.
Thank you to the authors of “One Token Is Enough” for publishing the paper, dataset, and code. This tool is a small practical build on top of their work.
I’d value blunt feedback: where does the report feel clear, and where does it overstate or under-explain the evidence?
Model routing is getting increasingly opaque. Curious what was the original pain point that convinced you this needed to exist as a standalone product?
the fingerprinting approach is clever, especially keeping credentials local. one thing i'd love to see is a way to schedule periodic verification on a cron or webhook basis, so i can get alerted if my api provider silently swaps the model under me. that would make this way more useful for production monitoring.
ran it on a couple of internal endpoints and it caught one Claude ID that was secretly routing to a smaller model, which honestly surprised me. The INCONCLUSIVE fallback is smart too.
The privacy choice of keeping credentials and raw samples local is genuinely thoughtful, especially for a tool that has to handle API keys to do its job.
About Verify LLM API on Product Hunt
“Check which model your AI agent is really using”
Verify LLM API was submitted on Product Hunt and earned 9 upvotes and 9 comments, placing #132 on the daily leaderboard. Verify LLM API checks whether the model behind your AI agent or API matches its claim. Tell a Codex agent to run verifyllmapi.com/run, or test an OpenAI- or Anthropic-compatible endpoint. It reuses the Agent’s login in fresh sessions, samples one-token outputs, and matches their distribution with public fingerprints using Jensen–Shannon distance. Credentials and raw samples stay local. The library covers 161 exact model IDs; unknown or weak matches return INCONCLUSIVE.
Verify LLM API was featured in API (98.4k followers), Developer Tools (516.2k followers), Artificial Intelligence (474.3k followers) and GitHub (41.3k followers) on Product Hunt. Together, these topics include over 221.8k products, making this a competitive space to launch in.
Who hunted Verify LLM API?
Verify LLM API was hunted by Evan Reed. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how Verify LLM API stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
Evan Reed