We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see the "performance" area of our model pages. E.g.
https://openrouter.ai/deepseek/deepseek-v4-flash-0731#perfor...
You know how there's a router mode to use the cheapest provider? That only takes into account uncached rates, last I checked. Make another one that takes into account effective rates (the ones that include cache).
I think the future -may- be a hosted routing client. Something similar to an auth client app. Why? A few reasons:
- Customers being able to decide their own routing with true logic is a huge feature. Open router provides the seamless switching/api, route switching decisions are available in a client.
- Similarly, providing hooks at this level gives a chance for stats/other things that are hard to plug into prod code elsewhere
- a true middle man hosting for other things like MCP may also turn into a real win once it is implemented.
Just a random thought though. My point about quality/cost being clobbered by bad providers remains. The fact that cache and quality is badly handled makes me doubt that training data choices are being respected. You need a more public trust/certification process for providers with real teeth when they cheat. I'm going to wait a bit to see how things evolve and check back later.
You need to validate what providers are actually serving. Add benchmarks, properly showcase what quantization they are serving on the model and KV cache, etc.
Until that happens, your service is doomed to be shitty.
Open to feedback on how to make this better.