Problem
Currently, Qoder lacks visibility into the token cache hit status during code generation or chat interactions. Users are unable to determine how much of their request leveraged cached context versus generating new tokens. This opacity makes it difficult for users (especially those on usage-limited plans) to understand their token consumption and optimize their prompts for cost-efficiency.
Solution
I propose adding a visual indicator or detailed metadata in the response area that displays the Token Cache Hit Status.
Specifically, it would be great to see:
-
A clear label showing the percentage or amount of tokens retrieved from the cache (e.g., “Cache Hit: 45%”).
-
Visual differentiation between cached content and newly generated content in the usage breakdown.
Use Case
This feature is essential when I am working on large projects with extensive context windows. I often reuse similar codebases or documentation, and knowing the cache hit rate would help me verify if Qoder is effectively reusing previous context. It allows me to adjust my workflow to maximize cache usage and reduce unnecessary token costs.
Priority
Medium - Important improvement
Additional Info
Many modern AI coding assistants (like Cursor or other LLM interfaces) provide breakdown details of prompt caching. Adding this transparency would significantly improve the user experience and trust in the tool’s efficiency.