I Built an AI Token Dashboard and Cut My API Bill by 60%

My $2,300 OpenAI Bill Was a Wake-Up Call - Here’s How I Fixed It

You know that sinking feeling when you see an unexpected charge? Well, that was me staring at a $2,300 bill from OpenAI. It hit me that something was seriously off. As I started digging through the logs, the culprit became painfully obvious: we had four developers, seven microservices, and zero insight into which ones were draining our token budget. If you’re curious, our comparison of AI tools illustrates just how different the costs can be across providers. The biggest surprise? An agent I thought was just handling some basic classification tasks was actually sending 12,000 tokens per request - thanks to someone copy-pasting an entire context window from a prototype without cleaning it up. Yikes!

That moment was my turning point. I decided to stop treating AI costs like a regular utility bill and instead manage them like an integral part of our production infrastructure. So, I rolled up my sleeves and built a dashboard that:

1. Intercepts every API call through middleware
2. Logs tokens, models, and endpoints into SQLite
3. Breaks down costs by agent, user, and hour

And guess what? We managed to cut our monthly bill by 60% in just 30 days - not because we switched to cheaper models, but because we finally saw where the waste was happening.

The Solution: A Simple Proxy That Reveals Everything

Here’s the reality: every AI agent consumes tokens, but most teams have no clue how many. My solution? A straightforward HTTP proxy that:

• Sits between your services and LLM providers
• Logs every request’s model, tokens, latency, and cost
• Stores all that info in SQLite for easy access
• Adds less than 5ms overhead

Here’s a sneak peek of what the code looks like:

from flask import Flask, request, jsonify
import requests, sqlite3, json, time

app = Flask(__name__)
UPSTREAM = "https://api.openai.com"

def log_to_db(model, prompt_tokens, completion_tokens, latency_ms, agent_id, cost):
  db = sqlite3.connect("token_burn.db")
  db.execute("INSERT INTO token_log VALUES (', ', ', ', ', ', datetime('now'))",
    (model, prompt_tokens, completion_tokens, latency_ms, agent_id, cost))
  db.commit()

@app.route("/v1/<path:subpath>", methods=["POST", "GET"])
def proxy(subpath):
  headers = {k: v for k, v in request.headers if k.lower() != "host"}
  t0 = time.time()
  resp = requests.request(request.method, f"{UPSTREAM}/v1/{subpath}",
    headers=headers, json=request.get_json(silent=True), stream=False)
  latency = int((time.time() - t0) * 1000)
  agent = request.headers.get("X-Agent-ID", "unknown")
  data = resp.json()
  usage = data.get("usage", {})
  pt = usage.get("prompt_tokens", 0)
  ct = usage.get("completion_tokens", 0)
  model = data.get("model", "unknown")
  cost = (pt * 0.00015 + ct * 0.0006) / 1000  # gpt-4o rates
  log_to_db(model, pt, ct, latency, agent, cost)
  return jsonify(data), resp.status_code

What We Found (And Fixed)

After a month of tracking, we discovered some tough truths:

1. Duplicate Agents: We had two separate services doing the same classification work, which accounted for 40% of our costs. By merging them, we saved $920 a month.

2. Context Overload: One agent was sending 48,000 tokens worth of chat history just for simple yes/no questions. By adding max_history: 5, we reduced its costs by 80%.

3. Peak Hour Waste: Our 2 PM cron job was reprocessing all daily data instead of just new changes, leading to a $180/hour spike.

Here’s a quick before-and-after comparison:

Metric	Before	After
Monthly Cost	$2,300	$920
Avg Tokens/Request	4,200	890
Peak Hour Cost	$180	$38

Who This Works For

This approach might be a good fit if:

• You have multiple developers or services using AI
• Your monthly bill is over $100
• You want to pinpoint waste without changing models

But it might not help if:

• You’re using providers that don’t return token counts
• Developers bypass the proxy
• You have just a single, simple use case

Alternatives Worth Considering

1. Helicone: An open-source proxy with a hosted dashboard (free for 100K requests)
2. LangSmith: A full-featured option that requires LangChain ($99/month)
3. Manual Logging: I gave this a shot first, but it only lasted three days before I threw in the towel!

Key Takeaways

1. Keep it simple: Even basic token logging can expose waste.
2. Look for ratios: If your prompt-to-completion ratios are over 10:1, that might indicate bloat.
3. Watch for peaks: Uneven usage can signal inefficiencies in scheduling.
4. Tag everything: Use X-Agent-ID headers to keep track of sources.

The proxy code, schema, and dashboard queries are all laid out above. It took me just an afternoon to build, and it paid for itself on day one. If you’re tired of those surprise AI bills, I encourage you to give it a try! What do you think? Have you faced similar challenges’

References & Further Reading

• OpenAI: Privacy
• Wikipedia: Privacy policy