🧠

AI Agent Memory / RAG Cost Calculator

Estimate embedding, vector storage, and query costs for agent memory and RAG. Compare managed and self-hosted options side by side.

Corpus & workload

Cached queries skip the vector DB search and reranker.

Cheap default; 1536 dims.

Pay for stored vectors plus read units.

Per-query reranking of top-k results.

Total first-period cost

—

Embedding + 12 months of operation

One-time embedding

—

Monthly total

—

Storage / mo

—

Query cost / mo

—

Reranker / mo

—

Cost per 1k queries

—

Effective chunks

—

Vector storage

—

Prices are directional estimates. Check each provider's current pricing before budgeting.

Vector DB cost comparison

Same corpus, embedding model, and query volume across all databases. Includes storage, uncached query search, query embedding, and reranker costs.

Vector DB Storage/mo Search/mo Query embed/mo Reranker/mo Total/mo
Pinecone Serverless
managed		 — — — — —
Weaviate Cloud
managed		 — — — — —
Qdrant Cloud
managed		 — — — — —
Zilliz / Milvus Cloud
managed		 — — — — —
Redis Cloud (vector search)
managed		 — — — — —
pgvector (RDS / self-hosted)
self-hosted		 — — — — —
Chroma (self-hosted)
self-hosted		 — — — — —

How the estimate works

  • Chunks: total corpus tokens ÷ effective chunk size after overlap.
  • Storage: each chunk stores a vector (dims × 4 bytes) plus metadata.
  • Embedding cost: one-time charge to embed all corpus tokens, plus per-query embedding tokens for uncached searches.
  • Query cost: uncached searches are charged per 1,000 by the vector DB plus any reranker cost.
  • Cache: cached queries skip search and reranker entirely.

Where to cut costs

  • • Use a cheaper embedding model if retrieval quality allows it.
  • • Raise cache hit rate with query embedding cache or response cache.
  • • Self-host pgvector or Chroma if you already run a server.
  • • Skip rerankers for internal prototypes; add them after product-market fit.
  • • Reduce dimensions or quantize vectors if your DB supports it.
Agent Stack Builder → LLM Cost Calculator → Context Window Cheat Sheet →

🤖 Use this tool in your agent

✓ Agent-ready code

Copy the snippet below into your agent, newsletter, or script. The tool page at hermesdispatch.dev/tools/agent-memory-cost-calculator/ is the canonical contract: inputs, outputs, and formulas.

python 
# Hermes Dispatch Tool — AI Agent Memory / RAG Cost Calculator
# Source: https://hermesdispatch.dev/tools/agent-memory-cost-calculator/
# Description: Estimate vector DB and embedding costs for AI agent memory and RAG.
# License: MIT (generated by hermesdispatch.dev)
#
# INSTALL:
#   1. Save this file as ~/.hermes/hermes-agent/tools/agent_memory_cost_calculator.py
#   2. Restart Hermes or run /reset in a session
#   3. The tool auto-registers if Hermes uses auto-discovery of tools/*.py
#
# MANUAL REGISTRY (if auto-discovery is off):
#   from tools.agent_memory_cost_calculator import register
#   register()

import json

DATA = {"currency": "USD", "defaults": {"documents": 10000, "tokens_per_doc": 500, "tokens_per_chunk": 512, "overlap_pct": 10, "query_tokens": 50, "queries_per_month": 100000, "cached_query_pct": 30, "include_reranker": true, "reranker": "cohere", "embedding_model": "openai-text-embedding-3-small", "vector_db": "pinecone", "storage_months": 12}, "presets": [{"slug": "support-bot", "name": "Small support bot", "icon": "\ud83d\udcac", "documents": 5000, "tokens_per_doc": 300, "queries_per_month": 50000, "cached_query_pct": 40, "vector_db": "qdrant", "embedding_model": "openai-text-embedding-3-small", "include_reranker": true, "reranker": "cohere"}, {"slug": "research-assistant", "name": "Research assistant", "icon": "\ud83d\udd2c", "documents": 50000, "tokens_per_doc": 1200, "queries_per_month": 20000, "cached_query_pct": 20, "vector_db": "pinecone", "embedding_model": "voyage-3", "include_reranker": true, "reranker": "cohere"}, {"slug": "enterprise-kb", "name": "Enterprise knowledge base", "icon": "\ud83c\udfe2", "documents": 250000, "tokens_per_doc": 800, "queries_per_month": 500000, "cached_query_pct": 35, "vector_db": "weaviate", "embedding_model": "openai-text-embedding-3-large", "include_reranker": true, "reranker": "cohere"}, {"slug": "codebase-rag", "name": "Codebase RAG", "icon": "\ud83d\udcbb", "documents": 100000, "tokens_per_doc": 400, "queries_per_month": 150000, "cached_query_pct": 25, "vector_db": "pgvector", "embedding_model": "local-bge", "include_reranker": false, "reranker": "local"}], "embedding_models": [{"slug": "openai-text-embedding-3-small", "name": "OpenAI text-embedding-3-small", "provider": "OpenAI", "dims": 1536, "cost_per_1m_tokens_usd": 0.02, "notes": "Cheap default; 1536 dims."}, {"slug": "openai-text-embedding-3-large", "name": "OpenAI text-embedding-3-large", "provider": "OpenAI", "dims": 3072, "cost_per_1m_tokens_usd": 0.13, "notes": "Best retrieval quality; 3072 dims."}, {"slug": "cohere-embed-english-v3", "name": "Cohere embed-english-v3", "provider": "Cohere", "dims": 1024, "cost_per_1m_tokens_usd": 0.1, "notes": "Good for classification and retrieval."}, {"slug": "voyage-3-lite", "name": "Voyage-3-lite", "provider": "Voyage AI", "dims": 512, "cost_per_1m_tokens_usd": 0.02, "notes": "Strong small embedding model."}, {"slug": "voyage-3", "name": "Voyage-3", "provider": "Voyage AI", "dims": 1024, "cost_per_1m_tokens_usd": 0.1, "notes": "High retrieval quality for agents."}, {"slug": "google-text-embedding-004", "name": "Google text-embedding-004", "provider": "Google", "dims": 768, "cost_per_1m_tokens_usd": 0.1, "notes": "Vertex AI / Google Gen AI embedding."}, {"slug": "local-bge", "name": "Local BGE-large / GTE (self-hosted)", "provider": "Self-hosted", "dims": 1024, "cost_per_1m_tokens_usd": 0, "notes": "No token API cost; GPU/CPU cost not included."}], "vector_dbs": [{"slug": "pinecone", "name": "Pinecone Serverless", "type": "managed", "storage_cost_gb_month_usd": 0.33, "query_cost_per_1k_usd": 0.06, "free_gb": 0, "free_queries": 0, "notes": "Pay for stored vectors plus read units."}, {"slug": "weaviate", "name": "Weaviate Cloud", "type": "managed", "storage_cost_gb_month_usd": 0.25, "query_cost_per_1k_usd": 0.05, "free_gb": 0, "free_queries": 0, "notes": "Managed vector database with hybrid search."}, {"slug": "qdrant", "name": "Qdrant Cloud", "type": "managed", "storage_cost_gb_month_usd": 0.2, "query_cost_per_1k_usd": 0.03, "free_gb": 1, "free_queries": 10000, "notes": "Generous free tier for small workloads."}, {"slug": "zilliz", "name": "Zilliz / Milvus Cloud", "type": "managed", "storage_cost_gb_month_usd": 0.15, "query_cost_per_1k_usd": 0.04, "free_gb": 0, "free_queries": 0, "notes": "Milvus-as-a-service."}, {"slug": "redis", "name": "Redis Cloud (vector search)", "type": "managed", "storage_cost_gb_month_usd": 0.5, "query_cost_per_1k_usd": 0.02, "free_gb": 0, "free_queries": 0, "notes": "Existing Redis stack pricing."}, {"slug": "pgvector", "name": "pgvector (RDS / self-hosted)", "type": "self-hosted", "storage_cost_gb_month_usd": 0.12, "query_cost_per_1k_usd": 0, "free_gb": 0, "free_queries": 0, "notes": "DB compute cost is separate."}, {"slug": "chroma", "name": "Chroma (self-hosted)", "type": "self-hosted", "storage_cost_gb_month_usd": 0, "query_cost_per_1k_usd": 0, "free_gb": 0, "free_queries": 0, "notes": "Local disk only; hardware cost separate."}], "rerankers": [{"slug": "cohere", "name": "Cohere Rerank v3", "provider": "Cohere", "cost_per_1k_queries_usd": 1.0, "notes": "Per-query reranking of top-k results."}, {"slug": "mixedbread", "name": "mixedbread.ai rerank", "provider": "mixedbread", "cost_per_1k_queries_usd": 0.2, "notes": "Lower-cost hosted reranker."}, {"slug": "local", "name": "Local cross-encoder", "provider": "Self-hosted", "cost_per_1k_queries_usd": 0, "notes": "GPU/CPU cost separate."}], "chunking": {"metadata_bytes_per_chunk": 250, "vector_bytes_per_dim": 4, "vector_overhead_pct": 15}, "faq": [{"question": "How much does RAG memory cost for an AI agent?", "answer": "RAG memory cost has three parts: a one-time embedding cost for your corpus, monthly vector storage, and per-query costs for embedding queries and searching the index. Reranking adds another per-query cost."}, {"question": "Which is cheaper: managed vector DB or self-hosted pgvector?", "answer": "For small workloads, managed services like Qdrant Cloud have free tiers. At scale, self-hosted pgvector or Chroma can be cheaper if you already run a server, but you pay for compute and maintenance instead."}, {"question": "Does chunk size affect RAG cost?", "answer": "Yes. Smaller chunks mean more chunks and slightly higher storage, but better retrieval. Larger chunks reduce chunk count but increase embedding tokens and can dilute relevance."}, {"question": "Should I include a reranker in my agent stack?", "answer": "Rerankers improve answer quality by re-ordering retrieved chunks, but they add $0.20-1.00 per 1,000 queries. Use one if retrieval quality is business-critical; skip it for cost-sensitive prototypes."}, {"question": "What is the cheapest embedding model for RAG?", "answer": "OpenAI text-embedding-3-small and Voyage-3-lite are roughly $0.02 per 1M tokens. Self-hosted models like BGE have no token API cost but need a GPU or CPU to run."}]}

def _ok(result):
    return json.dumps({"success": True, "data": result}, indent=2)

def _err(message):
    return json.dumps({"success": False, "error": message}, indent=2)


TOOL_NAME = "agent_memory_cost_calculator"
TOOLSET = "agents"

SCHEMA = {
  "type": "function",
  "function": {
    "name": "agent_memory_cost_calculator",
    "description": "Estimate vector DB, embedding, and reranker costs for agent memory/RAG.",
    "parameters": {
      "type": "object",
      "properties": {
        "documents": {
          "type": "integer",
          "description": "Number of documents in corpus."
        },
        "tokens_per_doc": {
          "type": "integer",
          "description": "Average tokens per document."
        },
        "tokens_per_chunk": {
          "type": "integer",
          "description": "Tokens per chunk."
        },
        "overlap_pct": {
          "type": "number",
          "description": "Chunk overlap percentage."
        },
        "queries_per_month": {
          "type": "integer",
          "description": "Queries per month."
        },
        "query_tokens": {
          "type": "integer",
          "description": "Average query tokens."
        },
        "embedding_model": {
          "type": "string",
          "description": "Model slug: openai-text-embedding-3-small, openai-text-embedding-3-large, cohere-embed-english-v3, voyage-3-lite, voyage-3, google-text-embedding-004, local-bge"
        },
        "vector_db": {
          "type": "string",
          "description": "DB slug: pinecone, weaviate, qdrant, zilliz, redis, pgvector, chroma"
        },
        "reranker": {
          "type": "string",
          "description": "Reranker slug or none: cohere, mixedbread, local"
        }
      },
      "required": []
    }
  }
}

def _run(args):
    docs = int(args.get("documents", DATA["defaults"]["documents"]))
    tokens_per_doc = int(args.get("tokens_per_doc", DATA["defaults"]["tokens_per_doc"]))
    tokens_per_chunk = int(args.get("tokens_per_chunk", DATA["defaults"]["tokens_per_chunk"]))
    overlap_pct = float(args.get("overlap_pct", DATA["defaults"]["overlap_pct"]))
    queries = int(args.get("queries_per_month", DATA["defaults"]["queries_per_month"]))
    query_tokens = int(args.get("query_tokens", DATA["defaults"]["query_tokens"]))
    embedding_slug = args.get("embedding_model", "openai-3-small")
    db_slug = args.get("vector_db", "pinecone")
    reranker_slug = args.get("reranker", "none")
    embedding = next((m for m in DATA["embedding_models"] if m["slug"] == embedding_slug), DATA["embedding_models"][0])
    db = next((d for d in DATA["vector_dbs"] if d["slug"] == db_slug), DATA["vector_dbs"][0])
    reranker = next((r for r in DATA["rerankers"] if r["slug"] == reranker_slug), None)
    total_tokens = docs * tokens_per_doc
    overlap_factor = 1 + overlap_pct / 100
    chunks = max(1, int(total_tokens / tokens_per_chunk * overlap_factor))
    embedding_input_tokens = chunks * tokens_per_chunk
    embedding_cost = (embedding_input_tokens / 1_000_000) * embedding.get("cost_per_1m_input", 0.02)
    storage_vectors = chunks
    storage_cost = storage_vectors * db.get("cost_per_1m_vectors_month", 0) / 1_000_000
    query_embedding_cost = (queries * query_tokens / 1_000_000) * embedding.get("cost_per_1m_input", 0.02)
    query_cost = queries * db.get("cost_per_1k_queries", 0) / 1000
    rerank_cost = (queries * reranker.get("cost_per_1k_queries", 0) / 1000) if reranker else 0
    total = embedding_cost + storage_cost + query_embedding_cost + query_cost + rerank_cost
    return _ok({
        "chunks": chunks,
        "embedding_cost": round(embedding_cost, 2),
        "storage_cost_monthly": round(storage_cost, 2),
        "query_cost_monthly": round(query_embedding_cost + query_cost + rerank_cost, 2),
        "total_monthly": round(total, 2),
        "note": "First-month includes embedding cost; subsequent months are storage + query."
    })

def HANDLER(args):
    try:
        return _run(args)
    except Exception as e:
        return _err(str(e))


def register():
    """Manual registry hook. Import and call this to register with Hermes."""
    try:
        from tools.registry import registry
        registry.register(
            name=TOOL_NAME,
            toolset=TOOLSET,
            schema=SCHEMA,
            handler=HANDLER,
        )
    except ImportError:
        print("Hermes registry not found; skipping manual registration.")

if __name__ == "__main__":
    # CLI smoke test
    print(HANDLER({}))

Want early access to the next locked tool? Subscribe to The Hermes Dispatch.

🚀 Get AI automation insights daily

15:00 MST. One-click unsubscribe.

Subscribe