#

semantic-cache

Here are 379 public repositories matching this topic...

maximhq / bifrost

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

Updated Nov 2, 2026
Go

codefuse-ai / ModelCache

A LLM semantic caching system aiming to enhance user experience by reducing response time via cached query-result pairs.

llm semantic-cache

Updated Jun 30, 2025
Python

redis / redis-vl-python

Redis Vector Library. The AI-native Python client for Redis.

python search redis mcp openai embedding redis-search vector-search huggingface vector-database large-language-models llm anthropic semantic-cache retrieval-augmented-generation llmcache

Updated Nov 2, 2026
Python

ai-gateway

ferro-labs / ai-gateway

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

mcp guardrails pii-detection llm llmops semantic-cache prompt-management llm-proxy litellm ai-gateway ai-infrastructure llm-cost llm-gateway openai-compatible llm-strategy token-saver

Updated Nov 2, 2026
Go

SmarterRouter

peva3 / SmarterRouter

SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profiling, and automatic failover for local AI labs.

docker self-hosted model-serving gpu-monitoring fastapi llm openai-proxy semantic-cache local-llm ollama llm-proxy ollama-api ai-gateway llm-router self-hosted-ai ai-cache

Updated May 10, 2026
Python

888newstep / ai-agent-platform

企业级 AI Agent 平台 | Spring Boot 3 + LangChain4j | ReAct 推理 + 多路召回 RAG + 语义缓存 + 多智能体协作

java docker spring-boot rag milvus ai-agent llm semantic-cache retrieval-augmented-generation langchain4j react-agent

Updated Nov 1, 2026
Java

deepmemteam / deepmem

Drop-in AI memory layer with 2x faster retrieval and 10x lower cost. Fully compatible with Mem0 API. Migrate in 5 minutes without any code changes. Self-host for free.

ai mcp memory-management ai-agents long-term-memory rag fastapi vector-database hybrid-search qdrant llm semantic-cache llm-memory ai-memory memory-system agent-memory ai-agent-memory mem0-alternative mem0-compatible

Updated Aug 17, 2026
Python

sensoris / semcache

Semantic caching layer for your LLM applications. Reuse responses and reduce token usage.

gemini openai llm anthropic semantic-cache genai

Updated Jan 2, 2026
Rust

aqstack / mimir

mimir is a drop-in proxy that caches LLM API responses using semantic similarity, reducing costs and latency for repeated or similar queries.

kubernetes golang caching proxy openai cost-optimization llm semantic-cache

Updated Dec 24, 2025
Go

vcache-project / vCache

Reliable and Efficient Semantic Prompt Caching with vCache

Updated Jul 30, 2026
Python

firish / webfetch

Own your LLM's web search: a local search->fetch->rank pipeline that replaces hosted web-search tools. Measured: matches hosted accuracy at 66% lower cost and up to 88% fewer tokens, plus a precision-tuned semantic caching with query-dependant TTL that no API offers.

mcp openai search-algorithms agents search-api web-search nli rag llm anthropic semantic-cache llm-tools agentic-workflow agent-skills agent-memory

Updated Jul 23, 2026
Python

RAMen

Rohit-Dnath / RAMen

RAMen is a fast in-memory data store like Redis, but built for AI: drop-in Redis protocol, native vector search, semantic caching, and a built-in MCP server for agents. Single Go binary, BSD-3.

redis golang mcp cache resp ai-agents key-value-store vector-search vector-database llm semantic-cache mcp-server redis-alternative valkey-alternative-in-memory-database

Updated Oct 22, 2026
Go

shinefuer / frugent

Cost-efficient coding-agent runtime: prefix-aware prompt-cache routing, poisoning-safe semantic cache, budgeted context compression, request batching/dedup, cascading model routing with DP-optimal policies, hard budget scheduling, span-level cost attribution, sandboxed tools, synthetic workload factory. NumPy core, CPU-only torch extra.

python numpy sandboxing observability cost-optimization prompt-caching semantic-cache llm-agents context-compression coding-agent token-budget model-cascade

Updated Oct 30, 2026
Python

GuglielmoCerri / khazad

Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets.

python redis embeddings gemini openai vector-search llm anthropic semantic-cache

Updated Jul 10, 2026
Python

redis-developer / adk-redis

Redis integration for Google Agent Development Kit (ADK) - Memory, Sessions, Search Tools, MCP

redis memory semantic-search adk long-term-memory vector-search hybrid-search semantic-cache mcp-server agent-memory session-memory google-adk adk-python redis-agent-memory

Updated Aug 20, 2026
Python

ashishpatel26 / omnicache-ai

Unified multi-layer caching library for AI/agent pipelines — LangChain, LangGraph, AutoGen, CrewAI, Agno, A2A

agent embeddings caching-strategies autogen rag caching-memory agno langchain semantic-cache langchain-python crewai langgraph aiagents llm-cache ai-cache gptcache-alternative

Updated Jun 4, 2026
Python

AlphaBitCore / nexus-gateway

Enterprise AI traffic gateway — unified compliance, routing across 20+ LLM providers, semantic cache, quotas, and audit. SDK / network / OS-layer intercept.

Updated Sep 9, 2026
Go

redis / redis-vl-java

Redis Vector Library (RedisVL) -- the AI-native Java client for Redis.

java redis ai embeddings vectors rag vector-search vector-database llm generative-ai semantic-cache llm-cache rag-chatbot semantic-routing agentic-ai

Updated Jun 5, 2026
Java

Das-rebel / a3m-router

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel ensemble merge. npm: adaptive-memory-multi-model-router

Updated Oct 21, 2026
TypeScript

Harras3 / Enterprise-Grade-RAG

This is a RAG based chatbot in which semantic cache and guardrails have been incorporated.

guardrails semantic-cache retrieval-augmented-generation

Updated Nov 11, 2024
HTML

Add this topic to your repo

To associate your repository with the semantic-cache topic, visit your repo's landing page and select "manage topics."