Popular repositories Loading

1. ioteverythin-display ioteverythin-display Public

   Touch-i: A wall-mounted Home Assistant control panel on the Waveshare ESP32-S3-Touch-LCD-4. Switches, Climate, Media, Presence. HACS integration with zero-code setup.

   C 5 1

2. speculative-tools speculative-tools Public

   Speculative execution engine for LLM agent tool calls. Predicts and pre-executes tools in parallel to reduce agent latency.

   Python 1

3. injectguard injectguard Public

   Statistical prompt injection detection for LLM applications. Ensemble of 10 heuristic features with calibratable weights, achieving 96% recall at 89% precision in under 100 microseconds per check. …

   Python 1

4. prefixroute prefixroute Public

   Prefix cache affinity routing for multi-replica LLM serving. Routes each request to the replica already holding its prompt prefix, with a load guard against hotspots, cutting mean time to first tok…

   Python 1

5. kvprefetch kvprefetch Public

   Decode-latency planner for offloaded KV cache: double-buffered layer-ahead prefetch cuts Llama-3-8B decode step latency 37.7 percent at 2k context (17.6 to 11.0 ms) on an A100 PCIe card, and locate…

   Python 1

6. moecache moecache Public

   Locality-aware expert caching with co-activation prefetch for offloaded fine-grained MoE inference. Pin-plus-prefetch lifts expert cache hit rate from 79.6 to 85.3 percent over LRU at equal GPU mem…

   Python 1