Skip to content
#

fp8-quantization

Here are 9 public repositories matching this topic...

Language: All
Filter by language
virtio-accel

Transport-neutral, vendor-agnostic virtual accelerator devices. First class cross-platform NPU backend support for Apple, Intel, AMD, Qualcomm devices, as well as Vulkan for cross-platform GPU support.

  • Updated Oct 23, 2026
  • Rust

A measured study of vLLM inference optimisation on Qwen3-8B - 145× throughput through ten single-flag changes, with accuracy, latency, and cost data for every step.

  • Updated Nov 16, 2026
  • Python

Production-grade disaggregated LLM prefill-decode KV-cache transport & FP8/INT8 compression engine in C++20/CUDA 12.8/AVX2 SIMD. Achieves 50.0% network bandwidth reduction, 0.14 ms P50 block transfer latency, 0.64 ms P99 SLA, and zero prefill TPOT interference.

  • Updated Nov 18, 2026
  • C++

Developed a foveated VLM search system using Qwen3-VL-2B, LoRA, vLLM, and FP8 quantization. Benchmarked 4 search strategies on 500 COCO queries; fine-tuning achieved 85.6% accuracy with 1.1% false positives. Reduced vision tokens 17.2x and latency 45% on 1920×1440 images using adaptive thumbnail-and-crop search.

  • Updated Nov 2, 2026
  • HTML

Add this topic to your repo

To associate your repository with the fp8-quantization topic, visit your repo's landing page and select "manage topics."

Learn more