You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Transport-neutral, vendor-agnostic virtual accelerator devices. First class cross-platform NPU backend support for Apple, Intel, AMD, Qualcomm devices, as well as Vulkan for cross-platform GPU support.
A measured study of vLLM inference optimisation on Qwen3-8B - 145× throughput through ten single-flag changes, with accuracy, latency, and cost data for every step.
Production-grade disaggregated LLM prefill-decode KV-cache transport & FP8/INT8 compression engine in C++20/CUDA 12.8/AVX2 SIMD. Achieves 50.0% network bandwidth reduction, 0.14 ms P50 block transfer latency, 0.64 ms P99 SLA, and zero prefill TPOT interference.
Developed a foveated VLM search system using Qwen3-VL-2B, LoRA, vLLM, and FP8 quantization. Benchmarked 4 search strategies on 500 COCO queries; fine-tuning achieved 85.6% accuracy with 1.1% false positives. Reduced vision tokens 17.2x and latency 45% on 1920×1440 images using adaptive thumbnail-and-crop search.