llama-cpp-alternative

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

Run Qwen3.8-27B at 140 tok/s on a single RTX 3090, up to 2x llama.cpp. Fast local LLM inference server with OpenAI- and Anthropic-compatible APIs, 256K context, native Windows and Linux.

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

C inference engine; own loader, kernels, and tokenizer. Runs hako-sho (3B) and hako-koi (7B) on CPU. No llama.cpp, no ollama.

From-scratch Metal inference engine that runs DeepSeek-V4.1-Flash (233 GB GGUF) with 200K context on a 64 GB Mac — 7x faster prefill, 12x faster decode than llama.cpp. OpenAI-compatible server + web UI.