#20
Colibrì (JustVugg/colibri) is a lightweight, pure-C inference engine designed to run GLM-5.2, a massive 744-billion-parameter Mixture-of-Experts (MoE) language model, on consumer hardware with as little as ~25 GB of RAM — no GPU required, no Python runtime, zero external dependencies. The core insight is that MoE models only activate roughly 40B parameters per token, meaning the model doesn't need to fit in fast memory but instead needs to be intelligently *placed* across a three-tier storage hierarchy: VRAM, RAM, and NVMe disk. The dense components (~17B params, ~9.9 GB at int4) stay resident in RAM, while the 19,456 routed experts (~370 GB total) are streamed on demand from disk using a per-layer LRU cache, a learned hot-store, and optional VRAM pinning. The engine is described as "a JIT, but for weights" — it learns which experts your workload routes to, pins the hottest ones automatically, and literally gets faster the more you use it via a persistent usage file. Key engineering features include router-lookahead prefetching (routing is 71.6% predictable one layer ahead), async I/O with batch-union deduplication, dual-SSD mirroring for doubled read bandwidth, O_DIRECT support, and a NUMA-aware multi-socket mode. Speculative decoding via GLM-5.2's native MTP head achieves 2.2–2.8 tokens per forward pass, with hard-won defaults around int8 draft heads and kernel consistency. The forward pass is validated token-exact against a Hugging Face Transformers oracle. A rich web dashboard visualizes live token metrics, per-tier expert residency, and a stunning "Brain" page showing all 19,456 experts as a living cortex with routing heat and topic affinity. Performance ranges from 0.05 tok/s on a 25 GB dev box to 5.8–6.8 tok/s on six RTX 5090s with full expert residency. The project supports Linux, macOS, and Windows, with an experimental Metal backend for Apple Silicon. It is notably early-stage but already versioned at v1.1.0,
ServingModelsAppslocal-firston-device
★ 19,706 total+3,158 this week
#54
Free Claude Code (FCC) is an open-source Python-based local proxy server that allows developers to run AI coding agents — Claude Code, OpenAI Codex, and Pi — through their own choice of AI provider backend, rather than being locked into Anthropic's default billing. The core idea is straightforward: FCC intercepts the API calls these coding agents make and reroutes them to whichever cloud or local model provider the user configures, enabling access to free tiers, cheaper alternatives, or fully local models. The project supports an impressive roster of 25+ providers, including NVIDIA NIM, OpenRouter, Google AI Studio (Gemini), Google Vertex AI, DeepSeek, Mistral, Amazon Bedrock, Hugging Face, Cohere, GitHub Models, Kimi, MiniMax, Cerebras, Groq, SambaNova, Fireworks AI, Cloudflare Workers AI, Ollama, LM Studio, and llama.cpp, among others. Configuration is managed through a local browser-based Admin UI, where users can paste API keys, search and select models from a dropdown, validate connections, and apply settings without touching config files manually. A notable feature is optional model-tier routing, which lets users map Claude Code's internal tiers (Fable, Opus, Sonnet, Haiku) to different provider models independently, enabling fine-grained cost and capability tradeoffs. The proxy preserves streaming, tool use, reasoning, and image input across compatible models, and exposes native model pickers inside both Claude Code and Codex. Installation is handled via a single curl or PowerShell command, with desktop launchers on Windows and macOS for background operation via system tray or menu bar. Linux users run a terminal command. The project also supports optional integrations including VS Code extensions, JetBrains ACP, Discord bots, Telegram bots with voice-note transcription, and token-based authentication for the local proxy. The codebase uses modern Python tooling: Python 3.14, uv for package management, Ruff for formatting, ty for type checking, Loguru for logging, and Pytest for testing. FCC is MIT-licensed and actively maintained with CI workflows. Its trending appeal is clear: it democratizes access to powerful AI coding agents by decoupling them from expensive proprietary APIs, letting developers use free-tier cloud models or fully local LLMs. This positions it squarely at the intersection of the AI coding agent boom and the growing demand for cost-effective, self-hosted AI infrastructure — making it highly relevant to developers watching GitHub trending for
ServingToolsAppscodinglocal-firstmultimodal
★ 42,425 total+1,401 this week
#78
9Router is a free, self-hosted AI router and "token saver" designed to sit between AI coding CLI tools and dozens of AI model providers, ensuring developers "never stop coding" while minimizing cost. It functions as a smart local proxy—running by default at http://localhost:20128—that connects popular AI code tools such as Claude Code, Cursor, Antigravity, GitHub Copilot, Codex, Gemini, OpenCode, Cline, and OpenClaw to over 40 AI providers and more than 100 models. The project's central value proposition is solving common pain points in AI-assisted development: unused monthly subscription quotas that expire, rate limits that interrupt work mid-session, expensive per-provider APIs costing $20–50/month, tool outputs (like git diff, grep, and ls) that rapidly consume tokens, and the tedium of manually switching between providers. Its headline feature is "RTK Token Saver," which automatically compresses tool_result content to save 20–40% of tokens per request. Beyond token savings, 9Router offers automatic tiered fallback that cascades requests from subscription-based tiers (Claude Code, Codex, GitHub Copilot) to cheap providers (GLM at ~$0.6/1M, MiniMax at ~$0.2/1M) to fully free tiers (Kiro, OpenCode Free, and Google Vertex with $300 credits), ensuring zero downtime as quotas exhaust. It also provides quota tracking to maximize subscriptions before reset, multi-account round-robin load balancing per provider, automatic token refresh, and format translation between OpenAI and Claude API schemas, making it broadly compatible with virtually any CLI tool through an OpenAI-compatible /v1 endpoint. Installation is straightforward via npm (`npm install -g 9router`), which launches a dashboard where users can connect a free provider like Kiro AI (offering ~50 credits/month free including Claude 4.5, GLM-5, and MiniMax) or OpenCode Free (no authentication required), then configure their CLI tool with the local endpoint, an API key copied from the dashboard, and a model such as kr/claude-sonnet-4.5. Because the repository package is private (named 9router-app), running from source or Docker is the expected local development path, supported by both development (`npm run dev`) and production (`npm run build`/`npm run start`) workflows using environment variables. The project is distributed through npm, Docker Hub, and GitHub Container Registry (GHCR), and it maintains internationalized documentation in Vietnamese, Chinese, Japanese, Russian, Thai, and Persian. Its maturity and traction are signaled by npm download badges, Docker pull counts, a Trendshift repository badge, and an extensive community-driven collection of video guides in multiple languages (Vietnamese, English, Urdu/Hindi, Indonesian, and Persian) produced by creators like Mì AI, Build AI With Hamid, CodeVerse Soban, Mai Gia, Krisswuh, Matin SenPai, and tuanminhhole, with contributors invited to submit their own videos via pull request. The tool supports a wide array of AI coding clients including Claude Code, OpenClaw, Codex, OpenCode, Cursor, Antigravity, Cline, Continue, Droid, Roo, Copilot, and Kilo Code, and integrates OAuth-based providers alongside API-key providers. In the context of AI agents, 9Router is notable because it acts as unified infrastructure enabling agentic coding tools and automated workflows (including examples like building automated Zalo AI bots with OpenClaw) to run cost-effectively and continuously across heterogeneous model backends without hitting limits. Its combination of aggressive token compression, intelligent
ServingToolsAppscodinglocal-firstworkflow-automation
★ 23,681 total+955 this week
#79
**llmfit** is a terminal tool (written in Rust) that helps users identify which LLM models will actually run well on their specific hardware by detecting RAM, CPU, and GPU/VRAM specs and scoring hundreds of models across memory fit, speed, quality, and context dimensions. It ships with an interactive TUI (default) and a classic CLI mode, supporting multi-GPU setups, MoE architectures, dynamic quantization selection, and local runtime providers including Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio. A standout feature is its community benchmarking system: users can measure real tok/s and TTFT on their hardware and submit results as PRs directly from the TUI—no external CLI or third-party account required—so verified measurements progressively replace estimates for identical hardware configurations. The project is actively maintained, published on crates.io, installable via Homebrew, Scoop, MacPorts, pip/uv, Docker, or a curl script, and has Windows binaries signed via SignPath.io. Its relevance to GGUF stems from its support for quantization-aware model fitting and llama.cpp as a backend, making it a practical discovery and planning tool for the local LLM/GGUF ecosystem. Sister projects extend its utility into model serving and Kubernetes agent management.
ServingModelsAppslocal-firston-device
★ 30,708 total+927 this week
#93
FreeLLMAPI is an open-source aggregation layer that exposes a single OpenAI-compatible `/v1` API endpoint on top of 28 free LLM providers, spanning roughly 235 model families and 339 free model endpoints that together add up to approximately 4 billion tokens per month of inference capacity. The core problem it solves is that individual free tiers from AI labs are small and painful to combine by hand—each has its own SDK, rate limits, and failure modes—so FreeLLMAPI collapses all of them behind one interface. Users point any OpenAI-compatible client library at their local server, and the router transparently routes requests across whichever providers they've added keys for. A key differentiator is a self-updating, signed model catalog synced from freellmapi.co twice a day, so new models, quota changes, and provider quirk fixes land automatically without a `git pull`. The project supports major providers including Google (Gemini), Groq (Llama, GPT-OSS, Qwen3), Cerebras, OpenCode Zen, Mistral, OpenRouter, Cloudflare, Cohere, Z.ai/Zhipu (GLM models), NVIDIA NIM, and HuggingFace, plus a custom provider for any OpenAI-compatible endpoint like llama.cpp, LM Studio, vLLM, or Ollama for chat, embedding, image, or audio models. Feature-wise, it is broad and mature: it implements the OpenAI chat completions and models endpoints, the Responses API (required by current Codex CLI versions) as a translating shim, a legacy completions endpoint for editor autocomplete (Continue ghost-text), and the Anthropic Messages API so Claude Code and Anthropic SDKs can run against the free pool with content-negotiated model listings. A notable "Fusion" feature fans a prompt out to a panel of diverse free models in parallel, then uses a judge model to synthesize a single answer, with configurable panel, judge, and strategy. It supports image generation and text-to-speech routing, streaming (SSE) and non-streaming responses, OpenAI-style tool calling with automatic rescue of text-emitted tool calls into structured JSON, structured outputs (json_object/json_schema translated to Gemini's responseSchema), and full sampling parameter passthrough (seed, top_k, min_p, penalties, logit_bias, logprobs) with per-provider dropping of unsupported params. Embeddings routing is family-based with strict same-model failover to preserve vector compatibility. Reliability is a major focus: automatic failover on 429/5xx/timeout errors with cooldowns and up to 20 retry attempts bounded by a wall-clock budget, sibling key rotation for dead keys, and full attempt trails on exhaustion. Its smart routing offers six selectable strategies (priority, balanced, smartest, fastest, reliable, custom) driven by live per-model measurements and a Thompson-sampling bandit. Unified models collapse the same logical model served by multiple providers into one entry with in-group failover, and model profiles let users save named fallback chains for coding, long-context, or vision use cases. Per-key rate tracking maintains RPM, RPD, TPM, and TPD counters per platform/model/key and even learns tighter ceilings from provider error bodies and quota headers. Sticky sessions keep multi-turn conversations on the same model for 30 minutes to avoid mid-conversation hallucination spikes, and an opt-in in-memory LRU response cache offers exact-match caching with SHA-256 keys, TTL/temperature gates, and zero provider quota on hits. Security is emphasized: provider API keys are encrypted with AES-256-GCM before hitting SQLite and decrypted only in memory; clients authenticate with a single unified `freellmapi-…` bearer token so upstream keys are never exposed. There is bulk key import/export (.env, JSON, CSV), periodic health checks that classify keys as healthy/rate_limited/invalid/error, and a React + Vite admin dashboard for managing keys, re
ServingMCPToolsmultimodallocal-firstsecurity
★ 17,114 total+683 this week
#105
Agnes AI is an OpenAI-compatible API gateway and model catalog developed by AgnesAI Labs, offering developers unified access to multimodal AI models spanning text, vision, image generation, and video generation. The platform exposes four primary models: `agnes-2.0-flash` (text and vision-language with 256K context, tool calling, and agent workflows), `agnes-image-2.0-flash` and `agnes-image-2.1-flash` (text-to-image and image editing), and `agnes-video-v2.0` (asynchronous text-to-video and image-to-video generation). Integration is straightforward via the OpenAI Python SDK or raw HTTP, making migration from existing OpenAI-based stacks minimal. The repository serves as the official documentation hub, including a model catalog, changelog, error code guides, FAQ, and bilingual (English/Chinese) support resources. Access tiers range from Free to Enterprise and Token Plans, with RPM and quota limits documented as of June 2026. Notable for its broad multimodal scope, OpenAI drop-in compatibility, and structured developer documentation, Agnes AI appears to be an emerging commercial AI API platform targeting developers building production agent and generative media workflows.
ServingAppsAgentsmultimodalvisionvideo
★ 1,886 total+595 this week