TGI is archived, Docker Model Runner changed local inference, and PRO's inference credits are $2 not 2 million. A 2026 field guide to running and serving LLMs — starting with the hardware you already own.
A chatbot once sold a Chevy Tahoe for a dollar because someone asked nicely. Here is what it took to put a real prompt-injection filter in front of my local LLM stack — and the one-character trick that walks straight past it.
“One more model won't hurt.” That sentence has probably caused more home-lab incidents than expired certificates, broken Docker volumes, and mysterious YAML indentation combined. It certainly caused mine. My NVIDIA DGX Spark was already busy. It was running chat models, an embedding service, speech-to-text, text-to-speech, vector search, automation
James Bond was my first memory. Watching my son play IO Interactive's 007 First Light handed the franchise down a generation — THEIA, the bluff, a reimagined Moneypenny, and why the games remember the fun the movies forgot.
Ever since I watched Iron Man, I have wanted my own JARVIS. Not necessarily the flying armor. Although, to be clear, I would accept the flying armor. What I really wanted was the assistant: something that could listen to me, understand my occasionally drunk-sounding broken English, remember previous conversations, search
Do you still open Google Analytics every Monday to see if your website’s traffic went up? Yeah… same here — until I automated the whole thing. In this post, we’ll create a workflow that fetches your site metrics from Google Analytics, formats them into clean HTML, and emails them
If you’ve ever watched your GPU twiddle its thumbs between prompts, this one’s for you. In this post we’ll cover what vLLM is, why it’s fast, how to run it with Docker Compose, and how to test it with real calls. I’ll also show concrete
Why Quantization Matters Large Language Models (LLMs) are huge. Even a “small” 7B parameter model can chew up 14+ GB in FP16 (16-bit floating point). If you’ve tried running one locally without a beefy GPU, you’ve probably noticed your machine crying in pain—or worse, swapping memory like
Opening AI harnesses and IDE extensions are evolving fast, and Microsoft’s Model Context Protocol (MCP) is emerging as the glue that makes custom tools discoverable and usable by editors like VS Code and Visual Studio Insiders. For this post, I’ll be using .NET 10 along with Visual Studio
If you've been playing with local LLMs and you're already a fan of automation tools like n8n, you're in for a treat. Today, we'll connect n8n to your local Ollama instance (with Open WebUI) using Docker Compose. The result? Automated AI workflows
