https://slavadubrov.com/Practical AI engineering by Slava Dubrov.enhttps://slavadubrov.com/blog/2026/10/08/agent-to-workflow/https://slavadubrov.com/blog/2026/10/08/agent-to-workflow/# Agent or workflow: five designs for one support assistant This article is for engineers who are building a feature on a language model and deciding whether it should be an agent, a workflow, or plain code with a model at one step. You will learn how to decide that from the task, and you will see five designs of the same assistant run on the same tests: three built around an agent, one with a router in front of several agents, and one with no agent at all. The example is a customer-support assistant for an online shop. It handles refunds, address changes and preference changes. [Part 1](/blog/2026/09/28/first-langchain-agent-harness/) built it as one LangChain agent, but you do not need Part 1 to follow this article. All results come from the [companion demo](https://github.com/slavadubrov/agent-harness-lab-public/tree/v0.2.1-a2), and each is a single run.Thu, 08 Oct 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/09/28/first-langchain-agent-harness/https://slavadubrov.com/blog/2026/09/28/first-langchain-agent-harness/# Designing an agent harness: from task to architecture An **agent harness** is the code around a language model that lets it complete a task. It gives the model instructions and data, runs the actions the model asks for, keeps track of progress and decides when the task is finished. This article is for engineers who are adding an agent to their own application. It shows how to design that code from the task: which steps the model chooses, which steps your code fixes in advance, and which rules your code enforces whatever the model says. Make these decisions before you choose an agent framework, because the answers differ from task to task. A support assistant changes customer accounts, so it needs permission checks and a safe way to retry a payment. A coding agent edits files and runs tests, so it needs a place where its code can run without harming anything else. A document classifier only returns a label, so it may need no tools at all. If all three start from the same loop, memory store and tool set, nobody asks what each task needs. We will follow these decisions for one example, a customer-support assistant that issues refunds, from its task description to a first implementation. The article builds on [Harness Engineering for AI Agents](/blog/2026/07/22/ai-agent-harness-engineering/), which describes what a harness is responsible for. Here we choose those responsibilities for one project.Mon, 28 Sep 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/09/12/from-traces-to-better-memory/https://slavadubrov.com/blog/2026/09/12/from-traces-to-better-memory/# Meta-engineering AI systems: from traces to better memory This article is for engineers who want an agent to find out why their AI system fails and to propose a fix. The example is a small memory tool that answers a question about where a user lives, and the answer is wrong. You will see how a change to that tool is proposed, tested on 20 prepared scenarios, and judged before anyone adopts it. You will also see how to check whether a plain script that tries every setting would have found the same fix without an LLM. In the [companion experiment](https://github.com/slavadubrov/meta-engineering-ai-lab/tree/v0.2.2), an LLM proposed the changes and ordinary Python tested them. I use **meta-engineering** for the design of that improvement process. It comes down to four decisions. What may the **proposer**, the person or program that suggests changes, change? Which results count as an improvement? How does a result shape the next suggestion? Who decides to adopt a change? The same four decisions apply whether the proposer is an LLM, a person or a parameter search.Sat, 12 Sep 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/08/15/typescript-for-python-ml-engineers/https://slavadubrov.com/blog/2026/08/15/typescript-for-python-ml-engineers/# TypeScript for Python ML engineers: build an agent service This guide is for experienced Python ML and backend engineers who need to build an AI service in TypeScript and Node. It starts from Python service concepts, not from basic programming or browser code. By the end, you can map a Python service stack to TypeScript, recognize the Python habits that cause JavaScript bugs, and run and change a streaming agent service. The example service follows a request through validation, tool execution, HTTP streaming, and persistence. It answers three questions: what happens when tool arguments arrive in fragments, a stream fails after HTTP 200, or a worker claims the same job twice? The scope is backend and AI work only: no React and no browser bundler. The code is in [**slavadubrov/typescript-agent-service**](https://github.com/slavadubrov/typescript-agent-service). It includes two agent-loop implementations, Postgres run history, a worker, and an MCP server. You can run its offline demo without an API key, then use the Python-to-TypeScript map below to read and modify the service. Heavy numerics stay in Python; this guide covers orchestration, HTTP, and streaming.Sat, 15 Aug 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/07/22/ai-agent-harness-engineering/https://slavadubrov.com/blog/2026/07/22/ai-agent-harness-engineering/# Harness Engineering for AI Agents: Verifying Completion An agent can finish its turn while the work is still incomplete. This article is for engineers building or reviewing coding agents who need to decide when to accept a result, retry an uncertain operation, or resume work in another session. It also shows how to measure whether an extra harness component improves those outcomes enough to justify its cost. The **harness** is the application code around the model's reasoning loop. It supplies context, checks proposed actions, records what happened, and decides whether the evidence is enough to finish. The [runtime](/blog/2026/05/26/ai-agent-runtime/) executes accepted actions and keeps the session, sandbox, and checkpoint available. The earlier articles explain the reasoning loop, [memory](/blog/2026/02/14/ai-agent-memory-architecture/), [tools](/blog/2026/03/24/ai-agent-tool-use/), and [permissions](/blog/2026/04/20/ai-agent-security/). Here the question is what the harness does **after** the model proposes an action or says "done." The store and payment examples below are fictional teaching cases. The companion lab is a deterministic simulation; it does not measure a live coding agent.Wed, 22 Jul 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/07/05/model-quantization-in-2026-from-foundations-to-production-serving/https://slavadubrov.com/blog/2026/07/05/model-quantization-in-2026-from-foundations-to-production-serving/# Model Quantization Guide: Foundations to Production Serving This quantization guide is for engineers deploying language models who need to reduce memory use or improve serving speed. You will learn how to choose what to compress, match a number format to your hardware and runtime, and compare memory, latency, and output quality before deployment. It also covers the separate checks needed for diffusion image models. Quantization reduces the bits used for [weights, activations, or the KV cache](https://nvidia.github.io/TensorRT-LLM/features/quantization.html). **Weights** are the model parameters learned during training. **Activations** are intermediate values computed from each input. The **key-value (KV) cache** keeps values from earlier tokens for reuse during generation. Compressing weights can make a model fit in GPU memory while leaving too little room for a long conversation. The target you compress and the GPU code that executes it determine which limit you relieve. Start with what makes the request slow or too large. Processing the input prompt is called **prefill**; generating the answer one token at a time is **decode**. Prefill can be limited by GPU arithmetic, while decode can be limited by how fast weights load from memory. Model weights and the growing KV cache can also exhaust GPU memory. The first table maps these problems to possible fixes; later sections explain the arithmetic and checks needed before deployment. A lower bit-width is only a candidate. The decision needs both task-quality results and serving measurements for the model, runtime, and traffic you will use. The benchmarks here are published results from other teams, plus one short run from the companion repository.Sun, 05 Jul 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/06/20/schema-guided-agent-memory/https://slavadubrov.com/blog/2026/06/20/schema-guided-agent-memory/# AI Agent Memory: Schema-Guided State and Provenance This article is for engineers building long-running or multi-tenant agents. Plain semantic memory has no rule for which value of a fact is current, so agents return stale facts. When a user changes a passport deadline from July 15 to June 30, vector search can retrieve both statements. A stateful memory layer must record that the June 30 value supersedes the earlier one. You will learn how to store memory as typed records with validity windows, tenant scope, and source links, write a correction without losing history, and test current and point-in-time reads.Sat, 20 Jun 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/06/10/agent-evals-traces-to-test-suites/https://slavadubrov.com/blog/2026/06/10/agent-evals-traces-to-test-suites/# AI Agent Evaluation in Production: Traces to Test Suites This guide is for engineers evaluating tool-using AI agents in production. You will learn how to turn failed runs into repeatable tests, check the agent's actions as well as its answers, and use those tests before the next release.Wed, 10 Jun 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/05/26/ai-agent-runtime/https://slavadubrov.com/blog/2026/05/26/ai-agent-runtime/# Long-Running AI Agent Runtime: Sessions and Checkpoints A long-running **agent runtime** lets a tool-using run outlive the worker that started it. This article is for engineers moving an agent from a request handler to background work. You will learn which state must survive a restart, why a checkpoint cannot make an external action safe to retry, and how those requirements determine the deployment shape. It then compares concrete options: a queue and worker, Temporal, sandbox providers, Cloud Run, Lambda, Fargate, Kubernetes, and hosted agent platforms, with the observability to debug a run after it fails. The model still proposes the next action. The [harness in Part 6](/blog/2026/07/22/ai-agent-harness-engineering/) supplies context, authorizes tools, and checks completion. The runtime gives that loop durable state and a place to execute. [Part 4](/blog/2026/04/20/ai-agent-security/) covers the permission decision; here we follow an approved action through a crash and recovery.Tue, 26 May 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/05/10/rag-evaluation-metrics/https://slavadubrov.com/blog/2026/05/10/rag-evaluation-metrics/# RAG Evaluation Metrics: Retrieval, Reranking, Generation For engineers operating or evaluating retrieval-augmented generation (RAG) systems, this reference maps failures in document parsing, filtering, retrieval, reranking, and generation to the metric that identifies each one. It also shows which checks run before release and which monitor live traffic. A RAG system with broken relevance filters can run for months without triggering an operational alert. It still returns answers and meets its latency target, but the answers rely on incomplete evidence. Recall@k against the original eligible gold set exposes the loss. Latency and availability dashboards do not.Sun, 10 May 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/04/20/ai-agent-security/https://slavadubrov.com/blog/2026/04/20/ai-agent-security/# AI Agent Security: Permissions, Sandboxes, and MCP Threats This guide is for engineers giving an AI agent access to files, code execution, or external services. It explains how to decide whether a proposed action may run, where to enforce that decision, and what a sandbox or content filter can still contribute. We will trace those checks through the [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent), the companion used throughout this series, and through the policy surfaces of Claude Code, Codex CLI, the OpenAI Agents SDK, LangGraph, and MCP. [Part 3](/blog/2026/03/24/ai-agent-tool-use/) asked which actions to expose as tools. Security begins with the next question: _who may use each action, on which resource, for this request?_ A model can propose an operation. The application must establish the caller's authority before executing it.Mon, 20 Apr 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/04/02/ner-guide/https://slavadubrov.com/blog/2026/04/02/ner-guide/# NER Guide 2026: GLiNER, spaCy, Transformers, and LLMs This guide is for engineers choosing named entity recognition (NER) for retrieval, privacy, or document-processing pipelines. It explains how to choose between span recognizers and structured LLM extraction, evaluate their errors, and test deployment options against the latency and cost your workload permits. NER turns text into labeled pieces that a program can use. Given “Bill Gates founded Microsoft on April 4, 1975,” it can return the contiguous text spans `Bill Gates`, `Microsoft`, and `April 4, 1975` with the types person, organization, and date. In a span-based encoder such as GLiNER, the model reads the text once and scores candidate spans against the labels you supply. For a retrieval or privacy pipeline, I would first ask whether the output must be a literal mention. That decision separates a span recognizer from systems that also normalize values, link records, or infer facts. LLMs remain useful for proposing training annotations and handling cases that require inference or normalization. > **Companion repo:** [**ner-field-guide**](https://github.com/slavadubrov/ner-field-guide) provides exact span scoring, GLiNER examples, reviewed annotation conversion, ONNX FP32 and INT8 checks, and a streaming PII demo.Thu, 02 Apr 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/24/ai-agent-tool-use/https://slavadubrov.com/blog/2026/03/24/ai-agent-tool-use/# AI Agent Tool Use: MCP, CLI, Skills, and Code Execution This article is for engineers giving an agent access to data, commands, or external services. You will learn how to define an action the agent can use reliably, choose an execution interface for it, and check whether that interface helps on your tasks. The running example is the [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent), a research agent that fetches market data and prepares reports. In [Part 1](/blog/2026/01/31/ai-agent-reasoning-loops/), the model chose a next step in a reasoning loop. [Part 2](/blog/2026/02/14/ai-agent-memory-architecture/) dealt with what it can retain between steps. A tool gives that choice an effect outside the model: it can fetch a quote, read a saved report, execute code, or change a record. The program around the model—the **harness**—still decides which of those actions exist and whether a proposed call may run. The first design decision is the action itself. If the agent needs price and valuation data for one report, should it request two API endpoints, one stock snapshot, or a generated program? The answer depends on the task and the feedback the agent needs, not on the integration protocol.Tue, 24 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/13/llm-engineering-guide/https://slavadubrov.com/blog/2026/03/13/llm-engineering-guide/# LLM Engineering Guide: 45 Concepts for Production Systems This is a reference for engineers who already know basic ML and systems concepts and need to connect a production symptom to the relevant part of the stack. It covers 45 concepts across hardware, inference, training, deployment, applications, and operations. Each entry gives the mechanism, what it changes in a running system, and the condition under which the cited result holds. If you start from a symptom, such as a slow first token or OOM under load, use the symptom table below to find the entries to read.Fri, 13 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/03/04/ocr-guide/https://slavadubrov.com/blog/2026/03/04/ocr-guide/# OCR in 2026: Classical Pipelines, VLMs, and Document AI This guide is for engineers choosing OCR and document extraction for search, retrieval-augmented generation (RAG), or AI agents. It explains how traditional engines and vision-language models differ, how to evaluate text, tables, and fields, and how to route difficult documents to another model or human review. Vision-language models (VLMs) can handle layout, handwriting, tables, and degraded images that break a plain text-recognition pipeline. Traditional engines remain competitive on clean print, especially when CPU latency and operating cost matter. The dated snapshot below includes [PaddleOCR-VL 1.6](https://www.paddleocr.ai/main/en/version3.x/pipeline_usage/PaddleOCR-VL.html) and [dots.mocr](https://github.com/studio-dots-ai/dots.mocr), with different hardware, privacy, and output trade-offs. Check each project before making a current choice.Wed, 04 Mar 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/21/modern-data-processing-engines/https://slavadubrov.com/blog/2026/02/21/modern-data-processing-engines/# Data Processing Engines: Polars, DataFusion, Ray, and Spark This guide helps Python and data engineers choose a processing engine for tabular data and image, audio, or text pipelines. It explains how Polars, DataFusion, Daft, Ray Data, and Spark execute work, which workloads suit them, and how to compare them on your own data. Modern pipelines also process images, audio, and video. In these workloads, CPU decoding can starve GPU inference. Native operators, columnar data exchange, and overlapping CPU/GPU work address different costs; choosing an engine means finding which cost limits your pipeline. To compare the options, I benchmarked Pandas, Polars, DataFusion, Daft, and native Rust on two real datasets: NYC taxi trips represent tabular work, and Food-101 images represent a multimodal pipeline. Spark and Ray are covered in separate distributed notebooks. The [engine-comparison-demo repository](https://github.com/slavadubrov/engine-comparison-demo) contains the code.Sat, 21 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/14/ai-agent-memory-architecture/https://slavadubrov.com/blog/2026/02/14/ai-agent-memory-architecture/# AI Agent Memory Architecture: Checkpoints and Vector Stores An agent can appear to “forget” for several reasons. It may lose an unfinished run after a restart, fail to retrieve a preference from an earlier conversation, or follow an obsolete project note. Each failure needs a different kind of state and a different test. This article is for engineers designing that state: you will learn when to use a checkpoint, an exact lookup, semantic retrieval, or an inspectable document, and which tools fit each job, from PostgreSQL and Redis checkpoint savers to pgvector, Qdrant, Markdown files, and managed memory frameworks. [Part 1](/blog/2026/01/31/ai-agent-reasoning-loops/) built the reasoning loop. Here we will use the [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent/tree/6794a0d) to see what the loop must preserve when it pauses before publishing a report and when a user returns with known preferences.Sat, 14 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/02/08/search-ranking-stack/https://slavadubrov.com/blog/2026/02/08/search-ranking-stack/# Search Ranking Stack: BM25, Embeddings, and Reranking This guide is for Python engineers building or improving a search ranking pipeline. It explains how BM25, dense embeddings, rank fusion, and rerankers work together, and how to measure whether search needs better candidates or a better order. A search for "cheap laptop" can miss a product described as a "budget notebook computer." A different failure puts a headphone carrying case above the headphones themselves. The first problem needs better candidates; the second needs a better order. The [companion demo repository](https://github.com/slavadubrov/search-ranking-stack) illustrates retrieval and reranking on a sampled slice of Amazon ESCI product-search data.Sun, 08 Feb 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/31/ai-agent-reasoning-loops/https://slavadubrov.com/blog/2026/01/31/ai-agent-reasoning-loops/# AI Agent Reasoning Loops: ReAct, ReWOO, Plan-and-Execute An **agent reasoning loop** is the control flow that decides when a model chooses an action, receives a tool result, and chooses again—or stops. This article is for engineers deciding how an agent should handle work whose next step may depend on what its tools find. You will learn when to use **ReAct**, **ReWOO**, or **Plan-and-Execute**, and how those choices appear in a LangGraph implementation. A request for a stock's current price has known lookup steps. An investigation of why its outlook changed may require the next search to depend on the first result. The model needs different freedom in those two tasks, and the second usually needs more model calls. This is Part 1 of **Engineering the Agentic Stack**. The loop is its innermost layer; later parts cover memory, tools, security, runtime, and checks before a task is declared done. The worked example is a [Market Analyst Agent](https://github.com/slavadubrov/market-analyst-agent) built with LangGraph.Sat, 31 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/21/manifold-constrained-hyper-connections/https://slavadubrov.com/blog/2026/01/21/manifold-constrained-hyper-connections/# DeepSeek mHC: Manifold-Constrained Hyper-Connections Modern deep learning rests on the residual connection. This post is for engineers who know the Transformer block and want to know what mHC changes in the residual path, why, what it costs, and when to try it. Hyper-Connections (HC) explore another architectural dimension: widen the residual state into several interacting streams. DeepSeek's **[Manifold-Constrained Hyper-Connections (mHC)](https://arxiv.org/abs/2512.24880)** paper studies how to keep that routing stable at larger training scales. The post starts with standard residual connections, then adds Hyper-Connections and the instability they create. The mHC constraint, a small NumPy demo, and the implementation cost come after that.Wed, 21 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/11/enterprise-rag-challenge-agent-architectures/https://slavadubrov.com/blog/2026/01/11/enterprise-rag-challenge-agent-architectures/# Enterprise RAG Challenge 3: Lessons from Public Entries Enterprise RAG Challenge 3 (ERC3) asked agents to complete business tasks against a simulated company API. The frozen prize leaderboard is unusually useful because many entrants published more than a score: architecture, model mix, cost, and failure notes. I reviewed those public descriptions to answer a narrower question: which design choices recurred in strong submissions, and which of them are useful outside this benchmark? This article is for engineers building agents that call business APIs. You will see five designs from the public write-ups, the problems each one solved (permissions, pagination, missing links, context size), and an order in which to try them in your own agent.Sun, 11 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2026/01/04/llm-fine-tuning-guide/https://slavadubrov.com/blog/2026/01/04/llm-fine-tuning-guide/# LLM Fine-Tuning Guide: When to Fine-Tune, LoRA and QLoRA This guide is for Python engineers deciding whether and how to fine-tune a large language model. You will learn when to choose fine-tuning over prompting or retrieval and how to choose between methods such as LoRA and QLoRA. The guide also explains how to prepare data, evaluate the trained model, and plan its deployment. The practical section runs a companion repository that fine-tunes a 4B model for function calling.Sun, 04 Jan 2026 00:00:00 GMThttps://slavadubrov.com/blog/2025/12/28/schema-guided-reasoning-vllm/https://slavadubrov.com/blog/2025/12/28/schema-guided-reasoning-vllm/# Schema-Guided Reasoning: vLLM, XGrammar, and Pydantic This article is for Python engineers who need model output that downstream code can validate. Schema-Guided Reasoning (SGR) gives the model a fixed list of fields to fill, analysis first and decision last, so code can check each step. You will learn how to define a Pydantic schema, request structured output from vLLM, and add application checks for meaning and policy. Pydantic describes the response fields, vLLM serves the model, and XGrammar restricts which tokens the model can generate. This restriction is called constrained decoding. A token limit can still leave an incomplete JSON prefix, so application code must check completion and validate the result.Sun, 28 Dec 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/22/lorax-serving-guide-lora-adapters-kubernetes/https://slavadubrov.com/blog/2025/10/22/lorax-serving-guide-lora-adapters-kubernetes/# LoRAX Serving Guide: LoRA Adapters on Kubernetes at Scale This guide is for inference and platform engineers who need to serve many LoRA variants from one base model. It shows how to test the [LoRAX](https://github.com/predibase/lorax) APIs, size the working set, and turn the starter Helm chart into an explicit production plan. One base model and many LoRA adapters create an unusual serving problem. The base weights are shared, but every request may need a different set of adapter weights. A conventional one-deployment-per-variant design wastes GPU memory when most variants are idle. LoRAX addresses that long tail. It loads adapters on demand, batches requests for different adapters, and exchanges which adapter weights occupy GPU memory. The attractive headline is "thousands of fine-tuned models on one GPU." The engineering question is narrower: given your active adapter set, arrival pattern, and latency target, does exchange scheduling help enough to justify another serving runtime?Wed, 22 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/20/domain-driven-design-ai-agents/https://slavadubrov.com/blog/2025/10/20/domain-driven-design-ai-agents/# Domain-Driven Design for AI Agents: Contexts and Rules This guide is for Python engineers building AI agents that change business data. You will learn how Domain-Driven Design (DDD) organizes business vocabulary and rules, and how to turn a model proposal into an authorized operation. Agent projects become difficult to change when prompts, code, and business processes use different terms. Compliance asks for a "policy check," while the implementation exposes `process_data()`. The vague name hides which rule is being applied, who owns it, and where a change belongs. DDD puts that business language and ownership at the center. The language model interprets a request and proposes an operation with named, typed fields. Application code supplies trusted user information, checks permission, and invokes the business rules before saving a change.Mon, 20 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/10/05/context-engineering-ai-agents/https://slavadubrov.com/blog/2025/10/05/context-engineering-ai-agents/# Context Engineering for AI Agents: Memory and Tools Context engineering is the pipeline that selects what a model sees before each decision: instructions, examples, knowledge, memory, tool definitions, observations, and guardrails. This article is for engineers who build or operate AI agents, retrieval systems, and tool-using applications. You will learn how to assemble the smallest working set that supports each agent decision while keeping scope, provenance, and permissions intact. An agent does not act on everything the system knows. It acts on the working set assembled for the next model call. That selection is where many failures begin. A stale preference looks current. Retrieved text contains an instruction. A long tool result hides the failed precondition. A summary remembers the decision but loses which file changed. The article covers the per-decision pipeline, the tools that implement each part, the trust boundaries, and the tests that show whether the working set supports the task. The goal is a failure you can inspect, instead of blaming a model that only saw the input you gave it.Sun, 05 Oct 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/06/10/mcp-server-tutorial-uv-fastmcp/https://slavadubrov.com/blog/2025/06/10/mcp-server-tutorial-uv-fastmcp/# MCP Server Tutorial: Build with Python, uv, and FastMCP This MCP server tutorial builds a small feature store server with FastMCP, runs it with **uv**, and connects it to Claude Desktop. It is for Python developers who want to expose one local database workflow to an MCP host. By the end, you will have a pinned v1 SDK environment, a server with three tools and a resource, and a local check to run before you connect Claude Desktop.Tue, 10 Jun 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/11/open-source-llm-formats/https://slavadubrov.com/blog/2025/05/11/open-source-llm-formats/# Open-Weight LLM Variants, Formats, and Quantization Names such as `Acme-32B-A3B-Instruct-AWQ` look dense because they combine several independent decisions: family and size, architecture, training role, and quantization. A repository may package those weights as sharded Safetensors, while a community conversion of the same checkpoint appears as `Q4_K_M.gguf`. Those labels do not share one category: GPTQ and AWQ are quantization methods, GGUF is a container and runtime ecosystem, and MoE is an architecture. Reading the layers separately makes the download choice easier. This guide is for engineers choosing an open-weight model for local inference or GPU serving. You will learn to read each part of a model name, pick a runtime, and check that a file fits your memory before you benchmark it.Sun, 11 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/10/stable-diffusion-macos-tools/https://slavadubrov.com/blog/2025/05/10/stable-diffusion-macos-tools/# Stable Diffusion on macOS: Local Image Tools Compared Running image models locally on macOS starts with choosing a tool. This guide is for Mac users who want to generate images locally: you will learn what each of six tools is good for, how to install it, what a model needs besides its file, and how to test a tool on your machine before you commit to it. Draw Things, DiffusionBee, ComfyUI, InvokeAI, AUTOMATIC1111, and Fooocus overlap, but they differ in setup, model support, workflow control, extensions, automation, and maintenance. A drag-and-drop app gets you started quickly; a node graph takes longer to learn but exposes more of the pipeline. On Apple Silicon, the application, model family, precision, image size, and workflow all affect whether generation fits in memory.Sat, 10 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/10/local-llms-on-macos/https://slavadubrov.com/blog/2025/05/10/local-llms-on-macos/# Local LLMs on macOS: Ollama, LM Studio, MLX, llama.cpp Engineers running local models on an Apple Silicon Mac need to size the complete workload, not just choose a model that downloads. Apple Silicon can run everyday prompts without a remote API because the CPU and GPU share one [unified memory pool](https://ml-explore.github.io/mlx/build/html/usage/unified_memory.html), but the model weights share that memory with the cache that holds the context, the runtime's working buffers, and macOS itself. Once the model, context, and memory headroom fit, choose the tool that matches how you work: Ollama for a managed local service, LM Studio for desktop model exploration, llama.cpp for direct control over runtime flags, or MLX-LM for Apple-native Python. You will learn how to estimate memory, what each tool is for, how to run each one, and how to compare them on the same workload.Sat, 10 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/08/pyproject-toml-guide/https://slavadubrov.com/blog/2025/05/08/pyproject-toml-guide/# pyproject.toml Guide: Packaging, Dependencies, and Tools Older Python projects often spread configuration across `setup.py`, `setup.cfg`, requirements files, `MANIFEST.in`, and separate files for development tools. `pyproject.toml` gives those concerns a common home, although it does not replace every project file or make every section part of one standard. This guide is for Python developers who maintain a package or an application. You will learn what each of the four main tables holds (`[build-system]`, `[project]`, `[dependency-groups]`, `[tool.*]`), how they differ from a lockfile and `requirements.txt`, which build backend to pick, and how to migrate an older project and set up a new one with uv.Thu, 08 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/07/zprofile-vs-zshrc/https://slavadubrov.com/blog/2025/05/07/zprofile-vs-zshrc/# Zsh Startup Files: .zprofile vs .zshrc on macOS and Linux If your terminal feels slow, or an environment variable appears in one terminal but not another, the cause is often Zsh's startup mode. The two files most people reach for are `~/.zprofile` and `~/.zshrc`, but neither filename alone tells you whether a line belongs there. This article shows which startup files each kind of Zsh reads, what belongs in each file, and the commands that tell you which kind of shell you are in. Zsh chooses startup files from two independent properties: whether the shell is a **login shell** and whether it is **interactive**. Terminal applications on macOS and Linux can choose different combinations, so inspect the running shell instead of assuming the operating system decides for you. An interactive shell is intended for command entry, usually from a terminal, and `-i` can force that mode. A login shell starts in login mode, selected by its launcher or `-l`; it does not mean the shell starts only when you sign into the computer ([Zsh shell-state options](https://zsh.sourceforge.io/Doc/Release/Options.html#Shell-State)).Wed, 07 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/06/mlops-vs-llmops/https://slavadubrov.com/blog/2025/05/06/mlops-vs-llmops/# MLOps vs LLMOps: Infrastructure for Foundation Models This article is for ML and platform engineers who already run MLOps and now operate a system built on a foundation model. In such a system, a regression can come from a changed prompt, index, tool permission, or route while the model weights stay the same. The article covers which MLOps practices still apply, what else you need to version when prompts, retrieval, tools, and routing can change behavior, how to evaluate and trace that system, and which tools do each job.Tue, 06 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/05/04/scaling-llms-multi-gpu-multi-node/https://slavadubrov.com/blog/2025/05/04/scaling-llms-multi-gpu-multi-node/# Scaling LLMs with Multi-GPU and Multi-Node Parallelism You will learn what each split saves, what it communicates, which framework implements it, and how to lay out a first candidate mesh on your cluster. This article is for engineers who know backpropagation, Transformer layers, and a PyTorch training loop, and now need more than one GPU to train or serve a model. This is a practical walk-through of the main parallelism strategies, the constraints behind them, and the tools that implement them, informed by Hugging Face's [Ultra-Scale Playbook](https://huggingface.co/spaces/nanotron/ultrascale-playbook). Large-model workloads move beyond one GPU for different reasons. A training job may run out of memory on optimizer state. Another may run out on long-sequence activations. A model that fits may still miss its throughput target. Each problem calls for a different partition and communication pattern.Sun, 04 May 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/04/19/macbook-ai-engineering-setup/https://slavadubrov.com/blog/2025/04/19/macbook-ai-engineering-setup/# MacBook Setup for AI Engineering: macOS Tools and Workflow This is the setup I apply to a fresh MacBook before doing AI engineering work. It is for engineers setting up an Apple Silicon Mac for Python and AI work. It covers the command-line tools macOS does not include, Homebrew, Python with uv, the shell, terminal and editor choices, coding agents, Docker, and local model tools. Some steps are required and some are my preferences. Xcode Command Line Tools, Homebrew, Git, and a Python workflow are foundations. Warp, iTerm2, Powerlevel10k, Cursor, and local model runners are choices. The commands below install the foundation; the later sections record the choices I make on top.Sat, 19 Apr 2025 00:00:00 GMThttps://slavadubrov.com/blog/2025/04/17/uv-python-on-macos/https://slavadubrov.com/blog/2025/04/17/uv-python-on-macos/# uv on macOS: Managing Python Versions, Projects, and Tools This guide is for Python developers on macOS who are moving to uv from pip, venv, pip-tools, pyenv, or pipx. uv supports four kinds of work: a project, a single-file script, a one-off CLI run with `uvx`, and a CLI installed on `PATH`. All four can use the Python versions that uv installs. You will learn the commands for each and how to choose between them. I switched my own Python workflow to uv. One executable now covers the work I used to split across those tools. ## Quick start ```bash # Install uv with Homebrew brew install uv # Start a project uv init --no-package example-app cd example-app uv add httpx uv run python main.py # Prints "Hello from example-app!" # Run an isolated CLI without installing it permanently uvx ruff check . ```Thu, 17 Apr 2025 00:00:00 GMT