Practical field notes on the parts of AI systems that have to survive production: agents, evaluation, security, memory, retrieval, model engineering, and infrastructure.
Recursive Harness Engineering explores systems that improve other AI systems, from controlled configuration experiments to approaches such as recursive agent harnessing.
Start with one of the series below, or browse every article by topic.
Series · Parts: 6
In order. Start at the loop.
Swipe through the parts →
Series · Parts: 2
Build the harness, then measure each change.
Series · Parts: 1
Controlled recursive improvement: one harness improves another.
Evaluation
Evaluate agent behavior and every stage of a retrieval-augmented generation system.
Retrieval & language
Build ranking pipelines, choose document AI systems, and structure language reliably.
Fast Answers
Choosing a model, tool, or approach? Get a short recommendation, the conditions that change it, and a link to the full explanation.
AI Agents Part 2
The same support tasks as an agent, an agent with approval, an agent in a workflow, a router and a workflow with no agent, compared on one test set.
· 22 min
AI Agents Part 1
How task requirements shape an agent harness: choosing control flow, tools, code execution, state, permissions and evidence before choosing a framework.
· 28 min
AI Agents Part 1
Recursive harness engineering in one small loop: an LLM turns a memory tool's wrong answer into a tested fix, and a plain script checks if an LLM was needed.
· 20 min
Developer Tooling · AI Agents
Map pydantic to Zod, FastAPI to Hono, and Celery to a Postgres queue, then build a streaming agent service in a pnpm TypeScript monorepo.
· 25 min
AI Agents Part 6
How to accept a tested artifact, recover uncertain tool calls, hand work to another session, and measure whether a harness component helps.
· 20 min
Model Engineering · AI Infrastructure
A practical guide to choosing LLM and diffusion-model quantization paths by bottleneck, hardware, runtime kernel, calibration data, and deploy checks.
· 33 min
AI Agents
Why production AI agents need schema-guided memory with temporal validity, provenance, and structured conflict handling instead of raw text vector recall.
· 16 min
AI Agents
How to evaluate production AI agents by turning traces into versioned regression datasets, trajectory metrics, calibrated judges, and CI checks.
· 27 min
AI Agents Part 5
Keep an agent run alive across worker failures, reconcile uncertain tool effects, and choose a runtime: queue and worker, Temporal, sandboxes, or hosted agents.
· 30 min
Retrieval and Language Systems
RAG evaluation metrics for parsing, retrieval, reranking, generation, citations, and production telemetry, with code and failure modes.
· 44 min
AI Agents Part 4
How to authorize an agent's proposed action, contain its executor, and test the boundaries that model guardrails cannot enforce.
· 26 min
Retrieval and Language Systems
NER in 2026 means choosing between GLiNER, spaCy, Transformers, and LLM extraction for latency, accuracy, and schema control.
· 41 min
AI Agents Part 3
Design an agent's action and result contract, then choose JSON calls, MCP, CLI, or code execution for the work it needs to do.
· 23 min
Model Engineering · AI Infrastructure
A practitioner's reference to 45 LLM engineering concepts for production systems, spanning inference, training, architecture, deployment, and operations.
· 73 min
Retrieval and Language Systems
OCR in 2026 means choosing between classical pipelines and VLMs for text, layout, tables, and document extraction.
· 22 min
AI Infrastructure
How Polars, DataFusion, Daft, Ray Data, and Spark run tabular and multimodal workloads, with code, a like-for-like benchmark, and decision rules.
· 19 min
AI Agents Part 2
Choose agent memory by the failure it must prevent, then pick the tool: checkpoint savers, key-value stores, pgvector or Qdrant, files, or a memory framework.
· 26 min
Retrieval and Language Systems
How to build a search ranking stack with BM25, dense embeddings, hybrid RRF, cross-encoder reranking, and LLM listwise reranking on Amazon ESCI.
· 28 min
AI Agents Part 1
Choose between fixed workflows, ReAct, ReWOO, and Plan-and-Execute by asking when tool results must change the next action.
· 15 min
Model Engineering
A technical walkthrough of DeepSeek's Manifold-Constrained Hyper-Connections (mHC), residual stream width scaling, Sinkhorn routing, and training stability.
· 13 min
AI Agents
What public Enterprise RAG Challenge 3 submissions reveal about prompt iteration, validation, context strategy, tool wrappers, and agent architecture.
· 13 min
Model Engineering
When to fine-tune LLMs, when to use RAG or prompting, and how LoRA, QLoRA, DoRA, Unsloth, Axolotl, datasets, evals, and deployment fit.
· 29 min
AI Agents · Model Engineering
How Schema-Guided Reasoning uses vLLM, XGrammar, Pydantic schemas, and constrained decoding to enforce structurally valid LLM outputs.
· 16 min
Model Engineering · AI Infrastructure
Evaluate and deploy LoRAX for long-tail LoRA serving, with current APIs, Kubernetes chart hardening, adapter security, cache-aware routing, and vLLM trade-offs.
· 16 min
AI Agents
How domain-driven design helps AI agents use shared language, bounded contexts, services, repositories, and deterministic business rules.
· 16 min
AI Agents
How to design context engineering for AI agents: context windows, instruction hierarchy, retrieval, memory, tools, guardrails, provenance, and compression.
· 25 min
AI Agents · Developer Tooling
Build a custom MCP server with uv and FastMCP, expose ML feature-store tools, test them locally, and connect the server to Claude Desktop.
· 10 min
Model Engineering
How to read open-weight model names by separating training role, architecture, quantization method, storage format, runtime, and hardware fit.
· 14 min
Developer Tooling
Compare local image-generation tools on macOS by setup, model support, workflow control, reproducibility, extensions, and maintenance status.
· 12 min
Model Engineering · Developer Tooling
Compare local LLM tools on macOS by interface, model format, memory headroom, API exposure, and Apple Silicon development workflow.
· 12 min
Developer Tooling
How pyproject.toml works for build systems, project metadata, runtime dependencies, development groups, CLI entry points, and tool configuration.
· 8 min
Developer Tooling
Zsh startup files explained: when ~/.zprofile and ~/.zshrc load on macOS and Linux, what belongs in each file, and how to diagnose the running shell mode.
· 7 min
AI Infrastructure
What foundation models changed in MLOps: release units, retrieval, tools, evaluation, tracing, serving, and the tools that do each job.
· 16 min
Model Engineering · AI Infrastructure
How to scale LLMs across GPUs and nodes with data, sharded, tensor, pipeline, context, and expert parallelism, and which framework implements each.
· 18 min
Developer Tooling
A MacBook setup checklist for AI engineering: Xcode tools, Homebrew, Python with uv, Docker, terminal setup, and VS Code.
· 7 min
Developer Tooling
How I use uv on macOS for Python installs, project dependencies, lock files, inline scripts, and one-off CLI tools.
· 9 min
Nothing here yet.
