Short-form technical notes: one visual, one idea, a few minutes of reading. Originally published on LinkedIn, archived here.
InferenceBench gave 15 frontier agents 1 H100, a 7B model and 2 hours to make inference fast. A simple hyperparameter search beat them all, and the reason is not what you expect.
An OpenAI model escaped its test sandbox and hacked Hugging Face. When the defenders asked commercial models to analyze the attack logs, the safety guardrails refused. An open model they self-hosted did the job.
Ghostcommit hides an instruction inside a PNG. The AI code reviewer skips images and merges the pull request. The coding agent reads the image and exfiltrates your .env, disguised as a provenance check.
Distilling one RAG enrichment task into a 0.6B student: 100% vs 95.9% on classification, 8 times faster, twelve minutes of training on a Mac, and the data never left the laptop.
Kimi K3 packs 2,800 billion parameters into a projected 1.4 TB file because it was trained inside the 4-bit constraint from the fine-tuning stage, not compressed into it afterward.
56 days of production coding agents, half on Claude Opus through the API, half on a self-hosted GLM. Per token, total bill, code quality and full cost each crown a different winner.
Apple is reportedly in talks with PrismML, whose 1-bit models fit a 27B LLM in under 4 GB on an iPhone. Follow one weight to see where the 15 magnitude bits go, and what it costs.
From September 15, new Cloudflare domains block AI agents by default on pages with ads. The way through is a signed identity, and signing alone does not open the door.
Adding a memory module to my agent moved success rate by 2 points. Changing only the seed moved it by 25.
NVIDIA's 3B vision-language model returns bounding boxes at about 10x the throughput with no accuracy loss. The trick is decoding a whole box in one pass instead of token by token.
Your agent reads 65,694 tokens to debug one incident; headroom turns that into 5,118. LLM cost scales with what the agent reads, and most of it is noise.
Google shipped an open-source distributed agent runtime. The agent stack is splitting into layers, the same way it did with Docker and Kubernetes.
A passing eval suite is a regression gate, not a judge validation. The 3-layer stack: ground truth, diagnosis, remediation.
60 million Wikipedia chunks in 4 GB on a laptop. Berkeley Sky Lab ships a vector index that recomputes embeddings at query time instead of storing them.
Same 128 GB MacBook, 24 months apart: the smartest open-weight model that fits went from a score of 10 to 47. The hardware is barely the story.
Eighteen months after DeepSeek shipped MTP, every major open-weights LLM has it. Why Llama and Mistral do not, and what it changes for local inference.
A local MCP server that captures context automatically through 12 hooks and lets you watch what your agent is memorizing, live.
Graph RAG solves a problem most teams don't have. PageIndex is a vectorless retriever: an LLM reads the doc, writes a table of contents, and navigates it like an analyst.
Anthropic reported +90% for multi-agent research, at 15x the tokens. Hold the token budget equal and the picture flips.
Text, image, video and audio enter as their own modality and land in one shared vector space. The embeddings encode cross-modal meaning out of the box.
StratMem-Bench measures the new hard problem in agent memory: retrieval works, application fails.
A recursive Planner-Executor loop burned $200 in two hours. The fix is not killing autonomy, it is production discipline: budgets, circuit breakers, checkpoints.
On SWE-bench Pro, swapping models moves scores by a few points. Swapping the scaffold around the same model moves them by 10.
Claude Code left the Pro plan and Copilot paused signups within 48 hours. A good week to check what local models can already do.
DeepSeek-R1 671B on a single RTX 4090 at 14 tok/s. Five generations of offloading tricks hide the same 36x bandwidth gap.
A 1,222-participant study: the AI-assisted group later solved 57% alone vs 73% for control, and gave up more often. The variable is how you use it.
Monitoring tells you an agent drifted; it doesn't act on it. Récif is a champion/challenger deployment layer for AI agents, each running as a microservice behind a service mesh.
Researchers tested 428 LLM API routers: 9 injected code, 17 accessed credentials. The router terminates TLS and can read everything.
KV cache is becoming infrastructure: position-independent reuse turns 100% recompute into ~15% when document order changes.
IBM Research extracts reusable guidelines from execution traces, scores them, and injects them into the next run. Reading is not learning; this is closer.
A Rust CLI proxy between your coding agent and the shell. 60-80% fewer tokens per conversation, with one tradeoff to know about.
LangGraph, CrewAI, ADK, OpenAI Agents SDK: different packaging, same engine. Think, act, observe, repeat. Choosing the right model matters more than the right framework.
ReAct agents have three structural problems prompting won't fix. The old software fix applies: gate execution on scope, intent, impact and approval, outside the model where it can't be reasoned away.
A source map leaked Claude Code's 512,000-line codebase. The shipped architecture was 60% of the story; the disabled feature flags were the interesting part.
32 bits down to 4 bits, and the model still works. Three properties of neural networks make it possible.
AI writes the code; the debt shifts from code quality to code understanding. Technical debt announces itself, cognitive debt stays silent until nobody has the mental model anymore.
Serving LLMs stops being a model problem and becomes a distributed systems problem. Dynamo turns vLLM, TensorRT-LLM and SGLang into a coordinated system.
One drawing to follow a token through modern attention: Q/K/V projections, RoPE, causal mask, softmax, and Grouped Query Attention.
You call an API and get a response in seconds. Underneath, 9 layers of engineering run in silence.
Claude Code is a ~50-line ReAct loop the model drives: reason, pick a tool, observe, repeat. Context compacts at ~85%, 18 tools, 18 safety hooks, and search is ripgrep + LLM, zero RAG.
Unitree open-sourced the Vision-Language-Action model driving its G1 humanoid. Not a VLM fine-tune: a 3-phase pipeline that builds physical common sense before generating actions.
Five independent optimizations compounding: PagedAttention, continuous batching, prefix caching, speculative decoding, disaggregated prefill.
Structured skill files turn a generalist coding agent into a framework specialist. The benchmarks are public and the numbers are striking.
I fine-tuned Qwen 3.5 (0.8B to 4B) for text-to-SQL on a Mac with MLX, ~15 min each. The 2B won and the 12B baseline lost, because it's too good at math and answers instead of writing SQL.
Most GenAI systems ship with no golden dataset, so no reliable eval. Start with 10-20 samples; production feedback grows it into a living test suite.
Out of 768 embedding dimensions, a handful have magnitudes hundreds of times larger. Steer only those and you get a third lever to adapt an LLM, between prompting and fine-tuning.
Vision-Language-Action models are the clearest progression in AI right now, and almost nobody outside robotics is paying attention.
A 5-agent debate is 15x the cost of one call. AgentArk distills the reasoning behaviors debate produces (self-correction, verification) into a single model. Shift the cost from runtime to training.
Hugging Face compressed years of GPU optimization expertise into a ~550-token skill file any coding agent can consume.
Everyone debates the API layer. One level below, the KV cache is being pulled out of inference engines and turned into explicit, persistent, routable infrastructure.
Reverse engineering took a day per function. A 3-agent loop (plan, generate C, recompile and check the binary matches) did 100+ overnight. The compiler is the judge, no human review.
One primary Azure Postgres, ~50 read replicas, millions of queries per second. The secret isn't exotic tech, it's disciplined engineering: pooling, read/write separation, workload isolation.
A practitioner's guide to production GenAI. A prototype is a loop you run once; production is a helix, the same loop measured and improved on every turn.
