[{"c":"AgentRx proposes a new attribution framework, but the overall process still has two steps: flag failures, then identify which failure was critical. AgentRx groups failures into nine categories ($K_1$–$K_9$): instruction or plan adherence failure, intent–plan mismatch, fabrication of new information or hallucination, invalid tool call, misinterpretation of tool output, insufficient specification of user intent, unsupported intent, guardrail triggered, and system failure. AgentRx takes three inputs: (i) a toolset $\\mathcal{T}$ with input and output schemas, (ii) an optional domain policy $\\Pi$, and (iii) a failed trajectory $T=\\langle s_{1},\\dots,s_{n}\\rangle$. Each step $s_k$ contains multiple logged events with fields such as agent name, tool name, step index, tool output, or natural-language conversation. AgentRx normalizes logs from different frameworks into a common intermediate representation (IR). The logs in the appendix show how the authors adapted different frameworks to this representation. A new framework would still need adjustments to its log adapter. Let $T_{\\leq k}$ denote the trajectory prefix through step $k$. AgentRx generates executable constraints from the tool schemas, domain policy, and trajectory prefixes. It evaluates those constraints step by step to produce an auditable validation log $V$. An LLM judge uses $V$ and the taxonomy checklist $K$ to predict the root-cause step $\\hat{s}$ and category $\\hat{y}$. The LLM first generates global constraints $\\mathcal{C}^{G}$ from the tool schemas and any available domain policy, then puts them in a global store $\\mathsf{G}$. At each step, it uses the task instruction $I$, the observed prefix $T_{\\leq k}$, and $\\mathsf{G}$ as context to generate dynamic constraints $\\mathcal{C}^{D}_{k}$, which it puts in a local store $\\mathsf{L}_k$. The constraints available at step $k$ are $\\mathcal{C}_k:=\\mathcal{C}^{G}\\cup\\mathcal{C}^{D}_k$. The guard $g_{C,k}=G_C(T_{\\leq k},s_k)\\in\\{0,1\\}$ determines whether a constraint applies. If the guard is 0, AgentRx skips the constraint. If it is 1, AgentRx evaluates the assertion $\\phi_{C,k}=\\Phi_C(T_{\\leq k},s_k)\\in\\{\\text{sat},\\text{viol}\\}$. The assertion can be checked programmatically or by an LLM. The evaluation at step $k$ is $$ \\text{Eval}_{C}(k)=\\begin{cases}(\\text{skip},\\emptyset)\u0026\\text{if }g_{C,k}=0,\\\\ (\\phi_{C,k},e)\u0026\\text{otherwise.}\\end{cases} $$ After evaluating every step, we want to know which constraints were violated, at which steps, and what evidence supports each finding. The validation log $V$ collects these records: $$ V\\;:=\\;\\{(k,C,e)\\mid C\\in\\mathcal{C}_{k},\\ G_{C}(T_{\\leq k},s_{k})=1,\\ (\\text{viol},e)=\\text{Eval}_{C}(k)\\}. $$ The LLM judge uses the task instruction, trajectory, taxonomy checklist, and validation log to output the critical failure step and root-cause category. The table below compares AgentRx with the adapted W\u0026W baseline. Table 1. Failure attribution accuracy across datasets. We compare modified W\u0026W (W\u0026W*) and AgentRx on failed-agent and critical-step accuracy. Mean $\\pm$ standard deviation over 3 runs. AgentRx improves step localization accuracy across all domains and improves failed-agent identification in most settings. The comparison uses the W\u0026W method: the authors convert trajectories from the AgentRx benchmark into the format W\u0026W expects and adjust the prompt so the LLM identifies the first unrecoverable failure step in the log.","d":"Reading notes on AgentRx, constraint validation, and root-cause diagnosis of agent failures.","dt":"2026-11-11","s":"blog","t":"AgentRx: Diagnosing AI Agent Failures from Execution Trajectories","u":"/blog/agentrx-notes/"},{"c":"BLADE BLADE uses Jev for attribution and divides the process into three stages. Notably, Jev makes the decisions in all three stages. The diagram below shows the BLADE framework. Figure 1. Overview of BLADE. (1) Jev reconstructs dependencies between events. (2) The dependencies select the handoffs to audit, and Jev checks how each agent decision uses its source against the task’s requirement spans $R$. (3) The diagnosis reads the full trajectory with the blame graph $G=(V,E,R,F)$; every event remains a candidate. Method BLADE represents an event as $e_i$ and defines four kinds of event relation: depends on, repeats, contradicts, and repairs. Jev selects the relation between events, with an additional None option. Let $\\mathcal{R}_{\\mathrm{link}}$ denote the four relation types and let $x_{t,r}$ denote the question about relation $r$ at event $e_t$. The probability assigned to an earlier event $e_u$ is $$ p_{ut}^{(r)}=P_{\\mathrm{Jev}}(e_{u}\\mid s_{\\tau},x_{t,r}). $$ BLADE retains relations with probability at least $\\theta=0.3$, forming the weighted edge set $$ E=\\bigl\\{ (e_{u},e_{t},r,p_{ut}^{(r)}):u \\lt t, r\\in\\mathcal{R}_{\\mathrm{link}},\\ p_{ut}^{(r)}\\geq\\theta\\bigr\\}. $$ Not every dependency is useful. Building edges for every qualifying dependency would require $O(N^2)$ complexity, so BLADE prioritizes the depends-on distribution when selecting edges. For each event $e_t$, let $\\mathcal{U}_{t}=\\{u \\lt t:p_{ut}^{(\\mathrm{dep})}\\geq\\theta\\}$, where $\\mathrm{dep}$ denotes depends-on. Let $\\mathcal{S}_{t}\\subseteq\\mathcal{U}_{t}$ contain the indices of at most $K=2$ sources with the highest $p_{ut}^{(\\mathrm{dep})}$. The selected handoffs are $$ \\mathcal{H}_{t}=\\bigl\\{(e_{u},e_{t}):u\\in\\mathcal{S}_{t}\\bigr\\}, $$ and $\\mathcal{H}=\\bigcup_{t=2}^{T}\\mathcal{H}_{t}$. Audit planning reuses dependency probabilities and requires no additional model calls. Each handoff $(e_u,e_t)$ carries an implicit semantic contract: the receiving event should preserve the source meaning, avoid treating uncertain information as established fact, and follow the task requirements. Unlike formal contract verification, these obligations are assessed through structured semantic judgments. BLADE extracts requirement spans $R$ from task $q$ and includes them with the full trajectory in the audit state $s_{\\tau,R}$. Jev classifies each event as an agent decision or evidence, denoted by $\\kappa(e_t)$, and evaluates how each selected receiving event $e_t$ uses its source $e_u$. For question $x_{ut}^{\\mathrm{use}}$, Jev assigns probabilities over nine predefined labels $\\mathcal{L}$: $$ p_{ut}(\\ell)=P_{\\mathrm{Jev}}\\big(\\ell\\mid s_{\\tau,R},x^{\\mathrm{use}}_{ut}\\big),\\qquad\\ell\\in\\mathcal{L}. $$ Here $\\ell_{ut}$ is the most probable label and $\\pi_{ut}$ its probability. The labels distinguish potential error introduction, inheritance or repair of an existing error, and cases with no retained finding. They fall into three groups, $\\mathcal{L}=\\mathcal{L}_{\\mathrm{kept}}\\cup\\mathcal{L}_{\\mathrm{breach}}\\cup\\mathcal{L}_{\\mathrm{inh}}$: $\\mathcal{L}_{\\mathrm{kept}}$ represents faithful use, no actual use of the source, or insufficient evidence to assess the handoff. $\\mathcal{L}_{breach}$ represents potential error introduction through unverified adoption, inapplicable information, misinterpretation, or a correction that introduces a new error. $\\mathcal{L}_{inh}$ represents propagation of an existing error or repair without introducing a new error. For handoffs entering an agent decision, BLADE retains non-kept judgments as structured findings. Let $\\mathcal{H}^{\\mathrm{dec}}=\\{(e_{u},e_{t})\\in\\mathcal{H}:\\kappa(e_{t})=\\mathrm{decision}\\}$. The resulting findings are $$ F=\\bigl\\{ (e_{u},e_{t},\\ell_{ut},\\pi_{ut}): (e_{u},e_{t})\\in\\mathcal{H}^{\\mathrm{dec}},\\ \\ell_{ut}\\notin\\mathcal{L}_{\\mathrm{kept}}\\bigr\\}. $$ Through this construction, BLADE turns the trajectory into a blame graph $G=(V,E,R,F)$, where $E$ contains dependency edges, $V$ contains nodes, and $R$ sup","d":"Reading notes on BLADE, multi-agent failure attribution, and information-use auditing.","dt":"2026-11-10","s":"blog","t":"BLADE: Auditing Information Use for Failure Attribution","u":"/blog/blade-notes/"},{"c":"The Problem The evaluation approach used by the Who\u0026When (WW) dataset has limitations. Its setup is: Input: A complete failed session log, from the user’s question through rounds of interaction among agents to an incorrect final answer or a stopping rule such as the maximum number of replanning rounds. Task: Ask an LLM to read the log and identify the earliest, uniquely responsible agent and step that caused the task to fail. Decisive error step: A step whose incorrect action could be replaced by a correct one so that the subsequent trajectory succeeds. The authors found that WW trajectories did not explicitly show step numbers, so the LLM had to track or infer each step. The evaluation criteria given to the LLM also differed from those used in expert annotations. The authors embedded a concise version of the human annotator guidance in the LLM prompt: Find the earliest decisive step that diverted the task and whose correction could lead to success. After these two changes, GPT-4o’s step attribution accuracy on the WW hand-crafted set with ground truth rose from 6.04% to 23.56%. WW’s attribution perspective is also narrow. Choosing a single absolute error from multiple errors is inherently uncertain: an LLM’s answer may differ from an expert’s while both identify plausible critical errors. Responsibility between agents can also be ambiguous. A subagent’s error may arise from the subagent itself or from an inadequate task description by the assigning agent; attributing it to only one agent is therefore incomplete. DoVer The authors propose DoVer in response. Its central framework can be summarized in three parts: Set aside human-annotated ground truth, since failures can be viewed from multiple perspectives, as in MP-Bench. DoVer returns to the definition of a decisive error: directly modify the decision at step $t$ and see whether the subsequent task succeeds. If it does, that step is treated as decisive. A long session can contain several ReAct loops, and multiple trials may fail because an LLM may initially explore by trial and error. DoVer separates each trial and looks for failure points within it, shortening context and enabling repair along different failed paths. Figure 1. Failure trace of Case 3 in WW-HC, illustrating ambiguity in failure attribution. The session consists of four distinct trials, each initiated by a plan update and executed via a ReAct-style Yao et al. (2023) loop. Different strategies (e.g., direct scrolling in Trial 1 vs. calendar navigation in Trial 2) yield separate error points, making single-step attribution across the session inherently ambiguous. Trial 2 (Steps 53–55) further shows inter-agent misalignment: the Orchestrator issued an invalid instruction, while the WebSurfer compounded the error by executing an unrelated action. Responsibility between an orchestrator and subagents is often entangled. DoVer distinguishes two levels of intervention: Directly edit the orchestrator’s plan or the prompt sent to a subagent. If a subagent cannot complete a task because it lacks a tool, create a new tool to help. This directly modifies the harness and can connect to RSI. Figure 2. DoVer (Do–then–Verify) Debugging Pipeline. (1) Trial segmentation: split the failed session log into trials using re-plan steps as cut points. (2) Failure attribution: for each trial, propose a hypothesis $h_{i}$ that marks a faulty step or agent. (3) Intervention generation: turn $h_{i}$ into an actionable intervention that edits either the plan or the attributed message or step in the original log. (4) Intervention execution: replay the trajectory in place, i.e., preserve all steps before the intervened step, then execute the intervention and measure progress of the new log. Colors indicate plan/re-plan (blue), execution (green), attributed failure (red), terminal failure (dark red), terminal success (dark green), intervention (yellow), new plan/re-plan (blue hatch), and new execution (green hatch). DoVer has four steps: Tri","d":"Reading notes on DoVer, multi-agent failure attribution, intervention, and validation.","dt":"2026-11-06","s":"blog","t":"DoVer: Intervention-Driven Multi-Agent Debugging","u":"/blog/dover-notes/"},{"c":"Core Idea The paper’s central idea is to find the critical error step $t^\\star$ in a failed trajectory $\\tau = (x_1, x_2, \\dots, x_N)$. The authors identify $t^\\star$ as the earliest error step directly and causally responsible for the final task failure: if that step had been corrected, the later cascade would not have occurred and the task would have succeeded. For an evaluation such as Who\u0026When that directly gives an LLM a failed trajectory, long context, numerous local errors, and cascading mistakes often lead to poor performance. Figure 1. Example of a critical error and its cascading effects. Method Figure 2. Overview of TrajDebug’s error-lifecycle tracing framework. Multi-Granularity History Compression For each step in a trajectory, TrajDebug has an LLM create views at three levels of detail: Table 1. Content and character limits of multi-granularity history views. View Code and character limit Retained content and purpose High-detail view th1 $\\le 3000$ characters per field Nearly verbatim: Retains original instructions, actions, environment observations, and local reasoning. Removes obvious filler, repeated phrasing, and decorative formatting; used for detailed evidence checks at the current and immediately adjacent steps. Medium-detail view th2 $\\le 1200$ characters per field Backbone summary: Merges related information and omits secondary elaboration while retaining the step’s main intent, specific action, state updates, errors, and key observations; used for mid-range history. Low-detail view th3 $\\le 600$ characters per field Minimal summary: Records rough progress, key entities, and unresolved commitments or hard constraints in as few words as possible; used for distant history. The compression process has two constraints: Preserve the user’s constraints and plans at all three granularities, such as prohibitions and subgoal plans. Preserve critical tokens verbatim rather than compressing or simplifying them, including specific item or order IDs and tool function names. Error Trigger Detection When the LLM checks step $t$, it receives context according to the following rules, which can reduce context length: Use th1, the original content, for $Step_t$. Use th2 for $Step_{t-x} \\to Step_{t-1}$. Use th3 for earlier history. The LLM outputs each detected error as: $$ e = (t, c, p, q_w, q_r) $$ Here, $t$ is the step; $q_w$ is the original erroneous commitment, which may be text or an action; $q_r$ is the original reference text containing the rule or fact violated by $q_w$; $c$ is the reference category; and $p$ is the execution phase in which the agent made the error. There are four reference categories $c$: Task conflict: $q_r$ comes from an earlier task instruction or system constraint. History conflict: $q_r$ comes from earlier environment feedback, tool output, or established facts. Intra-step conflict: $q_r$ comes from within the current step $t$. Environment anomaly: usually caused by external factors such as a lost connection or a 404 page. There are five execution phases $p$: Planning: Omitting subtasks, ordering them incorrectly, or drifting from the objective. Reasoning: Distorting evidence and choosing incorrectly, overgeneralizing, or inventing unchecked premises. Action: Choosing the wrong tool or acting inconsistently with stated intentions, using malformed arguments, or taking prohibited actions. Observation: Misreading tool output or ignoring a key result. Verification: Finishing without a required check, stopping before the task is complete, or retrying blindly in a loop. Error State Classification After obtaining the error set, the triggers still need clustering: the same error can be recorded multiple times, and already repaired errors can also be recorded. Further clustering removes duplicates. TrajDebug groups a set of triggers $\\mathcal{E}$ around the same violated reference object $O$ into an error instance $E = (\\mathcal{E}, O)$. The reference category $c$ and the specific violated reference $","d":"Notes on TrajDebug, critical error localization, error lifecycle tracing, and feedback.","dt":"2026-11-05","s":"blog","t":"TrajDebug: Tracing Error Lifecycles","u":"/blog/trajdebug-notes/"},{"c":"Who\u0026When Published in mid-2025, the paper uses relatively older models. The table shows the performance of OpenAI o1 and DeepSeek R1 on the dataset: Table 1. Failure attribution performance of OpenAI o1, DeepSeek R1, and GPT-4o The dataset contains algorithm-generated and hand-crafted agent systems: 127 systems in total, with human annotations. It distinguishes two levels of error: Agent-Level (Who, identifying the responsible agent) and Step-Level (When, identifying the specific step). Step-level performance is lower. Three evaluation methods are used: All-at-Once, Step-by-Step, and Binary Search. Their performance with GPT-4o is shown below: Table 2. GPT-4o failure attribution performance with and without ground-truth labels Dataset example: Figure 1. Examples of failure logs from algorithm-generated and hand-crafted systems in Who\u0026When. November 8 update: While working with the WW dataset, I found the AI-generated tasks rather unreasonable. This also helps explain why MP-Bench’s approach is reasonable. For example, one task asks for the penultimate word of the chorus of the fifth song on an album. The Manager assigns the task sequentially to three subagents: a lyrics expert, a linguistic verification expert, and a music historian. At Step 0, the lyrics expert forwards part of the Manager’s message: the Manager assumes the fifth song is F (it is actually A) and asks for confirmation that it is F. At Step 1, the music historian lists the album’s songs one by one but is biased by the user’s suggestion and concludes that F is fifth. Subsequent work based on this assumption naturally goes wrong. I think the responsible agent and step should be the music historian and Step 1, since F was explicitly a claim to be verified and the historian did not do so. The actual annotation, however, identifies the lyrics expert and Step 0. From another perspective this answer is understandable: WW selects the earliest step containing the incorrect statement. I evaluated Deepseek-V4.1-F1017 on 32 examples from Who\u0026When’s eval set and also ran 8 development examples. The overall results are as follows: Table 3. Comparison of my Who\u0026When evaluation results Experiment group Sample range Representation Validity rate Agent accuracy Step accuracy Joint accuracy Keyword heuristic eval/16 raw / canonical 100% 25.0% (4/16) 12.5% (2/16) 12.5% (2/16) dev development set dev/4 raw 100% 75.0% (3/4) 50.0% (2/4) 50.0% (2/4) dev development set dev/4 canonical 100% 100.0% (4/4) 50.0% (2/4) 50.0% (2/4) eval evaluation set eval/16 raw 100% 68.75% (11/16) 50.0% (8/16) 50.0% (8/16) eval evaluation set eval/16 canonical 100% 56.25% (9/16) 37.5% (6/16) 37.5% (6/16) The structured representation did not improve accuracy on WW. The LLM tended to attribute errors to the planner or decision-maker when using structured trajectories, whereas with less organized trajectories it leaned toward the part where the actual error appeared. In fact, among the 8/10 questions the LLM answered incorrectly, only one showed the multi-perspective problem I had expected; more often, the model seemed to lose its way in the long context and hallucinate or make an incorrect judgment. MP-Bench Different analysts may reasonably attribute a failure to different steps. This non-deterministic, multi-perspective nature arises from the interdependencies inherent in multi-agent systems. Multiple specialized agents can be combined interdependently into diverse, valid execution trajectories aimed at the same goal. Failure attribution therefore depends heavily on an observer’s internal reasoning about what constitutes a “correct” trajectory. The limitations observed in previous work stem from biased evaluation assumptions that overlook the multi-perspective nature of MAS failures, rather than from insufficient model capabilities. Figure 2. An example illustrating the multi-perspective nature of failure attribution in multi-agent systems. MP-Bench uses normalized discounted cumulative gain (nDCG@K), whi","d":"Notes on the Who\u0026When and MP-Bench multi-agent failure attribution benchmarks.","dt":"2026-11-04","s":"blog","t":"Multi-Agent Failure Attribution Benchmark Notes","u":"/blog/multi-agent-attribution-benchmarks/"},{"c":"Recent LLM capabilities have grown linearly or exponentially, sometimes exceeding even AI developers’ expectations. OpenAI reported that an internal model coordinated around 10,000 concurrent agents to produce a solution to the Navier–Stokes Millennium Prize problem. Anthropic researchers also described genetics research conducted with Claude and domain experts. Google’s Gemini 4 Argon helped researchers extend mathematical solutions related to magnetic confinement in fusion through multi-agent collaboration. This work relates to Grad’s 59-year-old conjecture. As task scale and execution time increase, it becomes harder to concentrate every required capability in a single agent. Multi-agent collaboration offers a principled way to address this challenge. Problems in Multi-Agent Systems Individual models still make local errors, including factual hallucinations, miscommunicated messages, and incorrect tool calls. These errors propagate through later interaction rounds and become difficult to detect and locate as the number of steps increases. Failure attribution and self-correction weaknesses become deeply intertwined, preventing agents from improving. This reveals a fundamental asymmetry. A complex multi-agent system’s capabilities far exceed the mechanisms available to understand its failures and enable autonomous improvement (RSI). Bridging this gap requires a closed-loop framework that identifies failures, reduces budget requirements, and guides structural self-improvement. Figure 1. An operational lifecycle overview of multi-agent systems, covering individual capabilities, collaboration, failure attribution, system self-evolution, and representative work. Individual Intelligence LLMs remain limited in complex settings such as automated software engineering, scientific experiment design, and multistep web navigation. They lack persistent memory and active perception of external environments, remaining constrained by the static knowledge acquired during training. A harness gives an LLM capabilities such as environmental perception, goal formulation, and tool execution: Agent = Harness + LLM. Figure 2. Individual agent capabilities organized along the reasoning pipeline. Memory supplies retrieved context and incorporates execution feedback, alongside planning and tool use. Modular optimization dominates, but coupling between modules remains insufficiently studied. Reasoning, memory, planning, and tool use depend on one another at every time step. Yet existing methods optimize individual modules almost entirely in isolation. Capability mismatches between modules often become the real bottleneck in end-to-end deployment. Current benchmarks largely assess only one or two capabilities, leaving this failure mode invisible. Reasoning Chain-of-thought (CoT) prompting elicits intermediate reasoning steps, substantially improving performance on complex tasks without changing model parameters. The central challenge then becomes how external mechanisms bridge static parametric knowledge and the dynamic demands of real-world tasks. From this perspective, existing methods can be organized by information flow through the reasoning pipeline into three stages: Input-stage enhancement: expanding the available information through external knowledge retrieval and multimodal perception. Reasoning-process enhancement: improving reasoning trajectories through search-space expansion and path verification. Output-stage regulation: improving reliability and factual accuracy through post-hoc evaluation and adaptive correction. Query Enhancement Retrieval-augmented generation (RAG) combines a parametric generator with a nonparametric retriever, allowing the model to obtain relevant external documents during inference. Subsequent approaches include: Training models to decide adaptively how to retrieve, and using reflection tokens to assess retrieved and generated content. Retrieving relevant subgraphs from knowledge graphs and performing multi-hop reaso","d":"Reading notes on Beyond Individual Intelligence: individual capabilities, reasoning, memory, planning, role assignment, multi-agent communication, orchestration, failure attribution, and self-evolution.","dt":"2026-10-23","s":"blog","t":"Multi-Agent Systems Survey Notes","u":"/blog/multi-agent-survey-notes/"},{"c":"The Agent Memory Lifecycle The lifecycle consists of four stages: memory extraction, storage, retrieval, and evolution. Memory extraction: transforming raw, unstructured observations $o_t$ into structured memory units $m$. Memory storage: indexing and organizing the extracted units within a memory structure $\\mathcal{M}$. Memory retrieval: retrieving relevant stored information $\\mathcal{M}_{\\text{rel}}\\subset\\mathcal{M}$ in response to a query $q$. Memory evolution: refining memory through post-processing, including internal self-evolution, such as consolidation and abstraction, and external self-exploration, such as new environmental feedback. Figure 1. Workflow of an AI agent system and detailed implementations of the memory system. A Multidimensional Taxonomy of Agent Memory Agent memory can be classified along several dimensions, including temporal scope, functional role, and representational structure. Temporal Scope Short-term memory: retains recent, immediately relevant information, offering fast access but limited capacity. It includes the current conversation context, ongoing reasoning traces, and transient state variables. It is usually discarded once the immediate task ends, although important elements may be consolidated into long-term storage. Long-term memory: persists across sessions, including accumulated knowledge, past interactions, learned patterns, and user preferences. It supports continuity across interaction episodes, transfer learning, and personalized, adaptive behavior over longer periods. Cognitive Structure of Memory Semantic memory: stores general, context-independent world knowledge and factual information, such as “Paris is the capital of France.” In a graph structure, it forms a stable ontological layer. Common sense and domain-specific facts provide a reliable foundation for agent reasoning. Procedural memory: encodes skills, routines, and fixed rules, representing how-to knowledge such as standard operating procedures or game rules. In practice, this allows agents to execute complex, established tasks automatically under standard conditions. Associative memory: establishes latent connections between concepts or data points within a knowledge base. Connecting related information, such as symptoms and diagnoses or products and categories, supports creative reasoning and analogy. It serves as connective tissue within a knowledge graph. Working memory: acts as the agent’s mental scratchpad for processing immediate experience. It temporarily holds the current dialogue turn, intermediate reasoning steps, and transient variables. Despite its short lifespan, it is the entry point for all experience memory. It offers very fast access and directly shapes the agent’s next action. Episodic memory: records past interaction sessions in chronological order, turning transient working memory into a persistent, queryable autobiographical history. For example, it may record that a user requested vegetarian options or cancelled an order. It lets the agent recall what happened and when, providing explicit temporal anchors. Affective memory: captures emotional tone or sentiment arising during interactions. Recording user feedback or frustration allows the agent to adjust its empathy and communication style in later turns. This adds a qualitative emotional dimension to raw interaction logs. Two Main Types Static knowledge memory and dynamic experience memory work together to support understanding the world’s rules and adapting to personalized human–agent interactions. Knowledge memory (passive and static): is the agent’s passive, static repository of objective, global, verifiable information. It resembles an internal reference library or textbook, containing canonical facts, general rules, established procedures, and general truths about the world. This memory is usually preloaded, rarely updated, and independent of specific contexts. Its purpose is to provide a stable, reliable foundation for reasoning and action.","d":"The agent memory lifecycle and taxonomy, with graph-based memory storage, retrieval, and continual evolution.","dt":"2026-10-20","s":"blog","t":"Graph-based Agent Memory: Reading Notes","u":"/blog/graph-based-agent-memory/"},{"c":"Pain Points Evaluating a candidate program usually only requires running it and checking the result. Evaluating an exploration policy requires letting it schedule many successive attempts, then observing the best solution found and the resources consumed. A direction with a low initial score may produce a breakthrough after several repairs; an early leader may soon stagnate. Feedback on policy quality is more delayed than feedback on individual programs. Rerunning the entire online discovery process after every controller change repeatedly incurs costs for model calls, code execution, and evaluation. Meanwhile, the space of exploration policies is large. Branch selection, stopping rules, parallel batches, and failure handling can all be combined in different ways. These are the two bottlenecks identified in the introduction: expensive, delayed feedback and many candidate meta-policies. A fixed exploration policy is simple and stable, but cannot change its search habits across rounds based on accumulated discovery experience. Dream-RSI aims to reduce the cost of comparing exploration policies, rather than directly designing a stronger fixed heuristic for a particular domain. How Dream-RSI Works 0. Dream-RSI Dream-RSI consists of online exploration and offline planning, alternating through the following cycle: Online exploration → save the discovery tree → evaluate and modify policies offline → select a policy for deployment → another round of online exploration. 1. Online Exploration First, an initial workspace is set up as the root node $r$. Each non-root node records its parent, preceding context, workspace snapshot, evaluation diagnostics, and score. Let $t$ index the outer online–offline loop, and let $\\pi_t$ denote the executable scheduling policy code used online in round $t$. The scheduler $\\pi_t$ selects a leaf of the current discovery tree or the root. This means continuing work from the selected node or starting a new approach. A coding agent, or an agent dedicated to the task, carries out the continuation. The resulting node saves its context and score and becomes a new leaf. Exploration stops when: The maximum number of online scheduling rounds is reached. One parallel batch counts as one round, which is not the same as one attempt. The scheduling policy decides not to continue. 2. Offline Planning Step 1 produces the discovery tree $\\mathcal T_t$ for the current round, abbreviated below as $\\mathcal T$. This tree consists of several chains because the online phase only extends leaf nodes. New branches cannot start from internal nodes, except for the root. Offline evaluation first evaluates the current policy $\\pi_t$, then its modified versions. The simulator holds the complete tree, but each replay starts from the root and reveals visited nodes to the policy incrementally. The policy chooses the workspace for the next step using only the information revealed so far. Let $\\mathcal T_i^{m,k}$ denote the tree revealed after policy version $m$ completes $k$ replay rounds on historical tree $i$. Then $\\mathcal T_i^{m,k}\\subseteq\\mathcal T_i$, and the policy code remains unchanged throughout a single replay. Suppose A1 is the earliest child created from the root; replay can proceed as follows: Round Starting points selected by the policy Simulator returns Round 1 Root $r$ Saved node A1 Round 2 Root $r$ Saved node B1 Round 3 A1, B1 Saved nodes A2, B2 Next Return an empty batch Stop without revealing A3 This policy creates no new experiments; it only produces a new scheduling trajectory: Select a non-root leaf of the currently revealed tree: return its existing next node. Select the root: open the next unrevealed branch in creation order. No recorded successor exists: return no new node for that selection. Replay ends when the policy returns an empty batch, reaches the maximum number of replay scheduling rounds, or reveals the entire historical tree. The score at the end of a replay is calculated as: $$ V=\\underbrace{\\max_","d":"Notes on Dream-RSI: its motivation, online exploration and offline replay, and a few thoughts on its limitations.","dt":"2026-10-16","s":"blog","t":"Dream-RSI: Reading Notes","u":"/blog/dream-rsi/"},{"c":"The PPO Algorithm Let’s start by defining a few concepts as the foundation for everything that follows: Action: the set of selectable actions, i.e., the moves the model can take next. Policy: a policy function that takes a state as input and outputs a probability distribution over actions, usually denoted by $\\pi$, e.g. $\\pi(action1|S_t)=0.78\\dots$ Trajectory: denoted by $\\tau$, a sequence of states and actions, where $S_{t+1}=f(S_t,a_t)$ is deterministic, or $S_{t+1}=f(*|S_t,a_t)$ is stochastic. Return: the cumulative sum of rewards from the current time step until the end. Expectation: $$E(x)_{x \\sim p(x)} = \\sum_{x} x * p(x) \\approx \\frac{1}{n} \\sum_{i=1}^{n} x_i \\quad (x_i \\sim p(x))$$Goal 1: train a policy network $\\pi$ that, in every state $S$, outputs actions such that the expected Return is maximized. Goal 2: train a policy network $\\pi$ that maximizes the expected Return over all trajectories. Policy Gradient Let $R(\\tau)$ be the total return obtained by trajectory $\\tau$. Under the current policy, the expected return over all possible trajectories is: $$E(R(\\tau))_{\\tau \\sim P_\\theta(\\tau)} = \\sum_{\\tau} R(\\tau)P_\\theta(\\tau)$$$$\\nabla E(R(\\tau))_{\\tau \\sim P_\\theta(\\tau)} = \\nabla \\sum_{\\tau} R(\\tau)P_\\theta(\\tau)$$$$= \\sum_{\\tau} R(\\tau)\\nabla P_\\theta(\\tau)$$$$= \\sum_{\\tau} R(\\tau)\\nabla P_\\theta(\\tau)\\frac{P_\\theta(\\tau)}{P_\\theta(\\tau)}$$$$= {\\color{#4285F4}{\\sum_{\\tau} P_\\theta(\\tau)} R(\\tau)}\\frac{\\nabla P_\\theta(\\tau)}{P_\\theta(\\tau)}$$$$\\approx \\frac{1}{N} \\sum_{n=1}^N R(\\tau^n) \\frac{\\nabla P_\\theta(\\tau^n)}{P_\\theta(\\tau^n)}$$$$= \\frac{1}{N} \\sum_{n=1}^N R(\\tau^n)\\nabla \\log P_\\theta(\\tau^n)$$The last step uses $\\nabla \\log f(x) = \\frac{\\nabla f(x)}{f(x)}$. A state is fully determined by the previous state and action, so the probability of a trajectory is the product of all its states and the actions taken in each state: $$= \\frac{1}{N} \\sum_{n=1}^N R(\\tau^n)\\nabla \\log \\prod_{t=1}^{T_n} P_\\theta(a_n^t|s_n^t)$$$$= \\frac{1}{N} \\sum_{n=1}^N R(\\tau^n) \\sum_{t=1}^{T_n} \\nabla \\log P_\\theta(a_n^t|s_n^t)$$$$= \\frac{1}{N} \\sum_{n=1}^N \\sum_{t=1}^{T_n} R(\\tau^n)\\nabla \\log P_\\theta(a_n^t|s_n^t)$$Dropping the gradient symbol, the objective for gradient ascent is $\\frac{1}{N} \\sum_{n=1}^N \\sum_{t=1}^{T_n} R(\\tau^n)\\log P_\\theta(a_n^t|s_n^t)$. When we perform the gradient update, the loss is defined as: $$\\text{Loss} = -\\frac{1}{N} \\sum_{n=1}^N \\sum_{t=1}^{T_n} \\color{#4285F4}{R(\\tau^n)} \\log \\color{#EA4335}{P_\\theta(a_n^t|s_n^t)}$$This means we need to collect the $R(\\tau^n)$ produced by every trajectory. Collecting data this way is very slow, consumes a lot of GPU/CPU memory, and has a deeper problem: $$\\frac{1}{N} \\sum_{n=1}^N \\sum_{t=1}^{T_n} {\\color{#4285F4}R(\\tau^n)}\\nabla \\log P_\\theta(a_n^t|s_n^t)$$The problem: $R(\\tau^n)$ is the total score of the entire game. But in reality, an action $a_t$ taken at time $t$ cannot influence anything that happened before time $t$. Using the total score to evaluate the current action is unreasonable. $$R(\\tau^n) \\rightarrow \\sum_{t'=t}^{T_n} \\gamma^{t'-t} r_{t'}^n = \\color{#4285F4}{R_t^n}$$We replace the total return with the future cumulative return $R_t^n$, measured from the current time $t$ until the end of the game. A discount factor $\\gamma$ (Gamma, between 0 and 1) is usually introduced here: the more distant a reward, the less reference value it carries. $r_{t'}^n$ denotes the immediate reward of a specific step. The update becomes: $$\\frac{1}{N} \\sum_{n=1}^N \\sum_{t=1}^{T_n} \\color{#4285F4}{R_t^n} \\nabla \\log P_\\theta(a_n^t|s_n^t)$$But a problem remains: what if every reward in an environment is positive, or every reward is negative? If everything is “bad”, the model would keep updating downward forever. So we should introduce a baseline and judge each action against that baseline. After subtracting the baseline: rewards are positive for good situations and negative for bad ones With the baseline in place, we get the following optimized formula: $$=\\frac{1}{N} \\sum_{","d":"A complete derivation from policy gradient to GAE, importance sampling, and PPO-Clip, and GRPO","dt":"2026-10-11","s":"blog","t":"PPO to GRPO","u":"/blog/ppo-to-grpo/"},{"c":"After SFT, a model has been trained at the token level, making its token probability distribution favor responses that follow the intended format. RL can further improve how it answers and the accuracy of its responses. During RL training, we do not want the model to stray far from the response format established by SFT, so we use KL divergence as a constraint to limit how far the model moves from the post-SFT model while trying to further improve accuracy. Let the complete response generated by the model be $o=(o_1,o_2,\\ldots,o_{|o|})$. The content generated before step $t$ is written as $o_{\\lt t}=(o_1,o_2,\\ldots,o_{t-1})$. The context prefix at step $t$ is then $h_t=(q,o_{\\lt t})$. Under the same context prefix, the probability that the reference policy model—the post-SFT model whose tone and format we want the model to follow—selects the same candidate token is $\\pi_{\\mathrm{ref}}(a\\mid h_t)=\\pi_{\\mathrm{ref}}(a\\mid q,o_{\\lt t})$. When the candidate position is written as a dot, $\\pi_\\theta(\\cdot\\mid h_t),\\pi_{\\mathrm{ref}}(\\cdot\\mid h_t)$, providing a specific token gives a probability value; going through all candidate tokens gives a list of probabilities. For a fixed context prefix $h$, assume that the two policies have the same support. Their common candidate set is: $$ \\begin{aligned} \\mathcal A_h \u0026=\\{a:\\pi_\\theta(a\\mid h)\u003e0\\}\\\\ \u0026=\\{a:\\pi_{\\mathrm{ref}}(a\\mid h)\u003e0\\}. \\end{aligned} $$Write the size of the set as $m_h$, and define two probabilities: the probability of selecting a token from the model currently being trained and from the reference model, respectively: $$ p_a=\\pi_\\theta(a\\mid h), \\qquad b_a=\\pi_{\\mathrm{ref}}(a\\mid h). $$Here, $p,b\\in\\mathbb R^{m_h}$. Both lists are normalized separately: $\\sum_{a\\in\\mathcal A_h}p_a=1,\\sum_{a\\in\\mathcal A_h}b_a=1$. Exact KL For a fixed prefix $h_t$, KL is defined as: $$ \\begin{aligned} \u0026D_{\\mathrm{KL}}\\!\\left( \\pi_\\theta(\\cdot\\mid h_t) \\Vert \\pi_{\\mathrm{ref}}(\\cdot\\mid h_t) \\right)\\\\ \u0026\\quad= \\mathbb E_{a\\sim\\pi_\\theta(\\cdot\\mid h_t)} \\left[ \\log\\frac{\\pi_\\theta(a\\mid h_t)} {\\pi_{\\mathrm{ref}}(a\\mid h_t)} \\right]. \\end{aligned} $$The left-hand side compares two complete next-token distributions. The expectation on the right means taking a weighted average of the values in brackets, weighted by the current policy’s probability of selecting each token. For a finite candidate set, the expectation can be expanded directly into a sum: $$ \\begin{aligned} \u0026D_{\\mathrm{KL}}\\!\\left( \\pi_\\theta(\\cdot\\mid h_t) \\Vert \\pi_{\\mathrm{ref}}(\\cdot\\mid h_t) \\right)\\\\ \u0026\\quad= \\sum_{a\\in\\mathcal A_{h_t}} \\pi_\\theta(a\\mid h_t) \\log\\frac{\\pi_\\theta(a\\mid h_t)} {\\pi_{\\mathrm{ref}}(a\\mid h_t)}. \\end{aligned} $$This step uses only the definition of the expectation of a discrete random variable. $\\log\\frac{\\pi_\\theta(a\\mid h_t)}{\\pi_{\\mathrm{ref}}(a\\mid h_t)}$ is the token’s log probability ratio; an individual term can be positive or negative. Only after summing over all candidates, weighted by their current probabilities, do we obtain a nonnegative KL penalty. If the two distributions are identical, all ratios are 1, taking $\\log$ gives 0, and there is no penalty. The sum for exact KL cannot be negative, so it can serve as a penalty Use the inequality that holds for all positive numbers: $$ \\log u\\leq u-1,\\qquad u\u003e0. $$Let: $$ u=\\frac{\\pi_{\\mathrm{ref}}(a\\mid h)} {\\pi_\\theta(a\\mid h)}. $$Rearranging gives: $$ \\log\\frac{\\pi_\\theta(a\\mid h)} {\\pi_{\\mathrm{ref}}(a\\mid h)} \\geq 1-\\frac{\\pi_{\\mathrm{ref}}(a\\mid h)} {\\pi_\\theta(a\\mid h)}. $$Multiply both sides by the current probability, then sum over all candidates: $$ \\begin{aligned} \u0026D_{\\mathrm{KL}}\\!\\left( \\pi_\\theta(\\cdot\\mid h) \\Vert \\pi_{\\mathrm{ref}}(\\cdot\\mid h) \\right)\\\\ \u0026\\quad\\geq \\sum_{a\\in\\mathcal A_h} \\left[ \\pi_\\theta(a\\mid h) -\\pi_{\\mathrm{ref}}(a\\mid h) \\right]\\\\ \u0026\\quad=1-1=0. \\end{aligned} $$This proof also uses the fact that each probability distribution sums to 1. The basic inequality above holds with equality only when $u$ equals 1.","d":"Notes on exact KL, k₁, k₃, and KL constraints and aggregation in GRPO.","dt":"2026-10-09","s":"blog","t":"KL Divergence","u":"/blog/kl-divergence/"},{"c":"Fine-tuning an entire model directly requires substantial computing resources and GPU memory. For Qwen3-0.6B (0.6 billion parameters), memory requirements cannot be determined from FP16 or FP32 alone: they also depend on the optimizer, sequence length, batch size, and techniques such as gradient checkpointing or offloading. For larger models, such as 7B or 13B, full fine-tuning on a single consumer GPU generally becomes more difficult. LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that trains only a small number of additional parameters while keeping the original model parameters frozen. Its central idea is that parameter changes during fine-tuning can be represented by low-rank matrices. Image source: IBM. Low-Rank Updates and Parameter Counts Suppose the original model’s weight matrix is $W \\in \\mathbb{R}^{d \\times k}$, and the fine-tuned weights are $W' = W + \\Delta W$. LoRA assumes that $\\Delta W$ can be decomposed into the product of two low-rank matrices: $$ \\Delta W = BA $$Here, $B \\in \\mathbb{R}^{d \\times r}$, $A \\in \\mathbb{R}^{r \\times k}$, and $r \\ll \\min(d, k)$ is the rank. During the forward pass, the output is: $$ h = Wx + \\Delta Wx = Wx + BAx $$The original model parameters $W$ remain frozen; only $B$ and $A$ are trained. Parameter comparison: the original model has $d \\times k$ parameters, while LoRA has $d \\times r + r \\times k = r(d + k)$ parameters. When $r \\ll \\min(d, k)$, LoRA has far fewer parameters than the original model. For example, with $d=4096, k=4096, r=8$, the original parameter count is $4096 \\times 4096 = 16,777,216$, while the LoRA parameter count is $8 \\times (4096 + 4096) = 65,536$—a reduction by a factor of 256! This reduction comes from counting parameters: each element of a trainable matrix contributes one scalar parameter. For positive integer dimensions $d,k,r$, the rank of the product matrix is at most $r$, and need not be exactly $r$. For this single weight matrix, the ratio of trainable parameters to the full parameter count is: $$ \\begin{aligned}\\frac{\\text{LoRA parameters}}{\\text{full parameters}}\u0026=\\frac{dr+rk}{dk} =\\frac{r(d+k)}{dk} =r\\left(\\frac1k+\\frac1d\\right),\\\\\\frac{8(4096+4096)}{4096\\cdot4096}\u0026=\\frac{65{,}536}{16{,}777{,}216} =\\frac1{256}=0.00390625,\\\\\\frac{16{,}777{,}216}{65{,}536}\u0026=256.\\end{aligned} $$ Intuitively, two narrow matrices replace one dense trainable update. The frozen base weights and activations still occupy GPU memory; this ratio does not mean that total memory use or runtime also falls by a factor of 256. Biases and any other modules trained alongside them must be counted separately. What Rank Means This constrains the range of updates that can be represented; it does not assert that every effective full fine-tuning update has low rank. Whether the chosen rank is sufficient depends on the task and the layers being adapted. It should be assessed through validation, rather than parameter counts alone. Full fine-tuning allows this weight matrix to change freely in parameter space, whereas LoRA requires the change to satisfy: $$ \\Delta W=BA $$This imposes a mathematical constraint: $$ \\operatorname{rank}(\\Delta W) = \\operatorname{rank}(BA) \\leq r $$When $r=8$, no matter how $A,B$ are trained, the resulting update matrix will never have a rank greater than $8$. This can be understood by following the path from input to output: $$ x\\in\\mathbb{R}^{4096} \\ \\xrightarrow{A}\\ z\\in\\mathbb{R}^{8} \\ \\xrightarrow{B}\\ \\Delta h\\in\\mathbb{R}^{4096} $$ Let the eight columns of $B$ be $\\mathbf b_1,\\ldots,\\mathbf b_8$. Then: $$ \\Delta h=Bz =z_1\\mathbf b_1+z_2\\mathbf b_2+\\cdots+z_8\\mathbf b_8 $$This means that for this trained LoRA branch, the output corrections produced by different inputs are all combinations of at most eight independent directions. Specifically: $A$ determines how much of each direction to use based on the input, by computing $z_1,\\ldots,z_8$. $B$ provides the output directions that can be combined. During training, both $A$ and $B$ l","d":"LoRA notes: low-rank updates, parameter counts, scaling, initialization, and common variants.","dt":"2026-10-08","s":"blog","t":"LoRA","u":"/blog/lora/"},{"c":"MCP (Model Context Protocol) MCP has three layers: the upstream, the MCP protocol, and the downstream. The downstream is called the MCP Server and is provided by developers or vendors. It implements the specific business logic and registers and exposes three core types of assets through the MCP specification: Tools: functions that the model can call, such as Git operations, SQL queries, and calls to internal microservices. Resources: data context that the model can attach in read-only form, such as local files, table schemas, and log streams. Prompts: standardized prompt templates and workflow presets for interactions. The upstream is called the MCP Client or host environment: commonly used agent engines such as Codex, Claude Code, and Pi. As clients, they manage lifecycles, protocol translation, and permissions: Establish process or network connections with multiple configured MCP Servers. Automatically discover the Tools and Resources exposed by each Server and pass them to the LLM when constructing the prompt. When the model decides to call a tool, assemble a request according to the standard protocol, dispatch it to the target Server, and return the execution result to the reasoning loop. Then there is the MCP protocol itself. It is a set of standards—a communication and lifecycle contract based on JSON-RPC 2.0—that strictly defines message formats, state transitions, and capability negotiation between the upstream and downstream, much like protocols such as TCP/IP. Take searching GitHub projects and sorting them by star count as an example of the overall workflow: the host obtains the search tool’s name, description, and input Schema through MCP, then provides the tool definition to the model. The model generates the tool name and arguments that conform to the Schema. After checking them, the host sends the request to the MCP Server through tools/call. The Server calls the GitHub API and returns the result, which the host then passes to the model. The tool provider defines the Schema; the model generates the call arguments. MCP unifies capability descriptions and invocation interfaces, reducing the need to repeatedly write communication adapters for different services. Each service’s business logic, authentication methods, permission requirements, and error handling still need to be implemented or configured separately. The host can map MCP tools into the tool-calling format supported by the model, so the model usually does not need to handle MCP messages directly. Resources and Prompts use their respective protocol interfaces; their contents should not all be converted into tools or automatically injected into the context by default. A major advantage of the MCP protocol is its rich community ecosystem. Anthropic and community developers have already created many ready-to-use MCP servers covering file systems, databases, API services, and other scenarios. This means you do not need to write tool adapters from scratch and can directly use these verified servers. Awesome MCP Servers (https://github.com/punkpeye/awesome-mcp-servers), a community-maintained curated list of MCP servers. MCP Servers Website (https://mcpservers.org/), the official MCP server directory website. Official MCP Servers (https://github.com/modelcontextprotocol/servers), servers officially maintained by Anthropic. Custom MCP servers can meet specific needs by encapsulating business logic, accessing private data, optimizing performance for particular use cases, or adding custom functionality. You can also publish your own MCP servers on community platforms. A2A (Agent-to-Agent) The MCP protocol addresses interactions between agents and tools, while the A2A protocol addresses collaboration between agents. In a task that requires multiple agents—such as a researcher, a writer, and an editor—to work together, they need to communicate, delegate tasks, negotiate capabilities, and synchronize their states. The traditional collaboration architecture, which is","d":"Notes on MCP, A2A, and ANP.","dt":"2026-10-05","s":"blog","t":"MCP, A2A, ANP","u":"/blog/mcp-a2a-anp/"},{"c":"After setting up the environment in .config/mini-swe-agent, we run: bash 5 lines python -m minisweagent.run.mini \\ -c mini.yaml \\ -t \"Create a test_agent.txt file and write Hello Agent inside it\" \\ --model \"deepseek/deepseek-chat\" \\ --agent-class minisweagent.agents.default.DefaultAgent This produces the latest execution trajectory in .config/mini-swe-agent. We can interpret the source code based on this trajectory. The framework of the entire Repo: run/mini.py:99-101: python 3 lines model = get_model(config=config.get(\"model\", {})) env = get_environment(config.get(\"environment\", {}), default_type=\"local\") agent = get_agent(model, env, config.get(\"agent\", {}), default_type=\"interactive\") Prompt Injection In default.py, we see: python 4 lines self.add_messages( self.model.format_message(role=\"system\", content=self._render_template(self.config.system_template)), self.model.format_message(role=\"user\", content=self._render_template(self.config.instance_template)), ) This performs prompt injection and user querying. The prompt is in mini.yaml, which instructs that if the Agent completes the task, the first sentence output should be echo COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT. This is very useful for our later task verification. Tool Action Next, observing the trajectory’s content and tool calls: json 11 lines \"content\": \"I'll start by analyzing the current directory structure to understand the environment.\", \"role\": \"assistant\", \"tool_calls\": [{ \"index\": 0, \"function\": { \"arguments\": \"{\\\"command\\\": \\\"pwd \u0026\u0026 ls -la\\\"}\", \"name\": \"bash\" }, \"id\": \"call_00_61SsIsH70VtP63w7RQ6G5957\", \"type\": \"function\" }], Expand all 11 lines The extra field of an assistant message has exactly four keys: json 6 lines extra = { \"actions\": [{\"command\": \"pwd \u0026\u0026 ls -la\", \"tool_call_id\": \"call_00_...\"}], \"response\": {...}, # The raw model response, inserted entirely \"cost\": 0.000327, \"timestamp\": 1789796928.007539, } extra is not sent to the model. On the model side, there is: python 2 lines def _prepare_messages_for_api(self, messages: list[dict]) -\u003e list[dict]: prepared = [{k: v for k, v in msg.items() if k != \"extra\"} for msg in messages] The extra field also contains usage costs and more specific messages for statistical purposes. Checking the agent’s tools, we find this agent only has one tool, which is bash. The advantage of this is that no additional sandbox installation is needed. Furthermore, read/edit/write/grep are essentially subsets of it—adding a read tool is equivalent to wrapping a shell around a small piece of bash’s capability. But in reality, specialized tools offer better precision and completeness, making them more suitable for model invocation. Conversely, using only bash heavily tests the model’s fundamental capabilities, though current models are indeed capable of handling the vast majority of scenarios. Flag / Return The exit mechanisms of Mini-Swe-Agent are divided into three types, defining Submitted, LimitsExceeded, and FormatError inherited from a base exception class. They are directly raised in the environment or internal validation points, unified and caught by the top-level main loop’s try...except, and appended with role: \"exit\" in messages. The advantage is the ability to jump out across the call stack with one click, distinguishing between normal completion and abnormal termination. However, this method relies on Python’s exception flow for normal control flow. The three exit mechanisms consist of five exception classes and one synthetic state: code 6 lines InterruptAgentFlow ├── Submitted ├── LimitsExceeded │ └── TimeExceeded ← Note: Inherits from LimitsExceeded, not directly from the base class ├── UserInterruption └── FormatError If there are too many format errors, too many iterations, or a timeout occurs, FormatError and LimitsExceeded are returned: python 19 lines if 0 \u003c self.config.max_consecutive_format_errors \u003c= self.n_consecutive_format_errors: self.add_messages( *e.messages,{ \"role\": \"exit\", \"content\": \"RepeatedFo","d":"A deep dive into the source code of Mini-SWE-Agent based on its execution trajectories, analyzing prompt injection, tool actions, exit mechanisms, and process management.","dt":"2026-09-20","s":"blog","t":"Mini-SWE-Agent Source Code Analysis","u":"/blog/mini-swe-agent/"},{"c":"A Markdown guide consolidating quality community posts, official blogs, research papers, open-source projects, and hands-on engineering practices into an actionable AI Agent learning checklist: Agent-Learning-Hub Anthropic’s engineering blog—a must-read is “Building Effective Agents”: Anthropic Engineering A highly-starred, open-source systematic tutorial on building and implementing agent systems from scratch: Hello-Agents Featured post - Comprehensive Roadmap for AI Application Development Harness-pattern agents: highly recommended to dive directly into the source code of Claude Code and Pi: Learn Claude Code; an insightful deep dive into the Pi agent: pi-coding-agent The LangChain ecosystem: after understanding the core Langchain library, try exploring and reproducing LangGraph and DeepAgents; leverage LangSmith for trace monitoring, trajectory inspection, and agent evaluation. Agent evaluation: an LLM-as-a-Judge framework tailored for LLMs and AI Agents; currently, OpenJudge stands out as a remarkably comprehensive open-source project.","d":"A curated collection of AI Agent learning resources and study paths gathered from various sources, to be continuously updated and expanded.","dt":"2026-09-16","s":"blog","t":"AI Agent Learning Resources \u0026 Roadmap","u":"/blog/ai-agent-resources-for-now/"},{"c":"These notes organize concepts, examples, derivations, and revision reminders by topic, with the original figures. Apply each result under its stated assumptions and the model and data specified by the exercise. Data Structures Linked lists Disjoint-set union (code, optimization operations, and visualization of root merging) Implementations of Prim’s and Dijkstra’s algorithms without priority queues Simulation of orthogonal lists and adjacency multilists (be able to draw them) Heap implementation code, and code that uses a heap to implement a priority queue Red-black trees (definitions and conclusions, insertion and deletion) Loser trees and replacement selection (loser tree + sequence numbers), the role of loser trees in merging, and the depth of a loser tree Computer Organization Special floating-point values and flag generation The principles and circuits of multiplication and division, the number of shifts in Booth multiplication and division, and rounding The procedure-call process, stack frames, and their respective responsibilities Multiprocessors and modern processors A comprehensive summary of the single-cycle datapath in Yuan’s textbook Operating Systems Interprocess communication methods Simplifying resource-allocation graphs and determining the conditions for deadlock Virtual machines and the detailed SPOOLing process File-system mounting Bitmaps and the grouped linked-list method Multiprocessor scheduling Wear leveling Computer Networks The roles of, and differences between, the layers of OSI and TCP/IP Physical-layer line coding for digital data, such as Manchester encoding; distinguish this from analog-signal modulation. PPP and P2P protocols Virtual circuits OSPF, BGP, and DHCP messages and their specific addresses The IP multicast process and IGMP Hamming-code checking and error correction Reservation-time calculations and NAV-value calculations in 802.11 Mobile IP Data Structures Things to remember: the various implementation procedures for external sorting; the three types of threaded binary trees; the applicability, purposes, and drawing methods of orthogonal lists and adjacency multilists; be able to write out the complete manual calculation processes for Dijkstra’s algorithm and the critical path; insertion and deletion in balanced trees, red-black trees, and B-trees; loser-tree sequence numbers and their use in replacement-selection sorting Stacks and Queues In linked queues and stacks, insertion and deletion update node links as well as the appropriate front, rear, or stack-top pointers. Inserting into an empty queue normally initializes both front and rear. After deleting the last data node, restore the empty-queue pointer state: without a sentinel, both front and rear are NULL; with a sentinel, the rear normally points back to it. Stack insertion and deletion operate at the top. The logic of a prefix expression is simply the reverse of a postfix expression: a postfix expression is converted from left to right, and a prefix expression is exactly the opposite. Here is a manual calculation example for ${((1 + 2) * 3) - (4 + 2)}$: work from left to right and from the inside outward, moving the operators to the front. ${(1+2)}$ becomes ${(+12)*3}$; then handle the right-hand side, ${(4+2) \\Rightarrow (+42)}$. Thus, ${((+12)*3)-(+42)\\Rightarrow (*+123)-(+42)\\Rightarrow -*+123+42}$. For a computer program, the steps are as follows: Reverse the infix expression. Replace every left parenthesis ( with a right parenthesis ), and every right parenthesis ) with a left parenthesis (. Apply the infix-to-postfix algorithm to the reversed token sequence, using a stack for operators. Reverse the handling of associativity for equal-precedence operators as well; do not reuse the usual pop condition unchanged. Treat each multidigit number as a single token. Reverse the final result again to obtain the prefix expression. For both prefix and postfix expressions, the idea for conversion to infix is the same: use a stack to ho","d":"Complete revision notes for data structures, computer organization, operating-system scenarios, and computer networks.","dt":"2026-09-09","s":"blog","t":"408 Exam Notes: Data Structures, Computer Organization, OS, and Networks","u":"/blog/kaoyan-408/"},{"c":"These notes organize concepts, examples, derivations, and revision reminders by topic, with the original figures. Apply each result under its stated assumptions and the model and data specified by the exercise. When working on a new problem, recognize its similarity to a previous problem. While solving it, discover gaps in your understanding of this type of problem, its tools, and its methods, or find that your understanding is not deep enough. By solving the new problem, update and refine your previous understanding at the same time. Limits, Derivatives, and Differentials Algebraic Transformations, Recurrences, and Product Derivatives For the hyperbolic sine function $y=\\dfrac{e^x-e^{-x}}{2}$, solve for the inverse as follows: First rewrite it as $2y+e^{-x}=e^x$, multiply both sides by $e^x$, and set $u=e^x\\gt 0$. The quadratic equation is $u^2-2yu-1=0$. Only the root $u=y+\\sqrt{y^2+1}$ is positive, so the inverse is $x=\\ln(y+\\sqrt{y^2+1})$. For the recurrence $x_n=f(x_{n-1})$, setting consecutive terms equal to $a$ gives the fixed-point equation $a=f(a)$. It identifies constant solutions or candidate limits. To find a sequence’s limit this way, first prove convergence and justify passing the limit through the recurrence, for example by continuity of $f$ at the limit. An arbitrary $n$th term still requires further work with the recurrence and initial value. $(660_{18})$ If $f(x)=u(x)v(x)$, the Taylor expansion of $f(x)$ is the product of the Taylor expansions of these two functions. This also shows that infinitesimals can be multiplied. The derivative of $f(x)=u(x)v(x)$ $f'(x)=u(x)v'(x)+v(x)u'(x)$; the proof is on page $p_{167}$. If a product of many factors or an $n$th derivative cannot be found directly, try simplifying it into the form above, then differentiate directly or use Leibniz’s formula: $$ \\frac{d^n}{dx^n} [u(x) v(x)] = \\sum_{k=0}^{n} \\binom{n}{k} u^{(k)}(x) v^{(n-k)}(x) $$Alternatively, use Taylor expansions, as with the multiplication of polynomials mentioned in the first point. Infinity, Equivalent Infinitesimals, and Power Limits Tending to infinity describes a limiting process; no individual value has to equal infinity. Under the absolute-value convention, for every $M\\gt 0$, all values sufficiently close to the limiting point must satisfy $|f(x)|\\gt M$; for a sequence, this must hold at every sufficiently large index. Unboundedness only requires exceeding arbitrarily large bounds somewhere, not remaining beyond them afterward. For example, $\\dfrac{1}{x^2}\\sin\\dfrac{1}{x}$ is unbounded as $x\\to0$, but its value is zero along a sequence of inputs tending to zero, so its absolute value does not tend to infinity. If $\\lim_{n\\to\\infty}x_ny_n=+\\infty$, consider the following deductions: We cannot conclude that at least one of $x_n$ and $y_n$ tends to $+\\infty$, although at least one must be unbounded; otherwise their product would be bounded. For example, let $x_n$ equal $1$ at odd indices and $n$ at even indices, and let $y_n$ equal $n$ at odd indices and $1$ at even indices. Then $x_ny_n=n\\to+\\infty$, but neither factor tends to infinity, since each equals $1$ infinitely often. If $x_n\\to a\\ne0$, then $x_n$ eventually has the sign of $a$ and stays away from zero. Since $y_n=(x_ny_n)/x_n$, we obtain $y_n\\to+\\infty$ when $a\\gt0$, and $y_n\\to-\\infty$ when $a\\lt0$. If the premise only states $|x_ny_n|\\to\\infty$, the conclusion is $|y_n|\\to\\infty$. For functions, if both factors tend to infinity in absolute value, their product does too. If both tend to $+\\infty$, the product also tends to $+\\infty$. Oscillation in sign may prevent a signed infinite limit, but it does not invalidate the absolute-value conclusion. Mere unboundedness is insufficient for this product-limit rule. Sequences obey the same rule as functions: if $|x_n|\\to\\infty$ and $|y_n|\\to\\infty$, then $|x_ny_n|\\to\\infty$; if both tend to $+\\infty$, so does the product. For any $M\\gt0$, all sufficiently large indices simultaneously satisfy $|x_n|\\gt\\sqrt ","d":"Complete revision notes for calculus and linear algebra.","dt":"2026-09-09","s":"blog","t":"Graduate Entrance Exam Math Notes: Calculus and Linear Algebra","u":"/blog/kaoyan-math/"},{"c":"Array Replacement - Rating 1700 Today is Thursday, August 27, 2026. Problem link: Array Replacement. The approach to this problem is somewhat subtle, so we can only approach it from the perspective of the operation itself. This is actually a classic swap-difference problem. Consider a triplet ($a_{i-1}$, $a_i$, $a_{i+1}$). After performing one operation on it, we obtain ($a_i$, $a_{i - 1} - a_i + a_{i + 1}$, $a_{i + 1}$). Let’s consider under what conditions replacing $a_i$ with $a_{i - 1} - a_i + a_{i + 1}$ is optimal. Clearly, the replacement should be made when $a_i \u003c a_{i - 1} - a_i + a_{i + 1}$. We can rearrange this expression to $a_i - a_{i + 1} \u003c a_{i - 1} - a_i$. Noting that this is very similar to the difference array, we consider defining an array $D$ such that $D_i = a_{i - 1} - a_i$. Let’s examine how operations on $a_i$ affect the array $D$: For $D_i$, it becomes $a_{i - 1} - (a_{i - 1} - a_i + a_{i + 1}) = a_{i} - a_{i + 1} = D_{i + 1}$. For $D_{i + 1}$, it becomes $(a_{i - 1} - a_i + a_{i + 1}) - a_{i + 1} = a_{i - 1} - a_i = D_{i}$. That is, in a single operation, we swap $D_i$ and $D_{i + 1}$, but only if they have the same parity. Therefore, we maintain a segment of $D$ values with the same parity, sort them by maximum value within that segment, and subtract the largest difference value from the head each time to ensure the lexicographical order is minimized. The code is as follows: cpp 30 lines // Retired? #include \u003cbits/stdc++.h\u003e #define int long long using namespace std; const int N = 1e6 + 10, mod = 1e9 + 7; void solve(){ int n; cin \u003e\u003e n; vector\u003cint\u003e a(n + 1), d(n + 1); for(int i = 1; i \u003c= n; i++) cin \u003e\u003e a[i], d[i] = a[i - 1] - a[i]; int l = 2; while (l \u003c= n) { int r = l; while (r \u003c= n \u0026\u0026 abs(d[l] % 2) == abs(d[r] % 2)) r++; sort(d.begin() + l, d.begin() + r); reverse(d.begin() + l, d.begin() + r); l = r; } int x = a[1]; cout \u003c\u003c x \u003c\u003c ' '; for(int i = 2; i \u003c= n; i++) x -= d[i], cout \u003c\u003c x \u003c\u003c ' '; cout \u003c\u003c '\\n'; } signed main(){ std::ios::sync_with_stdio(false), cin.tie(0), cout.tie(0); int t; cin \u003e\u003e t; while (t--) solve(); } Expand all 30 lines Robin Hood Archery - Rating 1900 Today is Friday, August 28, 2026. Problem link: Robin Hood Archery. First, let’s consider under what circumstances the Sheriff will not lose. Since Robin Hood always goes first, if there is a choice between a larger and a smaller number, he will definitely pick the larger one. Therefore, the only scenario where the Sheriff does not lose is a tie. Because he is always picking second, the only way they can get the same score—and thus ensure the Sheriff doesn’t lose—is if the numbers can be perfectly paired up (i.e., they are identical). Next, looking at the problem requirements, the number of queries is quite large. This means we need to frequently update and query within intervals. Furthermore, the queries do not need to be processed online (meaning we can compute the answers out of order and then output them in the original order of the questions). Therefore, we can consider using Mo’s Algorithm, which allows us to perform range modifications and queries with a time complexity of $O(n \\sqrt{n})$. We maintain a variable $res$. When the frequencies of all numbers within the interval are even, $res = 0$. The specific code is as follows: cpp 57 lines // Retired? #include \u003cbits/stdc++.h\u003e #define int long long using namespace std; const int N = 1e6 + 10, mod = 1e9 + 7; int pos[N], a[N], cnt[N], mp[N]; struct node{ int l, r, now; bool operator\u003c (const node \u0026w) const{ return (pos[l] ^ pos[w.l]) ? pos[l] \u003c pos[w.l] : ((pos[l] \u0026 1) ? r \u003c w.r : r \u003e w.r); } }p[N]; void solve(){ int n, m; cin \u003e\u003e n \u003e\u003e m; int len = sqrt(n); for(int i = 1; i \u003c= n; i++) cin \u003e\u003e a[i], pos[i] = i / len; for(int i = 1; i \u003c= m; i++){ int l, r; cin \u003e\u003e l \u003e\u003e r; p[i] = {l, r, i}; } sort(p + 1, p + 1 + m); int res = 0; auto add = [\u0026](int x) -\u003e void{ mp[a[x]]++; if(mp[a[x]] % 2 == 0) res--; else res++; }; auto del = [\u0026](int x) -\u003e void{ mp[a[x]]--; if(mp[a[x]] % 2 == 0) res--; el","d":"Starting Monday, solve one Codeforces problem each day, in order of increasing difficulty","dt":"2026-09-07","s":"blog","t":"Daily Codeforces Problem","u":"/blog/daily-question/"},{"c":"Overview Paper link: Retrieval-Augmented Generation Overall RAG architecture The overall process is as follows: for a specific question, the system first uses MIPS to retrieve relevant documents from the knowledge base, then “passes” them to the large model, which combines that knowledge to produce the final answer. Questions: the concrete details of the MIPS method; how the documents are passed to the large model (by augmenting the context, by modifying the query, or by making changes at the token level); what marginalization is, and why it is needed. RAG-Sequence Model and RAG-Token Model RAG-Sequence: the generated answer relies on only a single document. This model suits cases where the answer is strongly coherent and comes from a single source. When decoding the answer, the model generates a complete candidate answer sequence separately for each document, and at the end evaluates which complete sequence has the highest overall probability. $$p_{\\text{RAG-Sequence}}(y\\vert{}x) \\approx \\sum_{z \\in \\text{top-k}(p(\\cdot\\vert{}x))} p_\\eta(z\\vert{}x) p_\\theta(y\\vert{}x, z) = \\sum_{z \\in \\text{top-k}(p(\\cdot\\vert{}x))} p_\\eta(z\\vert{}x) \\prod_{i}^N p_\\theta(y_i\\vert{}x, z, y_{1:i-1})$$RAG-Token: every time a token is generated, the model consults the different documents again, synthesizing information from multiple sources; this suits answers that need to be pieced together from several sources. The retriever finds the top-$k$ documents. When the generator is about to predict the next word, it computes a next-word probability distribution based on each of the $k$ documents, aggregates them (marginalizes) to settle on that word; when predicting the word after that, it re-examines the $k$ documents and repeats the process. $$p_{\\text{RAG-Token}}(y\\vert{}x) \\approx \\prod_{i}^N \\sum_{z \\in \\text{top-k}(p(\\cdot\\vert{}x))} p_\\eta(z\\vert{}x) p_\\theta(y_i\\vert{}x, z, y_{1:i-1})$$Retriever: DPR DPR employs a Bi-encoder architecture, whose core working mechanism is as follows: Text vectorization: Document encoder: A $\\text{BERT}_{\\text{BASE}}$-based document encoder is used to map each document $z$ to a dense vector representation, denoted as $d(z)$. The formula is expressed as: $d(z) = \\text{BERT}_d(z)$. Query encoder: Similarly, a $\\text{BERT}_{\\text{BASE}}$-based query encoder is used to map the user’s input query $x$ to a dense vector representation, denoted as $q(x)$. The formula is: $q(x) = \\text{BERT}_q(x)$. Similarity Calculation and Probability Distribution: The probability $p_\\eta(z\\vert{}x)$ of retrieving a particular document is proportional to the exponential of the dot product (inner product) between the query vector and the document vector. Mathematically, this is expressed as: $$p_\\eta(z\\vert{}x) \\propto \\exp(d(z)^\\top q(x))$$ “Parametric memory” refers to knowledge that the model compresses and stores inside itself through its weights (e.g., the connection weights of a neural network) during training. The parameters of pre-trained models such as BERT or BART are their parametric memory. “Non-parametric memory” is entirely different. It is like an external knowledge base, or an external hard drive attached to the model; the knowledge is kept outside the model in raw, standalone data form. MIPS is responsible for quickly finding, among the vast collection of knowledge-base vectors, the top-$k$ vectors with the largest inner product (i.e., the most similar ones) with ${\\displaystyle q(x)}$. Computing dot products one by one would be far too slow, and MIPS exists precisely to do this quickly. In the paper, the concrete implementation uses the FAISS library combined with the HNSW algorithm. RAG’s Fine-tuning Process (Overall Workflow) Forward: The user inputs a question $x$. $\\text{BERT}_q$ turns it into a vector $q(x)$. $q(x)$ is used to run inner-product search (MIPS) over the frozen document store, retrieving the Top-K documents $z$ and their retrieval probabilities $p(z\\vert{}x)$. The generator BART concatenates the questi","d":"Notes and Reflections on the Paper “Retrieval-Augmented Generation”","dt":"2026-08-30","s":"blog","t":"NOTE：Retrieval-Augmented Generation","u":"/blog/noterag/"},{"c":"This collection reorganizes the algorithm notes I wrote on cnblogs between 2023 and 2024 into seven chapters. The original reasoning, derivations, and code are preserved (including the asides I scribbled back then); places that only had code now carry explanations, and places that ran long were trimmed slightly. For the unabridged originals, follow the source-note links at the top of each section. Chapter 1: Basic Algorithms 1.1 Prefix Sums and Difference Arrays Prefix sums answer “sum of any interval, many queries”: after preprocessing $s_i = s_{i-1} + a_i$, the sum over $[l, r]$ is $s_r - s_{l-1}$. The 2D version unfolds by inclusion-exclusion: $s_{i,j} = s_{i-1,j} + s_{i,j-1} + a_{i,j} - s_{i-1,j-1}$. cpp 2 lines ///s[i][j] = s[i-1][j] + s[i][j-1] + a[i][j] - s[i-1][j-1] printf(\"%d\\n\", s[x2][y2] - s[x2][y1-1] - s[x1-1][y2] + s[x1-1][y1-1]); A difference array is the inverse: if $a$ is the prefix sum of $b$, “add $c$ on $[l,r]$” is just $b_l \\mathrel{+}= c,\\ b_{r+1} \\mathrel{-}= c$, then one prefix-sum pass rebuilds the array. Three problems tie it together in my second note: Laser Bomb (2D prefix sums over every frame — a 3×3 frame actually traps a 2×2 bomb, so start accumulating at (1,1)); Increasing Sequence (after differencing, with positive sum $res$ and negative sum $ans$, the answer is $\\min(res,ans) + |res-ans|$ and the count of schemes is $|res-ans|+1$); and Classroom Reservations (difference array + binary search on the answer — check(mid) tests whether the first mid orders overflow, turning $O(nm)$ into $O((n+m)\\log m)$; that note keeps my 45-point brute force beside the full version). Source notes: 基础算法大全 · 再谈 前缀和，差分，离散化 1.2 Two Pointers and Coordinate Compression The classic two-pointer problem: longest substring without repeats. The right pointer advances; on a duplicate the left pointer moves until the window is clean. Both pointers travel at most $n$ steps — amortized $O(n)$: cpp 5 lines for (int i = 0; i \u003c n; i++) { s[a[i]]++; while (s[a[i]] \u003e 1) s[a[j++]]--; // until the window has no duplicate res = max(res, i - j + 1); } Coordinate compression handles “huge value range, few distinct values”: sort and dedupe the coordinates that appear, binary search for ranks. Classic application — Program Analysis (P1955): indices reach $10^9$, so compression is mandatory. Merge all equality constraints first, then check each inequality constraint against the merged sets. Source note: 基础算法大全 1.3 Fast Power and Big-Integer Arithmetic Fast power turns $O(k)$ exponentiation into $O(\\log k)$ by splitting the exponent into binary: cpp 8 lines long long qpow(long long a, long long b, long long k) { long long res = 1; while (b) { if (b \u0026 1) res = res * a % k; b \u003e\u003e= 1, a = a * a % k; } return res; } Big integers live in digit arrays (index 1 = ones place). Multiplication’s key fact: the product of position $i$ and position $j$ lands in position $i+j-1$: cpp 3 lines c[i + j - 1] = a[i] * b[j] + t + c[i + j - 1]; t = c[i + j - 1] / 10; c[i + j - 1] %= 10; Source notes: 快速幂 · 高精度 1.4 Greedy My greedy note was a sweep of interval models, one problem each: Interval point-cover / maximum disjoint intervals: sort by right endpoint, take greedily — “the earlier an interval ends, the more room it leaves” covers both questions. Minimum interval grouping: sort by left endpoint; a min-heap tracks each group’s maximum right endpoint. A new interval joins the group with the smallest right endpoint when they don’t intersect, otherwise it opens a new group. The heap size is the answer. Merging fruits: always merge the two smallest piles — a bare min-heap problem. Model transformations: radar placement maps each point to $[x-d, x+d]$ on the axis ($d=\\sqrt{r^2-y^2}$), reducing to point-covering; Tian Ji’s horse racing uses best-vs-best, worst-vs-worst, and sacrifices the worst horse against their best; the fishing problem enumerates the final lake and repeatedly takes the highest-yield lake from a heap. cpp 6 lines priority_queue\u003cint, vector\u003cint\u003e","d":"Two years of algorithm study notes from my cnblogs blog, reorganized into seven chapters: basic algorithms, search, dynamic programming, data structures, strings, graphs, and math","dt":"2026-08-29","s":"blog","t":"Algorithm Notes: The Grand Collection","u":"/blog/algorithm-collection/"},{"c":"Picking a Simulated Business Scenario and Building the Data I chose an automated research assistant, with the data generated randomly by AI. The main structure looks like this: text 5 lines D:\\Shixun\\harness-research\\ ├── data\\ # simulated literature and experiment data (read-only, Worker must not modify) ├── scripts\\ # tool scripts (lit_read / exp_stats / report_export) ... ├── evidence\\ # evidence directory: research notes, analysis logs, review opinions (organized by task_id) ├── .hermes.md # project rules (discovered and injected by Hermes automatically) Designing the Workflows Workflow Orchestration Why this approach Which layer it validates 1. Literature review notes Single Agent, direct execution 6 papers read sequentially, no dependencies and no parallel benefit; a single session can finish Execution layer: does the tool chain work, do Skills load, can audits leave a trail 2. Experiment plan design Parallel delegation via delegate_task The three subtasks — literature evidence / data audit / plan design — are independent of each other, and each needs its own context to avoid cross-contamination Orchestration layer: multi-agent division of labor, structured handoffs, merging by the main Agent 3. Full report review pipeline Kanban Cross-session, ordered dependencies (Planner → Worker → Reviewer), must be able to block and recover after failures State + control layers: state persistence, rework loop, exception recovery Workflow 1 answers “can a single Agent actually get work done” — the prerequisite for everything; Workflow 2 answers “can multiple Agents divide work without interfering with each other”; Workflow 3 answers “can things recover when they go wrong”. Building a Submittable Plugin Creating a versioned plugin Added to the workspace: text 5 lines D:\\Shixun\\harness-research\\plugins\\research_v2\\ ├─ __init__.py ├─ plugin.yaml ├─ schemas.py └─ tools.py plugin.yaml uses the new name: yaml 11 lines name: research_v2 version: 2.0.0 description: 科研 Harness 受限业务工具——规划读取、文献读取、实验统计、证据输出与独立评审 provides_tools: - project_read - lit_read - exp_stats - exp_head - evidence_read - report_export - review_export Expand all 11 lines Splitting tools into role-configurable toolsets Registered by role capability in __init__.py: toolset Tools Role research_planner project_read Planner — reads only the research question and project rules research_read lit_read, exp_stats, evidence_read Worker/Reviewer reading and verification research_write report_export Worker outputs business reports research_review review_export Reviewer outputs review opinions The remaining tools are registered per the table above. Only then can hermes tools enable/disable act as a real tool whitelist per Profile, instead of the three roles sharing all write capabilities. Tool whitelisting via hermes tools enable/disable Creating schemas.py Describes the schema of each role in the Agent setup. schemas.py: per-role schemas Adding a path whitelist in tools.py and forbidding evidence overwrites Using pathlib.Path.resolve(), each class of tool is confined to fixed directories: python 4 lines WORKSPACE_ROOT = Path(r\"D:\\Shixun\\harness-research\").resolve() LITERATURE_ROOT = (WORKSPACE_ROOT / \"data\" / \"literature\").resolve() EXPERIMENT_ROOT = (WORKSPACE_ROOT / \"data\" / \"experiments\").resolve() EVIDENCE_ROOT = (WORKSPACE_ROOT / \"evidence\").resolve() Report writing changed from mode w to exclusive creation x: python 2 lines with open(output_path, \"x\", encoding=\"utf-8\") as f: ... Path whitelisting and exclusive creation in tools.py Adding per-call business auditing No reliance on agent.log. Every tool call writes its own unique JSON audit file: text 1 lines evidence/\u003ctask_id\u003e/audit/\u003ctimestamp\u003e_\u003cuuid\u003e.json The log fields: json 12 lines { \"timestamp\": \"2026-08-08T10:00:00+08:00\", \"task_id\": \"task-003\", \"profile\": \"worker\", \"tool\": \"exp_stats\", \"input_paths\": [\"data/experiments/exp_adversarial.csv\"], \"parameters\": {\"columns\": [\"standard_acc\", \"robust_acc\"], \"group_by\": \"setting\"}, \"s","d":"Building a full harness around an automated research-assistant scenario: plugin, permissions, auditing, three workflows, and automated tests","dt":"2026-08-08","s":"blog","t":"Harness Engineering with Hermes","u":"/blog/hermes-harness/"},{"c":"Education Tianjin UniversitySep 2026 – Jun 2029 (expected) M.Eng. in Computer TechnologyTianjin, China Focus: large language models and agents. Yantai UniversitySep 2022 – Jun 2026 B.Eng. in Computer Science and TechnologyYantai, China Awards Lanqiao Cup National Finals — First Prize (top 3%)Jun 2026 National College Computer Ability Challenge (UG \u0026 PG Group) — First PrizeDec 2024 Group Programming Ladder Tournament — Individual Second Prize (top 6%)May 2026 Lanqiao Cup National Finals — Second PrizeJun 2025 Group Programming Ladder Tournament — Individual Third Prize / Team Third PrizeMay 2025 Chinese Mathematics Competition — First Prize, ShandongNov 2025 Baidu Star Programming Contest — Bronze, ShandongJul 2024 China Collegiate Computing Contest — Third Prize, ShandongOct 2024 Projects Personalized Recommender for Chinese Ethnic Music and Dance Python · PyTorch · Scrapy Accurate recommendation of text, image, audio and video content, using a hybrid approach of scheduled offline feature updates and online real-time scoring. Chinese QA Bot with BERT and mengzi-T5 Python · PyTorch · Transformers A high-accuracy Chinese reading comprehension QA system, mainly addressing two problems: insufficient semantic modeling of long texts and typo interference in user input affecting answer extraction. Chinese–English Translation with Bahdanau Attention and Transformer Python · PyTorch Built two classic NLP translation models from scratch with PyTorch — a Seq2Seq (GRU) with Bahdanau attention and a standard Transformer — for Chinese-English and English-Chinese translation on Opus/Tatoeba. Skills Deep learning — Proficient in building and training models with PyTorch. Self-studied STAT 157 and CS329P, independently implemented FCN/ResNet portrait blurring and sequence generation with Transformers and LSTMs; have Kaggle experience in semantic segmentation and image classification. Algorithms — A*, segment trees, dynamic programming, computational geometry, number theory; also familiar with low-level computer principles. Languages and tools — Python, C/C++, LaTeX; use Git/GitHub for version control, with decent code and documentation habits. English — CET-6; long-term reader of The Economist and other English publications. Interests Competitive programming、Blogging、Sharing. Contact Email: Gmail, QQ Code is on GitHub; older technical notes are on Cnblogs.","d":"","dt":"","s":"","t":"About me","u":"/about/"}]