BLADE: Auditing Information Use for Failure Attribution
Reading notes on BLADE, multi-agent failure attribution, and information-use auditing.
BLADE
BLADE uses Jev for attribution and divides the process into three stages. Notably, Jev makes the decisions in all three stages. The diagram below shows the BLADE framework.

Method
BLADE represents an event as $e_i$ and defines four kinds of event relation: depends on, repeats, contradicts, and repairs. Jev selects the relation between events, with an additional None option. Let $\mathcal{R}_{\mathrm{link}}$ denote the four relation types and let $x_{t,r}$ denote the question about relation $r$ at event $e_t$. The probability assigned to an earlier event $e_u$ is
BLADE retains relations with probability at least $\theta=0.3$, forming the weighted edge set
Not every dependency is useful. Building edges for every qualifying dependency would require $O(N^2)$ complexity, so BLADE prioritizes the depends-on distribution when selecting edges. For each event $e_t$, let $\mathcal{U}_{t}=\{u \lt t:p_{ut}^{(\mathrm{dep})}\geq\theta\}$, where $\mathrm{dep}$ denotes depends-on. Let $\mathcal{S}_{t}\subseteq\mathcal{U}_{t}$ contain the indices of at most $K=2$ sources with the highest $p_{ut}^{(\mathrm{dep})}$. The selected handoffs are
and $\mathcal{H}=\bigcup_{t=2}^{T}\mathcal{H}_{t}$. Audit planning reuses dependency probabilities and requires no additional model calls. Each handoff $(e_u,e_t)$ carries an implicit semantic contract: the receiving event should preserve the source meaning, avoid treating uncertain information as established fact, and follow the task requirements. Unlike formal contract verification, these obligations are assessed through structured semantic judgments. BLADE extracts requirement spans $R$ from task $q$ and includes them with the full trajectory in the audit state $s_{\tau,R}$.
Jev classifies each event as an agent decision or evidence, denoted by $\kappa(e_t)$, and evaluates how each selected receiving event $e_t$ uses its source $e_u$. For question $x_{ut}^{\mathrm{use}}$, Jev assigns probabilities over nine predefined labels $\mathcal{L}$:
Here $\ell_{ut}$ is the most probable label and $\pi_{ut}$ its probability. The labels distinguish potential error introduction, inheritance or repair of an existing error, and cases with no retained finding. They fall into three groups, $\mathcal{L}=\mathcal{L}_{\mathrm{kept}}\cup\mathcal{L}_{\mathrm{breach}}\cup\mathcal{L}_{\mathrm{inh}}$:
- $\mathcal{L}_{\mathrm{kept}}$ represents faithful use, no actual use of the source, or insufficient evidence to assess the handoff.
- $\mathcal{L}_{breach}$ represents potential error introduction through unverified adoption, inapplicable information, misinterpretation, or a correction that introduces a new error.
- $\mathcal{L}_{inh}$ represents propagation of an existing error or repair without introducing a new error.
For handoffs entering an agent decision, BLADE retains non-kept judgments as structured findings. Let $\mathcal{H}^{\mathrm{dec}}=\{(e_{u},e_{t})\in\mathcal{H}:\kappa(e_{t})=\mathrm{decision}\}$. The resulting findings are
Through this construction, BLADE turns the trajectory into a blame graph $G=(V,E,R,F)$, where $E$ contains dependency edges, $V$ contains nodes, and $R$ supplies task requirements for evaluating these interactions.
Given task $q$, the full trajectory $\tau$, and blame graph $G$, Jev answers the same decisive-step question used by the All-at-once baseline, enhanced with structured responsibility evidence:
BLADE’s performance on the evaluation datasets is shown below:

By using Jev throughout attribution, BLADE achieves a fast, low-cost approach. I wonder whether using an LLM to make the final decision after constructing the blame graph could improve results, since reasoning models may have stronger attribution capabilities when working with long contexts and structured language. In practice, Jev’s decision can attribute the failure to only one specific model, whereas some errors may result from a combination of incorrect information. Jev’s output therefore confines the result to a single agent.
BLADE’s evaluation framework also lacks generality and requires some adaptation for different harness frameworks.
Comments