BLADE: Auditing Information Use for Failure Attribution

Nov 10, 2026 4 min

Reading notes on BLADE, multi-agent failure attribution, and information-use auditing.

BLADE

BLADE uses Jev for attribution and divides the process into three stages. Notably, Jev makes the decisions in all three stages. The diagram below shows the BLADE framework.

Figure 1. Overview of BLADE. (1) Jev reconstructs dependencies between events. (2) The dependencies select the handoffs to audit, and Jev checks how each agent decision uses its source against the task’s requirement spans $R$. (3) The diagnosis reads the full trajectory with the blame graph $G=(V,E,R,F)$; every event remains a candidate.
Figure 1. Overview of BLADE. (1) Jev reconstructs dependencies between events. (2) The dependencies select the handoffs to audit, and Jev checks how each agent decision uses its source against the task’s requirement spans $R$. (3) The diagnosis reads the full trajectory with the blame graph $G=(V,E,R,F)$; every event remains a candidate.

Method

BLADE represents an event as $e_i$ and defines four kinds of event relation: depends on, repeats, contradicts, and repairs. Jev selects the relation between events, with an additional None option. Let $\mathcal{R}_{\mathrm{link}}$ denote the four relation types and let $x_{t,r}$ denote the question about relation $r$ at event $e_t$. The probability assigned to an earlier event $e_u$ is

$$ p_{ut}^{(r)}=P_{\mathrm{Jev}}(e_{u}\mid s_{\tau},x_{t,r}). $$

BLADE retains relations with probability at least $\theta=0.3$, forming the weighted edge set

$$ E=\bigl\{ (e_{u},e_{t},r,p_{ut}^{(r)}):u \lt t, r\in\mathcal{R}_{\mathrm{link}},\ p_{ut}^{(r)}\geq\theta\bigr\}. $$

Not every dependency is useful. Building edges for every qualifying dependency would require $O(N^2)$ complexity, so BLADE prioritizes the depends-on distribution when selecting edges. For each event $e_t$, let $\mathcal{U}_{t}=\{u \lt t:p_{ut}^{(\mathrm{dep})}\geq\theta\}$, where $\mathrm{dep}$ denotes depends-on. Let $\mathcal{S}_{t}\subseteq\mathcal{U}_{t}$ contain the indices of at most $K=2$ sources with the highest $p_{ut}^{(\mathrm{dep})}$. The selected handoffs are

$$ \mathcal{H}_{t}=\bigl\{(e_{u},e_{t}):u\in\mathcal{S}_{t}\bigr\}, $$

and $\mathcal{H}=\bigcup_{t=2}^{T}\mathcal{H}_{t}$. Audit planning reuses dependency probabilities and requires no additional model calls. Each handoff $(e_u,e_t)$ carries an implicit semantic contract: the receiving event should preserve the source meaning, avoid treating uncertain information as established fact, and follow the task requirements. Unlike formal contract verification, these obligations are assessed through structured semantic judgments. BLADE extracts requirement spans $R$ from task $q$ and includes them with the full trajectory in the audit state $s_{\tau,R}$.

Jev classifies each event as an agent decision or evidence, denoted by $\kappa(e_t)$, and evaluates how each selected receiving event $e_t$ uses its source $e_u$. For question $x_{ut}^{\mathrm{use}}$, Jev assigns probabilities over nine predefined labels $\mathcal{L}$:

$$ p_{ut}(\ell)=P_{\mathrm{Jev}}\big(\ell\mid s_{\tau,R},x^{\mathrm{use}}_{ut}\big),\qquad\ell\in\mathcal{L}. $$

Here $\ell_{ut}$ is the most probable label and $\pi_{ut}$ its probability. The labels distinguish potential error introduction, inheritance or repair of an existing error, and cases with no retained finding. They fall into three groups, $\mathcal{L}=\mathcal{L}_{\mathrm{kept}}\cup\mathcal{L}_{\mathrm{breach}}\cup\mathcal{L}_{\mathrm{inh}}$:

  1. $\mathcal{L}_{\mathrm{kept}}$ represents faithful use, no actual use of the source, or insufficient evidence to assess the handoff.
  2. $\mathcal{L}_{breach}$ represents potential error introduction through unverified adoption, inapplicable information, misinterpretation, or a correction that introduces a new error.
  3. $\mathcal{L}_{inh}$ represents propagation of an existing error or repair without introducing a new error.

For handoffs entering an agent decision, BLADE retains non-kept judgments as structured findings. Let $\mathcal{H}^{\mathrm{dec}}=\{(e_{u},e_{t})\in\mathcal{H}:\kappa(e_{t})=\mathrm{decision}\}$. The resulting findings are

$$ F=\bigl\{ (e_{u},e_{t},\ell_{ut},\pi_{ut}): (e_{u},e_{t})\in\mathcal{H}^{\mathrm{dec}},\ \ell_{ut}\notin\mathcal{L}_{\mathrm{kept}}\bigr\}. $$

Through this construction, BLADE turns the trajectory into a blame graph $G=(V,E,R,F)$, where $E$ contains dependency edges, $V$ contains nodes, and $R$ supplies task requirements for evaluating these interactions.

Given task $q$, the full trajectory $\tau$, and blame graph $G$, Jev answers the same decisive-step question used by the All-at-once baseline, enhanced with structured responsibility evidence:

$$ \hat{t} =\arg\max_{t\in\{1,\dots,T\}}P_{\mathrm{Jev}}\bigl(e_{t}\mid q,\tau,G\bigr), \hat{a} =a(e_{\hat{t}}). $$

BLADE’s performance on the evaluation datasets is shown below:

Table 1. Agent and step accuracy (%) without the reference answer. All methods are instantiated under the same Jev decision interface (Appendix B.2). HC: Hand-Crafted; AG: Algorithm-Generated. Macro Avg. equally weights the five subsets. Bold marks the best and underline the second best.
Table 1. Agent and step accuracy (%) without the reference answer. All methods are instantiated under the same Jev decision interface (Appendix B.2). HC: Hand-Crafted; AG: Algorithm-Generated. Macro Avg. equally weights the five subsets. Bold marks the best and underline the second best.

By using Jev throughout attribution, BLADE achieves a fast, low-cost approach. I wonder whether using an LLM to make the final decision after constructing the blame graph could improve results, since reasoning models may have stronger attribution capabilities when working with long contexts and structured language. In practice, Jev’s decision can attribute the failure to only one specific model, whereas some errors may result from a combination of incorrect information. Jev’s output therefore confines the result to a single agent.

BLADE’s evaluation framework also lacks generality and requires some adaptation for different harness frameworks.

Comments