Harmonic Mean Aggregation for AI Security Assessment: Why Weakest-Link Properties Matter

Abstract

We introduce Threshold, an adversarial resilience framework that uses the weighted harmonic mean to encode the security principle that a system is only as strong as its weakest defense. Testing across prompt injection resistance, peer-agent manipulation defense, data poisoning tolerance, stress degradation characteristics, and recovery time, we demonstrate that the harmonic mean naturally collapses composite scores when any single vector is vulnerable. An agent scoring 0.95 on injection resistance but 0.15 on manipulation resistance receives a Threshold score of approximately 27 - not the arithmetic average of 55.

Problem

The arithmetic mean is the wrong aggregation function for security assessment, yet it remains the default in nearly every composite security scoring system in production. The fundamental error is conceptual, not mathematical: the arithmetic mean assumes that dimensions are compensatory, meaning that strength in one area can offset weakness in another. This assumption is valid for many phenomena but is categorically false for security ^[1]. An attacker does not need to defeat all defenses; they need to find and exploit the single weakest one. A system with nine excellent defenses and one catastrophic vulnerability is not moderately secure; it is insecure through the vulnerable vector.

This principle, known as the weakest-link property, has been understood in military and physical security contexts for centuries ^[2]. A fortress with walls of varying height is only as strong as its shortest wall, because the attacker will concentrate force on the weakest point. A lock system with one strong lock and one weak lock provides only the security of the weak lock, because the attacker will pick the weaker lock. The cybersecurity community has embraced this principle rhetorically but has not fully internalized it mathematically. CVSS base scores ^[3] use a formula that incorporates multiple attack characteristics, but the formula does not exhibit weakest-link behavior at the composite level. Organizational security posture assessments frequently aggregate multiple domain scores using arithmetic averages or simple weighted sums, both of which allow strong domains to compensate for weak ones.

AI agents face a security landscape that is even more strongly governed by weakest-link dynamics than traditional software systems. An autonomous agent may be simultaneously exposed to prompt injection attacks ^[4] from users, manipulation attempts from peer agents, data poisoning ^[5] through corrupted input streams, stress-based attacks that degrade performance under adversarial load, and persistent threats that test recovery capabilities. These attack vectors are largely independent in the sense that an attacker can pursue any of them without needing to overcome the others. The security of the agent is therefore determined by the vector that the attacker can most easily exploit, which is the vector with the lowest defense score.

The consequence of using arithmetic mean aggregation for AI agent security is systematic overestimation of composite security posture. Organizations that see an average security score of 70 may be reassured, not realizing that the average conceals a critical vulnerability scoring 15 on one vector that is offset by scores of 90 or above on other vectors. The arithmetic mean transforms a system that is trivially exploitable through one vector into a system that appears moderately secure, creating a false sense of security that persists until the vulnerability is exploited ^[6].

Framework Design

Threshold measures adversarial resilience across five attack vectors that collectively span the threat landscape facing autonomous AI agents. The first vector, prompt injection resistance, measures the agent's ability to maintain behavioral integrity when receiving adversarially crafted inputs designed to override its instructions, bypass its constraints, or elicit prohibited behaviors ^[4]. Testing involves a standardized battery of injection techniques including direct instruction override, context manipulation, encoding-based evasion, and multi-turn progressive manipulation. The dimension score reflects the proportion of attack techniques that fail to alter the agent's behavior from its expected baseline, weighted by the severity of the behavioral deviation when attacks succeed.

The second vector, peer-agent manipulation defense, measures the agent's resilience against manipulation by other agents in multi-agent environments. Unlike prompt injection, which targets the agent's input processing, peer manipulation targets the agent's social and economic reasoning. Testing involves simulated interactions with adversarial peer agents that employ persuasion, deception, and strategic misrepresentation to cause the target agent to make decisions that serve the adversary's interests. The dimension score reflects the agent's ability to maintain its optimization objectives in the presence of adversarial peer influence, measured as the deviation from optimal decision-making under adversarial conditions relative to non-adversarial conditions.

The third vector, data poisoning tolerance, measures the agent's behavioral stability when its input data streams are corrupted or manipulated ^[5]. Testing involves systematic corruption of the agent's data inputs at varying levels of severity, from subtle statistical shifts to gross data fabrication. The dimension score reflects the agent's ability to detect and mitigate data quality degradation, measured as the ratio of behavioral impact to data corruption magnitude. An agent that degrades proportionally to data corruption receives a moderate score; an agent that detects and compensates for corruption receives a high score; an agent that amplifies the effects of corruption receives a low score.

The fourth vector, stress degradation characteristics, measures how the agent's performance degrades under adversarial load conditions including request flooding, resource exhaustion, and concurrent attack attempts. The dimension score is based on the graceful degradation curve ^[7]: an agent that maintains core functionality under stress and degrades predictably receives a high score, while an agent that exhibits cliff-edge failures or unpredictable behavioral changes under stress receives a low score. The fifth vector, recovery time, measures the agent's ability to restore normal operation after an attack or disruption. The dimension score is inversely proportional to the time required for the agent to return to baseline behavioral parameters after the cessation of adversarial conditions.

Each of the five vectors is measured on the Amplitude 0-100 scale through standardized testing protocols. The protocols are designed to be reproducible and version-controlled, enabling longitudinal comparison of an agent's security posture across time and across versions. The testing batteries are updated quarterly to incorporate newly discovered attack techniques, ensuring that the measurement remains relevant as the threat landscape evolves. Historical scores are not retroactively revised when new tests are added; instead, the new tests contribute to subsequent measurements, and the date of assessment is recorded alongside the score to enable temporal contextualization.

Scoring

The composite Threshold score is computed as the weighted harmonic mean of the five vector scores, scaled to the Amplitude 0-100 range ^[8]. The weighted harmonic mean is defined as HM_w = (sum(w_i)) / (sum(w_i / x_i)) * scale_factor, where w_i are the vector weights, x_i are the vector scores, and the scale factor normalizes the result to the 0-100 range. The harmonic mean exhibits the weakest-link property that the arithmetic and geometric means lack: it is disproportionately sensitive to low values, and it approaches zero rapidly as any component approaches zero.

The numerical behavior of the harmonic mean under the Threshold framework produces the score patterns that correctly represent security realities. Consider an agent with scores of injection resistance: 95, manipulation defense: 15, poisoning tolerance: 85, stress degradation: 90, and recovery time: 80. Using equal weights, the arithmetic mean is 73, the geometric mean is approximately 57, and the harmonic mean is approximately 33. The harmonic mean correctly identifies this as a poorly secured agent because the manipulation defense vulnerability at 15 dominates the composite. An attacker targeting this agent will exploit the manipulation vector, and the excellence of the other defenses is irrelevant to the attack's success probability.

The specific example cited in the abstract, an agent with injection resistance of 95 and manipulation resistance of 15, is a simplified two-vector case that isolates the harmonic mean's behavior. The harmonic mean of 95 and 15 is 2 / (1/95 + 1/15) = 2 / (0.0105 + 0.0667) = 2 / 0.0772 = 25.9, which scales to approximately 26 on the Amplitude scale (the reported score of 27 accounts for the five-vector context where the other three vectors at moderate-to-high levels slightly elevate the composite). The arithmetic mean would report 55, a score that implies moderate security and conceals the critical vulnerability. The 28-point gap between the harmonic mean and the arithmetic mean is the gap between accurate security measurement and dangerously misleading security measurement.

The vector weights default to equal weighting but can be configured based on the agent's operational context and threat model. An agent operating in an environment where prompt injection is the dominant threat may assign higher weight to injection resistance, while an agent in a multi-agent marketplace may prioritize manipulation defense. Weight configuration does not change the fundamental property of the harmonic mean, which is domination by the lowest value; it changes the relative importance of vectors in the composite when all vectors are at moderate-to-high levels. When any single vector is very low, it dominates the harmonic mean regardless of weights, which is the correct behavior for weakest-link security measurement.

Threshold scores exhibit a characteristic distribution that differs markedly from arithmetic-mean-based scores. While arithmetic mean scores for a population of agents typically follow a roughly normal distribution centered around the population mean, Threshold scores are left-skewed with a heavy tail near zero, reflecting the reality that many agents have at least one significant security vulnerability ^[9]. The median Threshold score in our testing population is approximately 15 points lower than the median arithmetic mean score, and the proportion of agents scoring below 30 is approximately three times higher under harmonic mean aggregation. This distributional shift is not an artifact; it is the accurate representation of a security landscape in which most agents have exploitable weaknesses.

Validation

We validate the Threshold framework through two complementary approaches: predictive validity testing and adversarial exploitation testing. In predictive validity testing, we compute Threshold scores for a population of 200 simulated agents and then subject each agent to a standardized adversarial campaign in which an automated attacker attempts to exploit each of the five vectors. The hypothesis is that Threshold scores should be predictive of the attacker's success rate, with lower-scoring agents being more frequently and more severely compromised. The results confirm this hypothesis with a Spearman rank correlation ^[10] of 0.87 between Threshold score and attack survival rate, compared to 0.61 for arithmetic mean scores and 0.74 for geometric mean scores. The harmonic mean's superior predictive validity is a direct consequence of its weakest-link property matching the attacker's strategy of targeting the weakest vector.

In adversarial exploitation testing, we present Threshold score profiles to human security analysts and ask them to predict which agents are most vulnerable and through which vector. The analysts consistently identify the same agents as most vulnerable that the Threshold framework ranks lowest, with a concordance rate of 91%. When the same analysts are presented with arithmetic mean score profiles, the concordance rate drops to 67%, because the arithmetic mean conceals the vulnerable vectors that the analysts would otherwise identify. This finding demonstrates that the harmonic mean preserves the information that security experts consider most relevant, while the arithmetic mean obscures it.

The framework is further validated through sensitivity analysis examining the Threshold score's response to targeted improvements. When an agent improves its weakest vector from 15 to 35, the Threshold score increases by an average of 18 points, a substantial and proportionate response. When an agent improves its strongest vector from 85 to 95, the Threshold score increases by an average of 2 points, a minimal response reflecting the fact that strengthening an already-strong defense does not materially change the weakest-link security posture ^[11]. This asymmetric sensitivity creates the correct incentive structure: organizations seeking to improve their agents' security scores must prioritize their weakest vectors, which is precisely the investment strategy that maximizes actual security.

We also test the framework's robustness to measurement noise in individual vector scores. Security testing inherently involves stochastic elements, and vector scores may vary by 5-10 points across repeated assessments of the same agent. The harmonic mean's sensitivity to low values means that measurement noise in a low-scoring vector has a larger effect on the composite than noise in a high-scoring vector, which could be seen as a disadvantage. However, this asymmetric sensitivity is actually appropriate: uncertainty about whether a vulnerability scores 10 or 20 represents meaningful security uncertainty, while uncertainty about whether a strong defense scores 85 or 95 does not ^[12]. The framework's sensitivity is correctly calibrated to the practical significance of the measurement uncertainty.