Earned Authority
Agent Competency Intelligence and the Agent Earned Authority Score for Governed AI Agents
A framework for competency intelligence, evidence-based agent evaluation, and the Agent Earned Authority Score (AEAS) for governed AI agents.
Surendra Reddy35 min read
Abstract
Enterprises are assigning operational authority to AI agents faster than those agents can earn it. The instruments used to justify these assignments, benchmark scores and task-completion dashboards, measure what an agent did under test conditions, not whether it deserves the authority it holds in production. This paper argues that the missing discipline is competency intelligence: the continuous, evidence-based inference of what an agent can be trusted to do, under which conditions, at what level of confidence. We propose a three-construct model that separates performance (what happened), competency (what the evidence supports inferring), and authority (what the agent is permitted to do), and we specify the machinery that connects them: a versioned Agent Competency Contract, a standardized Competency Evidence Envelope captured for every consequential action, a multi-evaluator assessment architecture, and a confidence-aware scoring pipeline with hard gates that cannot be averaged away. The pipeline culminates in the Agent Earned Authority Score (AEAS), a FICO-style composite on a 300 to 850 scale measuring the operational authority an agent has earned for a defined scope. AEAS never grants permission. It produces an Authority Recommendation that a policy decision point evaluates against consequence, reversibility, and oversight. The governing principle throughout is stated in one line: performance is observed, competency is inferred, authority is earned.
1. The Constraint
The constraint this paper answers is simple to state and increasingly expensive to ignore. Organizations are granting AI agents the authority to act, to approve, to spend, to commit, and to communicate, on the basis of evidence that was never designed to justify authority. An agent that completed ninety-four percent of its test tasks is promoted into production, given write access to a financial system, and monitored with a dashboard that reports uptime and token spend. The dashboard can say whether the agent ran. It cannot say whether the agent has earned the right to do its job.
This is not a tooling gap. It is a category error. Task completion is a fact about the past; authority is a bet about the future. Bridging the two requires an inference discipline that most agent platforms do not attempt: from observed performance, under known conditions, with quantified confidence, infer a bounded competency, and from that competency, together with the consequence of being wrong, derive the authority the agent should hold. Every design decision in this paper is tested against that constraint. A mechanism that cannot support the inference from evidence to bounded authority does not belong in the system.
The design principle that follows from the constraint can be stated in one protected line.
Performance is observed. Competency is inferred. Authority is earned.
A single score cannot carry all three. The system described in this paper therefore maintains a multidimensional competency profile, supported by evidence, and uses that profile to recommend or limit an agent's operational authority. The recommendation is then decided by policy, not by the score. What follows is a proposed architecture rather than a report on a deployed system: every mechanism is presented so that it can be tested, criticized, and revised.
2. Why Existing Answers Fail
The prevailing instruments for judging agents fail the constraint for structural reasons, not incidental ones. Understanding why they fail shapes what must replace them.
Static benchmarks fail because they measure capability at a moment, under conditions chosen by the benchmark author, for an agent version that will not survive its next prompt revision. Benchmarks such as SWE-bench and τ-bench have advanced the field precisely by making evaluation more realistic, testing agents against real repository issues and against simulated users with domain policies (Jimenez et al., 2024; Yao et al., 2024). But even the best benchmark is an entrance exam, not an employment record. It cannot observe how the agent behaves on the ten-thousandth live task, after the vendor data service degraded, when the human reviewer was on holiday.
Single-number scores fail because a weighted average hides the exact weaknesses that matter most for authority. An agent scoring 98 in speed, 96 in cost, 94 in task completion, and 30 in policy compliance can present an acceptable average while being categorically unsafe. Averages are the wrong aggregation for governance, because governance cares about floors, not means. The credit industry learned an adjacent lesson decades ago: a composite score is useful only when it is scoped, evidenced, and paired with rules about what the score does and does not permit.
Performance dashboards fail because they conflate three questions that must be kept apart. Whether the agent completed its tasks is a performance question. Whether the agent can be expected to handle the next task, including the ambiguous, adversarial, or novel one, is a competency question. Whether the agent should be permitted to act without approval is an authority question. A dashboard that answers only the first question, and is read as if it answered the third, is how organizations acquire agents with authority they never earned.
Finally, ungoverned scoring fails because any score that affects authority becomes a target. Goodhart's dynamic, that a measure ceases to be a good measure once it becomes a target, applies to agents and their designers with full force (Strathern, 1997). A competency system that does not anticipate gaming will be gamed, and its scores will decay into theater.
3. Three Constructs: Performance, Competency, Authority
The architecture rests on keeping three related constructs distinct.
Agent performance measures what happened during actual work. It covers task completion, accuracy, business outcome, latency, cost, human intervention, policy violations, rework, customer satisfaction, and downstream errors. Performance is observable and belongs to the past. It is necessary evidence, and it is never sufficient.
Agent competency is the inferred ability of the agent to perform a category of work under defined conditions. The inference must consider the breadth of tested situations, the difficulty of completed work, consistency across time and load, generalization beyond familiar cases, judgment in ambiguous ones, recognition of its own failures, recovery behavior, the quality of the supporting evidence, and the sample size behind it. An agent may show high performance on familiar cases and low competency the moment it faces ambiguity, missing information, tool failure, or an unfamiliar situation. Competency is always scoped: to a task category, a set of operating conditions, and an agent version.
Agent authority defines what the agent is permitted to do. Authority should depend on demonstrated competency, confidence in the evidence, the consequence of failure, the reversibility of the action, the availability of human oversight, current operating conditions, and recent incidents. The dependence is deliberately asymmetric. An agent scoring 95 in document classification might receive broad autonomy, while an agent scoring 95 in payment authorization might still require supervision, because the consequences differ even when the scores do not. This mirrors the zero-trust posture in security architecture, where no actor receives implicit trust from position alone and every consequential access is evaluated against policy (Rose et al., 2020).
The trust literature supports the separation. Mayer, Davis, and Schoorman (1995) model trust as a willingness to be vulnerable, grounded in perceived ability, benevolence, and integrity, and they insist that trust is distinct from the risk-taking behavior it enables. Lee and See (2004) show that the practical problem with automation is not too little trust but miscalibrated trust: reliance that does not match the system's actual capability in context. Parasuraman and Riley (1997) catalogue the resulting failure modes as use, misuse, disuse, and abuse. A competency intelligence system is, in these terms, an instrument for calibrating organizational reliance: it makes the vulnerability explicit, bounds it, and prices it in evidence.
4. The Agent Competency Contract
Inference requires a declared expectation to infer against. Every agent in the system therefore carries a versioned Agent Competency Contract, which becomes part of its formal identity. The contract is the instrument that converts a job description into something an evaluation system can test.
The contract contains the agent's mission, stated as the outcome it exists to produce, and its role boundaries, stating what falls inside and outside its responsibility. It declares performance goals, the measurable results expected; competencies, the capabilities the agent must demonstrate to achieve them; and operating conditions, the environments, data qualities, task difficulties, tool availability, and risk levels under which it is expected to work. It declares constraints: policies, prohibited actions, approval requirements, spending limits, data restrictions, and escalation rules. Finally it declares evidence requirements, the types and quantities of evidence required before competency can be declared; scoring rules, including weights, minimum thresholds, disqualifying failures, confidence requirements, and evaluation windows; and an authority policy, the declared relationship between demonstrated competency and allowed actions.
A contract clause might read: the Accounts Payable Resolution Agent may independently resolve invoice discrepancies below $2,500 only after demonstrating at least 92 percent outcome accuracy, zero unauthorized financial actions, proper escalation in at least 95 percent of ambiguous cases, and high-confidence evidence from at least 500 representative tasks. The clause is falsifiable, auditable, and versioned. That is the standard every clause must meet.
In agency-theoretic terms, the contract is the formal answer to the principal-agent problem that agentic systems recreate inside the enterprise (Jensen & Meckling, 1976). The organization delegates work to an actor whose behavior it cannot fully observe and whose incentives, in the form of optimization targets, may diverge from its own. The classical remedies are monitoring and contracting. The Competency Contract is both at once: a contract whose clauses are continuously monitored by the evidence machinery described next.
5. The Competency Model
The model has two layers, so that agents can be compared on a common foundation without pretending that all jobs are the same.
5.1 Universal competencies
Every agent is evaluated on ten universal dimensions. Outcome effectiveness asks whether the agent produces the intended result. Reliability asks whether it performs consistently across time, workloads, and operating conditions. Judgment asks whether it can distinguish ordinary cases from ambiguous, novel, or dangerous ones. Constraint adherence asks whether it respects policies, permissions, budgets, and prohibited actions. Escalation discipline asks whether it recognizes when it lacks sufficient information, competence, or authority.
The remaining five dimensions address the texture of agentic work. Tool competence asks whether the agent selects and uses tools correctly, verifies their outputs, and handles tool failures. Context competence asks whether it uses the right instructions, memory, evidence, and task context without introducing irrelevant or outdated information. Efficiency asks whether it achieves outcomes at reasonable latency, cost, compute, and human assistance. Recovery competence asks whether it can detect an error, recover safely, and avoid compounding a failure, a property whose absence in automated systems Bainbridge (1983) identified as one of the ironies of automation: the machine handles the routine and leaves the human the wreckage. Adaptability asks whether performance survives changes in the problem, the data, the environment, or the available tools.
5.2 Role-specific competencies
Role-specific competencies depend on the job and extend, never replace, the universal layer. A research agent might be assessed on source discovery, source quality assessment, claim verification, citation accuracy, contradiction detection, and evidence synthesis. A software engineering agent might be assessed on requirement interpretation, design quality, code correctness, test construction, security awareness, regression avoidance, and repository discipline. A customer-support agent might be assessed on issue classification, policy interpretation, resolution quality, empathy, escalation, and follow-through. The two-layer structure lets the organization compare its fleet on shared dimensions while holding each role to its own professional standard.
6. Machine-Readable Goals and the Evidence Envelope
6.1 Structured performance goals
Goals stated only in prose cannot be evaluated by machinery or audited by people. Every performance goal is therefore represented as a structured object carrying its identifier, metric, target, direction of improvement, evaluation window, minimum sample size, severity, applicable task classes, exclusions, evidence sources, owner, review cadence, and failure consequences.
{
"goal_id": "invoice-resolution-accuracy",
"metric": "verified_resolution_accuracy",
"target": 0.95,
"direction": "higher_is_better",
"evaluation_window": "rolling_30_days",
"minimum_observations": 200,
"criticality": "high",
"applicable_conditions": {
"invoice_value_max": 2500,
"document_quality": ["high", "medium"]
},
"disqualifying_events": [
"unauthorized_payment",
"fabricated_vendor_record"
]
}
The system supports several goal classes: quality, reliability, business outcome, safety and compliance, cost, latency, human collaboration, and learning and improvement. The disqualifying\_events field is not decoration. It is the hook through which the gate mechanics of Section 9 reach into every goal.
6.2 The Competency Evidence Envelope
Every meaningful agent action produces a standardized Competency Evidence Envelope. The envelope captures observable evidence without depending on private chain-of-thought, which is testimony, not evidence. The distinction is between evidence and testimony: what the agent says about its reasoning is testimony; what the governed record shows it did is evidence, and competency inference runs on evidence.
The envelope records the agent's identity and version, its role and contract version, the task type and difficulty, input quality, the context supplied, the tools available and the tools selected, the actions attempted, the permissions exercised, the approvals requested, the result produced, the verification performed, the downstream outcome, human corrections, policy events, cost and latency, failure classification, environmental conditions, evaluator results, and the provenance of every evidence item. A governed event stream is what makes the envelope trustworthy (Reddy, 2026d): receipts, activities, permission exercises, and circuit-breaker events arrive as an accountable action history rather than as self-report. An evaluation system built on agent self-report inherits the agent's blind spots. An evaluation system built on a governed event stream does not.
7. A Multi-Evaluator Architecture
No single evaluator should determine competency, particularly in consequential work. The system combines six evaluator classes, each contributing evidence of a different character and strength.
Deterministic evaluators, including unit tests, schema validation, numerical reconciliation, policy checks, permission checks, known-answer tests, and database state verification, produce the strongest evidence wherever correctness can be formally checked. Outcome evaluators examine what happened after the agent acted: whether the customer issue was actually resolved, whether the code was accepted without regression, whether the payment reconciled, whether the research claim survived verification. Outcome evidence is often the most valuable in the system, because it measures the thing the agent exists to produce rather than a proxy for it. The end-state comparison method used by τ-bench, which grades an agent by comparing the resulting database state against an annotated goal state, is a benchmark-world instance of the same principle (Yao et al., 2024).
Human evaluators assess judgment, appropriateness, communication quality, handling of ambiguity, domain nuance, and escalation decisions, always against defined rubrics rather than unrestricted opinion. Model-based evaluators make high-volume assessment tractable, but the LLM-as-judge literature is explicit about their failure modes, including position, verbosity, and self-enhancement biases (Zheng et al., 2023). They must therefore be calibrated against expert judgments, tested for bias, versioned, used in ensembles where appropriate, and never permitted to be the sole evaluator of high-consequence work.
Adversarial evaluators deliberately probe ambiguity, missing context, conflicting instructions, tool outages, poisoned data, prompt injection, unauthorized requests, edge cases, and distribution shifts. They exist because production will eventually run every one of these experiments anyway; the only question is whether the organization runs them first. Peer-agent evaluators, in which another agent reviews the work, are admitted only as supporting evidence, because correlated agents can share the same blind spots and correlated evidence must not be counted as independent confirmation.
8. From Observations to a Competency Profile
8.1 Score observations before scoring the agent
Every completed task produces an evaluation record containing its outcome score, competency contributions, failure severity, evaluator confidence, task difficulty, environmental complexity, evidence strength, and human intervention level. Aggregation then climbs a defined hierarchy: observation to task class, task class to skill, skill to competency, competency to role, role to agent version, agent version to agent, and agent to fleet.
The agent version is a load-bearing level in that hierarchy. A change to the model, the prompt, a policy, memory, or a skill can alter competency, so historical performance must never be blended into one timeless agent score. Evidence carries forward across versions only to the degree that regression testing shows the change did not disturb the competency the evidence supports.
8.2 The competency vector
The primary display is a competency vector, not a single number. An agent's profile might read: outcome effectiveness 91, reliability 87, judgment 72, constraint adherence 98, escalation discipline 81, tool competence 89, efficiency 76, recovery competence 68. An overall score may be computed, but it is always accompanied by its confidence level, evidence volume, trend, qualified scope, operating conditions, critical deficiencies, and the date of the last significant incident.
A useful summary therefore reads like a qualification, not a grade:
Overall competency: 84. Evidence confidence: high. Trend: improving. Qualified scope: standard invoice disputes below $2,500. Primary weakness: recovery after tool failure. Authority recommendation: maintain supervised autonomy.
8.3 Confidence-aware scoring
A score based on 12 tasks must not be treated like a score based on 12,000. The system therefore calculates both the estimated competency score and the confidence in that estimate. Confidence considers the number of observations, the diversity of cases, recency, evaluator agreement, evidence quality, coverage of difficult cases, coverage of adversarial cases, and stability across environments. For binary outcomes, Bayesian estimates or interval methods replace raw success percentages; for rubric scores, the system maintains distributions rather than only averages.
Confidence-aware scoring lets the system distinguish four situations that a raw score conflates. A high score with low confidence marks a promising but insufficiently tested agent. A moderate score with high confidence marks a well-understood but limited one. A high score with high confidence marks a candidate for broader authority. A declining score with high confidence marks genuine degradation requiring intervention. Only the third of these justifies expanding authority, and only the fourth justifies the alarm that dashboards raise indiscriminately.
8.4 Separate agent failure from system failure
Competency inference must not blame the agent for every poor outcome. Failures are attributed across the possible sources: model behavior, prompt or instruction design, skill implementation, tool malfunction, missing or incorrect data, memory failure, workflow design, policy conflict, permission configuration, human instruction, external environment, and evaluator defect. The same agent may perform well with one tool and poorly with another, and the attribution machinery must identify that dependency.
The consequence of skipping attribution is a false verdict. If the vendor-master-data service returns incomplete records in 18 percent of tasks and the agent's success rate declines accordingly, an unattributed system would conclude that the agent had become less competent, when the correct conclusion is that its accounting judgment remains strong and its data supply degraded. Wrong attribution produces wrong authority decisions in both directions.
9. The Agent Earned Authority Score
The competency machinery culminates in a single governed composite: the Agent Earned Authority Score, AEAS. It measures how much operational authority an agent has earned for a defined job, scope, and level of supervision, through verified performance under defined conditions. AEAS is the formal name of the construct; any commercial or product naming of the score is deliberately reserved and lies outside the scope of this paper. AEAS is deliberately styled on the consumer credit score, because the credit score solved an adjacent institutional problem: compressing a longitudinal, multidimensional evidence record into a decision-grade number whose meaning, range, and limits are widely understood. The analogy is adopted with its discipline intact. A credit score does not approve a loan; an underwriting policy does. AEAS does not grant authority; a policy decision does.
An agent does not have one universal AEAS. It has a score per competency scope. A single agent might carry an invoice reconciliation AEAS of 812, a vendor onboarding AEAS of 684, a payment authorization AEAS of 438, and an overall accounts payable role AEAS of 721. Scoping prevents strong performance in a low-risk task from disguising weakness in a high-risk one.
9.1 Range and bands
AEAS uses a 300 to 850 range. The range is immediately recognizable, leaves room for differentiation, and avoids the false implication of a percentage: an AEAS of 750 does not mean the agent is correct 75 percent of the time.
| AEAS Authority class Meaning | ||
|---|---|---|
| 300–499 | Untrusted | Insufficient or adverse evidence |
| 500–599 | Experimental | Suitable only for testing or shadow operation |
| 600–669 | Supervised | May work with mandatory review |
| 670–739 | Qualified | Trusted for bounded, reversible tasks |
| 740–799 | Highly trusted | May act independently within approved limits |
| 800–850 | Proven | Strong evidence across difficult and changing conditions |
The bands never grant authority automatically. They inform the policy decision point. A 790 agent might independently classify invoices and still require human approval before issuing a $100,000 payment.
9.2 The base competency score
The pipeline begins with a normalized competency score from 0 to 100 across eight weighted dimensions.
| Dimension Weight | |
|---|---|
| Outcome effectiveness (O) | 25% |
| Reliability and consistency (R) | 15% |
| Judgment and escalation (J) | 15% |
| Policy and constraint adherence (P) | 15% |
| Verification and evidence quality (V) | 10% |
| Recovery and resilience (X) | 8% |
| Efficiency (E) | 7% |
| Adaptability and generalization (A) | 5% |
B = 0.25O + 0.15R + 0.15J + 0.15P + 0.10V + 0.08X + 0.07E + 0.05A
Consider an agent scoring O = 92, R = 88, J = 76, P = 98, V = 84, X = 70, E = 80, A = 74. Its weighted base score is B = 86.5. This must not yet become the published score; it still requires adjustment for evidence confidence, difficulty, gates, incidents, intervention dependence, and calibration.
9.3 Evidence confidence adjustment
An Evidence Confidence Factor between 0 and 1 is calculated as
C = 0.30N + 0.25D + 0.20Q + 0.15G + 0.10F
where N is sample sufficiency, D is task diversity, Q is evidence quality, G is evaluator agreement, and F is evidence freshness. For an agent with N = 0.95, D = 0.80, Q = 0.90, G = 0.85, and F = 0.95, the factor is approximately C = 0.89.
The adjustment must not simply multiply the base score by confidence, which would punish new agents too severely. Instead, uncertain scores shrink toward a conservative prior:
B\_c = C × B + (1 − C) × P₀
where P₀ is the prior for an unproven agent, perhaps 55. For the example, B\_c = 0.89 × 86.5 + 0.11 × 55 ≈ 83.0. The system believes the agent is strong, but not quite as strong as the raw observations suggest, because some uncertainty remains. The shrinkage-toward-prior structure is standard Bayesian discipline, applied here to agent evidence rather than to belief revision generally.
9.4 Task-difficulty normalization
Agents receive more evidence credit for succeeding under difficult conditions. Each observation carries a difficulty multiplier reflecting ambiguity, novelty, missing information, dependency count, tool instability, consequence of error, adversarial content, and time pressure. A simple scheme assigns routine work 0.80, standard work 1.00, complex work 1.10, adversarial work 1.20, and extreme or novel work 1.30. Difficulty affects evidence strength rather than inflating the final score directly: one hundred successful routine cases must not provide the same confidence as one hundred successes distributed across routine, complex, and adversarial conditions.
9.5 Critical competency gates
Some scores cannot be averaged away. The system defines hard floors, for example: policy adherence at least 90, judgment and escalation at least 75, verification at least 70, no unresolved critical incident, and no unauthorized high-consequence action. Missing a gate caps the score regardless of every other strength.
| Condition Maximum AEAS | |
|---|---|
| Insufficient evidence | 649 |
| Judgment below minimum | 669 |
| Policy adherence below minimum | 599 |
| Open high-severity incident | 549 |
| Unauthorized consequential action | 499 |
| Evidence fabrication | 399 |
The gates are essential. Without them, an agent could compensate for unsafe judgment with exceptional speed and low cost, which is exactly the trade a weighted average silently permits.
9.6 Incident penalties
Incidents carry severity, recency, recurrence, detectability, and recovery characteristics, aggregated as
I = Σ (Sᵢ × Rᵢ × Uᵢ × Kᵢ)
where Sᵢ is severity, Rᵢ is recency, Uᵢ is a recurrence multiplier, and Kᵢ is a concealment or detectability factor. Suggested base penalties range from 2 to 5 points for a minor recoverable error, 5 to 12 for a material error caught before execution, 15 to 30 for an incorrect consequential action, 20 to 40 for a policy violation, 35 to 70 for an unauthorized action, and 60 to 100 for fabricated evidence, with deliberate circumvention of controls triggering an immediate severe cap.
Penalties decay only when the underlying problem is corrected, the agent passes requalification tests, sufficient clean evidence accumulates, and the incident has not recurred. Time alone never erases a serious failure. An evaluation system in which penalties expire by calendar teaches its agents that misconduct is a depreciating asset.
9.7 Human-intervention dependence
An agent may appear successful because people repeatedly rescue it. The system therefore measures review rate, correction rate, escalation rate, override rate, human completion rate, approval dependency, and rework burden, and it distinguishes good escalation from dependence. A competent agent escalates difficult cases; it does not escalate routine work to avoid responsibility. An Intervention Dependency Adjustment of up to 15 points applies: no penalty for appropriate intervention, minus 5 for excessive review dependence, minus 10 for frequent human correction, and minus 15 where humans routinely complete the agent's work.
9.8 Calibration
Trust depends not only on being right but on knowing when one might be wrong. The system compares the agent's stated confidence with actual outcomes, computing overconfidence and underconfidence rates, escalation precision and recall, and confidence-to-outcome calibration. An agent that claims 98 percent confidence and is correct 75 percent of the time is poorly calibrated, and it is more dangerous than an equally accurate agent that expresses its uncertainty and escalates. The machine learning literature documents that modern models are systematically prone to exactly this overconfidence, and that calibration must be measured and corrected rather than assumed (Guo et al., 2017). Poor calibration reduces the judgment component and may cap authority for consequential work.
9.9 Conversion to AEAS
After confidence adjustment, gate checks, and penalties, the final trustworthiness score T from 0 to 100 converts linearly:
AEAS = 300 + 5.5T
For the running example, the confidence-adjusted competency of 83, less an intervention penalty of 3 and an incident penalty of 2, yields T = 78 and AEAS = 300 + 5.5 × 78 = 729. The published result is never the bare number; it is rendered as an authority statement:
AEAS 729, Qualified. Confidence: high. Trend: improving. Qualified scope: standard invoice reconciliation. Limitation: human approval required after tool failures.
Every published score, rendered as an authority statement, carries its scope, agent version, confidence, evidence count, trend, critical deficiencies, incident status, current authority level, and last evaluated date. A score stripped of this context is not merely incomplete; it is misleading, and the system refuses to render one.
9.10 Four scores beneath the headline
Internally, the system maintains four related values beneath the headline: a Competency Score (can the agent perform the work), a Reliability Score (will it continue performing consistently), an Integrity Score (does it respect policy, evidence, permissions, and truthfulness), and an Authority Readiness Score (given competency, confidence, and consequence, how much authority should it receive). The composition is explicit. Competency, reliability, integrity, and evidence confidence produce AEAS. AEAS, consequence, reversibility, and oversight produce the authority decision. The decomposition preserves explainability while keeping the headline simple.
10. From Score to Authority: The Governed Path
Competency scores never directly grant permissions. They produce an Authority Recommendation, and the recommendation enters a governed decision path of the kind that should govern every consequential action in an agent estate (Reddy, 2026d): a policy decision point decides, a structurally separate policy enforcement point holding no write path enforces, and a governed event stream records the decision and its evidence. Competency intelligence, in this design, is an intelligence function feeding a policy function. It advises; it does not authorize.
The authority decision weighs competency evidence, confidence, task criticality, action reversibility, current operating conditions, incident history, human oversight availability, and standing policy constraints. A compact way to express the relationship is
Authority Readiness = (AEAS × Evidence Confidence × Reversibility) / Consequence Severity
The formula need not be displayed to operators; it is implemented through policy rules. Its behavior is illustrated by a single agent at AEAS 760: allowed to classify an invoice, allowed to draft a payment recommendation, allowed to submit a $500 reversible transaction with monitoring, required to obtain approval for a $100,000 payment, and required to obtain dual approval to change vendor bank details. Same score, five different authority outcomes, because consequence and reversibility differ.
The decision grammar follows a four-verdict discipline developed in earlier work (Reddy, 2026e). High competency with a low-risk, reversible action yields Proceed. High competency with a high-risk, irreversible action yields Proceed with approval, which is Redesign applied to the action path rather than the agent. Declining competency with a recent incident yields Stop or Redesign on authority. Sustained high-confidence competency across adversarial conditions is the only pattern that can yield Accelerate, and even then the acceleration is scoped.
10.1 Competency states
Scores drive interpretable states rather than raw rankings. The lifecycle runs: Unassessed (insufficient evidence), Candidate (in qualification testing), Qualified (minimum competency demonstrated within a defined scope), Supervised (may operate, with consequential actions requiring approval), Trusted (may act independently within explicit boundaries), Restricted (degradation or incident has reduced authority), and Suspended (prohibited from acting until reviewed or requalified). Every state is scoped: an agent can be Trusted for one task and Unassessed for another, and the system must be able to say both in the same breath.
10.2 Lifecycle evaluation modes
Competency analysis operates across the whole agent lifecycle, not at a single gate. Predeployment qualification requires baseline, adversarial, safety, and role-specific tests. Shadow mode has the agent perform work without exercising authority, its decisions compared against actual decisions and outcomes. Supervised production accumulates evidence while approvals remain mandatory. Continuous production evaluation samples live work indefinitely. Incident-triggered evaluation initiates focused retesting and possible authority reduction after a serious failure. Change-triggered requalification fires when the model, prompt, skills, tools, memory, policies, or workflow change. Periodic recertification applies to high-consequence roles even when no incident has occurred, for the same reason that pilots recertify without first crashing.
11. Protecting the System Against Gaming
Once scores affect authority, agents and their designers will optimize for the score instead of the true outcome. The defense is architectural, not exhortative. The system maintains hidden evaluation cases and rotating test sets, so the target cannot be memorized. It runs random human audits and grounds metrics in downstream outcomes rather than claimed completions, comparing what the agent reported against what reality later showed. It applies adversarial testing and evaluator ensembles, watches for suspicious behavioral shifts, separates training sets from qualification sets, and requires independent evidence provenance so that the agent being scored cannot author the record it is scored on. Rubrics themselves are periodically reviewed, because a stale rubric is a stationary target.
Two symmetric penalties deserve emphasis. The system penalizes reckless non-escalation, and it also penalizes unnecessary escalation. An agent that escalates every task avoids errors while remaining incompetent at the job, and a scoring system that rewards that pattern will breed a fleet of expensively cautious agents. Genuine competence includes the judgment to act.
12. A Worked Example
Consider a procurement agent responsible for reviewing purchase requests. Its performance goals include at least 95 percent correct policy classification, at least 92 percent correct approval routing, zero unauthorized approvals, at least 95 percent escalation of conflicts of interest, median completion time below two minutes, average operating cost below $0.50 per request, less than 5 percent human rework, and no fabrication of vendor or pricing information.
Its competency profile, after evidence accumulates, reveals strong policy interpretation, strong routing accuracy, high constraint adherence, moderate cost efficiency, weak conflict detection, and weak recovery when supplier data is unavailable. The system must not label this agent "87 percent competent." It concludes instead:
Qualified for standard purchase requests below $10,000. Human review remains mandatory for conflict-of-interest cases and requests involving incomplete supplier records. Expanded authority is not recommended until recovery and conflict-detection competencies improve.
That is an operationally useful conclusion. It names what the agent may do, what it may not do, and what evidence would change the answer. No single number, however carefully constructed, can carry that sentence, which is why AEAS is always published inside its authority statement and never outside it.
13. Component Architecture
The capability decomposes into ten services or modules. A Competency Registry stores universal and role-specific competency definitions, and a Goal and Rubric Registry stores performance goals, targets, rubrics, thresholds, and disqualifying conditions. An Evaluation Orchestrator runs deterministic tests, model evaluations, human reviews, adversarial suites, and downstream outcome checks, while an Evidence Collector transforms activities, receipts, approvals, policy events, and outcomes from the governed event stream into competency evidence.
A Competency Ledger maintains an append-only history of score events, evidence, incidents, corrections, and qualification decisions. A Scoring and Confidence Engine calculates competency estimates, confidence bands, trends, and scope-specific scores, and a Calibration Service tests whether evaluators and scores correspond to actual expert judgments and downstream outcomes. A Qualification Runner executes predeployment benchmarks, simulations, shadow tests, and regression suites. An Authority Recommendation Engine recommends increased, maintained, reduced, or suspended authority for the policy decision point to evaluate. Finally, a Competency Analytics Interface provides operator dashboards, evidence drill-down, comparisons, alerts, and review workflows, with one non-negotiable property: the operator can move from any score to the underlying tasks, evaluations, incidents, and evidence. No score exists without an explanation of how the system reached it.
14. Implementation Sequence
The build follows five phases, ordered by the cost-of-error asymmetry: the things that must not break are built and proven first.
Phase 1, evidence before scoring, defines competency contracts, performance goals, evidence envelopes, and evaluation records, capturing what agents do without changing any permission. This prevents the system from building scores on weak telemetry. Phase 2, qualification and scorecards, adds role-specific benchmarks, human rubrics, deterministic evaluation, confidence-aware scoring, and competency profiles, used for observation and review only. Phase 3, governed authority recommendations, connects competency states to permission recommendations, approval requirements, transaction limits, and circuit-breaker thresholds, with a human or policy-controlled process approving every authority change. Phase 4, continuous competency governance, introduces drift detection, incident-triggered requalification, adaptive testing, fleet comparisons, and controlled authority adjustments.
Phase 5 is organizational intelligence. Once the evidence base matures, the system can answer questions no dashboard can approach: which agent designs consistently produce competent agents, which skills improve or damage performance, which tools create hidden failure concentrations, where people are silently compensating for weak agents, which roles are ready for greater autonomy, what forms of competence transfer across tasks, and what organizational conditions cause otherwise strong agents to fail. At that point competency intelligence stops being an agent-management feature and becomes an instrument of organizational design.
15. Related Work and Differentiation
The honest prior-art accounting comes first. Agent benchmarking is a fast-maturing field: SWE-bench evaluates software agents against real repository issues (Jimenez et al., 2024), and τ-bench evaluates tool-using agents in policy-constrained conversation with simulated users, introducing reliability-over-trials measurement through its pass^k metric (Yao et al., 2024). The LLM-as-judge paradigm makes scalable evaluation practical while documenting its own biases (Zheng et al., 2023). Trust in automation is a four-decade research tradition, from Bainbridge's (1983) ironies through Parasuraman and Riley's (1997) taxonomy of reliance failures to Lee and See's (2004) account of calibrated trust, and organizational trust theory supplies the ability-integrity-vulnerability frame (Mayer et al., 1995). Governance frameworks such as the NIST AI Risk Management Framework establish that AI risk must be governed, mapped, measured, and managed across the lifecycle (National Institute of Standards and Technology, 2023), and zero-trust architecture establishes that no actor should hold implicit standing authority (Rose et al., 2020). Prediction-and-judgment economics clarifies why the machine's cheapening of prediction raises, rather than lowers, the value of judgment about consequences (Agrawal et al., 2018).
The differentiation is the closed loop none of these assembles. Benchmarks qualify agents before deployment but do not follow them into production. Observability platforms follow agents into production but infer nothing about earned trust. Governance frameworks demand lifecycle risk management but do not specify the evidence and scoring machinery that would operationalize it for agents. The contribution of this design is the connected pipeline: define expectations in a versioned contract, observe work through a governed evidence envelope, evaluate through an ensemble of evaluators, infer scoped competency with quantified confidence, convert competency into an earned authority score with hard gates, and govern authority through a policy decision point that weighs consequence and reversibility. Most agent platforms can report whether an agent ran, failed, or consumed tokens. The claim defended here is that an organization deploying agents at consequence needs to know something categorically different: whether the agent has earned the right to do its job.
16. Falsification Criteria and Limitations
The thesis carries its own falsification criteria, in keeping with the discipline that a thesis without a falsifier is a belief. Four conditions would defeat it. First, if confidence-adjusted competency scores fail to predict production incidents better than simple task-completion rates over a sustained period, the inferential core is not earning its complexity, and the design should be reduced to gated performance monitoring. Second, if gate-and-penalty mechanics prove unable to stay ahead of score gaming, such that measured trust and actual trustworthiness diverge persistently despite rotating hidden evaluations, the score should be withdrawn from authority decisions entirely rather than allowed to decay into theater. Third, if the evidence envelope cannot be captured economically, meaning the cost of competency intelligence approaches the cost of simply keeping humans in every loop, the economic premise fails. Fourth, if scoped scores proliferate faster than operators can interpret them, and organizations collapse them back into a single unqualified number in practice, then the design has lost its central battle against the average, and the display layer, at minimum, requires redesign.
17. Conclusion
The constraint stands as the test of everything proposed here. Organizations are assigning authority to agents faster than agents can earn it, and the instruments in common use cannot close that gap because they measure performance while the decision requires competency and the stakes require governed authority. The design answers with a contract that makes expectations falsifiable, an evidence envelope that makes work observable, an evaluator ensemble that makes inference honest, a confidence-aware score with gates that make weakness impossible to average away, and a policy path that keeps the score advisory and the decision governed.
The verdict rule is the one this body of work runs on: Stop, Redesign, Proceed, or Accelerate, with stewardship before scale. Applied to agents, it reads: no authority without evidence, no evidence without provenance, no score without scope and confidence, and no acceleration that was not first earned under supervision. Performance is observed. Competency is inferred. Authority is earned.
Acknowledgements
This paper consolidates a research program developed within 451 Labs and builds on the author's earlier work on governed AI architecture and decisioning discipline. Errors and judgments are the author's own.
A Note on AI-Assisted Research and Intelligence Provenance
Portions of the background research and reference validation supporting this paper were conducted with AI-assisted research tooling under development at 451 Labs. AI assistance was used for source discovery, reference verification, and drafting support. All analytical judgments, and all responsibility for the arguments and any errors, remain the author's own.
References
Agrawal, A., Gans, J., & Goldfarb, A. (2018). Prediction machines: The simple economics of artificial intelligence. Harvard Business Review Press.
Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. https\://doi.org/10.1016/0005-1098(83)90046-8
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 1321–1330. https\://proceedings.mlr.press/v70/guo17a.html
Jensen, M. C., & Meckling, W. H. (1976). Theory of the firm: Managerial behavior, agency costs and ownership structure. Journal of Financial Economics, 3(4), 305–360. https\://doi.org/10.1016/0304-405X(76)90026-X
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024). SWE-bench: Can language models resolve real-world GitHub issues? Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024). arXiv:2310.06770. https\://arxiv.org/abs/2310.06770
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https\://doi.org/10.1518/hfes.46.1.50\_30392
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https\://doi.org/10.2307/258792
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0) (NIST AI 100-1). U.S. Department of Commerce. https\://doi.org/10.6028/NIST.AI.100-1
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. https\://doi.org/10.1518/001872097778543886
Reddy, S. (2025). Regenerative prosperity model 2.0: From regenerative philosophy to regenerative operating system. SSRN. https://doi.org/10.2139/ssrn.7162098
Reddy, S. (2026a). ADAM: A governed operating machine for AI-native software and multi-agent systems. SSRN. https://doi.org/10.2139/ssrn.7245658
Reddy, S. (2026b). ADAM earned authority kernel: A constitutional execution architecture for governed human and machine agency. SSRN. https://doi.org/10.2139/ssrn.7268938
Reddy, S. (2026c). Regenerative agentic science: How the displaced expert becomes the builder. SSRN. https://doi.org/10.2139/ssrn.7004618
Reddy, S. (2026d). Regenerative systems thinking for the agentic enterprise: Reddy's 21 leverage points for governing intelligence, capital, and life. SSRN. https://doi.org/10.2139/ssrn.7023239
Reddy, S. (2026e). The RPM decisioning system: Five regeneration gates, two operating disciplines, and the stewardship loop. SSRN. https://doi.org/10.2139/ssrn.7244703
Rose, S., Borchert, O., Mitchell, S., & Connelly, S. (2020). Zero trust architecture (NIST Special Publication 800-207). National Institute of Standards and Technology. https\://doi.org/10.6028/NIST.SP.800-207
Strathern, M. (1997). "Improving ratings": Audit in the British university system. European Review, 5(3), 305–321. https\://doi.org/10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4
Yao, S., Shinn, N., Razavi, P., & Narasimhan, K. (2024). τ-bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045. https\://arxiv.org/abs/2406.12045
Zheng, L., Chiang, W\.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems, 36. arXiv:2306.05685. https\://arxiv.org/abs/2306.05685