ADLC metrics: How to measure agentic development security

TL;DR

• ADLC metrics cover two surfaces: agent output such as insecure code, vulnerable dependencies, and leaked secrets, plus agent behavior like shell commands and package installs.
• Traditional AppSec metrics such as PRs scanned, total findings, and scan pass rate scale with agent volume, so normalize confirmed findings per merged change or per 1,000 changed lines.
• Post-commit scanners miss what agents do before review, so track the share of risky actions stopped before execution, and separate deterministic policy blocks from model refusals.
• Measure agent-generated code by confirmed, newly introduced findings per merged agent change, compared with a human baseline, and report the share backed by reachability or exploitability evidence.
• Block risky or hallucinated packages before install, since install scripts run before post-install scans, and report reachable vulnerable dependencies alongside the total.
• Pair each security metric with a flow metric like blocked PR rate or fix acceptance rate, baseline six to eight metrics for a month, and set targets by risk tier.

ADLC metrics are the measurements that show whether the code AI coding agents produce, and the actions they take while producing it, are secure. That gives you two surfaces to measure. The first is the output: insecure code, vulnerable dependencies, and leaked secrets. The second is the behavior: shell commands, file reads, tool calls, and package installs that happen while the agent works.

The agentic development lifecycle describes how software gets built when AI coding agents such as Claude Code, Codex, Cursor, and GitHub Copilot write a large share of the changes. Developers set intent, agents plan and act, and security has to follow both the code and the agent's actions from prompt to production. For the full model, read our guide to the secure agentic development lifecycle.

IBM and others use ADLC to mean the agent development lifecycle, which covers building AI agents as products. This guide covers the agentic development lifecycle: building software with AI coding agents.

The guide organizes ADLC metrics into five categories:

• AI supply chain visibility: which agents, models, MCP servers, and skills are in use.
• Agent action security: whether risky actions are controlled before they run.
• Agent-generated code security: new, confirmed risk in agent-authored changes.
• Open source and package risk: what agents install and whether it's reachable.
• Developer flow: whether security keeps pace with agent throughput.

Why don't traditional AppSec metrics work for agentic development?

Traditional AppSec metrics were built for human-speed change and only see code after it's committed, while agents generate far more changes and act before anyone reviews them.

Volume distortion

Most AppSec dashboards report numbers that scale with activity, not risk. When agents open dozens or hundreds of pull requests a day, those numbers stop telling you anything useful.

• Percentage of PRs scanned hits 100% easily. It tells you a scanner ran, not whether it found what mattered.
• Total findings climbs with every agent-generated change. A rising count could mean more risk, more code, or a noisier tool. You can't tell which.
• Scan pass rate flattens. If agents produce many small, low-risk changes, the pass rate looks great while a handful of risky changes slip through.

The fix is normalization. Divide security outcomes by a unit of change instead of reporting raw totals:

• Per merged change: confirmed findings ÷ merged PRs in the period.
• Per 1,000 changed lines: confirmed findings ÷ (lines added or modified ÷ 1,000).

Per merged change works well for policy and process metrics. Per changed lines works better when agents produce changes of very different sizes. Pick one per metric and keep it stable so trends stay comparable.

Google Cloud's DORA 2024 Accelerate State of DevOps report supports this. It found that AI adoption was associated with lower software delivery stability, even as developers reported productivity gains. More change doesn't automatically mean safer change, and your metrics need to reflect that.

The pre-review blind spot

Agents act before code review. They run shell commands, read files, call tools, and install packages between one prompt and the next. A post-commit scanner sees the code that results. It doesn't see the npm install that ran an install script, the .env file the agent read, or the destructive command it executed in a working directory.

ADLC vs SDLC metrics

For more on how application security changes between the two models, see from SDLC to ADLC.

How do you measure AI supply chain visibility?

You measure AI supply chain visibility by comparing the agents, models, MCP servers, and skills actually observed in use with those your security team has reviewed and approved. Developers adopt these tools on their own, often with no central inventory, so the approved list usually describes intent rather than reality.

Discovered vs approved ratio

Formula: approved components in use ÷ discovered components in use

Calculate it per component type: agents, models, MCP servers, and skills. A ratio of 0.6 for MCP servers means 40% of the MCP servers running across your developer fleet haven't been reviewed. That gap is your shadow AI supply chain, and policy can't reach what you don't know exists.

Data source: agent hooks and endpoint telemetry that record which agent, model, and MCP server handled each session.

Owner: AppSec, with platform engineering supplying telemetry.

Time from first observed use to review

Formula: median days between the first time a component appears in telemetry and the date its review is recorded

The ratio tells you how big the gap is today. Time to review tells you whether your process can keep up. If developers adopt a new MCP server on Monday and it gets reviewed six weeks later, your inventory is always behind.

Permission review coverage

Formula: MCP servers and skills with a recorded permission review ÷ total MCP servers and skills in use

MCP servers and skills grant agents new capabilities: file system access, network calls, ticket systems, cloud APIs. A recorded review should capture what each one can read, write, and execute.

These three metrics give you the baseline for everything else. For background on the governance layer behind them, see our guide to AI coding agent governance.

How do you measure agent action security?

You measure agent action security by tracking how many risky actions are stopped by policy before they execute, across every agent in use. A log entry that shows an agent ran rm -rf on a directory is useful for forensics, but it isn't a control.

Pre-execution vs after-the-fact split

Formula: risky actions stopped before execution ÷ (risky actions stopped before execution + risky actions detected after execution)

Define "risky action" by category so the metric stays consistent:

• Destructive shell commands (deletes, force pushes, permission changes)
• Sensitive file reads (.env, credential files, SSH keys)
• Credential access or exfiltration attempts
• Package installs from unapproved sources

A high ratio means your controls act inside the agent loop. A low ratio means you mostly find out from logs and commits, after the action already ran.

Policy block vs model refusal

Record why an action didn't happen. Did a deterministic policy block it, or did the model decide not to do it? Without deterministic enforcement, safety depends on the model choosing to behave, and that isn't a control. Model behavior changes with versions, prompts, and context. Track the two separately:

• Policy-enforced blocks: repeatable and auditable
• Model refusals: useful signal, but not something you can promise an auditor

Supporting action metrics

• Policy coverage across agents: agents with active policy enforcement ÷ agents in use. If Claude Code has guardrails and Cursor doesn't, your coverage is partial no matter how good the Claude Code policy is.‍
• Blocks by action category: count of policy blocks, broken out by the categories above, normalized per 100 agent sessions. Watch for spikes after onboarding a new team or agent.‍
• Prompt-injection test success rate: successful injections ÷ injection test cases run. Use a fixed test suite so results are comparable over time.‍
• Regression rate after changes: test cases that newly fail after a model, prompt, or policy change ÷ total test cases. Agents change underneath you when vendors ship new models.

The OWASP Top 10 for LLM Applications from the OWASP GenAI Security Project gives you shared definitions for prompt injection, excessive agency, and supply chain risk. Mapping each action category to an OWASP entry makes the metrics easier to explain to auditors and leadership.

How do you measure the security of agent-generated code?

You measure the security of agent-generated code by counting newly introduced, confirmed vulnerabilities per merged agent-authored change and comparing that rate against a human-authored baseline on similar repositories. The steps below keep that number trustworthy.

Separate introduced from pre-existing findings

A scan of a changed file surfaces old problems along with new ones. If you attribute all of them to the agent, the metric measures your legacy debt, not agent behavior. Diff-based analysis tells you what the change actually introduced.

Formula: confirmed findings introduced by agent-authored changes ÷ merged agent-authored changes

Compare against a human baseline

Run the same calculation for human-authored changes on repositories with similar language, framework, and risk tier. The comparison answers the question leadership actually asks: are agents making us more or less secure? A raw rate without a baseline can't answer it.

Attribution takes some setup. Use commit trailers, PR labels, or agent telemetry to tag agent-authored changes. Changes that mix human and agent edits need a consistent rule, such as "agent-authored if the agent produced the majority of changed lines."

For a benchmark view of how coding agents perform on security, see the Agent Security League results.

Evidence-adjusted finding share

Raw finding counts reward noise. A tool that flags everything looks thorough until engineering asks which findings are real. Make the headline number the share of findings backed by evidence.

Formula: new findings on agent-authored changes with reachability or exploitability proof ÷ total new findings on agent-authored changes

Evidence means something a developer can check: a call path from an entry point to the vulnerable function, or a data flow from untrusted input to a sensitive sink. Track false positive rate next to it:

Formula: findings closed as false positive ÷ findings triaged

These two numbers together show whether your findings will hold up when an engineer pushes back. Business logic flaws such as insecure direct object references and broken authorization often pass pattern-based rules, so watch whether your tooling catches that class at all. A low finding count can mean clean code or blind tooling.

Secrets metric

Formula: confirmed live secrets introduced per period, split by agent-authored vs human-authored changes

Validate every detected secret as live or revoked. A revoked test key and a live production credential shouldn't count the same. Track the share of secrets caught at pre-commit versus found later in history, since a secret caught before it lands in Git never needs rotation.

‍

How do you measure open source and package risk introduced by agents?

You measure open source and package risk by tracking how many risky or nonexistent packages are blocked before install and how many vulnerable dependencies are actually reachable from your code. Agents install packages between an LLM suggestion and the next prompt, often without a human looking at the name.

Packages blocked before install

Formula: risky or nonexistent package installs blocked before execution ÷ total risky or nonexistent package install attempts

Timing matters here for the same reason it matters with agent actions. Install scripts run during installation, before any post-install scan sees the package. Malicious package campaigns now target that agent behavior. A post-install scan that flags a malicious package is reporting on something that already ran.

Package hallucination makes this metric concrete. Researchers studying LLM code generation found that models regularly suggest packages that don't exist, and often repeat the same invented names across runs. See We Have a Package for You!, a USENIX Security 2025 paper by Spracklen et al., for the methodology. Repeated names give attackers a predictable target: register the package, then wait for an agent to install it.

Reachable vs total vulnerable dependencies

Formula: vulnerable dependencies with a reachable call path from application code ÷ total vulnerable dependencies

Full-stack reachability across first-party code, open-source dependencies, and container images separates real exposure from noise. A vulnerable function in a transitive dependency that your code never calls is a lower priority than one on a live request path. Report both numbers. The total shows your inventory. The reachable count shows your exposure.

Unapproved dependency introduction rate

Formula: new dependencies outside your approved list or policy ÷ merged changes that add dependencies

This metric catches drift. If agents keep pulling in packages that fail your policy for license, maintenance, or provenance, the rate will show it before the dependency tree gets out of hand.

How do you tell if security is slowing agentic development down?

You tell if security is slowing agentic development down by pairing each security metric with a flow metric, such as blocked PR rate, fix acceptance rate, and median time to fix by severity. Without flow metrics, a program can make risk numbers look great by stalling delivery, and platform teams will rightly push back.

Flow counter-metrics

• Blocked PR rate: PRs blocked by a security check ÷ total PRs opened. A rising rate means findings show up for the first time at review instead of during development.
• Fix acceptance rate: suggested remediations merged without major rework ÷ suggested remediations. This measures whether developers and agents trust the fixes.
• Median time from finding to fix by severity tier: report critical, high, medium, and low separately. A blended median hides slow critical fixes behind fast low-severity ones.

Pair them deliberately

Why false positives hit flow first

False positives are a flow problem before they're a security problem. Each one costs a developer time to investigate. After enough of them, developers stop reading findings closely, fix acceptance drops, and real issues sit next to noise in the backlog. That's why false positive rate belongs on the scorecard next to the risk metrics.

As one reference point, Endor Labs reports 83% fewer blocked PRs and up to 95% false positive reduction for AI SAST in its own published results for AURI by Endor Labs. These are Endor Labs' reported figures, not industry benchmarks. Your own baseline is the number that matters.

How do you build an ADLC security scorecard?

You build an ADLC security scorecard by picking six to eight metrics across the five categories, baselining them for a month, then setting targets based on each repository's risk tier. Starting small keeps the data clean and the review meeting short.

Starter scorecard

The numbers below are hypothetical, for one team over one quarter. They're illustrative only, not industry benchmarks.

A healthy quarter for this hypothetical team looks like this: visibility and pre-execution control rise, confirmed findings per change fall, and blocked PR rate falls with them. If the security metrics improve while blocked PR rate climbs, the program is trading delivery for risk, and engineering will notice.

Scorecard caveats

• Set baselines before targets. A target set before you have a month of data is a guess. Any target in this guide is illustrative.
• Measure tests alongside incidents. A low incident count proves little if agents are rarely tested or their actions aren't observed. Report prompt-injection test volume and telemetry coverage next to any incident or failure metric.
• Report trends, not snapshots. A single month's number invites gaming. Three months of direction is harder to fake.
• Watch for gaming. If teams can raise the evidence-adjusted share by suppressing findings, track suppression rate too.
• Tier by risk. A payments service and an internal docs site shouldn't share a target.

For regulated teams, NIST SP 800-218A, the Secure Software Development Framework community profile for generative AI, gives you recognized practices to map these metrics against. Tagging each scorecard row with the related practice makes compliance reporting easier.

Where does ADLC metric data come from?

ADLC metric data comes from the places agents act: agent hooks and tool-call logs, PR review, dependency and container analysis, and secrets scanning. If a metric depends on data you only collect after merge, it can't measure anything that happened before review.

The challenge is collecting all of this without a different integration for every agent. AURI by Endor Labs integrates through Hooks, Skills, MCP, CLI, and API, so the same telemetry and policy apply across Claude Code, Codex, Cursor, GitHub Copilot, and other agents. The Coding Agent Governance overview in the Endor Labs documentation shows developer adoption, model use, and policy enforcement trends across a fleet, which covers several rows of the starter scorecard directly. 

If you're evaluating tooling for this, see Coding Agent Governance from Endor Labs.

Whichever tools you use, check two things: can the data source see what agents do before review, and can it show evidence a developer would accept?

FAQ

Is ADLC the agentic development lifecycle or the agent development lifecycle?

Both terms use the same acronym. The agentic development lifecycle covers building software with AI coding agents, which is what this guide measures. The agent development lifecycle, a term IBM and others use, covers building AI agents as products. The metrics overlap in places, such as prompt injection testing, but the goals differ.

Should you track the percentage of code written by AI agents?

Yes, as context. AI-generated code share helps you plan review capacity and decide where to scale controls. It isn't a security outcome on its own. A team where agents write 80% of the code can be more secure than one where they write 20%, depending on controls. Use it to normalize and segment other metrics, not as a headline.

How often should ADLC security metrics be reviewed?

Review operational metrics weekly: pre-execution stops, packages blocked, blocked PR rate, and confirmed findings per change. Review structural metrics monthly: policy coverage, evidence-adjusted share, and time to fix by severity. Report quarterly trends to leadership. Revisit definitions whenever you onboard a new agent or model, since those changes can shift every baseline at once.