Agentic AI: promise and peril for offensive security
Theresa Lanowitz, an analyst at Omdia, told Dark Reading that the rise of “agentic” AI – systems that can act autonomously toward a goal – is reshaping how penetration testers and red teams think about their work.
On one hand, she sees the technology as a force‑multiplier. An AI‑driven agent could crawl a network, enumerate services and even craft exploit payloads without a human typing each command. That speed could let organisations discover weaknesses before threat actors do.
On the other hand, Lanowitz warned that handing such power to an algorithm opens a new attack surface. If the model is trained on flawed data, it may produce unsafe commands, trigger denial‑of‑service conditions, or even be co‑opted by adversaries to automate large‑scale attacks.
Key technical concerns
- Loss of human judgement. Autonomous agents may miss nuanced context that seasoned testers catch.
- Model poisoning. Training data could be tainted, causing the AI to suggest malicious techniques.
- Auditability. Actions taken by an AI are often opaque, making post‑engagement reporting difficult.
Who should pay attention
Any security team that already uses scripting or automation in their red‑team toolbox should evaluate whether an agentic layer adds value or risk. Small‑to‑medium enterprises that lack deep offensive expertise are especially vulnerable to over‑reliance on a “black‑box” tool.
Practical steps to tame the beast
- Start with a pilot. Run the AI in a sandboxed environment and compare its findings against a manual test.
- Maintain human‑in‑the‑loop oversight. Require a senior tester to review every exploit suggestion before execution.
- Log and version‑control all AI‑generated scripts. This creates a trail for later audit.
- Regularly retrain or update the model with clean, vetted data to mitigate poisoning.
- Define clear usage policies – what scopes are allowed, what systems are off‑limits, and how to handle false positives.