AI Red Teaming Raises the Stakes for Security Operations

| 2 Min Read
AI is compressing red-team timelines and expanding their reach, while making operator judgment and verification more valuable.

AI is changing the economics of offensive security faster than many organizations have adapted. Red teams can now produce findings, scan infrastructure and prepare engagements at a pace that once required substantially more people and time. For security leaders, the practical challenge is twofold: understand how quickly AI can amplify an operation, and ensure that human expertise still directs and checks the work.

The distinction between testing AI systems and using AI during security operations helps clarify the shift. AI red teaming probes large language models, agent pipelines and issues such as prompt injection. Red teaming with AI applies AI tools to conventional offensive work. The latter is already changing reconnaissance, testing and reporting, with direct implications for both offensive and defensive teams.

More work, compressed into less time

DayBlink Consulting’s figures, collected across a decade of engagements and described by Robert Pimentel as representative of his red-team experience, point to a sharp increase in output. A three- or four-person team traditionally produced 60 to 70 findings over a year. In one after-hours exercise, an AI-augmented team generated more than 90. On a single application engagement, the comparison was seven to ten findings against more than 100 true positives.

The reported gains also extend to scale. Traditional large-scale scanning depends on careful sequencing and established pipelines, and can time out. With AI-assisted capabilities, teams can scan millions of IP addresses in two to three hours, at a stated cost of roughly $20,000 to $30,000. External application reconnaissance, which traditionally yielded 150 to 200 valid findings per year, reached more than 100 findings a month in the AI-augmented comparison.

MetricTraditional red teamingAI-augmented red teaming
Annual team output60 to 70 findings (3-4 person team)90+ findings generated in a single after-hours exercise
Single app engagement7 to 10 findings100+ true positive findings
Large-scale scanningTakes appropriate sequencing and pipelines, or times out entirelyEnables you to build capabilities that can scan millions of IPs in 2-3 hours (for roughly $20k-$30k)
Reconnaissance of external applications150-200 valid findings/year100+ findings a month

Data above aggregated by DayBlink Consulting over past 10 years of engagements and validated by Robert Pimentel as representative of his Red team experience.

These figures describe reported experience, not a guarantee that every team or target will see the same results. Still, the direction is clear: AI can let a small group cover more ground, run more checks and surface more potential issues within a tighter window. That alters the comparison between a well-equipped offensive team and a defense organized around manual review.

AI is entering every phase of an operation

The change goes beyond generating phishing messages or writing snippets of code. Organizations and threat actors are adopting a structured approach to integrating AI across operational stages, from preparation through post-engagement analysis.

  • Preparation and reconnaissance: Work that once stretched across weeks can be accelerated to minutes. AI agents can help gather open-source intelligence, assemble organizational charts, provision infrastructure and configure command-and-control servers. Experienced operators still need to make complex infrastructure choices, but automation can clear away much of the repetitive setup. That matters because preparation can become a drain on a team’s time even before an engagement begins.
  • Active testing: During an operation, AI can serve as an on-demand assistant for generating custom audit scripts, adjusting an approach and interpreting unfamiliar technology stacks. A workflow that asks one model to review another model’s output can add a check for correctness and risk before an action is taken. But the work still depends on operator judgment. AI can help with enumeration and testing; it does not yet replace the human ability to make sense of a changing target.
  • Reporting and deconfliction: After an engagement, teams can feed nine to ten months of raw command-and-control logs into a frontier model and produce a first draft of a deconfliction report in under 20 minutes. The model can map commands to the MITRE ATT&CK framework, cross-reference threat intelligence and recast technical details for senior executives. Those results still need review: operators must verify findings, check framework mappings and confirm the report’s accuracy before delivery.

Reporting is a particularly practical example of where AI can return time to specialists. Ticketing, follow-up and translating technical activity into readable findings are necessary, but they can consume hours that teams could otherwise spend analyzing systems. A fast draft helps; an unchecked draft can mislead. Verification remains part of the work.

Operator judgment separates useful output from noise

AI can automate basic scanning, enumeration and scaffolding. That makes foundational skills more consequential, not less. Operators still need to understand network architecture, recognize where trust boundaries break down and judge whether a result points to a meaningful weakness. People who combine that knowledge with AI assistance can move faster while focusing their attention on complex exploitation.

The gap is between practitioners who direct and validate these tools and those who paste their outputs into an operation without checking the mechanics. The source describes the first group as “unicorns”: people with critical thinking and adversarial judgment who use automation to carry out informed ideas. It calls the second “slop cannons”: operators who produce noisy, unverified work that can waste time or trigger alarms.

That distinction has consequences for hiring. Certifications can still signal useful knowledge, but they don’t show on their own whether someone can reason through a system or assess AI-generated results. Teams also need to look for practical curiosity, sound understanding of network architecture and experience building tools with frontier models. The valuable skill is not simply operating an AI assistant; it is knowing what to ask, what to trust and what to reject.

What security leaders need to build

Over the next 18 months, the recommendations in the source point to four priorities for security teams. First, organizations should set budgets for access to frontier models so practitioners can experiment and develop useful capabilities. The cost is part of the operational equation, especially for teams expected to match adversaries using AI.

Second, enterprises need continuous, programmatic adversarial testing. As organizations deploy thousands of agentic integrations, manual reviews alone cannot keep pace with the number of systems and possible interactions. Regular automated testing can help teams identify problems as those integrations change.

Third, planning should account for hybrid threats. As automated defenses harden digital perimeters, operations may combine automated technical attacks with physical or social engineering tactics. Red teams will need to examine how these methods interact, rather than treating digital systems as isolated targets.

Finally, teams should hire for curiosity and judgment. Practitioners who actively experiment with AI and know how to guide it can help organizations adapt their methods. As discussed in this analysis of cybersecurity skills in the AI era, the quality of a team’s capabilities matters alongside its size.

AI doesn’t remove the need for skilled operators; it changes how much those operators can accomplish and what they must be prepared to verify. Security leaders who treat AI as an operational capability will be better positioned to test more frequently and interpret results with care. The near-term edge will belong to teams that pair automation’s speed with human judgment at every stage.

Source: Joseph Rodriguez · www.csoonline.com

Comments

Please sign in to comment.