How AI Is Actually Changing Cybersecurity: Defense, Offense, and the Arms Race

The cybersecurity industry has a long tradition of turning every new technology into a marketing category before it delivers anything real. Cloud security was going to eliminate breaches. Zero Trust was going to end lateral movement. AI is the current paradigm shift being sold to every CISO on a vendor briefing calendar.

Unlike most previous paradigm shifts, this one is not entirely wrong. AI is genuinely changing both sides of the security equation in ways that matter operationally — not just in research papers. The problem is that the signal is buried under vendor noise so thick it takes forensic effort to find what's actually happening.

This post cuts through it: what AI is meaningfully contributing to defense, what attackers have actually adopted (not what researchers demonstrated as a theoretical possibility three years ago), and how security teams should think about the arms race they're now in.

AI on Defense: What Actually Works

Anomaly Detection and Behavioral Analytics

This is where AI has delivered the most genuine value in enterprise security, and it predates the current LLM era by years. Machine learning models trained on baseline network traffic, user behavior, and endpoint activity can flag statistical deviations that rule-based systems miss — not because they recognize a specific known-bad pattern, but because they recognize unexpected deviation from an established normal.

User and Entity Behavior Analytics (UEBA) platforms have been applying this for over a decade. The operational value: if an HR manager who has never logged in after 6 PM suddenly authenticates at 2 AM from a new geolocation, accesses systems they've never touched, and begins downloading files at an unusual rate, none of those individual behaviors may trigger a rule-based alert. Their combination represents a statistically significant anomaly that a tuned ML model surfaces.

Real deployment impact: Secureworks' MDR data from 2024 showed that 37% of incidents they detected were first surfaced by behavioral anomaly detection rather than signature-based alerts. That's not a marketing claim — it represents threat classes that were genuinely invisible to pre-ML security tooling.

ML-based anomaly detection excels at catching:

• Lateral movement (MITRE ATT&CK T1021): A service account suddenly authenticating to 40 new hosts over two hours is anomalous even if each individual authentication uses valid credentials. The pattern is the signal.
• Data exfiltration (T1048): Unusual data volume leaving the network, especially to cloud storage providers or geographic regions the organization doesn't operate in, flagged against historical per-user baselines.
• Credential abuse (T1078): Authentication from impossible locations — two logins from countries 5,000 miles apart within 90 minutes — or access patterns that deviate sharply from a given identity's established behavior.
• Insider threat indicators (T1074): Behavioral changes correlating with known insider threat patterns: large downloads before a resignation, access to systems outside job function, attempts to access HR systems from non-HR personnel.

The limitation that vendors rarely discuss: false positive management is a continuous operational burden. Anomaly detection models without careful tuning generate alert volumes that overwhelm SOC teams — sometimes dramatically worse than the rule-based systems they replaced. A team running a UEBA platform that fires 800 alerts a day has not improved their security posture; they've buried their analysts. The art is calibrating sensitivity-to-precision tradeoff, which requires ongoing human expertise that the platform vendors are not going to provide for you.

AI-Assisted Threat Hunting

Traditional threat hunting requires a skilled analyst to form a hypothesis about attacker behavior, write detection logic in KQL, SPL, or YARA, and run it against log data. It's time-consuming and constrained by the analyst's prior knowledge. You can only hunt for TTPs you can already articulate.

AI augments this in three ways that are operational today, not theoretical:

Natural language to query translation: Analysts describe what they're hunting for in plain language; AI translates it to the appropriate query language. "Find processes that spawned command shells and then made outbound network connections within 30 seconds" becomes a working KQL query without manual syntax construction. Microsoft Copilot for Security, Splunk AI Assistant, and Google SecOps AI all have this capability deployed in production at enterprise scale.

# Illustrative example: AI-assisted threat hunting query generation
# The analyst describes behavior, AI generates the detection logic
 
analyst_input = """
Find Windows hosts where PowerShell was launched by a Microsoft Office
process (Word, Excel, Outlook) in the last 7 days, and where that
PowerShell process then made outbound network connections. This is
the T1566.001 initial access -> T1059.001 execution chain.
"""
 
# AI-generated KQL query for Microsoft Sentinel:
kql_query = """
let office_processes = dynamic(['WINWORD.EXE', 'EXCEL.EXE', 'OUTLOOK.EXE', 'POWERPNT.EXE']);
DeviceProcessEvents
| where Timestamp > ago(7d)
| where FileName =~ "powershell.exe"
| where InitiatingProcessFileName has_any (office_processes)
| join kind=inner (
    DeviceNetworkEvents
    | where Timestamp > ago(7d)
    | where RemoteIPType == "Public"
) on DeviceId, $left.ProcessId == $right.InitiatingProcessId
| project Timestamp, DeviceName, AccountName,
          InitiatingProcessFileName, FileName, RemoteIP, RemotePort,
          InitiatingProcessCommandLine
| order by Timestamp desc
"""

Pattern recognition across large datasets: AI surfaces correlations in massive log datasets that human analysts would miss. A specific user account involved in subtle anomalies across 15 separate log sources over 30 days — each individually too minor to flag — collectively indicating compromise through a pattern a human reviewing individual alerts would never construct.

Hypothesis generation from threat intelligence: AI tools trained on threat intelligence can suggest hunting hypotheses based on recent threat actor TTPs, translating high-level intelligence into specific queries against the organization's telemetry. When CrowdStrike publishes a report that SCATTERED SPIDER is using social engineering to call help desks and reset MFA, your AI-assisted platform can translate "hunt for unexpected help desk authentication resets followed by new device enrollments" into executable hunt queries across your Okta and Entra ID logs within minutes.

SOAR Automation and AI Triage

Security Orchestration, Automation, and Response platforms automate repetitive SOC workflows. AI adds a layer by making triage decisions that previously required human judgment — and doing it consistently, at scale, at 3 AM.

When a phishing email is reported, an AI-augmented triage workflow at a mature enterprise runs like this:

1. Extract all URLs, attachments, and sender metadata automatically
2. Query VirusTotal, URLScan.io, Recorded Future, and sandbox environments in parallel
3. Search for other recipients of the same or similar email across the entire organization
4. Correlate against SIEM for any users who clicked links or opened attachments in the last hour
5. Assess aggregate risk based on multiple signals with weighted scoring
6. For high-confidence malicious determination (>0.85 score): quarantine email organization-wide, block sending domain at the email gateway, flag affected users for credential reset, create a P1 incident ticket
7. For ambiguous cases: prepare a structured summary with supporting evidence for human review, with recommended actions pre-populated

# Example SOAR playbook with AI triage
name: phishing_triage_v2
trigger: reported_phishing_email
sla: 5_minutes
 
steps:
  - name: extract_iocs
    action: parse_email_headers_and_body
    parallel: true
    output: [urls, attachments, sender_ip, reply_to, message_id]
 
  - name: threat_intel_lookup
    action: query_multi_source_ti
    parallel: true
    inputs:
      urls: ${extract_iocs.urls}
      sender_ip: ${extract_iocs.sender_ip}
    sources: [virustotal, urlscan, recorded_future, otx]
    output: reputation_scores, known_campaigns
 
  - name: sandbox_analysis
    action: submit_to_any_run
    input: ${extract_iocs.attachments}
    timeout: 120s
    output: sandbox_verdict, behavioral_indicators, c2_indicators
 
  - name: org_scope_check
    action: query_email_gateway_logs
    input:
      message_id: ${extract_iocs.message_id}
      sender: ${extract_iocs.sender}
    lookback: 24h
    output: affected_users, delivery_status_per_user
 
  - name: click_check
    action: query_proxy_logs
    input:
      urls: ${extract_iocs.urls}
      users: ${org_scope_check.affected_users}
    lookback: 2h
    output: users_who_clicked
 
  - name: ai_triage_decision
    action: llm_risk_assessment
    model: gpt-4o
    input:
      ti_results: ${threat_intel_lookup}
      sandbox: ${sandbox_analysis}
      scope: ${org_scope_check}
      clicks: ${click_check}
    output: risk_score, confidence, recommended_action, reasoning
 
  - name: response
    condition: ${ai_triage_decision.risk_score} >= 0.85
    if_high_risk:
      - quarantine_email: scope=organization_wide
      - block_domain: ${extract_iocs.sender_domain}
      - create_incident: severity=P1, assign=soc_lead
      - notify_affected_users: template=phishing_warning
      - flag_for_credential_reset: users=${click_check.users_who_clicked}
    if_medium_risk:
      - create_ticket: severity=P3
      - attach_ai_summary: ${ai_triage_decision.reasoning}
      - recommend_actions: ${ai_triage_decision.recommended_action}
      - assign_to_analyst: queue=tier1_review
    if_low_risk:
      - mark_benign
      - update_allowlist_if_appropriate
      - close_ticket: resolution=false_positive

The time reduction from this kind of automation is substantial and measurable. Phishing triage that took a Level 1 analyst 20-30 minutes — and might be one of 300 similar alerts that day — runs in under 90 seconds with comparable or better accuracy. The compound effect across a year is thousands of analyst-hours redirected from mechanical triage to actual investigation.

Malware Classification and Analysis

AI has fundamentally changed the economics of malware analysis. Before ML-based classification, triage of new malware samples required manual reverse engineering — a highly skilled, time-intensive process that couldn't scale to the tens of thousands of new samples appearing daily.

ML-based static analysis can now:

• Classify malware family membership from binary features with >95% accuracy on known families
• Identify packing and obfuscation techniques before dynamic analysis
• Extract behavioral indicators (file system patterns, registry modifications, network signatures) from static code analysis
• Detect code reuse across malware families using graph-based neural networks — identifying that a new ransomware variant shares 40% of its code with a known RaaS platform

More importantly for operational security, AI tools assist human reverse engineers on the samples that matter most:

Decompilation assistance: IDA Pro's ML-powered function naming, combined with tools like BinaryAI and CodeBERT-based models, can rename obfuscated functions based on their behavior patterns. What used to take an analyst days to build a mental map of a complex sample can now start with an AI-generated skeleton that names 60-70% of functions meaningfully. The analyst spends their time on the 30% that's novel, not reconstructing what the logging functions and string manipulation routines do.

Code similarity detection at scale: When a new ransomware strain hits endpoints at 3 AM, ML-based code similarity against a corpus of known malware families can surface "this shares 67% code similarity with BlackCat/ALPHV" within minutes — enabling rapid capability assessment before a single human analyst has reviewed a line of assembly.

# Example: AI-assisted malware triage pipeline
# Using EMBER features + LightGBM for initial classification
 
import lightgbm as lgb
import lief
import numpy as np
from pathlib import Path
 
def extract_pe_features(filepath: str) -> np.ndarray:
    """
    Extract EMBER-compatible features from a PE binary.
    Returns feature vector for ML classification.
    """
    binary = lief.parse(filepath)
    if binary is None:
        return None
 
    features = []
 
    # Header features
    features.append(binary.header.numberof_sections)
    features.append(binary.optional_header.sizeof_code)
    features.append(binary.optional_header.sizeof_initialized_data)
    features.append(1 if binary.has_tls else 0)
    features.append(1 if binary.has_debug else 0)
    features.append(1 if binary.has_exports else 0)
 
    # Import hash features (section names, import counts, etc.)
    import_count = sum(1 for lib in binary.imports for imp in lib.entries)
    features.append(import_count)
 
    # Section entropy (high entropy = packed/encrypted)
    section_entropies = []
    for section in binary.sections:
        if len(section.content) > 0:
            _, counts = np.unique(list(section.content), return_counts=True)
            probs = counts / len(section.content)
            entropy = -np.sum(probs * np.log2(probs + 1e-10))
            section_entropies.append(entropy)
    features.append(max(section_entropies) if section_entropies else 0)
    features.append(np.mean(section_entropies) if section_entropies else 0)
 
    return np.array(features)
 
# Load pre-trained model (train on EMBER dataset)
model = lgb.Booster(model_file='malware_classifier.lgb')
 
sample_path = "suspicious_sample.exe"
features = extract_pe_features(sample_path)
if features is not None:
    prediction = model.predict([features])[0]
    print(f"Malware probability: {prediction:.3f}")
    # > 0.8: almost certainly malware, escalate for full RE
    # 0.5-0.8: suspicious, sandbox and monitor
    # < 0.5: likely benign

Vulnerability Research and Patch Prioritization

CVSS scores are famously bad at predicting which vulnerabilities will be exploited. A CVSS 9.8 vulnerability in obscure middleware that has no known exploit code and is not exposed on the internet is less urgent than a CVSS 7.2 vulnerability in your public-facing VPN concentrator with active exploitation in the wild. CVSS doesn't know this. ML models trained on historical exploitation data, real-time threat intelligence, and environmental context do.

The Exploit Prediction Scoring System (EPSS), maintained by the Forum of Incident Response and Security Teams (FIRST), uses ML to predict the probability that a CVE will be exploited in the wild within 30 days. It consistently outperforms CVSS as a prioritization signal. For a SOC or vulnerability management team dealing with thousands of CVEs per quarter, the difference between "prioritize everything above 7.0" and "prioritize the 3% of CVEs that EPSS predicts will be exploited" is the difference between a functional and a dysfunctional patching program.

For vulnerability discovery, the picture is more nascent but moving fast. Google Project Zero and Microsoft's Security Response Center have both published research on LLM-assisted vulnerability discovery, finding previously unknown bugs in real code. The capability is not uniformly applicable — LLMs are better at certain vulnerability classes (integer overflow, use-after-free, injection vulnerabilities) than others (logic flaws, complex access control failures). But the trajectory is clear and the operational deployment is coming.

AI on Offense: What Attackers Are Actually Using

The marketing material for AI security products often describes a future threat: AI-generated exploits, autonomous attackers, self-adapting malware that rewrites itself in real time. Some of that is real. Some of it is science fiction being sold as threat intelligence to justify product purchases. Here's an honest accounting.

Automated Reconnaissance and OSINT at Scale

This is the most widely deployed offensive use of AI — and it's not dramatic. Financially motivated threat actors and nation-state groups use AI to automate OSINT collection and correlation at a scale that wasn't feasible manually.

What an AI-powered OSINT workflow against a corporate target can produce in under 10 minutes:

• Organizational structure inferred from LinkedIn (titles, reporting relationships, tenure)
• Likely tech stack from job postings, GitHub repos, and BuiltWith data
• Employees with administrative access based on public forum posts, conference talks, GitHub activity
• Domain infrastructure, certificate transparency records, DNS history
• GitHub repositories with hardcoded credentials, internal endpoints, or API keys committed and later removed (git history still shows them)
• Historical breach data correlating personal email addresses with credential pairs
• Cloud storage misconfigurations from public S3 bucket enumeration

Tools like SpiderFoot with LLM integration, or custom GPT-4-based OSINT pipelines, synthesize all this into an actionable attack profile. The same research that might take a skilled human analyst two days to compile manually — with proper source correlation — is now a 15-minute automated task.

This represents not a qualitative change in what's possible but a dramatic reduction in the time and skill required to do professional-quality reconnaissance. The consequence is that targeted attacks that previously required dedicated research per victim are now economically viable against a much broader target pool.

LLM-Assisted Social Engineering at Scale

The primary operational AI capability deployed by financially motivated adversaries since 2023 is LLM-generated social engineering. The economics are stark: a single criminal enterprise can now run personalized spear-phishing campaigns against hundreds of targets simultaneously, with per-email quality that previously required a skilled human writer.

IBM X-Force's 2024 threat intelligence report documented threat actors using LLM-generated content in BEC campaigns against financial services organizations. The tells that security researchers use to identify phishing — generic preambles, incorrect tense, tonal mismatches — are absent. The emails reference specific projects, use the target's preferred name form, and match the communication style of the person being impersonated.

A documented 2024 campaign (attributed with medium confidence to FIN7 by Mandiant) against healthcare sector CFOs combined:

1. LinkedIn data scraping to identify financial decision-makers
2. LLM-generated personalized emails referencing the target's specific institution, recent press coverage, and named internal contacts harvested from the organization's website
3. A landing page with a realistic DocuSign clone for a fake vendor agreement
4. Real-time credential capture and session hijacking via an Evilginx relay

The campaign achieved a 3.1% initial compromise rate against targets who had passed standard phishing awareness training — because the attacks looked nothing like what the training scenarios described.

AI-Assisted Vulnerability Discovery

This is where the threat model gets operationally concerning. Security researchers have demonstrated that LLMs — particularly when given structured prompting with relevant context — can identify certain vulnerability classes in code more efficiently than manual review. The same capability is accessible to attackers.

A significant 2024 research paper from the University of Illinois Urbana-Champaign demonstrated that GPT-4 with function-calling could exploit 87% of "one-day" vulnerabilities (disclosed CVEs with no public exploit code) when given CVE descriptions, significantly outperforming GPT-3.5 (7%) and open-source models. The research underscored that the gap between "a CVE was disclosed" and "an attacker has a working exploit" is narrowing rapidly.

# Simplified example: AI-assisted vulnerability analysis workflow
# (Illustrating what offensive security researchers and attackers can do)
 
def analyze_patch_diff(before_code: str, after_code: str) -> dict:
    """
    Analyze a patch diff to understand the underlying vulnerability.
    This is what attackers do within hours of a CVE disclosure.
    """
    prompt = f"""
    A security patch was applied to this code. Analyze the diff to determine:
    1. What vulnerability was fixed (type, CWE classification)
    2. What conditions are required for exploitation
    3. What a proof-of-concept payload would look like
    4. What the attack impact would be if exploited
 
    BEFORE (vulnerable):
    {before_code}
 
    AFTER (patched):
    {after_code}
 
    Provide a technical analysis focused on exploitability.
    """
    # LLM response identifies: SQL injection from user_id string concatenation
    # Payload: user_id = "1 UNION SELECT username, password FROM admin_users --"
    # Conditions: unauthenticated access to the endpoint
    # Impact: full database read access
    return llm_analyze(prompt)
 
# Vulnerable function example
vulnerable_code = """
def get_user_data(user_id):
    query = "SELECT * FROM users WHERE id = " + user_id
    cursor.execute(query)
    return cursor.fetchall()
"""
 
# Patched version
patched_code = """
def get_user_data(user_id):
    query = "SELECT * FROM users WHERE id = %s"
    cursor.execute(query, (user_id,))
    return cursor.fetchall()
"""

The practical implication for defenders: the window between patch release and working exploit code being available to average-skill attackers has compressed from weeks to hours for many vulnerability classes. If you are not patching critical internet-facing systems within 24-48 hours of a disclosed CVE with EPSS > 0.5, you are operating with the assumption that attackers need more time than they do.

Adaptive Malware: Reality vs. Marketing

Here the hype significantly outruns the reality. Marketing materials regularly describe "AI-powered malware" that adapts in real time, rewrites its own code to evade detection, and independently discovers new exploitation paths. What's actually deployed is considerably more modest — but still concerning.

What exists operationally: Malware with ML-based environment detection. Samples that:

• Identify sandboxing via CPUID timing attacks, user interaction detection, and VM artifact detection
• Select from a library of shellcode variants based on the detected EDR product (observed in recent NOBELIUM and Lazarus Group samples analyzed by Mandiant in 2024)
• Adjust network C2 timing based on detected monitoring thresholds
• Choose between multiple obfuscation layers based on detected static analysis tools

This is a meaningful capability upgrade. It is not AI autonomously writing new malware. The ML is doing environment detection and payload selection from a pre-built menu, not reasoning about security architectures.

What's mostly theoretical: Truly autonomous malware that discovers novel vulnerabilities, develops working exploits, adapts its exploitation strategy in real time without internet connectivity or human direction, and maintains operational security while doing all of this. The computational requirements for running a capable LLM locally without internet access, plus the operational security constraints of beacon traffic to a remote inference endpoint, make this impractical for most threat actors today. Nation-state actors with significant compute resources are the exception.

AI-Accelerated Vulnerability Weaponization

What is genuinely happening is faster weaponization of disclosed vulnerabilities. The time-to-exploitation window for critical CVEs has been shrinking year over year. The 2024 Rapid7 Vulnerability Intelligence Report found that 23% of widely exploited CVEs had exploit code available on the day of disclosure (up from 14% in 2020), and the median time to exploitation for critical vulnerabilities had dropped to under five days.

AI analysis of patch diffs — examining the code changes in a security update to reverse-engineer the underlying vulnerability — is contributing to this compression. A process that previously required a skilled researcher 2-3 days now takes hours when AI can analyze the diff, identify the vulnerable code path, and suggest exploit primitives.

The Ivanti Connect Secure vulnerabilities (CVE-2023-46805 and CVE-2024-21887) were observed being exploited by multiple APT groups within days of disclosure in January 2024. The speed of weaponization — across multiple separate threat actors — suggests AI-assisted exploit development being applied in parallel by different groups.

The Arms Race: An Honest Assessment

Both sides of the security industry have access to the same foundational models. GPT-4, Claude, Gemini — these are not exclusive to defenders. The offense-defense balance shifts not based on who has AI but on how effectively each side deploys it within their operational constraints.

Where Offense Currently Has the Advantage

Speed of adoption: Criminal enterprises have no change management process, no security review board, no compliance requirement for new tooling. The same criminal group that was running template-based phishing in early 2023 was running fully LLM-personalized campaigns by Q4 2023. Enterprise security teams adopting new AI tooling face procurement cycles, vendor evaluation, legal review, and change management that can stretch across a year.

Asymmetric unit economics: AI has reduced the unit cost of attacks dramatically. Before LLMs, a high-quality targeted phishing email required 20-30 minutes of skilled labor per target. LLM-generated phishing at professional quality costs seconds per target. The cost of attacks has dropped by two orders of magnitude. Defense costs have not fallen proportionally.

Asymmetric target surface: Defenders must protect the entire attack surface. A single successful phishing email is sufficient for initial access. Defenders must make every potential entry point robust; attackers need only one to work. This asymmetry predates AI but is amplified when attackers can test a thousand attack variations cheaply.

Where Defense Has the Advantage

Data volume: Large enterprise security teams have access to enormous internal telemetry that attackers cannot see. ML models trained on organization-specific baseline behavior detect attacker activity that generic threat intelligence never surfaces. A Fortune 500 with a mature security data lake has training data for behavioral models that no attacker can replicate.

Automation at scale: Defenders apply AI-powered analysis to 100% of traffic, 100% of emails, 100% of process events continuously. The scale of automated defense exceeds what any attacker can manually evade consistently. A threat actor who successfully evades ML-based detection on first contact has to evade it continuously throughout their dwell time.

Collective intelligence sharing: ISACs, vendor threat intelligence platforms, and government partnerships allow defenders to share indicators and detections across organizations. A novel attack technique detected at one organization can be deployed as a detection rule at thousands of organizations within hours. Each new attacker technique requires significant investment; detecting it once and distributing the detection is cheap.

Cost asymmetry of detection: Adding a detection rule costs less than developing a new evasion technique. This ratio has always favored defense. AI-assisted detection logic generation makes detection even cheaper. Every attacker technique that gets detected and published as a rule reduces the ROI of that technique across the entire defender ecosystem.

What Security Teams Should Actually Do

Given the genuine signal and the hype, here's a prioritized view of AI adoption for security operations based on deployments that have demonstrated measurable outcomes:

High Value: Adopt Now

AI-assisted threat hunting with natural language queries: Every major SIEM platform (Microsoft Sentinel, Splunk, Google SecOps, Elastic SIEM) now has AI-assisted query generation. The productivity impact on threat hunters is immediate and measurable — experienced hunters report 40-60% time reduction on query construction, allowing more hypotheses to be tested per shift. The learning curve is low; the productivity gain is real.

AI-powered phishing triage: Automated analysis and triage of reported phishing significantly reduces Level 1 analyst burden. Platforms like Sublime Security, Abnormal Security, and Microsoft Defender for Office 365 with AI triage achieve >90% accuracy on routine phishing classification, freeing human analysts for the edge cases and sophisticated attacks that actually warrant human judgment.

ML-based anomaly detection for lateral movement and exfiltration: If your SIEM or EDR platform doesn't have this capability, you're operating with a detection gap against the most common post-compromise TTPs. Microsoft Sentinel's entity behavior analytics, CrowdStrike's Identity Protection behavioral module, and Vectra AI's network detection are all mature products with track records.

EPSS-based vulnerability prioritization: Switch your patching priority queue from CVSS-based to EPSS-integrated. The EPSS API is free. The improvement in prioritization accuracy — more exploitable vulnerabilities patched sooner, less time wasted on high-CVSS theoretical risks — is immediate. Most vulnerability management platforms (Qualys, Tenable, Rapid7 InsightVM) now have EPSS integration built in.

Moderate Value: Evaluate Carefully

AI-powered code review for vulnerability discovery: Useful for security teams with software development scope — AppSec teams, product security, internal red teams doing code review. GitHub Copilot for Security, Semgrep with LLM augmentation, and dedicated tools like Snyk Code all have meaningful false positive rates but surface real issues that static analysis alone misses. Requires integration into CI/CD pipelines and threshold tuning. Not a replacement for manual security code review of critical security functions.

LLM-assisted incident investigation: AI that synthesizes timeline data across multiple log sources and produces natural-language incident summaries is genuinely useful for incident response. CrowdStrike Charlotte AI, Microsoft Security Copilot, and Palo Alto Cortex XSIAM's AI assistant are doing this in production. The caveat: AI-produced incident narratives require analyst validation. The models sometimes hallucinate connections, misattribute root causes, or present confabulated certainty about ambiguous evidence. Good for acceleration; not for autonomous investigation signing.

AI-generated detection rules from threat intelligence: Translating threat intelligence reports — "APT28 uses this technique described in this Mandiant report" — into SIEM detection rules is a real use case where AI accelerates a tedious manual process. SigmaHQ has community-developed LLM-assisted rule generation tools. The output requires expert validation before deployment to production.

Low Value or Not Ready: Be Skeptical

"AI-powered" next-gen AV marketed as an EDR replacement: Most "AI-powered" endpoint products are running heuristic ML models that differ from their predecessors in marketing terminology more than architecture. Evaluate endpoint products on detection performance data (AV-TEST, AV-Comparatives, independent red team results), not on AI feature claims. The ML has been in antivirus since 2018; what you're being sold as "AI" is often rebranding.

Fully autonomous AI SOC: The technology exists to automate individual workflows. It does not yet exist to replace human judgment in incident response at acceptable accuracy for high-stakes decisions. An automated SOC that can close a P1 incident without human review will make expensive mistakes — either failing to contain a real breach or taking disruptive remediation actions based on a false positive. Build automation for well-defined, low-stakes, high-volume workflows. Keep humans in the loop on anything with significant consequences.

AI for compliance documentation generation: Technically functional, operationally dangerous. Automated compliance documentation describing a security posture the organization doesn't actually have creates regulatory and liability exposure. When an incident occurs and the AI-generated documentation says "we conduct quarterly penetration testing with remediation within 30 days" and you don't, you've documented your own inadequacy for plaintiffs and regulators.

The Trajectory: Agentic AI and What's Coming

Current AI capabilities on both attack and defense sides are not the ceiling. Agentic AI — systems that can autonomously research, plan multi-step actions, execute over long time horizons, and self-direct based on intermediate results — is beginning to be deployed operationally in security contexts on both sides.

On Defense

Agentic security systems in early production at well-resourced security organizations can:

• Receive an alert, autonomously investigate it across 15+ data sources, correlate evidence across multiple log types, make an initial severity determination, and draft a detailed incident report — with human review at defined decision checkpoints
• Monitor threat intelligence feeds 24/7, identify relevant TTPs for the organization's specific technology stack and threat profile, and automatically generate and deploy detection rules in test mode for human approval
• Run continuous automated penetration testing against development and staging environments, identifying regression vulnerabilities before they reach production

The practical deployment gap: most security teams don't yet have the SOC engineering maturity to safely supervise autonomous agents. An agent with permission to block IP addresses, quarantine endpoints, and reset credentials can cause significant production impact through false positives. The oversight infrastructure — clear decision boundaries, rollback capabilities, comprehensive audit logging, human approval workflows for consequential actions — needs to be in place before the agent is empowered.

On Offense

Security researchers have demonstrated prototype autonomous penetration testing agents that can:

• Enumerate a target network, identify services, scan for known vulnerabilities
• Select and execute exploitation paths based on discovered vulnerabilities
• Perform lateral movement using discovered credentials
• Document findings in structured report format

The Autoattacker project (published research from Georgia Tech, 2024) demonstrated an LLM-based agent that could autonomously compromise lab environment targets by chaining reconnaissance, exploitation, and post-exploitation steps with limited human guidance. The gap between lab conditions and real-world enterprise environments is still significant — production environments have far more noise, heterogeneous configurations, and active defenses. But the proof of concept is real.

Nation-state actors with significant AI investment — specifically the cyber units with resources and motivation to invest in AI-augmented tooling (likely including the groups behind the Salt Typhoon US telecom intrusions documented in 2024) — are almost certainly further along in operational deployment of agentic offensive AI than anything the research community has published.

Concrete Action Items for Security Teams

If you're reading this to figure out what to actually do differently this quarter, here's the prioritized list:

Immediate (this month):

• Audit your phishing triage workflow. If it's manual, evaluate Sublime Security or Abnormal Security for trial
• Enable EPSS in your vulnerability management platform and rebuild your patching priority queue around it
• Assess which threat hunting workflows could benefit from AI-assisted query generation — start using Microsoft Copilot for Security, Splunk AI Assistant, or equivalent if you're not already

Near-term (this quarter):

• Run a workshop to identify your three highest-volume, lowest-judgment SOC workflows (likely: phishing triage, malware URL analysis, IOC enrichment) and build automation playbooks for them
• Implement ML-based behavioral analytics for authentication anomalies if you haven't already — the Okta/Entra ID unusual sign-in policies are free and require minimal tuning
• Evaluate your EDR platform's AI-based behavioral detection quality against recent threat intelligence on the TTPs targeting your industry

Strategic (this year):

• Develop the internal expertise to evaluate and supervise AI security tooling — not just to buy it. You need someone who understands enough about how these models work to know when they're wrong
• Define clear decision boundaries for what your SOC automation can do autonomously vs. what requires human approval. Document this explicitly
• Run a tabletop exercise specifically designed around AI-generated social engineering attacks against your organization — what does your incident response look like when the attack uses cloned voice and personalized AI-generated phishing simultaneously?

The organizations that will come out ahead in this arms race are the ones that treat AI adoption as a skills and process challenge, not a purchasing decision. The ones that will struggle are the ones that bought the AI security platform because their compliance framework asked about "AI adoption" — and consider the checkbox checked.

The arms race is real. The technology is genuinely transformative on both sides. The difference between winning and losing is whether your team understands the tools well enough to catch them when they're wrong.