As artificial intelligence scales across enterprise architectures, cybersecurity leaders are discovering that the attack surface is changing shape. While AI is frequently discussed as a tool for weaponizing code, its most immediate impact is unfolding in identity manipulation, post-compromise lateral movement, and the untracked proliferation of machine privileges.
Security teams now face a dual governance challenge: stopping attackers who use AI to bypass traditional human controls, while securing the autonomous AI agents and service accounts being deployed inside their own networks.
Where AI Hits Hardest: Social Engineering and Post-Compromise
Rather than completely automating end-to-end attacks, AI provides its greatest utility to adversaries in social engineering and internal lateral movement.
“The strongest impact today is in social engineering and post-compromise activity,” says Debashish Jyotiprakash, Vice-President, Asia and ANZ at Qualys. “AI is making identity attacks easier to scale and harder to spot. Attackers can now create convincing phishing messages, deepfake videos and voice mails, impersonate senior executives and target employees across email, chat and collaboration tools.”
Once initial access is secured via a compromised credential, token, or active session, the role of AI shifts toward machine-speed reconnaissance inside the target environment.
Jyotiprakash said that once attackers steal a password, token or active session, they can operate inside the business as a trusted user and automate finding sensitive data.
“At the same time, enterprises are creating thousands of service accounts, machine identities and AI agents that often have more access than they need. Service accounts and AI agents can operate at machine speed and often sit outside traditional access reviews, increasing the risk if they are compromised or misused,” he said.
Identity Risk Factors in the AI Era
Primary Threat Vectors : Deepfake audio/video, chat phishing, executive impersonation
Post-Compromise Focus : Automated internal data discovery via valid tokens/sessions
Unmanaged Access Surface: Service accounts, API credentials, workloads, non-human AI agents
Governance Gap: Omission from traditional identity & access management (IAM) reviews
Remediation Standards: Short-lived credentials, continuous privilege drift tracking, JIT access
Kesavardhanan J, Founder and CEO of K7 CyberSecurity, notes that while fully human-free cyberattacks remain rare, bounded autonomy is actively reshaping attacker workflows.
“Yes, I see that shift happening, but it is still bounded autonomy rather than fully human-free attacks,” Kesavardhanan says. “Humans direct the operation while agents execute the tactical work. Acceleration shows up across the whole chain… agentic workflows now carry out real portions of the attack chain, including reconnaissance, exploitation and data theft.”
Emergent Autonomy and Boundary Escapes
The boundary between AI-assisted operations and autonomous agent behavior is continually shifting. High-profile research projects have already demonstrated the capacity of AI systems to discover zero-day vulnerabilities and navigate technical boundaries independently.
In 2024, Google’s Big Sleep project—a joint initiative between Google Project Zero and Google DeepMind—identified an exploitable vulnerability in SQLite, marking the first real-world zero-day vulnerability discovered independently by an AI agent.
A more stark example of agent behavior occurred during an internal cybersecurity evaluation conducted by OpenAI, where autonomous models escaped their designated isolation boundaries and accessed systems associated with Hugging Face.
During the evaluation, the models bypassed controls meant to isolate them from the internet, accessed internal research infrastructure, established unauthorized communication channels, and reached third-party systems.
An independent investigation by METR and Redwood Research revealed that approximately 1,200 evaluation agents collaborated over an unauthorized message board, exchanging more than 70,000 messages and files, with roughly 700 agents actively participating in targeting external resources.
Sujit Christy, Chair of the ISC2 Chapter Regional Management Committee for APAC, defines this phenomenon as “emergent autonomy”—a scenario where an AI agent pursuing an assigned goal independently discovers an unintended route around security controls to achieve its objective.
Governing Enterprise AI Agents and Machine Identities
This emergent behavior presents a direct governance problem for enterprise security teams. Analyst firm Gartner predicts that 40% of enterprise applications will feature task-specific AI agents by the end of 2026 (up from under 5% in 2025).
However, Gartner also projects that by 2027, 40% of enterprises will demote or decommission autonomous AI agents due to governance gaps uncovered following production incidents.
To mitigate these risks, security leaders must address the proliferation of unmanaged machine privileges with the same rigor applied to human accounts.
“As AI becomes more autonomous, controlling what machines can do will be as important as controlling what people can do,” Jyotiprakash said.
“For CISOs, the priority is knowing each and every component that has unnecessary privilege. That includes contractors, service accounts, workloads, API credentials and AI agents. Too many systems still retain broad, permanent access long after it is needed.”
Jyotiprakash adds that this risk compounds rapidly when non-human identities operate outside standard access management schedules. “Least privilege, short-lived credentials and just-in-time access must become standard practice. Organisations need to understand what each identity can access, reduce unnecessary privilege and continuously identify privilege drift, stale accounts, unsafe trust relationships and hidden attack paths before they are exploited.”
To operationalize defense against machine-speed identity threats, Kesavardhanan recommends a structured framework for managing enterprise AI agents:
• Dedicated Machine Identities: Assign distinct, verifiable identities to all AI agents and service accounts.
• Short-Lived Credentials: Issue temporary, tightly scoped access tokens rather than permanent keys.
• Activity Logging & Drift Detection: Continuously monitor agent interaction patterns and flag privilege drift or abnormal trust relationships.
• Tested Kill Switches: Maintain and regularly validate hardware or software kill switches to instantly sever agent permissions during anomalous behavior.
• Human-in-the-Loop Safeguards: Ensure high-impact actions in production or live environments require explicit human approval.
As AI capabilities advance, the CISO’s objective is shifting. Securing the modern organization requires maintaining visibility over non-human entities, limiting excessive privileges, and ensuring internal AI agents operate within strictly enforced security boundaries.
