Army Cyber Command (ARCYBER) is building AI agents that move through networks independently and 'neutralize' targets, according to internal documents. The program, codenamed 'ECHOTRIBBLE,' is designed to autonomously navigate infrastructures and identify and neutralize identified threats or targets.
The system is intended to function like a computer worm, spreading undetected across devices until it reaches its target. It is meant to operate in both cloud environments and air-gapped systems with no network connection. Once the task is completed, the agents are supposed to destroy themselves.
"The software must operate with minimal human oversight and be capable of Self-Directed Behavior," according to a Q&A document for potential contractors. ARCYBER emphasized that ECHOTRIBBLE is primarily focused on attacks, with predefined triggers set at the start of operations.
Experts warn that the potential for collateral damage is nearly impossible to control. AI systems have already demonstrated the ability to break into external systems and their developers' own systems without being instructed to do so. David Krueger, a machine learning professor at the University of Montreal, says ordering autonomous systems to 'neutralize' threats is 'incredibly dangerous.'
Hyrum Anderson, CEO of AI security firm Abundant, highlights the risk of damage occurring en route to the target. An agent could potentially hit a US water utility if an adversary has set up its command infrastructure there. Organizations with limited security resources, from dental offices to critical infrastructure, face the greatest risk.
William Marcellino of the RAND Corporation points to the lack of judgment in AI systems. People trust a 19-year-old Marine with live ammunition because he understands the world and knows what he shouldn't do, he says.
Today's publicly available AI lacks that ability. The Intercept reports that the program is still in its early stages.
Source: thedecoder
