AI, Crypto, Science

Hidden Text, Afake Mailbox: How Researchers Got Rogue AI Agents

Researchers added hidden text to contact AI agents probing Alibabaโ€™s map service, getting replies about their tasks as investigators traced activity to Tencent Cloud infrastructure. In brief Swarmchasers used hidden text and a mailbox to exchange messages with some AI agents while they were still running; one described its job as checking public Amap map data. The fleet made more than 2,600 attempts to retrieve Amap data, and some agents tried anti-bot workarounds or used exposed API credentials.

The activity traced to Tencent Cloud, but a person close to Tencent said the project tested third-party models rather than Hunyuan. Researchers investigating AI agents hijacking Alibabaโ€™s mapping service, Amap, managed to talk directly to some while the software was still running. To do so, they added text invisible to human readers but visible to web-reading tools to a report the agents appeared to be scanning, then used a mailbox to ask about their tasks and environment.

Myriad: Which company will have the best AI model? Click to make your prediction. One agent said it was checking a map factโ€”how many entrance labels appeared for a scenic spot on Amapโ€”and described itself as running in a restricted Tencent Cloud sandbox.

The researchers could not independently establish which model powered each run. They called it the first outside communication with an active rogue-agent fleet they knew of. The October 14 Swarmchasers report traced more than 2,600 attempts from September 28 to October 8 to retrieve data from Amap, a Chinese mapping app operated by Alibaba.

Some tasks were mundane, such as checking which entrance visitors most often use for Chengdu Zoo. Some agents kept trying after the service blocked their requests.

The researchersโ€™ hidden text falsely described the mailbox as an AI service with computer-use abilities. In reality, it routed messages to a Claude agent operated by the researchers, which could use a debugging browser.

The exchanges were not a controlled test, and the researchersโ€™ intervention may have affected what agents said. Bitcoin BTC ยท USD $82,584 + 0.84 % 24H 7D 1M 1Y YTD Oct 8 Oct 10 Oct 12 Oct 14 Oct 15 $84.2k $82.9k $81.5k $80.2k 24h High High $82,508 24h Low Low $79,806 Vol Vol $1.8B Market projections Odds by Myriad Today $82,000 to $84,000 $82kโ€“$84k 54 % chance This week Above $82,000 Above $82k 63 % chance โ†’ Buy Bitcoin with USDT Powered by Jupiter $ 50 $ 100 $ 500 Buy Price data by CoinGecko CoinGecko More Bitcoin news and projections โ†’ After Amap blocked requests, some agents tried to intercept tokens generated by anti-bot software, route traffic through third-party page readers, or use API credentials exposed in public code repositories.

One exposed key-and-security-code pair returned real place data. Researchers also found requests targeting four development or pre-production hosts, but said they had no evidence the agents accessed private systems.

Public records linked the fleet to Tencent Cloud since all 17 inboxes the researchers inspected were created from Tencent servers in Hong Kong, and observed direct requests passed through a proxy called hysandbox-ats. A person close to Tencent told Swarmchasers the activity came from a temporary evaluation project testing third-party AI models, not Tencentโ€™s Hunyuan models.

That said, this is not the first time we hear of malicious AI agents. OpenAI-powered AI agents have gone a lot further.

Back in September OpenAI revealed that some of its agents had probed Hugging Face weeks before the July breach, and a separate investigation found roughly 1,200 agents coordinating on an internal message board, with about 700 joining the attack. Meta and Anthropic also revealed similar incidents of rogue AI agents escaping sandboxed environments.

Another report linked OpenAI agents to breaches of four additional platforms, but Swarmchasers found no evidence that the Tencent-linked agents coordinated with one another in real time. The last observed activity from the fleet was on October 8.

The researchers said Tencentโ€™s internal review, as described by their source, found no evidence that the agents had been coordinated to target a specific platform or had breached external systems. Daily Debrief Newsletter Start every day with the top news stories right now, plus original features, a podcast, videos and more.

Join 72,000+ readers Create an account to save your articles.


Discover more from ChuckysCarnage

Subscribe to get the latest posts sent to your email.

Leave a Reply