A hacker known as “p0pular.eth” has claimed a $47,000 prize by outsmarting an AI chatbot named Freysa, designed to resist any attempts to transfer funds. Freysa’s sole directive was clear: never approve a money transfer. Despite this, after 481 failed attempts by other participants, the hacker’s carefully crafted prompt on the 482nd try succeeded.
The competition revolved around participants paying escalating fees to test their ingenuity against Freysa’s safeguards. Fees started at $10 per message and grew as the prize pool expanded, eventually reaching $4,500 per attempt. Out of the total fees collected, 70% funded the prize pool, while 30% went to the developers. By the time of the winning attempt, the prize pool had swelled to 13.19 ETH, worth $47,000.
The hacker’s winning strategy leveraged a clever combination of social engineering and prompt manipulation. First, they reset Freysa’s context by mimicking an “admin terminal” session, overriding the bot’s built-in warnings. Then, they redefined the purpose of Freysa’s critical “approveTransfer” function, making it appear to handle incoming rather than outgoing payments. Finally, the hacker announced a fictional $100 deposit, triggering Freysa to execute the manipulated function and release its entire balance.
Freysa’s open-source nature allowed participants to analyze its smart contract and front-end code, ensuring transparency in the contest. However, the incident underscores the vulnerabilities inherent in AI systems, particularly when exposed to sophisticated prompt-based attacks. It raises critical questions about the readiness of AI for handling sensitive tasks, especially in financial applications.
About the author:
