An agent waiting for a build or a human approval need not occupy a running process for the entire wait. Infinity, an early open-source research runtime developed through an AWS and UC Berkeley collaboration, explores a different approach: save the conversation, suspend the agent and resume it when an event arrives.
How a waiting agent resumes
In AWS’s technical description, a bounded execution slice loads history, runs a model completion, stores the result and dispatches a tool. A later tool result or other message enters a queue and starts the next slice. Messages within a conversation are processed in order; different conversations can proceed in parallel.
This can reduce the need to keep an agent process alive while a long operation waits. Storage, queues, external tools and model calls still have costs and operational requirements of their own.
What the 75,000 figure means
AWS benchmarked 80,000 agents with a simulated model and in-memory stores on one thread of an AMD EPYC 9R14. Observing roughly 100 KB per idle agent, the team estimated that about 75,000 would fit in 8 GB, the memory capacity of a Raspberry Pi 5. That is a memory extrapolation, not a demonstration of 75,000 active real-model agents running on a Pi.
From research to deployment
The reference serverless setup combines SQS FIFO, Lambda, Aurora DSQL and DynamoDB. A practical deployment also has to handle repeat queue deliveries, preserve the order of each conversation and protect stored histories and tool results. Real workload, model latency and storage behavior need their own measurements; Infinity is presented as an early research release.
