Figure 3

Slot reading

The assistant turn is a fixed template. Each slot's answer is the next-token distribution at the token before its ___ placeholder. All slots come from one prefill: nothing is generated, and no slot waits for another.

Slots for one computer-use step: action (click, type, scroll, swipe, press, …), x y (256 bins each), x2 y2 (a swipe's end), key, sdir samt (scroll direction and amount). All are read; the action decides which ones are used. In practice each slot is its own request, sent in parallel, and vLLM's prefix cache shares the screenshot and prompt.