On a dense dashboard with five buttons that look alike, my coding agent reads the screenshot, guesses which one I meant, and edits the wrong one. Then I re-explain, it tries again, another round gone.
What fixed it for me: I stopped sending the raw image. I mark the exact element, write what should change, and hand over a small structured file instead. The agent edits the right thing on the first pass, and the handoff is around 700 tokens instead of several thousand for the picture.
Curious how the makers here deal with the "which element" problem. Prompt harder? Crop tighter? Something smarter?
