[spydon]

Description

Adds three options to flame snapshot, so that AI models can match what they see in a snapshot with the component tree and get images that stay within their size limits:

• --annotate (-a) draws the outline of every visible PositionComponent on the image, labeled with its id (the same id that flame tree prints) and its type, and prints the annotated components as Player (id: 220731871) box 368,268,64,64, with the box in the pixel coordinates of the image, before the path of the image. Text only consumers get the same mapping that way.
• --filter (-f) is a case insensitive regular expression on the type, like flame tree --filter, so that only the matching components are annotated. It needs --annotate.
• --max-size (-m) limits the longest side of the image to that many pixels. The game lowers the pixel ratio before rendering instead of resizing the image afterwards, so the aspect ratio is kept, nothing is cropped and the image stays sharp. When the ratio was lowered the command prints the size and the pixel ratio that was used, so that pixel coordinates can be converted back to canvas coordinates for the input commands.

All three work for the whole game, --world, --rect and --component, at any --pixel-ratio.

Drawing on the game side

The annotations are computed and drawn on the game side, while the snapshot is rendered, in the new packages/flame/lib/src/devtools/snapshot_annotations.dart. The getGameSnapshot and getComponentSnapshot extensions take the new annotate, annotationFilter and maxSize parameters, and return pixelRatio and, when annotating, annotations with the id, name and box of each component.

The game is the only place that knows all the transforms involved: the transforms of every position component up the tree, the viewfinder (position, zoom, angle), the viewport (position, anchor and the scale of a FixedResolutionViewport), and the snapshot mode itself (canvas, world rect or component bounds, and the pixel ratio). Doing it on the CLI side would mean sending all of that over and duplicating the camera math in a second place that has to be kept in sync. On the game side the outline of a component is found by mapping its four corners up the tree with positionOf, Viewport.localToGlobal and CameraComponent.localToGlobal, which are the same functions that the camera uses, so the outlines line up with the rendering by construction. A world is mapped through the camera that renders it, the backdrop of a camera through its viewport, and the box of a world, viewport or backdrop component is checked against that viewport, so components that the viewport clips away are left out. Components that are hidden with HasVisibility, that have no area or that are outside of the image are left out too, and the outlines are drawn as the rotated quad of the component, while the printed box is its axis aligned bounding box.

The annotations are drawn in image pixels after the game is rendered, so the labels have the same legible size at every pixel ratio. Every outline has a dark border under a colored line, every label has a solid colored background with dark text, and a label is placed at the first of a few positions around its box that doesn't cover the labels placed before it, or at the one that covers them the least.

Older Flame versions

An older game ignores the unknown parameters, so the CLI checks that the response has annotations when --annotate was passed and pixelRatio when --max-size was passed. Otherwise it fails with The running game does not support --annotate, update Flame to the latest version. and exit code 69, like a missing extension, before anything is written.

Testing

The geometry is tested directly instead of through goldens, so the tests don't depend on font rendering: through a zoomed and moved camera, a rotated camera (compared against the pixels of the rendered snapshot), a FixedResolutionViewport, viewport components, a camera backdrop (compared against the rendered pixels), nested and scaled components, viewport clipping, --world with a rect and a pixel ratio, single components, and the filtering rules. The maximum size is tested to never be exceeded through rounding. On the CLI side the options that are sent to the game, the output, the usage errors and the error for an older game are tested with a fake connection.

I also verified it end to end with a small game on flutter run -d flutter-tester with a rotated and zoomed camera, a fixed resolution viewport, a nested child and a viewport component, and checked that the outlines in the produced PNGs line up for the whole game, --max-size, --world and --rect --pixel-ratio 2.

The options are documented in the CLI docs, the service extensions in the debug docs, and there is an example in the readme.

Checklist

• I have followed the Contributor Guide when preparing my PR.
• I have updated/added tests for ALL new/updated/fixed functionality.
• I have updated/added relevant documentation in docs and added dartdoc comments with ///.
• [-] I have updated/added relevant examples in examples or docs.

Breaking Change?

• Yes, this PR is a breaking change.
• No, this PR is not a breaking change.

Related Issues

Part of #4131