Vision Analyze

Analyse images with Apple Vision for saliency, classification, horizon, aesthetics, and feature print.

What it does

Vision Analyze runs an image through Apple's on-device Vision engine and reports a saliency heatmap of where attention lands, scene and object classification labels, horizon angle, an aesthetic quality score, and a perceptual feature vector for comparing images. Each analysis is its own toggle, so only what is switched on is computed.

It has no visual output of its own, feeding numbers, labels and a heatmap image forward instead.

When to use it

• Which part of an image draws the eye: turn on Saliency and read saliency_map

• Automatic labels for what is in a shot: turn on Classify and read labels and top_label

• A tilted horizon detected for auto-levelling: turn on Horizon

• A rough quality score: turn on Aesthetics

• Two images compared for similarity: turn on Feature Print and compare the two features vectors downstream

Gotchas

• Every analysis is opt-in. Saliency and Classify default on; Horizon, Aesthetics and Feature Print default off. Check that a toggle is enabled when its output looks empty.

• Saliency Mode chooses between "attention", where eyes tend to look, and "objectness", distinct object regions. The two can produce quite different heatmaps for the same image.

• features is only meaningful when comparing two images. A single feature vector on its own carries nothing; the distance between two vectors is the signal.

Worked example

1. Add a Vision Analyze node and wire an image into Image.

2. Leave Saliency and Classify on. Look at saliency_map for the attention heatmap and top_label for the leading classification.

3. Turn on Horizon for the tilt angle for auto-levelling, or Aesthetics for a quality score.

4. Turn on Feature Print on two Vision Analyze nodes fed by different images, then compare their features outputs downstream to measure similarity.

See also

Object Detect · Vision Pose · Vision Text · Rasters (images)