AI Scene Classification¶

The AI Scene Classification batch tool uses AI to describe your video thumbnails and add the results as keywords. You can access this tool from the start page or by right-clicking a video and selecting Classify Scenes... from the context menu.

The tool offers three classification options that can be used independently or together. Each works from its own fixed vocabulary, and they differ in what they recognise and how they decide. A fourth option, Visual search index, adds no keywords; it prepares the thumbnails for visual search.

Only object detection can be extended with subjects of your own. If none of the three recognises something specific to your work, that is the route to take — see Custom Object Detection Models.

Scene Classification labels each thumbnail with the single best-fit place category from its list of 365 (e.g., kitchen, beach, airfield). One label per thumbnail — it answers "where is this?".

Object Detection finds specific objects from a set of 80 common types (e.g., car, person, dog) and can add several to the same frame. See Object Detection.

Image Tagging describes the whole frame at once, spanning scenes, objects, settings and activities (e.g., beach, crowd, airplane, sunset). It is the most general of the three, and the only one whose model is downloaded on demand. See AI Image Tagging.

The three overlap by design, and running them together is usually worth it: a single pass over your footage can tell you the setting, the objects in it, and a general description, and they all become searchable keywords.

The minimum confidence slider applies to Scene Classification and Object Detection — it controls how certain the AI must be before adding a scene or object keyword; higher values reduce false positives but may miss some. Image Tagging is governed by its own Tag sensitivity, which works differently — see How a tag is chosen.

Select which videos to process: All videos, the currently selected videos, or the filtered set. You can also choose to skip thumbnails that have already been classified. The frame skip setting controls how many thumbnails to skip between processed ones, where 1 means all thumbnails are processed.

All three options read your captured thumbnails, not the video file itself. How much detail they can find is therefore bounded by how often thumbnails were captured when the video was indexed, and then by the frame skip set here.

All three can also run automatically while videos are indexed — switch them on in the AI Classification preferences. Image tagging additionally needs its model downloaded before the setting has any effect.

Object detection can also run with your own fine-tuned model to detect custom object types — see Custom Object Detection Models.

Visual search index¶

Besides the three classification options, the AI Scenes tool has Visual search index. It records a compact description of every thumbnail — the index the visual search uses — and shows under the option how much of the catalog is covered. Tick it to add the thumbnails that do not have one yet. It reads every thumbnail regardless of the frame skip, and it is safe to stop and run again later: only the thumbnails still missing are added. After the first fifty it reports its speed and how long the rest will take, which matters on a computer without a graphics card, where it runs at about a second or two per thumbnail.

The same index is built automatically while videos are indexed when Build the visual search index while indexing is on in the AI Classification preferences (it is on by default), so a catalog indexed from now on needs no separate run. Both need the AI model that image tagging and trained tags share; without it the option is disabled.