Image Classifier
The Image Classifier detects objects in video using AI models. It can watch a camera continuously and report what it sees - people, vehicles, and other objects - or classify a single image or clip on request.
Overview
The Image Classifier runs an AI detection model over video frames and returns what it found: each object's type, a confidence score, and where it is in the frame. It can optionally track objects across frames so the same object keeps a stable identity and a direction of movement.
It works in two ways:
- Continuous - attached to a camera, it classifies frames as they arrive and publishes results whenever the scene changes.
- On request - given a single image or a short clip, it returns the detections for that one input.
Work is shared across instances: each instance handles up to a set number of streams at once and lets others take the rest, so you add capacity by running more instances. Detection is computationally heavy and can use a GPU where one is available.
Live video of a camera is classified once and shared by everyone watching it with classification on; classification stops when the last of those viewers turns it off or leaves. Playback of recorded video is classified separately for each viewer, from the recorded stream itself, so the boxes match the moment that viewer is watching. Live viewing and any number of playback sessions of the same camera can run with classification at the same time.
- Runs: When enabled. Continuous classification is off until turned on; the on-request classification is always available.
- Required: Only for video analytics. Not needed for telephony or alarm handling.
- Depends on: NATS, the AI runtime and a model file, and optionally a GPU.
Setting up the runtime and models
Before the Image Classifier can detect anything, the AI runtime (ONNX Runtime) and at least one model must be installed on the host. This is a one-time setup done with the classifier setup command:
This downloads the ONNX Runtime for the host's platform and the chosen model, and installs them into the directory set by imageclassifier.install.dir (by default ./models). Check what is installed with ./sbn-media classifier status.
A few notes on models:
- Ready-to-use models - D-FINE, RF-DETR, and YOLOX are downloaded as finished model files and need nothing further.
- Converted models - YOLOv8 and YOLO11 are converted on the host during setup, which needs Python with the Ultralytics package installed. If your site runs an internal model server, these are fetched ready-made instead and no Python is required.
- Re-identification - for the cross-camera "find similar" feature (see Video Analytics), install an embedder alongside the detector, for example
./sbn-media classifier setup dinov2-small(orsolider-smallfor people).
Useful flags: --runtime-only installs just the runtime, --force reinstalls over existing files, and -d <dir> overrides the install directory.
After setup, point imageclassifier.model.path at the installed model (or use a profile) and reload.
Configuration
The Image Classifier is configured under the imageclassifier namespace. View the defaults with ./sbn-media config eject and set overrides in sbn-media.local.yaml. The most useful settings:
Setting | Default | Description |
|---|---|---|
| false | Turn on automatic classification of attached streams. The on-request classification works regardless. |
|
| The detection model file to use. |
| 0.5 | Minimum confidence (0-1) for a detection to be reported. |
| 5 | Classify every Nth frame. Higher values use less CPU; |
| (all) | Limit detection to specific object classes; empty detects all the model knows. |
| (none) | Which device types to attach to automatically, for example cameras. |
| 4 | How many streams one instance classifies at once before others take over. |
| (CPU) | Where inference runs - for example |
|
| Where the AI runtime and downloaded models are kept. |
Advanced deployments can define named profiles (a model plus its detection settings) and groups (a profile plus detection areas), and can enable object tracking and a motion pre-filter. Several model families are supported, including YOLOv8, YOLO11, YOLOX, RF-DETR, and D-FINE; models can be supplied as files or downloaded.
After changing these in sbn-media.local.yaml, apply them with ./sbn-media config reload.
FAQ
Nothing is being detected even though the service is running.
Continuous classification is off until imageclassifier.enabled is true and the camera's device type is listed in imageclassifier.devices. Check also that imageclassifier.model.path points to a model file and that the confidence threshold is not set too high.
How do I limit it to people and vehicles?
Set imageclassifier.classes to just those classes. Everything else the model can detect is then ignored.
It uses a lot of CPU.
Raise imageclassifier.frame.interval so fewer frames are classified, limit imageclassifier.classes, or run inference on a GPU with imageclassifier.execution.provider. You can also cap streams per instance with imageclassifier.maxConcurrent and run more instances.
How many cameras can one instance classify?
Up to imageclassifier.maxConcurrent at once (4 by default); beyond that, other instances take the load. The practical number depends on the host, the model, and the frame interval.
Does turning classification off affect other viewers?
Only on live video, and only when you are the last viewer of that camera with classification on. Each playback session has its own classification, so ending one leaves the others running.
Related pages
- SBN Media Overview (
SBN-Media/overview) - Installing and Configuring SBN Media (
SBN-Media/installation) - Camera Stream (
SBN-Media/Devices/camerastream) - Video Analytics (
SBN-Media/Media-Processing/video-analytics) - Watchdog (
SBN-Media/Platform/watchdog)