Skip to main content

ControlForge Machine Vision

June 2026 | ControlForge v1.0.1069

Machine vision lets your ST logic see. You trigger an inference against a camera frame, the engine runs a backend (object detection, OCR, barcode, gauge reading, color/presence/measurement) on a worker, and the result comes back as objects your control logic reads — labels, confidences, bounding boxes, and capability-specific extras. Inference is asynchronous: a trigger returns a request_id immediately and your scan keeps running; you poll a readiness flag on later scans and read the results once the request reaches a terminal state. The same path is exposed over HTTP for HMIs, MES, and offline workflows, and every result can be appended to a tamper-evident audit chain for traceability.

There are 13 VISION_* ST functions plus 7 /api/vision/* HTTP endpoints. The ST functions are non-blocking: trigger, then poll across scans — never busy-wait inside one scan.

The flow — trigger → poll → read​

Inference takes far longer than one PLC scan, so the contract is:

  1. Trigger once with VISION_TRIGGER(camera, capability) — returns a request_id (UUIDv7), or '' on error. Latch the id.
  2. On following scans, poll VISION_READY(id). It returns TRUE once the request reaches a terminal state (done, error, or canceled).
  3. When ready, read the results: VISION_OBJECT_COUNT, then per-object VISION_OBJECT_LABEL / VISION_OBJECT_CONFIDENCE / VISION_OBJECT_FIELD. Always check VISION_ERROR(id) first.

Never call VISION_TRIGGER every scan in a tight loop — one trigger per part, gated by an edge, then poll.

ST functions​

Exact signatures from the live function registry (/api/docs/functions?search=VISION_):

VISION_TRIGGER(camera: STRING, capability: STRING) : STRING
    Submit vision scan via videohist camera; returns request_id (UUIDv7) or '' on error

VISION_TRIGGER_BACKEND(camera: STRING, capability: STRING, backend_name: STRING) : STRING
    Like VISION_TRIGGER but pins to a named backend (e.g. for shadow A/B model testing)

VISION_TRIGGER_FILE(file_path: STRING, capability: STRING) : STRING
    Submit scan against a frame file on disk (offline ST workflows; bypasses videohist)

VISION_STATUS(request_id: STRING) : STRING
    Lifecycle status: pending|running|done|error|canceled|unknown

VISION_READY(request_id: STRING) : BOOL
    True once status is done, error, or canceled (terminal state)

VISION_ERROR(request_id: STRING) : STRING
    Inference error message; empty if none

VISION_OBJECT_COUNT(request_id: STRING) : DINT
    Number of detected objects in the result

VISION_OBJECT_LABEL(request_id: STRING, idx: DINT) : STRING
    ResultObject.Class — barcode text, class name, gauge id, OCR transcription

VISION_OBJECT_CONFIDENCE(request_id: STRING, idx: DINT) : REAL
    Per-object confidence 0..1

VISION_OBJECT_FIELD(request_id: STRING, idx: DINT, field: STRING) : STRING
    Capability-specific extra (gauge_value, color_hsv, etc.) as STRING; convert numeric extras with STRING_TO_REAL

VISION_INFERENCE_MS(request_id: STRING) : REAL
    Backend inference time in milliseconds

VISION_FRAME_ID(request_id: STRING) : DINT
    videohist frame_id for spine correlation (time-travel debug)

VISION_CANCEL(request_id: STRING) : BOOL
    Cancel a pending or running request

The capability argument is one of: object_detection, ocr, barcode, color, presence, measurement, gauge_read, label.

Worked example — read a barcode on a part​

A part arrives at a station (rising edge on part_present). Trigger a barcode read, poll across scans, and latch the decoded text plus a pass/fail on confidence.

PROGRAM Inspect
VAR
    part_present    : BOOL;        // INPUT — photo-eye at the station
    part_present_d  : BOOL;        // edge memory
    req_id          : STRING;      // latched request_id
    waiting         : BOOL;        // a request is in flight
    code            : STRING;      // decoded barcode text
    conf            : REAL;        // confidence 0..1
    ok              : BOOL;        // result accepted
    n               : DINT;
    err             : STRING;
END_VAR

// 1) Trigger on the rising edge of part_present — exactly one request per part
IF part_present AND NOT part_present_d AND NOT waiting THEN
    req_id := VISION_TRIGGER('cam0', 'barcode');
    IF req_id <> '' THEN
        waiting := TRUE;
    END_IF;
END_IF;
part_present_d := part_present;

// 2) Poll on later scans until the request reaches a terminal state
IF waiting AND VISION_READY(req_id) THEN
    waiting := FALSE;
    err := VISION_ERROR(req_id);
    IF err = '' THEN
        // 3) Read results
        n := VISION_OBJECT_COUNT(req_id);
        IF n > 0 THEN
            code := VISION_OBJECT_LABEL(req_id, 0);        // decoded text
            conf := VISION_OBJECT_CONFIDENCE(req_id, 0);   // 0..1
            ok   := conf >= 0.80;
        ELSE
            ok := FALSE;                                   // no code found
        END_IF;
    ELSE
        ok := FALSE;                                       // inference failed — err has the message
    END_IF;
END_IF;
END_PROGRAM

For a capability-specific numeric extra — e.g. a gauge reading — use VISION_OBJECT_FIELD and convert:

VAR
    gid       : STRING;
    raw       : STRING;
    psi       : REAL;
END_VAR

gid := VISION_TRIGGER('cam1', 'gauge_read');
// ... poll VISION_READY(gid) on later scans ...
IF VISION_READY(gid) AND VISION_ERROR(gid) = '' AND VISION_OBJECT_COUNT(gid) > 0 THEN
    raw := VISION_OBJECT_FIELD(gid, 0, 'gauge_value');   // extra field name varies by capability
    psi := STRING_TO_REAL(raw);
END_IF;

To pin a specific model/backend for A/B comparison, swap VISION_TRIGGER for VISION_TRIGGER_BACKEND('cam0', 'barcode', 'zxing-shadow'). For an offline frame already on disk (no live camera), use VISION_TRIGGER_FILE('/data/frames/part_4711.jpg', 'object_detection').

If a part leaves the station before the result returns, abandon it cleanly:

IF waiting AND NOT part_present THEN
    IF VISION_CANCEL(req_id) THEN
        waiting := FALSE;
    END_IF;
END_IF;

HTTP API​

Seven endpoints under /api/vision/* mirror the ST surface for HMIs, MES integration, and audit/regulator workflows. All require a JWT (Authorization: Bearer <token>).

Engine info and capabilities​

curl -s http://localhost:8302/api/vision/info \
  -H "Authorization: Bearer $TOKEN"
# -> {"running": true, "stats": { ... engine stats + capabilities ... }}
# 503 {"error":"vision subsystem not enabled"} when vision is off

Synchronous scan (trigger + wait in one call)​

POST /api/vision/scan submits and blocks for the result up to timeout_seconds (default 5, max 60). capability is required; supply either a camera (snapshot from videohist) or a frame_path. Optional: backend_name, model_name, model_version, node_id, an roi ({x,y,w,h} pixels), and params (backend knobs).

curl -s -X POST http://localhost:8302/api/vision/scan \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"capability":"object_detection","camera":"cam0",
       "roi":{"x":100,"y":100,"w":400,"h":400}}'

On success (200) the body is {"request_id": "...", "result": { ... }}. The result is the inference Result:

{
  "request_id": "0190f...",
  "result": {
    "request_id": "0190f...",
    "capability": "object_detection",
    "backend_name": "yolo",
    "inference_ms": 18.4,
    "frame_id": 84213,
    "objects": [
      {
        "index": 0,
        "class": "bottle",
        "confidence": 0.93,
        "bbox": {"x": 120, "y": 140, "w": 60, "h": 220},
        "norm_bbox": [0.18, 0.21, 0.27, 0.55],
        "extra": {}
      }
    ],
    "started_at": "2026-06-22T12:00:00Z",
    "completed_at": "2026-06-22T12:00:00.018Z"
  }
}

If the wait cap is hit while inference is still running, the call returns 504 with {"request_id","status":"running","hint":"poll /api/vision/status/<id>"} — switch to polling.

Poll status (async clients, or after a /scan timeout)​

curl -s http://localhost:8302/api/vision/status/$REQUEST_ID \
  -H "Authorization: Bearer $TOKEN"
# -> {"request_id":"...","status":"running"}
#    status one of: pending|running|done|error|canceled
#    when done/error the body also carries "result": { ... }

Cancel​

curl -s -X POST http://localhost:8302/api/vision/cancel/$REQUEST_ID \
  -H "Authorization: Bearer $TOKEN"
# -> {"request_id":"...","status":"canceled"}   (404 if the id is unknown)

Audit chain — tamper-evident traceability​

When the audit chain is enabled, each vision result is appended as a hash-chained entry (optionally HMAC-signed), so a lot's inspection history can be proven untampered after the fact. Three read/verify endpoints:

# Chain summary — entry count + last-entry metadata
curl -s http://localhost:8302/api/vision/audit/info \
  -H "Authorization: Bearer $TOKEN"
# -> {"count": 1042, "last_index": 1041, "last_hash":"...",
#     "last_timestamp_ms": 1750000000000, "last_kind":"result"}

# Read entries, scoped by lot / correlation / index range.
# verify=1 additionally runs hash-chain verification (adds "chain_valid")
curl -s "http://localhost:8302/api/vision/audit?lot_id=LOT-123&verify=1" \
  -H "Authorization: Bearer $TOKEN"
curl -s "http://localhost:8302/api/vision/audit?correlation=$REQUEST_ID" \
  -H "Authorization: Bearer $TOKEN"
curl -s "http://localhost:8302/api/vision/audit?from=0&to=100" \
  -H "Authorization: Bearer $TOKEN"
# -> {"count": N, "entries":[ ... ], "lot_id":"LOT-123", "chain_valid": true}

# Regulator-side full verify (chain + optional HMAC). lot_id OR correlation required.
curl -s -X POST http://localhost:8302/api/vision/audit/verify \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"lot_id":"LOT-123","hmac_key":"deadbeef..."}'
# -> {"count": N, "chain_valid": true, "lot_id":"LOT-123"}
# on a break: adds "chain_break_index" and "chain_error"

The hmac_key is hex-encoded. Omit it to verify only the hash chain; supply it to verify the HMAC signature layer as well.

Notes & limits​

  • Asynchronous by design. ST triggers return a request_id instantly; results arrive on a later scan. Poll VISION_READY — do not block a scan waiting for inference. VISION_TRIGGER returns '' on a submit error (check it), and VISION_OBJECT_FIELD returns extras as STRING (convert numerics with STRING_TO_REAL).
  • Off until enabled. Both the vision subsystem and the audit chain are opt-in. When off, the ST functions report unavailable/empty and the HTTP endpoints return 503 ("vision subsystem not enabled" / "vision audit chain not enabled").
  • /api/vision/scan blocks; ST does not. The HTTP scan endpoint waits in-call (default 5 s, max 60 s) and is convenient for HMIs/scripts. ST always uses the trigger→poll pattern. There is no blocking VISION_SCAN ST function — use VISION_TRIGGER + VISION_READY.
  • frame_id ties a result to videohist for spine time-travel debugging — VISION_FRAME_ID(id) (ST) and result.frame_id (HTTP) reference the exact frame the inference ran on.
  • Capabilities: object_detection, ocr, barcode, color, presence, measurement, gauge_read, label. The shape of per-object extra/VISION_OBJECT_FIELD fields is capability-specific (e.g. gauge_value for gauge_read).
  • Confidence is 0..1. Gate accept/reject on VISION_OBJECT_CONFIDENCE (or result.objects[].confidence), and always check VISION_ERROR before trusting a result.