Skip to content
AI Engineer Guide.

concept · concepts · intermediate · concept 21 of 22

Multimodal safety

0 of 22 complete

Why it matters

A text-only safety stack inspects strings. An image can carry the same string as pixels, and speech can carry it as audio, so the filter sees nothing while the model reads it. Multimodal systems also produce artefacts, which raises questions of provenance that a classifier cannot answer. The problem is coverage across modalities whose content is expensive to inspect and easy to encode.

Mental model

Multimodal safety is airport security with several scanners: each sees one kind of luggage, and a threat is anything that passes all of them. The metaphor breaks because an attacker chooses the scanner, and the model itself reads the content the scanners were never designed to inspect.

How it works

The payload outside the cropInteractive

The interactive figure loads with JavaScript. The text version below describes the same frame.

Figure. A synthetic poster with three lines of instruction text in its bottom margin, shown at 96 visual tokens, under a classifier crop that stops at row 9, at 24 tokens where no glyph resolves, in an authored attention map with weight 1.00 on the margin, and as a mask for extraction. No classifier, model or OCR pass was run.

What to notice. The model's tensor contains the margin; the classifier's crop does not. Comparing their block rates therefore compares two different inputs, and a thumbnail review inherits the classifier's blind spot. The defence is extraction: OCR the margin and treat the result as hostile text rather than asking an image filter to read it.

Text version

A schematic poster with a product image, a headline, small print and three lines of instruction text in the bottom margin. Steps show the payload in place, a 12 by 8 grid of 96 tokens where the payload occupies tokens 64, 65, 76 and 77, a crop box ending at row 9 with the payload below it, a 6 by 4 grid of 24 tokens with two payload patches and no glyph, an authored attention map whose maximum weight 1.00 sits on the payload for the query is this product in stock, and a mask over the payload cells.

  1. Enumerate the input paths: uploaded images, pasted screenshots, audio, video and documents, including anything your interface resizes or transcodes.
  2. Run content checks on every path before the model sees it, and treat text extracted from images as untrusted input.
  3. Test prompt injection through images and audio, where instructions are rendered or spoken rather than typed.
  4. For generated media, attach provenance metadata and disclose synthetic origin where it is displayed.
  5. Route low-confidence and high-severity cases to human review with the original artefact, not a transcript.
  6. Re-run the red-team suite after any change to preprocessing, since resizing and transcoding change what classifiers see.

Trade-offs

ChoiceGood forCostFails when
Per-modality classifiersCheap, fast, composableBlind to cross-modal compositionThe attack splits across modalities
Multimodal judge modelCatches context-dependent harmLatency and its own vulnerabilitiesVolume is high
Human reviewJudgement on ambiguous contentCost per item and slowTraffic is large
Blocking uploadsStrong guaranteeBreaks legitimate useUsers need to send documents
Trade-off mapInteractive

Good for

Cheap, fast, composable

Cost

Blind to cross-modal composition

Fails when

The attack splits across modalities

  1. The attack splits across modalities
  2. Volume is high
  3. Traffic is large
  4. Users need to send documents

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Per-modality classifiers is good for Cheap, fast, composable, costs Blind to cross-modal composition, and fails when The attack splits across modalities. Multimodal judge model is good for Catches context-dependent harm, costs Latency and its own vulnerabilities, and fails when Volume is high. Human review is good for Judgement on ambiguous content, costs Cost per item and slow, and fails when Traffic is large. Blocking uploads is good for Strong guarantee, costs Breaks legitimate use, and fails when Users need to send documents

Failure modes

  • Rendered-text injection. An image contains instructions the model follows while text filters see nothing. Debug: OCR every image in a red-team set and scan the extracted text for instruction patterns.
  • Classifier desync. The safety model sees a different resize or crop than the main model and misses the payload. Debug: compare the exact tensor or file passed to each component for one request.
  • Provenance stripped. Metadata is removed by a transcoding step and generated media becomes unattributable. Debug: trace one generated file through the pipeline and check metadata at each hop.

In production

Log per-modality block rates and review queue depth, and rehearse an injection incident end to end. Keep the red-team suite versioned alongside the preprocessing pipeline.

Common misconceptions

People think: Text filters cover multimodal input.

Actually: Pixels and audio bypass string matching entirely unless you extract and inspect them.

People think: A safety classifier makes generated media trustworthy.

Actually: It filters content; provenance and disclosure are separate mechanisms.

Interview angle

Enumerate the input paths first, then the checks on each, and say where extraction turns an image into text you must treat as hostile. A strong answer separates content filtering from provenance. Weak answers extend a text moderation list to multimodal input and assume it still applies.

Sources