---
title: "Perceptual Fingerprinting"
description: "
"
url: https://instituteofprovenance.org/docs/fingerprinting
source: Institute of Provenance
---
# Perceptual Fingerprinting

Content trust requires answering two distinct questions about media: **has this content been modified?** (forensic analysis) and **is this the same content?** (identity matching). The Institute's fingerprinting research addresses each with a dedicated pipeline.

## Two Pipelines

### Forensic Pipeline — Manipulation Detection

The forensic pipeline detects whether an image or video has been tampered with. It identifies and localizes specific manipulation artifacts:

- **Splice detection** — Regions composited from different sources
- **Clone detection** — Duplicated regions within the same image
- **Inpainting detection** — Areas filled by generative removal tools
- **Smoothing detection** — Artificial blur or noise reduction applied to conceal edits

The forensic pipeline uses Luminance Waveform Analysis (LWA) — treating pixel rows and columns as 1D brightness signals and applying spectral analysis to detect coherence anomalies. See [Luminance Waveform Analysis](/docs/luminance-waveform-analysis) for the full specification.

### Discriminative Pipeline — Identity Matching

The discriminative pipeline extracts a compact identity vector that survives common transformations: compression, resizing, cropping, format conversion, color space changes, and re-encoding.

Identity features include:

- **Perceptual hash** — pHash gating for rapid pre-filtering
- **Color analysis** — Histogram and dominant color features
- **Spatial layout** — Block-level structural features
- **Spectral features** — Frequency-domain characteristics from LWA

The identity vector is stored as an XFPR record in the wire protocol, enabling network-wide similarity search. A query with a fingerprint vector returns matching records ranked by cosine similarity.

## Transform Awareness

Both pipelines are designed to handle real-world content transformations:

| Transformation | Forensic Pipeline | Discriminative Pipeline |
|---------------|-------------------|------------------------|
| JPEG compression | Distinguishes from manipulation | >90% similarity maintained |
| Resize / resample | Detects resampling artifacts | Scale-invariant features |
| Crop | Localizes removed regions | Partial match via block features |
| Format conversion | N/A | Format-agnostic features |
| Horizontal flip | Detects geometric changes | Mirror-aware comparison |
| Grayscale conversion | Luminance-only analysis | Color-independent features available |
| Rotation | Detects geometric changes | Rotation-aware comparison |

## Video Extension

Video fingerprinting extends both pipelines to the temporal domain. See [Video Fingerprinting](/docs/video-fingerprinting) for details on dual-track visual/audio analysis, A/V sync verification, and temporal coherence scoring.

## Integration with XION

Fingerprint data integrates with the XION trust infrastructure at two levels:

1. **At registration** — When content is signed and registered, its fingerprint is computed and stored as an XFPR record alongside the XIO and XSIGN records
2. **At verification** — A verifier can query the network with a fingerprint extracted from an unattributed copy of content to find the original registration and its trust chain