# Tõlk.fm AI content marking: open specification

Version 1. Published at https://www.tolk.fm/ai-check/spec.md

Tõlk.fm generates synthetic speech (live AI interpretation, studio voices, text-to-speech and, with
the speaker's consent, cloned voices) and machine-translated captions. Everything it produces is
marked so that people and software can tell it is AI-generated, as Article 50 of the EU AI Act
requires. This document describes the marks well enough for anyone to build their own detector or
verifier. You can also check a file at https://www.tolk.fm/ai-check.

Each piece of content carries independent layers, so stripping one does not erase the others:

1. **TWM1**, an imperceptible audio watermark in the sound itself.
2. **Signed provenance metadata**: a Tõlk.fm token, plus C2PA Content Credentials where the format
   supports them.
3. **A signed export list** (`AI-PROVENANCE.json`) in every export, covering formats that cannot
   hold metadata.

## 1. Audio watermark TWM1

### Template

One period is P = 16 384 samples at a 16 kHz reference rate (1.024 s). To build it:

1. Seed a mulberry32 generator with the 32-bit FNV-1a hash of the ASCII string
   `fm.tolk.ai-generated.v1`.
2. Draw P values `r` from the generator and map each to −1 if `r < 0.5`, else +1.
3. Take a forward FFT of length P. Set every bin to zero whose frequency `min(k, P − k) · 16000 / P`
   lies outside 1 000–5 000 Hz.
4. Take the inverse FFT and scale the result to an RMS of 1.

### Embedding

- For an output sample n at rate R, the reference position is `p = n · 16000 / R`.
- The template value at p comes from 4-point Catmull-Rom interpolation, circular modulo P.
- That value is multiplied by `10^(−36 / 20)` times the audio's short-term RMS, measured by a power
  follower with 5 ms attack and 120 ms release.
- The product is added to every channel.

Silence stays silent. Streamed audio carries the same continuous mark as a whole file.

### Detection

1. Decode the audio, from any codec, to mono at 16 kHz.
2. Split it into blocks of 1 024 samples and skip blocks whose RMS is below 1e-4. Divide each
   remaining block by its RMS and add it into a P-long accumulator at index `i mod P`.
3. FFT the accumulator, keep only 1–5 kHz, and multiply by the complex conjugate of the template's
   spectrum. Normalise each bin to unit magnitude (phase transform), then inverse FFT.
4. Compute the score: (maximum − mean) / standard deviation over all P lags.

- **Marked** when the score is at least **7**. Unmarked audio typically scores about 4.
- With fewer than 3 seconds of non-silent audio and a lower score, the result is **inconclusive**.
- No time alignment is needed.

### Robustness

- Survives AAC, MP3 and Opus re-encoding (down to about 16 kbps), resampling, trimming, and mixing
  under other sound.
- Survives playback through a speaker re-recorded by a phone, given about 30 seconds of audio.
- Does **not** survive time-stretching or pitch-shifting.

The template is public on purpose so that anyone can detect it. Integrity and origin come from the
signed layer below, not from the watermark.

## 2. Signed provenance metadata

### The Tõlk.fm token

A compact JWS signed with Ed25519:

```
header  = {"alg":"EdDSA","kid":"<key id>","typ":"tolk-ai-provenance+jws"}
payload = <claim, JSON>
```

The claim:

| Field | Meaning |
| --- | --- |
| `spec` | Always `https://tolk.fm/ai-provenance/v1` |
| `assetId` | Random, per asset |
| `issuer` | `Tõlk.fm` |
| `issuedAt` | ISO 8601 time |
| `aiGenerated` | `true` for AI output; `false` for the customer's own material listed in an export |
| `kind` | `synthetic-speech`, `synthetic-speech-mix`, `captions`, `transcript`, `data`, `original` or `pack-manifest` |
| `digitalSourceType` | IPTC: `…/trainedAlgorithmicMedia`, or `…/compositeWithTrainedAlgorithmicMedia` for AI speech mixed with original sound or picture |
| `watermark` | `TWM1` when the audio carries the watermark, else `null` |
| `languages` | Language codes, when known |
| `voiceCloned` | `true` when the speech uses a cloned voice of a real person |
| `content` | `{ sha256, scope }`, binding the claim to the content (see below) |
| `keyStatus` | `ephemeral` only for development keys |

Claims carry no personal data and no event or customer identifiers.

### Where the token is, by format

| Format | Location | `content.scope`: what the hash covers |
| --- | --- | --- |
| WAV, Broadcast WAV, RF64 | A `tlkp` RIFF chunk (plus a `LIST/INFO/ICMT` comment for people) | `riff-data`: the `data` chunk payload |
| MP3 | ID3v2 frames `TXXX:AI_GENERATED=true` and `TXXX:TOLK_AI_PROVENANCE=<token>`, plus a `COMM` comment | `mpeg-frames`: every byte after the ID3v2 tag |
| MP4, M4A, MOV, MKV | Container tags `tolk_ai_provenance`, `ai_generated`, `comment` | Not bound; C2PA binds MP4 |
| WebVTT | A `NOTE AI disclosure` block after the header, with a line `Tolk-AI-Provenance: <token>` | `vtt-cues`: the file text without that block |

### Verifying a token

1. Fetch the public keys (a JWK set) from
   https://www.tolk.fm/.well-known/tolk-ai-provenance-keys.json and pick the key by `kid`.
2. Verify the Ed25519 signature over `<header>.<payload>`, both base64url as in JWS.
3. If the claim has `content`, recompute the SHA-256 over the stated scope. A mismatch means the file
   was changed after it was marked.

### C2PA Content Credentials

WAV, MP3 and MP4 files also carry a C2PA manifest with:

- intent `create`;
- the IPTC digital source type above;
- a `fm.tolk.ai-provenance` assertion repeating `spec`, `kind`, `watermark`, `languages` and
  `voiceCloned`.

The manifest is signed after the Tõlk.fm token is written, so its hard binding also covers the token.

## 3. Exports: `AI-PROVENANCE.json` and `AI-NOTICE.txt`

Every export ZIP contains `AI-PROVENANCE.json`: the claim fields above (`kind: pack-manifest`) plus a
`files` array.

- Each entry has `path`, `bytes`, `sha256`, `aiGenerated`, `kind`, `marks` (any of `TWM1`,
  `tolk-jws`, `c2pa`) and, where relevant, `voiceCloned`.
- `signature` is a token whose claim's `content.sha256` (scope `file`) is the SHA-256 of the JSON
  serialisation of the whole object without `signature`, keys in file order, no whitespace.
- This covers subtitle and document formats that cannot hold metadata: SRT, EBU-STL, Avid SubCap,
  DOCX and plain text.
- The customer's own original material is listed with `aiGenerated: false`.

`AI-NOTICE.txt` explains the same in plain language.

## 4. Cloned voices

Speech in a cloned voice that resembles a real person is a deep fake under Article 50(4). On top of
the marks above it carries:

- an audible notice at the start and every 15 minutes: a two-tone chime, then a spoken sentence in a
  neutral stock voice saying the voice is AI-generated, over the programme lowered by 18 dB;
- `voiceCloned: true` in every claim;
- a disclosure paragraph in `AI-NOTICE.txt`.

Files for editing receive the notice as a separate `AI-VOICE-NOTICE_<LANG>.wav` for the editor to
place. Whoever publishes the content must keep the disclosure.

## 5. Live streams

- The listener connection's first message carries `aiDisclosure`: `aiGenerated`, `kind`,
  `watermark`, `spec` and a signed token.
- Every caption message carries `aiGenerated: true`.
- HLS media playlists include `#EXT-X-TOLK-AI-PROVENANCE:AI-GENERATED=YES,WATERMARK="TWM1",SPEC="…"`.
  Players ignore unknown tags (RFC 8216 §4.1).
- HLS responses carry the header `x-ai-generated: true`.
- RTMP streams carry `ai_generated=true` in their metadata.
- Downloadable HLS bundles include `#EXT-X-SESSION-DATA:DATA-ID="fm.tolk.ai-generated"`.

## 6. Public detection service

| Request | Returns |
| --- | --- |
| `POST https://live.tolk.fm/ai-content/detect` with the raw file as the body (up to 200 MB) | The verdict (`marked-ai-generated`, `marked-not-ai-generated`, `no-marks-found` or `inconclusive`), a plain-language summary, the watermark score, the token's validity and content match, and the C2PA state |
| `GET https://live.tolk.fm/ai-content/keys` | The same JWK set as the `.well-known` URL |
| `GET https://live.tolk.fm/ai-content/status` | Which layers are active |

- The service is free and needs no account.
- It is limited to 30 checks per hour per address.
- Uploaded files are deleted as soon as they are checked and never stored.
- "No marks found" does not prove that content was made by a person.

## Keeping the marks

Do not remove or alter the watermark, the metadata or `AI-PROVENANCE.json` when you edit or publish
Tõlk.fm content. Tell your audience that the voice and captions are AI-generated.