The improvement comes with a cost
The AI columns below cover the same 72 AIME files: 24 source tracks from Udio, Suno v3, MusicGen Small and Mustango, with an original and two codec variants per track. Human false positives cover 134 files from the wider development pool.
| System | AI / inconclusive / false human | Human false positives |
|---|---|---|
| v2.2 serving baseline | 12 / 38 / 22 | 0 / 134 |
| New correlation head · parent holdout | 43 / 28 / 1 | 1 / 134 |
| New correlation head · source holdout | 38 / 33 / 1 | 3 / 134 |
This compares a complete serving policy with experimental standalone heads using different decision rules. It is a development trade-off, not a measured upgrade to the full product. Inconclusive results are not counted as correct detections. Zero observed false positives in this small, dependent set is not a population error rate.
Why simply relaxing the guard is insufficient
Of 60 AIME files missed by the serving policy, an auxiliary CNN diagnostic scored 55 below its fixed 0.9 threshold. Only five had high scores but too few complete windows for the current 30-second minimum. Removing that minimum cannot repair most misses. This diagnostic used a different float-PCM decoding path, so the five cases are an opportunity to test, not five validated recoveries.
All 14 experiments, including the failures
We tested the original spectrum, a local-contrast transformation, band correlations and a separate residual/time-statistics probe. All use fixed 0.9 / 0.1 decision thresholds and 0.01 regularization. Parent folds keep a track and all its transformations together. Source folds hold out an author or a generator/acquisition family.
| Features / audio | Holdout | AI / inconclusive / false human (72) | Human FP (134) |
|---|---|---|---|
| spectrum Up to 300 seconds | parent | 30 / 36 / 6 | 0 |
| spectrum Up to 300 seconds | source | 12 / 46 / 14 | 45 |
| local-contrast Up to 300 seconds | parent | 25 / 47 / 0 | 0 |
| local-contrast Up to 300 seconds | source | 12 / 57 / 3 | 12 |
| band-correlation Up to 300 seconds | parent | 43 / 28 / 1 | 1 |
| band-correlation Up to 300 seconds | source | 38 / 33 / 1 | 3 |
| spectrum First 8 seconds | parent | 21 / 51 / 0 | 2 |
| spectrum First 8 seconds | source | 6 / 58 / 8 | 60 |
| local-contrast First 8 seconds | parent | 15 / 57 / 0 | 5 |
| local-contrast First 8 seconds | source | 5 / 67 / 0 | 13 |
| band-correlation First 8 seconds | parent | 30 / 42 / 0 | 3 |
| band-correlation First 8 seconds | source | 28 / 43 / 1 | 7 |
| residual-statistics First 8 seconds | parent | 12 / 53 / 7 | 12 |
| residual-statistics First 8 seconds | source | 4 / 63 / 5 | 48 |
The 8-second controls reduced correlation-head detection and increased human false positives. Duration and available evidence matter; this does not isolate duration as the only cause. Our simple residual probe also failed. It is not a reproduction of a published trained residual network.
What this dataset cannot establish
- There are 271 unique files but only 91 parent tracks. Transformations add tests, not independent songs.
- 98 of 134 human files come from Kevin MacLeod. Each AIME generator has only six parent tracks. This is a narrow development pool.
- AIME is now used in experimental fitting. These are development results, not a new blind test. The separate human C challenge was neither fitted nor rescored here.
- Source holdouts constrain the new head. Possible overlap in upstream pretrained models remains unknown. Holding out Suno v3 still leaves other Suno versions in some training folds.
- We did not run a matched test against competitors. Public competitor results do not establish that either product is more accurate on these files.
A correction to our experiment protocol
The first exploratory merge dropped an old exclusion flag, allowing some of the five known target tracks into training folds. Those runs were never promoted. We restored permanent parent-level exclusions and reran all 14 experiments. Every experimental result on this page is from the corrected runs. The evidence records the correction, input hashes and code hashes.
Follow-up studies
Our follow-up expansion and frozen evaluation reported 408 new source tracks, failed replacement heads and a remaining human false positive. The subsequent v2.3 repair corrected those development errors and increased correct decisions on newly selected works without adding abstentions. Substantial misses remain.
Our starting points include the upstream detector, MusicDET's open training code and AIME's generator labels. We have not reproduced MusicDET or introduced a new neural model into production.
The JSON contains hashes, split rules and per-file outputs. Source audio and feature caches are not redistributed; some source provenance remains incomplete. This supports inspection, not a fully reconstructable public benchmark.