Microtuning and MPE Inside Unified Audio AI Models: Where the Pitch Layer Breaks

The brief was four bars of ambient bed for a temple level, and it said one thing clearly: not Western tuning. Slendro-adjacent, roughly five near-equal steps to the octave, nothing that lands on a piano key. What came back from the generator was a beautiful gamelan-shaped texture — bronze-ish attack, long shimmering decay — playing an unambiguous D minor pentatonic. Every partial snapped to the 12-tone grid. This is the thing to understand about unified audio AI models before you build a product on top of one: they are extraordinary at timbre and close to useless as a pitch instrument.

The verdict, stated once so you can stop here if that settles it: if microtuning or per-note expression is load-bearing in what you ship, keep the pitch layer under your own control and buy these models for everything downstream of pitch — texture, foley, ambience, risers, room tone, transitions. That is not a missing checkbox a vendor ships next quarter. It follows from where pitch lives inside the architecture, and it fails at one specific point in the chain.

So walk the chain in causal order. Each stage discards something particular. Knowing which stage discards what is what tells you where to intervene.

Stage one: the interval you meant

Start with the thing that exists before any software: an interval, measured in cents. A cent is one hundredth of a 12-tone equal-tempered semitone, and the differences that matter here are small enough that people who don't work in pitch tend to dismiss them.

Equal temperament puts a major third at 400 cents. The just major third — the 5:4 ratio, the one that stops beating and locks — sits at about 386.3 cents. That gap of roughly 13.7 cents is the difference between a sustained brass pad that sounds like a chord and one that sounds like a chorus effect. The harmonic seventh, 7:4, lands near 968.8 cents against equal temperament's 1000; that 31-cent shortfall is why a barbershop tag or a well-tuned dominant ninth feels different from what a piano can produce.

For a rough sense of audibility: at A440, a 4-cent error against a reference produces about one beat per second. Trained ears catch that instantly on a sustained dyad. It is not a subtlety.

The use cases where this becomes a specification rather than a preference are concrete. A game studio building a fictional civilization wants a scale the player has never heard, which usually means 19-EDO or 31-EDO (a 31-EDO step is about 38.7 cents, roughly a third of a semitone). A documentary scored around maqam material needs quarter-tone inflections that 24-EDO approximates and 12-EDO cannot. A period drama wants a historical temperament with its wolf fifth intact. And the gamelan case above has a wrinkle worth naming: there is no standard slendro. Individual gamelan sets are tuned as sets, and the intervals vary between them. "Correct" here means matching a specific reference recording, not hitting a canonical table.

Separately from tuning there is expression: per-note vibrato that starts after the attack, a slide that moves one voice inside a held chord, pressure that opens a filter on the top note only. That is what MPE exists for, and it is a different axis from tuning that dies at the same place in the chain.

Stage two: how that intention gets encoded

Before any audio exists, there is a symbolic layer, and it is genuinely capable.

MPE (MIDI Polyphonic Expression) works around MIDI 1.0's channel-wide bend by rotating each note onto its own channel. In a standard lower zone, channel 1 is the master and channels 2–15 are member channels; the specification's default per-note pitch bend range is ±48 semitones, which is what makes wide continuous glides possible without quantization crunch. CC74 carries the timbre axis, channel pressure carries the third. Every note becomes an independently steerable object.

Scala files carry the tuning itself: a .scl file describes the scale as a list of cents values or ratios, and a .kbm file maps those degrees onto keys. Between them you can express essentially any tuning anyone has written down, including one you measured off a recording of a specific gamelan.

MTS-ESP distributes tuning to plugins in real time, so a whole session can shift temperament mid-cue rather than being frozen at load.

MIDI 2.0 raises the ceiling further with per-note pitch at 32-bit resolution and per-note controllers, removing the channel-rotation workaround altogether.

The important property of this entire layer: it is symbolic, pre-audio, and lossless with respect to pitch. A cents value is a number. Nothing has been rendered yet, so nothing has been approximated.

Stage three: the handoff, and where the resolution dies

Now hand that to a generative audio model. Look at what the interface actually accepts.

The primary control surface on unified audio AI models is a text prompt — a few hundred tokens of English. There is no field for a .scl file. There is no cents offset. "31-EDO" in a prompt is a string the model has almost certainly seen in metadata or filenames far less often than it has heard music in equal temperament, and prompt tokens do not bind to pitch the way a tuning table binds to an oscillator. You are describing, not specifying.

The second channel is audio conditioning: upload a reference and ask the model to follow its melody or structure. This is the only route with real information bandwidth, and it is where the loss is easiest to name. A common melody-conditioning approach extracts a chromagram from the reference — an energy distribution across twelve pitch classes per frame. Twelve bins. A quarter tone does not survive that representation; it lands as blur or leans to a neighbour. The conditioning signal is the microtonal content quantized to the exact grid you were trying to escape, before the model has generated a sample.

That is the first hard discard in the chain, and it happens at the API boundary.

Stage four: what the model does with pitch

Assume you got past conditioning. Inside an audio-domain model, pitch is not a parameter.

These systems generally work in a learned latent space produced by a neural audio codec — the waveform compressed into a sequence of discrete or continuous frames, commonly on the order of tens of frames per second, from which a decoder reconstructs audio. That latent has no pitch axis. Fundamental frequency is a distributed correlate spread across dimensions and entangled with timbre, register, and room. There is nowhere to write 386.3.

On top of that sits a training prior. The overwhelming majority of recorded music the model learned from is in or near 12-tone equal temperament. When the generation process resolves ambiguity — and diffusion or flow-matching models resolve enormous amounts of it — the prior pulls toward the grid. Call it tuning gravity. It is why a texture prompted as microtonal comes back as a 12-TET scale with a slightly detuned chorus on top: the model has learned detuning as a timbral gesture, an ensemble effect, not as a scale system.

The generation process makes this worse in a way worth understanding. These models are typically non-causal: they generate a whole clip, denoising globally rather than emitting one note after another. There is no per-note decision point to hook a controller onto. Even if you knew which latent dimensions correlated with pitch, there is no "note 3" to address — it does not exist as an object at any point in the computation.

That is the second hard discard, and it is architectural rather than a product gap.

Stage five: what actually comes back

A render arrives. Now the question is what you can repair.

Check the export specifics against your pipeline before anything else: sample rate and bit depth, whether you get a stereo mix or separated stems, whether loop metadata and tempo survive, and whether the stated key is present and correct (it is frequently absent, and when present, not reliable enough to drive a build script).

Stems are the single most consequential export feature for this workflow. A mixed stereo render is effectively unrepairable at the pitch level. Polyphonic retuning of a full mix means stretching reverb tails and shared partials along with the notes, and the artifacts arrive well before the tuning does. Given isolated stems, a polyphonic pitch editor with custom scale support can pull a monophonic or sparsely polyphonic line onto a target tuning with results that hold up in a mix.

The honest negatives on the output side, independent of tuning: percussive transients often come back softened, which matters when the loop has to sit under a gunshot; high-frequency content — cymbal wash, breath, sibilance — is where codec artifacts show up first and where a listener on decent headphones notices; and sustained solo vocal material remains the hardest category to get to a shippable state. Prompt roulette is real, and it has a cost line, which I will get to.

Stage six: where the pitch layer belongs instead

Given all of the above, there are three places to put the tonal content, and they trade off cleanly.

The working architecture for most teams is a hybrid: the model handles beds, foley, ambience, and non-pitched or pitch-agnostic layers, and the tonal line is either symbolic-then-rendered or played. Load a .scl into an instrument that respects it — Surge XT and Pianoteq both handle Scala tuning, and MTS-ESP-aware plugins take it live — and the pitch problem stops being a problem, because it moved to a layer where pitch is a number.

The hybrid has its own real costs, and I would rather state them than sell it. Setup is fiddly, and MTS-ESP support across a plugin folder is patchy enough that you will find at least one instrument that ignores it silently. Per-note MPE editing is slow work in every DAW I have used. And rendering symbolic output well requires someone with sound design taste, which is the skill the model was supposed to save you. What you get in exchange is determinism: the same MIDI and the same tuning table produce the same file every build, which is a requirement for shipping software and something no sampling-based generator gives you by default.

Stage seven: the gate at the end of the chain

Now the part that catches procurement teams, and it connects back to the architecture in a way that is not obvious.

The defining property of a unified model is that voice, sound effects, and music come out of one system through one endpoint. That is the same property that makes pitch un-addressable — a single latent space that does not privilege musical structure. It is also the property that determines your compliance surface, because regulation attaches to the output class and the model, not to your intent. If your service calls one endpoint for narration, foley, and score, the strictest obligation in that bundle governs the integration.

As of writing, the concrete constraints break down roughly as follows. In the EU, the AI Act's transparency provisions require synthetic audio to be marked in a machine-readable way and require disclosure where content imitates a real person. In the US, there is no single federal statute; the NO FAKES Act has been introduced to establish a federal right against unauthorized digital replicas of voice and likeness but was still pending as of writing, while state law — Tennessee's ELVIS Act among others — already reaches voice. China's deep synthesis and labelling measures require conspicuous labels plus embedded identifiers on synthetic media. C2PA Content Credentials are the emerging mechanism for carrying provenance in the file itself.

The questions to put to a vendor in writing, before anything ships:

• Does output carry a watermark or provenance manifest, is it retrievable, and does it survive your encoding pipeline?
• What are the commercial-use terms, and are outputs exclusive to you or potentially reproducible for another customer?
• Are your prompts and uploaded reference audio retained, and are they used for training? If you upload a session musician's performance as melody conditioning, you have moved their playing into someone else's logs — check the contract you have with them.
• Where is inference performed, and under which jurisdiction's data governance?
• Is there indemnification for third-party claims arising from output, and what does it exclude?

None of that is about microtuning. It applies to a four-bar temple bed with no voice in it, because you cannot show a regulator or a client that this particular request went through the instrumental part of a model that has no instrumental part.

How I'd decide

Named criteria, in the order I would weight them for a commercial deployment:

Pitch addressability. Is there any documented way to supply a tuning — Scala import, cents offsets, per-note control? If the answer is no, price the tool as a texture engine.

Conditioning fidelity. If audio conditioning exists, ask what representation it extracts. Chroma-class conditioning means microtonal content will not survive. Vendors that cannot answer this have told you something.

Stems and export. Stem count, sample rate, bit depth, loop metadata. Stems are what makes post-hoc repair possible at all.

Determinism. Does the same input reproduce the same output? Non-reproducible assets are a problem for build systems and for patching a shipped title.

License clarity over 12 months. Model the real cost, including regeneration churn — if you render eight takes per keeper, your per-asset cost is eight times the sticker rate. Check whether pricing is per-seat or per-render, and read the commercial-use clause rather than the pricing page summary.

Disclosure and provenance tooling, and data handling, as above.

Who it's wrong for, which is the criterion most evaluations skip.

Who this is for, and who should skip it

Buy the unified model if your pitched content is diatonic and Western, or if most of your audio budget goes to non-tonal material — ambience, impacts, UI, room tone, transitions. That is a large share of a game's audio and most of a podcast platform's needs, and this is where the technology is strong enough to change your staffing.

Keep the pitch layer in-house if you work in any non-12-EDO system, if your product depends on per-note expression, if you need bit-identical rebuilds, or if a client contract specifies tuning. In those cases the model is a texture supplier feeding a chain you control.

Skip it entirely, for now, if your deliverable is a solo lead vocal, or if your compliance posture cannot accommodate uncertainty about training data provenance.

What this piece does not answer

Three things, and I would rather name them than round them off.

First, nobody has published a microtuning-fidelity benchmark. There is no standard test that reports mean absolute cents deviation from a target scale across models, which means every claim here — including mine — rests on individual sessions rather than measurement anyone can replicate. You can build a crude version in an afternoon: render a sustained monophonic tone against a known target, analyze the fundamental, and report the deviation in cents across a few dozen renders. Do that before signing, and do it on your actual prompts.

Second, it is genuinely unresolved whether tuning gravity is a data problem or a representation problem. If it is data, a corpus with more non-Western and microtonal material moves the needle. If it is the latent representation, more data changes nothing and the fix is architectural. Watch for any vendor that exposes a tuning table as an input — that would be the signal that someone solved it rather than sidestepped it.

Third, the disclosure regime for purely instrumental synthetic audio is not settled. The voice provisions are clear in direction; whether a four-bar synthetic loop under a podcast intro carries the same marking obligation as a cloned voice is being worked out in implementation guidance, not in the statutes. Track the EU AI Act guidance and C2PA's audio specification work, and re-read your vendor's terms each renewal.

A model that can render anything and address nothing is a texture engine — price it, license it, and deploy it as one.