Voices

The sounds

The trouble starts with the inventory. The reconstruction calls for a set of sounds that no living language has kept together. There are three laryngeals: h₁, which I read as a plain [h]; h₂, a voiceless uvular fricative [χ], the scrape at the back of German Bach; and h₃, its voiced, lip-rounded partner [ɣʷ]. The stops at the back of the mouth come in three rows - palatal [c ɟ], plain [k ɡ] and labiovelar [kʷ ɡʷ] - and a labiovelar is one sound, the lips already rounding as the stop releases. Then there are the breathy-voiced stops [bʱ dʱ ɡʱ], resonants that carry a syllable all by themselves (like the l at the end of English bottle), a short vowel that turns up wherever a laryngeal sits between two consonants (so ph₂tḗr “father” comes out [pə.téːr]), and an accent made of pitch: the accented syllable is sung higher.

Every one of these exists somewhere. Hindi has the breathy stops and Arabic the uvular fricatives, Czech says syllabic r and l, Hungarian has palatal stops, and Swedish tells words apart by pitch. A speech engine trained on any one of those languages gets part of the inventory right and swaps in its nearest guess for the rest.

The phonetic values come from the handbooks: Fortson, Ringe, Beekes, Clackson and Byrd. Where they disagree I picked one reading and wrote the others down in the project's pronunciation notes. From there a converter turns written PIE into IPA by fixed rules, the same way every time, and everything that follows starts from its output.

Existing engines

The first attempt used eSpeak-NG, a formant synthesizer, which builds speech by rule out of a handful of resonances and (usefully) can be taught new phonemes. My PIE voice for it borrowed eSpeak's Hindi phoneme table, Arabic's pharyngeals and a pair of level tones. Measured, it did exactly what it was told: on h₂ówis the accented syllable sits at 98 Hz and the one after it at 82. Every sound landed where it belonged, in a voice nobody would ever mistake for a person.

eSpeak also has a nasty habit. Hand it a phoneme name it doesn't know and it drops the rest of the word, without a warning. The fix was to read every rendering back through eSpeak's own transcriber and compare the result, symbol by symbol, with the IPA I'd sent it.

The neural models had the opposite problem: they sound like people, and they can't say half the inventory. Kokoro-82M, the best open model that reads IPA directly, has no symbol for breathy voice, lip rounding, a syllabic resonant or a high tone, and it silently drops whatever it doesn't recognize. A bake-off put 19 of its voices and eSpeak through the hard sounds, each in several spellings - 634 clips in all. The Hindi voices came closest, since Hindi had already taught them breathy stops, but they still said the palatals as the affricates of Hindi च and ज. The commercial speech services accept IPA only within the sounds of each voice's own language.

Another route kept eSpeak's correct sounds and re-voiced them with a voice converter, which changes who seems to be speaking and leaves what's being said alone. The open converters kept either the melody or the laryngeals, and seldom both.

A vocal tract in software

VocalTractLab, Peter Birkholz's articulatory synthesizer, is something close to a marvel: a three-dimensional model of a human vocal tract. Its tongue, lips, jaw and velum move toward target positions, with a model of the vocal folds beneath them and a pair of lungs beneath those. At every instant the program turns the shape of the tract into a branching tube of air and computes the sound from pressure and airflow, the way a real throat does. It has no recordings to draw on, which means it can make any sound a human vocal tract can make - including ones no living language uses.

You write speech for it as a gestural score, a set of parallel tiers for the vowel, the lips, the tongue tip, the tongue body, the velum, the glottis, the pitch and the lungs. Each tier is a row of gestures, and each gesture is a target the articulator moves toward with its own time constant and then holds, so neighbouring sounds overlap and blend the way they do in a mouth. VocalTractLab ships with a converter for German. I wrote my own compiler, pie_vtl, which turns PIE IPA straight into a gestural score and knows every sound below.

p t k

A closure, with the glottis set for short-lag voicing: unaspirated, as the reconstruction requires.

c ɟ

The tongue body raised against the hard palate, a closure shape added to each voice and tuned on its own anatomy.

kʷ ɡʷ

The velar closure, with the lips rounding into the release.

bʱ dʱ ɡʱ

A voiced closure released into murmur: the vocal folds vibrating with a gap left open behind them. The size of the gap was found by bisection, until a held murmured vowel sat a few decibels below the modal one.

χ

The tongue body at the uvula and the glottis wide, with a layer of noise shaped to a uvular fricative's spectrum laid over it, since the simulation alone makes little friction that far back.

h

An open glottis and breath, the noise filtered through VocalTractLab's own transfer function for the neighbouring vowel, so every h takes its vowel's colour.

r̩ l̩

A schwa held under the tongue-tip shape of [ɹ] or [l], long enough to carry the accent.

r

VocalTractLab's tongue goes where it is sent and cannot be blown into a trill, so a trilled r is built from contacts 40 ms apart, twenty-five a second, the rate of a real trill.

Then came timing and pitch. The simulated articulators lag behind the score, and stop releases were landing 30 ms late until the compiler started moving the oral tiers 30 ms early. The accent is a rise on the pitch tier of about three semitones, as measured. Pitch drifts down across each line and drops two semitones at the end; a question finishes on a three-semitone rise instead. The male voice is VocalTractLab's JD3 speaker. The female voice is W02, from VocalTractLab 2.4, which has the newer model of the vocal folds.

There were ten versions, and each was measured before anyone listened to it. Every render has to pass a gate: the accent must lift at least 2.5 semitones, at least 97 percent of syllable nuclei must be voiced, no vowel may fall more than 8 dB below the clip's median, and every χ and h has to carry its share of energy between 2 and 8 kHz.

Two human voices

The simulated voices are exact, and they sound faintly like glass. To give them a human timbre, each render went through a speech-to-speech model, which re-voices a recording in another speaker's voice while keeping the timing to the millisecond. Every line went through twice, once into a male voice and once into a female one. Not every sound survived the trip. Converters treat a short burst of voiceless noise as background and throw it away, so a word-final h, and sometimes a χ, could simply vanish; the converter also smoothed away the flutter of the trill and the breath after bʰ dʰ gʰ.

Because the timing survives conversion, the repairs can be exact - the compiler records where every phone begins and ends. restore.py finds each h and χ the converter weakened and rebuilds it from noise shaped by the converted voice's own neighbouring vowel. In the Dawn hymn, the h at the end of the middle túH went from 0.00 to 0.57 of its energy in the 2–8 kHz band. retrill.py re-cuts the trill's contacts on the timing of each r, and murmur.py puts the breath back onto the release of every breathy stop.

One last gate set aside 746 clips with a weak h or χ, a voiceless syllable nucleus or a flat accent. The remaining 3,369 clips became the training set, with 123 more held out for validation: 3.8 hours in the male voice and 4.3 in the female, at 24 kHz.

Teaching Kokoro

Kokoro-82M is an open text-to-speech model with 82 million parameters, released under the Apache licence and descended from StyleTTS 2. It reads phonemes rather than letters, which is what made it possible to teach it a language with no living speakers. Its symbol table also happened to have four unused slots. Those became breathy voice ʱ, rounding ʷ, the syllabic mark and the acute accent, so the model reads the pitch accent as just another symbol. Every training label is the converter's IPA for its clip, character for character, so the model learned exactly the alphabet it's sent today.

Training followed the kikiri-tts recipe in two stages of ten epochs each, on a single GPU. The first stage teaches the model to rebuild the corpus audio. Its best epoch by validation loss started the second stage, which trains duration, pitch and style - the parts that make speech sound spoken. Each candidate epoch from the second stage read the held-out lines, was scored against the real clips on spectral distance, pitch error and duration, and then got a listen. The tenth epoch became the voices, pie_m and pie_f, each a voice pack drawn from 200 of its speaker's clips.

The voices at work

What's left is a short pipeline. A line of PIE goes through the converter to IPA and on into Kokoro, which hands back the audio along with the length of every phoneme. Those lengths place each word in time, which is how the hymn to Dawn on the front page lights each word as it's sung. It all runs on an ordinary processor, and it has read every entry in the dictionary aloud, in both voices.

Hear the hymn to Dawn Open the dictionary