Best AI voice cloning in 2026: how to clone your voice

I write about markets and AI for a living. Most of what I publish never becomes a video, because making one means setting up a camera and reading my own writing out loud, and I would rather spend those hours on the research. I wanted to feed in the text and get narration in my voice. A paid clone was the obvious first try and it was not good enough, so I trained my own.

The short answer

Fine-tune CosyVoice 3 or VibeVoice. Those are the only two that captured both my voice and the way I speak.

Press play on any row. Ranked on one question only: does it sound like me. Several of these are excellent synthesis that happens to be someone else. Every clip reads the identical script, and each zero-shot model cloned from the same 20 seconds cut out of the recording the fine-tunes trained on.

This table merges three blind rounds run over two days: an early screening round, the reader test below, and the ranking session near the end. Each round covered part of the field, and the ranks here are my ordering across all three. Read adjacent rows as ties.

Take the test

Try it yourself: one of these five is a real human

This is a reader exercise, not one of my ranking sessions. I assembled it from the same clips under the same conditions so you can try the thing I did: identical script, loudness matched, labels hidden. The session that actually produced the rankings is near the end of the article, and it used a different five files.

D is me. If you picked wrong, that is the finding: forty five minutes of phone audio was enough to make a synthetic version hard to separate from the real one.

The setup

How to clone your voice, in one pass

1. Record 30 to 45 minutes in one session. One room, one mic, one sitting. Mine was an iPhone with no interface and no treated room. Do not pool old clips from different days: that was the single biggest quality jump in my testing.
2. Convert to 24 kHz mono and skip enhancement. ffmpeg -i src.mov -ar 24000 -ac 1 -c:a pcm_s16le audio.wav. I tested an enhanced version of the same recording and it scored worse.
3. Cut into 4 to 12 second clips on sentence boundaries, then check the clip manifest against your recording length. My first chunker dropped over-long segments instead of splitting them and silently lost 27 of 52 minutes.
4. Transcribe with openai-whisper, model large-v3, running on the same GPU. Each training row is a (text, audio) pair.
5. Rent one 24 GB GPU and prove it is real before installing anything: python -c "import torch; assert torch.cuda.is_available()". A silent fall back to CPU will hide a dead machine for a whole run.
6. Read the project's Dockerfile. It is the only authoritative record of the environment. For CosyVoice the load-bearing details are Python 3.10 and pynini from conda-forge.
7. Train, saving a checkpoint per epoch. About 22 seconds per epoch for 45 minutes of audio. Do not assume the last checkpoint is the best one: mine peaked at epoch 4 of 31.
8. Render three takes per checkpoint, then transcribe each one and diff it against your script. Generation is stochastic and most of these models take no seed, so takes differ in how much of the script they actually speak. I measured 25%, 54% and 90% word coverage across three renders of one input. Keep the take with the highest coverage and re-render anything below about 90%.
9. Rank blind. Normalise loudness, pad to one duration, hide the labels until after you commit.

Clone my voice with an open-source TTS model. Work through these steps in
order and verify each one before moving on. Tell me when something fails; do
not silently retry.

=== 0. SAFETY: THIS PROMPT CONTAINS NO SECRETS. DO NOT ADD ANY. ===

Do not paste an API key, token, or password into this prompt or into any chat
window. Everything below authenticates interactively or through a file the
agent never prints.

Agent: never echo, log, or repeat back the contents of a credential file. If
you need to confirm a token loaded, print its LENGTH, not its value.

=== 0b. TOOLING YOU NEED FIRST ===

RunPod (rents the GPU). Connect the MCP server so the agent can create and
destroy pods:

    claude mcp add --transport http runpod -s user https://mcp.getrunpod.io/

This uses a browser OAuth flow, so no API key is stored on disk and nothing
sensitive passes through this prompt. Then add an ACCOUNT-LEVEL SSH key at
runpod.io/console/user/settings. The SSH proxy authenticates against that key,
not the pod's PUBLIC_KEY, and without it every connection fails with
"Permission denied (publickey)".

Hugging Face (hosts the base models, your corpus, and your trained weights).
Create a token at huggingface.co/settings/tokens. Prefer a FINE-GRAINED token
scoped to only the repos you will create, with write access. A global write
token can delete every repo you own, and an agent will be handling it.

Log in INTERACTIVELY so the token never lands in shell history or in a process
argument list:

    huggingface-cli login

To place it on a pod, read it from stdin and write it to a file. Never pass a
token as a command-line flag:

    read -rs HF_TOKEN && printf '%s' "$HF_TOKEN" > /root/.hf_token && unset HF_TOKEN
    chmod 600 /root/.hf_token

=== 1. RECORD ===

30-45 minutes in ONE session. One room, one mic, one sitting. A phone is fine.
Do not pool clips from different days: mixed sessions teach the model the room
differences along with the voice.
Scraped audio does not scale: 14.5 min of clips pulled from old videos tied
2.5 min of the same. One clean session of 30-45 min is what actually worked.
No noise reduction, no enhancement. Enhanced audio scored worse.

=== 2. BUILD THE CORPUS ===

    ffmpeg -i SOURCE.mov -ar 24000 -ac 1 -c:a pcm_s16le audio_24k.wav

Chunk into 4-12s clips: split on . ! ?, merge forward until each clip is >= 4s,
force-split anything over 12s at a comma or a gap > 0.35s.
NEVER discard a segment for being too long.

Print a manifest and check it against the source length:

    clips=360 total_min=45.36 min=4.00 mean=7.56 max=12.00

STOP if total_min is not within ~10% of what I recorded. An early version of my
chunker dropped over-long segments instead of splitting them and silently lost
27 of 52 minutes with no error.

Cut restarts, coughs, off-mic lines and dead air. A bad clip hurts more than a
missing one.

=== 3. TRANSCRIBE ===

openai-whisper, model large-v3, on the GPU. Write train.jsonl with one row per
clip: {"text": ..., "audio": ...}
Correcting the transcripts afterwards is OPTIONAL. I tested it and it did not
improve output quality. Skip unless I ask.

=== 4. PIN THE CORPUS TO HUGGING FACE, BEFORE ANY TRAINING ===

    huggingface-cli repo create my-voice-corpus --type dataset --private
    # upload the clips + train.jsonl

This is not bookkeeping. The corpus must be ONE artifact that every training run
fetches, or you will end up comparing models trained on different data without
realising it. Every training script should fetch the pin and assert the clip
count before starting.

=== 5. START A POD ===

Use the RunPod MCP tools. Ask for a 24GB GPU (RTX 4090 is enough for a 0.5B
model). Community 4090 is about $0.34/hr, secure about $0.74/hr.

CUDA MUST BE >= 12.8. create-pod returns cudaVersion: if it is 12.2 or 12.4,
DELETE the pod and create another. Those hosts crash-loop the container while
the API still reports RUNNING. Each rejected pod costs under a cent.

Then prove the GPU is real before installing anything:

    python -c "import torch; \
      print('GPUCHECK', torch.cuda.is_available(), \
      (torch.randn(999,999,device='cuda') @ torch.randn(999,999,device='cuda')).sum().item())"

If that fails, destroy the pod and get another. Never let training fall back to
CPU: a silent CPU fallback hid a dead GPU from me for an entire run.

=== 6. BUILD THE ENVIRONMENT ===

FIND THE PROJECT'S docker/Dockerfile AND FOLLOW IT. It is the only authoritative
record of the environment. Guessing at it cost me many hours.

For CosyVoice 3 specifically:

    # the image may not ship conda at all; install miniconda if `conda` is missing
    conda create -y -n cosyvoice python=3.10 --override-channels -c conda-forge
    conda install -y -n cosyvoice --override-channels -c conda-forge pynini==2.1.5
    pip install "setuptools<81" wheel
    pip install torch==2.3.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
    pip install -r requirements.txt          # hold out openai-whisper
    pip install openai-whisper --no-build-isolation
    pip install huggingface_hub hf_transfer

Four traps, all of which fail SILENTLY or misleadingly:
  - `conda activate` does nothing in a non-interactive shell. Use absolute
    paths: /opt/conda/envs/NAME/bin/python. Otherwise your verification passes
    against the WRONG interpreter.
  - Anaconda default channels now need a ToS accept that fails unattended.
    Use --override-channels -c conda-forge.
  - openai-whisper fails to build inside isolation and aborts the ENTIRE
    requirements install, so nothing else lands.
  - Some images export HF_HUB_ENABLE_HF_TRANSFER=1 without shipping
    hf_transfer, so every Hugging Face download raises until you install it.

Assert every import before training:

    /opt/conda/envs/cosyvoice/bin/python -c \
      "import torch,torchaudio,hyperpyyaml,whisper,pkg_resources; print('DEPS_OK')"

=== 7. TRAIN ===

Save a checkpoint EVERY epoch. Expect ~22 seconds per epoch for a 45-minute
corpus on a 4090, so roughly 11 minutes total.

Checkpoints are ~2GB each, so prune to a sparse ladder as you go or a 150GB
disk fills at about epoch 36.

Do NOT assume the last checkpoint is the best. Audition early ones. Mine peaked at epoch 4 of 31.
Epoch count scales INVERSELY with corpus size: one epoch on 45 min is ~3x the
gradient steps of one epoch on 15 min, so a bigger corpus peaks at a LOWER
epoch number.

=== 8. PUSH EVERY CHECKPOINT TO HUGGING FACE ===

    huggingface-cli repo create my-voice-model --private
    # upload each kept checkpoint, winner first

NON-NEGOTIABLE, and do it BEFORE destroying the pod. Verify by reading the file
sizes back off Hugging Face.

A stopped RunPod pod is NOT a saved pod. /workspace is container overlay
storage, not a network volume. I have seen both failure modes in one hour: a
stopped pod that could not restart ("not enough free GPUs on the host"), and one
that restarted on a different host with /workspace completely empty.

=== 9. RENDER, 3+ TAKES PER CHECKPOINT ===

Generation is stochastic and most of these models take NO seed, so one take is
a lottery draw rather than a measurement.

Render at least 3. Do NOT pick by duration: in both cases I measured, the
shortest takes were the ones missing words, because truncation shortens a take.
Transcribe each render, diff it against the input text, and keep the highest
word coverage. Treat a wide spread in coverage as checkpoint instability rather
than as bad quality.

If the inference call returns a GENERATOR, consume ALL of it and concatenate.
CosyVoice yields one segment per sentence group; taking only the first silently
truncated a 29.5s clip to 19.7s for me, with no error.

Verify completeness: transcribe each render and diff it against the input text.
Expect >90% word coverage.

=== 10. BLIND TEST ===

Normalise every candidate to -16 LUFS, pad to one duration, and force an
identical byte size. Loudness, length and file size all leak which file is
which.

Shuffle the labels, seal the mapping, and do not reveal it until after I have
ranked. Keep sets to 5-6 files.

Balance the arms: do not put 4 checkpoints of one model against 1 of another.
One bad render sinks a lone entry.

=== 11. TEAR DOWN ===

Once checkpoints are pushed AND verified by read-back, TERMINATE the pod. Do
not stop it. Stopping still bills for storage and may lose the disk anyway.

=== REPORT BACK ===

Manifest numbers, per-epoch loss, the checkpoint list, take durations and word
coverage per render. Name what failed.

What mattered

Two levers moved the result. The rest were a wash or worse.

One clean recording beat everything I scraped

I started by scraping audio out of videos I had already published, because it was free and it was there. My first clone came from 2 minutes 33 seconds of it and was good enough to be my keeper for weeks. Here it is:

Scraping more of it stopped helping quickly. I built a second corpus of 14 minutes 28 seconds from the same videos and it tied with the 2.5 minute version in a blind ranking.

Then I sat down and recorded 45 minutes in one session, one room, one microphone, one sitting. That corpus produced both of the winners at the top of this page.

Record properly instead of scraping. Clips pulled from videos shot across different rooms and days carry all those rooms into the model. Forty five minutes recorded on purpose is a couple of hours of your time and it is the single biggest quality jump I found.

The peak arrives early

With one model I extended a run from 3200 to 6400 steps because the curve looked like it was still climbing. The first 3200 were the whole gain. My CosyVoice keeper peaked even earlier, at epoch 4 of 31, so plan on auditioning early checkpoints rather than training longer.

Blind test 2

The ranking that picked the winners

A different five files from the test at the top of this page: both ElevenLabs voices, both of my fine-tunes, and the real recording. This is the set that decided the ranking, and it includes both paid clones so the comparison is against the service at its best and its worst.

Read C and D together. I called the fine-tuned model "honestly best" and, separately, flagged the real recording as "probably legit me." I did identify the human. I also rated a synthetic clone at the top of the set while doing it. Both paid clones landed below both fine-tunes.

That is the practical bar for narration, and 45 minutes of phone audio cleared it.

Cost

What each option actually costs

The whole project, including every failed experiment and dozens of training runs, came to about $17 of GPU time. That is less than one month of the cheapest tier that supports professional voice cloning.

I did not provision or run any of this infrastructure by hand. I described what I wanted to an AI coding agent, it stood up the environment, rented the GPU, ran the training, and shut the machine down when it was done. The plan above is the instruction set, and everything it needs to get right is already in it.

The cost is a few dollars of rented GPU and the time it takes you to record.

One honest asterisk on that comparison: the $22 a month buys inference too, and self-hosting does not. Training is a one-off, but every minute of narration afterwards needs a GPU. At my volume that is pennies of pod time per batch. If you are generating a hundred minutes a month you should price it against the tier that covers it, not against the training run.

Generalisation

Does it hold up on text it has never seen?

A clone that only works on the paragraph you tested is not useful. This is 124 words the model never saw in training, including finance jargon it has no special reason to pronounce correctly.

Word coverage across three renders came back at 96%, 91% and 93% when transcribed and diffed against the source, with the residual differences being transcription artifacts rather than missing speech.

Before you ship it

Licence, consent, and the things that decide this in practice

Check the licence before you pick a winner

"You own the weights" in the cost table is doing more work than it looks. You own the adapter you trained, on top of a base model whose terms you did not write. Checked on Hugging Face, 24 August 2026:

Two of these want reading rather than glancing. IndexTTS-2 ships a LICENSE.txt headed "bilibili Model Use License Agreement", with its own definitions of derivative works and acceptance-on-use terms. XTTS-v2 is Coqui Public Model License 1.0.0, which permits non-commercial use only, and that restriction covers the model's outputs as well as its weights. The model that came last is also the one you are least free to publish from.

One provenance note: I trained on vibevoice/VibeVoice-1.5B, a community copy, and Microsoft's first-party microsoft/VibeVoice-1.5B is also live and also MIT. Prefer the first-party repo, and read the licence on the specific checkpoint you pull, since a mirror can carry different terms than its source and terms change between releases.

Clone your own voice, and say that it is synthetic

Everything here is my voice, my recording, my consent. Cloning someone else's without permission is a legal problem in a growing number of places: Tennessee's ELVIS Act covers voice explicitly, right-of-publicity law applies in many US states, and the EU AI Act carries transparency obligations for synthetic media. None of that is legal advice, and all of it is worth ten minutes before you point this at a voice that is not yours. Label synthetic narration as synthetic.

Where this test stops

Seven systems is what I ran, and the field is bigger. F5-TTS, Fish Speech, Higgs Audio, and Sesame CSM went untried. F5-TTS is the one I would run next, because it fine-tunes and therefore competes in the same category as the winners rather than the zero-shot group. Length is the other open question: my generalisation test runs 124 words, the use case is ten-minute narration, and long-form drift is where these models tend to struggle.

Versions

Text to speech moves monthly, so these results are pinned to what I ran in August 2026: FunAudioLLM/Fun-CosyVoice3-0.5B-2512, vibevoice/VibeVoice-1.5B, IndexTeam/IndexTTS-2, Qwen/Qwen3-TTS-12Hz-1.7B-Base, coqui/XTTS-v2, and chatterbox-tts from PyPI. Assume a model released after this article behaves differently, and re-run the test rather than trusting the table.

What I would tell someone starting today

Fine-tune. Every zero-shot model I tried got half of it and missed the other half, and it split evenly: two landed my voice and missed the rhythm, two landed the rhythm and missed the voice. None got both.

Record one clean session before you touch a model. Forty five minutes from a phone in a quiet room beat everything I scavenged.

Then rank blind. Every time I evaluated with the labels visible, I confirmed what I already believed.