Measured 21 November 2026 on the full FLEURS test
Spanish speech-to-text accuracy benchmark
On all 908 clips of the Spanish FLEURS test, measured on 21 November 2026, KolWrite Oracle Max had a word error rate of 1.76% (95% interval 1.52% to 2.02%), the lowest of the 48 engines in the comparison, level with one published figure (1.76%); none is lower. Every transcript is published, so you can check the score.
Word error rate of every engine in the comparison
1.76% word error rate, 95% interval 1.52% to 2.02%KolWrite Oracle Max on all 908 clips of the Spanish FLEURS test, 21 November 2026
Oracle Max made 415 word errors (261 substitutions, 72 deletions, 82 insertions) over 23,525 reference words in 908 clips, 3.09 hours of speech. Resampling the 908 clips 10,000 times gives a 95% interval of 1.52% to 2.02%. 661 of the 908 clips were transcribed with no error.
Show all 48 enginesShow fewer
For 34 of the 47 engines the leaderboard publishes the transcript of every clip. We scored those files and ours with the same code, then resampled the 908 clips 10,000 times, using the same resampled clips on both sides of each comparison. The 95% interval of the difference lies entirely in favour of Oracle Max in 33 of the 34 comparisons; it includes zero, so the difference is not clear either way, in 1 (mistralai/Voxtral-Small-24B-2507, 1.87%); and it lies against Oracle Max in none. The other 13 engines publish no per-clip file for this test, so they can be compared only by their published figures. Set against Oracle Max's interval of 1.52% to 2.02%, 4 inside it (one with an equal figure) and 9 above it. Because their transcripts are not public, a figure inside our interval is not shown to be better or worse than ours.
All results
| Engine, as the leaderboard names it | Word error rate | 95% interval | Clips with no error | Oracle Max lead, points (95% interval) | Against Oracle Max |
|---|---|---|---|---|---|
| KolWrite Oracle Max | 1.76% | 1.52% to 2.02% | 661 | – | – |
| microsoft/azure-speech-09-2026 | 1.76% | published only | – | – | Equal figure |
| elevenlabs/scribe_v2 | 1.85% | published only | – | – | Figure inside our interval |
| mistralai/Voxtral-Small-24B-2507 | 1.87% | 1.63% to 2.13% | 639 | 0.11 (−0.08 to 0.29) | No clear difference |
| reson8/rumble-2 | 1.90% | published only | – | – | Figure inside our interval |
| openai/gpt-transcribe | 1.98% | published only | – | – | Figure inside our interval |
| yoob/yoob-asr-pro-v1 | 2.04% | published only | – | – | Figure above our interval |
| assemblyai/universal-3-5-pro | 2.04% | published only | – | – | Figure above our interval |
| salesforce-sovereign-intelligence/SOV-Transcribe-1 | 2.15% | 1.90% to 2.41% | 603 | 0.39 (0.22 to 0.55) | Lower for Oracle Max |
| assemblyai/universal-3-pro | 2.21% | published only | – | – | Figure above our interval |
| sprag/symphony | 2.26% | published only | – | – | Figure above our interval |
| soniox/stt-async-v5 | 2.28% | published only | – | – | Figure above our interval |
| openai/whisper-large-v3 | 2.30% | 2.04% to 2.58% | 584 | 0.54 (0.36 to 0.73) | Lower for Oracle Max |
| nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-FP8 | 2.33% | 2.05% to 2.64% | 604 | 0.57 (0.37 to 0.78) | Lower for Oracle Max |
| nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B | 2.34% | 2.05% to 2.65% | 609 | 0.57 (0.37 to 0.79) | Lower for Oracle Max |
| mistralai/Voxtral-Mini-3B-2507 | 2.51% | 2.24% to 2.78% | 571 | 0.74 (0.52 to 0.98) | Lower for Oracle Max |
| TheStageAI/thewhisper-large-v3-turbo | 2.58% | 2.32% to 2.85% | 559 | 0.82 (0.60 to 1.04) | Lower for Oracle Max |
| Qwen/Qwen3-ASR-1.7B-hf | 2.61% | 2.35% to 2.88% | 544 | 0.85 (0.64 to 1.06) | Lower for Oracle Max |
| speechmatics/enhanced | 2.61% | published only | – | – | Figure above our interval |
| nvidia/canary-1b-v2 | 2.63% | 2.35% to 2.91% | 550 | 0.86 (0.67 to 1.07) | Lower for Oracle Max |
| HojoAI/Hojo-ASR-Multi-V1 | 2.65% | 2.37% to 2.92% | 550 | 0.88 (0.66 to 1.10) | Lower for Oracle Max |
| FluidVoice-Labs/fluid-speech-pro | 2.65% | 2.37% to 2.94% | 549 | 0.88 (0.66 to 1.11) | Lower for Oracle Max |
| OpenLLM-France/Luciole-8B-Audio-1.0 | 2.66% | 2.37% to 2.95% | 564 | 0.89 (0.64 to 1.15) | Lower for Oracle Max |
| openai/whisper-large-v3-turbo | 2.73% | 2.44% to 3.02% | 549 | 0.96 (0.75 to 1.18) | Lower for Oracle Max |
| meta/muse-voice-transcribe | 2.74% | published only | – | – | Figure above our interval |
| mistralai/Voxtral-Mini-4B-Realtime-2602 | 2.80% | 2.45% to 3.21% | 554 | 1.04 (0.74 to 1.39) | Lower for Oracle Max |
| Reson8/Rumble-2-Flash | 2.82% | 2.52% to 3.12% | 538 | 1.05 (0.84 to 1.28) | Lower for Oracle Max |
| microsoft/Phi-4-multimodal-instruct | 2.86% | 2.56% to 3.16% | 553 | 1.10 (0.84 to 1.36) | Lower for Oracle Max |
| hf-audio/omniasr-llm-7b-v2 | 2.93% | 2.63% to 3.23% | 534 | 1.17 (0.92 to 1.42) | Lower for Oracle Max |
| nvidia/parakeet-tdt-0.6b-v3 (fast-gpu-asr) | 3.11% | published only | – | – | Figure above our interval |
| nvidia/parakeet-tdt-0.6b-v3 | 3.25% | 2.94% to 3.57% | 496 | 1.49 (1.22 to 1.77) | Lower for Oracle Max |
| CohereLabs/cohere-transcribe-03-2026 | 3.26% | 2.93% to 3.62% | 513 | 1.50 (1.20 to 1.81) | Lower for Oracle Max |
| modulate/multilingual | 3.31% | published only | – | – | Figure above our interval |
| FermionResearch/Phonon-2.5 | 3.40% | 3.06% to 3.76% | 488 | 1.64 (1.35 to 1.93) | Lower for Oracle Max |
| hf-audio/omniasr-llm-3b-v2 | 3.41% | 3.08% to 3.73% | 501 | 1.64 (1.37 to 1.92) | Lower for Oracle Max |
| OpenMOSS-Team/MOSS-Transcribe-Diarize | 3.54% | 3.19% to 3.91% | 464 | 1.78 (1.49 to 2.08) | Lower for Oracle Max |
| microsoft/VibeVoice-ASR-HF | 3.58% | 3.08% to 4.13% | 538 | 1.82 (1.38 to 2.32) | Lower for Oracle Max |
| ibm-granite/granite-speech-4.1-2b-nar | 3.61% | 3.27% to 3.95% | 462 | 1.84 (1.59 to 2.10) | Lower for Oracle Max |
| google/gemma-4-E4B-it | 3.71% | 3.26% to 4.27% | 502 | 1.95 (1.54 to 2.47) | Lower for Oracle Max |
| hf-audio/omniasr-llm-1b-v2 | 3.78% | 3.42% to 4.15% | 475 | 2.01 (1.71 to 2.32) | Lower for Oracle Max |
| hf-audio/omniasr-ctc-7b-v2 | 3.90% | 3.53% to 4.26% | 451 | 2.13 (1.83 to 2.46) | Lower for Oracle Max |
| hf-audio/omniasr-ctc-3b-v2 | 3.98% | 3.66% to 4.32% | 423 | 2.22 (1.93 to 2.52) | Lower for Oracle Max |
| Qwen/Qwen3-ASR-0.6B-hf | 4.14% | 3.79% to 4.49% | 416 | 2.38 (2.10 to 2.66) | Lower for Oracle Max |
| nvidia/nemotron-3.5-asr-streaming-0.6b | 4.23% | 3.87% to 4.61% | 425 | 2.46 (2.18 to 2.76) | Lower for Oracle Max |
| qvac/TranscribePsy-250M-EU-sub | 4.71% | 4.34% to 5.10% | 398 | 2.95 (2.63 to 3.28) | Lower for Oracle Max |
| hf-audio/omniasr-ctc-1b-v2 | 5.26% | 4.86% to 5.67% | 354 | 3.50 (3.16 to 3.84) | Lower for Oracle Max |
| hf-audio/omniasr-llm-300m-v2 | 5.28% | 4.88% to 5.69% | 369 | 3.52 (3.17 to 3.87) | Lower for Oracle Max |
| hf-audio/omniasr-ctc-300m-v2 | 9.37% | 8.86% to 9.88% | 151 | 7.61 (7.16 to 8.07) | Lower for Oracle Max |
Our scoring of the engines' per-clip files differs from the leaderboard's published figures by at most 0.03 percentage points on this set.
How it was measured
- Product. KolWrite cloud transcription, tier Oracle Max, called through the public KolWrite API on the production service on 21 November 2026, with the language set to Spanish, speaker separation off and no custom vocabulary. These are the calls a customer makes.
- Test data. The Spanish test split of FLEURS (Google, CC BY 4.0) in the copy the Open ASR Leaderboard scores (dataset hf-audio/open-asr-leaderboard-multilingual-datasets, configuration
es, revision2a6022fd), all 908 clips, 3.09 hours of read sentences. The leaderboard edited 165 of the 908 reference sentences (brackets and guillemets removed, missing text restored). Every engine in the table is scored against that text, and so is Oracle Max. Nothing was left out. All 908 clips returned text. - Scoring. The code of the Open ASR Leaderboard (huggingface/open_asr_leaderboard, commit
c4696193) in its multilingual mode: a basic multilingual text normaliser with numbers written as words, words joined where the two sides differ only by a space, then one error rate over all the words of all clips, aligned so that a compound written as two words on one side and one word on the other counts as no error. Before the run, our scoring code reproduced the leaderboard's figures for all 34 engines whose per-clip transcripts it publishes, to within 0.03 percentage points. - Statistics. Percentile bootstrap over clips, 10,000 resamples, random seed 20261121 (NumPy
default_rng). In each paired comparison, both engines are scored on the same resampled clips. - The other engines. The leaderboard's own results, in the versions it names, as shown on its page of 20 November 2026 (leaderboard, commit
7f8e5314). We did not run them. For 34 of them the leaderboard also publishes the transcript of every clip, which we rescored. - Pairing. In eleven of the engines' transcript files the clips are named by number, not by file name. We mapped them back to the clips and checked every reference sentence: six follow the order of the dataset and five the order of falling duration. In the latter, 26 clips share the same text and duration with others, so their assignment inside the group is assumed. That affects only which clip is paired with which, never an engine's own total.
- Publisher. ITANU Studios Ltd, the maker of KolWrite. We measured our own product, and we publish the files so that anyone can check the score.
What this does not show
- One test set of read or prepared speech, one speaker per clip. Recordings of meetings, phone calls, noise and overlapping speakers are different, and the order of the engines can differ on them.
- One tier, one language and one run. A transcription service does not return identical text on every run, so a repeat will land near 1.76%, inside or close to the interval, and not exactly on it. The interval covers the sample of clips, not the run-to-run variation.
- The leaderboard's edited references are the ones every engine is scored on. Scored against the original FLEURS text instead, the figures of every engine would differ.
- FLEURS has been public since 2022. Any large model may have met its sentences in training, Oracle Max and every other engine in the table alike.
- The leaderboard drops a clip from an API engine's row when that engine's call failed. We never drop a clip; none failed here.
- The other engines' figures are those of the versions and dates the leaderboard lists. Newer versions were not measured. Speed and price are not measured here.
- This page says nothing about how Oracle Max ranks in a language or on a test that is not listed on the benchmark overview.
Check it yourself
1. Download the per-clip file. oracle-max-spanish-fleurs-908-2026-11-21.csv has all 908 rows: the reference sentence, what Oracle Max returned, both texts after cleaning, and the error counts of each clip (id is the clip's identifier in the test set). SHA-256: c881a5c3e72d2a8a7406a0ed9951127a8128a8df96b12351f3b8c9fa2545f686. The reference sentences are from FLEURS (Conneau et al., 2022), © Google, used under CC BY 4.0 with the Open ASR Leaderboard's edits; our transcripts in the file may be reused under the same licence.
2. Recompute the score. score_wer.py (SHA-256 070513ff9c648f3ba8c9f44aa9950a0f82a8228ff6740722c81004a8257a0cf3) reads that file with Python's standard library, recounts every clip's errors from the cleaned texts and prints the figures:
python3 score_wer.py oracle-max-spanish-fleurs-908-2026-11-21.csv --compounds join-merge
Prints
clips: 908
reference words: 23525
word errors: 415
word error rate: 1.76%
95% interval: 1.52% to 2.02% (10000 resamples over clips, seed 20261121)
clips with no error: 661
recount of errors and words from the cleaned texts: 908 of 908 clips agree with the file
The interval is the published one: the script redraws the same 10,000 resamples with NumPy's random generator (seed 20261121), reimplemented in plain Python, so it needs no package. The option --compounds join-merge makes the recount treat a compound written as two words on one side and one on the other as no error, as the leaderboard does.
3. Run Oracle Max on your own account. Create an API key and add credits, then send the clips of the FLEURS Spanish test split one by one (about US$11.12 for all 908 at US$3.60 per audio hour). With the API of this site (model oracle_max, language es, no speaker labels, no custom vocabulary):
# 1. Upload one clip (WAV, under 4 MiB). The answer contains fileUrl.
curl -X POST https://app.kolwrite.com/upload \
-H "x-api-key: $KOLWRITE_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"filename\": \"clip.wav\", \"contentType\": \"audio/wav\", \"content\": \"$(base64 < clip.wav | tr -d '\n')\"}"
# 2. Start the job with that fileUrl. The answer contains jobId.
curl -X POST https://app.kolwrite.com/transcribe \
-H "x-api-key: $KOLWRITE_API_KEY" \
-H "Content-Type: application/json" \
-d '{"audio_url": "FILE_URL_FROM_STEP_1", "model": "oracle_max", "language": "es", "output_format": "json", "word_timestamps": false}'
# 3. Poll every 5 to 10 seconds until status is COMPLETED. Join result.segments[].text with spaces.
curl https://app.kolwrite.com/transcribe/JOB_ID -H "x-api-key: $KOLWRITE_API_KEY"
Our measurement went through the KolWrite platform API (https://api-prod.kolwrite.com, authenticated with an access key from the app's API access settings, documented here), where the same tier is called oracle-max. This is the exact request it sent for each clip:
curl -X POST https://api-prod.kolwrite.com/api/v1/uploads \
-H "Authorization: Bearer $KOLWRITE_KEY" -F "[email protected];type=audio/wav"
# the answer's data.upload_id goes into the job:
curl -X POST https://api-prod.kolwrite.com/api/v1/jobs \
-H "Authorization: Bearer $KOLWRITE_KEY" -H "Content-Type: application/json" \
-d '{"type": "transcription", "input": {"type": "upload_ref", "upload_id": "UPLOAD_ID"}, "profile": "oracle-max", "options": {"language": "es", "speaker_separation": {"mode": "none"}}}'
# poll GET /api/v1/jobs/JOB_ID until data.status is "completed"; the text is data.result.segments[].text
Where the data lives. Dataset: FLEURS, revision 2a6022fd. Leaderboard and its per-clip files: hf-audio/open_asr_leaderboard, commit 7f8e5314. Scoring code: huggingface/open_asr_leaderboard, commit c4696193.
Engine and product names belong to their owners and are used only to report the results the leaderboard published for them.
Questions and answers
Is this benchmark independent?
No. ITANU Studios Ltd, the maker of KolWrite, ran it on its own product. The other engines' figures are the leaderboard's, not ours. We publish every transcript and the scoring method so that anyone can check the Oracle Max score, and the leaderboard's own files let anyone check the rest.
Which languages does this site list?
The languages in which Oracle Max has the lowest word error rate on a named public test: Hebrew, Spanish, French, Arabic, German and Portuguese. Where it is level with the lowest, or below the totals a leaderboard publishes, the page says so. Other languages are not listed.
Will I get exactly the same figure if I run it myself?
Close to it, not exactly. A transcription service does not return identical text on every run, so a repeat should land near 1.76%, inside or near the interval.
Where do I report a mistake on this page?
Write to [email protected] with the clip id or the figure. We correct the page and the file and say what changed.