How ProsGrow Reached 7,200 Audio Hours per Hour with Whisper

The problem: recover capacity without changing the speech model

An archive-transcription service needs to process more audio while meeting a defined quality target. ProsGrow began with the public Whisper implementation used in MLPerf Inference v6.0 and reproduced its supported profile on an eight-GPU RTX PRO 6000 Blackwell node. The full-duration baseline delivered 291.771 audio samples/s.

We profiled engine batching behavior and found that the public profile was not optimal for this setup. A supported profile adjustment delivered 299.444 samples/s across two full-duration runs: 2.63% more throughput on the same eight GPUs. The selected profile also passed the benchmark’s accuracy threshold.

Baseline → ProsGrow tuned

The two tuned runs differed by only 0.123% of their mean. For conservative capacity planning, we use the lower observed result, 299.260 samples/s. The baseline has one full-duration measurement; the tuned profile has two.

What profiling revealed about engine batching

Average GPU utilization rose from 94.417% to 97.373% and 97.312% in the two selected runs. Together with the throughput results, this supports improved batching cadence as the explanation for recovered capacity. It is an inference from telemetry, not a claim about undocumented engine internals.

ProsGrow evaluated the complete pipeline, selected a supported engine profile, and validated it through full-duration repeats and separate accuracy checks. The public source, checkpoint, dataset, decoding behavior, LoadGen seeds, and evaluator stayed fixed. The same FP16 source checkpoint and NVFP4 decoder matrix-multiplication plugin remained in use; no source performance patch was active.

The trade-off: more throughput with slightly more active power

The first tuned full run increased average aggregate GPU power from 3,428 W to 3,508 W, while throughput increased by 2.69%. GPU-only energy per 1,000 source-audio hours moved slightly lower, from 0.4885 to 0.4868 kWh. The capacity gain did not require a lower measured energy efficiency, but the energy difference is small.

Quality remained an explicit gate. The tuned profile’s AccuracyOnly run covered all 1,633 samples and produced 2.131770% word error rate with the MLCommons evaluator, below the allowed 3.046429%. This establishes a passing result for the evaluated corpus; the local baseline did not receive a separate accuracy run in this comparison.

External context: comparable throughput with fewer GPUs

The report’s primary public Whisper comparison is the HPE eight-GPU result of 294.821 samples/s. Our 299.444 samples/s mean is 1.57% higher node throughput. Both use eight RTX PRO 6000 GPUs, while host and software details differ.

The audited public MLPerf v6.0 snapshot includes an HPE ten-GPU result of 297.661 samples/s. Our local eight-GPU mean was 0.60% higher, with 20% fewer accelerators. Dividing by GPU count gives 37.431 versus 29.766 samples/s/GPU, a 25.75% difference.

This is useful system-efficiency context. The engineering delta isolated on our own node is the 2.63% gain from engine-profile selection. The external comparison also includes host and software-stack differences, so its full per-GPU advantage cannot be attributed to that one change.

What this changes for a transcription service

On the evaluated corpus, the two-run mean corresponds to approximately 7,202 hours of source audio per wall hour, compared with 7,017 at baseline. That is about 185 additional source-audio hours processed per active node-hour, derived from the benchmark’s average clip duration. A service with a sustained archive backlog can use the gain to shorten processing windows on the same installed GPUs.

ProsGrow would qualify archive transcription against throughput, transcription quality, queue age, and data boundaries. Live transcription needs its own service contract: concurrent connections, partial-result behavior, finalization latency, and recovery. An Offline sample rate cannot determine live-stream capacity.

Speech cost at 10% paid demand

The September 15 capability report models $0.0001321 per sold audio minute using a $100,000 node, three-year amortization, 70% benchmark-to-service efficiency, and 10% paid demand. Its dated API price inputs are OpenAI hosted whisper-1 at $0.006/minute and Deepgram Nova-3 monolingual pre-recorded PAYG at $0.0043/minute. The modeled infrastructure figure is 97.80% and 96.93% lower, respectively.

The calculation converts measured capacity to approximately 30,247.5 sold audio minutes per calendar hour, then divides approximately $3.995 in hourly infrastructure cost by that capacity. Cost includes $100,000 ÷ (3 × 8,760 hours) in hardware amortization and electricity at $0.10/kWh, with assumed whole-node power of 1.5 kW idle and 5.5 kW active, weighted by 10% demand. Service efficiency and paid demand are separate assumptions.

Holding hourly cost and other assumptions fixed, the measured 2.63% throughput gain alone reduces unit cost by about 2.56%. The larger API-price gap comes from the modeled economics of private batch capacity and depends heavily on demand.

This is an infrastructure-to-price comparison, not established customer savings. It excludes maintenance, cooling/PUE, labor, networking, storage, support, redundancy, and SLA reserve. Local Whisper Large v3, hosted whisper-1, and Nova-3 differ in model and service features; equivalent-service quality was not established.

Granite API and live speech

The report also measures a separate Granite queued API service. Host placement and request batching raised throughput from 19,062.88 to 28,368.99 RTFx, a 48.82% gain against its original local API baseline. RTFx means audio seconds processed per wall second. The batch API evaluated 62,880 utterances with zero degenerate outputs and a worst WER of 3.4942%.

Granite’s separate async Offline result is 68,932.91 RTFx. The report’s live-ASR test sustained 5,568 WebSocket connections in three 24-second same-host loopback tests. These use different models, scoring and timing boundaries. Each headline occupies the node in a separate test; external traffic, long-running live operation and recovery still need validation.

Methodology and limitations

• System and stack: the August 20–21, 2026 tests used eight NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs and two AMD EPYC 9575F CPUs, TensorRT 10.14.1.48, TensorRT-LLM 1.3.0rc0, and CUDA 13.1. Public reference code and model/data hashes were pinned and checked.
• Workload and units: the MLPerf Whisper data contract repacks LibriSpeech into 1,633 mono, 16 kHz audio samples. Each sample incurs 30 seconds of padded Whisper compute; actual source duration averages 24.0505 seconds. Audio samples/s, generated tokens/s, and source-audio hours are different units.
• Performance validity: the baseline and both tuned headline runs met the 10-minute minimum and were LoadGen VALID. Short diagnostic runs selected candidates; they are not mixed with full-duration headline results. The 0.123% range covers two tuned repeats and is not a confidence interval.
• Accuracy and compliance: these are local MLPerf-derived measurements, not official MLPerf submissions or MLCommons-reviewed results. TEST01 performance verification passed. Direct output comparison found mismatches; the permitted fallback scored baseline and audit outputs at identical 2.168070% WER and redeemed that check. This was fallback redemption, not a byte-for-byte accuracy pass. The full AccuracyOnly WER remains 2.131770%.
• Production scope: power figures cover GPUs only, excluding the host and cooling. Audio-hour conversions use this corpus and assume a sustained queue. Customer accents, languages, noise, overlap, vocabulary, storage, networking, retries, utilization, and latency require their own measurements.