Speech Inference: Seconds, Samples and Mel Features

Follow the recording through three different units

A speech-to-text offer can describe input in audio seconds, samples or frames and still leave the work unclear. A recording's duration does not identify the representation passed to the model. A feature-array size does not identify the text eventually returned. Compare the transformations between those quantities before interpreting a capacity or price claim.

This explanation uses WhisperFeatureExtractor's constructor defaults in Hugging Face Transformers v4.57.1: 16,000 samples per second, a 30-second target window, a 160-sample hop and 80 mel bins. These are named implementation defaults, rather than a version recommendation or specifications for every Whisper checkpoint. The example stipulates a matching 80-bin model configuration; an actual offer must identify its checkpoint and frontend configuration together.

For mono audio, one waveform sample supplies one amplitude value at a point on the recording's time axis. A rate of 16,000 samples per second means 16,000 such values per second. The frontend derives a time-frequency representation through short-time spectral analysis, mel filtering and logarithmic processing. Its time frames have a separate axis of mel features. Those bins are frequency-related features within a frame, rather than additional recordings or text tokens.

References: [H1][H2]

ASR-422: 35 recording seconds become 60 padded window seconds

Fictional ASR-422 supplies two already-mono recordings at an established 16,000 samples per second. Recording A lasts 10 seconds and B lasts 25 seconds. The example uses the named frontend's maximum-length padding to 30 seconds, without a different maximum or extra padding to a multiple. Neither recording needs truncation. These are invented inputs and arithmetic, with no audio processing or model execution.

A contains 10 x 16,000 = 160,000 raw waveform samples. B contains 25 x 16,000 = 400,000. Together they supply 560,000 samples over 35 recording seconds. That duration includes whatever the recordings contain; it does not establish 35 seconds of voiced speech.

Each 30-second target contains 30 x 16,000 = 480,000 waveform sample positions. A therefore adds 480,000 - 160,000 = 320,000 padding positions. B adds 480,000 - 400,000 = 80,000. Across both, 560,000 supplied samples plus 400,000 padding positions equal 960,000 represented waveform sample positions.

The two target windows account for 60 seconds at the established sample rate. They do not create 60 seconds of recorded speech. A's supplied duration is still 10 seconds, even though this frontend path represents it in a 30-second window. Keeping supplied duration and target-window duration separate resolves that apparent contradiction.

References: [H2]

The hop sets time spacing; mel bins form another axis

A hop is the advance between successive spectral analysis positions. Here, 160 samples / 16,000 samples per second = 0.01 second, or 10 milliseconds. This spacing is distinct from the analysis window: the tagged implementation uses a 400-sample Fourier-analysis window. Successive windows overlap, so a frame is not an independent, nonoverlapping 160-sample recording.

The constructor derives its sample-window size from duration times sampling rate and its maximum frame count by integer division by the hop. In this tagged extraction path, the final spectral time frame is removed. For ASR-422's divisible, padded length, the resulting time axis is 480,000 / 160 = 3,000 frames per recording.

Each time frame carries 80 mel-bin values. One recording therefore produces an 80 x 3,000 feature array, containing 240,000 scalar positions. The pair has shape 2 x 80 x 3,000 and contains 480,000 feature scalar positions. The leading two counts recordings, 80 counts features per time frame and 3,000 counts time frames per recording.

The number 480,000 now appears twice with different meanings: waveform sample positions for one padded recording, and feature scalar positions across both recordings. Their numerical equality is coincidental. It is not a one-to-one conversion, a common billing unit or evidence that those two representations occupy the same memory. Always retain both the unit and the population attached to a count.

References: [H2]

Check what preparation retained and what execution actually skipped

The tagged path pads the waveform before spectral feature extraction. Added waveform zeros do not imply a matching block of zero-valued log-mel features. The representation has undergone a transformation, so its feature values and its position count are different questions.

An optionally returned attention mask describes positions for downstream interpretation. Its presence does not establish that padding storage or spectral computation was avoided. ASR-422 supplies no measured time, byte count or complete runtime-memory footprint. Dividing raw samples by represented samples would describe a layout fraction, rather than accelerator utilization or an achievable speedup. The related padding guide examines the corresponding distinction for text inputs and request boundaries.

Input preparation must also preserve the intended content. Mono is a supplied condition here, rather than permission to discard a stereo channel. The extractor checks a supplied sampling-rate declaration against its expected rate; changing that declaration does not resample the waveform. If an offer starts with different audio, its actual preparation path must explain how it reaches the input being compared.

Truncation is a separate change: it removes samples beyond the retained boundary. It cannot be counted as equivalent padding efficiency when the required recording is no longer complete. For longer recordings, establish the actual long-form pipeline, including segmentation, any overlap and the way segment outputs are combined. A 30-second frontend window alone establishes no universal 30-second transcription-service limit.

References: [H1][H2]

A mel frame is not a transcript token

The Transformers v4.57.1 processor reference separates audio feature extraction from text tokenization and decoding predicted token IDs into text. The audio representation feeds the speech task; the returned transcript belongs to a different output representation.

ASR-422's 3,000 time frames per recording therefore do not imply 3,000 generated text tokens. Nor do its 80 mel bins describe vocabulary size. Recording duration and feature shape alone supply neither output word counts nor transcript correctness.

This matters at the completion boundary. A report that stops when features are ready has finished preprocessing, rather than delivered the requested transcription. A report that stops at predicted IDs leaves text decoding outside its interval. If the buyer requires a complete transcript with particular timestamps or formatting, the evidence must reach that agreed deliverable. Quality needs an assessment of the returned result against the same acceptance requirement; a frontend shape cannot supply it.

References: [H1]

Compare the two offers at the same completed transcription

Consider two expressly fictional offers for ASR-422. A describes the input as 35 audio seconds. B describes 480,000 frontend feature scalar positions. Both stipulate the same recordings, the same compatible checkpoint and the frontend path above. Both are intended to return a complete English transcription of each recording under the same acceptance criteria. Neither supplies a price or performance result.

Their headlines count different stages of the same stipulated task. B's smaller number than the pair's 960,000 waveform sample positions establishes no reduction in total work. A's 35-second label likewise leaves the 60 seconds of padded frontend windows invisible. Neither label tells the buyer how many acceptable transcripts will complete within a response-time requirement.

For real offers, reconcile the actual checkpoint revision, frontend settings and executed software implementation with the shared recordings. Keep the intended language and transcription task fixed; a translated output would answer a different requirement. If either offer changes segmentation, retained audio or requested output, resolve that difference before treating its performance as comparable.

Then obtain elapsed-time and outcome evidence through the same completed deliverable. State whether input loading, required preprocessing, queueing, generation, text decoding and required output formatting are included. A measurement beginning at precomputed features omits the waveform preparation illustrated here. A measurement excluding queueing also leaves a different part of the buyer's waiting time unanswered.

Keep hardware, execution precision, batch composition and concurrent load attached to each result. Report failures, incomplete transcripts and results that miss the agreed quality or response-time criteria alongside successful completions. Matching feature shapes cannot make those outcomes equivalent, and results from materially different complete systems cannot isolate frontend padding as the cause of a difference.

Only after that comparison can an actual commercial offer be interpreted against accepted output. Establish what its audio-second or other meter counts, which preparation is included and how failed or repeated work is charged. ASR-422 provides no such terms. It supports a precise workload description, while leaving cost, capacity and quality undecided until matching evidence exists. The related GPU benchmark guide covers that broader complete-system comparison.

Take separately selected equipment to a physical deployment inquiry

If the selected speech-inference system also needs equipment receiving, rack integration, onsite installation or IT cabling, Leviathan's Complete Physical Deployment service and related project inquiry provide a way to discuss that physical scope. Bring a nonconfidential equipment list, site, rack arrangement, required connections and target window.

This educational frontend explanation is not an ASR hosting or software-delivery offer. Speech frontend and model integration, training, tuning, evaluation and software commissioning remain with the appointed software team. The physical inquiry concerns separately selected equipment and the agreed installation handoff.

References: [L1]