AudioPreprocessing

org.llm4s.speech.processing.AudioPreprocessing

Functional audio preprocessing utilities. These are pure transformations described as functions that return either errors or processed audio.

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any
Self type

Members list

Value members

Concrete methods

def resamplePcm16(bytes: Array[Byte], source: AudioMeta, targetRate: Int): Result[(Array[Byte], AudioMeta)]

Resample signed little-endian PCM (8, 16, 24 or 32 bit, despite the name) to targetRate using Java Sound.

Resample signed little-endian PCM (8, 16, 24 or 32 bit, despite the name) to targetRate using Java Sound.

The contract:

  • '''Arguments are checked first.''' The target rate and the source's sample rate must be between MinSampleRate and MaxSampleRate Hz, the channel count between 1 and MaxChannels, and the bit depth a multiple of 8 from 8 to 32. Anything else is a Left(ValidationError) naming the field (targetRate, source.sampleRate, source.numChannels or source.bitDepth), and nothing reaches Java Sound.
  • '''The output has exactly round(frames * targetRate / sourceRate) frames''' (half up), where frames is the number of whole frames in bytes. A trailing partial frame is ignored, as it is by toMono and trimSilence. Java Sound's converter delays the signal by its filter latency and appends the same number of frames as a tail; the length fit drops that tail, so the output is the source delayed by a fraction of a millisecond (at most 0.3 ms for 24 to 16, 16 to 24, 44.1 to 16, 16 to 48, 8 to 48, 48 to 8 and 22.05 to 44.1 kHz, measured on JDK 21) and the source's final fraction of a millisecond is not in it.
  • '''Empty input gives empty output''', and so does input too short to make a single output frame.
  • '''Equal rates return a copy of the whole frames''' without converting.
  • '''An output larger than MaxOutputBytes (256 MiB) is a Left(ValidationError) on targetRate''', checked from the expected frame count before anything is allocated. The output is then written into one array of exactly the expected size.
  • '''A converter that delivers fewer than the expected frames''' (by more than MaxShortfallFrames, or no frames at all) is a Left(ProcessingError); a shortfall within that is padded with silence.
  • '''It cannot loop without making progress:''' reading stops at the end of the stream, at a read that returns no bytes, and once the expected number of bytes has been read, whichever comes first.

Value parameters

bytes

signed little-endian PCM

source

the sample rate, channel count and bit depth of bytes

targetRate

the rate to convert to, in Hz

Attributes

def standardizeForSTT(bytes: Array[Byte], meta: AudioMeta, targetRate: Int): Result[(Array[Byte], AudioMeta)]

Compose multiple steps functionally

Compose multiple steps functionally

Attributes

def standardizeForSTT(audio: GeneratedAudio, targetRate: Int): Result[(Array[Byte], AudioMeta)]

standardizeForSTT for generated audio; MP3 is a ValidationError, as it is not PCM.

standardizeForSTT for generated audio; MP3 is a ValidationError, as it is not PCM.

Attributes

def toMono(bytes: Array[Byte], meta: AudioMeta): Result[(Array[Byte], AudioMeta)]

Convert to mono by averaging channels (PCM16 little-endian).

Convert to mono by averaging channels (PCM16 little-endian).

Attributes

def trimSilence(bytes: Array[Byte], meta: AudioMeta, threshold: Int): Result[(Array[Byte], AudioMeta)]

Trim leading and trailing silence using a simple amplitude threshold on PCM16.

Trim leading and trailing silence using a simple amplitude threshold on PCM16.

Attributes

def wrap(bytes: Array[Byte], meta: AudioMeta, format: AudioFormat): GeneratedAudio