streamComplete is a Source of the provider's chunks; complete is a Future. The provider call blocks, so it never runs on a stream or actor dispatcher thread: a stream runs it on a thread of its own, and complete runs it on the ExecutionContext you pass, which should be a blocking one - for example system.dispatchers.lookup(Dispatchers.DefaultBlockingDispatcherId).
Cancelling a stream interrupts the provider's thread, which is how llm4s providers are cancelled: they keep the interrupt and return Left(CancelledError). A Future cannot be cancelled; use streamComplete when the call has to be.
An LLMError is carried by an LLMException, the failure of the stream or the Future.
The provider's chunks, as they arrive, as a Source. The call starts when the stream is materialized, once per materialization.
The provider's chunks, as they arrive, as a Source. The call starts when the stream is materialized, once per materialization.
'''Backpressure''': the provider calls back on its own thread and cannot be paused, so its thread is blocked while bufferSize chunks wait for the consumer. Nothing is dropped, and no more than bufferSize chunks are held.
'''Cancellation''': cancelling the stream (or take, via a stage that finishes early, a failed downstream) interrupts the provider's thread.
'''Errors''': if the call fails mid-stream, the chunks already received are emitted first, then the stream fails with LLMException. A bufferSize below 1 fails the stream at once.