Skip to content

Streaming Chat Completion

Stream chat completion tokens as they are generated using the IChatClient interface.

This example assumes using HuggingFace; is in scope and apiKey contains your HuggingFace API key.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
using var client = new HuggingFaceInferenceClient(apiKey);
IChatClient chatClient = client;

await foreach (var update in chatClient.GetStreamingResponseAsync(
    [new ChatMessage(ChatRole.User, "Say hello in one word.")],
    new ChatOptions
    {
        ModelId = "Qwen/Qwen2.5-Coder-32B-Instruct",
        MaxOutputTokens = 32,
    }))
{
    Console.Write(update.Text);
}