Skip to content

Scroll till the end of the page if you just want code

Setup

We will take the code from [Getting started] tutorial as our starting point. 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
using LangChain.Providers.HuggingFace.Downloader;
using LangChain.Providers.LLamaSharp;
using static LangChain.Chains.Chain;

// get model path
var modelPath = await HuggingFaceModelDownloader.GetModelAsync(
    repository: "TheBloke/Thespis-13B-v0.5-GGUF",
    fileName: "thespis-13b-v0.5.Q2_K.gguf",
    version: "main");

// load model
var model = LLamaSharpModelInstruction.FromPath(modelPath);

// building a chain
var prompt = @"
You are an AI assistant that greets the world.
World: Hello, Assistant!
Assistant:";

Getting the chain output

Almost every possible link in a chain are having having at least one input and output.

Look here: 

1
2
3
var chain =
    Set(prompt, outputKey: "prompt")
    | LLM(model, inputKey: "prompt", outputKey: "result");

This means that, after link Set get executed, we are storring it's result into "prompt" variable inside of chain context. In its turn, link LLM gets "prompt" variable from chain context and uses it's as input.

LLM link also has output key argument. Let's use it to save the result of llm. 

1
var result = await chain.RunAsync("result");

Now the LLM link saves it's result into "result" variable inside of chain context. But how do we extract it from there?

chain.Run() method has an optional argument "resultKey". This allows you to specify variable inside of chain context to return as a result. 

1
Console.WriteLine(result);

Output: 

1
Hello, World! How can I help you today?

Complete code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
using LangChain.Providers.HuggingFace.Downloader;
using LangChain.Providers.LLamaSharp;
using static LangChain.Chains.Chain;

// get model path
var modelPath = await HuggingFaceModelDownloader.GetModelAsync(
    repository: "TheBloke/Thespis-13B-v0.5-GGUF",
    fileName: "thespis-13b-v0.5.Q2_K.gguf",
    version: "main");

// load model
var model = LLamaSharpModelInstruction.FromPath(modelPath);

// building a chain
var prompt = @"
You are an AI assistant that greets the world.
World: Hello, Assistant!
Assistant:";


var chain =
    Set(prompt, outputKey: "prompt")
    | LLM(model, inputKey: "prompt", outputKey: "result");


var result = await chain.RunAsync("result");


Console.WriteLine(result);