Skip to content

Scroll till the end of the page if you just want code
Ok, so you want to explore Large Language Models(LLM's) like ChatGPT with help of greatest programming language in the world(c#)?

Scroll till the end of the page if you just want code

Let's do this!

You, probably, heard that ChatGPT API costs some money, but don't worry. In this example we will use so called local models. You can think of it as a smaller version of ChatGPT that can run on your computer. Even without graphics card!!!! And completly free!

Installation

Create new console application and use nuget to install these packages(dont't forget to check Include prerelease):

LangChain - meta package for development, it includes the Core package, OpenAI, HuggingFace and LLamaSharp providers

You also need to install ONE of these packages for the backend if you are using the LLamaSharp provider 

1
2
3
LLamaSharp.Backend.Cpu  # cpu for windows, linux and mac (mac metal is also supported)
LLamaSharp.Backend.Cuda11  # cuda11 for windows and linux
LLamaSharp.Backend.Cuda12  # cuda12 for windows and linux

Preparing the model

Lets download our model first. For this example we will be using model called Thespis with 13 billions parameters and quantization Q2_K.

You can check it here: Thespis-13B-v0.5-GGUF

You can look for other models here: TheBloke. We can only run models in GGUF format.

What is quantization? - you may ask

In simple words in is precision of the model. Models with small quantization are easily getting confused with complex prompts, but are working much faster.

So, finally, let's write some code! 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
using LangChain.Providers;
using LangChain.Providers.HuggingFace.Downloader;
using LangChain.Providers.LLamaSharp;
using static LangChain.Chains.Chain;

// get model path
var modelPath = await HuggingFaceModelDownloader.GetModelAsync(
    repository: "TheBloke/Thespis-13B-v0.5-GGUF",
    fileName: "thespis-13b-v0.5.Q2_K.gguf",
    version: "main");

This line will download the model and save it locally for future usage. After model is downloaded it will return path to the .gguf file. _You can manually download any model you want and insert path to it directly. Without using HuggingFaceModelDownloader.*_

Now it's time to load our model into memory: 

1
2
// load model
var model = LLamaSharpModelInstruction.FromPath(modelPath).UseConsoleForDebug();

Now let's build a chain!

Building a chain

This is minimal chain to make LLM work: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
// building a chain
var prompt = @"
You are an AI assistant that greets the world.
World: Hello, Assistant!
Assistant:";

var chain =
    Set(prompt, outputKey: "prompt")
    | LLM(model, inputKey: "prompt");

await chain.RunAsync();

We can see here 2 chains(or links) working together: Set and LLM.

  • Set - setting value for the chain context variable prompt
  • LLM - getting value from chain context variable prompt and passing it to specified model

Links are connected together with '|' symbol.

Finally we run the entire chain. After some seconds you will see entire dialog in console window:

1
2
3
You are an AI assistant that greets the world.
World: Hello, Assistant!
Assistant: Hello, World! How can I help you today?

Complete code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
using LangChain.Providers;
using LangChain.Providers.HuggingFace.Downloader;
using LangChain.Providers.LLamaSharp;
using static LangChain.Chains.Chain;

// get model path
var modelPath = await HuggingFaceModelDownloader.GetModelAsync(
    repository: "TheBloke/Thespis-13B-v0.5-GGUF",
    fileName: "thespis-13b-v0.5.Q2_K.gguf",
    version: "main");


// load model
var model = LLamaSharpModelInstruction.FromPath(modelPath).UseConsoleForDebug();


// building a chain
var prompt = @"
You are an AI assistant that greets the world.
World: Hello, Assistant!
Assistant:";

var chain =
    Set(prompt, outputKey: "prompt")
    | LLM(model, inputKey: "prompt");

await chain.RunAsync();