Ollama with C# and .NET: Run a Local LLM in ASP.NET Core

Running an LLM locally is useful for development, experimentation, and privacy-sensitive workloads where sending data to a third-party API isn't an option.

In this guide we connect Ollama to an ASP.NET Core application, first with nothing but HttpClient, then with the approach we now use in every .NET project on this site: OllamaSharp behind the Microsoft.Extensions.AI IChatClient interface. Along the way we cover streaming, typed JSON output, embeddings, choosing a model, and the two settings that silently broke our results until we found them.

In short:

• Install Ollama, ollama pull a model, and it serves an HTTP API on port 11434.
• For a quick test, HttpClient + POST /api/generate is enough (Steps 2–3).
• For real apps, register OllamaApiClient from OllamaSharp as an IChatClient. Your code stays the same if you later switch to OpenAI, Azure OpenAI or Claude.
• Set a longer HTTP timeout, raise the context window, and use 127.0.0.1 instead of localhost on Windows.

Architecture

ASP.NET Core
     ↓
   Ollama
     ↓
 Local LLM

In more detail, this is what the finished setup looks like:

ASP.NET Core endpoint
     ↓  IChatClient / IEmbeddingGenerator  (Microsoft.Extensions.AI)
OllamaApiClient  (OllamaSharp)
     ↓  HTTP, 127.0.0.1:11434
Ollama server  →  model loaded on GPU or CPU (qwen2.5-coder:7b, nomic-embed-text, …)

Your application code only depends on the interfaces at the top. Ollama, the model and the hardware underneath can change without touching your endpoints.

Step 1 — Install and run Ollama

Install Ollama from ollama.com (Windows, macOS, Linux), then pull a model and start the server:

ollama pull llama3.1
ollama serve

Ollama exposes a local HTTP API, by default on http://localhost:11434.

A few things that are easy to miss:

• The server may already be running. On Windows and macOS the desktop app starts the server for you, and the Linux install script sets it up as a service. If ollama serve fails because port 11434 is already in use, that's why. Skip it.

• Check that it's up before you write any C#:

  Text

  Copy

  curl http://127.0.0.1:11434/api/tags
  ollama list

  /api/tags returns the models you have pulled as JSON. If the model your code asks for isn't in that list, every request fails.

• Running in Docker instead. The official image works the same way. CPU only:

  Text

  Copy

  docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
  docker exec -it ollama ollama pull llama3.1

  For NVIDIA GPUs, add --gpus=all (it needs the NVIDIA Container Toolkit). Running Ollama, a vector database and SQL Server together with Docker Compose will get its own article.

Choosing a model for local development

Pick the model for the job, not the biggest one that fits. These are the models we use on this site and their download sizes from the Ollama library:

How to judge whether a model will run well on your machine:

• The download size is roughly the minimum memory the weights need. The context window (KV cache) needs memory on top of that, and a larger num_ctx needs more.
• Check where it actually runs. Send one request, then run ollama ps. The PROCESSOR column shows 100% GPU, 100% CPU, or a split. A split means the model didn't fit in VRAM and part of it runs on the CPU, which is much slower.
• 7B models are a good default for development on a typical developer machine. They are also where we found real limits. In our tool-calling tests, a 7B model with full context mostly answered directly instead of calling tools, and it "confirmed" its own wrong results.

Step 2 — Call it from ASP.NET Core

The smallest possible integration is a typed HttpClient that posts to Ollama's /api/generate endpoint:

public class OllamaClient(HttpClient http)
{
    public async Task<string> GenerateAsync(string prompt)
    {
        var response = await http.PostAsJsonAsync("/api/generate", new
        {
            model = "llama3.1",
            prompt,
            stream = false
        });
 
        response.EnsureSuccessStatusCode();
        var result = await response.Content.ReadFromJsonAsync<OllamaResponse>();
        return result!.Response;
    }
}
 
public record OllamaResponse(string Response);

Two details make this work. stream = false matters because /api/generate streams by default, and without it you'd get many JSON lines instead of one object. The response field in Ollama's reply maps onto OllamaResponse.Response, because ReadFromJsonAsync uses case-insensitive web defaults. The same reply also includes eval_count, prompt_eval_count, load_duration and total_duration (in nanoseconds). Add them to the record if you want to log token counts and timings.

Step 3 — Register the client

builder.Services.AddHttpClient<OllamaClient>(client =>
{
    client.BaseAddress = new Uri("http://localhost:11434");
});

That is enough for a demo. Before you rely on it, change two things:

builder.Services.AddHttpClient<OllamaClient>(client =>
{
    client.BaseAddress = new Uri("http://127.0.0.1:11434"); // avoid the IPv6 localhost penalty
    client.Timeout = TimeSpan.FromSeconds(180);             // the first request also loads the model
});

• Timeout. HttpClient gives up after 100 seconds by default. A non-streaming request has to load the model (the first call, or after Ollama unloads it from memory, which happens 5 minutes after the last request by default) and then generate the whole answer. On a slow machine or with a long prompt, that can take longer than 100 seconds. We use 180 seconds in the Text-to-SQL tutorial.
• 127.0.0.1, not localhost. When we traced the AI Database Agent, the first trace found a 2-second IPv6 localhost penalty on every new connection to Ollama and Qdrant. Using the IPv4 address directly removed it.

Step 4 — The better way: OllamaSharp + Microsoft.Extensions.AI

The hand-written client works, but you end up re-implementing chat history, streaming, tool calls and JSON schemas yourself. In .NET there's now a standard abstraction for this:

• Microsoft.Extensions.AI defines IChatClient (chat and streaming) and IEmbeddingGenerator<TInput, TEmbedding> (embeddings), plus middleware for logging, OpenTelemetry, caching and function invocation.
• OllamaSharp is the .NET client for the Ollama API. Its OllamaApiClient implements IChatClient and IEmbeddingGenerator<string, Embedding<float>> directly, so you don't need an adapter.

dotnet add package OllamaSharp
dotnet add package Microsoft.Extensions.AI

Register it once in Program.cs. This is the registration from our Text-to-SQL tutorial:

using Microsoft.Extensions.AI;
using OllamaSharp;
 
builder.Services.AddChatClient(sp =>
{
    var http = new HttpClient
    {
        BaseAddress = new Uri("http://127.0.0.1:11434"),
        Timeout = TimeSpan.FromSeconds(180),
    };
    return new OllamaApiClient(http, "qwen2.5-coder:7b");
})
.Use(inner => new ConfigureOptionsChatClient(inner,
    o => o.AddOllamaOption(OllamaOption.NumCtx, 8192)))
.UseLogging();

What each line is for:

• OllamaApiClient(HttpClient, defaultModel) lets you control the timeout and base address. The simpler new OllamaApiClient(new Uri(...), "model") constructor does not set a timeout, so you get the 100-second default.
• AddOllamaOption(OllamaOption.NumCtx, 8192) raises the context window on every request. This was the most important line in that project (see "Settings that matter in practice" below). .ConfigureOptions(o => ...) on the builder is a shorter way to write the same thing.
• .UseLogging() logs every request and response through your ILogger. In production you can add .UseOpenTelemetry() here too. We cover what those traces showed in Part 6 of the AI Database Agent series.

AddChatClient registers a singleton, so the HttpClient above is created once and reused. Now inject IChatClient anywhere:

app.MapPost("/chat", async (ChatRequest req, IChatClient chat, CancellationToken ct) =>
{
    ChatResponse response = await chat.GetResponseAsync(req.Prompt, cancellationToken: ct);
    return Results.Ok(new { answer = response.Text });
});
 
record ChatRequest(string Prompt);

For a system prompt or a multi-turn conversation, pass a list of ChatMessage objects (ChatRole.System, ChatRole.User, …) instead of a string. Then add the reply back with history.AddMessages(response). Ollama is stateless, so you send the whole history with every request.

Which .NET package should you use for Ollama?

If you use .NET Aspire, the Community Toolkit has an OllamaSharp integration (CommunityToolkit.Aspire.OllamaSharp) that registers the same IChatClient from a connection name.

Pulling a model and checking the server from code

OllamaApiClient also exposes the Ollama-specific APIs. This is useful for a startup check or a setup command:

var ollama = new OllamaApiClient(new Uri("http://127.0.0.1:11434"));
 
if (!await ollama.IsRunningAsync())
    throw new InvalidOperationException("Ollama is not running on 127.0.0.1:11434.");
 
var models = await ollama.ListLocalModelsAsync();
if (!models.Any(m => m.Name.StartsWith("qwen2.5-coder:7b")))
{
    await foreach (var status in ollama.PullModelAsync("qwen2.5-coder:7b"))
        Console.WriteLine($"{status?.Percent}% {status?.Status}");
}

Step 5 — Stream the response

The original version of this article ended with "swap GenerateAsync for a streaming implementation". With IChatClient, that's a different method on the same interface. GetStreamingResponseAsync returns an IAsyncEnumerable<ChatResponseUpdate>:

app.MapPost("/chat/stream", async (ChatRequest req, IChatClient chat, HttpResponse response, CancellationToken ct) =>
{
    response.ContentType = "text/plain; charset=utf-8";
 
    await foreach (ChatResponseUpdate update in chat.GetStreamingResponseAsync(req.Prompt, cancellationToken: ct))
    {
        await response.WriteAsync(update.Text, ct);
        await response.Body.FlushAsync(ct);
    }
});

Test it with curl -N -X POST http://localhost:<port>/chat/stream -H "Content-Type: application/json" -d "{\"prompt\":\"Explain IChatClient in two sentences\"}". The text should arrive in pieces rather than all at once. If it arrives as one block, something between the server and the client is buffering the response.

Passing ct through matters: if the browser disconnects, the request is cancelled and the app stops reading the stream. This minimal version streams plain text. For a real chat UI you'll want Server-Sent Events or SignalR, and you'll need to handle disconnects and record token usage. That will get its own article on streaming LLM responses in ASP.NET Core. Our ASP.NET Core AI chatbot already streams over SignalR, and the ASP.NET Core vs FastAPI comparison builds the same streaming endpoint in both frameworks.

Step 6 — Get typed JSON back instead of text

When code consumes the answer, ask for a type, not prose. GetResponseAsync<T> generates a JSON schema from your C# type. OllamaSharp passes it to Ollama's format parameter, which constrains the output to that schema:

public enum TicketCategory { Bug, Billing, FeatureRequest, Other }
 
public record TicketTriage(TicketCategory Category, int Priority, string Summary);
 
var result = await chat.GetResponseAsync<TicketTriage>(
    $"Triage this support ticket. Priority 1 (urgent) to 4 (low).\n\n{ticketText}",
    cancellationToken: ct);
 
if (result.TryGetResult(out TicketTriage? triage))
{
    // triage.Category, triage.Priority, triage.Summary
}

Valid JSON doesn't make the answer correct. A 7B model can return a well-formed object with the wrong category, so validate the values like any other untrusted input. We apply the same rule to generated SQL in Preventing SQL Injection in AI-Generated SQL. If you start from a sample JSON response, our JSON to C# converter generates the record types for you.

Step 7 — Generate embeddings with the same library

The same OllamaApiClient class implements IEmbeddingGenerator. Point it at an embedding model and register it next to the chat client:

ollama pull nomic-embed-text

IEmbeddingGenerator<string, Embedding<float>> embedder =
    new OllamaApiClient(new Uri("http://127.0.0.1:11434"), "nomic-embed-text");
 
builder.Services.AddEmbeddingGenerator(embedder);

ReadOnlyMemory<float> vector = await embedder.GenerateVectorAsync("How do I reset my password?");
Console.WriteLine(vector.Length); // dimension of the embedding model

Use one embedding model for both indexing and querying. If you change models, re-embed everything. To see these vectors used in a full pipeline, go to Build RAG with ASP.NET Core and Qdrant, or read What Is RAG? for the concepts.

Settings that matter in practice

These are the settings that changed our results, not just our latency:

Context window (num_ctx). Ollama's context window defaults to a few thousand tokens, regardless of the model's maximum (32K for qwen2.5-coder). When a prompt is longer than that, Ollama truncates it without an error. In our Text-to-SQL work, the schema prompt didn't fit in the 2,048-token default of the Ollama version we were running. The model never saw part of the schema. Raising NumCtx to 8,192 fixed it. You can set it per request (AddOllamaOption(OllamaOption.NumCtx, …), as in Step 4) or for the whole server (OLLAMA_CONTEXT_LENGTH). Newer Ollama versions raised the default (the current docs list 4,096), so check your version. A larger value uses more memory.

Model unloading (keep_alive). Ollama keeps a model in memory for 5 minutes after the last request, then unloads it. The next request pays the load time again. That's fine for development. For a demo or an internal tool, set OLLAMA_KEEP_ALIVE (for example 30m, or -1 to keep it loaded) so the first user after a quiet period doesn't hit a timeout.

Concurrency. By default each model processes one request at a time (OLLAMA_NUM_PARALLEL=1). Other requests wait in a queue. If two users call your endpoint at once, the second one waits for the first one's whole answer. Keep this in mind when you set timeouts and before you load-test.

Network exposure. Ollama listens on 127.0.0.1 only. To reach it from another machine or a container, set OLLAMA_HOST (for example 0.0.0.0:11434). Ollama's API has no authentication, so don't expose it beyond a network you trust.

Troubleshooting

A note on tool calling

IChatClient supports tool calling through UseFunctionInvocation() and AIFunctionFactory. Ollama supports it for models trained for it. Whether a 7B model calls your tools reliably is a different question. In Part 5 of the AI Database Agent series, qwen2.5-coder:7b wrote a JSON object naming the tool into its text instead of making a structured call, so the experiments switched to the general qwen2.5:7b. With full context in the prompt, that model mostly answered directly instead of calling the tools. A full tool-calling walkthrough will get its own article.

Why this matters for the Foundry

Every article in the AI + .NET discipline builds toward the same flagship project — this local Ollama connection is the same building block the AI Database Agent uses before it ever touches a real SQL Server instance.

In practice that means the IChatClient registration in Step 4, qwen2.5-coder:7b for SQL generation, and nomic-embed-text for the business glossary retrieval in Part 4. The source is on GitHub.

What's next

Swap GenerateAsync for a streaming implementation over Server-Sent Events or SignalR once the API needs to support a live chat interface.

With IChatClient in place, these are the natural next steps:

• Generate SQL from plain English, safely: How to Build Text-to-SQL with ASP.NET Core
• Answer questions from your own documents: Build RAG with ASP.NET Core and Qdrant
• Decide where the AI endpoint should live: ASP.NET Core vs FastAPI for AI APIs
• Before you let a model act on data: AI Agent Security: What Developers Need to Know

FAQ

Do I need OllamaSharp, or can I just use HttpClient?

HttpClient is enough for a single non-streaming call (Steps 2–3). OllamaSharp is worth adding once you need streaming, chat history, embeddings, structured output or tool calling. It also implements IChatClient, so you can swap Ollama for a hosted model later without rewriting your endpoints.

Is Microsoft.Extensions.AI.Ollama still supported?

No. The package is deprecated on NuGet, which recommends OllamaSharp instead, and no further updates are planned. Use OllamaApiClient from OllamaSharp as your IChatClient.

Which Ollama model should I start with in C#?

Start with a 7B–8B model that fits entirely in your GPU memory. ollama ps should show 100% GPU. We use qwen2.5-coder:7b for SQL and code, llama3.1 for general chat, and nomic-embed-text for embeddings. Test on your own prompts before you choose a bigger model.

Why does my first request time out but later ones are fast?

The first request loads the model into memory. Ollama unloads it again after 5 idle minutes by default. Raise the HttpClient timeout above the 100-second default, use streaming, or set OLLAMA_KEEP_ALIVE to keep the model loaded.

Can I use the same code with OpenAI, Azure OpenAI or Claude later?

Yes, as long as your code depends on IChatClient and IEmbeddingGenerator rather than on OllamaApiClient. Only the registration in Program.cs changes. The Ollama-specific options (like NumCtx) are simply not used by other providers.