Rystem.OpenAi 10.0.10

Unofficial Fluent C#/.NET SDK for accessing the OpenAI API (Easy swap among OpenAi and Azure OpenAi)

Last update with Cost and Tokens calculation

A simple C# .NET wrapper library to use with OpenAI's API.

Help the project

Contribute: https://www.buymeacoffee.com/keyserdsoze

Contribute: https://patreon.com/Rystem

Stars

Requirements

This library targets .NET 9 or above.

Adv

Watch out my Rystem framework to be able to do .Net webapp faster (easy integration with repository pattern or CQRS for your Azure services).

What is Rystem?

Setup

Install package Rystem.OpenAi from Nuget. Here's how via command line:

Install-Package Rystem.OpenAi

Documentation

Table of Contents

• Unofficial Fluent C#/.NET SDK for accessing the OpenAI API (Easy swap among OpenAi and Azure OpenAi)
  • Last update with Cost and Tokens calculation
  • Help the project
    • Contribute: https://www.buymeacoffee.com/keyserdsoze
    • Contribute: https://patreon.com/Rystem
  • Stars
  • Requirements
    • Adv
    • What is Rystem?
  • Setup
  • Documentation
    • Table of Contents
  • Startup Setup
  • Dependency Injection
    • Add to service collection the OpenAi service in your DI
  • Dependency Injection With Azure
    • Add to service collection the OpenAi service in your DI with Azure integration
    • Add to service collection the OpenAi service in your DI with Azure integration and app registration
    • Add to service collection the OpenAi service in your DI with Azure integration and system assigned managed identity
    • Add to service collection the OpenAi service in your DI with Azure integration and user assigned managed identity
  • Use different version
  • Dependency Injection With Factory
  • Without Dependency Injection
  • Models
    • List Models
    • Retrieve Models
    • Delete fine-tune model
  • Chat
    • Usage examples
    • Function chaining
  • Images
    • Create Image
    • Create Image Edit
    • Create Image Variation
  • Embeddings
    • Create Embedding
    • Distance for embedding
    • Which distance function should I use?
  • Audio
    • Create Transcription
    • Create Translation
    • Speech
  • File
    • List files
    • Upload file
    • Delete file
    • Retrieve file
    • Retrieve file content
  • Fine-Tunes
    • Create fine-tune
    • List fine-tunes
    • Retrieve fine-tune
    • Cancel fine-tune
    • List fine-tune events
    • List fine-tune events as stream
    • Delete fine-tune model
  • Moderations
    • Create moderation
  • Utilities
    • Cosine Similarity
    • Tokens
    • Cost
    • Setup price
  • Management
    • Billing
    • Deployments
  • How the OpenAI Assistant Works
  • Assistant Features
  • Assistant
  • Thread
  • Run
  • VectorStore
  • Examples
    • Create a Simple Assistant
    • Use the Assistant for Code Interpretation
    • Create a Thread and Manage Messages
    • Execute a Run and Retrieve Steps
    • Stream Responses from the Assistant
    • Work with VectorStore

Startup Setup

📖 Back to summary
You may install with Dependency Injection one or more than on integrations at the same time. Furthermore you don't need to use the Dependency Injection pattern and use a custom Setup.

Dependency Injection

📖 Back to summary

Add to service collection the OpenAi service in your DI

    var apiKey = configuration["Azure:ApiKey"];
    services.AddOpenAi(settings =>
    {
        settings.ApiKey = apiKey;
        //add a default model for chatClient, you can add everything in this way to prepare at the best your
        //client for the request
        settings.DefaultRequestConfiguration.Chat = chatClient =>
        {
            chatClient.WithModel(configuration["OpenAi2:ModelName"]!);
        };
    }, "custom integration name");

    var openAiApi = serviceProvider.GetRequiredService<IFactory<IOpenAi>>();
    var firstInstanceOfChatClient = openAiApi.Create("custom integration name").Chat;
    var openAiChatApi = serviceProvider.GetRequiredService<IFactory<IOpenAiChat>>();
    var anotherInstanceOfChatClient = openAiChatApi.Create("custom integration name");

Dependency Injection With Azure

Add to service collection the OpenAi service in your DI with Azure integration

When you want to use the integration with Azure.

    builder.Services.AddOpenAi(settings =>
    {
        settings.ApiKey = apiKey;
        settings.Azure.ResourceName = "AzureResourceName (Name of your deployed service on Azure)";
    });

Add to service collection the OpenAi service in your DI with Azure integration and app registration

See how to create an app registration here.

    var resourceName = builder.Configuration["Azure:ResourceName"];
    var clientId = builder.Configuration["AzureAd:ClientId"];
    var clientSecret = builder.Configuration["AzureAd:ClientSecret"];
    var tenantId = builder.Configuration["AzureAd:TenantId"];
    builder.Services.AddOpenAi(settings =>
    {
        settings.Azure.ResourceName = resourceName;
        settings.Azure.AppRegistration.ClientId = clientId;
        settings.Azure.AppRegistration.ClientSecret = clientSecret;
        settings.Azure.AppRegistration.TenantId = tenantId;
    });

Add to service collection the OpenAi service in your DI with Azure integration and system assigned managed identity

See how to create a managed identity here.
System Assigned Managed Identity

    var resourceName = builder.Configuration["Azure:ResourceName"];
    builder.Services.AddOpenAi(settings =>
    {
        settings.Azure.ResourceName = resourceName;
        settings.Azure.ManagedIdentity.UseDefault = true;
    });

Add to service collection the OpenAi service in your DI with Azure integration and user assigned managed identity

See how to create a managed identity here.
User Assigned Managed Identity

    var resourceName = builder.Configuration["Azure:ResourceName"];
    var managedIdentityId = builder.Configuration["ManagedIdentity:ClientId"];
    builder.Services.AddOpenAi(settings =>
    {
        settings.Azure.ResourceName = resourceName;
        settings.Azure.ManagedIdentity.Id = managedIdentityId;
    });

Use different version

📖 Back to summary
You may install different version for each endpoint.

     services.AddOpenAi(settings =>
            {
                settings.ApiKey = azureApiKey;
                //default version for all endpoints
                settings.DefaultVersion = "2024-08-01-preview";
                //different version for chat endpoint
                settings.DefaultRequestConfiguration.Chat = chatClient =>
                    {
                        chatClient.ForceModel("gpt-4");
                        chatClient.WithVersion("2024-08-01-preview");
                    };
            });

In this example We are adding a different version only for chat, and all the other endpoints will use the same (in this case the default version).

Dependency Injection With Factory

📖 Back to summary
You may install more than one OpenAi integration, using name parameter in configuration. In the next example we have two different configurations, one with OpenAi and a default name and with Azure OpenAi and name "Azure"

    var apiKey = context.Configuration["OpenAi:ApiKey"];
    services
        .AddOpenAi(settings =>
        {
            settings.ApiKey = apiKey;
        });
    var azureApiKey = context.Configuration["Azure:ApiKey"];
    var resourceName = context.Configuration["Azure:ResourceName"];
    var clientId = context.Configuration["AzureAd:ClientId"];
    var clientSecret = context.Configuration["AzureAd:ClientSecret"];
    var tenantId = context.Configuration["AzureAd:TenantId"];
    services.AddOpenAi(settings =>
    {
        settings.ApiKey = azureApiKey;
        settings.DefaultRequestConfiguration.Chat = chatClient =>
            {
                chatClient.ForceModel("gpt-4");
                chatClient.WithVersion("2024-08-01-preview");
            };
        settings.Azure.ResourceName = resourceName;
        settings.Azure.AppRegistration.ClientId = clientId;
        settings.Azure.AppRegistration.ClientSecret = clientSecret;
        settings.Azure.AppRegistration.TenantId = tenantId;
    }, "Azure");

I can retrieve the integration with IFactory<> interface (from Rystem) and the name of the integration.

    private readonly IFactory<IOpenAi> _openAiFactory;

    public CompletionEndpointTests(IFactory<IOpenAi> openAiFactory)
    {
        _openAiFactory = openAiFactory;
    }

    public async ValueTask DoSomethingWithDefaultIntegrationAsync()
    {
        var openAiApi = _openAiFactory.Create();
        openAiApi.Chat.........
    }

    public async ValueTask DoSomethingWithAzureIntegrationAsync()
    {
        var openAiApi = _openAiFactory.Create("Azure");
        openAiApi.Chat.........
    }

or get the more specific service

    private readonly IFactory<IOpenAiChat> _chatFactory;
    public Constructor(IFactory<IOpenAiChat> chatFactory)
    {
        _chatFactory = chatFactory;
    }
    public async ValueTask DoSomethingWithAzureIntegrationAsync()
    {
        var chat = _chatFactory.Create(name);
        chat.ExecuteRequestAsync(....);
    }

Without Dependency Injection

📖 Back to summary
You may configure in a static constructor or during startup your integration without the dependency injection pattern.

      OpenAiServiceLocator.Configuration.AddOpenAi(settings =>
        {
            settings.ApiKey = apiKey;
        }, "NoDI");

and you can use it with the same static class OpenAiServiceLocator and the static Create method

    var openAiApi = OpenAiServiceLocator.Instance.Create(name);
    openAiApi.Embedding......

or get the more specific service

    var openAiEmbeddingApi = OpenAiServiceLocator.Instance.CreateEmbedding(name);
    openAiEmbeddingApi.Request(....);

Models

📖 Back to summary
List and describe the various models available in the API. You can refer to the Models documentation to understand what models are available and the differences between them.
You may find more details here, and here samples from unit test.

List Models

Lists the currently available models, and provides basic information about each one such as the owner and availability.

    var openAiApi = _openAiFactory.Create(name);
    var results = await openAiApi.Model.ListAsync();

Retrieve Models

Retrieves a model instance, providing basic information about the model such as the owner and per missioning.

    var openAiApi = _openAiFactory.Create(name);
    var result = await openAiApi.Model.RetrieveAsync("insert here the model name you need to retrieve");

Delete fine-tune model

Delete a fine-tuned model. You must have the Owner role in your organization.

    var openAiApi = _openAiFactory.Create(name);
    var deleteResult = await openAiApi.Model
        .DeleteAsync(fineTuneModelId);

Chat

📖 Back to summary
Given a chat conversation, the model will return a chat completion response.
You may find more details here, and here samples from unit test.

The IOpenAiChat interface provides a robust framework for interacting with OpenAI Chat models. This documentation includes method details and usage explanations, followed by 20 distinct examples that demonstrate real-world applications.

Methods Overview

1. Execution Methods

ExecuteAsync(CancellationToken cancellationToken = default)

• Executes the configured request and retrieves the result in a single response.
• Usage: Best for one-off requests where the response can be processed at once.

ExecuteAsStreamAsync(bool withUsage = true, CancellationToken cancellationToken = default)

• Streams the results progressively, enabling real-time processing.
• Usage: Ideal for scenarios where partial results need to be displayed or acted upon immediately.

2. Message Management

Adding Messages

• AddMessage(ChatMessageRequest message)
  Adds a message with detailed configuration (Role, Content).
• AddMessages(params ChatMessageRequest[] messages)
  Adds multiple messages at once.
• AddMessage(string content, ChatRole role = ChatRole.User)
  A simplified method to add a single message.

Specialized Messages

• AddUserMessage(string content)
  Adds a user-specific message.
• AddSystemMessage(string content)
  Adds a system-specific message for setting context.
• AddAssistantMessage(string content)
  Adds an assistant-specific message.

Retrieving Messages

• GetCurrentMessages()
  Retrieves all messages added to the current request.

Content Builder

• AddContent(ChatRole role = ChatRole.User)
  Adds content dynamically with a builder.
• AddUserContent(), AddSystemContent(), AddAssistantContent()
  Builders for specific message roles.

3. Parameter Configuration

Generation Parameters

• WithTemperature(double value)
  Adjusts randomness (range: 0 to 2).
• WithNucleusSampling(double value)
  Enables nucleus sampling (range: 0 to 1).
• WithPresencePenalty(double value)
  Penalizes repeating tokens (range: -2 to 2).
• WithFrequencyPenalty(double value)
  Penalizes frequent tokens (range: -2 to 2).

Token and Choice Limits

• SetMaxTokens(int value)
  Sets the maximum tokens for the response.
• WithNumberOfChoicesPerPrompt(int value)
  Sets how many response options to generate.

Stop Sequences

• WithStopSequence(params string[] values)
  Adds one or more stop sequences.
• AddStopSequence(string value)
  Adds a single stop sequence.

Bias and User Context

• WithBias(string key, int value), WithBias(Dictionary<string, int> bias)
  Adjusts the likelihood of specific tokens appearing.
• WithUser(string user)
  Adds a unique user identifier for tracking.
• WithSeed(int? seed)
  Sets a seed for deterministic responses.

4. Response Format Management

• ForceResponseFormat(FunctionTool function), ForceResponseFormat(MethodInfo function)
  Forces responses to follow specific function-based formats.
• ForceResponseAsJsonFormat(), ForceResponseAsText()
  Ensures responses are structured as JSON or plain text.

5. Tool and Function Management

• AvoidCallingTools(), ForceCallTools(), CanCallTools()
  Configures tool-calling behavior.
• ClearTools(), ForceCallFunction(string name)
  Manages specific tools and their calls.

Usage Examples

Basic Interaction

Description: A simple user message and response.

var chat = openAiApi.Chat
    .AddUserMessage("Hello, how are you?")
    .WithModel(ChatModelName.Gpt4_o);

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Streaming Interaction

Description: Streaming a response progressively.

await foreach (var chunk in openAiApi.Chat
    .AddUserMessage("Tell me a story.")
    .WithModel(ChatModelName.Gpt4_o)
    .ExecuteAsStreamAsync())
{
    Console.Write(chunk.Choices?.FirstOrDefault()?.Delta?.Content);
}

Configuring Temperature

Description: Adjusting response randomness.

var chat = openAiApi.Chat
    .AddUserMessage("What is your opinion on AI?")
    .WithTemperature(0.9);

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Adding Multiple Messages

Description: Sending multiple messages to set context.

var chat = openAiApi.Chat
    .AddSystemMessage("You are a helpful assistant.")
    .AddUserMessage("Who won the soccer match yesterday?")
    .AddUserMessage("What are the latest updates?");

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Using Stop Sequences

Description: Limiting the response with stop sequences.

var chat = openAiApi.Chat
    .AddUserMessage("Explain the theory of relativity.")
    .WithStopSequence("end");

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Adding a Function Tool

Description: Using functions for structured responses.

var functionTool = new FunctionTool
{
    Name = "calculate_sum",
    Description = "Adds two numbers",
    Parameters = new FunctionToolMainProperty()
        .AddPrimitive("number1", new FunctionToolPrimitiveProperty { Type = "integer" })
        .AddPrimitive("number2", new FunctionToolPrimitiveProperty { Type = "integer" })
        .AddRequired("number1")
        .AddRequired("number2")
};

var chat = openAiApi.Chat
    .AddUserMessage("Calculate the sum of 5 and 10.")
    .AddFunctionTool(functionTool);

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Streaming with Stop Sequence

Description: Streaming with an enforced stop condition.

await foreach (var chunk in openAiApi.Chat
    .AddUserMessage("Describe the universe.")
    .WithStopSequence("stop")
    .ExecuteAsStreamAsync())
{
    Console.Write(chunk.Choices?.FirstOrDefault()?.Delta?.Content);
}

Presence Penalty

Description: Encouraging diverse topics in the response.

var chat = openAiApi.Chat
    .AddUserMessage("Tell me something new.")
    .WithPresencePenalty(1.5);

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

Frequency Penalty

Description: Reducing repetitive phrases.

var chat = openAiApi.Chat
    .AddUserMessage("What is recursion?")
    .WithFrequencyPenalty(1.5);

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

JSON Response

Description: Forcing the response to be in JSON format.

var chat = openAiApi.Chat
    .AddUserMessage("Summarize the book '1984'.")
    .ForceResponseAsJsonFormat();

var result = await chat.ExecuteAsync();
Console.WriteLine(result.Choices?.FirstOrDefault()?.Message?.Content);

You can use some JsonProperty attribute like:

• JsonPropertyName: name of the property
• JsonPropertyDescription: description of what the property is.
• JsonRequired: to set as Required for OpenAi
• JsonPropertyAllowedValues: to have only a range of possible values for the property.
• JsonPropertyRange: to have a range of values
• JsonPropertyMaximum: to have a maximum value for the property
• JsonPropertyMinimum: to have a minimum value for the property
• JsonPropertyMultipleOf: to have only a multiple of a value for the property

After the configuration you can use this function framework in this way:

    var openAiApi = _openAiFactory.Create(name);
    var response = await openAiApi.Chat
        .RequestWithUserMessage("What is the weather like in Boston?")
        .WithModel(ChatModelType.Gpt35Turbo_Snapshot)
        .WithFunction(WeatherFunction.NameLabel)
        .ExecuteAndCalculateCostAsync(true);

    var content = response.Result.Choices[0].Message.Content;

Function chaining

You may find the PlayFramework here

Images

📖 Back to summary
Given a prompt and/or an input image, the model will generate a new image.
You may find more details here, and here samples from unit test.

The IOpenAiImage interface provides functionality for generating, editing, and varying images using OpenAI's image models. This document covers each method with explanations and includes 20 distinct examples demonstrating their usage.

1. Image Generation

GenerateAsync(string prompt, CancellationToken cancellationToken = default)

• Description: Generates an image based on a textual prompt.
• Usage: Use for scenarios where a visual representation of an idea is required.
• Returns: ImageResult containing the generated image's details.

GenerateAsBase64Async(string prompt, CancellationToken cancellationToken = default)

• Description: Generates an image and returns it as a Base64 string.
• Usage: Ideal for embedding images directly into web or mobile applications without saving files.
• Returns: ImageResultForBase64 with the image encoded as Base64.

2. Image Editing

EditAsync(string prompt, Stream file, string fileName = "image", CancellationToken cancellationToken = default)

• Description: Edits an image using a text prompt and an image file.
• Usage: Modify existing images based on creative or functional requirements.
• Returns: ImageResult with the edited image's details.

EditAsBase64Async(string prompt, Stream file, string fileName = "image", CancellationToken cancellationToken = default)

• Description: Edits an image and returns it as a Base64 string.
• Usage: Enables editing workflows with direct Base64 output for web integration.
• Returns: ImageResultForBase64.

3. Image Variation

VariateAsync(Stream file, string fileName = "image", CancellationToken cancellationToken = default)

• Description: Creates variations of an existing image.
• Usage: Generate alternate versions of an image for creative exploration.
• Returns: ImageResult.

VariateAsBase64Async(Stream file, string fileName = "image", CancellationToken cancellationToken = default)

• Description: Creates variations of an image and returns them as Base64 strings.
• Usage: Useful for embedding variations in platforms that consume Base64 directly.
• Returns: ImageResultForBase64.

4. Additional Configurations

WithMask(Stream mask, string maskName = "mask.png")

• Description: Adds a mask to guide image editing.
• Usage: Define specific areas of an image to be edited or preserved.

WithNumberOfResults(int numberOfResults)

• Description: Sets the number of images to generate (1 to 10).
• Usage: Control how many images are returned in a single operation.

WithSize(ImageSize size)

• Description: Specifies the size of generated images (e.g., 256x256, 512x512, 1024x1024).
• Usage: Select resolution based on the intended use case.

WithQuality(ImageQuality quality)

• Description: Sets the quality of generated images.
• Usage: Choose between standard and high-quality outputs based on performance needs.

WithStyle(ImageStyle style)

• Description: Specifies the artistic style of generated images.
• Usage: Create images with specific aesthetic or thematic styles.

WithUser(string user)

• Description: Sets a unique identifier for tracking and abuse prevention.
• Usage: Helps monitor usage and identify specific user requests.

Create Image

Creates an image given a prompt.

    var openAiApi = _openAiFactory.Create(name)!;
    var response = await openAiApi.Image
        .WithSize(ImageSize.Large)
        .GenerateAsync("Create a captive logo with ice and fire, and thunder with the word Rystem. With a desolated futuristic landscape.");
    var uri = response.Data?.FirstOrDefault();

Download directly and save as stream

    var openAiApi = _openAiFactory.Create(name)!;

    var response = await openAiApi.Image
        .WithSize(ImageSize.Large)
        .GenerateAsBase64Async("Create a captive logo with ice and fire, and thunder with the word Rystem. With a desolated futuristic landscape.");

    var image = response.Data?.FirstOrDefault();
    var imageAsStream = image.ConvertToStream();

Create Image Edit

Creates an edited or extended image given an original image and a prompt.

    var openAiApi = _openAiFactory.Create(name)!;
    var location = Assembly.GetExecutingAssembly().Location;
    location = string.Join('\\', location.Split('\\').Take(location.Split('\\').Length - 1));
    using var readableStream = File.OpenRead($"{location}\\Files\\otter.png");
    var editableFile = new MemoryStream();
    await readableStream.CopyToAsync(editableFile);
    editableFile.Position = 0;

    var response = await openAiApi.Image
        .WithSize(ImageSize.Small)
        .WithNumberOfResults(2)
        .EditAsync("A cute baby sea otter wearing a beret", editableFile, "otter.png");

    var uri = response.Data?.FirstOrDefault();

Create Image Variation

Creates a variation of a given image.

    var openAiApi = _openAiFactory.Create(name)!;

    var location = Assembly.GetExecutingAssembly().Location;
    location = string.Join('\\', location.Split('\\').Take(location.Split('\\').Length - 1));
    using var readableStream = File.OpenRead($"{location}\\Files\\otter.png");
    var editableFile = new MemoryStream();
    await readableStream.CopyToAsync(editableFile);
    editableFile.Position = 0;
    var response = await openAiApi.Image
        .WithSize(ImageSize.Small)
        .WithNumberOfResults(1)
        .VariateAsync(editableFile, "otter.png");

    var uri = response.Data?.FirstOrDefault();

Embeddings

📖 Back to summary
Get a vector representation of a given input that can be easily consumed by machine learning models and algorithms.
You may find more details here, and here samples from unit test.

The IOpenAiEmbedding interface provides methods to generate embeddings for text inputs, enabling downstream tasks such as similarity search, clustering, and machine learning model inputs. This documentation explains each method and includes 10 usage examples.

1. Input Management

WithInputs(params string[] inputs)

• Description: Adds an array of strings to be processed for embeddings.
• Usage: Use when multiple inputs are provided simultaneously.

ClearInputs()

• Description: Removes all previously added inputs.
• Usage: Resets the input list, useful for reconfiguring the operation.

AddPrompt(string input)

• Description: Adds a single input string for embedding.
• Usage: Use when inputs are added incrementally or one at a time.

2. User Identification

WithUser(string user)

• Description: Adds a unique identifier for the user, aiding in monitoring and abuse detection.
• Usage: Helpful in multi-user applications or for logging purposes.

3. Embedding Configuration

WithDimensions(int dimensions)

• Description: Sets the desired dimensionality for the output embeddings.
• Usage: Supported only in specific models where dimension configuration is allowed.

WithEncodingFormat(EncodingFormatForEmbedding encodingFormat)

• Description: Specifies the encoding format of the embeddings (e.g., Base64, Float).
• Usage: Define the format based on downstream processing needs.

4. Execution

ExecuteAsync(CancellationToken cancellationToken = default)

• Description: Executes the embedding operation asynchronously and returns the result.
• Usage: Call after configuring inputs and parameters.

Create Embedding

Creates an embedding vector representing the input text.

    var openAiApi = name == "NoDI" ? OpenAiServiceLocatorLocator.Instance.Create(name) : _openAiFactory.Create(name)!;

    var results = await openAiApi.Embeddings
        .WithInputs("A test text for embedding")
        .ExecuteAsync();

    var resultOfCosineSimilarity = _openAiUtility.CosineSimilarity(results.Data.First().Embedding!, results.Data.First().Embedding!);

Create Embedding with custom dimensions

Creates an embedding with custom dimensions vector representing the input text. Only supported in text-embedding-3 and later models.

    var openAiApi = name == "NoDI" ? OpenAiServiceLocatorLocator.Instance.Create(name) : _openAiFactory.Create(name)!;

    var results = await openAiApi.Embeddings
        .AddPrompt("A test text for embedding")
        .WithModel("text-embedding-3-large")
        .WithDimensions(999)
        .ExecuteAsync();

Distance for embedding

For searching over many vectors quickly, we recommend using a vector database. You can find examples of working with vector databases and the OpenAI API in our Cookbook on GitHub. Vector database options include:

• Pinecone, a fully managed vector database
• Weaviate, an open-source vector search engine
• Redis as a vector database
• Qdrant, a vector search engine
• Milvus, a vector database built for scalable similarity search
• Chroma, an open-source embeddings store

Which distance function should I use?

We recommend cosine similarity. The choice of distance function typically doesn't matter much.

OpenAI embeddings are normalized to length 1, which means that:

Cosine similarity can be computed slightly faster using just a dot product Cosine similarity and Euclidean distance will result in the identical rankings

You may use the utility service in this repository to calculate in C# the distance with Cosine similarity

Audio

📖 Back to summary
You may find more details here, and here samples from unit test.

The IOpenAiAudio interface provides methods to handle audio processing tasks such as transcription, translation, and customization of audio analysis. Below is a detailed breakdown of each method.

1. Audio File Input

WithFile(byte[] file, string fileName = "default")

• Description: Adds an audio file as a byte array for processing.
• Parameters:
  • file: Byte array representing the audio file.
  • fileName: Name of the audio file (default: "default").
• Usage: Useful when the audio file is loaded into memory as bytes.

WithStreamAsync(Stream file, string fileName = "default")

• Description: Adds an audio file as a stream asynchronously.
• Parameters:
  • file: Stream representing the audio file.
  • fileName: Name of the audio file (default: "default").
• Usage: Ideal for large files streamed directly without loading entirely into memory.

2. Transcription

TranscriptAsync(CancellationToken cancellationToken = default)

• Description: Transcribes the audio into the input language.
• Returns: An AudioResult containing the transcription details.
• Usage: Extract text content from audio in its original language.

VerboseTranscriptAsSegmentsAsync(CancellationToken cancellationToken = default)

• Description: Transcribes the audio into a verbose representation in the input language.
• Returns: A VerboseSegmentAudioResult with detailed transcription data.
• Usage: Suitable for scenarios requiring detailed transcriptions with additional context or metadata.

VerboseTranscriptAsWordsAsync(CancellationToken cancellationToken = default)

• Description: Transcribes the audio into a verbose representation in the input language.
• Returns: A VerboseWordAudioResult with detailed transcription data.
• Usage: Suitable for scenarios requiring detailed transcriptions with additional context or metadata.

3. Translation

TranslateAsync(CancellationToken cancellationToken = default)

• Description: Translates audio content into English.
• Returns: An AudioResult containing the translated text.
• Usage: Convert audio content from any supported language to English.

VerboseTranslateAsSegmentsAsync(CancellationToken cancellationToken = default)

• Description: Translates audio into a verbose representation in English.
• Returns: A VerboseSegmentAudioResult with detailed translation data.
• Usage: Obtain comprehensive translation output with additional metadata.

VerboseTranslateAsWordsAsync(CancellationToken cancellationToken = default)

• Description: Translates audio into a verbose representation in English.
• Returns: A VerboseWordAudioResult with detailed translation data.
• Usage: Obtain comprehensive translation output with additional metadata.

4. Customization

WithPrompt(string prompt)

• Description: Adds a text prompt to guide the model's transcription or translation style.
• Parameters:
  • prompt: Text to provide contextual guidance or continue a previous segment.
• Usage: Helps maintain consistency or tailor the model’s output style.

WithTemperature(double temperature)

• Description: Sets the sampling temperature (range: 0 to 1). Higher values increase randomness, while lower values make output more deterministic.
• Parameters:
  • temperature: Value for controlling randomness.
• Usage: Adjusts the balance between creativity and focus in the output.

WithLanguage(Language language)

• Description: Specifies the input audio's language using ISO-639-1 codes.
• Parameters:
  • language: Language code of the input audio.
• Usage: Improves transcription/translation accuracy and reduces latency by specifying the language explicitly.

WithTranscriptionMinutes(int minutes)

• Description: Sets the number of minutes allocated for transcription tasks.
• Parameters:
  • minutes: Duration in minutes.
• Usage: Controls the time allocation for transcription operations.

WithTranslationMinutes(int minutes)

• Description: Sets the number of minutes allocated for translation tasks.
• Parameters:
  • minutes: Duration in minutes.
• Usage: Controls the time allocation for translation operations.

Create Transcription

Transcribes audio into the input language.

    var openAiApi = _openAiFactory.Create(name)!;
    var location = Assembly.GetExecutingAssembly().Location;
    location = string.Join('\\', location.Split('\\').Take(location.Split('\\').Length - 1));
    using var readableStream = File.OpenRead($"{location}\\Files\\test.mp3");

    var editableFile = new MemoryStream();
    readableStream.CopyTo(editableFile);
    editableFile.Position = 0;

    var results = await openAiApi.Audio
        .WithFile(editableFile.ToArray(), "default.mp3")
        .WithTemperature(1)
        .WithLanguage(Language.Italian)
        .WithPrompt("Incidente")
        .TranscriptAsync();

example for verbose transcription in segments

    var openAiApi = _openAiFactory.Create(name)!;
    var location = Assembly.GetExecutingAssembly().Location;
    location = string.Join('\\', location.Split('\\').Take(location.Split('\\').Length - 1));
    using var readableStream = File.OpenRead($"{location}\\Files\\test.mp3");

    var editableFile = new MemoryStream();
    readableStream.CopyTo(editableFile);
    editableFile.Position = 0;

    var results = await openAiApi.Audio
        .WithFile(editableFile.ToArray(), "default.mp3")
        .WithTemperature(1)
        .WithLanguage(Language.Italian)
        .WithPrompt("Incidente")
        .VerboseTranscriptAsSegmentsAsync();

    Assert.NotNull(results);
    Assert.True(results.Text?.Length > 100);
    Assert.StartsWith("Incidente tra due aerei di addestramento", results.Text);
    Assert.NotEmpty(results.Segments ?? []);

Create Translation

Translates audio into English.

    var openAiApi = _openAiFactory.Create(name)!;

    var location = Assembly.GetExecutingAssembly().Location;
    location = string.Join('\\', location.Split('\\').Take(location.Split('\\').Length - 1));
    using var readableStream = File.OpenRead($"{location}\\Files\\test.mp3");
    var editableFile = new MemoryStream();
    await readableStream.CopyToAsync(editableFile);
    editableFile.Position = 0;

    var results = await openAiApi.Audio
        .WithTemperature(1)
        .WithPrompt("sample")
        .WithFile(editableFile.ToArray(), "default.mp3")
        .TranslateAsync();