Skip to main content

GenAI Integration: API

  • A GenAI task leverages an AI model to enable intelligent processing of documents in runtime.

    • The task is associated with a document collection and with an AI model.
    • It is an ongoing task that:
      1. Continuously monitors the collection;
      2. Whenever needed, like when a document is added to the collection, generates user-defined context objects based on the source document data;
      3. Passes each context object to the AI model for further processing;
      4. Receives the AI model's JSON-based results;
      5. And finally, runs a user-defined script that potentially acts upon the results.

  • The main steps in defining a GenAI task are:

  • In this article:

Defining a Connection string​

  • Choose the model to connect with, by what you need from your GenAI task.
    E.g., If you require security and speed above all for the duration of a rapid development phase, you may prefer a local AI service like Ollama.
  • Make sure you define the correct service: both Ollama and OpenAI are supported but you need to pick an Ollama/OpenAI service that supports generative AI, like Ollama llama3.2 or OpenAI gpt-4o-mini.
  • Learn more about connection strings here.

Example:​

using (var store = new DocumentStore())
{
  // Define the connection string to Ollama
  var connectionString = new AiConnectionString
  {
      // Connection string name & identifier
      Name = "ollama-cs", 
      
      // Ollama connection settings
      OllamaSettings = new OllamaSettings(
          // LLM model for text generation
          model: "llama3.2",
          // local URL
          uri: "http://localhost:11434/")
  };
  
  // Deploy the connection string to the server
  var operation = new PutConnectionStringOperation<AiConnectionString>(connectionString);
  var putConnectionStringResult = store.Maintenance.Send(operation);
}

Syntax:​

public class AiConnectionString
{
  public string Name { get; set; }
  public string Identifier { get; set; }
  public OllamaSettings OllamaSettings { get; set; }
  ...
}

public class OllamaSettings : AbstractAiSettings
{
  public string Model { get; set; }
  public string Uri { get; set; }
}

Defining the GenAI task​

  • Define a GenAI task using a GenAiConfiguration object.
  • Run the task using AddGenAiOperation.
GenAiConfiguration config = new GenAiConfiguration
{
  // Task name
  Name = "FilterSpam",

  // Unique task identifier
  Identifier = "FilterSpam",

  // Connection string to AI model
  ConnectionStringName = "ollama-cs",

  // Task is enabled
  Disabled = false,

  // Collection associated with the task
  Collection = "Posts",

  // Context generation script - format for objects to be sentto the AI model
  GenAiTransformation = new GenAiTransformation {
      Script = @"
          for(const comment of this.Comments)
          {
              ai.genContext({Text: comment.Text, Author: comment.Author, Id: comment.Id});}"
  },

  // AI model Prompt - the instructions sent to the AI model
  Prompt = "Check if the following blog post comment is spam or not",

  // Sample object - a sample response object to format the AI model's replies by
  SampleObject = JsonConvert.SerializeObject(
      new
      {
          Blocked = true,
          Reason = "Concise reason for why this comment was marked as spam or ham"
      }),

  // Update script - specifies what to do with AI model replies
  UpdateScript = @"    
          // Find the comment
          const idx = this.Comments.findIndex(c => c.Id == $input.Id);  
          // Was detected as spam
          if($output.Blocked)
          {
              // Remove this comment
              this.Comments.splice(idx, 1);
          }",

  // Max concurrent connections to AI model
  MaxConcurrency = 4
};

// Run the task
var GenAiOperation = new AddGenAiOperation(config);
var addAiIntegrationTaskResult = store.Maintenance.Send(GenAiOperation);

GenAiConfiguration​

ParametersTypeDescription
NamestringTask name
IdentifierstringUnique task identifier, embedded in documents metadata to indicate they were processed along with hash codes for their processed parts
ConnectionStringNamestringConnection string name
DisabledboolDetermines whether the task is enabled or disabled
CollectionstringName of the document collection associated with the task
GenAiTransformationGenAiTransformationContext generation script - format for objects to be sent to the AI model
PromptstringAI model Prompt - the instructions sent to the AI model
SampleObjectstringA sample response object to format the AI model's replies by
If both a SampleObject and a JsonSchema are provided the schema takes precedence
JsonSchemastringA JSON schema to format the AI model's replies by
If both a SampleObject and a JsonSchema are provided the schema takes precedence
UpdateScriptstringUpdate script - specifies what to do with AI model replies
MaxConcurrencyintMax concurrent connections to the AI model (each connection serving a single context object