Skip to main content

Indexing Time Series

Time series indexes vs Document indexes​

Auto-Indexes:​

  • Time series index:
    Dynamic time series indexes are Not created in response to queries.

  • Document index:
    Auto-indexes are created in response to dynamic queries.

Data source:​

  • Time series index:

    • Time series indexes process segments that contain time series entries.
      The entries are indexed through the segment they are stored in, for example, using a LINQ syntax that resembles this one:
from segment in timeseries
from entry in segment
...

  • The following items can be indexed per index-entry in a time series index:

    • Values & timestamp of a time series entry
    • The entry tag
    • Content from a document referenced by the tag
    • Properties of the containing segment (see TimeSeriesSegment)

  • Document index:

    • The index processes fields from your JSON documents.
      Documents are indexed through the collection they belong to, for example, using this LINQ syntax:
from employee in employees
...

Query results:​

  • Time series index:
    When querying a time series index, each result item corresponds to the type defined by the index-entry in the index definition, (unless results are projected). The documents themselves are not returned.

  • Document index:
    The resulting objects are the document entities (unless results are projected).

Ways to create a time series index​

There are two main ways to create a time series index:

  1. Create a class that inherits from one of the following abstract index creation task classes:

  2. Deploy a time series index definition via PutIndexesOperation:

Examples of time series indexes​

Map index - index single time series from single collection:​

  • In this index, we index data from the "StockPrices" time series entries in the "Companies" collection (TradeVolume, Date).

  • In addition, we index the containing document id (DocumentID), which is obtained from the segment,
    and some content from the document referenced by the entry's Tag (EmployeeName).

  • Each tab below presents one of the different ways the index can be defined.

public class StockPriceTimeSeriesFromCompanyCollection : AbstractTimeSeriesIndexCreationTask<Company>
{
  // The index-entry:
  // ================
  public class IndexEntry
  {
      // The index-fields:
      // =================
      public double TradeVolume { get; set; }
      public DateTime Date { get; set; }
      public string CompanyID { get; set; }
      public string EmployeeName { get; set; }
  }
  
  public StockPriceTimeSeriesFromCompanyCollection()
  { 
      // Call 'AddMap', specify the time series name to be indexed 
      AddMap("StockPrices", timeseries =>
              from segment in timeseries
              from entry in segment.Entries
              
              // Can load the document referenced in the TAG:
              let employee = LoadDocument<Employee>(entry.Tag)
              
              // Define the content of the index-fields:
              // =======================================
              select new IndexEntry()
              {
                  // Retrieve content from the time series ENTRY:
                  TradeVolume = entry.Values[4],
                  Date = entry.Timestamp.Date,
                  
                  // Retrieve content from the SEGMENT:
                  CompanyID = segment.DocumentId,
                  
                  // Retrieve content from the loaded DOCUMENT:
                  EmployeeName = employee.FirstName + " " + employee.LastName 
              });
  }
}
  • Querying this index, you can retrieve the indexed time series data while filtering by any of the index-fields.
using (var session = documentStore.OpenSession())
{
  // Retrieve time series data for the specified company:
  // ====================================================
  List<StockPriceTimeSeriesFromCompanyCollection.IndexEntry> results = session
     .Query<StockPriceTimeSeriesFromCompanyCollection.IndexEntry,
         StockPriceTimeSeriesFromCompanyCollection>()
     .Where(x => x.CompanyID == "Companies/91-A")
     .ToList();
}

// Results will include data from all 'StockPrices' entries in document 'Companies/91-A'.

Map index - index all time series from single collection:​

public class AllTimeSeriesFromCompanyCollection : AbstractTimeSeriesIndexCreationTask<Company>
{
  public class IndexEntry
  {
      public double Value { get; set; }
      public DateTime Date { get; set; }
  }
  
  public AllTimeSeriesFromCompanyCollection()
  {
      // Call 'AddMapForAll' to index ALL the time series in the 'Companies' collection 
      // ==============================================================================
      AddMapForAll(timeseries =>
          from segment in timeseries
          from entry in segment.Entries
              
          select new IndexEntry()
          {
              Value = entry.Value,
              Date = entry.Timestamp.Date
          });
  }
}

Map index - index all time series from all collections:​

// Inherit from AbstractTimeSeriesIndexCreationTask<object>
// Specify <object> as the type to index from ALL collections
// ==========================================================

public class AllTimeSeriesFromAllCollections : AbstractTimeSeriesIndexCreationTask<object>
{
  public class IndexEntry
  {
      public double Value { get; set; }
      public DateTime Date { get; set; }
      public string DocumentID { get; set; }
  }
  
  public AllTimeSeriesFromAllCollections()
  {
      AddMapForAll(timeseries =>
          from segment in timeseries
          from entry in segment.Entries
              
          select new IndexEntry()
          {
              Value = entry.Value,
              Date = entry.Timestamp.Date,
              DocumentID = segment.DocumentId
          });
  }
}

Multi-Map index - index time series from several collections:​

public class Vehicles_ByLocation : AbstractMultiMapTimeSeriesIndexCreationTask
{
  public class IndexEntry
  {
      public double Latitude { get; set; }
      public double Longitude { get; set; }
      public DateTime Date { get; set; }
      public string DocumentID { get; set; }
  }
  
  public Vehicles_ByLocation()
  {
      // Call 'AddMap' for each collection you wish to index
      // ===================================================
      
      AddMap<Plane>(
          "GPS_Coordinates",timeSeries =>
              from segment in timeSeries
              from entry in segment.Entries
              select new IndexEntry()
              {
                  Latitude = entry.Values[0],
                  Longitude = entry.Values[1],
                  Date = entry.Timestamp.Date,
                  DocumentID = segment.DocumentId
              });

      AddMap<Ship>(
          "GPS_Coordinates",timeSeries =>
              from segment in timeSeries
              from entry in segment.Entries
              select new IndexEntry()
              {
                  Latitude = entry.Values[0],
                  Longitude = entry.Values[1],
                  Date = entry.Timestamp.Date,
                  DocumentID = segment.DocumentId
              });
  }
}

Map-Reduce index:​

public class TradeVolume_PerDay_ByCountry : 
  AbstractTimeSeriesIndexCreationTask<Company, TradeVolume_PerDay_ByCountry.Result>
{
  public class Result
  {
      public double TotalTradeVolume { get; set; }
      public DateTime Date { get; set; }
      public string Country { get; set; }
  }

  public TradeVolume_PerDay_ByCountry()
  {
      // Define the Map part:
      AddMap("StockPrices", timeSeries =>
          from segment in timeSeries
          from entry in segment.Entries
          
          let company = LoadDocument<Company>(segment.DocumentId)
          
          select new Result
          {
              Date = entry.Timestamp.Date,
              Country = company.Address.Country,
              TotalTradeVolume = entry.Values[4]
          });

      // Define the Reduce part:
      Reduce = results =>
          from r in results
          group r by new {r.Date, r.Country}
          into g
          select new Result
          {
              Date = g.Key.Date,
              Country = g.Key.Country,
              TotalTradeVolume = g.Sum(x => x.TotalTradeVolume)
          };
  }
}

Syntax​

AbstractTimeSeriesIndexCreationTask​

// To define a Map index inherit from:
// ===================================
public abstract class AbstractTimeSeriesIndexCreationTask<TDocument> { }
// Time series that belong to documents of the specified `TDocument` type will be indexed. 

// To define a Map-Reduce index inherit from:
// ==========================================
public abstract class AbstractTimeSeriesIndexCreationTask<TDocument, TReduceResult> { }
// Specify both the document type and the reduce type

// Methods available in AbstractTimeSeriesIndexCreationTask class:
// ===============================================================

// Set a map function for the specified time series
protected void AddMap(string timeSeries,
  Expression<Func<IEnumerable<TimeSeriesSegment>, IEnumerable>> map);

// Set a map function for all time series 
protected void AddMapForAll(
  Expression<Func<IEnumerable<TimeSeriesSegment>, IEnumerable>> map);

AbstractMultiMapTimeSeriesIndexCreationTask​

// To define a Multi-Map index inherit from:
// =========================================
public abstract class AbstractMultiMapTimeSeriesIndexCreationTask { }

// Methods available in AbstractMultiMapTimeSeriesIndexCreationTask class:
// =======================================================================

// Set a map function for all time series with the specified name
// that belong to documents of type `TSource`
protected void AddMap<TSource>(string timeSeries,
  Expression<Func<IEnumerable<TimeSeriesSegment>, IEnumerable>> map);

// Set a map function for all time series that belong to documents of type `TBase`
// or any type that inherits from `TBase`
protected void AddMapForAll<TBase>(
  Expression<Func<IEnumerable<TimeSeriesSegment>,IEnumerable>> map);

AbstractJavaScriptTimeSeriesIndexCreationTask​

// To define a JavaScript index inherit from:
// ==========================================
public abstract class AbstractJavaScriptTimeSeriesIndexCreationTask
{    
  public HashSet<string> Maps; // The set of JavaScript map functions for this index
  protected string Reduce;     // The JavaScript reduce function
}

Learn more about JavaScript indexes in JavaScript Indexes.

TimeSeriesIndexDefinition​

public class TimeSeriesIndexDefinition : IndexDefinition

While TimeSeriesIndexDefinition is currently functionally equivalent to the regular IndexDefinition class from which it inherits, it is recommended to use TimeSeriesIndexDefinition when creating a time series index definition in case additional functionality is added in future versions of RavenDB.

TimeSeriesIndexDefinitionBuilder​

public class TimeSeriesIndexDefinitionBuilder<TDocument>
{ 
  public TimeSeriesIndexDefinitionBuilder(string indexName = null)  
}

Note:

  • Currently, class TimeSeriesIndexDefinitionBuilder does Not support API methods from abstract class AbstractCommonApiForIndexes, such as LoadDocument or Recurse.

  • Use one of the other index creation methods if needed.

TimeSeriesSegment​

  • Segment properties include the entries data and aggregated values that RavenDB automatically updates in the segment's header.

  • The following segment properties can be indexed:

public sealed class TimeSeriesSegment
{
  // The ID of the document this time series belongs to
  public string DocumentId { get; set; }

  // The name of the time series this segment belongs to
  public string Name { get; set; }

  // The smallest values from all entries in the segment
  // The first array item is the Min of all first values, etc.
  public double[] Min { get; set; }

  // The largest values from all entries in the segment
  // The first array item is the Max of all first values, etc.
  public double[] Max { get; set; }

  // The sum of all values from all entries in the segment 
  // The first array item is the Sum of all first values, etc.
  public double[] Sum { get; set; }

  // The number of entries in the segment
  public int Count { get; set; }

  // The timestamp of the first entry in the segment
  public DateTime Start { get; set; }

  // The timestamp of the last entry in the segment
  public DateTime End { get; set; }

  // The segment's entries themselves
  public TimeSeriesEntry[] Entries { get; set; }
}
  • These are the properties of a TimeSeriesEntry which can be indexed:
public class TimeSeriesEntry
{
  public DateTime Timestamp;
  public string Tag;
  public double[] Values;

  // This is exactly equivalent to Values[0]
  public double Value;
}