Skip to main content

Configuration: ETL Options

ETL.ExtractAndTransformTimeoutInSec​

Number of seconds after which extraction and transformation will end and loading will start.

  • Type: int
  • Default: 30
  • Scope: Server-wide or per database

ETL.MaxNumberOfExtractedDocuments​

  • Max number of extracted documents in an ETL batch.
  • If value is not set, or set to null, the number of extracted documents fallbacks to ETL.MaxNumberOfExtractedItems value.
  • Type: int
  • Default: 8192
  • Scope: Server-wide or per database

ETL.MaxBatchSizeInMb​

  • Maximum size in megabytes of a batch of data (documents and attachments) that will be sent to the destination as a single batch after transformation.
  • If value is not set, or set to null, the size of the batch isn't limited in the processed ETL batch.
  • Type: Size
  • Size Unit: Megabytes
  • Default: 64
  • Scope: Server-wide or per database

ETL.MaxFallbackTimeInSec​

  • Maximum number of seconds the ETL process will be in a fallback mode after a load connection failure to a destination.
  • The fallback mode means suspending the process.
  • Type: int
  • Default: 900
  • Scope: Server-wide or per database

ETL.MaxNumberOfExtractedItems​

  • Max number of extracted items (documents, counters, etc) in an ETL batch.
  • If value is not set, or set to null, the number of extracted items isn't limited in the processed ETL batch.
  • Type: int
  • Default: 8192
  • Scope: Server-wide or per database

ETL.OLAP.MaxNumberOfExtractedDocuments​

Max number of extracted documents in OLAP ETL batch.

  • Type: int
  • Default: 64 * 1024
  • Scope: Server-wide or per database

ETL.ProcessHealthStatusFailedThreshold​

  • Error-rate threshold for the Failed task health state. A task whose recent error rate exceeds this value is classified as Failed.
  • See the Task errors page to learn how the rate is calculated and how to set a value.
  • Type: float
  • Default: 0.9
  • Range: [0, 1]
  • Scope: Server-wide or per database

ETL.ProcessHealthStatusImpairedThreshold​

  • Error-rate threshold for the Impaired task health state. A task whose recent error rate exceeds this value is classified as Impaired.
  • See the Task errors page to learn how the rate is calculated and how to set a value.
  • Type: float
  • Default: 0.1
  • Range: [0, 1]
  • Scope: Server-wide or per database

ETL.Queue.AzureQueueStorage.TimeToLiveInSec​

Lifespan of a message in the queue.

  • Type: int
  • Default: 604800 (7 days)
  • Scope: Server-wide or per database

ETL.Queue.AzureQueueStorage.VisibilityTimeoutInSec​

How long a message is hidden after being retrieved but not deleted.

  • Type: int
  • Default: 0
  • Scope: Server-wide or per database

ETL.Queue.Kafka.InitTransactionsTimeoutInSec​

Timeout to initialize transactions for the Kafka producer.

  • Type: int
  • Default: 60
  • Scope: Server-wide or per database

ETL.Raven.LoadRequestTimeoutInSec​

  • Number of seconds after which the Raven ETL load request will time out.
  • Can be overridden per task by setting the LoadRequestTimeoutInSec property in the Raven ETL configuration, or via Studio.
  • Type: int
  • Default: 300
  • Scope: Server-wide or per database

ETL.SQL.CommandTimeoutInSec​

  • Number of seconds after which the SQL command will time out.
  • Can be overridden by setting the CommandTimeout property in the SQL ETL configuration.
  • Type: int?
  • Default: null (use provider default)
  • Scope: Server-wide or per database

In this article