What is Smuggler
-
Smuggler is a tool for moving data between databases.
Use it to export documents, indexes, and other items from a database as JSON,
and to import that data into another database. -
Smuggler is exposed through the
DocumentStore.Smugglerproperty. -
An export can write to a
.ravendbdumpfile or stream directly into another database.
An import reads that data back into a database. -
You can narrow what is transferred by collection, by item type,
and by applying a transform script to each document. -
In this article:
ForDatabase
By default, DocumentStore.Smuggler works on the database set in the DocumentStore.Database property.
To run Smuggler against a different database, switch to it with the ForDatabase method.
var northwindSmuggler = store
.Smuggler
.ForDatabase("Northwind");
Export
Export writes the selected data to a .ravendbdump file, or streams it directly into another database.
Use DatabaseSmugglerExportOptions to choose what to include.
Example: export indexes and documents to a file
// export only Indexes and Documents to a given file
var exportOperation = await store
.Smuggler
.ExportAsync(
new DatabaseSmugglerExportOptions
{
OperateOnTypes = DatabaseItemType.Indexes
| DatabaseItemType.Documents
},
@"C:\ravendb-exports\Northwind.ravendbdump",
token);
await exportOperation.WaitForCompletionAsync();
Export specific collections
By default, all collections are exported.
To export only specific collections, list their names in the Collections option.
// Export only the "Orders" and "Categories" collections to a file
var exportOperation = await store
.Smuggler
.ExportAsync(
new DatabaseSmugglerExportOptions
{
Collections = new List<string> { "Orders", "Categories" }
},
@"C:\ravendb-exports\Northwind.ravendbdump",
token);
await exportOperation.WaitForCompletionAsync();
See ExportAsync and DatabaseSmugglerExportOptions in the Syntax section.
Import
Import reads data from a .ravendbdump file or a stream and writes it into the database.
Use DatabaseSmugglerImportOptions to choose what to include.
Example: import documents from a file
// import only Documents from a given file
var importOperation = await store
.Smuggler
.ImportAsync(
new DatabaseSmugglerImportOptions
{
OperateOnTypes = DatabaseItemType.Documents
},
// import the .ravendbdump file that you exported (i.e. in the export example above)
@"C:\ravendb-exports\Northwind.ravendbdump",
token);
await importOperation.WaitForCompletionAsync();
Import specific collections
By default, all collections are imported.
To import only specific collections, list their names in the Collections option.
// Import only the "Orders" and "Categories" collections from the file
var importOperation = await store
.Smuggler
.ImportAsync(
new DatabaseSmugglerImportOptions
{
Collections = new List<string> { "Orders", "Categories" }
},
@"C:\ravendb-exports\Northwind.ravendbdump",
token);
await importOperation.WaitForCompletionAsync();
See ImportAsync and DatabaseSmugglerImportOptions in the Syntax section.
ImportIncrementalAsync
A RavenDB backup folder contains one full backup file and any number of incremental backup files accumulated since
the last full backup.
Use ImportIncrementalAsync to import all logical-backup files
from a backup folder into an existing database in a single call, instead of importing each file individually.
After finding all backup files in the specified folder, the method orders them chronologically and imports them in sequence.
Indexes and Subscriptions are excluded from all files except the last, ensuring only their most recent state is applied.
ImportIncrementalAsync imports your backup into an existing database.
To create a new database from backup, use RestoreBackupOperation.
Example
// Import all backup files from a backup folder into an existing database
await store.Smuggler.ImportIncrementalAsync(
new DatabaseSmugglerImportOptions(),
@"C:\RavenDB\Backups\Northwind.2024-01-15T12-00-00");
See ImportIncrementalAsync and DatabaseSmugglerImportOptions in the Syntax section.
TransformScript
TransformScript lets you modify or filter out each document during an export or import, using JavaScript that you provide.
The JavaScript engine is the same one used for patching operations, so you have identical syntax and capabilities, plus the ability to filter out a document by throwing a skip exception.
var id = this['@metadata']['@id'];
if (id === 'orders/999-A')
throw 'skip'; // filter-out
this.Freight = 15.3;
Syntax
Methods
Smuggler methods
- ForDatabase
- ExportAsync
- ImportAsync
- ImportIncrementalAsync
Switches Smuggler to operate on a database other than the document store's default database.
public DatabaseSmuggler ForDatabase(string databaseName);
Usage:
var northwindSmuggler = store.Smuggler.ForDatabase("Northwind");
| Parameter | Type | Description |
|---|---|---|
| databaseName | string | The database that Smuggler will operate on. |
| Return value | |
|---|---|
DatabaseSmuggler | A Smuggler instance scoped to the specified database. |
Exports the selected data to a file, to a stream, or to another database.
Task<Operation> ExportAsync(
DatabaseSmugglerExportOptions options,
string toFile,
CancellationToken token = default);
Task<Operation> ExportAsync(
DatabaseSmugglerExportOptions options,
Stream toStream,
CancellationToken token = default);
Task<Operation> ExportAsync(
DatabaseSmugglerExportOptions options,
DatabaseSmuggler toDatabase,
CancellationToken token = default);
Usage:
var exportOperation = await store.Smuggler.ExportAsync(options, toFile, token);
| Parameter | Type | Description |
|---|---|---|
| options | DatabaseSmugglerExportOptions | Options used during the export. See DatabaseSmugglerExportOptions. |
| toFile | string | Path to the file where exported data is written. |
| toStream | Stream | Stream that exported data is written to. |
| toDatabase | DatabaseSmuggler | A DatabaseSmuggler instance used as the destination. |
| token | CancellationToken | Token used to cancel the operation. |
| Return value | |
|---|---|
Operation | An Operation instance you can use to wait for completion and subscribe to progress events. |
Imports data from a file or a stream into the database.
Task<Operation> ImportAsync(
DatabaseSmugglerImportOptions options,
string fromFile,
CancellationToken cancellationToken = default);
Task<Operation> ImportAsync(
DatabaseSmugglerImportOptions options,
Stream stream,
CancellationToken token = default);
Usage:
var importOperation = await store.Smuggler.ImportAsync(options, fromFile, token);
| Parameter | Type | Description |
|---|---|---|
| options | DatabaseSmugglerImportOptions | Options used during the import. See DatabaseSmugglerImportOptions. |
| fromFile | string | Path to the file the data is imported from. |
| stream | Stream | Stream with the data to import. |
| cancellationToken | CancellationToken | Token used to cancel the operation, in the overload that imports from a file. |
| token | CancellationToken | Token used to cancel the operation, in the overload that imports from a stream. |
| Return value | |
|---|---|
Operation | An Operation instance you can use to wait for completion and subscribe to progress events. |
Imports the logical-backup files found in a backup folder, in chronological order.
Task ImportIncrementalAsync(
DatabaseSmugglerImportOptions options,
string fromDirectory,
CancellationToken cancellationToken = default);
Usage:
await store.Smuggler.ImportIncrementalAsync(options, fromDirectory, token);
| Parameter | Type | Description |
|---|---|---|
| options | DatabaseSmugglerImportOptions | Options applied to each file imported from the directory. Tombstones and CompareExchangeTombstones are added automatically. |
| fromDirectory | string | Path to the backup folder containing the backup files to import. |
| cancellationToken | CancellationToken | Token used to cancel the operation. |
| Return value | |
|---|---|
Task | Completes when all backup files in the folder have been imported. |
Classes
Smuggler options classes
- DatabaseSmugglerExportOptions
- DatabaseSmugglerImportOptions
Options that control what an export includes.
class DatabaseSmugglerExportOptions
{
DatabaseItemType OperateOnTypes
DatabaseRecordItemType OperateOnDatabaseRecordTypes
bool IncludeExpired
bool IncludeArtificial
bool IncludeArchived
bool RemoveAnalyzers
string TransformScript
int MaxStepsForTransformScript
string EncryptionKey
List<string> Collections
int? MaxReadOpsPerSecond
bool SkipCorruptedData
ExportCompressionAlgorithm? CompressionAlgorithm
}
| Property | Type | Description |
|---|---|---|
| Collections | List<string> | List of specific collections to export. If empty, all collections are exported. Default: empty |
| OperateOnTypes | DatabaseItemType | Indicates what should be exported. Default: Indexes, Documents, RevisionDocuments, Conflicts, DatabaseRecord, ReplicationHubCertificates, Identities, CompareExchange, Attachments, CounterGroups, Subscriptions, TimeSeries, TimeSeriesDeletedRanges |
| OperateOnDatabaseRecordTypes | DatabaseRecordItemType | Indicates what should be exported from the database record. Default: Client, ConflictSolverConfig, Expiration, ExternalReplications, PeriodicBackups, RavenConnectionStrings, RavenEtls, Revisions, Settings, SqlConnectionStrings, Sorters, SqlEtls, HubPullReplications, SinkPullReplications, TimeSeries, DocumentsCompression, Analyzers, LockMode, OlapConnectionStrings, OlapEtls, ElasticSearchConnectionStrings, ElasticSearchEtls, PostgreSQLIntegration, QueueConnectionStrings, QueueEtls, IndexesHistory, Refresh, DataArchival, QueueSinks, SnowflakeEtls, SnowflakeConnectionStrings, EmbeddingsGenerations, AiConnectionStrings, GenAiEtls, AiAgents |
| IncludeExpired | bool | Should expired documents be exported. Default: true |
| IncludeArtificial | bool | Should artificial documents be exported. Default: false |
| IncludeArchived | bool | Should archived documents be exported. Default: true |
| RemoveAnalyzers | bool | Should analyzers be removed from indexes. Default: false |
| TransformScript | string | JavaScript applied to every exported document. See TransformScript. |
| MaxStepsForTransformScript | int | Maximum number of steps the transform script can run before failing. Default: 10000 |
| EncryptionKey | string | Encryption key used to encrypt the exported file. Default: null |
| MaxReadOpsPerSecond | int? | Limits the read-operation rate during the export, to throttle the load placed on the database. Default: null (no limit) |
| SkipCorruptedData | bool | When true, the export continues if corrupted data is encountered (for example, lost compression dictionaries), recording an error entry in the result report instead of stopping. Default: false |
| CompressionAlgorithm | ExportCompressionAlgorithm? | Compression algorithm used for the exported file: Zstd or Gzip. Default: null |
Options that control what an import includes.
class DatabaseSmugglerImportOptions
{
DatabaseItemType OperateOnTypes
DatabaseRecordItemType OperateOnDatabaseRecordTypes
bool IncludeExpired
bool IncludeArtificial
bool IncludeArchived
bool RemoveAnalyzers
string TransformScript
int MaxStepsForTransformScript
string EncryptionKey
List<string> Collections
int? MaxReadOpsPerSecond
bool SkipCorruptedData
bool SkipRevisionCreation
}
| Property | Type | Description |
|---|---|---|
| Collections | List<string> | List of specific collections to import. If empty, all collections are imported. Default: empty |
| OperateOnTypes | DatabaseItemType | Indicates what should be imported. Default: Indexes, Documents, RevisionDocuments, Conflicts, DatabaseRecord, ReplicationHubCertificates, Identities, CompareExchange, Attachments, CounterGroups, Subscriptions, TimeSeries, TimeSeriesDeletedRanges |
| OperateOnDatabaseRecordTypes | DatabaseRecordItemType | Indicates what should be imported into the database record. Default: Client, ConflictSolverConfig, Expiration, ExternalReplications, PeriodicBackups, RavenConnectionStrings, RavenEtls, Revisions, Settings, SqlConnectionStrings, Sorters, SqlEtls, HubPullReplications, SinkPullReplications, TimeSeries, DocumentsCompression, Analyzers, LockMode, OlapConnectionStrings, OlapEtls, ElasticSearchConnectionStrings, ElasticSearchEtls, PostgreSQLIntegration, QueueConnectionStrings, QueueEtls, IndexesHistory, Refresh, DataArchival, QueueSinks, SnowflakeEtls, SnowflakeConnectionStrings, EmbeddingsGenerations, AiConnectionStrings, GenAiEtls, AiAgents |
| IncludeExpired | bool | Should expired documents be imported. Default: true |
| IncludeArtificial | bool | Should artificial documents be imported. Default: false |
| IncludeArchived | bool | Should archived documents be imported. Default: true |
| RemoveAnalyzers | bool | Should analyzers be removed from indexes. Default: false |
| TransformScript | string | JavaScript applied to every imported document. See TransformScript. |
| MaxStepsForTransformScript | int | Maximum number of steps the transform script can run before failing. Default: 10000 |
| EncryptionKey | string | Encryption key used to read an encrypted .ravendbdump file. Default: null |
| MaxReadOpsPerSecond | int? | Limits the read-operation rate during the import, to throttle the load placed on the database. Default: null (no limit) |
| SkipCorruptedData | bool | When true, the import continues if corrupted data is encountered, recording an error entry in the result report instead of stopping. Default: false |
| SkipRevisionCreation | bool | When true, revisions are not created for the imported documents. Default: false |