Skip to main content

Monitoring: FAQ

Frequently Asked Questions​

My requests are slow - where do I start?​

Open the Cluster Dashboard for the whole-cluster picture: its CPU, Traffic, and Indexing widgets report each metric for every cluster node.
Then open the Running Queries view to list the queries the server is currently running, and inspect the requests themselves using Traffic Watch.
When a query's duration exceeds the configured threshold, the server raises a request-latency performance hint on its own.

Can I stop a query that runs for too long?​

Yes. The Running Queries view lists the queries running on the node you are connected to, and you can abort a query from its row.
The abort action does not stop the query on the spot. RavenDB signals the query to stop, and the query ends shortly afterwards.

Is the disk the bottleneck?​

The IO Stats views draw a timeline of the disk operations RavenDB performs. Use these views to spot the operations that are slow to complete.
The IO Stats widget on the cluster dashboard reads per-drive counters from the operating system, like IOPS and throughput.
When disk writes are extremely slow, the server raises a slow-writes performance hint.

Disk space is running out - what is taking it?​

The Storage Reports show the disk space a database uses, broken down by component, and the storage of the server-wide data RavenDB keeps outside any user database.
The Storage widget reports the used disk space and total capacity of every cluster node at a glance.
When disk space is not reclaimed after documents are deleted, see I deleted documents, but the database keeps growing below.

I deleted documents, but the database keeps growing​

RavenDB replaces every deleted document with a marker called a tombstone, so that indexing, replication, and the ETL and incremental backup tasks can handle the deletion.
Tombstones take up disk space until RavenDB removes them, and the periodic tombstone cleanup clears only the tombstones that no process still needs.
When a process lags behind, tombstones accumulate, and the space they take grows with every deletion.
The Tombstones view shows the progress of this cleanup for each collection, and the number of tombstones each process still has to handle.

An ETL or AI task stopped working - how do I find out why?​

When an ETL task or an AI task (Embeddings Generation or GenAI) fails to process an item or a batch, the failure is stored as a task error.
The Task Errors view lists these errors, and shows each task's health classification: Healthy, Impaired, or Failed.
To follow the task as it runs, open the Ongoing Task Stats view. This view draws the task's operations over time.

How do I get alerted before trouble?​

The server raises alerts in Studio's notification center when it detects a condition that requires your attention, like low disk space.
To be alerted outside Studio, connect an external monitoring tool.
The Zabbix community template includes triggers for common operational concerns, like an expiring certificate, and a Prometheus server can raise alerts when a metric crosses a threshold you have set.

How do I silence a notification I do not need?​

Use the Notifications.FilterOut configuration key to filter out the notifications you do not need: a whole notification type, or only the notifications raised for a specific reason.
The Notification center article lists the values this key accepts, and the way to read an alert's Reason.

The server hangs or burns CPU - what is it doing?​

The Stack Traces view captures the stack traces of all server threads at once, and draws the traces as a graph that groups threads running the same code, so you can figure out the exact code a misbehaving server runs.
To capture the stack trace of a single thread rather than that of all threads, use the advanced-debug view's Threads Runtime Info tab. This tab also lists the server's threads and the CPU each thread uses, so you can locate the thread behind the load.

How do I raise the logging detail without restarting the server?​

Use Studio's Admin Logs view to change the logging settings, including the minimum logging level.
The settings take effect immediately, without a server restart.
You can also make the same change from your own code, using the logging client API.

How do I read RavenDB metrics in an external monitoring tool?​

RavenDB exposes its metrics in the formats common monitoring systems read, so an external tool can collect RavenDB metrics the same way it collects metrics from any other service:

  • A tool that reads JSON over HTTP, like Telegraf, can poll the JSON monitoring endpoints.
  • A Prometheus server can collect RavenDB metrics through the Prometheus endpoint, store their values over time, graph them, and raise alerts.
  • Zabbix, PRTG, and other SNMP-based tools can read the metrics as SNMP OIDs.
  • An OpenTelemetry Collector, or any other consumer of OTLP (OpenTelemetry Protocol), can receive the metrics RavenDB sends over the protocol.

What do I send the support team when I report an issue?​

When you encounter an incident or an issue with RavenDB and turn to the support team, the team needs relevant information to diagnose and resolve the issue.
Use the Collect Info for Support article to learn about collecting this information, including the creation of a debug package and the download of server logs.
Before sending a debug package to support, you can inspect it yourself using the Debug Package Analyzer, and possibly locate the problem on your own.

In this article