Processing Invoices Using Data Subscriptions

Egor Shamanaev

Development Team Lead

Published on December 8, 2024

Tags:DemoBackground TasksC#Use Case

In this article we will tackle the problem of processing invoices in an asynchronous manner using the RavenDB Data subscriptions feature.

We will create a data subscription on Orders collection, and use a Subscription Worker to process the newly added Orders documents. In this particular article we are going to process in an ongoing fashion, but since Subscriptions state is persisted, it can be a process that runs on a schedule, like overnight or on weekends.

In the subscription batch processing we will calculate the overall products cost, prepare the invoice PDF file and store it as an Attachment to the Invoices document.

Additional subscription can be defined for processing the Invoices documents, and sending an email with an Attachment that was created.

Intro

Typically after paying for your goods on an online store you would get the confirmation right away, but the invoice will be sent as a separate email afterwards. Did you ever wonder why it works this way? The reason is that the store wants to confirm the purchase immediately, and do the actual processing of the order in the background, so you can return to the shop homepage and possibly purchase even more.

That's from the business point of view, but what about user experience?

Processing invoices synchronously would hurt online store responsiveness, leading to long response times. Think about waiting a few minutes for order confirmation. The customer may refresh or even close the page, which would cancel the order.

Breakdown

After a customer adds a new order and gets confirmation with an order identifier, we want to start the invoice processing.

Let's break down the invoice processing into steps:

1. Get & start processing newly added Order document
2. Load the list of ordered products
3. Calculate order total sum
4. Generate PDF and save it to memory stream
5. Mark the Order document with InvoiceCreated=true (so it will get processed only once)
6. Add the PDF stream as attachment to the order document
7. Save changes to the RavenDB database

Document model

We will have Orders, Products and Invoices collections.

public class Order  
{  
    public string Id { get; set; }  
    public List\<LineItem\> LineItems { get; set; }  
    public string Address { get; set; }  
    public DateTime OrderDateUtc { get; set; }  
    public bool InvoiceCreated { get; set; }  
    public string InvoiceId { get; set; }  
}

public class LineItem  
{  
    public string ProductId { get; set; }  
    public decimal TotalPrice { get; set; }  
    public int Quantity { get; set; }  
}

public class Product  
{  
    public string Id { get; set; }  
    public string Name { get; set; }  
    public string Description { get; set; }  
    public decimal Price { get; set; }  
}

public class Invoice  
{  
    public string Id { get; set; }  
    public string OrderId { get; set; }  
    public bool EmailSent { get; set; }  
}

Subscription

The subscription task definition will be on Orders collection on documents that have InvoiceCreated = false.

await DocumentStore.Subscriptions.CreateAsync(new SubscriptionCreationOptions  
{  
    Name \= \_subsName,  
    Query \= "from Orders where InvoiceCreated \= false"  
});

The subscription worker (subscription.Run() method) will receive a batch of Orders documents each time. Inside this batch we will process each Orders document and prepare an invoice PDF. Please see the full code below.

await using var subscription = DocumentStore.Subscriptions.GetSubscriptionWorker<Order>(_subsName);

await subscription.Run(async batch =>
{
    var streams = new List<Stream>();
    try
    {
        var session = batch.OpenAsyncSession();
        foreach (var item in batch.Items)
        {
            var order = item.Result;
            if (order.InvoiceCreated)
            {
                // in case of fail over we might get an already processed items 
                // we have to check if we already created the invoice
                continue;
            }

            order.InvoiceCreated = true;

            if (order.LineItems.Count == 0)
            {
                // no products, we can't create an invoice, but need to save the order
                continue;
            }
            else
            {
                // load products data
                var products = await session.LoadAsync<Product>(order.LineItems.Select(x => x.ProductId));

                var invoice = new Invoice()
                {
                    EmailSent = false,
                    OrderId = order.Id
                };
                await session.StoreAsync(invoice);
                var invoiceId = session.Advanced.GetDocumentId(invoice);
                var mem = await CreateInvoiceForOrderAsync(invoiceId, order, products);
                MemoryStream stream = new MemoryStream(mem);
                streams.Add(stream);
                session.Advanced.Attachments.Store(
                    invoice,
                    $"Invoice_{order.Id}_{order.OrderDateUtc}.pdf",
                    stream,
                    "application/pdf"
                );
            }
        }
        await session.SaveChangesAsync();
    }
    finally
    {
        foreach (var stream in streams)
        {
            await stream.DisposeAsync();
        }
    }
});

The subscription worker is opened using default SubscriptionWorkerOptions, which means the subscription worker strategy is OpenIfFree.

The code here is simply creation of a subscription worker. Additional examples can be found in the Subscription Consumption Examples documentation article.

In the batch processing code, we check order.InvoiceCreated and if it is true we skip the item. The reason for that is that in case of subscription connection failover we might receive an item a second time, meaning we could get an Order that we already created an invoice for.

After checking the value of order.InvoiceCreated, we open a Session and load the related Products that were ordered.

Then we set order.InvoiceCreated to true. We have to set this property to true because after we add the invoice, the Order document will be updated, and setting order.InvoiceCreated to true ensures the updated Order document will no longer match the Subscription criteria.

The session in a subscription is bound to the processing node, so we can be sure that editing the Order document in the subscription session edits the document on the same server that is processing the subscription.

Afterwards, if there are products, we create an Invoice document and call the CreateInvoiceForOrderAsync() method, which prepares the PDF file and returns it (see the method code below). We then load the PDF file into a MemoryStream and store it as an attachment of the created Invoice document.
The last step is to call session.SaveChangesAsync(), which persists the changes to the database.

PDF Generation Implementation

The CreateInvoiceForOrderAsync() method calculates the overall product costs, prepares the invoice PDF document, and returns it as a byte[].

private Task<byte[]> CreateInvoiceForOrderAsync(string invoiceId, Order order, Dictionary<string, Product> products)
{
    // we want to create a new pdf invoice for the order
    // we will generate the pdf and save it as attachment to the order
    // we will also update the order to mark that the invoice was created

    using var memStream = new MemoryStream();
    using var document = new Document(
        new PdfDocument(
            new PdfWriter(memStream, new WriterProperties())
        )
    );

    document.Add(new Paragraph(new Text($"INVOICE '{invoiceId}':").SetBold()));
    document.Add(new LineSeparator(new DottedLine()));
    document.Add(new Paragraph(new Text("")));

    document.Add(new Paragraph(new Text("ORDER").SetBold()));
    document.Add(new Paragraph($"Order Id: {order.Id}"));
    document.Add(new Paragraph($"Order Date: {order.OrderDateUtc}"));

    document.Add(new LineSeparator(new DashedLine()));
    document.Add(new Paragraph("PRODUCTS").SetBold());

    var total = 0m;

    foreach (var item in order.LineItems)
    {
        var product = products[item.ProductId];

        document.Add(new Paragraph($"Quantity: {item.Quantity}"));
        document.Add(new Paragraph($"Product Name: {product.Name}"));
        document.Add(new Paragraph($"Product Description: {product.Description}"));
        document.Add(new Paragraph($"Products Price: {item.TotalPrice}"));

        total += item.TotalPrice;
    }

    document.Add(new LineSeparator(new DashedLine()));

    var totalParagraph = new Paragraph();
    totalParagraph.Add(new Text("TOTAL: ").SetBold());
    totalParagraph.Add($"{total}$");

    document.Add(totalParagraph);

    document.Close();

    return Task.FromResult(memStream.ToArray());
}

Summary

• Why async invoice processing matters: Synchronous invoice generation hurts store responsiveness and risks order cancellations. Offloading it to a background worker keeps confirmation instant.
• Subscription setup: A RavenDB Data Subscription on the Orders collection filters for documents where InvoiceCreated = false, ensuring each order is picked up exactly once.
• Batch processing: The subscription worker loads related Products, calculates the order total, generates a PDF using iText, and stores it as an attachment on the corresponding Invoice document.
• Reliability and scheduling: Subscription state is persisted in RavenDB, so the worker handles connection failovers safely and can run continuously or on a timed schedule such as overnight.