Effective Backup Strategies for Azure Tables

Abstract:

In the dynamic world of cloud services, where data security and backup are of utmost importance, Azure Tables in the Microsoft Azure cloud platform presents a unique challenge. This technical post delves into the practical and efficient backup strategies for Azure Tables, highlighting various tools and methods to safeguard your data.

Azure Tables backup strategies

Azure Tables, robust and scalable for storing structured NoSQL data, lacks a specific built-in backup functionality. This poses a significant challenge for users needing to secure their data against potential incidents or data loss. While Azure Data Factory’s “Copy Data” component can copy tables to another Azure storage service or even to a non-Azure service, it might be an overkill for simple backup needs due to its extensive features and potential extra costs.

An effective approach involves using Container Apps and Azure Functions to trigger scheduled backups. These can retrieve data from Azure Tables and store it in Azure Blob Storage or other services. Additionally, beyond the familiar SDK, there are other tools like PowerShell cmdlets, CLI commands, and AzCopy, all utilizing the Azure API.

AzCopy emerges as a particularly effective tool for rapidly performing operations on Azure Blob Storage using simple command lines. It allows for automated processes through shell commands in languages such as C#. In a C# context, these shell commands can be executed using Process.Start. This method enables the dynamic construction of AzCopy commands necessary for copying a table to a blob.

After exporting the data to the blob storage you could be able to establish a blob storage automated back up strategy.

Example of AzCopy Command to backup a table:

AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:https://myaccount.blob.core.windows.net/mycontainer/ /SourceKey:key1 /DestKey:key2

Restoring Data with AzCopy:

AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer /Dest:https://myaccount.table.core.windows.net/mytable /SourceKey:key1 /DestKey:key2 /Manifest:"myaccount_mytable_20140103T112020.manifest" /EntityOperation:"InsertOrReplace"

This mechanism is so flexible that you can even split the tables to adjust them to an specific exported file size. Also, you can take advantage of the concurrency to export the tables in parallel based on the PKs while setting up the number of concurrent threads using the /NC option. I recommend that you adjust the paralelism to the number of cores you have in the processing unit you use to make the backup.

AzCopy now also supports parameter files (response files) for easier and more flexible configuration.

You can find all the details for the implementation of these recomendations here

Alternatively, a more conventional solution involves mixing the Azure.Data.Tables SDK with the Data Movement library for efficient uploading to Blob Storage. This approach offers advantages like file compression and not needing AzCopy’s executable in the app distribution. Comparatively, AzCopy simplifies the export process into a single line of command, whereas using the SDK requires manual coding. AzCopy can also be used to download tables to a local disk for compression before sending to Blob Storage, which can save storage space.

Another option is migrating your Azure Tables solution to CosmosDB. It provides automatic backup options and the existing code to manage your tables will work the same with CosmosDB; but it comes with higher costs due to its additional features.