Data Cleanup and Deletion

EventStoreDB
1

Hi Team, Can we delete or cleanup data of EventStore automatically to free up storage space?

2

Could you please be a bit more specific? Do you want to expire events, delete streams, or kill the whole database?

3

Sorry for the delay in response, but I want to cleanup expired events or which is already processed and life cycle of event is completed.

4

There are multiple ways to delete events from ESDB:

  • Set $maxCount or $maxAge stream metadata fields
  • Truncate streams
  • Delete streams
    (all described on the same page, different sections)

Deleted events don’t get removed from the log immediately as the log is immutable and can’t be manipulated at random. You’d need to execute the scavenge operation to get rid of the deleted events in the log. Scavenging is well-documented now, so you should be able to understand how it works, and how to run it.

There’s no way to do scavenging automatically at the moment, but you can have a scheduled cron job that would run at the time you define. Remember these things:

  • do not run scavenge during peak load times
  • do not execute scavenge on all the nodes at the same time
    You can, for example, scavenge one node on Saturday night, then one node on Sunday afternoon, and the third node on Sunday night (just an example)