Partial information removal for a stream

EventStoreDB
#1

Hi, I need to partially remove information of a stream.

For example, imagine I have the stream Client+123 with the following data:

{
“ClientId”: “123”,
“Name”: “Luke Skywalker”,
“Address”: “fake address”,
“Country”: “Ireland”
}

If the client requests a removal of his personal data, we would have to remove the existing events and only keep the following data:

{
“ClientId”: “123”,
“Country”: “Ireland”
}

How can I achieve that? I’m wondering if it would work by making a soft delete of the stream, and then attaching a new event with the data I want to keep. But I’m not sure what would do the scavenging in that case.

#2

Without more details of the case at hand :
You can also use a “GDPRed” event,
a reaction listen to the GDPRed event , rewriting the stream and after that truncates it before the gdpred event
image

the reasons for the “GDPRed” event is

  • remembering the fact that it was an intentional removal
  • it can trigger any consumer ( projections & reaction ) to do their own cleanup

The actual physical removal of the data will only happen after a scavenge though

#3

We had a similar challenge, removing user data for GDPR compliance. We solved it by encrypting all sensitive user data keys. And if a user wants to be deleted, we simply throw away the encryption key.

2 Likes
#4

In many cases, the simplest solution is to keep PII in a different stream.

#5

I think we are going to need something like this, because the part of the data we want to remove might be dynamic. So it wouldn’t be easy to split that data in a different stream or encrypt it differently.

Regarding this approach, I have a couple of questions:

  1. How do you force a truncate for a stream? By doing a soft delete of the stream?
  2. If you truncate a stream and you add some new events after that, will the scavenge remove only the events created before the scavenge and keep the newer events?
#6

You can truncate the whole stream by deleting it (soft-delete is the default), or you can truncate the stream at any position by updating stream metadata https://github.com/Eventuous/eventuous/blob/0538e082162537fed9c3bb31eacef5707a2c4067/src/EventStore/src/Eventuous.EventStore/EsdbEventStore.cs#L172-L204

The numbering of events within the stream will keep increasing, so the versioning will always work correctly.

#7

That TruncateStream method is exactly what I was looking for, and I couldn’t find it in the documentation.

Thanks Alexey

#8

As you can see it’s not part of the client, it’s the code from my library. Feel free to use it :slight_smile: