Truncate Before/Max Count, Scavenging & Projections

EventStoreDB
#1

Hi Guys,

If I were to set the $tb or $maxCount on a stream’s metadata and I have a number of projections that would process events on those streams. I’m thinking of an archive scenario where I was looking to set the $tb on a stream to the archived position, but am wondering how that might affect subscriptions & projections. For my scenario I only need to keep the most recent event for a given category of streams provided that some projections and subscriptions have fully caught up to that position.

If the projection processing were to fall behind, will they still process the events in the stream that aren’t visible due to the max count / age / tb?

Does the projection’s processing position have an impact on scavenging?

Will those events (provided that scavenging hasn’t occurred) still be delivered if a catch up subscription is used (say if the subscription was on $et-XXXX)?

Hope my questions make sense.

Cheers,

Tom

#2

Hi Thomas

Luckily all fairly straightforward questions, though I fear with a different answer to what you want!

If the projection processing were to fall behind, will they still process the events in the stream that aren’t visible due to the max count / age / tb?

No

Does the projection’s processing position have an impact on scavenging?

No (other than temporary streams it may have created for e.g. results)

Will those events (provided that scavenging hasn’t occurred) still be delivered if a catch up subscription is used (say if the subscription was on $et-XXXX)?

Unlikely (I’d say no but haven’t verified this and I can see an edge case where it may still happen…)

Cheers

James

#3

Hi James,

I thought that might be the case and was planning to build for that, but thought I might ask to see if it wasn’t required.

Thanks for the quick reply.

Cheers,

Tom

#4

If an event is sent to the subscription before $tb is set or maxage etc is hit then yes it will be delivered (eg it could be in a buffer) it matters at the time of seeing the item not on it’s arrival

#5

out of curiosity in the case of many subscribers which should we wait for? What about when a subscriber no longer wants to receive messages? Doing this would change he subscription model from consumer driven to server driven where the server would have to track every subscriber and you would need a registration / deregistration process as well. This would be totally trivial to introduce on top of the server driven subscription mode (competing branch) but totally breaks a client driven model