Projections - Idempotency and event order

EventStoreDB
1

Hi! I have a question about projections (client ones).

I would like to use a projection to keep the balance of debits and credits of a bank account. I need to ensure that this balance is accurate because I will perform business checks reading this model.

I will have an stream per account, like : account-{id}

I have planned to use:

fromCategory("account")
            .foreachStream()
            .when({ // the code

My questions:

  • Are projections idempotent? I mean, are they ensuring that an event is never processed twice?
  • Are projections atomic? I mean, are they ensuring that one event is processed at time for projection, so no concurrency errors?
  • Are projections ensuring order of events? Meaning, there are no race conditions by which I can receive out-of-order events

I would like to understand how they behave since I din’t find this in your docs and testing edge cases it is kind of difficult.

Thanks in advance!

2

In short yes , idempotency , 1 event at time and ordered

I’d suggest you look at this :

( the last one about ordering is relaxed when using fromStreams() see here )

3

Cool! Thanks for your quick answer. It would be great if you could add these capabilities in the docs, maybe a new section, since it is not clear checking the docs.

1 Like
4

Hi again, I have a follow up question, as I already commented before I will have a stream per account, like : account-{id}

And:

fromCategory("account")
            .foreachStream()
            .when({ // the code

This code will lead to millions of partitions, because I have an stream per account and millions of accounts in my system, is that ok in terms of performance? Do you recommend to have a different approach for this kind of read model?

5

EventStoreDB is capable of storing billions of streams & events
Yes :slight_smile:

6

My question was more about having millions of partitions within the projection, is that also ok in terms of performance or shall I create my own projection in a dedicated datastore? Thanks in advance! :slight_smile:

7

So, really high cardinality.

8

  • on the number of partition :
    it shouldn’t be a problem, think of it as each partition having it’s own state ,

  • is that also ok in terms of performance or shall I create my own projection in a dedicated datastore?
    You should perform some realistic perf tests.
    User projections do create overhead in term of CPU and additional events needs to appended to the database ( so there is a write amplification that uses IOPS and disk space )
    Scavenging also needs to happen ( to deleted old checkpoints used by the projection engine )

9

I think the more fundamental question here is: what do you intend to do with all those partitions and how will you learn about them (by convention perhaps)?

10

They would be the read model, keeping the state and balance for each account … I will perform some tests as Yves was suggesting. Thanks for your expertise and quick answers :star_struck: