I recently got intrested in event sourcing. In the examples I saw they often use a bank as an example to display the possibilities.
What I often see is that they store events like Deposited{5}, Withdrawn{10}, etc.
This felt logical at first. Until I started thinking about commands that failed. They might not have an event, so I start loosing information.
So I thought about storing the commands besides the events. So something like Deposit{5}, Withdraw{10}, etc
But these commands do not affect the state. So they don’t belong in the same stream of data as the events I’m storing.
All pretty clear. But still it felt a little bit weird storing stuff double like this. One for the command and then more or less the same for the event.
Then I was thinking about the events. Although the user wanted to deposit 5 euro, the result is a new value for the balance of the account?
So command: Deposit{5}
Event: BalanceUpdated{105} (knowing balance was 100)
command: Withdraw{10}
Event: BalanceUpdated{95}
I can store a link to the command in the metadata of the event to see it was a deposit and a withdrawel. Why should or shouldn’t I store it like this? For things like owner we also store the new value of the object that we want to change. Why not for something like the balance?
If you want to track denied commands, add the denial as an event.
Withdrawal{100}
WithdrawalDenied{100, Insufficient Funds}
Storing the balance is a bad plan because it is derived state.
Check out my talk Event Sourcing for Cloud Developers at BuildStuff a couple of years ago on YouTube it describes what state to save and why.
@chris.condron thanks for the idea about how to store failed events. That is a way to also handle it indeed.
I understand that storing an event with just the new balance value is a derived state, but thats actually what I’m trying to figure out. So let me continue on the example.
Today the bank wants to make another deposit in the customer account because the interest are being payed.
So what do we do here? Deposit{10}? That doesnt feel right and since the 10 is a derived value lets store the event DepositInterest{5%} ? But wait that 5% is actually derived from the much more complex formula based on the date, the amount in the account, the age of the customer, the opening date of the account and so on and so on… Maybe some of that values or also derived again of something else. So what do we do here? Are we going to store all those values so that we can recreate the complete formula with this event? When do we start using derived values or will we never? For me, this is one of the harder questions of using event sourcing vs storing current state. So any advice about good reads about that subject will be appreciated too.
And I’ll look up your youtube video’s, perhaps they will answer all my questions .
@daxyhr, I have a bit different perspective than Chris. Sometimes it’s worth adding a bit of “redundancy”. I agree that we should try to avoid putting the derived state into our events. We should try to make them as concise and small as possible, but not smaller
In this particular case, as you noted, the balance may be calculated using complex formulas. What’s more, those formulas tend to change. So, I would keep the events types focused on the specific event as Chris suggested, but I’d add balance into the events, e.g.:
DepositMade{Amount:5, Balance:105}
Withrawal{Amount10, Balance:95},
etc.
This is giving us several benefits:
we’re keeping the business logic in place on the write model. Read models do not have to be aware of that.
we have the balance information frozen, even if the balance calculations changes (or e.g. taxes). We might avoid then versioning the balance formulas,
read model generation is more straightforward, plus what’s more important - it’s easier to handle idempotency. If you have the ordering guarantee, then applying this event twice will have the same effect. If you just have the transaction amount, then you need to have a dedicated mechanism for that.
Of course, this is kinda grey matter when to allow “redundancy” and where not to. A lot of things needs to be taken into considerations. You can also read more in my article: Events should be as small as possible, right?.
Thats actually the point I’m “fighting” against without even knowing it. hahaha. I completely agree that the read model should not have to understand the logic. Where with Deposit{5} it needs to know it needs to take the accounts balance and add 5, storing the new balance value would simply be a replace value. The same as we do when we enter a new email address event. Or update the Account owners name.
Perhaps I would store it like:
BalanceUpdated {
State: {
Balance: 105
}
Command: {
Deposit: 5
}
}
This way 95% of the read models can just use the replace value X with Y based on the fields in the State the same as we would handle and UpdateEmailEvent or ChangeOwnerEvent, where the other 5% can do other things based on the stored Command.
If you flatten or try to unify the events (e.g. DepositMade, Withdrawal, TransactionVoided into BalanceUpdated), you’re risking losing the important business information.
It’s hard to precisely say what would be better for your case, but I’d recommend keeping the events focused on the business operation result. Events are business facts. Of course, we should consider how they’re consumed. E.g. cash withdrawal from ATMs will have eventually different properties than credit card payment. Also, those different events may trigger different flows (e.g. fraud detection etc.). By making events generic, you’re risking losing the biggest benefits of using Event Sourcing: having the business context and not losing the data.
While event modelling, it’s critical not to think about the state change but the business fact that occurred. Generic doesn’t have to mean simpler.
Ofcourse we could add the Amount in both cases like in your example above, but then the read model needs to know again which fields it should and which fields it shouldnt process. So my feeling says it should be one of the two (either the NewBalance or the DepositValue) and the rest should be in the metadata? And the one that requires the least knowledge from the reader is the NewBalance value?
Eventually I want to setup something based on the Domain we have here and see if i can push it, so I get some more time to figure out if event sourcing would be viable for what we are doing, because its all new to us .
Or probably im just thinking wrong and im trying to have to much control on the read models?
If you remove the amount, you’ll end up in the opposite situation to the initial one. You’ll have to know the previous balance to be able to calculate the amount. For those events, the specific information about this fact is that you deposited 5$, then withdrawn 10$.
As Chris noted: balance is derived information. We may decide to add it for the reasons I explained above, but that doesn’t make it event-specific information. We shouldn’t be dropping the critical pieces of information. By doing that, we are losing the data. If the algorithm or taxes change (and you don’t version it, just keeping the latest logic), then how you will know what the transaction amount was?
As I mentioned before, we should focus on recording all the specific information related to the results of a business operation. Read models should derive from that. To do it right, it’s needed to shift the mindset from thinking what the state is to what has happened.
We can pragmatically add additional information or do some enrichment transformations suited for the read models, but that shouldn’t be the basis of our considerations. It should be an optimisation that we carefully crafted.
Read models definitions are changing. New may be created based on the already registered events. That’s one of the biggest benefits of Event Sourcing. As long as you have the information recorded, you can use it later on. If you follow those rules, you don’t even have to design read models while working on the write model. If you’re tying them together, then you’re losing the autonomy of those models. Events are facts how they’re interpreted, depending on the reader. As long as it has enough information, then we’re good.
@oskar.dudycz Thanks for clearing that up. You are correct that im trying to come up with a solution based on my knowledge of storing state and I’m trying to optimise based on that knowledge . Thats why I’m glad that I found this forum, because I really hope I can explain this way of working to other people (especially where I work) at one point in the future.
Events are facts how they’re interpreted, depending on the reader.
I would strongly advise against keeping the balance in the event in any way.
I think the the use of “deposit” might be making this more difficult to see.
If we start using more accurate process models it might become clearer.
Let’s add in just “Recognition of Funds” and 'Source of Funds" to get started.
A cash deposit made in person before “close of business day” (often 3pm even if the branch is open until 5pm) is available immediately.
A foreign wire transfer may take several days
A cash ATM deposit may recognize some portion but not the full amount.
So if I have in the account 100
and I have an ATMDeposit (1000) (100 is recognized immediately and the rest on the next day)
as a customer my balances might be
Available 200
Account 1100
the Teller may see that I have balances of
Ledger 100
Deposit Credit 100
Pending 1000
Available 200
Account 1100
If the calculated balance is included in the events, this kind of thing, and especially any changes the business makes to this become very difficult.
Events should be the history of business facts that happened and not include any calculated values
Thanks for the example. That made things alittle bit more clear.
One more question about the:
Withdrawal{100}
WithdrawalDenied{100, Insufficient Funds}
Would it be also possible to just store the WithdrawalDenied? Because the Withdrawal itself never happend? It would just mean that the read model that shows the balance total for example can completely ignore the whole WithdrawalDenied event, since it does not affect the balance state the same as an OwnerChanged doesn’t?
Yes you can.
I can see my example above wasn’t clear
“Withdrawal” is the command and isn’t stored.
“WithdrawalDenied” is an event and is stored
the parameters “100”, and “Insufficient funds” are the requested amount and reason code on the failed command. The name of the event indicates the type of command that was rejected.
So all of the data on the stored event are about the incoming request and not the internal state.
Ok thank for the clarification. The think error I was making is that I tried to “solve” everything at the writer, where I should solve it on the reader that is responsible for a certain task/view on the events. Different readers, with different responsibilities.
Nice read. Although after giving it some more thought since I opened this topic, I start to agree that the balance shouldn’t be part of these event at all. This logic should not be part of the write model, but it should be part of a readmodel and if you don’t want to have other readmodels to contain this logic too, they should depend on a readmodel that does.
Actually If I grab your example: DepositRecorded { Amount : 50} CashWithdrawnFromATM { Amount : -50} IncomingTransferRecorded { Amount : 100} CreditCardPaymentMade { Amount : -20}
I would make a readmodel that catches all four events. Process them by calculating the BalanceUpdate and store the BalanceUpdate as a separate event (instead of appending it to the 4 events above).
So now other readmodels can choose to track only the BalanceUpdate if they dont care about the transactions made, or track DepositRecorded, CashWithdrawnFromATM, IncomingTransferRecorded and CreditCardPaymentMade. Or any combination of events they require to make the projection that is needed.
And I’m still on the fence if I should record the BalanceUpdated at all… Because I think the best would be if those other projections would just read from a model that calculated the current Balance. If they need the current balance.
Who knows maybe next week I think different about the subject again, but thats how I feel about it currently. But I’m still very new to event sourcing, so I might switch my opinion a couple of more times .
@daxyhr, a separate BalanceUpdated event is one of the options. Still, I’d rather had a transformation (enrichment) that’s taking the event and then publishing it (or storing it in a separate, dedicated stream). That’d keep the main stream shorter and focused on the business use case. Personally, I do not like to add artificial events to the main stream.
Plus, you have to remember that events are not only about projections. They might be triggering other workflows, not only inside the single module that’s publishing it but also to external services. As I wrote in the article, focusing too much on the state is a fallacy and might be leading to the wrong decision.
Still, it’s hard to say what would be better or worse without having the specific requirements. Good design is not created in a vacuum If BalanceUpdated is a business fact, then it’s might be worth recording it one way or another. Each design can be fine, as long as it’s aligned with the specific case. It’s all about tradeoffs.
.
.