Deployment best practices

EventStoreDB
1

I’m new to event sourced databases.
How I’m used to working with relational databases: you store database schemas in your code repository. You write database migrations when your schema need to change. On deployment the migrations are automatically applied to the database.

But I’m a bit lost how to get a similar workflow with EventStore. Take projections for example. I know that I can store code that is generating projections in my codebase and push into via an API.

So the minimalist deployment setup would automatically push all new projection code and reset the projections. But this would only work if those projections don’t emit new events, so it doesn’t seem like a good idea.

So what do you guys do to deploy code? Just by hand?

The same goes for permissions configuration, stream metadata configuration and so on. Do you store it together with your code base or do you manually deploy it?

Hope my desire is clear: to have the whole application logic stored in code and versioned together with the rest of codebase.

2

Hi Dmitri,

I’ve found that in an Event Sourced/CQRS world a lot more lives in your services.

I’m not sure how representative this is of everyone else but here’s what we do…

We store eventstore config in a repo and use it to build our ES server nodes using Packer. Everything else lives in our services.

Our read models do the job of projections so we don’t use them - in fact we found they were a bit tricky to work with for the reasons you’ve noted and coupled the DB to the app more tightly than we’d have liked.

We deploy code in docker containers via a build pipeline. No, not by hand, ever :slight_smile:

Hope my desire is clear: to have the whole application logic stored in code and versioned together with the rest of codebase.

Why would you store application logic the database?

Cheers

Adrian

3

Hi Adrian! Thanks a lot for the answer!

We store eventstore config in a repo

What does this config look like? A number of requests to configure ES on first launch? Or does it have some configuration file I’m not aware of?

Or do you mean /etc/eventstore/eventstore.conf?

and coupled the DB to the app more tightly than we’d have liked.

Yes, exactly, I don’t want them to be so coupled.

If you don’t use projections things get a lot simpler. But still how do you init streams, how do you configure permissions, streams metadata and things like that?

Also curious, in what kind of storage you write the state of read models?

As for me, I still want to get projections to work, for the sake of simplicity. I have created a “projections” folder in my app where I just store them as JS files. Then on deployment I’ll go through all files and publish a projection with the same name as a file and the reset the projection. For now it should be cheap for me, and by the time things grow I hope I’ll find a better solution.

But honestly I expected a tooling for such things out of the box, maybe with some mental model like DB schemas to reason about it.

Dmitri

4

Hi Dmitri

Or do you mean /etc/eventstore/eventstore.conf?

Yes

If you don’t use projections things get a lot simpler.

But still how do you init streams,

I guess I’m assuming you are using CQRS like us, but in our apps the aggregates create a stream for each resource at runtime as an when it’s needed.

how do you configure permissions,

We haven’t needed to. We just limit access to the event store to services that should be able to access it.

streams metadata and things like that?

Again, we don’t need it

Also curious, in what kind of storage you write the state of read models?

Read Models are ephemeral. They construct their state at startup and then subscribe to what they need. If it dies then it just rebuilds and resubscribes when restarted

As for me, I still want to get projections to work, for the sake of simplicity.

Didn’t you say things get a lot simpler if you don’t use projections?

Then on deployment I’ll go through all files and publish a projection with the same name as a file and the reset the projection.

So, you’ll need the projections for your read models I guess… in which case wouldn’t that mean that to start a read model you need the projections in the database to be setup right? How do you know? What happens if it’s not? How do you handle the case where 2 models need different versions of the “same” projection?

For all of those reason, we don’t use them. If you build your read models as I described above then they fufill the job of a projection while being able to stop, start and be upgraded independently.

But honestly I expected a tooling for such things out of the box, maybe with some mental model like DB schemas to reason about it.

It sounds like you should find a new mental model. Event Sourcing turns relational thinking on its head. Try reading up on CQRS and other Event Source architectures and then taking a fresh look at ES. It might seem a bit easier then.

(p.s. sorry, that’s a bit of a stock answer on this group but its true)

Hope this helps

Adrian

5

Hey Adrian!
I did read up a lot on Event Sourcing and CQRS by now, I come from Neos CMS community and we are rebuilding our Content Repository in an event sourced way (https://www.youtube.com/watch?v=qG_x35CxtFE), though admittedly there are smarter people than me doing that (actually I’m a frontend guy).

I understood your solution to the problem: just use the bare minimum of what EventStore provides. It might be indeed very wise.

My current app is super simple: I receive information about payments and I build some projections like total amount of money received, then I just pass them through nodejs frontend and output on the website via ajax… For this stuff projections help by providing an out of the box solution that I don’t need to spend the time on.

in which case wouldn’t that mean that to start a read model you need the projections in the database to be setup right? How do you know? What happens if it’s not?

Yes. I’ll just do it on every code deployment: kill all projections and push the new ones. I could improve the logic to check if the projection’s code actually change. I think that should work…

How do you handle the case where 2 models need different versions of the “same” projection?

Duh, don’t ask! Perhaps if my app becomes that complicated I’ll stop using projections…

So the question to everybody else: do any of you actually use projections and other features besides just writing to streams? If yes then please answer my original question.

Regards,

Dmitri