Partitioning strategy for multiple business domain

EventStoreDB
#1

Hi,

Short version

We’re looking at migrating some of our systems to EventStore, and I’m trying to find some resources which can help with deciding how to partition and isolate the data from multiple business areas.

Are there any Best Practice guides for hosting multiple business domains within an EventStore instance?

Long version

Let’s say I have two related but loosely coupled business domains (e.g. customers and orders vs marketing’s offers and email lists). In an RDBMS I might have a single SQL server with two databases - one for customers and orders, and one for marketing activity, with the databases owned by two different development teams. If we migrate this to EventStore, our dev teams are keen to be quite granular and potentially create a stream per customer, per order, per marketing promotion, etc.

If we do this, and the two dev teams share an EventStore instance, what options do we have for isolating the streams “owned” by the customer and order team from the streams “owned” by the marketing promotion team? I get that you can assign ACLs to streams, but if we’re creating them on the fly per customer, order, promotion, etc, how does a server admin ensure the right permissions are assigned to the appropriate stream? The default ACL assumes there’s only one ACL set that needs to be assigned, so I don’t think we can use that.

Should we run a separate EventStore cluster per business domain? Or should we use naming conventions in stream names - e.g. customer100, order999, etc (in which case how do we enforce these conventions and prevent the marketing team from inadvertently creating a stream called customer200?).

Any advice would be greatly appreciated.

Cheers,

Mike

#2

I’ve seen namespacing done (such as prepending every stream with the domain they belong to, ex: “myDomain.customer-123”).

#3

We have also looked at supporting the concept of a "database" the
issue with supporting this is there are a LOT of options people want
in terms of varying levels of sharing of resources. As example should
they share the http port? Should replication go over the same channel
or distinct channels? Should all dbs be under the same replication
scheme or replication scheme per db?

#4

Hi Austin,

Thanks - it’s useful to know how other people have approached this scenario.

In our case the dev teams want system-level guarantees that they can’t inadvertently “cross the streams” (pun intended), which means that it looks like the only route is to create a separate cluster for each domain. That has some implications for support costs, but that a problem for the project sponsor, not me :-).

Cheers,

Mike

#5

Hi Greg,

I can imagine it’s a pretty big can of worms to try to think about a change of that scale on a mature product.

From a technical perspective I’ve got some scripts which will help with spinning up and managing multiple clusters to isolate domain data so it’s not a major hurdle - I just wanted to check if there was a recommended way of doing it first. If you were designing a “database” approach, I’d suggest just doing the same port and same replication scheme in first instance, as you could always provision multiple clusters as per today if you really needed control at that level, but it would help with simple cases like mine. That’s just me being a bit selfish though :-).

Cheers,

Mike

#6

Another option is to use ACLs and distinct users to prevent services
from accessing each other's data.

#7

If the application is creating streams for new customers, orders, etc on the fly though for two separate domains, how to I permission the streams with the correct ACLs at creation time? I know I can set up a single default ACL per cluster for new strerams, but unless I’m missing something, I don’t think that that helps if I have two different sets of permissions that have to be applied depending on the which application created the stream.

#8

Write ACL before first event. There are per stream ACLs