We’ve a scenario where we’d like to replicate a non-clustered EventStore node to a cluster. We’ve a component that reads the $all stream and writes those events to the replica. The specific scenario is that we’d like to replicate data to a cluster and then switch to using the cluster within minimal downtime. Some questions:
Currently, the replicator filters event types starting with $. Does this make sense or should all events be replicated? On the one hand, stats events seem superfluous, one the other, metadata events could be important? For example, deleted streams don’t come through.
The replicator copies the event in its entirety, and provides an expected version when writing. It orders writes to individual streams, but parallelizes across streams. Is this acceptable? It seems to be working, but I’m not sure if it could cause issues.
We noticed that when using the built-in IEventStoreConnection.SubscribeToAllFrom function, when the replicator catches up and switches to push mode, we start running into WrongExpectedVersionException. If we leave it in pull/poll mode, we don’t see those issues. Could be an issue with our code, but I’m wondering if you’ve seen this before?
In using this process to switch to a cluster, we’re looking to see if we can eliminate downtime. One option is to create a layer on top of the EventStore client, and in a similar vein to the EventStore client itself, take care of switching to the async replica upon a trigger. In this case, the trigger could be an explicit request indicating the replica is caught up, rather than a heartbeat timeout. Is there a better way to do this with EventStore?