Backups

Greetings and congratulations on the release!

I’m new to the concept of Event Sourcing; it’s however so interesting that I’m about to give it a try; a question that just popped up from one of the guys in my team…

Can you back up the eventStore regularly on a schedule? what’s the recommended desaster recovery/prevention strategy for this type of store?

thanks.

Geo

Sure. The whole db is immutable files.

With the multinode version though just setup warm replicas (aside from hot nodes)

Hi Greg. Have you ever gotten a chance to write blog post about how backup would work, in which order we should backup the files?

Thanks, Slav

We have been adding docs like crazy so we will add it in docs.

An xcopy of all files should work just fine. AFAIK, just make sure to backup checkpoints first and then actual chunks.

Best,

Rinat

Yup. This worked for us.

XCOPY method works reliable ONLY if EventStore is stopped while you taking back. Not just idle, but completely stopped. Otherwise you can end up with a backup that will be refused by EventStore saying that DB is corrupted (there are at least few things that CAN go wrong if you copy files in unfortunate order while something is written into DB). And idle ES doesn’t mean there are no stats events written or projections are not writing anything.

So, if you want to back up DB of live ES, you need to do the following:

  1. copy dbdir\writer.chk into backup\truncate.chk

  2. copy dbdir\index subfolder into backup\index

  3. copy all files in dbdir\ except dbdir\truncate.chk into backup\

This process makes sure that after restoring from backup you’ll end up with correct DB.

you taking backup, obviously :slight_smile:

What about using shadow copy to copy files out? Would that make any difference at all if db is live?

Thanks, Slav

The only issue with a straight xcopy is you can get an extra chunk and
would have to delete it. We are working on this (probably an option on
startup like --verify-back-up)

I noticed there's an OEM section on the licence page now. Under what conditions is this required? Running SingleNode in-process for instance?

And while I'm at it: Do you think it's a viable solution to run the eventstore embedded in an occasionally connected client (assuming chunk-sizes and threads can be tweaked suitably)?

/Peter

For your occasionally connected clients how many events are we talking
about having there? For many I would think a file would work just
fine.

At the moment we clear the local evenstore at end-of-day (this is a point-of-sale-application), if we're connected and syncronized, so we're talking a few thousand, max. My issue is more with robustness than scalability, I'd rather use something battletested and hardened than the pretty naive file-based approach we use now. (which seem to work ok, but there's been report of corruption in one instance).

/Peter

As Greg mentioned in documents, corruption could be caused by a failure of commodity hardware. Single-node event store will not necessarily be able to deal with this situation. More POS instances you have - higher the chance of such failure.

Best,

Rinat

Peter

You asked about OEM licnesing/support.

The Single Node is available to you under a BSD license; you can effectively do what you want with it provided you fulfill the copyright notice and disclaimer requirements https://github.com/EventStore/EventStore/blob/master/LICENSE

There are many organisations who will use the Event Store (single node) in similar ways to yourself, some for internal systems some in systems for sale. Quite a few have asked for support on the single node version, many of these are going to be running multiple instances on EPOS terminals, etc. just like you. It wouldn’t work for you if we charged for support for each of these, in your case 300 or so (although I would like that!).

Therefore, the OEM licensing/support is available on both the single node and HA version at exactly the same price. Although you don’t need it to distribute the single node version with your product, support might be a good idea to ensure you keep those tills ringing. If any of your store/customers have such a large store that they need the HA version then that is available to.

email me off this list if you wish to discuss further - chris.watt[at]geteventstore.com