We’re about to migrate some data, not that much if you ask me, from a sql database to eventstore. To do so we have a small project that extracts the relevant data from the SQL database and inserts it into eventstore. The first part takes about 15 s to complete, as I said, not that much data.
We have also implemented some services that subscribes to events and build up views as events are written, that works fine under normal load, but now when we have an initial larger set of events it takes for ever. The views we build up are one in a sql database and one in a elasticsearch database. I’m pretty sure it is because we are only consuming one event at a time and that causing many reads and writes against the view databases since each event results in one lookup and then one write/update.
Is there a smarter way to do this? It must be. If possible I would like to have the save solution for the migration part as well as for the live running part, the views should be updated the same way disregarding if it is an event resulting from an migration or from the actual system.