Externally managed offsets

EventStoreDB
1

Hi,

Let’s say we have the following situation :

  • a mobile device maintains it’s own event log, with its own offsets / sequence numbers

  • we want to periodically sync the device’s event log to a GetEventStore stream

I cannot seem to find a way to tell GetEventStore to use the client’s sequence numbers instead of the automatically generated ones.

(which would allow me querying the event store by offsets rather than needing secondary indexes)

So my questions:

  • Is what I’m trying to do doable with GetEventStore?

  • Does that even make sense from your perspective?

Regards,

Sami

2

What if two devices have the same sequence numbers? The sequence
numbers represent the order the events were received not the order
they were generated.

The best way if I am understanding your problem correctly is similar
to how git works. Remember what the device thinks the expected version
should be. Then push events to that expected version number. If
something has changed you will get a concurrency violation. Then pull
events, "merge locally", then push again with new expected version
number. Assuming you want consistency.

Without consistency just push with expectedversion.any and use a
metadata field to represent your own ordering (sequences, times,
vector clocks, etc).

Make sense?

Greg

3

Thanks for your answer Greg!

4

Can you explain your exact use case?

5

So, here is my use case :

  • each ios app instance has some sort of appId UUID that is generated the first time the app is launched

  • all user interactions in the app are modelled as events and recorded to a local event store on the ios device

  • the appId is the aggregate root Id and is thus shared by all events for this particular stream for this particular install of the app

[ {
eventType: “RunStartedEvent”

eventId: “62ACB3E1-DDB7-4507-B0CD-B13167B8DA00”

appId: “857E7B39-DF4D-4259-BF9D-7816D66A3D8A”

offset: 0

payload: { …}
},

{
eventType: “RunStoppedEvent”

eventId: “76736CCB-50DE-416E-B0AA-EBD553EC0182”

appId: “857E7B39-DF4D-4259-BF9D-7816D66A3D8A”

offset: 1

payload: { …}
}]

  • Once in a while, when enough data has been collected, the app decides to push this data to the server,

which currently uses a custom event store based on MongoDB, that allows querying by appId and client offset range.

  • In most cases the offset should start from zero, but we could still have some legacy streams that do not start from zero for some reason.

So, I am trying to figure out whether I could phase out our current implementation of the event store in favor of Get Event Store.

One aspect is that I would like to be able to query directly by appId + client offset range, without keeping a mapping of client offsets to getEventStore offsets somewhere…

Is that clearer?

Regards,

Sami

6

Are the sequences just starting later in some cases EG position 5 but
then going 6,7,8,9, etc?

7

There shouldn’t theoretically be any gap, but I guess I would need to run a batch on the old data to make sure that no bug was introduced at any point that lost some data…

8

If not then the simple solution is to use a
"IDontGiveAShitAboutThisSequence" event and just populate forward.
Then the sequences align and no issues.

9

OK, so GetEventStore doesn’t want me to mess with its own sequence numbers, and then the realistic solution is to insert dummy events to align them.

That makes sense, it should allow me to phase out my legacy MongoDB Event Store.

Creating a decent, well-performing event store is trickier than it looks like initially :wink:

Thanks!

Sami