Advice for offline mobile client application

EventStoreDB
1

Hi everyone,

I am new to event sourcing, although I have been a professional developer for about 6 years, mainly Xamarin mobile apps for Droid and iOS. I am familiar with hooking into APIs which drive the apps I work on, but have limited back end experience.

I am working on a project which I think can really benefit from using event sourcing and it is going well so far. I have been reading a lot and watching tonnes of presentations by Greg and others and I have (I think) a decent grasp of what I want to achieve and how I am going to do it for the most part.

I think occasionally connected mobile clients are particularly complex domain to apply this in for my first go at ES or CQRS, so I am finding it challenging, but it is too good an opportunity to pass up and also it is a dry run for later client facing work.

I would like some advice on one element in particular, if possible. I will try to explain without going into too much detail. Sorry if it is a bit long winded.

Background

    •   A group of people will be invited to a project by a project admin.

    •   They will be given geographically-based tasks (Go to these places, do these things).

    •   Whilst in the field, they will need to work offline, collecting positional data and augmenting it with their observations.

    •   When they come back online they need to be able to sync their data back to the server for the project admin to analyse.

I have taken my lead from Greg’s talk here:

https://skillsmatter.com/skillscasts/1980-cqrs-not-just-for-server-systems

I currently have ES up and running locally and I am pushing / pulling to it directly over http from the app for the time being.

The idea is that ultimately I will

    •   Pull all unseen events from the server to the client on login / launch (Using EventStore as the master book of record)

    •   Keep a copy of the domain logic in the client so that I can execute commands offline

    •   Keep these commands and the pending events they generate in separate buckets to the pulled server events (using NEventStore w/ SQLite in the apps)

    •   When pushing pending events to the server, if optimistic concurrency check fails, raise 'need to pull first'.

    •   A pull in this case will:

  • Clear local events for that stream

  • Pull new events / rebuild domain models

  • Re-execute commands and resolve any conflicts

    •   If push succeeds, clear local commands. Next pull will replace local events with server events.

Question

Ok, so the thing I am a bit stuck on is the group auth. Obviously eventually I don’t want to be pulling all events down to the mobile client, only those from streams which the user is authenticated to access.

The authentication will be group – based and controlled by the project admin. I am using Azure ADB2C via App Center as my directory service (client facing Active Directory essentially).

I am sketchy on the details of how I will deal with the server until I get there as it is the bit I have little experience with. I know I will need to have a VM which has a client facing API that acts as a proxy for ES which runs behind it on localhost. Access to the API will be secured with the Active Directory. When a user is authenticated I call ES with the ES-TrustedAuth header which contains the users ID and groups.

Beyond that it is a bit blurry.

Should the streams should have a GroupId in their meta data, then I can run a projection to partition them by that ID? That would get me to a point where I had all the events for one group in a single stream.

I’m still not sure how I then secure that projection’s endpoint at group level, unless I can somehow dynamically create an ACL group for it using the group’s ID, and then add users to that ACL group.

If I were to update a stream’s metadata to have an ACL with a $someGroupGuidId entry, then include an ES-TrustedAuth header in my request of “myUserId; someGroupGuidId” would that ‘just work’?

Perhaps I am missing something or looking at this from totally the wrong angle. I expect I am, given my lack of server-side experience. Just a pointer on which direction to go and some things to read or anything would be awesome J

Cheers! Ryan

2

Ok, I found out a little more about AD groups so I am starting to see through the fog a tiny bit.

Once a user has been added to a group and I auth them with AD, the controller can get the user’s group claims, which will be a series of guids.

If all events have a GroupID in their meta which uses an AD group guid, I can partition by that ID as mentioned in my last post, so I would have a stream at a known url, i.e. …/streams/myADgroupGuid123xyz

I could also create an ACL group with the same GUID, then lock access to the stream to members only (so stream /streams/myADgroupGuid123xyz has an ACL which only lets group myADgroupGuid123xyz access it)

Now when a user logs in, I can go through each group ID in their auth claims, infer the team url, auth them in ES for the group using the ES-TrustedAuth header (if I can make it work, see my other group post…) and grab the events for that stream.

Does that seem at least partly sensible?

3

I tell you what would be really handy in my case - just a way to say to ES ‘Give me all the events from all streams I am authorised to access’

Or, failing that, ‘give me a list of all the stream urls I am authenticated to access’

That way I wouldn’t need to partition by group ID or anything. Streams could be secured by group and the user added to that group, then they can just pull all their data easily.

I guess nothing like that exists?

I think a lot of my problems are coming up as I need to pull all events for a user to a standalone offline device, I guess that is unusual as usually ES event processing is used in a server-to-server context and the client is only interested in read models.

4

"
I tell you what would be really handy in my case - just a way to say to ES ‘Give me all the events from all streams I am authorised to access’

This could be done now I believe using linkTos! IIRC the linkTo will do a security check on the origin stream and if you don’t have access will not resolve. As such you could write a linkTo for each event to a stream foo then read from it. There would of course be a runtime overhead for this (write amplification) but it would work …

fromAll().

when({

$any : function(s,e) {

linkTo(“mySuperAwesomeAllStreamThatICanAlsoFilter”, e);

}

})

Then you can read from mySuperAwesomeAllStreamThatICanAlsoFilter … If the user does not have permissions to read the linkTo I believe they will get a notResolved linkTo though I would need to test it to validate. It should be a pretty easy test though if you have an existing DB

This is more feasible than the latter …

Or, failing that, ‘give me a list of all the stream urls I am authenticated to access’" …

This could be billions of streams :open_mouth: Once we start discussing paging remember that this list is mutable (things can be added/removed while you are reading it!)


Virus-free. www.avast.com

5

Aha! That is great, thank you :slight_smile:

So essentially using that projection I get a copy of the All stream but with all the things I don’t have access to filtered out (or marked as ‘unresolved’ so easily ignorable). Sounds ideal.

I’m still a little unsure of how I will auth the users against the streams. As the access will be group based, I would lean towards putting them in group XYZ and locking the stream read / write to that group.

The trouble is, I can’t see a way of creating groups via the API, unless I missed something.

It looks like I can dynamically add users to a group using ES-TrustedAuth (which isn’t working for me right now, see my other post), but the group itself has to be created in the web portal?

Or does adding a user to a non existent group auto create it? That would be handy.

The other option of course is adding a user to each stream’s ACL directly, but there could be zillions to edit when a new user joins a group so it seems a non starter.

6

Or perhaps that should be ‘adding a non existent group to a stream’s ACL auto creates it’

Essentially, does a group exist just by being mentioned in an stream’s ACL / user’s trusted auth header or does it have to be explicitly created, and if so can it be done remotely?

7

Ohhh… looking in the portal, it looks like there are two set groups only - admin and ops, no way to create new ones at all.

Is that what you were trying to tell me earlier with the cheeseburger example on Github?

In that case, absolutely that feature would be great, and essential for this project really :slight_smile:

8

ES-TrustedAuth is unrelated here I think. Its just saying let something else do the authentication etc then it passes in the user/group memberships as opposed to using internal users/group management. As an example I could configure a system that looked into some company-wide authentication service to provide this information as opposed to using what is in eventstore.


Virus-free. www.avast.com

9

You can add users to groups and use groups in ACLs now. There is no “explicit creation” of a group its just a tag. When setting privs you can include this tag.

EG

User: Greg groups : “foo1,foo2”

Stream : str-123 : {$r : “foo1”}


Virus-free. www.avast.com

10

Ah, awesome, just what I had hoped.

I just noticed that the web portal only has admin and ops or none as options, so I thought perhaps it was restricted to them.

So essentially I can tag any stream as being part of a group just by including it’s tag in the meta ACL, and auth any user to that group by including that tag in the auth header. Perfect.

Thanks again for all your help on this, really really appreciated :slight_smile:

Cheers,

Ryan