Protocol Buffers for events?

EventStoreDB
1

I’m considering using Protocol Buffers for the actual events themselves, for a number of reasons. Is anyone doing this currently? Any reason not to use them for events?

Thanks

2

eventstore supports it.

A few things will not work.

1) Stream browsing UI (well it kind of will but is essentially useless)
2) Projections (they will not know how to read your data

In general though storage etc works without any issue.

3

Thanks, I figured it would support it since I saw the application/octet-stream content-type stuff. I am not planning on using projections at this time, but I assume I could probably use protobuf.js if I really wanted to down the road. Have you heard of anyone else using protocol buffers for their events? I think it has some advantages but I am wary of swimming against the stream.

Thanks,

Preston

4

You could feasibly do it, there is a raw version of data that can be provided.

5

We use protobuf for our events, a bit irritating sometimes that we cannot browse data, but overall we would still pick protobuf over json.
There is always the possibility of changing to different serialization by migrating streams in a cluster to a new cluster that changes the serialization.

6

There isn't really a good way to support being able to view your
events as we don't have schemas for them

7

Yeah, I know. Anyway, we only use the awesome by_category projection which is a dealbreaker! :slight_smile:
With a little of work we probably could create an endpoint that did unmarshalling wrt. protobuf->json,

however most of our data ends in PostgreSQL or similar anyway so our need is not critical.

8

Greg, would it be feasible to introduce some kind of plugin system wrt. the UI of Eventstore a la
ES -> serialization plugin -> ES UI wrt. being able to show data?

9

It is possible... PR? :slight_smile: If you want I can show you where to look.
Basically its a plugin that would convert protobufs etc to json which
would work in place. The one difficult thing would be supporting the
varying mime types as it would need a hook as well

10

We are doing it and are quite happy with it. In our context it gives more flexibility e.g. renaming of event properties with no consequences but improved clarity. It's about 4 times faster consuming protobuffed streams than json so it enables some scenarios that would be out of reach if json was used. On a different note why would you spend cpu cycles parsing text when its completely unnecesary..