So while looking up for examples or connectors for 3rd party i am finding way more stuff for Kafka than Eventstore. Is it not so popular? if not what is the reason. I love using eventstore but the fact of missing 3rd party connectors and writing them on my own would be a long process.
Maybe we all should start contributing the plugins, connectors we created so everyone can find those open sourced?
For example i would want to send events to big query or to azure stream analytics, i would have to code that. but these connectors are available for Kafka out of the box.
@oblutu, at first, I’d like to highlight that Kafka and EventStoreDB are different types of products. Both are dealing with events. However, Kafka is messaging (streaming) tool, while EventStoreDB is a database. So it’s a bit like comparing oranges to apples. Kafka is designed to move data efficiently from one place to another, while EventStoreDB is that “one” or “another” So EventStoreDB would be more a source or a sink. Of course, we have subscriptions functionality that can be used as a way to push data forward. However, I’d compare that more to the CDC rather than streaming solutions.
I fully agree with your point that it’d be good to have more out of the box connectors. It would help in adoption. We’re aware of this issue. We have a beta version of a tool called internally “Replicator” that may be a basis for future connectors. We’re also planning to provide the Kafka Connector.
It’s not an oversight, but rather fitting that with our other urgent priorities. Those things will come, but it’s hard right now to provide the exact deadline.
We’re more than open to supporting the community in the OSS effort to deliver connectors.
thank you for shedding some light on this. I’m glad there’s a beta version, would you be open sourcing that in beta stage so more people can test while in beta and help resolve bugs and/ or issues?
We’re currently doing PoC with the subgroup of customers to ensure that we have the most important scenarios covered. I’m not sure yet when we’ll put it officially into the public. I think that we’ll have more info after a few pilot runs. I’ll send a follow-up in this thread when we have more detailed information.
@zbarrier in fact, yes. I am maintaining the MassTransit integration library for EventStoreDB and building a rider was (still is) in plans. It’s just the fact that building a rider is quite a big undertaking, so I am still not done with that.
@alexey.zimarev thanks, I started looking into adding an EventStoreDB rider recently for the gRPC client which is what lead me to the older posts I mentioned above. I agree, it is a big undertaking. I’m not sure where it is on your timeline, I know you guys are super busy, so I may take a stab at it. If I produce anything that may be helpful, I’ll be sure to share.
So EventStoreDB would be more a source or a sink. Of course, we have subscriptions functionality that can be used as a way to push data forward. However, I’d compare that more to the CDC rather than streaming solutions.