We have a rare and interesting problem when working with the http api of EventStore. In our process we have multiple subscriptions each in their own thread, that each iterate over an event stream and pipe the events to their respective handlers. Each subscription has it’s own .net HttpClient for reading slices (with long polling) and another one for reading the actual events. We are reading the slices as atom xml so we can deserialize them with the SyndicationFeed class, as this seems to be a bit faster than json deserialization. Now to the interesting part: very rarely, like once in a couple of days, the HttpClient that should read the xml slices returns a json data from a specific event. So now the XmlReader that provides the data to SyndicationFeed rightfully throws an XmlException. We added logging over the time to find out why the XmlException was thrown and after not being able to figure out what causes the problem we added retry logic that just repeats the request, which basically works. It seems that one HttpClient instance receives the response from an other instance, which should be impossible I think, as each instance afaik holds its own tcp connections that shouldn’t be shared among other instances.
To be clear, we don’t know if the problem is on EventStore’s side or in our client code, because the problem happens so seldom that we were not able to reproduce while running Fiddler.
Did somebody ever have a similar issue? Does anybody think that using the HttpClient together with http pipelining and long polling could cause these problems? Are there good ways to analyze a problem like that?
Thanks a lot!
Sebastian
but we are happy to help