Reading and writing to the same stream over the same connection at the same time

I am developing a javascript client and I encountered an issue which I am not sure if it’s something in my code or if eventstore doesnt work quite like I think it does (eventstore works great, thank you for your work!).

I have a script I run to test what I have so far in my client. Right now, I’m implementing catch-up subscriptions.

My current script immediately starts reading stream events froward from an event number to a final event number in batches.

It also immediately starts writing to the same stream on an interval.

Some of the batches complete until the first event is written to the stream.

In my logging I see that right before the write, a tcp read message is sent to the eventstore, but once the writing starts, I only receive WriteCompleted and StreamEventAppeared events

and the eventstore never sends any sort of response for the last read request.

If I disable the writing, all the batches complete. If I delay the writing sufficiently, all the batches complete and the subscription works as expected.

Would eventstore “wipe out” read requests on a stream once it receives write requests to the same stream? Any other ideas what might be going on?

Perhaps posting some code would be a good idea?

I thought maybe I should post code, but the thing is, if it’s my code, it could be anywhere at this point. I think I’m just seeking some understanding about how eventstore would handle this situation. My main question is assuming I successfully send a ReadStreamEventsForward command to eventstore over tcp and before receiving a ReadStreamEventsForwardCompleted I start sending WriteEvents commands every 500 ms, I should eventually receive a ReadStreamEventsForwardCompleted response?

Here is my test script but it would be a lot to wade through: https://github.com/chrismcleod/eventstore/blob/feature/catchup-subscription/src/index.ts

all operations are linearized in the client.

Ok great, thank you for fast replies!

https://github.com/EventStore/EventStore/blob/release-v4.0.2/src/EventStore.ClientAPI/Internal/EventStoreNodeConnection.cs#L245

Ok so I have reproduced this in a single-file test. The problem I am experiencing is when I am reading and writing, sometimes a read is sent and no response for that correlation id is ever received. Also there are no errors in the server log. I can work around the issue by retrying the read, but I’m wondering if that is a good way to handle this.

import { parse, unparse, v4 } from “node-uuid”;

import { EventStore } from “./eventstore”;

import { connect } from “net”;

const connection = connect({ port: 1113, host: “192.168.99.100” }, () => {

const read = (from: number) => {

readTimeout = setTimeout(() => {

console.log(“Read timedout retrying…”)

read(from);

}, 1000);

console.log(Reading from ${from}...);

const message = EventStore.ReadStreamEvents.create({

eventStreamId: “$ce-user”,

fromEventNumber: from,

maxCount: 10,

resolveLinkTos: true,

requireMaster: false

});

const encoded = EventStore.ReadStreamEvents.encode(message).finish() as Buffer;

const auth = Buffer.alloc(15);

auth.writeUInt8(5, 0);

auth.write(“admin”, 1);

auth.writeUInt8(8, 6);

auth.write(“changeit”, 7);

const packet = Buffer.alloc(22 + encoded.length + auth.length);

packet.writeUInt32LE(18 + encoded.length + auth.length, 0);

packet.writeUInt8(0xB2, 4);

packet.writeUInt8(1, 5);

v4(undefined, packet, 6);

auth.copy(packet, 22, 0);

encoded.copy(packet, 37, 0);

if (connection.writable) {

connection.write(packet, () => {

console.log(“Sent Read…”);

});

}

}

const write = (data: any) => {

console.log(“Writing…”);

const message = EventStore.WriteEvents.create({

eventStreamId: user-${v4()},

expectedVersion: -2,

events: [ {

eventId: Buffer.from(v4()),

eventType: “CreateUser”,

dataContentType: 1,

metadataContentType: 1,

data: Buffer.from(JSON.stringify({ a: “b” })),

metadata: Buffer.from("{}")

}],

requireMaster: false

});

const encoded = EventStore.WriteEvents.encode(message).finish() as Buffer;

const auth = Buffer.alloc(15);

auth.writeUInt8(5, 0);

auth.write(“admin”, 1);

auth.writeUInt8(8, 6);

auth.write(“changeit”, 7);

const packet = Buffer.alloc(22 + encoded.length + auth.length);

packet.writeUInt32LE(18 + encoded.length + auth.length, 0);

packet.writeUInt8(0x82, 4);

packet.writeUInt8(1, 5);

v4(undefined, packet, 6);

auth.copy(packet, 22, 0);

encoded.copy(packet, 37, 0);

if (connection.writable) {

connection.write(packet, () => {

console.log(“Sent write…”);

});

}

}

let buff: Buffer = Buffer.alloc(0);

let size: number | null = null;

let readTimeout: number;

connection.on(“data”, (buffer) => {

console.log(buffer);

if (size === null) size = buffer.readUInt32LE(0);

buff = Buffer.concat([ buff, buffer ]);

if (buff.byteLength < size) return;

const code = buff.readUInt8(4);

if (code === 0x01) {

buffer.writeUInt8(0x02, 4);

connection.write(buffer);

} else if (code === 0xB3) {

if (readTimeout) clearTimeout(readTimeout);

const key = unparse(buff, 6);

const response = EventStore.ReadStreamEventsCompleted.decode(buff.slice(22));

if (!response.isEndOfStream) read(response.nextEventNumber);

}

buff = Buffer.alloc(0);

size = null;

});

read(0);

const int = setInterval(write, 100);

setTimeout(() => clearInterval(int), 10000);

});

Also, though the read response never comes through, I do still get responses from the writes.

Hi Chris

The C# client has an operation timeout of 7 seconds by default. If no response has been received from the server by the time this timeout is up, it will retry the operation up to 10 times.
Both the timeout and the max number of retries are configurable.

Do you have any indication as to why you are sometimes not getting responses from the server, though? Is your Event Store under load?

ES will shed load when it is overloaded. Timeouts etc will be hit.
What kind of environment are you running in?

Ok this is perfect, I was not aware ES would shed load and knowing the C# client has a default timeout makes me feel much better about implementing a timeout.

I am running this on a macbook pro core i5 (2.2GHZ) 8GB ram using the docker container. I dont expect great performance from my macbook, it’s just a dev machine.

Thank you both for the answers.