How to reliably get a testcontainer with projections running

EventStoreDB
1

I have some flaky integration tests using testcontainers for go, where they sometimes fail because the $category-SESSION stream is not found. I interpret that as the projection has not yet started up.
I wait for the testcontainer to be available like this:

	WaitingFor: wait.ForAll(
			wait.ForHTTP("/gossip").WithPort(nat.Port("2113/tcp")).WithStatusCodeMatcher(
				func(status int) bool { return status == 200 }),
			wait.ForHTTP("/stats").WithPort(nat.Port("2113/tcp")).WithStatusCodeMatcher(
				func(status int) bool { return status == 200 }),
			wait.ForLog(`.*Found the following new projections in.*`).AsRegexp(),
		),

Is there a better log statement to wait for? Or maybe some other method to ensure that projections are available?

Thanks in advance

2

You could wait for the projection status endpoint to return { "status": "Running" } to make sure the projection is running after startup. However, you are reading a $category-SESSION which will only appear as of the moment the category system projection observes SESSION-* streams and emits the first event that is in such a stream. Ergo, even if you wait for the projection and subsystem to be running, that stream will not be there until your test emits such an event. If your tests put these events in as part of their setup you could make the setup of the test probe and wait for that stream to appear before moving forward with the rest of the logic that assumes that streams is present. Overall, it may be wise to have that code handle the absence of the stream gracefully. Stream not present is similar to stream being empty in this case.

HTH.

3

To complement Yves 's information :
Are you reading or subscribing ?

4

That sounds doable, i will try that. The usecase is i have a scheduled job cleaning up expired sessions and soft-deleting the streams via that projection. It’s reading it from where it left off last night and then reading towards the end of the projection stream until it meets an active session or the end of the stream.

The integration test does the session seeding itself so i think it’s fine to just wait for the projection to be ready like you describe, but just to complete my understanding of projections, please confirm this:

If the test emits an event to a session stream before the projection is initiated - won’t the projection just catch up and also include that event emitted when the projection was not running? I know it will still likely fail the test, because the test will also do the attempt to read the projections stream too soon - but let’s say the test then slept for n seconds, wouldn’t that before mentioned event appear on the projection stream and the test succeed?

Thanks for the fast replies btw!

5

Yes the projection , when started will start from where it left off ( and if run for the first time will run through the history of events until the last one in the DB )

In the case you describe though:
I would use a catch-up subscription instead of nightly batch type processing

  • you would still need to store where you left off as you do for the reading, in case that process needs to be restarted (i.e. your logic stays the same, just how you feed it to your logic )
  • advantages: no night job, the sessions will be expired as soon as the trigger you already have defined are reached

The test would be :

  • set up the subscription
  • emit events as needed for your test.
  • wait for your logic to run
  • Assert what needs to be asserted .
  • ( and some time out on the test )
    • depending on where the test is ran you might not get enough resources to make it fast
      • we had that with integration tests executed in GitHub runners
        • they are not made all equal in term of resources
6

Sleeping in tests is somewhat of an anti-pattern. It usually implies a problem with the overall design of either the test or the system under test or both. Your test might benefit from waiting until the linkTo ($>) event appears in the $category-SESSION, making sure that any subsequent code is able to read the stream. That said, overall it’s wise to not depend on the stream being present and to deal gracefully with the absence of the stream, making the code more resilient. Alternatively, you could subscribe to the stream instead of reading it, which deals with the absence of the stream automatically.