User projection from one id pattern to another

EventStoreDB
1

I have a need to set up a custom projection, but I can’t figure out from the docs where to even begin, and was hoping I could get some help here.

The input streams being written to EventStore have the following stream id pattern:
{environment}-{context}-{source}-{tenant}
For example:
production-myservice-dbimporter-abcd1234

A particular context (like “myservice”) can have several sources, and in order to be able to delete and redo them they need to be in their own streams, also so that the source can resume upload by looking at revision of its own stream for that context. Tenant id is technically optional, as some sources are multi-tenant, but the individual metadata for events always contains tenant id.

From this I would like to project to:
$projections-{environment}-{context}-{tenant}
(e.g. $projections-production-myservice-abcd1234)
and
$projections-{environment}-{context}
for when consumers don’t care which tenant it is getting (again the tenant id is in each event anyway).

I can’t find many examples of how to set up user projections, and I have no idea what to do with the API being described in the docs. So if anyone have ideas on how to construct this, it would be very much appreciated. Thanks!

2

I got lucky with my Google searches. Praise RNGesus. If anyone else finds this thread and have similar needs, here’s what I ended up with to solve the above. I changed the stream id pattern to “events-{environment}-{context}-{source}-{tenant}” so that I could start with fromCategory(“events”) as selector:

fromCategory("events")
.when({
    $any: function(s, e) {
      metadata = JSON.parse(e.metadataRaw);
      if (metadata == null)
      {
          return;
      }
      environment = metadata.environment;
      source = metadata.source.split("/")[0];
      tenantId = metadata.tenant;
      tenantStreamId = "$projections-events-"+environment+"-"+source+"-"+tenantId;
      sourceStreamId = "$projections-events-"+environment+"-"+source;
      linkTo(tenantStreamId, e, metadata);
      linkTo(sourceStreamId, e, metadata);
    }
})

A super-important trick is to use the “New Projection” and “Debug” option in the EventStore web admin UI. That, and getting lucky with finding others who have had similar issues is the solution until there are examples of how to use the projection features in the official documentation.

3

I am wondering what did you mean by “New Projection” and “Debug” option as “New projection” is a button that you click to add a new projection, and there’s no “Debug” option in new projection screen. What you need to do in new projections settings is to set the mode to “Continuous” and tick “Emit enabled” as you produce links.

I don’t think you needed to change the pattern, you could have used “fromAll()” and then check the meta as you do already to verify that it contains the fields you need.

4

@alexey.zimarev what I meant is that the trick to develop projections at least somewhat more easily is to do “New Projection” and then “Debug” on the created projection, so you can see what is going on. That in combination with logging, plus luck with Google searches, is what made it possible for me to create the projection.

And then it is important to NOT set the mode to “Continuous” during development, as I basically just want to run it in admin UI as a test, see if the log output is what you thought, and then continue from there. Once the script is finished I could copy it into my app, which installs it on startup.