Emit vs linkTo vs copyTo

EventStoreDB
1

Hello,

I have stream which will be written directly to form my application.

I would like to add more event to it from a projection, which will use fromStream([’$et-foo’, ‘$et-bar’]) when &all to monitor certain other streams.

What should I use in the projection: emit(), linkTo() or copyTo()?

I also assume that the projection should be continuous.

In my initial tests the projection seems to be working (processed events is as expected) but I cannot see events in the target stream.

I can see the events when emitting to a brand new stream.

Thank you in advance,

Jakub

PS. I’m currently away with intermittent access to internet, may not reply immediately.

2

The difference is:

emit - emits a completely new events with arbitrary JSON content

linkTo - writes a pointer to an event into another stream

copyTo - copies an event into another stream.

If you post the source of your projection we can likely help further.

James

3 
Here's my projection:



fromStreams(['$et-AccountManagement_BranchCreated', '$et-AccountManagement_BranchCodeChanged', '$et-AccountManagement_BranchRenamed', '$et-AccountManagement_BranchDeleted'])
    .when({
        $any: function (state, ev) {
            if (ev.metadataRaw) {
                var metadata = JSON.parse(ev.metadataRaw);

                if (metadata.Tenant)
                    linkTo(metadata.Tenant + '-Sales_Branch-' + ev.data.Id, ev);
            }
        }
    });

Jakub

4

Have you tried debugging it to ensure your linkTo is hit - it looks ok at first glance (on a phone screen and without running it!)

https://www.youtube.com/watch?v=NFnsFfOMMkI demonstrates how to do this

James

5

Jakub,

why do you join system streams instead of just doing

fromAll().when({

AccountManagement_BranchCreated: your_function,


AccountManagement_BranchRenamed: your_function .....



..

}



function your_function(state, ev) { ....
6

I was using fromStreams as my function is basically just toLink and is identical for each event type.

My requirement is to append to certain streams all events of certain types. Using et- streams seemed natural. I have very small numer of events and performance is not an issue.

If you recommend using fromAll I can switch without any problem.

Any guidance is very helpful as the documentation is a bit sparse yet. (No hard feelings intended, I do understand your priorities!)

7

Et- streams are just technical details of how fromAll().when works. You can use the same function on multiple event types as I showed.

I’d not recommend using any $ steams directly unless it is absolutely necessary.

The same without them should be more readable and it will not depend on any system changes.

-yuriy

8

Thank you, Yuriy.

Could you please check me rewritten projection? It faults on the first run but restarting it succeeds.

fromAll()
    .when({
        AccountManagement_BranchCreated: process,
        AccountManagement_BranchCodeChanged: process,
        AccountManagement_BranchRenamed: process,
        AccountManagement_BranchDeleted: process
    });

function process(state, ev) {
    if (ev.metadataRaw) {
        var metadata = JSON.parse(ev.metadataRaw);

        if (metadata.Tenant)
            linkTo(metadata.Tenant + '-Sales_Branch-' + ev.data.Id, ev);
    }

    return state;
}
9

Jakub,

You should be able to access parsed metadata as ev.metadata. It is defined as calculated property which parses metadataRaw on the first access.

-yuriy

10

Yuriy,

Thank you your your suggestions. I’ve rewritten my projection as

fromAll()
    .when({
        $init: function () {
            return { }; // initial state
        },

        Core_AccountManagement_BranchCreated: function (state, ev) {
            if (ev.metadata && ev.metadata.Tenant) {
                linkTo(ev.metadata.Tenant + '-Core_Sales_Branch-' + ev.data.Id, ev);
            }

            return state;
        },
        Core_AccountManagement_BranchCodeChanged: function (state, ev) {
            if (ev.metadata && ev.metadata.Tenant) {
                linkTo(ev.metadata.Tenant + '-Core_Sales_Branch-' + ev.data.Id, ev);
            }

            return state;
        },
        Core_AccountManagement_BranchRenamed: function (state, ev) {
            if (ev.metadata && ev.metadata.Tenant) {
                linkTo(ev.metadata.Tenant + '-Core_Sales_Branch-' + ev.data.Id, ev);
            }

            return state;
        },
        Core_AccountManagement_BranchDeleted: function (state, ev) {
            if (ev.metadata && ev.metadata.Tenant) {
                linkTo(ev.metadata.Tenant + '-Core_Sales_Branch-' + ev.data.Id, ev);
            }

            return state;
        }
    });

The projection can be saved and started without any issues.

However, when I start writing the events the projection errors and its status changes to Faulted. If I start it again the status changes to Running but I see more errors in the log.

The error log is attached.

Could you please advise?

Many thanks,
Jakub

PS. I will be offline for a few days.

127.0.0.1-2113-single-node-err.log (35.2 KB)

11

Jakub,

1) Does the same happen if you stop the projection, then restart it and
then start again?

2) Also, if i did not miss anything all the handlers are the same, so you
can rewrite it as:

function handle (state, ev) {
    if (ev.metadata && ev.metadata.Tenant) {
        linkTo(ev.metadata.Tenant + '-Core_Sales_Branch-' + ev.data.Id, ev);
    }

    return state;
}

fromAll()
    .when({
        $init: function () {
            return { }; // initial state
        },

        Core_AccountManagement_BranchCreated: handle,
        Core_AccountManagement_BranchCodeChanged: handle,
        Core_AccountManagement_BranchRenamed: handle,
        Core_AccountManagement_BranchDeleted: handle,
    });

-yuriy

12

Jakub,

Do you write to the streams this projection emits to somewhere else? different projection? via clientAPI or http?

-yuriy

13

Yes, I write to this stream from client API. I want to add some events from different stream using the projection in question.

14

Starting and stopping works when no other events exist (empty eventstore). I get the same errors when other events exist.

I can provide the database if it would be easier for you to diagnose the issue. It’s just one chunk. I’ve already provided it before. :wink:

Jakub

15

Jakub,

The problem is projections assume exclusive access to the output stream. Otherwise it becomes impossible to make projections recoverable in case of a crash. You can typically write your events to independent stream and then join streams with this or another projection.

-yuriy

16

Yuriy,

I was afraid that’s the case.

No worries, I’ll change my system so I don’t write myself to the projection output stream.

A note in the documentation may be useful.

Many thanks for your support.
It’s much appreciated!

Jakub