Emit in projection doesn't create/write to new stream

Farhan_Nasim · April 12, 2020, 2:04am

Hi All,

I am using EventStore 5.0.1-1 official Docker image.
I have only one stream (called title:A4CAABEE-D8A1-4581-9444-61950FB7584E, with 2 events) and only the system projections enabled.
I would like to re-write the stream to a new stream called new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E.

Here follows my OneTime projection called migration:

fromStream(‘title:A4CAABEE-D8A1-4581-9444-61950FB7584E’)
.when(function (s, e) {
var newStreamName = ‘new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E’;
emit(newStreamName, e.eventType, e.data, e.metadata);
})

The detailed definition of the projection as returned by ES server for clarity.

➜ ~ curl -i http://localhost:2113/projection/migration/query%3Fconfig=yes -H “Accept: application/json” -u “admin:changeit”
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, PUT, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:33:11 GMT
Content-Length: 574
Keep-Alive: timeout=15,max=100
{
“msgTypeId”: 262,
“name”: “migration”,
“query”: “fromStream(‘title:A4CAABEE-D8A1-4581-9444-61950FB7584E’)\r\n.when(function (s, e) {\r\n var newStreamName = ‘new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E’;\r\n emit(newStreamName, e.eventType, e.data, e.metadata);\r\n})\r\n”,
“emitEnabled”: true,
“definition”: {
“allEvents”: true,
“categories”: [],
“events”: [],
“streams”: [
“title:A4CAABEE-D8A1-4581-9444-61950FB7584E”
],
“options”: {}
},
“outputConfig”: {
“resultStreamName”: “$projections-migration-result”
}
}%

The projection seems to have finished properly.

➜ ~ curl -i http://localhost:2113/projections/any -H “Accept: application/json” -u “admin:changeit”
{
“coreProcessingTime”: 0,
“version”: 2,
“epoch”: -1,
“effectiveName”: “migration”,
“writesInProgress”: 0,
“readsInProgress”: 0,
“partitionsCached”: 1,
“status”: “Completed/Stopped/Writing results”,
“stateReason”: “”,
“name”: “migration”,
“mode”: “OneTime”,
“position”: “Phase: 1 (completed)”,
“progress”: 100.0,
“lastCheckpoint”: “Phase: 1 (completed)”,
“eventsProcessedAfterRestart”: 1,
“statusUrl”: “http://localhost:2113/projection/migration”,
“stateUrl”: “http://localhost:2113/projection/migration/state”,
“resultUrl”: “http://localhost:2113/projection/migration/result”,
“queryUrl”: “http://localhost:2113/projection/migration/query%3Fconfig=yes”,
“enableCommandUrl”: “http://localhost:2113/projection/migration/command/enable”,
“disableCommandUrl”: “http://localhost:2113/projection/migration/command/disable”,
“checkpointStatus”: “”,
“bufferedEvents”: 0,
“writePendingEventsBeforeCheckpoint”: 0,
“writePendingEventsAfterCheckpoint”: 0
}
➜ ~ curl -i http://localhost:2113/projection/migration/state -H “Accept: application/json” -u “admin:changeit” HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:47:35 GMT
Content-Length: 306
Keep-Alive: timeout=15,max=100
{“eventBody”:"{“eventVersion”:1,“occurredOn”:“1586655651224”,“titleId”:{“id”:“A4CAABEE-D8A1-4581-9444-61950FB7584E”},“copyAddedCount”:2,“oldCopyCount”:1,“newCopyCount”:3}",“eventId”:-1,“occurredOn”:“1586655651224”,“typeName”:“com.fnasim.esbookstore.inventory.domain.model.TitleCopyAdded”}%

However, the new stream new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E is not created.

➜ ~ curl -i http://localhost:2113/streams/new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E -H “Accept: application/json” -u “admin:changeit”
HTTP/1.1 404 Not Found
Access-Control-Allow-Methods: POST, DELETE, GET, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Content-Type: text/plain; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:50:59 GMT
Content-Length: 0
Keep-Alive: timeout=15,max=100

Did I miss anything or got something wrong in the projection definition? Further details are available on request. Please help.

yorick.laupa · April 12, 2020, 9:07am

Looks like you didn’t allow the projection to emit events. Adding emit=true in the URL query parameter list should fix your problem.

Farhan_Nasim · April 12, 2020, 9:47am

Hi Yorick,

Thanks for the reply. However, I can see emit had been very much enabled.

Here follows the projection query response:

➜ ~ curl -i http://localhost:2113/projection/migration/query%3Fconfig=yes -H “Accept: application/json” -u “admin:changeit”
HTTP/1.1 200 OK
Access-Control-Allow-Methods: GET, PUT, OPTIONS
Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion
Cache-Control: max-age=0, no-cache, must-revalidate
Vary: Accept
Content-Type: application/json; charset=utf-8
Server: Mono-HTTPAPI/1.0
Date: Sat, 11 Apr 2020 00:33:11 GMT
Content-Length: 574
Keep-Alive: timeout=15,max=100
{
“msgTypeId”: 262,
“name”: “migration”,
“query”: “fromStream(‘title:A4CAABEE-D8A1-4581-9444-61950FB7584E’)\r\n.when(function (s, e) {\r\n var newStreamName = ‘new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E’;\r\n emit(newStreamName, e.eventType, e.data, e.metadata);\r\n})\r\n”,
“emitEnabled”: true,
“definition”: {
“allEvents”: true,
“categories”: [],
“events”: [],
“streams”: [
“title:A4CAABEE-D8A1-4581-9444-61950FB7584E”
],
“options”: {}
},
“outputConfig”: {
“resultStreamName”: “$projections-migration-result”
}
}%

And, just in case, the projection configs view:

yorick.laupa · April 12, 2020, 10:07am

Hi Farhan,

I just saw in one of your cURL results that emit is set to true. However, emitting events only work with continuous projections. Based on your previous image, it appears you configured a one-time projection.

Please switch to continuous projection and see if it solves your problem.

Farhan_Nasim · April 12, 2020, 10:23am

Hi Yorick,

Yes, last projection had been One-Time.

emitting events only work with continuous projections

Didn’t know that. Any reason for that constraint?

Have created a new Continuous projection (migration_new) with the same operation, however, still no good.

➜ ~ curl -i http://localhost:2113/projection/migration_new/query%3Fconfig=yes -H “Accept: application/json” -u “admin:changeit”

HTTP/1.1 200 OK

Access-Control-Allow-Methods: GET, PUT, OPTIONS

Access-Control-Allow-Headers: Content-Type, X-Requested-With, X-Forwarded-Host, X-Forwarded-Prefix, X-PINGOTHER, Authorization, ES-LongPoll, ES-ExpectedVersion, ES-EventId, ES-EventType, ES-RequiresMaster, ES-HardDelete, ES-ResolveLinkTos

Access-Control-Allow-Origin: *

Access-Control-Expose-Headers: Location, ES-Position, ES-CurrentVersion

Cache-Control: max-age=0, no-cache, must-revalidate

Vary: Accept

Content-Type: application/json; charset=utf-8

Server: Mono-HTTPAPI/1.0

yorick.laupa · April 12, 2020, 10:51am

Your script seems wrong. Please use that one instead:

fromStream('title:A4CAABEE-D8A1-4581-9444-61950FB7584E')
    .when({
        $any: function (s, e) {
            var newStreamName = 'new_title:A4CAABEE-D8A1-4581-9444-61950FB7584E';
            emit(newStreamName, e.eventType, e.data, e.metadata);
        }
    })

``

Farhan_Nasim · April 12, 2020, 10:57am

You are absolutely right! With the script corrected, both versions (One-Time and Continuous) works now.

Thanks a lot, Yorick.