Can't get counts for streams working user per-stream projections

EventStoreDB
1

I’m trying to work out the total number of events in some streams I have using Rob Ashton’s projection per-stream approach: http://codeofrob.com/entries/creating-a-projection-per-stream-in-the-eventstore.html.

These streams are created by an initial projection that looks like so:

fromStream(‘tests’)
.whenAny(function(state, evnt) {
linkTo(‘test-’ + evnt.data.Name, evnt);
});

So this will create streams like: test-Jimmy, test-Greg, test-Sarah etc that contain references to events in the main “tests” stream. I now want to get the count for each stream in the “test-” category. So I have another projection like so:

fromCategory(‘test’)

.whenAny(function(state, ev) {

if (state.count) {

state.count++;

} else {

state.count = 1;

}

return state;

});

But this projection (called “testCounts”) doesn’t have any state. I tried the following URLs:

I think I’m close, but any help would be very welcome.

2

Actually that second query was my playing around trying to get things to work. This is the version I tried based on Rob’s post:

fromCategory(‘test’)

.foreachStream()

.whenAny(function(s, e) {

if (!s.count) { s.count = 0 } s.count++

});

3

Ultimately, I just want to get the number of events in each “test-” stream.

4

There are better ways of doing this (I rewrote your query).

  1. make sure your streams actually exist eg test-greg etc

  2. try this query

fromCategory(‘test’)

.foreachStream()

.when({

$init : function(s,e) {return {count : 0}},

$any : function(s,e) s.count += 1} //mutate in place works

});

5

Cheers Greg, it’s still not working for me though.

I tried creating an entirely new database. Then I pumped some events into an index called people. Each event had a name: Greg, Jimmy, Sarah.

I then ran the initial projection to create a stream per-name using:

fromStream(‘people’)
.whenAny(function(state, evnt) {
linkTo(‘person-’ + evnt.data.Name, evnt);
});

As before, this works perfectly. I can now see the streams: test-Greg, test-Jimmy and test-Sarah. I ran this is a “one-time” projection with all the boxes ticked.

Then I took the query you gave me and made a new projection:

fromCategory(‘people’)
.foreachStream()
.when({
$init : function(s,e) {return {count : 0}},
$any : function(s,e) { s.count += 1} //mutate in place works
});

I ran this in the same way: one-time only with all the boxes ticked. But again, when I navigate to the state URLs:

I just get a blank response in the browser.

The projection shows this which looks intriguing:

Events/sec:Buffered events:0Events processed:0Partitions cached:2Reads in-progress:0Writes in-progress:0Write queue:0Write queue (chkp):0Checkpoint status:Position:$ce-people: -1Last checkpoint:$ce-people: -1

In the console I see:

[04836,14,19:22:33.868] SLOW BUS MSG [bus]: CreateAndPrepare - 81ms. Handler: ProjectionCoreService.
[04836,14,19:22:33.868] SLOW QUEUE MSG [Projection Core #2]: CreateAndPrepare - 81ms. Q: 0/0.
[04836,11,19:22:33.875] ‘StreamPerPerson’ projection source has been written
[04836,14,19:22:33.875] Creating an event distribution point at ‘people: -1’
[04836,14,19:22:33.875] The ‘96201dab-39e0-478a-981d-852c6d5a0827’ projection subscribed to the ‘490a34d5-dc89-49f7-92b5
-ca3a5e6cf4b5’ distribution point
[04836,14,19:22:33.954] The ‘96201dab-39e0-478a-981d-852c6d5a0827’ subscription has unsubscribed (reader: 490a34d5-dc89-
49f7-92b5-ca3a5e6cf4b5)
[04836,14,19:22:34.283] Writing checkpoint for StreamPerPerson at people: 5 with expected version number -1
[04836,14,19:22:34.541] Checkpoint has be written for projection StreamPerPerson at sequence number 0 (current)
{“allStreams”:false,“allEvents”:true,“byStreams”:true,“byCustomPartitions”:false,“categories”:[“people”],“streams”:[],“e
vents”:[],“definesStateTransform”:false,“options”:{“resultStreamName”:null,“partitionResultStreamNamePattern”:null,"$for
ceProjectionName":null,"$includeLinks":false,“reorderEvents”:false,“processingLag”:0}}
Category people requested
All events requested
[04836,11,19:23:42.420] ‘streamPerPersonCounts’ projection source has been written
[04836,12,19:23:42.420] Creating an event distribution point at ‘$ce-people: -1’
[04836,12,19:23:42.420] The ‘dee1e4d3-34cd-4138-94f2-877917b67997’ projection subscribed to the ‘3abaa5fd-78aa-42d9-af05
-56b5da2aa4c9’ distribution point
[04836,12,19:23:42.420] The ‘dee1e4d3-34cd-4138-94f2-877917b67997’ subscription has unsubscribed (reader: 3abaa5fd-78aa-
42d9-af05-56b5da2aa4c9)

All this is happening on Windows 8, using version 2.0.1 for .NET.

It’s probably something stupid that I’ve missed, but I would be super-grateful if anyone can help me. Even if you make me look stupid it’s still a win.

6

That last query was wrong, it should have been category “person” not “people”, like this:

fromCategory(‘person’)
.foreachStream()
.when({
$init : function(s,e) {return {count : 0}},
$any : function(s,e) { s.count += 1} //mutate in place works
});

But I still get the empty response unfortunately.

7

Just downloaded ES 3 rc2 and gave that a shot. When I ran the final query as one-time only I got this error:

A concurrency violation detected, but the projection is not running. Current state is: CompletingPhase. The reason for the restart is: ‘Checkpoint stream has been written to from the outside’

although the report said it did try to process 1 event.

I then tried running the projection as continuous (again with all the boxes ticked) but ran into the same problem as version 2.0.1

8

More info, the state requests are actually timing out on the server:

wget : The remote server returned an error: (408) Request Timeout.

At line:1 char:1

This is still with version 3 rc2

9

Can you run it as a query?

10

When running as a query it executes fine. On completion it shows a result of the query which contains a few obscure json fields:

{
  "$ph": 1,
  "$cp": true
}

But the state column is empty, unfortunately.

11

What is in results? you dont get one state as there are many states (one/stream)

12

13

14

what version are you running? You mentioned 3.0.

Also have you enabled the projections stream-by-category and by-category?

Greg

15

Initially I was using 2.0.1 but now I’m trying against 3.0. I’m starting like this: .\EventStore.SingleNode.exe --db .\ESData --run-projections=all

No, I haven’t used those settings. Are they start args, too?

16

Go to projections tab and enable them (might take a few minutes depending on the size of your database). You can see statuses

17

Cheers Greg I will give it a shot tomorrow.

18

btw enabling these will likely be default in the future we don’t want to make you take the cost of them now…

19

That seems to work fine on the query tab by enabling those. It makes sense that they are disabled by default, although if you’re not planning to make that happen for a while maybe you could update one of the ES blog posts (or maybe you did and I missed it).

Cheers again, though. I’m really fond of projections so far.

20

Nick, do you run standard projections on your system?

-yuriy