partitionBy

EventStoreDB
1

I just switched to dev and noticed this doesn’t exist here?

Which branch should I be working on? Or should I not be using partitionBy any more?

2

Rob, nothing has changed in regard to partitionBy. How do you see that it doesn’t exist anymore?

3

With this projection:


fromStream('github')
  .partitionBy(function(ev) {
      return ev.body.repo.language
  })
  .when({
    "$init": function() {
      return {
         commits: 0, sadcount: 0, happycount: 0
      }
    },
    "PushEvent": function(state, ev) {
       for(var i =0 ; i < ev.body.payload.commits.length ; i++)
           collectHappinessIndexOfCommit(ev.body.payload.commits[i], state)
    },
  })


I get the message "The requested feature is not implemented", if I remove the partition it works fine. - if I'm in the projections branch, this works as expected
4

I must have broken something. Let me check.

5

:slight_smile:

When this works I will take over the world.

6

pushed. It just should do nothing instead of throwing exception. NoopResultEmitter :slight_smile:

7

I thought it did partitioning and stuff - in the projections branch it would create me a lovely results object with each of the results - is this not the case any more?

8

Rob,

  1. I create branches with projections- prefix when I do something bigger on projections. The branch named “projections” was the first one, so it has so simple name. I’m now deleting it as I’m sure I’m not going to merge it. Typically projections- branches are unstable, so it’s better to watch “dev”.

  2. We are introducing so called projection results stream(s) (similar to what we had as -state streams) and a way to transform current projection state into projection result before the writing to -result stream. It should allow you encapsulating more details in the projection state than you need to expose outside. If you do not transform projection state these -result streams should be the same for you as -state streams before.

However, there is currently no way to make a projection to write results without a call to transformBy() function. So, for now, you can end your projection with :

.transformBy(function(s) { return s; })

which will enable -result streams.

I’ll add a separate

.outputTo('stream-name/pattern)

function to configure output stream names and/or enable results eben without transformBy()

-yuriy

9

with the state noy the result. results are different

10

I’ll read this in a few hours and try and understand - Greg tried to get that across to me by e-mail the other evening but it went right over my head

I’ll give it a go

11

Okay, now I’ve read this (couldn’t wait until I have time so I’m making it)

What I want

I have a stream with a pile of stuff in it, each thing has a “language” associated with it, previously I had two ways to manage this

Option one

  • Create a projection that does linkTo(‘language-’ + ev.body.language’, ev)

  • Create another projection which does fromCategory(‘language’).foreachStream()

I have/had two problems with this,

  • one was there wasn’t an easy way to get a list of languages (fine, i can index myself if I really want to)

  • two was it never seemed to really work

Option two

  • Do the partitioning with a partitionBy in a single projection

  • Look at /all-states to get all the states with their counts

I really liked option two for its simplicity

I don’t see how what you said above maps to this.

12

option 1 can in psrallelized. option 2 is serial.

13

Yeah, I get that, it’s just a shame that it has never worked for me :wink:

14

So

If I have a projection:

fromStream(‘github’)

.when({

“PushEvent”: function(s, ev) {

linkTo(‘pushlanguage-’ + ev.body.repo.language, ev)

}

})

And another projection

fromCategory(‘pushlanguage’)

.foreachStream()

.when({

$init: function(s, ev) {

return { count: 0 }

},

“PushEvent”: function(s, ev) {

s.count++

}

})

This should work? Where should I see the results of this?

(There are streams being created because of the first projection, pushlanguage-Cobol etc, so that’s not the problem)

15

Are you running dev? There is transformedby and yuriy has pushed the results output

16

OutputTo(stream, pattern for the results streams)

17

I’m running dev

So what do I need to do to

a) Make this projection work

b) See the results

18

Rob,

there was a problem with categorizing linktos earlier, which you got when you first tried this approach some time ago. Ipushed a fixed for categorization issue today into dev. So, the linkTo(‘categoryname-’… approach should work now. there are other ways to do it as Greg mentioned.

-yuriy

19

Oh cool

I like that there are options available to me, it’s a bit unclear as to what they are though (I guess I could read the source code but man, I’ve been putting that off for months now)

I’ll try and make it work with fromCategory, is there an equivalent to all-states for this? (Or do I need to do one of these mysterious other options? :))

20

Rob,

just realized that the most trivial case will not work (will push a fix in a minute), but in general

the structure of projection is the following now:

formall/fromStream/fromCategory()

[.foreachStream() ]
.when()
[.transformBy/filterBy()]
[.outputTo(resultStreamName, pattern-to-name-result-streams-per-stream)]

sample:

fromAll().foreachStream().when({

$init: function() {

return {count: 0};

},

$any: function(s, e) {

s.count++;

}

})

.filterBy(function(s){

return s.count % 10 === 0;

})

.transformBy(function(s) {

return { Count: s.count};

})

.outputTo(“test132”, “test13-{0}”)