When running with --mem-db, is there a way to force the system projections to be Running?

EventStoreDB
1

We are trying to automate some end-to-end smoke-tests and thought it would be awesome to use --mem-db mode.

When we run with ./EventStore.ClusterNode.exe --mem-db --run-projections=all we expected that we’d see:

$by_category, $by_event_type, $stream_by_category, and $streams to be in the Running state, but they’re all Stopped.

We noticed we could manually curl -X POST --user admin:changeit http://localhost:2113/projection/$by_event_type/command/enable -H “Content-Type: 0” etc to start these, but is there any other way without needing to do this?

We also tried --run-projections=system and had the same result.

Thanks!

Josh

2

Yes because they have to be enabled. Even when running projections a
vast portion of people don't want all of them running. Once you have
enabled them they stay on but they need to be enabled first.

3

OK, so issuing the curl commands or equivalent POST code in another tool is the best / only way to turn those on. Cool thanks

4

Or using projection manager (which can also do this)

5

You mean through the UI on http://localhost:2113/web/index.html#/projections right?

That just sends X number of POST requests for each item that’s not running. Since we’re running these tests against an API, I think we’ll add a method to our client that does this same thing for us without the UI.

6

See projection manager in client api

https://github.com/EventStore/EventStore/blob/dev/src/EventStore.ClientAPI/ProjectionsManager.cs#L39

Or there is a command line app called padmin you can shell to

Greg

7

We ended up just doing this:

cd /c/Program\ files/eventstore

./EventStore.ClusterNode.exe --mem-db --run-projections=all &

sleep 15

curl -X POST --user admin:changeit http://localhost:2113/projection/$by_event_type/command/enable -H “Content-Length: 0”

curl -X POST --user admin:changeit http://localhost:2113/projection/$by_category/command/enable -H “Content-Length: 0”

curl -X POST --user admin:changeit http://localhost:2113/projection/$stream_by_category/command/enable -H “Content-Length: 0”

curl -X POST --user admin:changeit http://localhost:2113/projection/$streams/command/enable -H “Content-Length: 0”

working well!

8

Cool! I never thought of doing that.

Couldn’t this be a cmd line switch? i.e. --start-projections=$by_event_type;$by_category;foo;bar

9

You only need to start them once. It would only ever be used the first
time it started up. All other times they start automatically (if ever
started before)

10

btw there is another tool shipped called padmin that can build the
http requests for you from the command line.

11

I get that, but during development I’m constantly starting fresh without a --db so I can simulate first-time runs. I always have to log into the web interface and start the damn $by_category projections each time I start fresh. It’s annoying. Additionally there are surely use cases for starting up an ES instance, starting system projections, running for a short time to gather and output data, and then die… Just seems it would be nice to have a cmd-line switch that started projections as part of ES startup.

Though like I said, I never though of doing the CURL requests like the poster. So this could be easily scripted.

12

Its been mentioned every time this issue has come up. Use curl or
padmin.exe (it builds the http requests as well).

"Just seems it would be nice to have a cmd-line switch that started
projections as part of ES startup"

why when I can write:

EventStore.Cluster.Node.exe --.......
padmin start $by_category

13

btw if you are running tests etc on top then use in client api in your
setup fixture etc

https://github.com/EventStore/EventStore/blob/dev/src/EventStore.ClientAPI/ProjectionsManager.cs#L39

14

Hmm. “EventStore.PAdmin.exe” fails by default, says I need some kind of config file?

ES starts up and doesn’t yield, so I need a second console to run this padmin?

I suppose the question really is… How can I open a single console app, start ES AND start a system projection in one line?

15

START /B EventStore.ClusterNode.exe

its the equivalent of EventStore.ClusterNode.exe & in unix

Then put this and your other lines in a .bat file

myes.bat

START /B EventStore.ClusterNode.exe --mem-db --run-projections=all
timeout /t 5
curl -X POST --user admin:changeit
http://localhost:2113/projection/\$by_event_type/command/enable -H
"Content-Length: 0"
curl -X POST --user admin:changeit
http://localhost:2113/projection/\$by_category/command/enable -H
"Content-Length: 0"
curl -X POST --user admin:changeit
http://localhost:2113/projection/\$stream_by_category/command/enable
-H "Content-Length: 0"
curl -X POST --user admin:changeit
http://localhost:2113/projection/\$streams/command/enable -H
"Content-Length: 0"

as an example then type myes.bat at command line

instead of curl try:

padmin enable $by_category admin changeit

running padmin I get:

C:\Users\Greg\src\EventStore>bin\padmin\padmin.exe

padmin <command> <arg1 arg2 ... argN>
commands available :

enable <projection-name> [<login> <password>]
disable <projection-name> [<login> <password>]
create <onetime | adhoc | continuous | persistent> <projection-name> <query file

[<login> <password>]

list <all | onetime | adhoc | continuous | persistent> [<login> <password>]
status <projection-name> [<login> <password>]
state <projection-name> [<login> <password>]
statistics <projection-name> [<login> <password>]
show-query <projection-name> [<login> <password>]
update-query <projection-name> <query file> [<login> <password>]
delete <projection-name> [<login> <password>]

and it works.

16

I understand… bat scripts, timeouts, and external tools that need to be downloaded separately from ES. My comment was simply that it’s a lot of ceremony for something that a --start-projections=xxx could alleviate. I’ll leave it at that. I think ES is a great product and I’m only interested in seeing it improve.

17

"I understand... bat scripts, timeouts, and external tools that need
to be downloaded separately from ES."

padmin ships with es

18

OK, thanks Greg, this is easier than curl.

Chris, are you running the command from CMD / powershell or from Git Bash?

When running from Git Bash, I had to escape the projection name, like this:

jgough@JGOUGH /c/Program Files/eventstore

$ ./EventStore.PAdmin.exe enable $by_category admin changeit

Config file failed to load (Could not find file 'c:\Program Files\eventstore\pad

min.esconfig’.).

Should be located near exe file, named ‘padmin.esconfig’, key=value formatted).

Using defaults

Loading config values…

Enabling $by_category…

$by_category enabled

[29772,13,16:56:11.707] Enabling ‘$by_category’ projection

[29772,13,16:56:11.707] ‘$by_category’ projection source has been written

If I don’t escape it, the shell tries to find a variable named by_category, and that is missing so I get this:

$ ./EventStore.PAdmin.exe enable $by_category admin changeit

Config file failed to load (Could not find file 'c:\Program Files\eventstore\pad

min.esconfig’.).

Should be located near exe file, named ‘padmin.esconfig’, key=value formatted).

Using defaults

Loading config values…

Enabling …

padmin execution failed : System.ArgumentNullException: name

at EventStore.Padmin.Command.Enable(ProjectionsManager manager, String[] comm

andArgs) in c:\EventStore\src\EventStore.Padmin\Command.cs:line 16

at EventStore.Padmin.Program.Execute(ProjectionsManager manager, String[] arg

s) in c:\EventStore\src\EventStore.Padmin\Program.cs:line 98

at EventStore.Padmin.Program.TryExecute(Dictionary`2 config, String[] args) i

n c:\EventStore\src\EventStore.Padmin\Program.cs:line 81

The command works from CMD.exe without any escaping though.