What’s wrong with this docker compose file?
Purpose: Create a test eventstore cluster on the local machine for running unit tests.
Constraint: The three nodes cannot be defined in a single service of size ‘3’ because our production rancher system requires them defined separately to deploy to specific hosts.
Problem: Leader election is timing out
Attached is docker-compose.yml and the three output log files.
docker-compose.yml (2.99 KB)
1.txt (14.9 KB)
2.txt (15.5 KB)
3.txt (16.1 KB)
Here is the compose file on rancher that actually works:
It’s different in the sense that it knows how to assign public and private IPs to eventstore configuration, and uses labels to assign specific nodes to specific hosts.
version: ‘2’
services:
eventstore3:
image: eventstore/eventstore:release-4.0.3
environment:
EVENTSTORE_CLUSTER_DNS: eventstore1
EVENTSTORE_CLUSTER_GOSSIP_PORT: ‘2112’
EVENTSTORE_CLUSTER_SIZE: ‘3’
EVENTSTORE_EXT_HTTP_PORT: ‘2113’
EVENTSTORE_EXT_HTTP_PREFIXES: http://*:2113/
EVENTSTORE_EXT_TCP_PORT: ‘1113’
EVENTSTORE_INT_HTTP_PORT: ‘2112’
EVENTSTORE_INT_HTTP_PREFIXES: http://*:2112/
EVENTSTORE_INT_TCP_PORT: ‘1112’
EVENTSTORE_RUN_PROJECTIONS: All
EVENTSTORE_START_STANDARD_PROJECTIONS: ‘True’
entrypoint:
volumes:
/backup/eventstore3/data:/var/lib/eventstore
/backup/eventstore3/log:/var/log/eventstore
ports:
2113:2113/tcp
1113:1113/tcp
command:
-c
'apt-get install curl && export HOST_PUBLIC_IP=$$(curl http://rancher-metadata/2015-12-19/self/host/agent_ip)
&& grep -q -F “IntIpAdvertiseAs: $$(hostname -i)” /etc/eventstore/eventstore.conf ||
echo “IntIpAdvertiseAs: $$(hostname -i)” >> /etc/eventstore/eventstore.conf &&
grep -q -F “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” /etc/eventstore/eventstore.conf ||
echo “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” >> /etc/eventstore/eventstore.conf
&& /entrypoint.sh’
labels:
io.rancher.scheduler.affinity:host_label: eventstore=eventstore3
io.rancher.scheduler.global: ‘true’
eventstore1:
image: eventstore/eventstore:release-4.0.3
environment:
EVENTSTORE_CLUSTER_DNS: eventstore1
EVENTSTORE_CLUSTER_GOSSIP_PORT: ‘2112’
EVENTSTORE_CLUSTER_SIZE: ‘3’
EVENTSTORE_EXT_HTTP_PORT: ‘2113’
EVENTSTORE_EXT_HTTP_PREFIXES: http://*:2113/
EVENTSTORE_EXT_TCP_PORT: ‘1113’
EVENTSTORE_INT_HTTP_PORT: ‘2112’
EVENTSTORE_INT_HTTP_PREFIXES: http://*:2112/
EVENTSTORE_INT_TCP_PORT: ‘1112’
EVENTSTORE_RUN_PROJECTIONS: All
EVENTSTORE_START_STANDARD_PROJECTIONS: ‘True’
entrypoint:
volumes:
/backup/eventstore1/data:/var/lib/eventstore
/backup/eventstore1/log:/var/log/eventstore
ports:
2113:2113/tcp
1113:1113/tcp
command:
-c
'apt-get install curl && export HOST_PUBLIC_IP=$$(curl http://rancher-metadata/2015-12-19/self/host/agent_ip)
&& grep -q -F “IntIpAdvertiseAs: $$(hostname -i)” /etc/eventstore/eventstore.conf ||
echo “IntIpAdvertiseAs: $$(hostname -i)” >> /etc/eventstore/eventstore.conf &&
grep -q -F “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” /etc/eventstore/eventstore.conf ||
echo “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” >> /etc/eventstore/eventstore.conf
&& /entrypoint.sh’
labels:
io.rancher.scheduler.affinity:host_label: eventstore=eventstore1
io.rancher.scheduler.global: ‘true’
eventstore2:
image: eventstore/eventstore:release-4.0.3
environment:
EVENTSTORE_CLUSTER_DNS: eventstore1
EVENTSTORE_CLUSTER_GOSSIP_PORT: ‘2112’
EVENTSTORE_CLUSTER_SIZE: ‘3’
EVENTSTORE_EXT_HTTP_PORT: ‘2113’
EVENTSTORE_EXT_HTTP_PREFIXES: http://*:2113/
EVENTSTORE_EXT_TCP_PORT: ‘1113’
EVENTSTORE_INT_HTTP_PORT: ‘2112’
EVENTSTORE_INT_HTTP_PREFIXES: http://*:2112/
EVENTSTORE_INT_TCP_PORT: ‘1112’
EVENTSTORE_RUN_PROJECTIONS: All
EVENTSTORE_START_STANDARD_PROJECTIONS: ‘True’
entrypoint:
volumes:
/backup/eventstore2/data:/var/lib/eventstore
/backup/eventstore2/log:/var/log/eventstore
ports:
2113:2113/tcp
1113:1113/tcp
command:
-c
'apt-get install curl && export HOST_PUBLIC_IP=$$(curl http://rancher-metadata/2015-12-19/self/host/agent_ip)
&& grep -q -F “IntIpAdvertiseAs: $$(hostname -i)” /etc/eventstore/eventstore.conf ||
echo “IntIpAdvertiseAs: $$(hostname -i)” >> /etc/eventstore/eventstore.conf &&
grep -q -F “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” /etc/eventstore/eventstore.conf ||
echo “ExtIpAdvertiseAs: $$HOST_PUBLIC_IP” >> /etc/eventstore/eventstore.conf
&& /entrypoint.sh’
labels:
io.rancher.scheduler.affinity:host_label: eventstore=eventstore2
io.rancher.scheduler.global: ‘true’
For anyone who’s interested, here’s how I got it all working:
https://gist.github.com/bboyle1234/07ad0d223fa57f0548bb0666097c545f
A disadvantage of the system is that node “eventstore1” must be alive in order for cluster gossiping to work.
I would appreciate suggestions about how to improve my work, within the constraint that the three nodes must have different container names / hostnames.
It might be cool if the EVENT_STORE_CLUSTER_DNS variable could accept multiple hostnames, or multiple host:port combinations, OR,
It might be cool if the EVENT_STORE_GOSSIP_SEED variable could accept host:port as well as instead of IP:port.
“A disadvantage of the system is that node “eventstore1” must be alive in order for cluster gossiping to work.”
You know that you can assign multiple ips to a DNS entry?
if not clear make an entry eventstore-cluster-discover and put all the nodes in it 
Yeah
Thank you, that’s a great suggestion and logical next step.