Node: Failed to Discover after 10 attempts

Clients NodeJS
1

We’ve recently setup an EventStore Cloud cluster, single node. Using the connection string provided by the cluster page UX, we’re attempting a connection and receiving:

Error: Failed to discover after 10 attempts.
    at discoverEndpoint (/Users/code/test/node_modules/@eventstore/db-client/dist/Client/discovery.js:45:11)
    at Client.resolveUri (/Users/code/test/node_modules/@eventstore/db-client/dist/Client/index.js:178:43)
    at Client.createChannel (/Users/code/test/node_modules/@eventstore/db-client/dist/Client/index.js:151:25)
    at Client.createGRPCClient (/Users/code/test/node_modules/@eventstore/db-client/dist/Client/index.js:112:37)
    at /Users/code/test/node_modules/@eventstore/db-client/dist/Client/index.js:57:28
    at Subscription2.initialize (/Users/code/test/node_modules/@eventstore/db-client/dist/streams/utils/Subscription.js:27:18)

Here’s what our code doing the connecting looks like:

import { stringify } from 'querystring';
import { EventStoreDBClient } from '@eventstore/db-client';

      const options = {
        keepAliveInterval: 1000 * 60,
        keepAliveTimeout: 1000 * 60 * 10,
        tls: false
      };
      const params = stringify(options);
      this.client = EventStoreDBClient.connectionString`${connectionString}?${params}`;

There’s no documentation around this error, so we don’t even know where to begin to debug.

1 Like
2

The error can be translated to a simple “I can’t connect to the node”.

I am not sure where your application runs, but if you run it locally, your machine needs to be connected to your cloud VPC that is peered with ES Cloud via some sort of point to site connection (VPN). https://developers.eventstore.com/cloud/use/#networking

3

I am having the same problem with a 3 node cluster using docker. Did you ever find a solution?

4

Hey, what is your connection string and your docker setup?

5

My docker compose:

## Inference Pool Docker Compose
#
# Event Store Cluster
# https://developers.eventstore.com/server/v24.6/installation.html#use-docker-compose
# Sets up a 3 node cluster with certficates 
#
#
# Nest JS API
# The api of the application. Runs on port 3000
#
# Client (Vite / React)
#  

services:
  # Setup service for generating certificates
  setup:
    image: eventstore/es-gencert-cli:1.0.2
    entrypoint: bash
    user: "1000:1000"
    command: >
      -c "mkdir -p ./certs && cd /certs && es-gencert-cli create-ca && es-gencert-cli create-node -out ./node1 -ip-addresses 127.0.0.1,172.30.240.11 -dns-names localhost && es-gencert-cli create-node -out ./node2 -ip-addresses 127.0.0.1,172.30.240.12 -dns-names localhost && es-gencert-cli create-node -out ./node3 -ip-addresses 127.0.0.1,172.30.240.13 -dns-names localhost && find . -type f -print0 | xargs -0 chmod 666"
    container_name: setup
    volumes:
      - ./certs:/certs

  # Node 1 for Event Store Cluster
  node1.eventstore: &template
    image: eventstore/eventstore:24.2.0-jammy
    container_name: node1.eventstore
    env_file:
      - vars.env
    environment:
      - EVENTSTORE_INT_IP=172.30.240.11
      - EVENTSTORE_ADVERTISE_HTTP_PORT_TO_CLIENT_AS=2111
      - EVENTSTORE_GOSSIP_SEED=172.30.240.12:2113,172.30.240.13:2113
      - EVENTSTORE_TRUSTED_ROOT_CERTIFICATES_PATH=/certs/ca
      - EVENTSTORE_CERTIFICATE_FILE=/certs/node1/node.crt
      - EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/certs/node1/node.key
    healthcheck:
      test: [ "CMD-SHELL", "curl --fail --insecure https://node1.eventstore:2113/health/live || exit 1" ]
      interval: 5s
      timeout: 5s
      retries: 24
    ports:
      - 2111:2113
    volumes:
      - ./certs:/certs
      - node1-data:/var/lib/eventstore # Data volume for node1
      - node1-logs:/var/log/eventstore # Logs volume for node1
    depends_on:
      - setup
    restart: always
    networks:
      clusternetwork:
        ipv4_address: 172.30.240.11

  # Node 2 for Event Store Cluster
  node2.eventstore:
    <<: *template
    container_name: node2.eventstore
    environment:
      - EVENTSTORE_INT_IP=172.30.240.12
      - EVENTSTORE_ADVERTISE_HTTP_PORT_TO_CLIENT_AS=2112
      - EVENTSTORE_GOSSIP_SEED=172.30.240.11:2113,172.30.240.13:2113
      - EVENTSTORE_TRUSTED_ROOT_CERTIFICATES_PATH=/certs/ca
      - EVENTSTORE_CERTIFICATE_FILE=/certs/node2/node.crt
      - EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/certs/node2/node.key
    healthcheck:
      test: [ "CMD-SHELL", "curl --fail --insecure https://node2.eventstore:2113/health/live || exit 1" ]
      interval: 5s
      timeout: 5s
      retries: 24
    ports:
      - 2112:2113
    volumes:
      - ./certs:/certs
      - node2-data:/var/lib/eventstore # Data volume for node2
      - node2-logs:/var/log/eventstore # Logs volume for node2
    networks:
      clusternetwork:
        ipv4_address: 172.30.240.12

  # Node 3 for Event Store Cluster
  node3.eventstore:
    <<: *template
    container_name: node3.eventstore
    environment:
      - EVENTSTORE_INT_IP=172.30.240.13
      - EVENTSTORE_ADVERTISE_HTTP_PORT_TO_CLIENT_AS=2113
      - EVENTSTORE_GOSSIP_SEED=172.30.240.11:2113,172.30.240.12:2113
      - EVENTSTORE_TRUSTED_ROOT_CERTIFICATES_PATH=/certs/ca
      - EVENTSTORE_CERTIFICATE_FILE=/certs/node3/node.crt
      - EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/certs/node3/node.key
    healthcheck:
      test: [ "CMD-SHELL", "curl --fail --insecure https://node3.eventstore:2113/health/live || exit 1" ]
      interval: 5s
      timeout: 5s
      retries: 24
    ports:
      - 2113:2113
    volumes:
      - ./certs:/certs
      - node3-data:/var/lib/eventstore # Data volume for node3
      - node3-logs:/var/log/eventstore # Logs volume for node3
    networks:
      clusternetwork:
        ipv4_address: 172.30.240.13

  # NestJS Application
  nestjs-app:
    build: .
    ports:
      - '3000:3000'
    depends_on:
      - node1.eventstore
      - node2.eventstore
      - node3.eventstore
    environment:
      - EVENTSTORE_DB_URL=esdb://admin:[email protected]:2111,node1.eventstore:2112,node1.eventstore:2113?tls=true&tlsVerifyCert=false

    networks:
      clusternetwork:
        ipv4_address: 172.30.240.14

  # Client Application
  client:
    build:
      context: ./client
    ports:
      - '5173:5173'
    environment:
      - VITE_API_URL=http://localhost:5173 # Adjust this based on your API URL

networks:
  clusternetwork:
    name: eventstoredb.local
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.30.240.0/24

volumes:
  eventstore-volume-data:
  eventstore-volume-logs:
  node1-data: # Volume for node1 data
  node1-logs: # Volume for node1 logs
  node2-data: # Volume for node2 data
  node2-logs: # Volume for node2 logs
  node3-data: # Volume for node3 data
  node3-logs:

Connection string:

  - EVENTSTORE_DB_URL=esdb://admin:[email protected]:2111,node2.eventstore:2112,node3.eventstore:2113?tls=true&tlsVerifyCert=false
6

I have update my connection string based on the client documentation (Getting started | EventStoreDB Documentation) instead of the one in the server guide for the cluster install:

esdb+discover://admin:[email protected]:2113,node2.eventstore:2113,node3.eventstore:2113?tls=true&tlsVerifyCert=false

and I’m still getting the same error:

nestjs-app-1      | esdb+discover://admin:[email protected]:2113,node2.eventstore:2113,node3.eventstore:2113?tls=true&tlsVerifyCert=false 
nestjs-app-1      | Error: Failed to discover after 10 attempts.
nestjs-app-1      |     at discoverEndpoint (/app/node_modules/@eventstore/db-client/dist/Client/discovery.js:45:11)
nestjs-app-1      |     at async Client.resolveUri (/app/node_modules/@eventstore/db-client/dist/Client/index.js:270:43)                          
nestjs-app-1      |     at async Client.createChannel (/app/node_modules/@eventstore/db-client/dist/Client/index.js:244:25)                       
nestjs-app-1      |     at async Client.createGRPCClient (/app/node_modules/@eventstore/db-client/dist/Client/index.js:200:37)
nestjs-app-1      |     at async /app/node_modules/@eventstore/db-client/dist/Client/index.js:145:28                                              
nestjs-app-1      |     at async ReadStream.initialize (/app/node_modules/@eventstore/db-client/dist/streams/utils/ReadStream.js:25:18)

I can curl the nodes in the nest js docker container with out issue, I can also make requests from outside the container on localhost ports 2111-2113. The only thing I can’t seem to achieve is connecting via gRPC client. Any help would be greatly appreciated.

7

If you pass gossip seeds directly, you don’t need esbd+discover.

I think your mistake is to not use localhost instead of node*.eventstore. Your application is running inside or outside of docker?