Too many open files errors causing restart loop

We have a 3-node cluster running on Ubuntu Trusty where one of the nodes is consistently failing to initialize due to an error during ReadIndex rebuilds. The log states:

Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: [80860,20,17:54:48.628] Loading PTable (Version: 4) ‘744580c8-0a71-4dd0-b97e-0c0c1bbbb5ad’ (1000000 entries, cache depth 16) done in 00:00:00.3474961.
Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: [80860,20,17:54:48.633] PTables merge started (specialized for <= 2 tables).
Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: [80860,20,17:54:48.635] Error in TableIndex.ReadOffQueue
Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: Too many open files
Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: [80860,20,17:54:48.659] Global Unhandled Exception occurred.
Jun 29 17:54:48 DASDSEVENTNIX04 eventstored[80860]: Too many open files
Jun 29 17:54:48 DASDSEVENTNIX04 systemd[1]: eventstore.service: Main process exited, code=exited, status=255/n/a
Jun 29 17:54:48 DASDSEVENTNIX04 systemd[1]: eventstore.service: Unit entered failed state.
Jun 29 17:54:48 DASDSEVENTNIX04 systemd[1]: eventstore.service: Failed with result ‘exit-code’.
Jun 29 17:54:48 DASDSEVENTNIX04 systemd[1]: eventstore.service: Service hold-off time over, scheduling restart.

We have increased the max open files limits on the system to 500,000 from the default 1024 (max supported by the OS is ~700k) per https://gist.github.com/luckydev/b2a6ebe793aeacf50ff15331fb3b519d

We have tried to rebuild the box and replaced the node completely with a fresh install that repopulated from the rest of the cluster over a period of several hours and things were fine for a few days.

The issue does not seem to be affecting the other two nodes in the cluster.

I am not sure how to proceed from here as there don’t seem to be any similar issues reported.