I appreciate the answer to this problem is do not event source everything / as much. The problem is the goal is 10x growth which goes up to 10gig a day.
This isn’t a storage problem per say - its more a cost * time problem. In this situation old streams are probably not going to be read again - really streams once they are over 5 days old will not get read again. IE the investment in storage for them over time will never give a return after 5 days (Except a few cases with audit requirements - but keeping them is fine).
There must be many systems that operate at this level but are cost-sensitive.
What is the best practice?
I could just delete them, but Feels Bad Man.
Is there a way of moving part of the store onto cold storage (and then back?!)?
I feel like I’ve backed myself into a simple logical contradiction - just want someone to confirm my options are simply
a) delete it
or b) pay for it.
or even better give me more, make this a real interesting bit of pain