Why the 100GB hard limit? If you can stream from disk you should be able to process infinite data sets if the operations allow it. If you need to do stuff like deduplication, then it will depend on how much RAM you have available.
There's no "limit", just a kind of practical size I have seen. It's a single VM.
You will need NVMe to prevent disk being a bottleneck (NVMe is now 8 GB/s vs 200 MB/s for S3).
