The node memory in luatex is automatically reallocated when needed, so you have run out of memory or (more likely) run against the limit of the amount memory the operating system is willing to give to a single process.
Typically, this means a macro programming error, not a fault of luatex itself. If you run the input file with
\tracingall added, you will probably see (a lot of) recurring output indicating a bad recursion somewhere. No way to tell for sure without your input, but that diagnosis is a pretty safe bet.
So why the low limit? The number you see is not in bytes but in memory words, which are typically 8 bytes. The reallocation happens using realloc(), which means that both the new and the old value are active at the same time. What you see is the old number, so the new number will be nearly 2Gb ((1+1.25) * 8 * 100987760). Add in the rest of the luatex memory in use, and with only 2GB of real memory you should no longer be surprised of the fatal error.