I have just successfully allocated 1 Hundred Million Nodes in a neural network. This was just to see if my PC and my program could handle it. I had to compile this for x64 because 32 bit mode cannot handle the memory allocation above about 2GB. But x64 works beautifully.
But size is just promise of additional capability – more important is architecture. SO I am back to much smaller, much more specialized neural circuits.
