Neuromorphic hardware is going to jump many orders of magnitude over classic hardware. When we get a RAM that can execute multiple layers in parallel at once, per clock tick, we'll see whole AI ecosystems cooperating to get a solution in a fraction of the time a single modern NN would take.