Couple of questions that I have been pondering:
As the nature is with chips/memory, it has historically been a race to the bottom as they become commoditized, after this initial hysteria, this too shall level out.
This has been true, certainly for memory and storage, but the cycles have been becoming more shallow. The number of suppliers have been dwindling, with three large suppliers with a large technology and market lead, and then a few about 3 to 4 generations behind in technology (that are benefiting from the supply shortage (The leading China one has required tremendous financial support and has been hampered by the lack of access to leading edge manufacturing equipment). The cost of a new, from ground factory to build the chips (the "fab") now costs $15 to $20B, most of which is in new equipment that is also supply constrained. That is why it still takes 3 to 4 years to get a new factory in operation. The NAND side has a few more players (less concentrated) and is less constrained by equipment, which why it remains more vulnerable. There are also some new wildcards --- customers are now signing five year min/max purchase agreements (even with deposits) and the next generation of HBM (4.5) features a logic layer that can be customized to the architecture of each logic chip.
With all the backlash about where datacenters are built and the energy they consume, water used. Doesn't that make an orbital datacenter more sensible and economical?
The lifecycle of chips/servers/memory is short. Newer, faster, less energy reliant options will be created tomorrow and need constant replacement and upgrading. Is there a company that exists today that can dispose of and recycle/reuse these components?
I think we covered this up above, but the launch costs and replacement cycles are economically challenging to justify today vs terrestrial solutions, and there are numerous technological problems due to the rigors and risks of space that need to be solved. No one is saying it is impossible, but it does appear it will take a while.
For current terrestrial use, there are recycling systems in place, although the economics can be challenging to justify. It adds to the challenges of space, with the lifecycles potentially shortened and disposal another challnege.
My experience was more around transactional systems but this is an workload I expect will be aided by an unconventional technology breakthrough. I saw an article, about a company trying to reduce the AI memory footprint by 30x.
Yes, there are have been companies chasing methods for quite a while, from software models and tool chains to alternative system architecture, CPU and memory solutions. A good part of the issue today is that the default approaches to AI use models and systems that are very memory transaction centric, and the pursuit of higher weights while retaining precision has fueled the curve. The models and usages at their heart at times boil down to 95%+ multiply-accumulate operations, which are essentially memory transactions. So different models are segmenting for different uses (cost/precision/memory footprint) while other techniques (Turboquant, for example) are being explored. There are other more revolutionary approaches as well, but these have struggled to gain traction. These things typically all involve tradeoffs, though, and the industry will begin to sort things out, and new solutions will incrementally emerge.