Mott the Hoople
Sweet Jane
it is but to discuss the point of view of biology. What is described as ASI sounds like a cascade system. Moore's law would probably stop applying at that time. Biochemical cascade systems like compliment and blood clotting occur rapidly at multiple states and cycles and expand at exponential rates. They also have this in common, without some form of closed loop negative feedback control mechanism the cascade system crashes on itself in complete collapse because it can not sustain the exponential rate of the reaction. In biology the consequence of such a collapse of a biochemical cascade system is the death of the biological system.
The same is probably true with mechanical cascade system like ASI appears to be. If the system of exponential growth described is controlled by an open loop feedback system, which are usually required for exponential growth, then it is an inherently unstable system and will likely crash. What if human level AI is only possible under a closed loop negative feed back control mechanism? Would this behave as a limiting agent preventing ASI?
In other words is human level AI in machines even possible without these machines having a mechanical equivalent of homeostasis?
In addition the authors understanding of evolution as it relates to extinction is flawed. Though it is true that 99.5% of species who have ever existed on this planet are now extinct the analogy used is incorrect. Most did not just fall off the life bar into extinction. In fact very few actually have. Most species have, over great extent of time, have adopted a significant adaptation here and another there and accumulate enough adaptations over time to become something distinctly different. In other words they didn't fall off the bar of life into extinction, they just changed walking along it.
So that brings up the question about Human level AI or ASI that the article didn't address, what if both humans and mechanical AI cause environmental changes that each must adapt to and instead causing human life to go off either side of the bar of life what if we just keep walking along another biological system that has adapted to environmental change?
In other words what if for human level AI or ASI to exist and grow it would still require a symbiotic relationship with humans to survive? What if humans adapt to be ASI's homeostatic control mechanism?
