Bifurcation structure of equilibria of iterated softmax

We present a detailed bifurcation study of iterated renormalization process driven by softmax transformation parametrized by a temperature parameter. For each emerging equilibrium we give exact characterization of stable/unstable manifolds of the linearized dynamics. As the system cools down, new equilibria emerge in a strong structure until finally a complex skeleton of saddle type equilibria surrounding an unstable maximum entropy point, with decision enforcing “one-hot” stable equilibria emerges.