Intelligence at the Edge of Chaos

🔀 Intelligence at the Edge of Chaos

This research investigates how intelligent behavior emerges in artificial systems by studying the connection between the complexity of rule-based systems and the abilities of models trained to predict these rules. The researchers used elementary cellular automata (ECA), simple one-dimensional systems with varying complexity, to train large language models (LLMs). Their results show that models trained on more complex ECAs demonstrate greater intelligence, excelling in reasoning and chess move prediction tasks. A key finding is the importance of training at a "sweet spot" of complexity—known as the "edge of chaos"—where systems are structured yet difficult to predict, fostering intelligent behavior. Additionally, models trained on complex rules develop sophisticated solutions by incorporating information from previous states, which improves their ability to generalize and perform well on various tasks.

📎 Link to paper