Freeze a Pong game halfway through a rally. You can see where the ball is, but can you tell where it will go next? Its previous position reveals the direction that the current picture leaves out.
That is the question behind the Markov principle: what must the current state retain so the earlier history adds nothing to our prediction of the next state and reward?
In this week’s article, you can train and run paired agents in Pong, a door-and-key world, and an ordered-checkpoint task. Each pair changes only what the agent observes. Try the experiments and see how motion, memory, and reward context each affect learning.
Prefere ler em português? O princípio de Markov está aqui: https://aflora.page/pt-br/learn/o-principio-de-markov/