Markov chain

In mathematics, a Markov chain, named after Andrey Markov, is a discrete-time stochastic process with the Markov property. Having the Markov Property means for a given process knowledge of the previous states is irrelevant for predicting the probability of subsequent states. In this way a Markov chain is "memoryless", no given state has any causal connection with a previous state.

At each point in time the system may have changed states from the state the system was in the moment before, or the system may have stayed in the same state. The changes of state are called transitions. If a sequence of states has the Markov property, then every future state is conditionally independent of every prior state.