We shall consider essentially stochastic dynamical systems with circonspect time and finite state space, or finite Markov chains. This framework already contains the essential difficulties (intuition instance for long term problems), and allows one to give at the same time an insight of algorithms, mathematical techniques and qualitative properties