>> % This is a diary of a matlab session to illustrate >> % simple Markov Chain calculations >> >> % Entering the one step state transition matrix P >> >> P = [.2 .75 0 .05 0 .3 .3 .3 .1 0 0 .55 .4 .05 0 0 0 0 .5 .5 .6 0 0 0 .4] P = 0.2000 0.7500 0 0.0500 0 0.3000 0.3000 0.3000 0.1000 0 0 0.5500 0.4000 0.0500 0 0 0 0 0.5000 0.5000 0.6000 0 0 0 0.4000 >> >> % Entering the initial state vector pi0 >> pi0 = [1 0 0 0 0] pi0 = 1 0 0 0 0 >> % To find the n step transition matrix Pn simply raise >> % P to nth power - for example to find P5 >> P5 = P^5 P5 = 0.2050 0.3837 0.1978 0.1192 0.0944 0.2108 0.3766 0.1981 0.1187 0.0959 0.2020 0.3835 0.2044 0.1186 0.0916 0.2730 0.3690 0.1215 0.1052 0.1313 0.2152 0.3955 0.1971 0.1133 0.0789 >> % This could be used to find pi5 by >> pi5 = pi0*P5 pi5 = 0.2050 0.3837 0.1978 0.1192 0.0944 >> % Notice that if we find Pinfinity all it's row will be >> % equal and are the steady state pi >> P6 = P5*P P6 = 0.2127 0.3776 0.1942 0.1181 0.0973 0.2126 0.3800 0.1922 0.1174 0.0977 0.2104 0.3790 0.1968 0.1179 0.0959 0.2441 0.3823 0.1593 0.1092 0.1051 0.2091 0.3884 0.1975 0.1168 0.0882 >> P7 = P6*P P7 = 0.2142 0.3797 0.1910 0.1172 0.0980 0.2151 0.3792 0.1909 0.1170 0.0978 0.2133 0.3797 0.1924 0.1172 0.0973 0.2266 0.3854 0.1784 0.1130 0.0967 0.2113 0.3820 0.1955 0.1176 0.0937 >> P13 = P7*P6 P13 = 0.2156 0.3804 0.1902 0.1166 0.0972 0.2156 0.3804 0.1902 0.1166 0.0972 0.2156 0.3804 0.1902 0.1166 0.0972 0.2155 0.3804 0.1903 0.1167 0.0972 0.2156 0.3804 0.1902 0.1166 0.0972 >> % One can see P13 approximately Pinfinity >> % Alternate method for determining state vector for n = 1,2, ... >> % Iteratively solve pin = pi0*P for n = 1,2, ...5 >> >> ans =[pi0]; >> for n = 1:5 pi = pi0*P; ans = [ans pi]; pi0 = pi; end >> ans ans = 1.0000 0 0 0 0 0.2000 0.7500 0 0.0500 0 0.2650 0.3750 0.2250 0.1100 0.0250 0.1805 0.4350 0.2025 0.1170 0.0650 0.2056 0.3772 0.2115 0.1212 0.0845 0.2050 0.3837 0.1978 0.1192 0.0944 >> % Notice that the first row in ans is pi0 >> % the second row is pi1, third row is pi2, and so on >> >> >> % Alternate (better ) method to find steady state pi >> % is to solve pi = pi*P - the m file mcss.m does this >> % to use mcss must enter P and N - size of P matrix >> N = 5 N = 5 >> mcss sspi = 0.2156 0.3804 0.1902 0.1167 0.0972 >> % To determine first passage time probabilities fij^(n) >> % use the m-file mcfpt.m - to use specifiy >> % i, - the initial state, j - the final state, n - the >> % number of steps and P - the one step transition matrix >> P = P = 0.2000 0.7500 0 0.0500 0 0.3000 0.3000 0.3000 0.1000 0 0 0.5500 0.4000 0.0500 0 0 0 0 0.5000 0.5000 0.6000 0 0 0 0.4000 >> i = 1; >> j = 5; >> n = 4; >> mcfpt fij = 0 0.0250 0.0550 0.0585 >> % Note the answer fij - the first element iis for n = 1 >> % the second element for n = 2 , third n = 3 and so on >> clear >> % To calculate the Mean First Passage Time from i to j >> % need to specify P, N - the size of P , desired state j >> % Note in matlab the states will be numbered starting from 1!!!! >> % and must specify pi0 the initial state vector >> % As an example - same as in notes >> P = [0 .75 .25 .25 0 .75 .25 .25 .5] P = 0 0.7500 0.2500 0.2500 0 0.7500 0.2500 0.2500 0.5000 >> >> N = 3 N = 3 >> j = 3; >> pi0 = [ 1 0 0 ]; >> % The m-file to compute the mean first passage time is mcmfpt.m >> mcmfpt ETij = 2.1538 >> exit