Applications of some formulas for finite Markov chains

We present some explicit matrix formulas for a finite state Markov chain. The first gives sums of probabilities along some general subsets of paths. Another formula yields the probability mass function (pmf) of the random variable which adds costs along subsets of paths. We then discuss how these formulas can be used to efficiently compute expected values of a function of the sum of costs along paths, as well as related applications. We conclude by describing a procedure allowing us to avoid using Monte Carlo simulation in stochastic approaches to solving some general boundary value problems. Instead, we show how to evaluate the relevant expected values exactly for discretizations of the original continuous problem.