An Efficient Solution Technique for a Class of Markov Chains Models