This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
An Efficient Dynamic Algorithm for Maintaining All-Pairs Shortest Paths in Stochastic Networks
June 2006 (vol. 55 no. 6)
pp. 686-702
This paper presents a new solution to the Dynamic All-Pairs Shortest Path Routing Problem, using a linear reinforcement learning scheme. The particular instance of the problem that we have investigated concerns finding the all-pairs shortest paths in a stochastic graph, where there are continuous probabilistically-based updates in edge-weights. We present the details of the algorithm with an illustrative example. The algorithm can be used to find the all-pairs shortest paths for the "statistical” average graph, and the solution converges irrespective of whether there are new changes in edge-weights or not. On the other hand, the existing algorithms will fail to exhibit such a behavior and would recalculate the affected shortest paths after each edge-weight update. There are two important contributions of the proposed algorithm. The first contribution is that not all the edges in a stochastic graph are probed and, even if they are, they are not all probed equally often. Indeed, the algorithm attempts to almost always probe only those edges that will be included in the final list involving all pairs of nodes in the graph, while probing the other edges minimally. This increases the performance of the proposed algorithm. The second contribution is designing a data-structure, the elements of which represent the probability that a particular edge in the graph lies in the shortest path between a pair of nodes in the graph. All the algorithms were tested in environments where edge-weights change stochastically and where the graph topologies undergo multiple simultaneous edge-weight updates. Its superiority in terms of the average number of processed nodes, scanned edges, and the time per update operation, when compared with the existing algorithms, was experimentally established.

[1] G. Ausiello, G. Italiano, A. Marchetti-Spaccamela, and U. Nanni, “Incremental Algorithms for Minimal Length Paths,” J. Algorithms, vol. 12, no. 4, pp. 615-638, 1991.
[2] A.F. Atlasis, N.H. Loukas, and A.V. Vasilakos, “The Use of Learning Algorithms in ATM Networks Call Admission Control Problem: A Methodology,” Computer Networks, vol. 34, no. 3, pp. 341-353, 2000.
[3] A.F. Atlasis and A.V. Vasilakos, “The Use of Reinforcement Learning Algorithms in Traffic Control of High Speed Networks,” Advances in Computer Intelligence and Learning, pp. 353-369, Kluwer, 2002.
[4] R. Bellman, “On a Routing Problem,” Quarterly Applications Math., vol. 16, pp. 87-90, 1958.
[5] E.W. Dijkstra, “A Note on Two Problems in Connection with Graphs,” Numerische Mathematik, vol. 1, pp. 269-271, 1959.
[6] C. Demetrescu and G. Italiano, “Fully Dynamic All-Pairs Shortest Paths with Real Weights,” Proc. 42nd IEEE Ann. Symp. Foundations of Computer Science, pp. 260-267, 2001.
[7] C. Demetrescu and G.F. Italiano, “A New Approach to Dynamic All Pairs Shortest Paths,” Proc. 35th Ann. ACM Symp. Theory of Computing, pp. 159-166, 2003.
[8] S. Even and H. Gazit, “Updating Distances in Dynamic Graphs,” Methods of Operations Research, vol. 49, pp. 371-387, 1985.
[9] P.G. Franciosa, D. Frigioni, and R. Giaccio, “Semi-Dynamic Shortest Paths and Breadth First Search in Digraphs,” Proc. Symp. Theoretical Aspects of Computer Science (STACS), pp. 33-46, 1997.
[10] R.W. Floyd, “Algorithm 97 (SHORTEST PATH),” Comm. ACM, vol. 5, no. 6, p. 345, 1962.
[11] J. Fakcharoemphol and S. Rao, “Planar Graphs, Negative Weight Edges, Shortest Paths, and Near Linear Time,” Proc. 42nd IEEE Symp. Foundations of Computer Science, pp. 232-241, 2001.
[12] M. Henzinger, P. Klein, S. Rao, and S. Subramanian, “Faster Shortest-Path Algorithms for Planar Graphs,” J. Computer and System Sciences, vol. 55, no. 1, pp. 3-23, 1997.
[13] V. King, “Fully Dynamic Algorithms for Maintaining All-Pairs Shortest Paths and Transitive Closure in Digraphs,” Proc. 40th IEEE Symp. Foundations of Computer Science, pp. 81-99, 1999.
[14] J. Kabudian, M.R. Meybodi, and M.M. Homayounpour, “Applying Continuous Action Reinforcement Learning Automata (CARLA) on Global raining of Hidden Markov Models,” Proc. Int'l Conf. Information Technology: Coding and Computing (ITCC '04), vol. 2, Apr. 2004.
[15] S. Lakshmivarahan, Learning Algorithms Theory and Applications. New York: Springer, 1981.
[16] P. Loubal, “A Network Evaluation Procedure,” Highway Research Record, vol. 205, pp. 96-109, 1967.
[17] M.R. Meybodi and H. Beigy, “New Learning Automata Based Algorithms for Adaptation of Backpropagation Algorithm Parameters,” Int'l J. Neural Systems, vol. 12, no. 1, pp. 45-67, 2002.
[18] J. Moy, “OSPF Version 2,” Internet Draft, RFC 2178, 1997.
[19] S. Misra and B.J. Oommen, “Stochastic Learning Automata-Based Dynamic Algorithms for the Single-Source Shortest Path Problem,” Proc. Int'l Conf. Industrial and Eng. Applications of Artificial Intelligence and Expert Systems (IEA/AIE Conf.), pp. 239-248, 2003.
[20] J. McQuillan, I. Richer, and E. Rosen, “The New Routing Algorithm for the ARPANET,” IEEE Trans. Comm., vol. 28, no. 5, pp. 711-719, 1980.
[21] J. Murchland, “The Effect of Increasing or Decreasing the Length of a Single Arc on All Shortest Distances in a Graph,” Technical Report LBS-TNT-26, London, 1967.
[22] K. Najim and A.S. Poznyak, Learning Automata: Theory and Applications. Oxford, U.K.: Pergamon Press, 1994.
[23] P. Narvaez, K.-Y. Siu, and H.Y. Tzeng, “New Dynamic Algorithms for Shortest Path Tree Computation,” IEEE/ACM Trans. Networking, vol. 8, no. 6, pp. 734-746, 2000.
[24] K.S. Narendra and M.A.L. Thathachar, Learning Automata. Prentice-Hall, 1989.
[25] B.J. Oommen and E.V. De St. Croix, “Graph Partitioning Using Learning Automata,” IEEE Trans. Computers, vol. 45, no. 2, pp. 195-208, Feb. 1995.
[26] B.J. Oommen and T.D. Roberts, “Continuous Learning Automata Solutions to the Capacity Assignment Problem,” IEEE Trans. Computers, vol. 49, pp. 608-620, 2000.
[27] A.S. Poznyak and K. Najim, Learning Automata and Stochastic Optimization. Berlin: Springer-Verlag, 1997.
[28] G.I. Papadimitriou and A.S. Pomportsis, “Learning-Automata-Based TDMA Protocols for Broadcast Communication Systems with Bursty Traffic,” IEEE Comm. Letters, vol. 4, no. 3, pp. 107-109, 2000.
[29] V. Rodinov, “The Parametric Problem of Shortest Distances,” USSR Computational Math. and Math. Physics, vol. 8, no. 5, pp. 336-343, 1968.
[30] H. Rohnert, “A Dynamization of the All-Pairs Least Cost Problem,” Proc. Second Ann. Symp. Theoretical Aspects of Computer Science (STACS '85), pp. 279-286, 1985.
[31] G. Ramalingam and T. Reps, “On the Computational Complexity of Dynamic Graph Problems,” Theoretical Computer Science, vol. 158, no. 1, pp. 233-277, 1996.
[32] P. Spira and A. Pan, “On Finding and Updating Spanning Trees and Shortest Paths,” SIAM J. Computing, vol. 4, no. 3, pp. 375-380, 1975.
[33] M.S. Obaidat, G.I. Papadimitriou, and A.S. Pomportsis, “Learning Automata: Theory, Paradigms and Applications,” IEEE Trans. Systems, Man, and Cybernetics, vol. 32, no. 6, pp. 706-709, 2002.
[34] M.S. Obaidat, G.I. Papadimitriou, A.I. Pomportsis, and H.S. Laskaridis, “Learning Automata-Based Bus Arbitration for Shared-Medium AM Switches,” IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 32, no. 6, pp. 706-709, 2002.
[35] M.S. Obaidat, G.I. Papadimitriou, and A.I. Pomportsis, “Efficient Fast Learning Automata,” Information Science, vol. 157, pp. 121-133, 2003.
[36] B.J. Oommen and J.K. Lanctot, “Discretized Pursuit Learning Automata,” IEEE Trans. Systems, Man, and Cybernetics, vol. 20, no. 4, pp. 931-938, 1990.
[37] B.J. Oommen and M. Agache, “A Comparison of Continuous and Discretized Pursuit Learning Schemes,” Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 1061-1067, 1999.
[38] L. Peterson and B. Davie, Computer Networks: A Systems Approach. Morgan Kaufmann, 2000.
[39] A.S. Poznyak and K. Najim, Learning Automata and Stochastic Optimization. Berlin: Springer-Verlag, 1997.
[40] G.I. Papadimitriou and A.S. Pomportsis, “Learning Automata-Based DMA Protocols for Broadcast Communication Systems with Bursty Traffic,” IEEE Comm. Letters, vol. 4, no. 3, pp. 107-109, 2000.
[41] F. Seredynski, “Distributed Scheduling Using Simple Learning Machines,” European J. Operational Research, vol. 107, pp. 401-413, 1998.
[42] M. Schwartz and T. Stern, “Routing Techniques Used in Computer Communications Networks,” IEEE Trans. Comm., vol. 28, pp. 539-552, 1980.
[43] A. Tanenbaum, Computer Networks, fourth ed. Prentice Hall, 2003.
[44] M.A.L. Thathachar and P.S. Sastry, “A New Approach to Designing Reinforcement Schemes for Learning Automata,” Proc. IEEE Int'l Conf. Cybernetics and Soc., Jan. 1984.
[45] M.A.L. Thathachar and P.S. Sastry, “A Class of Rapidly Converging Algorithms for Learning Automata,” IEEE Trans. Systems, Man, and Cybernetics, vol. 15, pp. 168-175, 1985.
[46] M.A.L. Thathachar and P.S. Sastry, “Estimator Algorithms for Learning Automata,” Proc. Platinum Jubilee Conf. Systems and Signal Processing, Dec. 1986.
[47] M.A.L. Thathachar and P.S. Sastry, Networks of Learning Automata. Kluwer Academic, 2003.
[48] A.V. Vasilakos, M.P. Salouros, A.F. Alasssis, and W. Pedrycz, “Optimizing QoS Routing in Hierarchical ATM Networks Using Computational Intelligence Techniques,” IEEE Trans. Systems, Man, and Cybernetics, Part C, vol. 33, no. 3, pp. 297-312, 2003.

Index Terms:
Routing, all pairs shortest path, dynamic, learning automata, stochastic graphs.
Citation:
Sudip Misra, B. John Oommen, "An Efficient Dynamic Algorithm for Maintaining All-Pairs Shortest Paths in Stochastic Networks," IEEE Transactions on Computers, vol. 55, no. 6, pp. 686-702, June 2006, doi:10.1109/TC.2006.83
Usage of this product signifies your acceptance of the Terms of Use.