The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2010 vol.59)
pp: 545-560
Ole-Christoffer Granmo , University of Agder, Grimstad
ABSTRACT
In a multitude of real-world situations, resources must be allocated based on incomplete and noisy information. However, in many cases, incomplete and noisy information render traditional resource allocation techniques ineffective. The decentralized Learning Automata Knapsack Game (LAKG) was recently proposed for solving one such class of problems, namely the class of Stochastic Nonlinear Fractional Knapsack Problems. Empirically, the LAKG was shown to yield a superior performance when compared to methods which are based on traditional parameter estimation schemes. This paper presents a completely new online Learning Automata (LA) system, namely the Hierarchy of Twofold Resource Allocation Automata (H-TRAA). In terms of contributions, we first of all, note that the primitive component of the H-TRAA is a Twofold Resource Allocation Automaton (TRAA) which possesses novelty in the field of LA. Second, the paper contains a formal analysis of the TRAA, including a rigorous proof for its convergence. Third, the paper proves the convergence of the H-TRAA itself. Finally, we demonstrate empirically that the H-TRAA provides orders of magnitude faster convergence compared to the LAKG for simulated data pertaining to two-material unit-value functions. Indeed, in contrast to the LAKG, the H-TRAA scales sublinearly. Consequently, we believe that the H-TRAA opens avenues for handling demanding real-world applications such as the allocation of sampling resources in large-scale web accessibility assessment problems. We are currently working on applying the H-TRAA solution to the web-polling and sample-size detection problems applicable to the world wide web.
INDEX TERMS
Nonlinear knapsack problems, hierarchical learning, learning automata, stochastic optimization, resource allocation.
CITATION
Ole-Christoffer Granmo, "Solving Stochastic Nonlinear Resource Allocation Problems Using a Hierarchy of Twofold Resource Allocation Automata", IEEE Transactions on Computers, vol.59, no. 4, pp. 545-560, April 2010, doi:10.1109/TC.2009.189
REFERENCES
[1] G.K. Bhattacharyya and R.A. Johnson, Statistical Concepts and Methods. John Wiley and Sons, 1977.
[2] P.E. Black, "Fractional Knapsack Problem," Dictionary of Algorithms and Data Structures, 2004.
[3] K.M. Bretthauer and B. Shetty, "The Nonlinear Knapsack Problem—Algorithms and Applications," European J. Operational Research, vol. 138, pp. 459-472, 2002.
[4] B.C. Dean, M.X. Goemans, and J. Vondrak, "Approximating the Stochastic Knapsack Problem: The Benefit of Adaptivity," Proc. 45th Ann. IEEE Symp. Foundations of Computer Science (FOCS '04), pp. 208-217, 2004.
[5] B. Fox, "Discrete Optimization via Marginal Analysis," Management Sciences, vol. 13, no. 3, pp. 211-216, 1966.
[6] M.R. Gibson, J.W. Ohlmann, and M.J. Fry, "An Agent-Based Stochastic Ruler Approach for a Stochastic Knapsack Problem with Sequential Competition," Computers and Operations Research, vol. 37, no. 3, pp. 598-609, 2010.
[7] O.-C. Granmo and B.J. Oommen, "On Allocating Limited Sampling Resources Using a Learning Automata-Based Solution to the Fractional Knapsack Problem," Proc. 2006 Int'l Intelligent Information Processing and Web Mining Conf. (IIS:IIPW '06), pp. 263-272, 2006.
[8] O.-C. Granmo, B.J. Oommen, S.A. Myrer, and M.G. Olsen, "Determining Optimal Polling Frequency Using a Learning Automata-Based Solution to the Fractional Knapsack Problem," Proc. 2006 IEEE Int'l Conf. Cybernetics and Intelligent Systems (CIS) and Robotics, Automation and Mechatronics (RAM), pp. 73-79, 2006.
[9] O.-C. Granmo, B.J. Oommen, S.A. Myrer, and M.G. Olsen, "Learning Automata-based Solutions to the Nonlinear Fractional Knapsack Problem with Applications to Optimal Resource Allocation," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 37, no. 1, pp. 166-175, Feb. 2007.
[10] O.-C. Granmo and B.J. Oommen, "A Hierarchy of Twofold Resource Allocation Automata Supporting Optimal Web Polling," Proc. 21st Int'l Conf. Industrial and Eng. Applications of Artificial Intelligence and Expert Systems (IEA/AIE '08), pp. 347-358, 2008.
[11] O.-C. Granmo and B.J. Oommen, "A Hierarchy of Twofold Resource Allocation Automata Supporting Optimal Sampling," Proc. 22nd Int'l Conf. Industrial and Eng. Applications of Artificial Intelligence and Expert Systems (IEA/AIE '09), pp. 523-534, 2009.
[12] J. Kangasharju, F. Hartanto, M. Reisslein, and K.W. Ross, "Distributing Layered Encoded Video through Caches," IEEE Trans. Computers, vol. 51, no. 66, pp. 622-636, June 2002.
[13] H. Kellerer, U. Pferschy, and D. Pisinger, Knapsack Problems. Springer, 2004.
[14] G.Y. Lin, Y. Lu, and D.D. Yao, "The Stochastic Knapsack Revisited: Structure, Switch-Over Policies, and Dynamic Pricing," Operations Research, vol. 56, pp. 945-957, 2008.
[15] K.S. Narendra and M.A.L. Thathachar, Learning Automata: An Introduction. Prentice Hall, 1989.
[16] B.J. Oommen, "Absorbing and Ergodic Discretized Two Action Learning Automata," IEEE Trans. Systems, Man, and Cybernetics, vol. 16, no. 2, pp. 282-293, Mar./Apr. 1986.
[17] B.J. Oommen, "Stochastic Searching on the Line and Its Applications to Parameter Learning in Nonlinear Optimization," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 27, no. 4, pp. 733-739, Aug. 1997.
[18] B.J. Oommen and L. Rueda, "Stochastic Learning-Based Weak Estimation of Multinomial Random Variables and Its Applications to Pattern Recognition in Non-Stationary Environments," Pattern Recognition, vol. 39, pp. 328-341, 2006.
[19] S. Pandey, K. Ramamritham, and S. Chakrabarti, "Monitoring the Dynamic Web to Respond to Continuous Queries," Proc. 12th Int'l World Wide Web Conf., pp. 659-668, 2003.
[20] T.C. Perry and J.C. Hartman, "An Approximate Dynamic Programming Approach to Solving a Dynamic, Stochastic Multiple Knapsack Problem," Int'l Trans. Operational Research, vol. 16, no. 3, pp. 347-359, 2009.
[21] K.W. Ross and D.H.K. Tsang, "The Stochastic Knapsack Problem," IEEE Trans. Comm., vol. 37, no. 7, pp. 740-747, July 1989.
[22] J. Sachs, "A Stochastic Knapsack Model for the Capacity Evaluation of (Multi-) Radio Access Networks," Int'l J. Systems Science, vol. 32, no. 3, pp. 79-87, 2006.
[23] M. Snaprud, N. Ulltveit-Moe, O.-C. Granmo, M. Rafoshei-Klev, A. Wiklund, and A. Sawicka, "Quantitative Assessment of Public Web Sites Accessibility—Some Early Results," Proc. Accessibility for All Conf., 2003.
[24] E. Steinberg and M.S. Parks, "A Preference Order Dynamic Program for a Knapsack Problem with Stochastic Rewards," J. Operational Research Soc., vol. 30, no. 2, pp. 141-147, 1979.
[25] M.A.L. Thathachar and P.S. Sastry, Networks of Learning Automata: Techniques for Online Stochastic Optimization. Kluwer Academic Publishers, 2004.
[26] M.L. Tsetlin, Automaton Theory and Modeling of Biological Systems. Academic Press, 1973.
[27] J.L. Wolf, M.S. Squillante, J. Sethuraman, and K. Ozsen, "Optimal Crawling Strategies for Web Search Engines," Proc. 11th Int'l World Wide Web Conf., pp. 136-147, 2002.
45 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool