This Article 
 Bibliographic References 
 Add to: 
Fundamental Limits of Bayesian Inference: Order Parameters and Phase Transitions for Road Tracking
February 2000 (vol. 22 no. 2)
pp. 160-173

Abstract—There is a growing interest in formulating vision problems in terms of Bayesian inference and, in particular, the maximimum a posteriori (MAP) estimator. This approach involves putting prior probability distributions, $P(X)$, on the variables $X$ to be inferred and a conditional distribution $P(Y|X)$ for the measurements $Y$. For example, $X$ could denote the position and configuration of a road in an aerial image and $Y$ can be the aerial image itself (or a filtered version). We observe that these distributions define a probability distribution $P(X,Y)$ on the ensemble of problem instances. In this paper, we consider the special case of detecting roads from aerial images [9] and demonstrate that analysis of this ensemble enables us to determine fundamental bounds on the performance of the MAP estimate (independent of the inference algorithm employed). We demonstrate that performance measures—such as the accuracy of the estimate and whether the road can be detected at all—depend on the probabilities $P(Y|X), P(X)$ only by an order parameter $K$. Intuitively, $K$ summarizes the strength of local cues (as provided by local edge filters) together with prior information (i.e., the probable shapes of roads). We demonstrate that there is a phase transition at a critical value of the order parameter $K$—below this phase transition, it is impossible to detect the road by any algorithm. In related work [25], [5], we derive closely related order parameters which determine the time and memory complexity of search and the accuracy of the solution using the $A^{\ast}$ search strategy. Our approach can be applied to other vision problems and we briefly summarize results when the model uses the “wrong prior” [26]. We comment on how our work relates to studies of the complexity of visual search [21] and to critical behaviour (i.e., phase transitions) in the computational cost of solving NP-complete problems [19].

[1] D.J. Amit, Modeling Brain Function. Cambridge Univ. Press, 1989.
[2] R. Balboa, PhD thesis, Dept. of Computer Science. Univ. of Alicante, Spain, 1997.
[3] P. Cheeseman, B. Kanefsky, and W. Taylor, “Where the Really Hard Problems Are,” Proc. 12th Int'l Joint Conf. Artificial Intelligence, vol. 1, pp 331-337, 1991.
[4] J. Coughlan, D. Snow, C. English, and A.L. Yuille, “Efficient Optimization of a Deformable Template Using Dynamic Programming” Proc. Computer Vision and Pattern Recognition (CVPR '98), 1998.
[5] J.M. Coughlan and A.L. Yuille, “Bayesian A* Tree Search with Expected O(N) Convergence Rates for Road Tracking,” Proc. EMMCVPR '99 York, England, 1999.
[6] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley&Sons, 1991.
[7] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[8] D. Geiger and T.-L. Liu, “Top-Down Recognition and Bottom-Up Integration for Recognizing Articulated Objects,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, M. Pellilo and E. Hancock, eds., Venice, Italy, May 1997.
[9] D. Geman and B. Jedynak, An Active Testing Model for Tracking Roads in Satellite Images IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 1, pp. 1-14, Jan. 1996.
[10] M. Isard and A. Blake, “Contour Tracking by Stochastic Propagation of Conditional Density,” Proc. European Conf. Computer Vision, pp. 343-356, 1996.
[11] D.W. Jacobs, "Robust and Efficient Detection of Salient Convex Groups," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 23-37, Jan. 1996.
[12] M. Kass, A. Witkin, and D. Terzopoulos, ”Snakes: Active Contour Models,” Proc. First Int'l Conf. Computer Vision, pp. 259-268, 1987.
[13] Perception as Bayesian Inference, D.C. Knill and W. Richards, eds., Cambridge Univ. Press, 1996.
[14] S. Konishi, A.L. Yuille, J. Coughlan, and S.C. Zhu, Fundamental Bounds on Edge Detection: An Information Theoretic Evaluation of Different Edge Cues Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 573-579, 1999.
[15] P. Parodi, R. Lancewicki, A. Vijh, and J.K. Tsotsos, “Empirically-Derived Estimates of the Complexity of Labelling Line Drawings of Polyhedral Scenes,” Artificial Intelligence, vol. 105, no. 1-2, pp. 47-75, 1998.
[16] J. Pearl, Heuristics. Addison-Wesley, 1984.
[17] B.D. Ripley, Pattern Recognition and Neural Networks. Cambridge Univ. Press, 1995.
[18] S.J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, Upper Saddle River, N.J., 1994.
[19] B. Selman and S. Kirkpatrick, “Critical Behaviour in the Computational Cost of satisfiability Testing,” Artificial Intelligence, vol. 81, nos. 1-2, pp. 273-295, 1996.
[20] J.K. Tsotsos, “Analyzing Vision at the Complexity Level,” Behavioural and Brain Sciences, vol. 13, no. 3, Sept. 1990.
[21] J.K. Tsotsos, “On the Relative Complexity of Active versus Passive Visual Search,” Int'l J. Computer Vision, vol. 7, no. 2, Jan. 1992.
[22] J.T. Lewis, C.E. Pfister, and W.G. Sullivan, “Entropy, Concentration of Probability, and Conditional Limit Theorems,” Markov Processes Related Fields, vol. 1, pp. 319-396, 1995.
[23] Y.N. Wu and S.C. Zhu, “Equivalence of Ensembles and Fundamental Bounds,” Proc. Int'l Conf. Computer Vision, Corfu, Greece, 1999.
[24] A.L. Yuille and J. Coughlan, “Twenty Questions, Focus of Attention, and A*: A Theoretical Comparison of Optimization Strategies,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, M. Pellilo and E. Hancock, eds., Venice, Italy, May 1997.
[25] A.L. Yuille and J.M. Coughlan, “Convergence Rates of Algorithms for Visual Search: Detecting Visual Contours,” Proc. NIPS '98, 1998.
[26] A.L. Yuille and J.M. Coughlan, “High-Level and Generic Models for Visual Search: When Does High Level Knowledge Help?” Proc. Computer Vision and Pattern Recognition CVPR '99, Fort Collins, Colo., 1999.
[27] S.C. Zhu, Y.N. Wu, and D. Mumford, “Minimax Entropy Principle and Its Application to Texture Modeling,” Neural Computation, vol. 9, no 8, Nov. 1997.
[28] S.C. Zhu and D. Mumford, “Prior Learning and Gibbs Reaction-Diffusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, Nov. 1997.
[29] S.C. Zhu, Y. Wu, and D. Mumford, “Filters Random Fields and Maximum Entropy(Frame)—Towards a Unified Theory for Texture Modeling,” Int'l J. Computer Vision, vol. 27, no. 2, pp. 107-126, 1998.
[30] S.C. Zhu, “Embedding Gestalt Laws in Markov Random Fields,” . Submitted to IEEE CS Workshop Perceptual Organization in Computer Vision.

Index Terms:
Bayesian inference, phase transitions, curve tracking.
Alan L. Yuille, James M. Coughlan, "Fundamental Limits of Bayesian Inference: Order Parameters and Phase Transitions for Road Tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 2, pp. 160-173, Feb. 2000, doi:10.1109/34.825754
Usage of this product signifies your acceptance of the Terms of Use.