Issue No.06 - June (2011 vol.33)
pp: 1132-1146
Berthold Klaus Paul Horn , Massachusetts Institute of Technology, Cambridge
Purnima Ratilal , Northeastern University, Boston
Nicholas Constantine Makris , Massachusetts Institute of Technology, Cambridge
We present methods for estimating forces which drive motion observed in density image sequences. Using these forces, we also present methods for predicting velocity and density evolution. To do this, we formulate and apply a Minimum Energy Flow (MEF) method which is capable of estimating both incompressible and compressible flows from time-varying density images. Both the MEF and force-estimation techniques are applied to experimentally obtained density images, spanning spatial scales from micrometers to several kilometers. Using density image sequences describing cell splitting, for example, we show that cell division is driven by gradients in apparent pressure within a cell. Using density image sequences of fish shoals, we also quantify 1) intershoal dynamics such as coalescence of fish groups over tens of kilometers, 2) fish mass flow between different parts of a large shoal, and 3) the stresses acting on large fish shoals.
Force estimation, density prediction, compressible flow estimation, minimum energy flow.
Berthold Klaus Paul Horn, Purnima Ratilal, Nicholas Constantine Makris, "Force Estimation and Prediction from Time-Varying Density Images", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 6, pp. 1132-1146, June 2011, doi:10.1109/TPAMI.2010.185
[1] A. Amini, “A Scalar Function Formulation for Optical Flow,” Proc. European Conf. Computer Vision, pp. 125-131, 1994.
[2] P. Anandan, “A Computational Framework and an Algorithm for the Measurement of Visual Motion,” Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[3] M. Andrews, Z. Gong, and P. Ratilal, “High Resolution Population Density Imaging of Random Scatterers with the Matched Filtered Scattered Field Variance,” The J. Acoustical Soc. of Am., vol. 126, no. 3, pp. 1057-1068, 2009.
[4] J. Barron, D. Fleet, and S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, 1994.
[5] G.K. Batchelor, An Introduction to Fluid Dynamics. Cambridge Univ. Press, 1967.
[6] R. Battiti, E. Amaldi, and C. Koch, “Computing Optical Flow across Multiple Scales: An Adaptive Coarse-to-Fine Strategy,” Int'l J. Computer Vision, vol. 6, no. 2, pp. 133-145, June 1991.
[7] D. Béréziat, I. Herlin, and L. Younes, “A Generalized Optical Flow Constraint and Its Physical Interpretation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 487-492, 2000.
[8] D. Béréziat and J.-P. Berroir, “Motion Estimation on Meteorological Infrared Data Using a Total Brightness Invariance Hypothesis,” Environmental Modeling and Software, vol. 15, pp. 513-519, 2000.
[9] T. Corpetti, D. Heitz, G. Arroyo, E. Mémin, and A. Santa-Cruz, “Fluid Experimental Flow Estimation Based on an Optical-Flow Scheme,” Experiments in Fluids, vol. 40, pp. 80-97, 2006.
[10] I.D. Couzin and J. Krause, “Self-Organization and Collective Behavior in Vertebrates,” Advances in the Study of Behavior, vol. 32, pp. 1-75, 2003.
[11] G. Danuser and C.M. Waterman-Storer, “Quantitative Fluorescent Speckle Microscopy: Where It Came from and Where It Is Going,” J. Microscopy, vol. 211, pp. 191-207, Sept. 2003.
[12] W.J. Devenport, M.C. Rife, S.I. Liapis, and G.J. Follin, “The Structure and Development of a Wing-Tip Vortex,” J. Fluid Mechanics, vol. 312, pp. 67-106, 1996.
[13] V. Devlaminck and J.-P. Dubus, “Estimation of Compressible or Incompressible Deformable Motions for Density Images,” Proc. Int'l Conf. Image Processing, vol. 1, pp. 125-128, Sept. 1996.
[14] L.G. Ekatarina, M.I. Maxim, I.A. Fazly, and J.R. McIntosh, “Force Production by Disassembling Microtubules,” Nature, vol. 438, pp. 384-388, 2005.
[15] W. Enkelmann, “Investigation of Multigrid Algorithms for the Estimation of Optical Flow Fields in Image Sequences,” Computer Vision, Graphics, and Image Processing, vol. 43, no. 2, pp. 150-177, Aug. 1988.
[16] D.M. Farmer, M.V. Trevorrow, and Q.B. Pederson, “Intermediate Range Fish Detection with a 12-kHz Sidescan Sonar,” The J. Acoustical Soc. of Am., vol. 106, pp. 2481-2490, 1999.
[17] J. Fitzpatrick, “A Method for Calculating Fluid Flow in Time Dependent Images Based on the Continuity Equation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 78-81, 1985.
[18] D.J. Fleet and A.D. Jepson, “Computation of Component Image Velocity from Local Phase Information,” Int'l J. Computer Vision, vol. 5, pp. 77-104, 1990.
[19] S. Freeman, “Cell Division,” Biological Science, Prentice Hall, 2002.
[20] Z. Gong, M. Andrews, S. Jagannathan, R. Patel, J.M. Jech, N.C. Makris, and P. Ratilal, “Low-Frequency Target Strength and Abundance of Shoaling Atlantic Herring (Clupea Harengus) in the Gulf of Maine during the Ocean Acoustic Waveguide Remote Sensing 2006 Experiment,” The J. Acoustical Soc. of Am., vol. 127, pp. 104-123, 2010.
[21] T. Guerrero et al., “Dynamic Ventilation Imaging from Four-Dimensional Computed Tomography,” Physics in Medicine and Biology, vol. 51, no. 4, pp. 777-791, 2006.
[22] D.J. Heeger, “Optical Flow Using Spatiotemporal Filters,” Int'l J. Computer Vision, vol. 1, pp. 279-302, 1988.
[23] B. Horn and B. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[24] I. Huse and E. Ona, “Tilt Angle Distribution and Swimming Speed of Overwintering Norwegian Spring Spawning Herring,” Int'l Council for the Exploration of the Sea J. Marine Science, vol. 53, pp. 863-873, 1996.
[25] S. Inoue and H. Sato, “Cell Motility by Labile Association of Molecules: The Nature of Mitotic Spindle Fibers and Their Role in Chromosome Movement,” The J. General Physiology, vol. 50, pp. 259-292, 1967.
[26] S. Inoue and E.D. Salmon, “Force Generation by Microtubule Assembly/Disassembly in Mitosis and Related Movements,” Molecular Biology of the Cell, vol. 6, no. 12, pp. 1619-1640, 1995.
[27] S. Jagannathan, I. Bertsatos, D. Symonds, H.T. Nia, A. Jain, M. Andrews, Z. Gong, R. Nero, L. Ngor, M. Jech, O.R. Godø, S. Lee, P. Ratilal, and N. Makris, “Ocean Acoustic Waveguide Remote Sensing (OAWRS) of Marine Ecosystems,” Marine Ecology Progress Series, vol. 395, 2009.
[28] K.E. Kasza, A.C. Rowat, J. Liu, T.E. Angelini, C.P. Brangwynne, G.H. Koenderink, and D.E. Weitz, “The Cell as a Material,” Current Opinion in Cell Biology, vol. 19, pp. 101-107, 2007.
[29] L.D. Landau and E.M. Lifshitz, Mechanics. Pergammon Press, 1976.
[30] H. Lodish, A. Berk, P. Matsudaira, C.A. Kaiser, M. Krieger, M.P. Scott, S.L. Zipursky, and J. Darnell, Molecular Cell Biology. Freeman and Company, 2004.
[31] B. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. Defense Advanced Research Projects Agency Image Understanding Workshop, pp. 121-130, 1981.
[32] N.C. Makris, P. Ratilal, D. Symonds, S. Jagannathan, S. Lee, and R. Nero, “Fish Population and Behavior Revealed by Instantaneous Continental Shelf Scale Imaging,” Science, vol. 311, pp. 660-663, 2006.
[33] N.C. Makris, P. Ratilal, S. Jagannathan, Z. Gong, M. Andrews, I. Bertsatos, O.R. Godø, R. Nero, and M. Jech, “Critical Population Density Triggers Rapid Formation of Vast Oceanic Fish Shoals,” Science, vol. 323, no. 5922, pp. 1734-1737, 2009.
[34] P.L.M. de Maupertuis, “Accord de Diffrentes Lois de la Nature Qui Avaient Jusqu'ici paru Incompatibles” Mm. As. Sc. Paris, p. 417, 1744.
[35] D. Metaxas and D. Terzopoulos, “Shape and Nonrigid Motion and Structure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 580-591, June 1993.
[36] O.A. Misund, A. Fernö, T. Pitcher, and B. Totland, “Tracking Herring Schools with a High Resolution Sonar. Variations in Horizontal Area and Relative Echo Intensity,” Int'l Council for the Exploration of the Sea J. Marine Science, vol. 55, pp. 58-66, 1998.
[37] H.-H. Nagel, “On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results,” Artifical Intelligence, vol. 33, pp. 299-324, 1987.
[38] Y. Nakajima, H. Inomatad, H. Nogawab, Y. Satoa, S. Tamuraa, K. Okazakic, and S. Torii, “Physics-Based Flow Estimation of Fluids,” Pattern Recognition, vol. 36, no. 5, pp. 1203-1212, 2003.
[39] R. Ortega, G. Devés, and A. Carmona, “Bio-Metals Imaging and Speciation in Cells Using Proton and Synchrotron Radiation X-Ray Microspectroscopy,” J. Royal Soc. Interface, vol. 6, pp. S649-S658, 2009.
[40] T. Potapova and G. Gorbsky, “Classic Mitosis in a Vertebrate Cell,” ASCB Image and Video Library, VID-31, http://cellimages.,233 , Aug. 2007.
[41] A. Pentland and B. Horowitz, “Recovery of Nonrigid Motion and Structure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 730-742, July 1991.
[42] T.J. Pitcher and J. Parrish, The Behavior of Teleost Fishes, T. J. Pitcher, ed., pp. 363-439. Chapman and Hall, 1993.
[43] J.L. Prince, “Motion Estimation from Tagged MR Image Sequences,” IEEE Trans. Medical Imaging, vol. 11, no. 2, pp. 238-249, June 1992.
[44] K. Rhode et al., “Validation of an Optical Flow Algorithm to Measure Blood Flow Waveforms in Arteries Using Dynamic Digital X-Ray Images,” Proc. SPIE Conf., p. 1414, 2000.
[45] P.G. Saffman, M.J. Ablowitz, E. Hinch, J.R. Ockendon, and P.J. Olver, Vortex Dynamics. Cambridge Univ. Press, 1992.
[46] J. Simpson and J. Gobat, “Robust Velocity Estimates, Stream Functions and Simulated Lagrangian Drifters from Sequential Spacecraft Data,” IEEE Trans. Geosciences and Remote Sensing, vol. 32, no. 3, pp. 479-493, May 1994.
[47] A. Singh, “An Estimation-Theoretic Framework for Image Flow Computation,” Proc. Third IEEE Int'l Conf. Computer Vision, pp. 168-177, 1990.
[48] S.M. Song and R.M. Leahy, “Computation of 3D Velocity Fields from 3D Cine CT Images of a Human Heart,” IEEE Trans. Medical Imaging, vol. 10, no. 3, pp. 295-306, Sept. 1991.
[49] D. Suter, “Motion Estimation and Vector Splines,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 939-942, June 1994.
[50] D. Terzopoulos and K. Waters, “Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Processes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 569-579, June 1993.
[51] J. Toner and Y. Tu, “Long-Range Order in a Two-Dimensional Dynamical XY Model: How Birds Fly Together,” Physical Rev. Letters, vol. 75, no. 23, pp. 4326-4329, 1995.
[52] J. Toner and Y. Tu, “Flocks, Herds, and Schools: A Quantitative Theory of Flocking,” Physical Rev. E, vol. 58, pp. 4828-4858, 1998.
[53] S. Uras, F. Girosi, A. Verri, and V. Torre, “A Computational Approach to Motion Perception,” Biological Cybernetics, vol. 60, pp. 79-87, 1988.
[54] A. Vlasenko and C. Schnörr, “Variational Approaches to Image Fluid Flow Estimation with Physical Priors” Imaging Measurement Methods for Flow Analysis, pp. 247-256, Springer, 2009.
[55] A.M. Waxman, J. Wu, and F. Bergholm, “Convected Activation Profiles and Receptive Fields for Real Time Measurement of Short Range Visual Motion,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 723-771, 1988.
[56] R.P. Wildes et al. “Recovering Estimates of Fluid Flow from Image Sequence Data,” Computer Vision and Image Understanding, vol. 80, pp. 246-266, 2000.