New Animation Technique Uses Motion Frequencies to Shake Trees
UK researchers have developed a new way of making high-quality animations of trees from video footage. Animators for films or computer games could use this approach, called animated foliage, to automatically generate trees that move in a natural way. Animated foliage overcomes many difficulties in creating lifelike tree movements, which normally require hand drawing.
The technique was developed by senior lecturer Peter M. Hall and engineering doctorate student Chris Li at the University of Bath and funded by the UK Engineering & Physical Sciences Research Council. It lets an animator draw a circle around an image of a tree in a single video frame. A Matlab program then uses machine-vision techniques to automatically extract a 3D model of the tree from the rest of the footage. The user can then specify the desired tree movement, even making it dance to music.
Tree-modeling research has been significant over the years. Hall said this project advances the field in five ways. First, it collects the data from video, so that the motion is inherent to the trees rather than modeled afterward by specifying a wind field. Second, the program minimizes the amount of human interaction, making the technique suitable for novices. Third, it can generate new trees automatically. Additionally, it uses a unique 3D reconstruction technique that avoids many problems seen in other animation techniques. Finally, it models tree motion as frequencies, shaking the trees rather than blowing on them.
Tracking the Trees
The research was partly funded to develop a tracker program that follows objects in a video. Hall said, "No one in the world knows how to do that for the vast variety of objects, and the way people do track objects remains a mystery."
The animated-foliage program tracks branches by following the big branches first, then the medium-size child branches, and so on down to small branches. This tracker isn't sensitive enough to go all the way to twigs.
One part of the tracker works out how the branches are connected together. If the tracker gets this wrong, then it can't follow the smaller branches, so it tries again until it gets it right. "Once the branching structure is right and the tracker can follow the tree, it's easy to collect data about the swaying motion," said Hall.
Users initiate the process by outlining the tree they want with a mouse cursor.
Modeling the Motion
Once the program has captured data about the swaying motion, it builds a probabilistic generative model for the fork shapes in tree branches. The model uses information about a branch and its two children that make up the fork. For a given sort of tree, such as an oak or pine, the fork triangles must be more or less the same shape, such as broad and fat or tall and thin.
To make a new tree, the program makes new forks and pieces them together like bricks. It can grow new trees by adding forks that start at the trunk and branch upward. Unlike the tracker, the modeler can make a tree all the way to its twigs.
One challenge Hall and Ping faced was stopping the tree from leaning to one side as it grows. They solved this program by reusing the original, user-created tree outline and fitting the growth pattern to fill that space as evenly as possible. Users can create a differently shaped tree by drawing a new outline of the original.
"This way is much better than just moving the branches of the original tree [from video] by some small random amount," noted Hall. "If we did that we'd just get the same tree again and again, but our way gives a unique individual every time."
Shaken Not Blown
The modeler can control tree movements for different weather conditions and seasons. It copies a tree's movement by extracting the dominant few motion frequencies, using a Fourier transform of the angular velocity. It looks at the shape of branch forks to compute the average shape, along with how much and in what ways it can change.
The researchers noted that the trees sway with resonant frequencies. As a parent branch sways, it pushes its children in just the right way, much like a human pushes a child's swing in time to work properly.
To simulate different weather conditions, the model adjusts the power in the push. To get the tree to dance, the modeler pushes the tree in time to the music frequencies. Hall said, "This is very different from state-of-art trees, which force users to specify a wind. It would be, at best, very hard to make trees dance that way."
Animated-foliage trees are initially built bare of leaves. The tracker sees through the leafy canopy to make a guess about the branches below. The program can add leaves as polygons that approximate the original tree’s texture. Users can also add other shapes, such as blossoms and fruit, and render trees as if they were painted or drawn.
Hall said the technique has some weaknesses. The video must show a leafy tree. To change this, they would need to develop a new tracker.
All the tree forms have one parent and two children. This works well for most trees, but splitting branches into three or more children on occasion would increase the model's accuracy for a larger range of trees. However, it would also require a more complex statistical model.
Another problem is that the branch shapes are guesses and might not be fully accurate. Animators, gamers, and architects are usually satisfied with the results, but tree scientists would be disappointed.
The branches themselves do not bend. Instead, they rotate around the forking point they come from. To bend a branch more, users add more points along the branch’s length.
Hall would like to apply the same user-interface style to create models of other object types, such as clouds and fire. How to capture such object motions remains an open question. Researchers have only recently begun using dynamic textures to model natural phenomena.
There are no definitive plans for the animated-foliage research, except to use it as a platform for further study. A finished application would require a better user interface and translation into a standard language such as C++, Hall said. "Turning research code into a product takes money and time we don't have, but we'd be delighted to work with a partner to create a product."
George Lawton is a freelance journalist based in Guerneville, CA. Contact him at glawton.com.