Visual Effects with Matt Hausman

In the fifth of a series of production diaries, effects animation supervisor Matt Hausman discusses the art of water and wave effects in Sony Pictures Animation's 3D-animated feature, Surf's Up.

by Matt Hausman

At the onset of development the Surf's Up effects team was given two main directives in regards to the water and wave effects: make it look "80% real" and make the entire process capable of producing 20 minutes of final surfing footage as efficiently as possible. For anyone who's worked on a large-scale project such as this it will come as no surprise that the latter of those goals was the one that took the most work. A couple of months were required to hammer out the basic approach and produce a test that, while crude in comparison to the end product, more than adequately proved the overall technical strategies. It then took well over a year to iron out the specifics and craft a production pipeline that spanned five departments: character rigging, layout, animation, effects and lighting.

In a collaborative effort between the effects and character rigging departments, a "wave rig" was developed in Maya that provided the motion of the surfing waves. The rig deformed a rectangular patch such that any lateral cross section of the patch could be animated through the evolution of a two- dimensional wave shape from flat water to tubing, through collapse and back to flat again. By offsetting in time the animation of neighboring cross sections of the wave patch the motion of a tubing wave could be achieved. The waves were animated in this way entirely by hand without procedural mechanisms and with a great deal of time spent getting the motion to appear to generally obey a narrow subset of real-world characteristics such as forward velocity, lateral break speed, a lip that fell at a speed close to gravity, plausible volume preservation and corresponding surface stretching. For composition and timing purposes the ability to pre-visualize dominant wave and surfing features, such the white water explosion of the crashing wave and the wake from a surf board, was incorporated into the rig as well.

This is a still from an early pre-production test rendering of a pipeline wave. Despite the fact that the surface foam, whitewater and the shape of the wave are not the final quality elements, the wave was beginning to take shape. All artwork © 2007 Sony Pictures Animation.

While the function of the wave rig was to provide the gross animation of the surfing waves it was not designed to provide the small, higher-frequency waves of the water surface. Inspired in part by Tessendorf 's work, a Houdini and RenderMan based system for simulating open ocean waves was developed for the overall displaced surface of the water. The system employed "wave trains," simply defined as the sum of continuous wave patterns of varying period, amplitude, direction and speed. By creating sets of Gerstner-style wave trains whose speeds, by default, were physically based but whose frequency ranges and angles of propagation were hand-tailored, several water surface "styles" were settled upon. These ranged from almost dead calm to stormy and chaotic. The frequencies of the wave trains were segregated into three ranges: low, medium and high, each with individual control over amplitude, cuspiness and speed. Provision was made for general noise-based and specific hand tailored control of areas of amplitude reduction of the wave trains for a more varied and natural look of the ocean surface. The peaks of waves could be determined and isolated in the shader to create areas of aerated water or to be used as the source of emission for particle effects. Data, output from the simulation system, describing the frequency ranges and propagation angles and speeds of the wave trains was input into the water displacement shader for rendering.

Ambient foam, foam created from crashing waves, splashes, surfboard wakes and shore break all were critical all components of the look of the water in Surf's Up. From the start it was important to create methods for general and specific foam placement, erasure, dissolution and animation. Used not only to create a more realistic look, different foam patterns and formations were employed to distinguish wave styles and locations from each other. Three distinct patterns of foam were designed from live action reference and consultation with the visual development department: a patchy foam used for choppier water and splashes from rocks and characters; a more elegant graphic style referred to as "web foam" taken from specific photographic examples and used with calmer water at the North Beach location and a convected bubbly foam used with the beach break system of small waves lapping at the shoreline. A distinction was made between "standing" foam, foam that was generated with procedural noise functions in the shader, and "interactive" foam — foam that was specifically placed or the result of a specific event like a splash or wake. Interactive foam used the same noise functions as standing foam but was placed on the water using point clouds sampled in the reference space of the water with attributes describing search radius and density. Once enough points, collected in the reference space, crossed a density threshold, foam would appear in the additive space of the points' search radii on the corresponding part of the wave surface. Additionally, and similar to the methods of amplitude reduction of the wave trains, areas of foam could be erased or reduced procedurally with noise fields or specifically with artist-designed maps projected onto the ocean surface.

An early lighting test showing the different shadow casting techniques used to get detail out of the particle renders.

All the spray effects in Surf 's Up -- the whitewater, lip spray, surfboard sprays, character splashes, rock splashes, etc. -- were rendered, entirely or in part, as dense clouds of RiPoints calculated at render time in RenderMan. To accomplish this, a proximity-based particle instancing scheme called Cluster was developed as a RenderMan DSO. The instancing algorithm produced new points along and around the vectors between pairs of seed points from sparse particle simulations with many parameters controlling point size, distribution, density, opacity fall off and attribute blending. Because the final particle counts required for whitewater or lip spray in a given shot would most often exceed the memory limitations of the machines on the render farm, methods for rendering subsections of the elements had to be developed. A scheme for slicing Cluster renders into layers based on distance from the camera plane and managing the compositing of those layers was implemented. Throughout the show the Cluster DSO was optimized in an attempt to render as many points as possible and ultimately was capable of rendering without slicing about 45 million motion blurred points. For a big Mavericks wave shot in Surf 's Up, the combined point count of the whitewater, lip spray and foam ball (the white water explosion inside the tube) could easily reach 500 million points. The clustered effects were lit using deep shadows which were rendered from each light: typically a key, rim and fill. The final beauty render of the element was a "utility" pass with equal contributions of the key, rim and fill lights segregated into the RGB channels of the image to be balanced and color-corrected into the shot during the compositing stage. Extra passes for specular glints, particle life and density variation were also provided to increase the detail of the element. To save time, especially during sliced renders, matting of other objects was handled by rendering deep shadows of the occluding geometry from the shot camera and sourcing them into the whitewater and spray shader for opacity variance.

An illustration showing three different "wave-train" levels. These "wave-trains," or series of waves, were used in every water shot at various levels to add the textural detail to the surface of the water.

In retrospect it seems odd that a computer generated movie with so much water in it would have been made without the use of a fluid solver at any point. But that fact underscores the overall methodology used by the effects and animation teams on Surf 's Up which was initiated in response to the following quandary: how to efficiently create a lot of realistic looking surfing waves in a production pipeline where the primary animation of the waves occurs during layout and key features of the wave need to be pre-visualized and altered during Animation. As more sophisticated ocean simulation techniques become available, CPUs become faster and memory more expansive, future answers to this question may not rely on any of the strategies outlined above. However, given the demands of an animated feature, where keeping creative and technical options open for as long as possible throughout the pipeline is strongly desired, the approach of layering linked yet discreet solutions to the primary wave features proved highly successful.

Matthew Hausman is a CG Supervisor at Sony Pictures Imageworks, where he recently completed work on Sony Pictures Animation's Surf's Up as effects animation supervisor. He previously served as effects animation lead on The Polar Express, for which he was recognized with an Annie Awards nomination for outstanding individual achievement. Hausman's other credits include Cast Away, Robert Zemeckis' What Lies Beneath and Stuart Little. Before joining Imageworks, he worked with Blue Sky/VIFX on such projects as Star Trek: Insurrection.