Learning Physics-Based Motion Style with Nonlinear Inverse Optimization

C. Karen Liu, Aaron Hertzmann, Zoran Popovic

ACM Transactions on Graphics (SIGGRAPH)

Volume 24, Number 3, Pages 1071-1081, 2005

Motivation

Creating expressive and realistic character motion remains one of the main challenges in computer animation.

Goal of This Research

The goal is to create realistic character motion by extracting the essential style of locomotion arising from physics-based models.

Goal of This Paper

This paper presents a new physics-based approach to creating realistic character motion. A dynamic model is presented which includes an abstract representation of an actor's muscles and tendons, as well as parameters that encode an actor's relative preference for applying torques at some joints more than others. New motions are created by minimizing the total muscle torques according to these preferences using spacetime optimization.

A set of tasks is represented by environment and goal constraints C . Any new motion X is computed by minimizing an energy objective function E(X;H) which equals the total torque due to muscle forces. The vector H is the set of physical style parameters. The basic method of this paper consists of two stages:

1. Parameter estimation: Given a motion capture sequence X_T and constraints C , we can estimate the parameter vector H that gave rise to the motion. This is done using nonlinear inverse optimization to find a H for which X_T is the minimizer of E(X;H) .
2. Generation of new motion: By keeping the same motion style H but changing the constraints C , we can minimize the same energy function to generate new motions X that satisfy the new set of constraints while preserving the same motion style.

Related Work

It is controversial whether optimization is even a good model for human motion [2].

Hypotheses about locomotion from biomechanics used in this paper's model are:

1. There is a distinct preference for using certain joints rather than others due to variation in joint strength, stability, and other factors [13].
2. Biological systems use passive elements in their musculoskeletal structure to store and release energy, thereby reducing total power consumption [1].
3. Animals vary stiffness of their joints when performing different tasks, e.g. leg stiffness is much higher during running than during walking [11] [12].

Robot controller simulation has been successfully applied to realistic computer animation [9] [24] [25] [41] [28] [45] [47] [50] [49], although creating controllers for specific tasks and for specific motion styles is difficult. Spacetime constraints cast motion synthesis as a problem of minimizing some measure of energy such as muscle exertion [33] [43] [32] [36] [39] [52] or joint angle acceleration [10]. For complex characters, Newtonian physics constraints are highly nonlinear, preventing spacetime optimization from converging to a good solution. This framework works well for high-energy motion but not for low-energy motions like walking and running that have many possible solutions satisfying the constraints. Parameterizing motion from a collection of example motions yields more realistic motions [44].

An active area of research is the learning of simple models of motion style from example motions [3] [4] [7] [18] [26] [27] [30] [31] [40]. Motion filtering, warping, and retargetting [16] [42] [46] [48] [51] [53] can modify existing motions to create new motions, but are restricted to small modifications since the real physics is not modeled.

The importance of muscle and spring tension in motion is applied to keyframe animation [35], but in a way that requires all parameters to be determined by an animator.

Previous inverse optimization algorithms search for energy functions in which the measured data is optimal [21], but so far existing methods apply only when the optimization problem has a restricted structure, such as linear programming and network flow problems. Approximate inverse optimization is an open problem.

Maximum likelihood and Bayesian learning methods can learn energy functions defined in terms of probabilities, but lead to an intractable integral. Previous methods used random sampling techniques to optimize this integral [14] [23] [22]. Other energy learning methods exist for classification and regression [29].

Methods that learn dynamical systems from data include NeuroAnimator [19] and parameter estimation from a video sequence [5] [6].

Results

The initial configuration is chosen by an automatic procedure. Arbitrary initial configurations do not always generate realistic motions.

Style parameters estimated from a single motion sequence appeared correct since they were used to generate a motion with the same style that was constrained to have the same footprints as the input motion, and the generated motion appeared visually identical to the original motion. New motions were created using these style parameters by using new footprint constraints. The sharper the turn in the footprint path, the more the character leans its torso into the turn. ``Happy" and ``sad" styles of walking were learned and used to generate uphill and downhill walking motions. A running style was learned and applied to walking to generate a motion resembling power-walking. Styles with greater shoe elasticity and carrying a 3-kg suitcase were also learned and used to generate a variety of motions. The generated motions appear to capture the essential overall features of the real motion of a human.

What This Paper Contributes

This paper contributes a new way to generate visually realistic character motions using a physics-based representation that incorporates several factors of locomotion from biomechanics research. The use of optimization in this framework lends support to the biomechanical theory that human motion is optimal in some sense. This paper also captures an idea of motion ``style" that is based on physical properties of the character model and captured motion.

What This Paper Does Not Contribute

The generated motions are only validated visually: no physical measurements of contact forces are used to compare real human motion to the motions generated by nonlinear inverse optimization. Also, initial conditions for generated motions must be chosen carefully since arbitrary initial configurations can lead to unrealistic motions.

Bibliography

1

R. M. Alexander.
Elastic Mechanisms in Animal Movement.
Cambridge University Press, 1988.

2

R. M. Alexander.
Design by numbers.
Nature, 412:591, 2001.

3

O. Arikan and D. A. Forsyth.
Synthesizing constrained motions from examples.
ACM Transactions on Graphics, 21(3):483-490, 2002.

4

O. Arikan, D. A. Forsyth, and J. F. O'Brien.
Motion synthesis from annotations.
ACM Transactions on Graphics, 22(3):402-408, 2003.

5

K. S. Bhat, S. M. Seitz, J. Popovic, and P. K. Khosla.
Computing the physical parameters of rigid-body motion from video.
Lecture Notes in Computer Science, 2350:551-566, 2002.

6

K. S. Bhat, C. D. Twigg, J. K. Hodgins, P. K. Khosla, Z. Popovic, and S. M. Seitz.
Estimating cloth simulation parameters from video.
In Eurographics/SIGGRAPH Symposium on Computer Animation, pages 37-51. ACM Press, 2003.

7

M. Brand and A. Hertzmann.
Style machines.
Proceedings of SIGGRAPH 2000, pages 183-192, 2000.

8

P. de Leva.
Adjustments to zatsiorsky-seluyanov's segment inertia parameters.
Journal of Biomechanics, 29(9):1223-1230, 1996.

9

P. Faloutsos, M. van de Panne, and D. Terzopoulos.
Composable controllers for physics-based character animation.
In Proceedings of SIGGRAPH 2001, pages 251-260, 2001.

10

A. C. Fang and N. S. Pollard.
Efficient synthesis of physically valid human motion.
ACM Transactions on Graphics, 22(3):417-426, 2003.

11

C. T. Farley and D. C. Morgenroth.
Leg stiffness primarily depends on ankle stiffness during human hopping.
Journal of Biomechanics, 32:267-273, 1999.

12

D. P. Ferris, K. Liang, and C. T. Farley.
Runners adjust leg stiffness for their first step on a new running surface.
Journal of Biomechanics, 32:787-794, 1999.

13

R. J. Full, T. Kubow, J. Schmitt, P. Holmes, and D. Koditschek.
Quantifying dynamic stability and maneuverability in legged locomotion.
Integrative and Comparative Biology, 42:129-157, 2002.

14

C. J. Geyer and E. A. Thompson.
Constrained monte carlo maximum likelihood for dependent data.
Journal of the Royal Statistical Society: Series B, 54:657-699, 1992.

15

P. E. Gill, M. A. Saunders, and W. Murray.
Snopt: An sqp algorithm for large-scale constrained optimization.
Technical Report NA 96-2, University of California, San Diego, 1996.

16

M. Gleicher.
Retargetting motion to new characters.
In Proceedings of SIGGRAPH 1998, pages 33-42, 1998.

17

F. S. Grassia.
Practical parameterization of rotations using the exponential map.
Journal of Graphics Tools, 3(3):29-48, 1998.

18

K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popovic.
Style-based inverse kinematics.
ACM Transactions on Graphics, pages 522-531, 2004.

19

R. Grzeszczuk, D. Terzopoulos, and G. Hinton.
Neuroanimator: Fast neural network emulation and control of physics-based models.
In Proceedings of SIGGRAPH 1998, pages 9-20, 1998.

20

J. He, R. Kram, and T. A. McMahon.
Mechanics of running under simulated low gravity.
Journal of Applied Physiology, 71:863-870, 1991.

21

C. Heuberger.
Inverse combinatorial optimization: A survey on problems, methods, and results.
Journal of Combinatorial Optimization, 8:329-361, 2004.

22

G. E. Hinton.
Training products of experts by minimizing contrastive divergence.
Neural Computation, 14(8):1771-1800, 2002.

23

G. E. Hinton and T. J. Sejnowski.
Learning and relearning in boltzmann machines.
In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing, Volume 1: Foundations, pages 282-317, 1986.

24

J. Hodgins, W. Wooten, D. Brogan, and J. O'Brien.
Animating human athletics.
In Proceedings of SIGGRAPH 1995, pages 71-78, 1995.

25

J. K. Hodgins and N. S. Pollard.
Adapting simulated behaviours for new characters.
Computer Graphics (SIGGRAPH), pages 153-162, 1997.

26

L. Kovar and M. Gleicher.
Automated extraction and parameterization of motions in large data sets.
ACM Transactions on Graphics, pages 559-568, 2004.

27

L. Kovar, M. Gleicher, and F. Pighin.
Motion graphs.
ACM Transactions on Graphics, 21(3):473-482, 2002.

28

J. Laszlo, M. van de Panne, and E. L. Fiume.
Interactive control for physically-based animation.
Proceedings of SIGGRAPH 2000, pages 201-208, 2000.

29

Y. LeCun and F. Huang.
Loss functions for discriminative training of energy-based models.
In Proceedings of the International Workshop on Artifical Intelligence and Statistics, 2005.

30

J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard.
Interactive control of avatars animated with human motion data.
ACM Transactions on Graphics, 21(3):491-500, 2002.

31

Y. Li, T. Wang, and H.-Y. Shum.
Motion texture: A two-level statistical model for character motion synthesis.
ACM Transactions on Graphics, 21(3):465-472, 2002.

32

C. K. Liu and Z. Popovic.
Synthesis of complex dynamic character motion from simple animations.
ACM Transactions on Graphics, 21(3):408-416, 2002.

33

Z. Liu, S. J. Gortler, and M. F. Cohen.
Hierarchical spacetime control.
Computer Graphics (SIGGRAPH), pages 35-42, 1994.

34

F. E. Mount, M. Whitmore, and S. L. Stealey.
Evaluation of neutral body posture on shuttle mission sts-57 (spacehab-1).
Technical Report TM-2003-104805, NASA, 2003.

35

M. Neff and E. Fiume.
Modeling tension and relaxation for computer animation.
In ACM SIGGRAPH Symposium on Computer Animation, pages 81-88, 2002.

36

M. G. Pandy.
Computer modeling and simulation of human movement.
Annual Review of Biomedical Engineering, 3:245-273, 2001.

37

D. J. Pearsall, J. G. Reid, and R. Ross.
Inertial properties of the human trunk of males determined from magnetic resonance imaging.
Annals of Biomedical Engineering, 22:692-706, 1994.

38

N. S. Pollard and P. S. A. Reitsma.
Animation of humanlike characters: Dynamic motion filtering with a physically plausible contact model.
In Yale Workshop on Adaptive and Learning Systems, 2001.

39

Z. Popovic and A. Witkin.
Physically based motion transformation.
Computer Graphics (SIGGRAPH), pages 11-20, 1999.
Summary available .

40

K. Pullen and C. Bregler.
Motion capture assisted animation: Texturing and synthesis.
ACM Transactions on Graphics, 21(3):501-508, 2002.

41

M. H. Raibert and J. K. Hodgins.
Animation of dynamic legged locomotion.
Computer Graphics (SIGGRAPH), 25:349-358, 1991.

42

C. Rose, M. F. Cohen, and B. Bodenheimer.
Verbs and adverbs: Multidimensional motion interpolation.
IEEE Computer Graphics & Applications, 18(5):32-40, 1998.

43

C. Rose, B. Guenter, B. Bodenheimer, and M. Cohen.
Efficient generation of motion transitions using spacetime constraints.
Computer Graphics, pages 147-154, 1996.

44

A. Safonova, J. K. Hodgins, and N. S. Pollard.
Synthesizing physically realistic human motion in low-dimensional behavior-specific spaces.
ACM Transactions on Graphics, 2004.

45

H. C. Sun and D. N. Metaxas.
Automating gait animation.
Proceedings of ACM SIGGRAPH 2001, pages 261-270, 2001.

46

S. Tak and H.-S. Ko.
A physically-based motion retargeting filter.
ACM Transactions on Graphics, 24(1):98-117, 2005.

47

N. Torkos and M. van de Panne.
Footprint-based quadruped motion synthesis.
In Graphics Interface '98, pages 151-160, 1998.

48

M. Unuma, K. Anjyo, and R. Takeuchi.
Fourier principles for emotion-based human figure animation.
Proceedings of SIGGRAPH 95, pages 91-96, 1995.

49

M. van de Panne and E. Fiume.
Sensor-actuator networks.
In Proceedings of SIGGRAPH 1993, pages 335-342, 1993.

50

M. van de Panne, R. Kim, and E. Fiume.
Virtual wind-up toys for animation.
In Proceedings of Graphics Interface 94, pages 208-215, 1994.

51

M. A. O. Vasilescu.
Human motion signatures: Analysis, synthesis, recognition.
Proceedings of the International Conference on Pattern Recognition '02, 3:456-460, 2002.

52

A. Witkin and M. Kass.
Spacetime constraints.
Computer Graphics (SIGGRAPH), 22:159-168, 1988.

53

A. Witkin and Z. Popovic.
Motion warping.
Computer Graphics (SIGGRAPH), pages 105-108, 1995.