用于自动摄影的声明式摄影机控制——中文全译

Animations generated by interactive 3D computer graphics applications are typically portrayed either from a particular character's point of view or from a small set of strategically-placed viewpoints. By ignoring camera placement, such applications fail to realize important storytelling capabilities that have been explored by cinematographers for many years.

In this paper, we describe several of the principles of cinematography and show how they can be formalized into a declarative language, called the Declarative Camera Control Language (DCCL). We describe the application of DCCL within the context of a simple interactive video game and argue that DCCL represents cinematic knowledge at the same level of abstraction as expert directors by encoding 16 idioms from a film textbook. These idioms produce compelling animations, as demonstrated on the accompanying videotape.

The language of film is a complex one, which has evolved gradually through the efforts of talented filmmakers since the beginning of the century. As a result the rules of film are now so common that they are nearly always taken for granted by audiences; nonetheless, they are every bit as essential as they are invisible. Most interactive 3D computer graphics applications (e.g., virtual chat managers, interactive fiction environments, and videogames) do not exploit established cinematographic techniques. In particular, most computer animations are portrayed either from a particular character's point of view or from a small set of strategically-placed viewpoints. By restricting camera placement, such applications fail to realize the expository capabilities developed by cinematographers over many decades. Unfortunately, while there are several textbooks that contain informal descriptions of numerous rules for filming various types of scenes (Arijon 1976; Lukas 1985; Mascelli 1965), it is difficult to encode this textbook knowledge in a manner that is precise enough for a computer program to manipulate.

In this paper, we describe several of the principles of filmmaking, show how they can be formalized into a declarative language, and then apply this language to the problem of camera control in an interactive video game. Specifically, we describe the Declarative Camera Control Language (DCCL) and demonstrate that it is sufficient for encoding many of the heuristics found in a film textbook. We also present a Camera Planning System (CPS), which accepts animation traces as input and returns complete camera specifications. The CPS contains a domain-independent compiler that solves DCCL constraints and calculates the dynamical control of the camera, as well as a domain-independent heuristic evaluator that ranks the quality of the candidate shot specifications that the compiler produces. We demonstrate a simple interactive video game that creates simulated animations in response to user input and then feeds these animations to CPS in order to produce complete camera specifications as shown on the accompanying videotape.

Our prototype video game serves as a testbed for applications of DCCL and CPS. However, there are number of alternative applications to which both DCCL and CPS might be applied. Within the realm of video games, Multi-user Dungeons (MUDs), and interactive fiction, automated cinematography would allow an application to convey the subjective impression of a particular character without resorting to point-of-view shots.¹ Because many MUDs operate over long periods of time, an automated cinematography system could provide users with customized summaries of events they had missed while they were away. Alternatively, automated cinematography could be used to create natural interactions with the “intelligent agents” that are likely to take part in the next generation of user interfaces. Automated cinematography could also be used to assist naive users in the creation of desktop videos, or for building animated presentations. In the latter case, Karp and Feiner have shown (Karp & Feiner 1990; 1993) that animated presentations can be effectively designed on a computer, reducing costly human involvement and allowing presentations to be customized for a particular viewer or situation.

Although a film can be considered to be nothing but a linear sequence of frames, it is often helpful to think of a film as having structure. At the highest level, a film is a sequence of scenes, each of which captures some specific situation or action. Each scene in the film is composed of one or more shots. A single shot covers the small portion of a movie between when a camera is turned on and when it is turned off. Typically, a film is comprised of a large number of individual shots, with each shot's duration lasting from a second or two in length to perhaps tens of seconds.²

This section provides an informal description of the Declarative Camera Control Language DCCL. The specification of DCCL is important because it allows CPS to formalize, encode, and implement common film idioms, such as the one presented in Figure 2.

There are four basic primitive concepts in DCCL: fragments, views, placements, and movement endpoints; these primitives are combined to specify higher-level constructs such as shots and idioms.

The Camera Planning System (CPS) codifies and implements the previously described cinematographic principles of camera placement for the case of simple movement sequences on the part of one or two actors.

As input, CPS requires an animation trace: information about the positions and activities of each character, as well as directives stating which actors are to be filmed over what intervals of time. In our interactive game, this information is produced by a simple computer simulation generated in response to a user command. Given this trace information, the CPS chooses which subintervals of time to capture with the camera and from which vantage points and with which lenses (i.e., what field of view). The animation can then be played back to the user using the intervals and camera placements selected by the CPS.

The primary data structure used by CPS is the film tree (Figure 7), which represents the film being generated. Of primary consequence are the scene and candidate idiom levels of the tree: each scene in the film is associated with one or more possible idioms, with each idiom representing a particular method of filming the parent scene. The CPS operates by expanding the film tree until it has compiled detailed solutions for each idiom, and then selecting appropriate candidate idioms and frames for each scene.

Internally, the CPS is implemented as a three-stage pipeline involving a sequence planner, DCCL compiler, and a heuristic evaluator, as shown in Figure 8. An idiom database (not shown) provides idiom specifications relevant to each scene in the animation being filmed.

We are particularly interested in interactive uses of automatic cinematography. Therefore, we decided to build a simple interactive game that would use CPS to film actions commanded by a human player. The basic plot of the game is very simple. The main character, Bob, must walk around a virtual world (in our case, SGI's Performer Town) looking for the Holy Grail. The game is made more interesting by the introduction of Fido, an evil dog that seeks to steal the Grail. From time to time, the game generates animations of Fido's activities and instructs CPS to edit these animations into the action. Eventually, Bob and Fido interact: if Fido gets the grail, Bob has to chase Fido in order to get it back. The user commands Bob through a pop-up menu. These commands fall into four basic categories: telling Bob to look in a particular direction, to move to a particular point on the screen, to pick up an object, or to chase another actor.

The implementation of the game and its various animation/simulation modules was done in C++ using the IRIS Performer toolkit running on SGI workstations, and based partially on the perfly application provided by SGI (a Performer-based walkthrough application). The game operates as a finite-state machine that produces animation traces as the user issues commands to the game engine, with the CPS acting as a separate library whose sole inputs are the animation trace and the database of idioms (Figure 9). The game itself (not counting CPS or the code present in the existing perfly application) required approximately 10,000 lines of C++ code. The CPS system is also written in C++ (despite the Lisp-like appearance of DCCL) and implemented with about 19,000 lines of code.

The sample game interaction presented at the end of our video is intended to demonstrate a number of the activities possible in the game, as well as the various DCCL idioms. For presentation purposes, the planning time required by CPS was edited out of the video; Table 2 gives performance data taken from a similar run-through of the game on an SGI Onyx.

The subject of using principles from cinematography to control camera positions and scene structure has received relatively little attention in the computer graphics or AI communities. We survey most of the related work here.

He, Cohen, and Salesin (1996) have developed a system for controlling camera placement in real-time using some of the ideas behind DCCL. Their work focuses on filming dialogues between multiple animated characters, and uses a finite state machine to select and generate camera positions.

A number of systems have been described for automatically placing the camera in an advantageous position when performing a given interactive task (Gleicher & Witkin 1992; Mackinlay, Card, & Robertson 1990; Phillips, Badler, & Granieri 1992). However, these systems neither attempt to create sequences of scenes, nor do they apply rules of cinematography in developing their specifications.

In work that is closer to our own, Karp and Feiner (Karp & Feiner 1990; 1993) describe an animation planning system for generating automatic presentations. Their emphasis is on the planning engine itself, whereas the work described in this paper is more concerned with the problem of defining a high-level declarative language for encoding cinematic expertise. Thus, the two approaches complement each other.

Strassman (Strassman 1994) reports on Divaldo, an ambitious experiment to create a prototype system for “desktop theatre.” Unlike our focus on camera placement, Strassman attempts to create semi-autonomous actors who respond to natural language commands. CPS is also complementary to Divaldo.

Drucker et al. (Drucker, Galyean, & Zeltzer 1992; Drucker & Zeltzer 1994; 1995) are concerned with the problem of setting up the optimal camera position for individual shots, subject to constraints. Specific camera parameters are automatically tuned for a given shot based on a general-purpose continuous optimization paradigm. In our work, a set of possible cameras is fully specified by the shot descriptions in DCCL and the geometry of the scene. The final selection from among this set of different shots is made according to how well each shot covers the scene. Our approach avoids the need for generic optimization searches, and it is guaranteed to result in a common shot form. The cost is a greatly reduced set of possible camera specifications.

Several useful texts derive low-level camera parameters, given the geometry of the scene (Foley et al. 1990; Hearn & Baker 1994; Blinn 1988).

We close by summarizing the contributions of this paper and describing the directions we intend to pursue in future work. The main contributions of this paper include:

Surveying established principles from filmmaking that can be used in a variety of computer graphics applications.

Describing a high-level language, DCCL, for specifying camera shots in terms of the desired positions and movements of actors across the screen. We have argued that DCCL represents cinematic knowledge at the same abstraction level as expert directors and producers by encoding sixteen idioms from a film textbook (Arijon 1976) (e.g., Figure 2).

Presenting a domain-independent compiler that solves DCCL constraints and dynamically controls the camera.

Describing a domain-independent heuristic evaluator that ranks the quality of a shot specification using detailed geometric information and knowledge of desirable focal lengths, shot durations, etc.

Describing a fully-implemented film camera planning system (CPS) that uses the DCCL compiler and heuristic evaluator to synthesize short animated scenes from 3D data produced by an independent, interactive application.

Incorporating CPS into a prototype game, and demonstrating sample interactions in the game (see videotape).

This research was funded in part by Office of Naval Research grants N00014-94-1-0060 and N00014-95-1-0728, National Science Foundation grants IRI-9303461 and CCR-9553199, ARPA/Rome Labs grant F30602-95-I-0024, an Alfred P. Sloan Research Fellowship (BR-3495), and industrial gifts from Interval, Microsoft, Rockwell, and Xerox.

Arijon, D. 1976. Grammar of the Film Language. New York: Communication Arts Books, Hastings House, Publishers.

Blinn, J. 1988. Where am I? What am I looking at? IEEE Computer Graphics and Applications 76–81.

Christianson, D. B., Anderson, S. E., He, L., Weld, D. S., Salesin, D. H., and Cohen, M. F. 1996. Declarative camera control for automatic cinematography. TR UW-CSE96-02-01, University of Washington Department of Computer Science and Engineering.

Drucker, S. M., and Zeltzer, D. 1994. Intelligent camera control in a virtual environment. In Proceedings of Graphics Interface ’94, 190–199. Banff, Alberta, Canada: Canadian Information Processing Society.

Drucker, S. M., and Zeltzer, D. 1995. Camdroid: A system for intelligent camera control. In Proceedings of the SIGGRAPH Symposium on Interactive 3D Graphics ’95.

Drucker, S. M., Galyean, T. A., and Zeltzer, D. 1992. CINEMA: A system for procedural camera movements. In Zeltzer, D., ed., Computer Graphics (1992 Symposium on Interactive 3D Graphics), volume 25, 67–70.

Foley, J. D., van Dam, A., Feiner, S. K., and Hughes, J. F. 1990. Computer Graphics, Principles and Practice. Reading, Massachusetts: Addison-Wesley Publishing Company, second edition.

Gleicher, M., and Witkin, A. 1992. Through-the-lens camera control. In Catmull, E. E., ed., Computer Graphics (SIGGRAPH ’92 Proceedings), volume 26, 331–340.

He, L., Cohen, M. F., and Salesin, D. H. 1996. Virtual cinematography: A paradigm for automatic real-time camera control and directing. To appear at SIGGRAPH ’96.

Hearn, D., and Baker, M. P. 1994. Computer Graphics. Englewood Cliffs, New Jersey: Prentice Hall, second edition.

Karp, P., and Feiner, S. 1990. Issues in the automated generation of animated presentations. In Proceedings of Graphics Interface ’90, 39–48.

Karp, P., and Feiner, S. 1993. Automated presentation planning of animation using task decomposition with heuristic reasoning. In Proceedings of Graphics Interface ’93, 118–127. Toronto, Ontario, Canada: Canadian Information Processing Society.

Lukas, C. 1985. Directing for Film and Television. Garden City, N.Y.: Anchor Press/Doubleday.

Mackinlay, J. D., Card, S. K., and Robertson, G. G. 1990. Rapid controlled movement through a virtual 3D workspace. In Baskett, F., ed., Computer Graphics (SIGGRAPH ’90 Proceedings), volume 24, 171–176.

Mascelli, J. V. 1965. The Five C’s of Cinematography. Hollywood: Cine/Grafic Publications.

Phillips, C. B., Badler, N. I., and Granieri, J. 1992. Automatic viewing control for 3D direct manipulation. In Zeltzer, D., ed., Computer Graphics (1992 Symposium on Interactive 3D Graphics), volume 25, 71–74.

Strassman, S. 1994. Semi-autonomous animated actors. In Proceedings of the AAAI-94, 128–134.

¹ Most current games, of which Doom is the classic example, still provide each participant with a single point-of-view shot; however, a number of games such as Alone in the Dark, Fade 2 Black and Virtua Fighter have begun to employ a wider variety of perspectives.

² A notable exception is Alfred Hitchcock's Rope, which was filmed in a single shot, albeit with disguised breaks.

³ To understand why this argument is necessary, recall that a go-by fragment results in a static camera position directed at an actor who moves across the field of view. An example of a go-by fragment is the first shot of Figure 2. Note that Arijon (1976) expresses the shot not by specifying the temporal duration, but rather by indicating (with arrows) the range of motion that the actor should cover while the film is rolling. DCCL uses movement endpoints to allow the same type of declarative specification and relies upon the compiler to calculate the temporal bounds that will yield the proper range of motion.

⁴ This explains the definition of out-right and out-left. Arijon (1976) specifies that shots in which an actor moves off-screen (or onto the screen) should be cut three frames after (or before) the actor disappears (or appears). As a result we define out-right and out-left in terms of the distance traveled while three frames transpire.

⁵ Typically, the camera is restricted to one side of the line of interest. However, opportunities to “switch sides” sometimes present themselves, such as when an actor turns to walk in a neutral direction.

⁶ Indeed, if there are n scenes and each scene has k candidate idioms as children, then the film tree represents nᵏ possible idiom combinations.