next up previous
Next: Alignment and resampling Up: No Title Previous: Organization

Description of the reconstruction pipeline


Most of the work done in the reconstruction pipeline is preprocessing to correct the alignment of the example heads, build the eigenspaces, and create the linear estimators to fill in data not obtained from FLIRT. Each step in the pipeline is detailed below, and relevant programs from the software catalog (Appendix A) are listed. The sequence of operations is summarized in Figure 2.1.

Figure 2.1: Overview of the reconstruction pipeline.

  1. Head models are aligned in 3D (and, in FLIRT's version of the alignment code, color corrected) so that eyes, nose, and mouth line up as best as possible. This eliminates the need to encode information about the orientation of the user's head in the eigenspace. The merits of this ``view-based'' approach are discussed in [3]. ( align, alignToFLIRT)

  2. The aligned models are converted back into cylindrical-coordinate range and texture files, which now have the property that the eyes, nose, and mouth of each of the heads are in approximately the same 2D location in all of the data files. ( iv2float)

  3. The range map of the canonical head (the one to which the other models were aligned) is used as the basis to create masks for the modular eigenspaces --- see Figure 2.2. These masks highlight separately the eyes, nose, mouth, and hair of the canonical head. In addition, for comparison, a mask highlighting the entire head region is generated (corresponding to a single, rather than modular, eigenspace). All of these masks crop out information above and below the region of the head, so that the shoulder blades and any invalid information above the top of the head are removed.

    Figure 2.2: The attenuation masks for the eyes, hair, mouth, and nose.

  4. The masks are smoothed using a diffusion algorithm to eliminate hard boundaries around the highlighted regions, to avoid hard-edge artifacts in the modular eigenspaces. ( diffuse)

  5. The intensities of the texture components of the aligned head files are scaled down by a dataset-dependent amount, to avoid having the power of the texture component (which is three times as large as the structure component, per file) overwhelm the structure component. In other words, when the eigenspace is presented with some structure information and some texture information for a head, it should weight both equally when representing the head, as opposed to essentially discarding the structural information. ( lambdaCalc, combiner)

  6. Each of the head files is multiplied by each of the masks in turn to create a set of input heads per mask. ( combiner)

  7. An eigenvector decomposition is performed on each mask's head set. These are the modular eigenspaces used for reconstruction. ( eigenDecomp)

  8. For each modular eigenspace, the matrix of eigenvectors is reduced to the dimensions that will be acquired from FLIRT, and the linear estimator computed; see Chapter 4. The result is the matrix used for coding the incoming head data at run time. At this point the (offline) preprocessing stage is completed. ( estimatorCalc)

  9. At run time, the texture image of the user is acquired from FLIRT and its dot product is taken with each of the vectors in each of the modular eigenspaces' estimators. The resulting coefficients are used to weight the full eigenvectors when taking the weighted sum of eigenvectors for reconstruction. The result is divided by the sum of the masks, and the texture component is scaled back up. The output is a head file containing structure and texture information approximating the user's head. ( reconTrack)

next up previous
Next: Alignment and resampling Up: No Title Previous: Organization

Kenneth B Russell
Mon May 5 14:33:03 EDT 1997