There are many ways to look at an eigenvector or eigenfunction analysis of a data set or linear system: it decorrelates the data as much as possible; it finds the fundamental modes of the system; it minimizes mean-squared error for reconstructing the original data points or vectors. However, since we are performing coding of the input data, we will consider it a coding problem.
All of the elements in our data set, the head models, can be considered points or vectors in a high-dimensional space by concatenating the rows of the range and texture maps to form one long vector per head; each vector contains roughly 65,000 dimensions for a subsampled head. We perform an eigenvector decomposition on this data set (a matrix containing these vectors as its columns) by finding the covariance matrix of this data set and then finding the eigenvectors of that covariance matrix. (Actually, we perform an well-known equivalent operation which prevents the need to compute the full 65,000 by 65,000 covariance matrix; this is described in, for example, , and is equivalent to the singular value decomposition.)
The eigenvectors of a covariance matrix can be chosen orthonormal, because a covariance matrix is by definition symmetric [7, p. 273,]. Let be the matrix of normalized eigenvectors of the covariance matrix of a data set, where each row is one eigenvector. Then we can consider the projection of a new column vector onto the subspace spanned by these eigenvectors, , as a coding operation () combined with a decoding operation (multiplication by ). Assuming we have used all of the eigenvectors of the covariance matrix, the following is true for all , where is an element of the original data set:
That is, the eigenspace projection of is an identity operation (modulo roundoff error). We want to verify that the modular eigenspace operation holds in a similar case.
Let be a diagonal matrix representing mask 1 (for example, the mask highlighting the region around the eyes), be the matrix of eigenvectors (as rows) of the data set of input heads multiplied by , and and be another mask and its eigenvectors (for example, the mouth mask). Then assuming we have kept all the eigenvectors of the two modular eigenspaces, the equation which we wish to verify for all in the original data set is
Because of the identity in Equation 5.1, the projection of into eigenspace 1 is an identity operation; the same is true for into eigenspace 2. Therefore this equation simplifies in this restricted case to
We have implemented the above formula; that is, the sum of the projections of an input vector onto the modular eigenspace is divided by the sum of the contributions of the masks. However, when is not in the original data set, or if we have dropped higher-frequency eigenvectors from the eigenspace, the projection of will not be an identity operation. Since the pseudoinverse of will in general not be equal to the inverse of , please note that this equation does not hold in general. It does hold when the modular eigenspaces are orthogonal to each other and the nonzero entries on the diagonal are unity, because it reduces to the case of a projection of onto an orthonormal basis by taking its dot product with the basis vectors. We can enforce this orthogonality constraint by making the masks orthogonal (i.e., no diffusion, and no overlapping regions).
The rationale behind using modular eigenspaces is that they manually delineate which regions of the face (eyes, nose, mouth) are approximately decorrelated. Separate, specialized eigenspaces can code high-resolution versions of these various features, and they can be combined independently, providing more reconstruction parameters. However, we found that when the modular eigenspaces were not orthogonal, the reconstructions sometimes had errors including large variations in the head shape (Figure 6.3). From experiments described in Chapter 6, we found that forcing the modular eigenspaces to be orthogonal caused other stability problems. We are considering techniques to reduce the modular eigenspace reconstruction error by interpolating nonlinearly among the modular eigenspaces' projections. Our current work involves using a search technique to automatically find masks which minimize cross-validation error on a given data set.