In this project, I morph my face into someone else's, compute the mean face of a population dataset, and extrapolate from the mean to create caricatures. Morphing is defined by geometric warping as well as color cross-dissolving.
I picked an image of myself and a headshot of Kate Moss. Using photoshop, I remove the background of both images, roughly line up the faces, and crop them to the same dimensions for a smoother morph.
I manually define correspondence points on the images using the labeling tool that a former 180 student kindly provided. I compute the average between the correspondence pairs and define a single triangulation with scipy.spatial.Delaunay. The four corner points are chosen so the triangulation covers the entire picture including the background. This is important for smoothness in later steps.
As an intermediate step, I compute the mid-way face. This involves three main steps:
The most involved portion is step 2. Given the triangulation, original image points and mid-way points,
I compute the affine transformation A between each corresponding triangle, from the original to the target.
Then, I obtain every pixel within the target triangle using skimage.draw.polygon.
Applying inverse warping here, I transform these target points back to the original image through multiplication
with the inverse affine transformation matrix A_inv. This brings the
points back to the original image basis, but the pixel coordinates are not guaranteed to be integer values,
barring simple color mapping. So, I interpolate
the color for each pixel using scipy.interpolate.griddata and
nearest neighbor method.
Importantly, every point and pixel value of the original image are passed in the interpolation function, so the
resulting colors can be smooth.
Here are results of warping my face and Kate's face to the intermediate geometry, and the combination of the
two with colors cross-dissolved. I purposely chose two faces with very different shape and coloration to
demonstrate the changes better. A fun observation is in the eye color of the combined mid-face. My eyes take
on a very slightly blue-black hue during interpolation and Kate's eyes are hazel.
Since my eyes are much closer together than Kate's, the overlapping face has very large eyes that almost look blueish,
making the resulting face look very ambiguous and interesting. The result can look different if I were to meticulously
define correspondence points around the pupils, but I actually really enjoy how the eyes look!
I define morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac), allowing for intermediate percentages of warping and dissolving at different time steps. Varying them at equally space intervals, I form the morph sequence with 25 images.
Given the Danes dataset, I calculate the average shape of the entire population and subpopulations of male and female. I choose only images with suffixes -1m.jpg and -1f.jpg, since these all have similar neutral face expressions. Pre-existing correspondence points are defined and extracted from the dataset. Below is an example of the average triangulation over the whole population.
Here is the population breakdown: population: 40 | male population: 33 | female population: 7. It is very male dominated, so the average population face is very similar to the male subpopulation.
I also compute the morph sequence of a few select faces into the average face of the entire population.
The female morphs look somewhat awkward, so I also morph them into the average female subpopulation face, which look much better. Compared to male samples, these are more blurry due to sparsity of data.
Finally, this is my face morphed into the average Danes dataset face and the female subpopulation average.
Along with the results of warping just the facial geometries.
I create caricatures of my face by extrapolating outside the normal warp range from the average face, exaggerating certain features by over-warping to that geometry. The warping over-emphasize towards either set of facial features. As shown, when the extrapolation is towards the Danes average, the jawline is broad and nose is shortened. When extrapolated towards my facial geometry, the eyebrows are thick and jaw is pointy.
I change the ethnicity of my face to Brazilian using an average Brazilian woman's face that I found online. I go through the steps of morphing to change both the facial geometry and colors. The results show my face warped into the average Brazilian woman's face geometry, as well as a mid-face morph. The biggest difference is seen in the rounder face shape and shorter forehead. The cross-dissolved colors really tied the changes together!
