To better see the similarity among them, I first took the images from one Japanese blog. After simple normalization and registration, I can get an aligned, animated GIF looping through the 20 contestants as shown below.

These images remind me of a course project in computational photography I did two years ago. I wonder how the "averaged" face of these 20 contestants look like. Thus, I morph all the face images into a mean face shape, then average over the picture values using the code I developed in the course project. Here is the result.

Using the shape of the face images, I can generate a short movie of face morphing from contestant number 1 to 20. We can now see how one face is smoothly transformed into another.

Another application is to synthesize faces with new shapes. For example, I can morph each face a bit toward the mean shape. This results in a

*stabilized*version of animated GIF looping from contestant 1 to 20 as shown below.
So far, the above visualization figures are qualitative, we may be able to get more insights by quantitative analysis.

First, we construct the eigenspaces from these contestants faces. However, the standard Principle Component Analysis cannot be directly applied due to the slight pose variation and occlusion by the hair. Instead, we perform a robust version of PCA, e.g., [Robust PCA, Low-Rank Matrix Recovery], to factor the low rank part and the sparse errors. Specifically, I use the implementation provided in [Low-Rank Matrix Recovery].

[The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009)]

Here are three sample results.

[The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009)]

Here are three sample results.

Second, by performing singular value decomposition, we get the eigenfaces and the corresponding eigenvalues. Here we can see that the eigenvalues vanish after 7, suggesting that the rank of the image data is 6.

Here we visualize the largest 6 eigenfaces. These eigenfaces encodes the variation within the 20 contestants.

Third, by projecting faces onto the eigenspace, we can analyze how these faces are distributed in the eigenspace. We show here the coefficients of the 20 contestants. Note that most of the energies are concentrated on first two eigenvalues.

Thus, we plot the coefficients corresponding to the first two eigenvectors. We can now see how similar the appearance of these contestants are.

Fourth, we can try to construct the affinity matrix which encodes the pairwise similarity. (blue indicates similar and red indicates dissimilar).

This affinity matrix facilitates many interesting applications. For example, by summing up over columns or rows, we get a measure of how a contestant is different from the rest. In the plot below, x-axis denotes the contestant index (from 1- 20). The y-axis indicates how distinct the contestant is. This measure can also be interpreted as the salient object among the 20 contestants using the surround center information divergence.

(If you are interested in saliency, please see our paper on ICPR or slide.)

From the plot above, we can see that contestant 1, 2 and 6 are more distinct (or more salient) than the rest of the contestants. On the other hand, contestant 7, 12, and 15 are more common (i.e., closed to the averaged face) than others.

Salient contestants (1, 2, 6)

Salient contestants (1, 2, 6)

It is interesting that one actually used these most salient contestants (1, 2, 6) to support their opinions on how they are dissimilar [link]. (I am not sure whether they did the same eigen analysis or not.)

Common contestants (7, 12, 15)

Here are the pictures of all contestants.

Another application of using the affinity matrix is to come up with a displaying order with maximized or minimized similarity from one to another. You may use one of them to support your opinion on whether they are similar or not.

The heuristic algorithm works as follows. Start from a random contestant, choose the next contestant (that has not been selected before) with maximum or minimum similarity with the previous contestant. Repeat the process until we find the ordering.

Here is the result of ordering with

*maximized*pairwise similarity. Note that the transition between contestants are more smooth.

Here is the result of

*minimized*pairwise

*similarity. They look more dissimilar in this displaying order.*

Resources:

- If you would like to try it out on your own, you can download my code (in Matlab) here (12.7 MB)

- You can also just download the cropped, aligned, and mean faces here (12.6 MB)

Further readings:

- Average faces
- Face of tomorrow project by Mike Mike. [link]
- Face Research has a online demo for you to make an average face [link]
- Face morphing [wiki]
- Principle component analysis
- PCA [wiki]
- Eigenfaces [scholarpedia] [wiki]
- Low-Rank Matrix Recovery and Completion via Convex Optimization [link]
- Saliency
- Salience (neuroscience) [wiki]
- Hamiltonian path/circuit [wiki]

**Updates (May 4, 2013)**

1. The title of the original post was wrong. They are in fact contestants for Miss Daegu. I had corrected the title. Sorry for the confusion.

2. The poll results are out and is available on the official website

The top candidates are contestant number 5, 1, 15, 20, and 19. We may be able to "guess" how the quantitative analysis of similarity is related to the result.

The first thing I observed is that one of the most salient (No. 1) and most common contestants (No. 15) won the 2nd and the 3rd places, respectively. This result seems to coincide with the psychological study on averaged faces.

Thomas R. Alley and Michael R. Cunningham, Psychological Science, 1991

We then plot the coefficients corresponding to the first two eigenvectors for top 5 contestants (marked in red). It seems that the selection process somehow avoided choosing both candidates with similar appearances.

**Update (July 20, 2013)**

One of the most salient contestant (number one), Yoo Ye-bin, crowned the Miss Korea 2013.

Cool! Thanks for sharing interesting work.

ReplyDeleteThanks! It was a fun project.

DeleteAMAZING. I appreciate you sharing this.

ReplyDeleteNice and fun project. Cheers.

ReplyDeleteNice,

ReplyDeleteHowever, I'd have been interested with a study of the difference with a group on "normal" (i.e. not models) Korean girls.

interesting..

ReplyDeletethe candidates for this test should be proven that they underwent surgical procedure so we can see the "convergence" of their faces after surgery. also, sample pictures should have been makeup-less since they tend to use the same style of makeup and therefore be more similar especially in the eyes part.

ReplyDeleteinteresting, though.

I agree. This post is very far from a serious study on the effect of plastic surgery. If more sample faces (under controlled lighting, poses, and w/wo makeup) are available, then we may be able to see more interesting results.

DeleteOMG.

ReplyDeleteWOW this post is so coooooool!! Thought computer vision is not my major, I cannot understand it fully, I am considering whether it's possible to add a section to compare the average of the variance between the "normal people" and that between the "20 contestants." I believe we can see more interesting and inspiring results XD

ReplyDeleteTrue!

DeleteYes, that will be an interesting comparison. Mike Mike had a project which computes average faces for people in different cities.

Deletehttp://www.businessinsider.com/faces-of-tomorrow-2011-2#altogether-now-23

You can see the mean face of Korean near the top-right corner.

i think your talking about ANOVA (Analysis of statistical variance)

DeleteAlthough the comparison is interesting, it would only tell you about korean preference or what is considered beautiful in korean society, not anything about plastic surgery necessarily.

DeleteMakeups do make these pageants contestants look very similar rather plastic surgery. Here is the link of them before and after makeups:

ReplyDeletehttp://big5.xinhuanet.com/gate/big5/news.xinhuanet.com/photo/2013-04/28/m_124642965.htm

That explain my problem watching korean movies: I can't differentiate one girl from another. Men I have no problems; women look all the same to me.

ReplyDeleteBTW, american porn have the same problem: all women have the same long hair, eyes makeup and overall face shape. It's the same face repeated thousand times.

It's not a real picture. They used Photoshop.

ReplyDeletehttp://pantip.com/topic/30421514

You want to rewrite the code in R for me? :)

ReplyDeletewow what a lazy ass! :)

Deletehttp://stackoverflow.com/questions/5527145/convert-matlab-code-to-r

DeletePerv

ReplyDeletescience dude! fucking science.

DeleteI'm in Korea on business now haha... I keep falling in love

DeleteSo, I have a question. Did one 7, 12, or 15 end up winning?

ReplyDeletemy theory would be that, since they are all clearly trying to achieve an ideal, that the "most similar" of them should have the best chances, i.e. contestants 7, 12 and 15

DeleteThe poll results are out and available on the official website

Deletehttp://misskorea.mpluskorea.com/missdaegu2013_poll

Seems like both typical and atypical faces are popular.

可以用它來作判定美醜的基準嗎，例如，

ReplyDelete1.與範本的近似權值，越接近 "理想值" 就越美...

2.臉型眼眉鼻口相對位置與比例...整容業者會很想知道呢

3.弄個 App 把照片 "自動美容"...應該會大受歡迎

4.調整某些參數...拿來製作 美女臉蛋 的 3D 模型 ...

There was a work on enhancing facial attractiveness using data-driven approach. This might be close to what you have in mind.

Deletehttp://www.youtube.com/watch?v=lVbrUuwK-8g

SO NYEO SHI DAE RULEZZZZ

ReplyDelete可以用它來作判定美醜的基準嗎，例如，

ReplyDelete1.與範本的近似權值，越接近 "理想值" 就越美...

2.臉型眼眉鼻口相對位置與比例...整容業者會很想知道呢

3.弄個 App 把照片 "自動美容"...應該會大受歡迎

4.調整某些參數...拿來製作 美女臉蛋 的 3D 模型 ...

Agree!

Thus, we have mathematically proven that they arr rook same.

ReplyDeleteActually, I didn't prove anything in the post. We could not draw any conclusion from such limited samples.

DeleteAmazing. Nice to see practical applications of scientific analysis.

ReplyDeleteVery interesting and nice job!

ReplyDeleteI am wondering whether you can derive how unusual

their dis-similarity distribution is in a quantitive way?

such as, the standard deviation of their distribution is much smaller than a random sample!

Thanks! Yes, if we have some sample images of normal people in Korea, then, we might be able to see some interesting results. However, it may be hard to draw meaningful conclusion as we didn't control the shooting environment, e.g., lighting, pose, tone mapping.

DeleteCool! You can even find out which surgery clinic is shared by different contestants.

ReplyDelete你好，请问一下对齐的脸是自己处理的吗？还是你拿到的数据就是脸部（尤其是双眼）已经对齐的了呢？如果是您自己处理的对齐，能否说下使用的方法吗？

ReplyDelete我从网上找了这些照片，用了一些方法检测到每张照片中双眼的位置，但是每个人双眼距离不同，所以双眼无法对齐，平均脸就看着不像一张脸了，非常想知道对齐方法，谢谢您！

Hi Kevin,

DeleteNope, the raw data is not aligned.

To align the images using eye positions.

1. I manually labeled the eye positions for each faces.

2. Compute mean locations of these eye positions.

3. Compute similarity transformation from the eye positions in one image to the averaged position.

4. Wrap each image according to the similarity transformation computed in (3)

To compute the averaged face, I did something more. I first compute the mean shape. Then, wrap each image to the mean shape. Lastly, perform pixel-wise average.

You may take a look of my code to see exactly how these steps are performed.

Yes, they have very talented hair and makeup people, but more importantly, which one had the best ballet technique? Which one played the concert harp? Which one wrote the best essay on the future of Korean feminism? Which one is an anthropology major and talented violinist? Which one is most likely to re-write your code?

ReplyDeleteso beautiful photo gallery and blog

ReplyDeleteEverest Base Camp Treks

Trekking in Nepal

Trekking Company in Nepal

Annapurna base camp Treks

Manaslu circuit trek

Everest trekking

Annapurna circuit Treks

Langtang Valley Trekking

Upper Mustang Trekking

Kanchanjunga Trekking

Trekking Guide in Nepal

Peak Climbing in Nepal

Package tour in Nepal

Hi! This is really well done and smart working..I'm impressed much.

ReplyDeleteThanks you for sharing us.It's pleasure to follow you.

Lots of loves from Turkey.

Thanks! :D

DeleteThis comment has been removed by a blog administrator.

ReplyDeleteGood work thanks for sharing best Facial Plastic Surgery Post :)

ReplyDeleteHowdy! This is my 1st comment here so I just wanted to give a quick shout out and say I truly enjoy reading your posts. Thanks a lot!beauty surgery

ReplyDeleteThis comment has been removed by the author.

ReplyDeleteGreat work I enjoyed this post! I'm going to download and try the code now just for fun :) Thanks for sharing, really! Looking forward to more posts :)

ReplyDeleteNice work!

ReplyDeleteRemarkable blog! I have no words to praise, it has really allured me.PerfectNose UK

ReplyDeleteTrekking in Nepal

ReplyDeleteTrekking in Nepel

ReplyDeleteThis is an interesting blog on redefining open mind and I have learned a lot from it. In deed the modeling profession has greatly transformed and the participants are not only beauty queens but also intellectuals. Thanks for sharing this article with us and in case you need writing services you can hire our services by clicking on Paper Redoing Assistance.

ReplyDelete