I have been digging this topic since morning and I have my own views on Google's manual penalty.
I have straight question. Does Google manually review highly traffic and competitive keywords' Google listing? (without any report)
I don't think it should be. But this blogger www(dot)wisdek)dot)com/blog/Your-Guide-to-Understanding-Google-Manual-Penalty is saying that Google manually review top 3 sites for keyword like 'web design'.
What I believe is when Google detect unnatural links pointing to any website or they received spam report to that website, only then they manual review and apply penalty to the culprit website.
I am keen to hear your real experiences and thoughts. Any inputs will be appreciated.