I’m also a huge fan of data driven decision making so I clicked through on Pema Hegan’s Twitter update to read an experiment in A/B Testing my Résumé | Paul Butler. It is a fascinating read – very inventive, very smart and very savvy. Paul certainly deserves, at least, to get an internship out of this fascinating and well executed experiment.
My only comment is that this shows the best and worst of A/B testing in action. The best is the work, thought and robust methodology that went into this. The worst however is that, as Kevin Hillstrom at Mine That Data always reminds us, you can optimize the heck out anything, wringing the last 5-10% out of it but at some point you have to just rebuild it as brand new.
American Copywriter also covered the strange route a piece of creative can take if you optimize it for one thing (in this case, clicks from prepubescent boys!).
And everywhere in-between.