Mechanical Turkey

These days, much of the time that I would like to devote to working on my web crawler has been usurped by randomizations with pre-made meal plans.  I fret that I’m woefully behind on my goal of having ten pre-made meal plans (one for each category) by launch; though I have been moderately successful in getting some friends, as well as  some in the professional realm, to assist with this project.  One thing I have not been able to get much assistance with is recipe photography.

It’s very important to me to have ten really good, well tested and well thought-out meal plans for launch.  I want to show off what the site can do with these plans.  I want to individually advertise each of these plans on Facebook and Google.  I want whatever press I can muster up for launch to eat these plans up, and keep users coming back for more.

Obviously, one of the keys to good recipe content is good recipe pictures.  Unfortunately, most of the content I’m getting from nutritionists and dieticians is in the form of a Word document or PDF, and usually sans pictures.  I’ve had a little luck getting some friends to try some of the recipes out and snap some shots, however this approach would result in me losing friends faster than achieving my goals.

Yesterday, I decided to try a new approach.  Yes, once again, Amazon’s Mechanical Turk service.

Those who frequently peruse my ramblings might be asking, “But Mike?  Didn’t you try out Mechanical Turk before and declare it a scam-ridden wasteland of automated succubots, ensuring any user dumb enough to use it would surely spend more time cleaning up the mess it created than any time savings yielded?”

Well, yes I probably did say something similar to that.  But this is different!  Let me tell you why!

Protip: Ignore Amazon when they tell you Turk is designed for short, quick repetitive tasks.

When I tried Mechanical Turk before, I was using it for ingredient matching.  For example, I’d upload a list of ingredient usages found across thousands of recipes.  The goal would be to match something like “4 cups of shredded cheddar cheese” to the number 4, the unit “cups” and the “shredded” form of “cheddar cheese”.  This was somewhat of a crowd sourced version of the natural language parsing technology I have since developed.

This approach failed because the results would be tainted by scammers that are hoping to make a quick buck.  I’m quite certain some of them are running automated scripts to just fill out random answers to any Turk HIT they can find.  This caused issues such as over 1,000 HITs getting matched with Captain Crunch cereal.  This created a mess that was nearly as difficult to clean up as the BP oil spill.

Turk experts would chime in here saying building a reliable team of workers is something that doesn’t just happen overnight.  I should have tests to pass first, approve each worker individually, assign each HIT to multiple workers to ensure a consensus, blah blah blah.  Unfortunately, I had time for none of that so I’ve instead declared Mechanical Turk to be a bad fit for that particular task.  It turned out writing a natural language parser was a lot more geeky anyway.

So, I gave this another shot with recipe pictures.  Who wouldn’t want to cook a recipe and take a picture of it for money, right?  I decided to start out with a meal plan being developed by Say I Do Nutrition Services in Seattle.  This plan has seven recipes, and I already had pictures for two of them.  I created five Mechanical Turk HITs that basically said, “Cook this recipe, take a picture of it, make sure it’s not a crappy picture or I’ll reject it, and don’t just steal a picture of something similar online because I’ll know.”

I gave each worker 12 hours to complete the job after they accepted the HIT, allowing people to make it for their families for dinner last night.  Oh, and I also restricted the work force to only those in the U.S.  I’m not racist or xenophobic or anything, I just think most of the scams and bot-thingies are going on over seas, perhaps funded by fake Nigerian princes.  For this job, I require nice southern housewives trying to feed their two-and-a-half kids.

I also paid $20 a HIT, which I think should more than cover all the ingredients involved.

The Results

NOT Beef Stew

Within about 20 minutes, I got one HIT completed for a beef stew recipe.  Considering that recipe takes four hours to cook, I knew it was a fake.  Sure enough, attached to the HIT was a link to a random picture on the Internet of a sandwich.  Mmm, sandwich.  But no, this was not what I was hoping for.  Within a few clicks, I was able to reject the HIT, sending it back to the available HIT pool for someone else, as well as block the user so they could bother me no more with their sandwich-based trickery.  Only worrying about five HITs was far easier to manage, so this slight detour was hardly annoying and perhaps slightly entertaining.

One thing that Mechanical Turk lacks is the ability to see how many workers have accepted a HIT.  In fact, you get no feedback at all until they actually submit the results.  For this reason, nothing happened for about five hours.

Broiled Salmon

Then, all evening, I started getting various pictures submitted of each of the five recipes.  A lot of the submissions contained a dozen or more pictures, from various angles and lighting.  Several were very artistic, such as the recipe for wine stew included a glass of red wine next to it. A few included comments saying how much they enjoyed the recipe, and that this was the best job they’ve ever found on Mechanical Turk.  I had absolutely no problems with any of the photos submitted, and I added all five workers to a custom group so I can use them again later.

Bonus!

Another highlight of this approach was free advertising.  Not only did I see a huge spike in website traffic right after I posted the HITs, I also got a few unsolicited emails, such as this one:

I wouldn’t be able to make the meal though I would love to. I just wanted to let you know that I am impressed with the recipe site and will be using it in the future! Thank you!

Several of the workers also said they loved the site and would be coming back as well.  This makes sense, of course, as the posted job seeks the exact same target demographic as my website itself does; people who can and love to cook.  Not only did I get photos for all five recipes, I also got a few new users for the site!

Though this approach is a bit expensive, at $100 bucks for 5 recipe photos, it might be the best way to get ten meal plans up and ready for launch.  I don’t think going much lower than $20 per recipe would yield the quality I expect, plus I think it’s fair to at least cover the cost of the ingredients.

With that said, I think I’ll try to cook as many recipes as I can myself, and continue to pester friends and family to help out as well, but knowing I can quickly get photos of any leftover recipes I have within a day definitely lifts away a lot of the stress associated with building this content.

Flank Steak	Wine Stew
Salmon Stir Fry	Tofu Stir Fry