Up all night… again.

As Saturday night becomes early Sunday morning, I find myself frantically tying up loose ends in order to prepare for the “UI integration” phase of KitchenPC, which marks the final coding phase before the private beta release.  I spent the evening tracking down resolutions to a few of those elusive bugs that I’ve been putting off for far too long now.  The ones of interest are:

HTML special codes were being double-escaped

This seems like it should be easy to fix, but for HTML purification, I was using an open-source project called AntiSamy.  When I started, this project was the only one I could find that was a viable white-list approach to purifying HTML and preventing XSS attacks.  There was a few major features missing (like validation of inline CSS styles,) but I was able to construct the input HTML using TinyMCE to not rely on those features.  However, for some reason certain HTML character codes would get escaped, regardless of what the allow-list said.  HTML text that included   would be allowed, but anything else, such as → would get turned into the super annoying →.  WTH?  After putting this off long enough, I decided to track down the issue but instead found the entire project is now pretty much abandoned.  Before I could dig up the source and try to fix this myself, I ran across the recently updated Microsoft AntiXSS Library.  This library, previously part of the Exchange Server SDK (which would actually reference Exchange Server DLLs that I don’t have!), was since released as an open-source, free-to-use library for sanitizing HTML and HTML fragments.  It took all of about 5 minutes to switch my code over, and it works absolutely beautifully!  The one problem was that I had to go through several dozen existing recipes and clean them up by hand.

Archive support for recipes

Now that anyone can edit any recipe, I got scared and decided I should probably archive recipe versions.  Luckily, this was a pretty easy feature to implement.  It required a new table with a recipe id and a text field.  Every time a recipe is updated, I serialize the old version of the recipe as text and stick it in the table.  This will eventually lead to an ability to see old versions of the recipe, highlight differences, and roll back to previous versions similar to most Wiki software out there.  For now, it’s just a safeguard so that I will have some way (albeit hacky) to restore recipes that get corrupted due to accidental user interference or malicious play.  This feature makes me feel a bit better about allowing users to edit any recipe they wish.

Fixed invalid conversion data in database

This one has been annoying me for quite some time.  Sometimes when the user created a shopping list from a recipe, the web service would throw an exception as the form the ingredient was used in could not be converted into the form it’s purchased in.  There was really no good way to enforce this integrity through database schema design, so these errors had to be tracked down manually.  Fortunately, I found a good way to find them all at once, and keep the database clean.  For recipe modeling, I keep certain recipe and ingredient “relationships” in memory in a giant graph.  This graph stores ingredients in their purchased form, but reads every recipe usage up front when the site app loads.  I’ve modified this code to log an error if it runs into any conversion problems.  The code it calls is basically the same code path as adding a recipe to the shopping list.  Furthermore, I’ve been doing performance tests on a database with 10,000 “randomized” recipes.  These tests have exposed invalid conversion data on ingredients that were not even being used on the production DB.  I’ve managed to track these all down and fix them, so there shouldn’t be any such annoying bugs for beta.

Misc bugs that don’t depend on the new UI

I’ve also fixed a few smaller bugs, made some KitchenPC Mobile improvements, and a few finishing touches here and there.  The bugs I have left (and there’s only 6 of them at this point) all will have to wait until the new UI integration.  They are all marked as “Minor” bugs.

KitchenPC Modeling Engine is done!

This is also huge news.  The modeler is now code-complete and running lightning fast.  Generating a model with 10,000 recipes in the database takes about 2 seconds, and the results are excellent.  The overall algorithm design was pretty basic (I had that working in one night), but the hard part turned out to be how to really “define” success.  Computers will do exactly what you tell them, so it really helps if you know what you want them to do.  A lot of variables come into play here, which assume a knowledge of what the user really wants out of this feature.  For example, would a user prefer to use more of their pantry items if this also meant they had to buy more ingredients at the store?  Or are users incredibly cheap and want to buy as few new things as possible, even if they won’t get complete usage out of all their pantry items.  Someday, I believe the meal planner will have an “Advanced” section to allow the user to control these variables, but for beta I have to make a few assumptions about what the user really wants.  I’m really glad I didn’t end up contracting this work out to someone who doesn’t know or care about my users.

Waiting on the UI

On Friday, my Polish guys surprised me by finishing a day early on the final modifications to the prototypes (which are basically Photoshop mockups of each page.)  I’ve signed off on all these changes, and “cutting” (the process of slicing up the bitmap images into HTML) will begin Monday.  I’m hoping the designers will be able to provide me with HTML as each page is completed, which will allow me to work in parallel with them.  I considered putting up a few “teaser” screen shots of the finalized KitchenPC UI, but decided to unleash the final design in all its glory when it goes online.

Final steps

I expect the UI integration to take a week or two, and there will no-doubt be a few last minute bugs to address.  However, the next big step is going to be deciding on a place for KitchenPC to live.  I’m still bouncing back between Amazon EC2 and Rackspace CloudServers, but in reality if I don’t like my decision, I can always move later (it’s just a beta.)

I’d love to say the hard part is over, and it kind of is; getting a product you’re proud of up on the Internet is a huge step, and I really think people are going to enjoy what I’ve created.  I think it will be by far the best meal planning tool available on the Internet (Sorry Monki boy,) and the meal modeling features are just something that hasn’t been done before.  I can’t think of anything I should have done differently or any major regrets with the overall design of the product.  It’s simply going to rock!  I’m more proud of this than any product I worked on at Microsoft during my 12 year career there.

After KitchenPC goes live online, I’ll begin shifting my focus to the business side of things.  This is foreign territory to me, but hugely exciting.  The first step, of course, is to actually find 1,000 beta users.  Rather than just picking 1,000 people off the streets of the Internet, I’d like to do something a bit more intelligent and find a way to target my 1,000 to people who will really use the product and deliver valuable feedback.  Invites will of course go out to the 280+ people who took the survey, and I’ll probably give anyone who asks a few invite codes, but I’ll be spending some time over the next week or so figuring out the best approach for finding this beta group.

During the beta, I’ll be looking at usage metrics, studying how the server performs under certain loads, and addressing any performance problems critical to making possible a wider deployment.  These numbers will also give me an idea of costs associated per user, which is a must for any business plan.  Luckily, my site isn’t hugely data intensive so I expect PostgreSQL to quietly yawn as users explore the site.  This will also be my chance to see if I made any hugely wrong assumptions about the product, and consider the need for any major redesign.

I’ll also be trying to get a bit of media attention, perhaps from the likes of TechCrunch or what-not.  Issuing a formal press-release is probably a good idea, though timing is questionable on that one.  This blog will start to take a turn as well.  I’ll be talking more about future business plans for KitchenPC (such as my visions for revenue generation,) and detailing my adventures trying to get the word out.  Stay tuned!