Without Exception

Part of being an entrepreneur is getting used to the fact that work is infinite in nature.  Actually accomplishing something doesn’t reduce the number of items in your to-do list, it simply creates more work.  The more productive you are, the more work will be created, causing an annoying feedback loop similar to putting a microphone too close to the speaker.

I was reminded of this over the long Thanksgiving weekend while hacking together the first few k-locks of code that will ultimately become the common mobile library for KitchenPC.  This is a UI agnostic API that will eventually power things like Windows Phone, Surface, Windows 8 apps, and other devices.

The main function of this library is to abstract client-to-server communications between a mobile or tablet device and the server.  It works similar to a web service proxy, like one that Visual Studio will generate for you from a WSDL contract.  However, I wanted to write one from scratch because, one, that’s the way I roll, and two, I can write a version that’s faster, more portable, and more light weight.

It’s written on the CoreCLR (which has been ported to various platforms, including iOS and Android) which provides only a subset of the raw power of the full .NET Framework.  I think developing a core KitchenPC framework on this platform can ease development for KitchenPC apps on various devices in the future, and also some day evolve into a full fledged open-source KitchenPC SDK.  I wanted to make the code pure, without any outside dependencies, and as portable, easy to manage and distribute as possible.

While basic WCF libraries are available on the CoreCLR, they’re prone to a few problems.  One, they only implement SOAP transports.  While KitchenPC can speak SOAP, the Javascript libraries use JSON to communicate with the server, which is much less verbose and efficient.  Two, my initial attempts to get that code running on the Mono framework were far from successful.  Even Xamarin has marked the WCF Stack as Alpha Quality.  If I wanted something truly efficient, truly cross-platform, and truly low-level, I was just going to have to write it myself.  So that’s what I did.

So, now you’re aware of the amount of work I signed up for, here comes the part where it spawned into more work, like crazed bunnies in the spring.

The KitchenPC web services are built with the idea in mind that return values should always be valid and indicate a successful result.  Errors should be handled as exceptions, which are serialized through SOAP faults or JSON exceptions.  This was the design decision I made, simply because it seemed cleaner to me.  I could just throw exceptions at any place in the stack, without having to litter try/catch blocks all over the place.  I’ve used a bunch of web-based APIs that return things like Result, which have a Success property indicating if the call was successful, an Error property with error information, etc.  The Exchange Server API is designed in this manner.

Well, turns out, when the ASP.NET Script Services throw an exception and that exception information is serialized out to the client, the resulting HTTP code is 500 (Internal Server Error).  This probably makes sense, as you’d want to be able to trap that code and handle that condition as an error.  In HTTP terms, the result looks like this:

HTTP/1.1 500 Internal Server Error
Cache-Control: private
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/7.5
jsonerror: true
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sat, 24 Nov 2012 22:42:33 GMT
Content-Length: 91

-- JSON serialized version of your Exception object

Now, with the CoreCLR WebClient libraries, to my knowledge (and after many hours of trying), there is simply no way to actually get access to that serialized JSON exception in the response.  Once WebClient sees a HTTP 500, it throws a generic WebException.  Basically, you’d do something like:

WebClient client = new WebClient();
client.Headers["Content-Type"] = "application/json; charset=UTF-8";

client.UploadStringCompleted += delegate(object sender, UploadStringCompletedEventArgs e)
{
   if (e.Error != null)
   {
      // e.Error will be a generic WebException, with nothing helpful of any sort
   }

   // On a successful call, e.Result will contain the HTTP response
};

client.UploadStringAsync(new Uri(url), json);

If the web service call failed, an HTTP 500 would be returned, which would cause e.Error to be set to a generic WebException object. The Message property would say something like “NotFound”, or something unhelpful. If you tried to access e.Result, you’d get another exception. Great.  They made it as hard as possible to do anything low-level with this API.

I tried every way I could think of to intercept the actual HTTP response before an exception is thrown (writing all sorts of really messy code,) but the framework was just not having it. It’s no wonder so many applications say things like “An error has occurred. Please try again later.” – It’s bloody impossible to ever get the information you need!

I suppose the only way around it would be to re-implement the HTTP protocol at the socket level, but who has that kind of time?

Without access to this exception information, I could not do things like tell the user their password was wrong, or if a menu could not be created and why.  Sure, I could guess what the error most likely was, but that’s lame.

Even if there were a way around this, I ran into another issue.  Through the course of debugging this mess, I discovered a major bug on KitchenPC that only repros on production.  I wasn’t even getting back exception data even if I were able to parse it.  The HTTP response looked like:

{"Message":"There was an error processing the request.","StackTrace":"","ExceptionType":""}

That’s right, the production KitchenPC server was eating exception information, which I rely on for various site UI functionality (such as logon errors, duplicate menu names, registration issues, etc)!  All this code was broken, and I hadn’t even noticed since I do very little testing in production.

Turns out, ASP.NET will only serialize exception information if you have <customErrors mode="Off" /> set in web.config, and debug mode enabled. This, of course, exposes all sorts of other debug information with site crashes as well, which you may or may not want your users to have access to. There doesn’t appear to be a way to allow ASP.NET web services to serialize certain exceptions, without running in debug mode, since the creators of ASP.NET saw this as sensitive data, and definitely not something you’d be depending on for your UI to function properly!  Uh oh.

During the KitchenPC beta, I just ran the code in debug mode as I actually wanted detailed errors to be displayed, logged, etc.  It was a beta product, I needed to log things, trap things, and if something happened, I wanted users to be able to report as much information as they could on it.  Thus, I never noticed this design was potentially flawed in production use.  I don’t think it was until I switched to .NET 4.5 and started using web.config transforms that I really had a solid production configuration.  So, many of the error handling features have been broken on my site for the past few weeks.  Sigh.

I knew the only approach was to re-think how exceptions are handled.  First, exceptions should be exceptional.  An exception should be thrown if and only if that condition was completely unexpected.  Things like bad passwords and duplicate user names are perfectly reasonable, and thus should be communicated with a valid return object and an HTTP 200 result.

I then spent all of Sunday night revisiting every single API, ripping out the exception throwing code and implementing a new base return type which has properties such as Error.  Ugh, I’ve now implemented the exact design I said at the beginning of this post that I hated so much, but I guess that’s how things must work.

It’s of course very important to get all these API issues worked out now rather than later.  Once I release mobile and tablet apps, changing APIs can be quite a hassle.  With web code, I can simply change the JavaScript files and invalidate the CDN cache when I deploy changes.  With mobile applets, I really don’t want to have to force users to upgrade to a new version to continue running their app.

These new changes have been rolled out to production, so there should be no more missing error messages when you enter the wrong password.  Now, all web service calls return either a Result object, or an object that inherits from Result.  This makes the KitchenPC API much more consistent, and allows me to handle result errors internally within my proxy so that I can throw exceptions when the result contains an error.

I think overall, it was a good redesign.  Now, maybe I can actually get some work done.