Improving the way ASP.NET handles 404 requests

Sometimes things go wrong, information goes out of date, pages go missing. When these things happen it’s up to the humble 404 error page to let your users and the search engines know that things have changed. ASP.NET has a built in custom error pages feature to make it easy to set up your 404 page but here are a couple of tips to make the 404 experience a little better.

Return the right status code

By default the page handling a 404 error page doesn’t return a 404 status code to the browser. It displays the error message that you provided to the user but doesn’t have any extra information to flag the page as an error page.

This is called a soft 404. Soft 404 pages aren’t as good as ones that return the 404 status code because returning the 404 status code lets anything accessing your file that the page is an error page rather than a real page of you site. This is mostly useful for search engines because then they know they should remove dead pages from their index so users won’t follow dead links into your site from results pages.

Pages that return 404 status codes are also useful for error detection because they’ll be recorded in your server logs so if you have unexpected 404 errors, they’ll be easy to find. Here’s an example of the 404 error report in Google Webmaster tools:
Google webmaster tools has a nice report of any 404 links it found

Here’s how it looks if we use Firebug to spy on what’s happening when we load an invalid url into an ASP.NET application. It first shows a 302 redirect as the ASP.NET custom error handling code redirects from the original page to the custom 404 error handler. Then it shows a 200 ok code because the page is just a regular page and doesn’t know it should do any different:
firebug shows a 302 redirect and then a 200 ok status code

This is very easy to fix. You just need to explicitly set the Response.StatusCode property to 404 in the 404 error page:

  1. protected void Page_Load(object sender, EventArgs e)
  2. {
  3.     Response.StatusCode = 404;
  4. }

Once this is done, we still get the 302 redirect and then the 404 error code that should be returned:
now firebug reports a correct 404 status code when we hit a url that doesn't exist

Redirecting to the error page

The ASP.NET custom error handling logic automatically redirects the user to your 404 error page. The only bad thing is they do a browser redirect rather than a server redirect so the url of the page changes for the user from the url that they typed in, to the url of your error page.

So if a user entered in a url like they’d be redirected to

It doesn’t seem very slick to me. If the user typed in the url manually they will probably get quite confused (and probably a little annoyed) when the url they typed in has suddenly changed. If they simply made a mistake they’ll need to type in the whole url again. The physical location of the error page just doesn’t seem like information that will be important to the user while the url of the page they were trying to access is.

The good news is that it’s quite easy to create a HttpHandler that can listen for 404 errors and show the error page that is set up in the web.config without changing the url.

Here’s an example of what the code would look like. Notice that if you’re using a HttpModule to handle the 404 error pages, you can set the status code to the right thing in the module and don’t need to do it in the page:

  1. public class ErrorModule : IHttpModule
  2. {
  3.     …
  4.     public void Init(HttpApplication context)
  5.     {
  6.         // Listen for errors
  7.         context.Error += context_Error;
  8.     }
  10.     void context_Error(object sender, EventArgs e)
  11.     {
  12.         var context = HttpContext.Current;
  14.         // This example only handle 404 errors
  15.         // You could also add some similar logic for 500 internal server
  16.         // errors (logic errors) and do some logging
  17.         var error = context.Server.GetLastError() as HttpException;
  18.         if (error.GetHttpCode() == 404)
  19.         {
  20.             // we can still use the web.config custom errors information to
  21.             // decide whether to redirect
  22.             var config =
  23.                 (CustomErrorsSection)WebConfigurationManager.GetSection("system.web/customErrors");
  24.             if(config.Mode == CustomErrorsMode.On ||
  25.                 (config.Mode == CustomErrorsMode.RemoteOnly && context.Request.Url.Host != "localhost"))
  26.             {
  27.                 // set the response status code – if we set it here we don't need
  28.                 // to also set it in the page itself
  29.                 context.Response.StatusCode = 404;
  31.                 // redirect to the 404 error page from the web.config
  32.                 if (config.Errors["404"] != null)
  33.                     HttpContext.Current.Server.Transfer(config.Errors["404"].Redirect);
  34.                 else
  35.                     HttpContext.Current.Server.Transfer(config.DefaultRedirect);
  36.             }
  37.         }
  38.     }
  39.     …
  40. }

After the module is created it just needs to be registered in the httpModules section of the web.config file:

  1. <httpmodules>
  2.     …
  3.     <add name="ErrorModule" type="ErrorModule, App_Code"/>
  4. </httpmodules>

Complete example

Creating a good 404 experience

Noone likes 404 errors but they’re a part of being online. To finish up, here are some ideas about what you can do to make the 404 pages on your site as useful as possible to the people who end up seeing them:

Posted on 11 Feb 09 by Helen Emerson (last updated on 11 Feb 09).
Filed under ASP.NET