Have you ever clicked on the link on some page and got beautiful 404 page saying "Please make sure that you have typed the URL correctly"? Have you ever tried to open link that you found in google, the one that you had opened milion times before, and then saw the same page? I have. I know how annoying it can be. After all it is so easy to create custom 404 error page and display a message that is a little bit more intelligent.
Let's start with our own 404 page. Create new aspx file and call it whatever you want. PageNotFound is what I always use. I think it's good idea not to use the same template that we use for all the rest of our site - after all it is an error page. Pretty simple design, short message, maybe logo and few links with the most common used pages would be just fine. Now we need to set IIS to use this page for 404 errors instead of the default one. It's pretty simple. If you have never done this before take a look here http://technet.microsoft.com/en-us/library/bb727138.aspx. It will take you less than 5 minutes.
Ok, so we have custom error page, but what we were supposed to do was to display some intelligent message. That's the place where we can use url referrer. We can access it from code behind without any problems (Request.UrlReferrer). What does it give to us? In general we have 4 possibilities:
- Empty referrer. It means that user either typed the url and made a mistake, clicked on link sent by email of instant messenger or used a bookmark. This is what we can display to them: "Got a wrong link? Mis-typed the url? Your bookmarks are out of date?".
- Referrer is search engine. If referrer is from Google, Yahoo or any other search engine it is clear that index of this engine is most probably out of date. So tell user about it. We can even try harder. Several lines of code like those:
Show code >< Hide code
string referrer = "";
if (Request.UrlReferrer != null)
{
referrer = Request.UrlReferrer.AbsoluteUri.ToLower();
}
if (referrer.Contains("google.") ||
referrer.Contains("ifind.freeserve") ||
referrer.Contains("ask.co") ||
referrer.Contains("altavista.co") ||
referrer.Contains("msn.co") ||
referrer.Contains("yahoo.co") ||
referrer.Contains("looksmart.co"))
{
string site = referrer.Split('/')[2];
string reason = "Incorrect url found in [" + site + "] search engine (searching for: '";
string[] paramsString = referrer.Split('?');
if (paramsString.Length > 1)
{
List<string> searchTerms = new List<string>(paramsString[1].Split('&'));
List<string> queryStrings = new List<string>(4) {"q=", "p=", "ask=", "key="};
foreach (string searchTerm in searchTerms)
{
foreach (string queryString in queryStrings)
{
if (searchTerm.StartsWith(queryString, StringComparison.OrdinalIgnoreCase))
{
reason += searchTerm.Replace(queryString, "").Replace("+", " ");
}
}
}
reason += "')";
}
}
give us information about what user looked for. Maybe we're able to give them a hint with few most probable links? - If referrer is from some other site then we know that link on that site is incorrect. We're not able to do anything about this but we can always inform user that it's not our fault and that the link that they used is wrong.
- The last possibility is that referrer is from our site. It would be good to apologize and fix the link.
Yeah, fix the link ... Easy to say but ... we would have to know that we have a bad link somewhere on our site first. It is easier than you think. Let's send an email to us. Another piece of code will send a mail to our addresses with information on where user came from and what wasn't found:
Show code >< Hide code private void SendEmail(List<string> emails)
{
MailMessage objEmail = null;
try
{
objEmail = new MailMessage
{
Subject = CreateSubject(),
Body = CreateMessage(),
From = new MailAddress("error404@yourdomain.com")
};
foreach (string email in emails)
{
objEmail.To.Add(email);
}
SmtpClient smtpClient = new SmtpClient("127.0.0.1"); // assuming that smtp is running locally
smtpClient.Send(objEmail);
}
catch (Exception exc)
{
LOG.Error(String.Format("Exception while sending email {0}", objEmail), exc);
}
}
private string CreateSubject()
{
return string.Format("your site 404 error [ {0} ]", Url);
}
private string url;
private string Url
{
get
{
if (null == url)
{
string[] urls = Request.RawUrl.Split(new[] { ";" }, StringSplitOptions.RemoveEmptyEntries);
if (urls.Length > 1)
{
url = Request.RawUrl.Substring(urls[0].Length + 1);
}
if (url.Contains(":80/"))
{
url = url.Replace(":80", "");
}
}
return url;
}
}
private string CreateMessage()
{
const string message = "404 error on yoursite.com\r\n\r\n" + //
"URL: {0}\r\n" + //
"Referrer: {1}\r\n" + //
"IP: {6}\r\n" + //
"Browser: {2}\r\n" + //
"Crawler: {3}\r\n" + //
"User (if logged in): {4}\r\n" + //
"Possible reason: {5}";
string browserName = "";
string crawler = "";
string referrer = "";
string user = "";
if (Request.Browser != null)
{
browserName = Request.Browser.Browser;
crawler = Request.Browser.Crawler.ToString();
}
if (Request.UrlReferrer != null)
{
referrer = Request.UrlReferrer.AbsoluteUri.ToLower();
}
if (User.Identity.IsAuthenticated)
{
user = User.Identity.Name;
}
return String.Format(message, Url, referrer, browserName, crawler, user, GetReason(referrer, Url),
Request.UserHostAddress);
}
private static string GetReason(string referrer, string url)
{
try
{
string reason;
if (String.IsNullOrEmpty(referrer))
{
reason = "Mis-typed URL - empty referrer";
}
else if (referrer.Contains("google.") ||
referrer.Contains("ifind.freeserve") ||
referrer.Contains("ask.co") ||
referrer.Contains("altavista.co") ||
referrer.Contains("msn.co") ||
referrer.Contains("yahoo.co") ||
referrer.Contains("looksmart.co"))
{
string site = referrer.Split('/')[2];
reason = "Incorrect url found in [" + site + "] search engine (searching for: '";
string[] paramsString = referrer.Split('?');
if (paramsString.Length > 1)
{
List<string> searchTerms = new List<string>(paramsString[1].Split('&'));
List<string> queryStrings = new List<string>(4) {"q=", "p=", "ask=", "key="};
foreach (string searchTerm in searchTerms)
{
foreach (string queryString in queryStrings)
{
if (searchTerm.StartsWith(queryString, StringComparison.OrdinalIgnoreCase))
{
reason += searchTerm.Replace(queryString, "").Replace("+", " ");
}
}
}
reason += "')";
}
}
else if (url.Split('/')[2] != referrer.Split('/')[2])
{
reason = "Incorrect url on the referring site ( " + referrer + " )";
}
else if (url.Split('/')[2] == referrer.Split('/')[2])
{
reason = "Incorrect url on yoursite.com ( " + referrer + " )";
}
else
{
reason = "unknown reason";
}
return reason;
} catch (Exception exc)
{
LOG.Error("exception while looking for a reason", exc);
}
return "Unknown";
}
What is more, if many users try access the same not existing url (e.g. we change the url of one of the pages on our site), we can redirect them where they most probably wanted to be. I recommed using 301 redirect (permanently moved) instead of 302 (temporarily moved). I'm not going to draw water in a sieve trying to convience anyone of pros and cons of 301 redirect - if anyone is really interested in this, type 301 vs 302 in google. It's enough of this in the web already, let's do not create any artifical crowd ;)
Let's go further with 404. Our custom 404 page will inform us not only about not existing pages, but about the missing images, javascript files or even hacking attempts. If you will notice that some ips flood you with suspicious requests, you can always ban them. And if you are tired of getting and getting 404 emails that you are not interested in, you can always create a list of regexes and filter what do you want to send and what not.
Agree with me or not, but custom error pages for 404 or 500 errors should be one of the first steps in web application development.
January 26, 2009 at 9:59 AM
I'll definetely use that feature in my projects:)
January 27, 2009 at 9:27 PM
Nice post for the start, I agree completely that custom error pages are one of those small things which people tend to forget about but are important!
September 22, 2009 at 2:25 PM
September 22, 2009 at 2:27 PM
Pls reply me at darshanps@projectdemo.biz
thanks
December 21, 2009 at 11:32 AM
Nice post!
I have created customerrorpage.aspx for the 404 error and I have set the url in IIS-7 like (http://test/general/CustomerrorPage) it is working fine, but i would like to navigate to custom error page without specifying it in the url ,file in IIS. any help is appreciated!