Thursday, January 20, 2011

Craig's Directory - Clean-up

I decided yesterday, to have a look at a sample of the websites I had approved over the last 18 months. Surprisingly, quite a few of the websites no longer exist, or the page being linked to has been removed?

Any guesses how long it would take me to check over 56 thousand web page manually? My broadband provider would love me, but sadly, my wife wouldn't.

I therefore developed a small webbot to do the checking for me. After checking a website it writes the response status codes (200, 404, etc) into the database. All I have to do now is review all the negative response codes and remove the defunct websites from my listings.

So, if you find your website no longer exists in Craigs Directory, apologies, but please check the page I was linking to - it may no longer exist. If this is the case, please re-submit your website.

Regards,
Craig

No comments:

Post a Comment