Monday, January 31, 2011

Craigs Directory - Clean-up Complete?

Well, it took almost 2 weeks, by my little webbot has finished it's work and checked over 60 thousand websites submitted to Craigs Directory.

In the end, 2,585 websites have been removed from the indexes. Quite a few of these websites were removed due to 404's (page not found) but most of the errors were DNS (Domain Name Server) errors - i.e. URL not recognized?

Now the hard work starts... I have to manually go through all approved websites and ensure that they still adhere to the submission guidelines. I have found that some individuals submit valid websites, have them approved and then modify their submission to allow some of their "suspect" websites to be indexed - I will find you...

Regards,
Craig

No comments:

Post a Comment