Well, it took almost 2 weeks, by my little webbot has finished it's work and checked over 60 thousand websites submitted to Craigs Directory.
In the end, 2,585 websites have been removed from the indexes. Quite a few of these websites were removed due to 404's (page not found) but most of the errors were DNS (Domain Name Server) errors - i.e. URL not recognized?
Now the hard work starts... I have to manually go through all approved websites and ensure that they still adhere to the submission guidelines. I have found that some individuals submit valid websites, have them approved and then modify their submission to allow some of their "suspect" websites to be indexed - I will find you...
Regards,
Craig
No comments:
Post a Comment