Google Webmaster Tools offer entrepreneurs and solopreneurs like us the opportunity to really understand what the visitors to our site(s) want from us without having to pay and arm and a leg. I have been guilty of using paid solutions first, often assuming that “you get what you pay for”. But I have to hand it to the folks over at Google. Their tools kick butt!
So today I’m going to walk you through how to do an url crawl or web crawl of your site. I stumbled upon this tool over at my Google account and decided to try it when I had to move over all of my old html urls at GirlShrink to the new wordpress urls. I used this tool to discover how many 404 errors I was receiving — and boy there were a LOT more than I thought.
Even if you aren’t moving over a site like I did – it’s a great idea to use this tool to discover if other sites are linking to you, but with the incorrect url. Believe it or not this happens more than you would think, and it’s important to know because if someone is trying to link to your fabulous article – you want their traffic to arrive to the correct destination.
For example, a link to my ezine sign up page at GirlShrink is http://girlshrink.com/better-choices-newsletter
But quite a few sites were linking to it and added an extra space in between words or added a number to the url by mistake. Those links were getting 404 errors. Now instead of contacting those webmasters and asking them to correct the link (because of course who is really going to do that), I can now just redirect that traffic to the correct url by setting up a Google 301 redirect.
I wouldn’t have had that information if I hadn’t used Google’s Web Crawl tool. So now that I’ve explained the why – let me explain the how…
1. Sign into your Google account. If you don’t have one, now is the time to create one. Everyone should have their own gmail email, even if it’s just for signing up for a bunch of newsletters, as well as a few other tools that we will talk about in future lessons.
2. Click on Webmaster Tools.
3. You can add as many sites as you want to your one Google account, but for today just add the name of the site that you want to crawl today where you see the box to “add site”.
4. Once you have set up your account, and see it in your dashboard, now you can click on it. *Hint* – It’s a good idea to have a site map set up on your site to help Google give you more accurate stats.
5. Now you should see a screen like this:
6. While I fixed over 200 errors last week, you can see that Google has detected that I still have seventy-six 404 errors. Ugh! That is the problem with moving over a really big site, but at least I have the information and know what I need to fix.
If I click on “details”, I can see each and every url that is the problem:
7. If I click on the link under “linked from” I can see the sites that are linking to this particular url. Sometimes you will find that you have linked to your own pages badly, other times you will find that other sites have linked to you incorrectly.
Since it was a huge job, I used this to prioritize which links I needed to fix immediately. All links from other (3rd party) sites were of course first priority. My interlinks were second priority.
And that’s it! A simple solution for checking the linking structure of your site and making sure that are not a victim of the dreaded 404 errors.
Let me know if you’ve discovered any errors on your website that you were unaware of (I love it when that happens!), or what you think of your new Google account. Look forward to hearing your thoughts…