Over time, as new websites are launched, URLs are changed and pages are removed or unpublished, broken links begin to litter the search results, and significantly impact the rankings of websites. As an SEO, one of the first things you learn is that broken links are one of the biggest no-no’s in our digital world. Often times they are a first priority when beginning to optimize a website. A good place to reference your broken links in search is through Google Search Console’s (formerly Webmaster Tools) crawl errors report.

Search Console Crawl Errors Report

Every time I  head into Google’s Search Console to review the crawl errors report, I’m hopeful that the number of errors is low. If a website is properly managed, one can be fortunate enough to have little to no 404s, 500s and other errors. In certain cases when a website isn’t properly managed, there can be thousands. In order to start correcting the errors associated with your domain, you first have to get there. Here is how you can do that:

1. Head to Google’s Search Console
2. Select your primary domain (or submit and verify it)
3. In the left hand side navigation click the “Crawl” drop-down
4. Select Crawl Errors

Once you’ve arrived here, you’ll see server errors, soft 404, access denied, and not found for desktop and smartphone. Generally, I download all of these errors to a Google Sheet via the “Download” button. This will make it easier to test that links are still broken and to keep track of how you fixed them.

How to Fix/Remove Crawl Errors

How you resolve a crawl error depends on the error. In my experience, the majority of crawl errors are 404’s (soft, not found), so a 301 redirect should permanently resolve the issue. At times, you’ll receive a 500 (server) error that is the result of the site temporarily being down. 500s like this are especially telling of the importance of testing. Whether you have a 400 or 500 error, you must test to make sure that the link is actually broken, or you could end up creating a loop or directing users away from a perfectly good web page. Speaking of examples, here are some common situations that cause 404’s in search results:

Common Broken Link Example: Old Website (.asp/.aspx/.htm/.html) and Website Launches

Older websites used to have URLs that ended in .asp, .aspx, .htm and .html. Many newer platforms, like WordPress, eliminate those endings, because the websites are built in a different way. Let’s say I’m a plumber and my original website had a page for my services. The URL on the original website for this page was plumberdomainname.com/services.html. I decided that I wanted a new website, went ahead with it, and it’s now launched. I have a services page on my new website is well, and the URL is plumberdomainname.com/services/. The issue that arises is that Google and other search engines still have your old website indexed, so plumberdomainname.com/services.html is still appearing, and when a user clicks it, it takes them to a broken page. If you redirect users from /services.html to /services/, it will bring them to the correct page, eliminate that broken link in search, and help Google and other search engines begin to understand your new website.

Common Broken Link Example 2: Changed URL’s/slugs

Let’s say you’re a food critic and your SEO team handled your site launch properly. You’re in a new WordPress site and you’ve been trained on how to add blog posts. You recently added a blog post, but decided you didn’t like the URL ending (also called a slug) that you put on the blog post. This blog post was titled “The best types of cheese to pair with wine and charcuterie.” You originally had the URL set up as so: foodcriticdomain.com/the-best-types-of-cheese-to-pair-with-wine-and-charcuterie/. You didn’t like how long this slug was, so you changed it to foodcriticdomain.com/cheese-wine-charcuterie/. It’s a great SEO practice to keep slugs short, but there’s one problem: you now have a broken link in search from that original URL. How do you fix it? Redirect the original URL to the new one.

Common Broken Link Example 3: Employment Listings

Perhaps you’re in a business where jobs are constantly in demand, so job listings are always being published and unpublished on your website. Most of the time a new listing is created with a new URL and content, but sometimes the same job appears and disappears, so you publish and unpublish it regularly. Whenever you unpublish a listing, that link is still indexed in search, so you’ll need to temporarily redirect it (with a 302) to, perhaps, your employment page for the most relevant information at that time.

How to Prepare for a New Site Launch or Proactively Check Existing Links in Search

While Google is pretty quick to recognize these broken links, lets say they haven’t and you’d like to be proactive about finding them, because your new website has already launched, or you’re just curious about the links you have out there. Going into Google, you can type site:yourdomainname.com and many, if not all, of the indexed pages from your website will appear. If you’re going to launch a new site and would like to be proactive (as a business owner, marketing manager or SEO team) you can take a few steps to ensure your website has a smooth transition, eliminates or minimizes broken links, and rankings aren’t crushed with your new URLs.

1. Download a list of URLs existing on your current site with a software like Screaming Frog
2. Plan for your new launch by deciding if you’re keeping existing pages, deleting some or adding any new ones, and note what the URL will be or where it will be redirected to
3. When the new site is launched, implement the redirects
4. You should still monitor the crawl error report going forward, because you never know what else may be out there in search. Better safe than sorry!

How to Notify Google That You’ve Fixed The Existing Crawl Errors

This is very simple. Once you’ve fixed the a crawl error, you check the box for the particular URL, and then click “Mark as Fixed.” If you’ve successfully fixed the crawl error, it will not reappear in the report. Oftentimes, if there are a decent amount of crawl errors, I’ll fix them all, then select all of them and mark them as fixed all at once.

Why You Should Monitor Crawl Errors

You never know what is lingering out there in the search index, and sometimes websites can have a lot of pages, posts and other items that are difficult to keep track of. If you monitor the crawl errors report, you’ll be sure that your domain is clean and free of broken links in search, which is a big positive for your SEO efforts.

With this work, you’ll already be a step ahead of a few of your competitors, and you can move onto other SEO efforts. Oftentimes, broken links can limit the success of other good SEO practices, so as you move onto other SEO work, the results may be amplified to consistently improve your rankings and drive qualified traffic to your website.

Helpful Links:

https://support.google.com/webmasters/answer/35120?hl=en

https://webmasters.googleblog.com/2012/03/crawl-errors-next-generation.html
“”