Screaming frog is a a useful SEO spider tool that can be used to crawl your website's
links, images, CSS, scripts, and apps from the SEO perspective. If you
didn't know about it already, then where have you been? :). It is a
small desktop software that will let you crawl an entire site on demand,
and will help resolve common SEO issues with your website. But crawling isn't all it's good for. It can do much, much more. In this post, we'll show you some of the most important uses of the Screaming Frog SEO Spider tool.
The functionality that Screaming Frog provides is immense. And you can
mix-and-match some features to create functionality of your own - the
possibilities are countless. We will attempt to create a power tutorial
on Screaming Frog soon. But for the time being, just check out these 5
great ways to SEO Optimize your website with Screaming Frog.
1. Verifying crawl errors
Google Webmaster Tools can be often slow to respond to the changes on
your site. For example, it might not find all the fixes you've made to
your broken links, You can, however use Screaming Frog to find out
broken link problems such as 3xx (eg 301), 4xx (eg 404s) and 5xx. At the
top, there's a bar where you can insert the URL of a website you need
to crawl. Once crawled, you can filter the results by HTML content only,
and then if you go to Advanced Export, you can export the links based
on their status codes (3xx, 4xx, 5xx etc).
You can, for example export your 404s, and then compare them with the ones you get in Webmaster Tools. You can also take all links from the Webmaster Tools (export as CSV, then copy all links and paste them in a text document). Now, if you go to Mode in Screaming Frog, and select List instead of Spider, you will be able to import a text file, which you can then check for errors. This will tell you all the 404s that are actually there, and will eliminate all the fixes that Google Webmaster Tools missed out on.
2. Creating a sitemap
You can use Screaming Frog to create a sitemap. All you have to do is,
go to Mode tab at the top, and click on Spider. Then, from the
configuration menu, click on Exclude. Here, you can exclude certain
types of URLs. For example, WordPress often adds something like
/?replytocom=123 when a user comments (for eg,
domain.com/my-post/>replytocom=12). You don't want such URLs to be in
your sitemap. So in the exclude menu, you can add this "\?repytocom."
(without quotes) to exclude such URLs. Check the Screaming Frog User Guide to find out more about excluding URLs.
Now do a full crawl of your site, and once that is complete, simply export the data by clicking on Advanced Export >> XML Sitemap.
Now do a full crawl of your site, and once that is complete, simply export the data by clicking on Advanced Export >> XML Sitemap.
3. Checking a sitemap for errors
Similarly, you can check an XML sitemap for errors. If you have your XML
file with you, go into Mode, and select List. Then, Select the XML
file, and import it into Screaming Frog. Let it crawl the links. Once
done, it will show you all broken links, and any other such problems.
This can really help keep your sitemaps fresh and healthy.
4. Check for successful site migration
When you change your site's domain, Google Webmaster Tools often goes
berserk, giving tonnes of 404 errors. You can, however, use Screaming
Frog to do it quickly for you. If you have a list of your old URLs, you
can import them in SF, and then find out which of them are giving you
404s. You can do a full-site crawl as well to find some of the broken
links. And you can also import data from Google Webmaster Tools and then
check them against any 404 errors.
5. Building links
As we discussed in our link-building exercise,
you can generate authentic backlinks to your site by helping other
webmasters out. If you do a site crawl for another site, and give them a
list of their broken links (source and destination), you can fall into
their good graces. It is indeed a great way to increase your outreach
and build connections, because it tells others that you're actually
reading their blog/website, and care for it. You can use this strategy
to build backlinks :)
Did you like this post? Got any questions? I know this might be very confusing, so don't hesitate to ask in the comments section below. We'd love to help.
0 Comments: