Matthew Laurin

Using The Screaming Frog SEO Tool On Your Site

The Screaming Frog LogoWhenever we find good SEO tools, we love to share them.  You’ll find a lot of duds out there but there are also a fair amount of really good tools that can make your life a lot easier.  This is especially true when it comes to the basics.  This post will be an overview of getting some basic SEO info about your site with the Screaming Frog SEO Tool.

Disclaimer: Screaming Frog is not paying us to do this post and they don’t even know about it.  It’s just an honest look at a well-made tool.  So feel free to use it or keep using whatever works for you.

Getting Data from Screaming Frog

This post is just covering the basics (which we’ll get to in a moment) so keep in mind that there are a lot of tasks you can use the data for.  We could literally write volumes on what you could use it for but in for the sake of time, we are just looking at extracting data for interpretation on the following factors:

  • Proper optimization of title tags, meta descriptions, URLs, Images and header tags
  • Response code errors
  • Canonical URLs
  • Special Directives
  • Page load times
  • Generating reports on all this stuff

Download Screaming Frog

You can download the tool here.  Note that you can get the whole thing for free but there are some limitations.  For example you won’t be able to crawl more than 500 URIs with the tool.  Note that this means you will be able to crawl and get information on this many URIs and NOT URLs or pages.  So if the spider crawls a Javascript file that counts as one URI.

The Screaming Frog Home Page

If you have less than 500 pages on your site then you’re getting a pretty good deal.  You just won’t be able to save reports and re-open them and there is some other limited functionality.  You will however be able to see a lot of useful data about your pages.

If you want to buy the program, the price is pretty reasonable.  It’s only about 167 dollars U.S.  Well worth the money for all that it does.

Once you have the program downloaded, installed and open, you’re ready to crawl.  Copy and paste your home page URL into the text box at the top of the interface and click the start button.  Depending on the size of your website, the crawl could happen instantly or it could take a while.  To put that in context, a few hundred page site may take a moment or two whereas something with thousands of pages may take 20 minutes or more.

The screaming frog default interface

Titles, Metas, and URLs

After the crawl is finished, you’ll notice a lot of data in all the tabs.

Page Title Tab:

Page titles tab in screaming frogHere you can see information related to the page titles of your pages.  For optimization in search, titles should be in the range of 55-65 characters.  For target keyword pages, they should also contain the phrase for which the page is trying to rank.

Screaming Frog gives information on the title tag length, the content and the page with which it is associated.  You can even export the report on its own to filter out pages that have titles, those that are too short and those that are too long in order to make optimization changes.

The meta description tag in screaming frogMeta Description Tab

Here you can get a lot of the same quantitative information as on the title tab.  See which pages have meta descriptions and which ones don’t.  See which are in the optimal range (around 145-160 characters) and which are not.

The URI Tab in Screaming FrogURI Tab:

In the URI tab you can see length of your page URLs as well as if the URI is relevant to a target keyword phrase.  You can also get a lot of other information about individual pages which we’ll go over in a moment.

Note that in all of the meta description, title and URI tabs (as well as other tabs in the interface), you can get a visual representation of what these elements look like in search.  The SERP snippet tab in the window pane at the bottom of the screen shows you what changes could look like before you make them live.  It’ll also show you where characters get cut off and where your optimal length should be.

Page Response Codes

In the reponse codes tab, you can see how your pages are behaving.  For a good user experience, it’s good to take care of any pages that are causing errors.  For example 404 errors or 500 internal server errors can be very frustrating for users and present missed opportunities in search.

Also, improperly redirected pages (302 instead of a 301) can cause a problem when certain pages should be indexed and others shouldn’t be.  You can use the page response code tab to see at-a-glance which pages are working properly and which ones aren’t.

In the response code tab, you can also see information on page load times for the URIs listed.  There isn’t a hard and fast rule for site speed.  There is no mandate that says your site has to load in 3 seconds or you lose rank in search.  There is however a range and if your site is taking longer than 8-10 seconds to load, you might want to think about optimizing it for speed.  There is an excellent post on the Moz blog that gives much more detailed information on site speed.

Special Directives

On the directives tab you can see data on directives for your site.  What’s a directive?  In computer programming, a directive is an instruction on how a computer program processes an input.  Screaming frog will show data on directives like those found in a robots file or a meta tag.

You can use this portion of reporting to see what if any page URLs are being excluded from search as well as any canonical URL directives.

Screaming Frog is an excellent tool for quickly gathering a lot of information about a website as it relates to its performance in search.  The creators were also generous enough to make a sizeable chunk of the program free so you can test drive it before paying for the full version.


Written by:

Loading Facebook Comments ...

No comments yet.

You must be logged in to post a comment.