On-page SEO can feel like a monster with a thousand heads. You fix one thing, but two more pop up. It’s frustrating when you’re pouring time and money into your website, but it’s not getting the attention it deserves from a Google search.
That’s why I’m going to show you how to use Screaming Frog to improve on page SEO. You’ve probably heard of this tool. It looks a little intimidating at first, I get it.
But you don’t need to be a tech wizard to use it effectively. Learning a few key functions will change how you approach your website. Let me show you my process for how to use Screaming Frog to improve on page SEO.
Table of Contents:
- What is Screaming Frog and Why Do You Need It?
- Getting Started: Your First Crawl
- How to Use Screaming Frog to Improve On Page SEO: 7 Key Checks
- Advanced Screaming Frog Features
- Conclusion
What is Screaming Frog and Why Do You Need It?
The Screaming Frog SEO Spider is a powerful desktop program. It crawls your website just like a search engine bot would. The SEO spider gathers all the data from your pages and organizes it in one place.
This allows you to perform a thorough SEO audit and spot problems quickly. You can find broken links, missing page titles, redirect chains, and many other common SEO issues. Think of the Screaming Frog SEO spider as your personal detective for technical SEO.
The tool has a free version that crawls up to 500 URLs. This is often enough for small business websites to start a basic SEO auditing process. A paid license unlocks more features, such as advanced configuration, JavaScript rendering, and API integrations, with no crawl limit, but you can get a lot done for free.
Getting Started: Your First Crawl
First, you need to download and install the Screaming Frog SEO software. It works on PC, Mac, and even Linux. Once you open the application, you’ll see a large address bar at the top.
This box is where you put your website’s URL. Just type in your homepage address and hit the “Start” button. That’s it, you’ve started your first crawl and the Screaming Frog SEO tool will begin gathering crawl data.
You’ll see a progress bar and lots of data filling the screen as it works. A crawl can take a few minutes or much longer, depending on your site architecture and the total number of pages. Once complete, you can save the crawl to review later or to track progress over time.
How to Use Screaming Frog to Improve On Page SEO: 7 Key Checks
Once the crawl hits 100%, you have a treasure trove of information. Now comes the fun part: finding things to fix. We’ll focus on some of the biggest on-page SEO wins.
1. Finding and Fixing Page Title Issues
Page titles are incredibly important. They tell Google and visitors what your page is about. They’re also what people see in search results and in the tab of their browser.
Go to the “Page Titles” tab in Screaming Frog. Here you can see the titles meta data for every single page on your site. Use the filter dropdown to look for specific problems.
Look for pages with “Missing” titles. Also, filter for “Duplicate” titles, as every important page on your site should have a distinct title. You’ll also want to look for titles that are too long (over 600 pixels) or too short, which you can see in the length columns.
| Issue | Why It Matters |
|---|---|
| Missing | Google has to guess your page’s topic. |
| Duplicate | It confuses search engines and users. |
| Too Long | Gets cut off in search results. |
| Too Short | Missed chance to use keywords. |
Fixing these is simple but takes time. You need to go into your website’s editor and write better, more descriptive titles for each page. Aim for titles that accurately describe the page content and include your target keyword.
2. Optimizing Your Meta Descriptions
While meta descriptions don’t directly impact rankings, they heavily influence whether someone clicks on your link in the search results. A good meta description acts like a free advertisement for your page.
Click on the “Meta Description” tab. It works just like the page titles tab. You can filter for missing, duplicate, too long (over 960 pixels), or too short descriptions.
Your goal is to have a compelling and relevant description for every important page. Think about what a searcher wants to know and answer that question. A great description encourages them to click and continue reading on your site.
3. Auditing Your Heading Tags (H1s and H2s)
Headings give your content structure. They help both people and search engines understand the hierarchy of information on a page. This structure is a signal of content quality.
Your most important heading is the H1. In Screaming Frog, you have H1 and H2 tabs. Go to the H1 tab first and use the filter to check for “Missing” H1s or pages with “Multiple” H1s.
Ideally, every page should have one and only one H1 tag. This tag should align with the page title and primary topic. Your H2s act as subheadings that improve readability and help define the page’s topical relevance.
4. Uncovering Broken Links (404s)
Clicking a link that goes nowhere creates a terrible user experience. Search engines don’t like it either, as it makes your site seem poorly maintained and can harm your SEO performance.
To find these, go to the “Response Codes” tab and use the filter to show only “Client Error (4xx)” codes. This will show you all the broken internal and external links on your website. Fixing these is a fundamental part of technical SEO.
To find where that broken link is, click on a 404 URL in the top window. Then, look at the bottom window and click the “Inlinks” tab. This will show you every single page on your site that links to that broken page, letting you fix it at the source.
5. Checking for Missing Image Alt Text
Alt text is the text that describes an image. It is important for screen readers that help visually impaired users browse the web. It also helps Google understand what your images are about, which can help them rank in image search.
Go to the “Images” tab. Here you will see every image file on your site. Use the filter to show images with “Missing Alt Text”.
This gives you a list of every image that needs attention. Go into your website and add descriptive alt text. For example, instead of “image123.jpg”, write something like “brown leather sofa in a modern living room”.
6. Analyzing Your Internal Linking Structure
Internal links are links that go from one page on your site to another. They are extremely powerful for SEO. They help users find more of your content and help spread authority throughout your website.
A good way to analyze your internal linking is in the main “Internal” tab. Find the column called “Crawl Depth”. This number tells you how many clicks it takes to get to that page from the homepage, which is a key part of your site architecture.
Important pages should have a low crawl depth. Another great column is “Inlinks,” which shows how many other pages link to that page. You can also use the reporting functions to visualize site architecture with a tree graph to quickly spot structural problems.
7. Identifying Thin or Duplicate Content
Google wants to show its users high-quality, helpful content. Thin content refers to pages with a low word count that give little value. Duplicate content is when you have the same, or very similar, text on multiple URLs.
You can identify potential thin content by sorting the “Word Count” column in the “Internal” tab from low to high. Pages with very few words might need to be improved, consolidated, or removed.
For duplicate content, go to the “Content” tab. Use the filter to look for “Exact Duplicates”. Screaming Frog identifies these pages by a hash value, showing you groups of pages that are identical so you can address them with redirects or canonical tags.
Advanced Screaming Frog Features
Beyond the basics, the paid version of Screaming Frog offers advanced tools for a deeper analysis. These features are particularly useful for complex websites and in-depth audits. They can help you uncover a wider range of SEO issues.
Crawling JavaScript Websites
Many modern JavaScript websites use frameworks like React, Angular, or Vue. These JavaScript rich websites often require a browser to render web pages and display content. A standard crawler may not see this content, leading to an incomplete audit.
Screaming Frog’s integrated chromium wrs (Web Rendering Service) solves this problem. By changing the rendering mode from Text Only to JavaScript, the SEO spider can crawl JavaScript and see the pages as a user would. This allows you to crawl dynamic, JavaScript rich sites accurately.
This is crucial for auditing sites where JavaScript websites render content on the client-side. The chromium wrs function lets you fully crawl JavaScript websites and analyze the final rendered HTML. This helps you crawl JavaScript websites render web content correctly and check for any issues.
API Integrations
You can greatly enhance your crawl data by connecting to external APIs. The ability to connect Google Analytics and Search Console lets you pull in user data like sessions, bounce rate, clicks, and impressions for each URL.
You can also connect to PageSpeed Insights to fetch Core Web Vitals data in bulk. This allows you to check web vitals like LCP, FID, and CLS across your entire site. Integrating with search console PageSpeed insights provides a holistic view of SEO performance and user experience.
All this data can be viewed alongside the crawl data within the interface or exported to Google Sheets for further analysis. The console page speed integration is particularly powerful for identifying pages with poor core web performance. Simply connect Google accounts in the API access menu to get started.
Generating XML Sitemaps
A well-structured XML sitemap helps search engines find and index all of your important pages. Screaming Frog makes it easy to generate xml sitemaps. After a crawl is complete, you can go to the Sitemaps menu and create an XML sitemap with a few clicks.
You can configure the sitemap to include or exclude certain pages, set priorities, and control which page types are added. This feature is a quick and effective way to manage your xml sitemaps without needing a separate plugin or tool.
Advanced Auditing Tools
The software includes several other features for detailed analysis. You can compare crawls to see what has changed on a site over time, which is perfect to track progress on your SEO efforts. This helps you verify that fixes have been implemented correctly.
The custom extraction feature lets you scrape any data from a page’s HTML using CSS Path, XPath, or Regex. This is useful for auditing structured data implementation, checking for analytics code, or extracting product prices. A comprehensive approach to common SEO involves using all available tools.
Conclusion
This tool can seem like a lot at first. But hopefully, you now have a clear path to follow. By checking these seven areas and exploring the advanced features, you are tackling the most common SEO issues that hold websites back.
Knowing how to use Screaming Frog to improve on page SEO is a skill that will pay off for years to come. Start with your first crawl and pick one or two of these SEO issues to fix this week. This is a powerful step towards getting more traffic and growing your business.
As you clean things up, you are making your website better for your customers and for search engines. Regular use of the Screaming Frog SEO spider will become a central part of your digital marketing strategy. If this is a bit much for you, you can always hire a SEO company to take your digital marketing to the next level.
