How to Find Broken Internal Links Before They Multiply
What a broken internal link is
An internal link is a link from one page on your site to another page on the same site. A broken internal link is one where the destination page no longer returns a 200 OK status code. The page it points to might return a 404 Not Found, a 410 Gone, or some other error response.
The link itself is still there in your HTML. It looks fine in the source code. The problem is invisible until someone clicks it or a crawler follows it.
Why they multiply
Broken internal links don't usually appear all at once. They accumulate. A page gets deleted without a redirect. A URL structure changes during a site migration. A CMS auto-generates slugs differently after an update. Each of these events can create multiple broken links simultaneously, and none of them trigger any visible warning on the live site.
Over time, a site that started with zero broken links can accumulate dozens without anyone noticing. The links were working when they were created. The pages they point to changed later.
How to find them with Screaming Frog
Open Screaming Frog and enter your homepage URL in the address bar at the top. Click Start. The tool will begin following every link it can find on your site, recording the HTTP status code it receives from each URL.
When the crawl finishes, click the Response Codes tab. Filter by 4xx. Every URL shown here returned a client error — most commonly 404 Not Found. These are your broken pages.
Now you need to know which pages link to each broken URL. Click on any 404 URL in the list. At the bottom of the screen, click the Inlinks tab. This shows you every page on your site that contains a link to the broken URL. These are the pages you need to update.
What to do with what you find
For each broken URL, you have two options. If the content that was at that URL still exists somewhere on your site under a different URL, set up a 301 redirect from the old URL to the new one. If the content no longer exists and there's no reasonable replacement, update the internal links that point to it to point somewhere else — or remove those links entirely.
Export the full list of 404 URLs and their inlinks before you start making changes. This gives you a record of what you fixed and lets you verify that each fix worked.
How often to check
Running a crawl after any significant content operation is a reasonable practice. After a site migration, after deleting a batch of old pages, after changing your URL structure. For sites that update frequently, a monthly crawl catches problems before they accumulate into a larger cleanup project.
What Orphaned Pages Are and Why They Represent Wasted Effort
The definition
An orphaned page is a page that exists on your site but has no internal links pointing to it from other pages on the same site. It's reachable if you know the exact URL. But nobody navigating through your site will ever find it, and search engine crawlers that follow links to discover content won't find it either.
How pages become orphaned
The most common cause is navigation restructuring. A page was linked from a section of the site that got reorganized. The links in that section were updated to point to newer content, but the old page wasn't deleted. It still exists. It just has no path to it.
Another common cause is publishing without linking. A page gets created and published, but the author forgets to add a contextual link from a related page, or assumes the sitemap submission will be enough. The sitemap might get the page indexed once, but without internal links, the page accumulates almost no internal authority and rarely maintains ranking.
Campaign landing pages are frequently orphaned after a campaign ends. The page was created for a specific purpose, traffic was driven to it directly, and when the campaign ended, the links were removed. The page stayed.
Why it matters
Internal links serve two purposes. They help users navigate. They also distribute what's sometimes called link equity or PageRank through the site's internal link graph. A page with no internal links pointing to it receives none of this distributed authority. It starts every crawl at a disadvantage.
Beyond authority, orphaned pages represent content investment that isn't paying off. Someone wrote that content. Someone edited it. The time spent creating it is wasted if the page is invisible to both users and crawlers.
Finding orphaned pages
This is one area where Screaming Frog requires a bit of extra setup. The tool can tell you which pages it discovered through link-following, but it can't tell you about pages it didn't discover — because it didn't find them.
To find orphaned pages, you need to give Screaming Frog a list of all your URLs. You can do this by uploading your XML sitemap. Go to Mode, select List, and upload your sitemap file. Now Screaming Frog will check every URL in the sitemap rather than only the ones it discovers through crawling.
After the crawl, go to the Crawl Analysis tab and look for pages with zero inlinks. These are your orphaned pages.
What to do
For each orphaned page, decide whether it should exist at all. If the content is outdated or irrelevant, delete it and set up a redirect or return a 410 Gone. If the content is worth keeping, add internal links to it from related pages. A contextual link from a related article is worth far more than a link buried in a footer or sitemap.
How Redirect Chains Slow Down Crawling Without Anyone Noticing
What a redirect chain is
A redirect chain occurs when following a URL requires passing through more than one redirect before reaching the final destination. URL A redirects to URL B. URL B redirects to URL C. URL C is the actual page. That's a chain of two redirects, which is already worth fixing. Chains of three, four, or more are not uncommon on sites that have been through multiple migrations or domain changes.
Why they happen
Redirect chains are almost always unintentional. They build up layer by layer over time. A site migration creates a set of redirects from old URLs to new ones. Later, another URL restructure creates a new set of redirects. The new redirects point to the new URLs, but the old redirects still point to the intermediate URLs rather than the final destinations. Nobody went back to update the original redirects.
Domain changes are another major source. A site moves from HTTP to HTTPS, creating one layer of redirects. Later it moves from www to non-www, creating another. Later it moves to a completely new domain. Each migration adds a layer.
Why it matters for crawling
Search engine crawlers have a crawl budget. They allocate a certain amount of time and resources to crawling each site. When a crawler follows a redirect chain, it spends more time per URL than it would for a direct response. On a large site with many redirect chains, this can meaningfully reduce how much of the site gets crawled in each session.
There's also a performance impact for real users. Each redirect in a chain adds a round trip between the browser and the server. A chain of three redirects adds three round trips before the final page even starts loading.
Finding redirect chains with Screaming Frog
Run a standard crawl. When it finishes, go to the Response Codes tab and filter by 3xx. Look at the Redirect To column for each URL. If the destination of a redirect is itself a redirect, you have a chain.
Screaming Frog also has a built-in redirect chain report. Go to Reports in the menu and select Redirect Chains. This gives you a direct list of all chains detected during the crawl, along with the full path from start to end.
Fixing redirect chains
The fix is straightforward in concept: update each redirect so it points directly to the final destination rather than to an intermediate URL. If A redirects to B and B redirects to C, update A's redirect to point directly to C. Then verify that C actually returns a 200 OK.
After fixing chains, update any internal links that point to the intermediate URLs. Internal links should always point to the final, canonical URL of a page rather than to a URL that redirects to it.
Why Your XML Sitemap Probably Lists Pages That Return Errors
What an XML sitemap is supposed to do
An XML sitemap is a file that lists the URLs you want search engines to know about and consider for indexing. It's a communication tool. You're telling crawlers: these are the pages that exist and are worth visiting. That communication only works if the URLs in the sitemap actually return valid, indexable content.
Why sitemaps accumulate errors
Most sitemaps are generated automatically by CMS plugins or platform tools. They include every published URL at the time of generation. The problem is that sitemaps don't automatically update when pages are deleted, when URLs change, or when pages are set to noindex.
A page gets deleted. The sitemap still lists it. The URL now returns a 404. A page gets moved to a new URL. The old URL is in the sitemap and returns a 301 redirect. A page gets a noindex tag added. It's still in the sitemap, which is a direct contradiction — you're telling crawlers to visit this URL in the sitemap and also telling them not to index it in the page's meta tag.
What each type of sitemap error means
A sitemap URL returning a 404 means the page was deleted or moved without the sitemap being updated. A sitemap URL returning a 301 means the URL has been redirected to a different URL — the sitemap should list the final destination URL instead. A sitemap URL with a noindex tag means you're submitting a page for indexing that you've also told crawlers not to index. A sitemap URL returning a 500 error means the page has a server error that needs to be fixed regardless of the sitemap.
Auditing your sitemap
In Screaming Frog, go to Mode and select List. Upload your sitemap file. Run the crawl. Every URL in your sitemap will be checked and its status code recorded. Export the results and filter for anything that isn't a 200 OK response.
Cross-reference this list with your sitemap file. Remove URLs that return 404 or 410. Update URLs that return 301 redirects to point to the final destination URL. Remove URLs that have noindex tags.
After fixing the sitemap
Resubmit your sitemap through Google Search Console after making changes. This signals to Google that the sitemap has been updated and prompts a re-fetch. Monitor the Coverage report in Search Console over the following weeks to see if the error count decreases.
How to Run a Complete Technical Health Check in One Afternoon with Screaming Frog's Free Version
Before you start
Download Screaming Frog SEO Spider from screamingfrog.co.uk. The free version is fully functional up to 500 URLs per crawl. You don't need to create an account or enter payment details to use it. Install it on your computer — it runs locally, not in a browser.
Have a spreadsheet ready. You'll be exporting data from several different reports and you want somewhere to track what you find and what you've fixed.
The crawl setup
Open Screaming Frog. In the address bar at the top, enter the homepage URL of the site you want to audit. Before clicking Start, go to Configuration and check a few settings. Make sure "Follow Internal Nofollow Links" is checked if you want to see all internal links regardless of nofollow attributes. Make sure "Check External Links" is checked if you want to verify that external links on your site are working too.
Running the crawl and reading the results
Click Start. Watch the URL count increase as the crawler discovers and visits pages. When it stops, you have a complete picture of every URL the crawler could reach from your homepage through internal links.
Start with the Response Codes tab. This is the most important view for a technical health check. Filter by 4xx to see broken pages. Filter by 3xx to see redirects. Note the count of each before you start fixing anything.
Move to the Page Titles tab. Look for pages with missing titles, duplicate titles, or titles that are too long or too short. These are quick wins that don't require any technical work to fix.
Check the Meta Description tab with the same approach. Look for missing or duplicate descriptions.
Go to the H1 tab. Every page should have exactly one H1. Look for pages with missing H1s or multiple H1s.
The redirect chain report
Go to Reports in the top menu. Select Redirect Chains. This report shows every chain of redirects that the crawler encountered. Export it and work through each chain, updating redirects to point directly to the final destination.
The sitemap audit
After completing the standard crawl, switch to List mode. Go to Mode and select List. Click Upload and select your sitemap file. Run a second crawl. This crawl checks every URL in your sitemap rather than only the ones discoverable through link-following. Compare the results to your standard crawl to identify orphaned pages — URLs that appear in the sitemap but weren't discovered through internal links.
Prioritizing what to fix
Not everything needs to be fixed today. Broken internal links on high-traffic pages are the highest priority. Redirect chains longer than two hops are worth fixing soon. Sitemap errors are worth cleaning up in a dedicated session. Orphaned pages can be addressed systematically over time.
Document everything. A technical audit that produces a spreadsheet of findings and fixes is far more useful than one that produces a mental note to "fix some things later."