When it comes to Technical SEO it’s easily to get mired in the details. When I do a technical SEO site audit, it’s not uncommon for me to find 60 or more issues, which can be overwhelming and make it more challenging to know where to start. The thing is not every technical SEO issue is critical to fix. So the question is:
Which technical SEO issues are the most important?
While you can find many listicles of top technical SEO issues, zeroing on the one or two fixes that will really make a difference for your traffic is a matter of having both knowledge and experience. And what many articles miss, is that it’s not just the type of issue you need to consider, but also the scope.
Here are the key questions you can ask yourself to help determine the urgency of the issue.
How many of my pages does this issue impact? If your entire site is blocked so search bots can’t crawl it, that issue is going to be way more urgent to fix than if just a few of your pages are blocked.
Are my important pages at risk? For many websites, a small percentage of pages are responsible for the majority of organic traffic. Additionally you might have pages important for business reasons. If a technical SEO problem is impacting your best selling product then that issue is of outsized importance. If your top category pages are not linked into the site, that’s more important that some old blog posts that are orphaned (orphan pages are pages that are not linked to from any page on your website).
There are also certain classes of technical SEO problems that usually are more serious than others. Generally if the issue is a major impediment to Google and other search engines processing the page it’s more impactful issue than perhaps getting your tags (like your image alt tags) perfectly set.
Below I’ve organized the critical Technical SEO issues into 3 types. This ordering can be used as a checklist that is in roughly priority order.
Just remember that a problem with a larger scope (many more pages impacted) or an outsized business impact means the issue jumps up the priority ladder.
For some of the items, I’ve given you a tip on how to diagnose the issue in question.
3 Types of Critical Technical SEO Issues
The 3 Classes of Technical SEO issues that are important to fix are:
- Crawling (and Rendering) Problems
- Discovery Inefficiencies
- Indexing Problems
Crawling (and Rendering) Problems
Crawling is the first step to getting your website’s pages to show in the search results. Google sends “spiders” or “bots” to discover new and updated content on your site. It starts with pages it knows about, and then follows the links on those webpages to find new URLs. Google’s primary web crawler is known as Googlebot, and it has a close sibling, Googlebot-Smartphone which crawls the mobile version of your pages.
Google’s systems operate in a pipeline, crawling is first, then indexation is next. The Google index — a massive database of discovered URLs — powers the search results you see. If a page is not indexed, there is no chance of it ranking for a query. And if there is a problem with Googlebot crawling your page then it’s unlikely to be indexed. Not indexed means no traffic, so crawling issues are usually critical to address.
We’ll talk about Indexation Issues in the 3rd class of Technical SEO issues.
Here are some common crawling issues
Robots.txt Disallows
I’ve already mentioned a common cause of crawling problems, which is an improperly configured robots.txt file. This file can contain Disallow directives preventing search bots like Googlebot from crawling pages on your site.
If you see this in your robots.txt file, then your site isn’t being crawled at all.
User-agent: *
Disallow: /
In this case, User-agent: * means this section applies to all robots. Disallow: / tells the robot not to visit any pages on the site. By simply removing the “/” from the Disallow, you can unblock your site and allow search bots to crawl it. Amazing the difference one character makes.
You want to give crawlers access to your important pages as well as to their assets such as images, CSS, fonts and scripts.
Lack of Parity between Mobile and Desktop
Since most websites are Responsive these days I don’t see this very often but it is a must fix if encountered. Lack of parity means that the version shown to mobile devices is missing key content and links that is otherwise found on the desktop version.
Google crawls most (if not all by this point) sites mobile first, so incomplete information on your mobile site is a problem.
I often find this issue on inspection using Chrome Dev Tools to spoof different user agents, but comparing a crawl that crawled as Googlebot-Smartphone against one that crawled as Googlebot is a good way to go as well.
Poor Handling of Status Codes
When a browser or user agent requests a web page, the server sends back information that includes a status code known as the HTTP status code. This code informs the browser about the disposition of the page requested, such as whether the page is available (or not) or has moved. If the status code is incorrect this can cause SEO issues.
In particular sending a 404 (Not Found) HTTP status code will cause the bot to stop and not crawl the page at all. Other problems include (in order of importance):
- Providing a 200 code when the code should be a 404 and vice versa
- Having a redirect loop that leads the bot in a circle
- Sending a temporary redirect (302) when it should be a permanent redirect (301)
There are several browser extensions that will show you the status code of a URL that renders in your browser. Ayima Redirect Path is one such extension, you can also use httpstatus.io.
Less Efficient Crawling
Another problem that can impact crawling is a slow website. If your site is slow to respond, this can reduce how much Google crawls your website. Make sure your Time to First Byte (TTFB) metric is reasonable and not unduly slow. Large and unoptimized images can also slow down crawling.
Page Speed Insights and webpagetest.org are two of several tools that will give you TTFB timings for a web page.
I should point out that resolving Page Speed issues (which falls under the larger umbrella of Core Web Vitals) are usually not “needle movers” of organic traffic. But if the speed of your site is really bad, this could be an exception.
Rendering and JavaScript
Most modern websites use JavaScript to add functionality and interactivity to web pages. In the last couple of years Google has gotten quite good at crawling sites with client side rendering JavaScript, but there’s no doubt that depending on CSR adds complexity and potential issues with SEO.
Client Side rendering is JavaScript that is executed in the browser rather than on the server to create or enhance the web page the user sees. As good as Google with crawling rendered pages, it isn’t always perfect, and other search engines struggle even more.
When it encounters a JavaScript page, Google initially crawls the HTML that comes from the server and then sends the page along to the Web Rendering Service, a separate Google system that executes the JavaScript to render the page so that it can be processed. In this phase, the page with the executed JavaScript should be very similar to how the page looks in a browser like Chrome or Firefox.
Below is Google’s graphic that shows how these systems work together to process JavaScript pages.
Like with crawling issues, rendering challenges impact the downstream Google systems of indexing and serving (providing results for user’s queries). Some common challenges I’ve run into include:
- Not using a href tags for links and instead relying on JavaScript events such as onclick that search engines will not recognize as links
- Blocking JavaScript, CSS and other assets that the page uses in robots.txt
- Relying on JavaScript execution to populate meta tag information or links
- Relying on user actions such as scrolling to reveal important content
Bots do not scroll or click so everything that is needed (important content and links) should be present at page load and not dependent on user action.
In diagnosing these problems, the URL Inspection Tool in Google Search Console is helpful. Be sure to take a look at not only the screenshot but also the rendered HTML which I’m pleased to report that you can now search.
Is the content you see in your browser findable in the HTML provided by GSC?
You definitely don’t want to see a blank screenshot in Google Search Console! A blank screenshot indicates Google isn’t rendering the page properly.
Raw HTML from the server should have links, tags and content
In practice I’ve also found that if the raw HTML (sometimes called pre-rendered HTML) that the server sends over is empty (that is all content and links are dependent on JavaScript execution), that I’m not likely to get the SEO performance I’m looking for, so I recommend that the initial HTML from the server should have all the links defined with a href tags, the meta tags uniquely set and have at least the H1 and a paragraph or two of content.
A great resource for looking at the differences between raw and rendered HTML for a given web page is the Chrome extension: View Rendered Source. Some crawlers such as Screaming Frog will highlight JavaScript dependencies such as links that are only found when the page is fully rendered after JavaScript execution.
You can also view the raw HTML with Chrome Dev Tools, here are the steps:
- Bring up Chrome Dev Tools in the Chrome browser for the web page (I use Shift+Cmd+C on Mac , but you can try F12 or Control + Alt + I on Windows)
- Click on the Network Tab
- Refresh the page (circular arrow in Chrome)
- Scroll up to the top of the list and locate the initial HTML document (usually first) and click on it
- Click on the Preview tab
Tag Conflicts
Another potential problem area for Rendering issues is when tags such as the canonical and meta robots tags change after JavaScript execution and the page is rendered. The View Rendered Source Chrome extension I mentioned above can help you zero in on this issue.
It doesn’t matter if the rendered page is indexable, if the initial raw HTML has a meta robots tag set to noindex, Google won’t bother sending the page to the WRS (Rendering Service) and the page will never get indexed.
JavaScript SEO is a big topic, so if you want dig in more, this article from iPullRank is a good place to start.
Discovery Inefficiencies
Discovery inefficiencies are another type of crawling problem that occur when you make it difficult for the search bots to find your pages.
Site Structure and Orphan Pages
Typically a well organized site should feature your important pages prominently and have a shallow click depth. If your pages are too many clicks away from the home page, or worse, are not even linked to from the site (orphan pages), then that is a signal to the search engines that the page is not important and it will struggle to rank and sometimes not even be indexed.
To find orphan pages you can compare a list of crawled URLs with either the XML sitemap and/or with pages that appear in Google Analytics.
Internal Linking
Good internal linking is an underrated SEO superpower. Not only does it facilitate a good site structure, distributing your link equity across your pages, but the anchor text used in the links helps Google to determine the page’s relevance for similar search queries. Internal linking helps you cluster your pages into thematic topics, helping search engines understand the relevance of your pages to specific search queries,
In her brilliant talk on improving internal linking using machine learning, Lazarina Stoy shared the different aspects of internal linking that should be audited.
Internal linking tends to be more of a SEO performance enhancer than blocker, but if the problem is serious enough and affects important and many pages then it needs to be on the urgent list.
Sitemap
An XML sitemap is a file that lists the pages of a website and may provide additional information about each page to help search engines understand the site’s structure and content. By including the URL of your page in a XML sitemap, you increase the chances of your content being discovered, ranked, and displayed in search engine results pages (SERPs). The term “XML” refers to the format used to structure the sitemap data.
XML sitemaps are essential for SEO because they serve as a roadmap for search engines. You want to include only the pages of your website that are valuable for search. All URLs listed in a sitemap should be canonical (not canonicalized and indexable) and return a HTTP Status Code of 200.
Too many non canonical and non 200 URLs in your XML sitemap great diminishes its effectiveness and impedes discovery of your pages.
It’s also important that your newly created pages are added to your XML sitemap soon after they are created (automatically updated XML sitemaps are preferred). Out of date XML sitemaps are doing you no favors, I often find orphan pages in XML sitemaps that are old pages that we want to remove.
Indexation Issues
It takes a massive amount of resources to crawl and index the web, which continues to grow exponentially in size. At this point it is not possible for Google to index everything so increasingly, if Googlebot encounters a low value page, it just won’t index it.
One place to identify Indexation Issues is by reviewing the Google Search Console reports:
Indexation Issues can be caused by many technical SEO issues, but they can also be a symptom of content and on page issues as well. If your page has poor tagging (including a poor title tag) as well as not very helpful content then just fixing the technical SEO issues may not be enough.
Tackling technical SEO issues that results in poor Indexation is often a matter of using links and tags to guide Google to your key pages that are important for search while helping it ignore the pages that are of low value.
Duplicate and Thin content
When Google detects multiple pages with identical or very similar content, it may not index all of them. If your website uses URL parameters to generate different variations of a page, these pages are often similar enough that you’ll want to canonicalize them to a representative URL using the rel canonical tag. In other cases you’ll want to add more content to differentiate your pages more.
Pages with thin, low-quality, or insufficient content may not be considered valuable by search engines, leading to non-indexation. In larger sites it is not uncommon to unintentionally create thin pages that need to be addressed. In this case you should help Google out and add a meta robots noindex to the page. Note that noindexing poor pages that are in the Google Index (rather than not) is often more impactful.
Many crawlers will capture word count for the pages crawled, so one way to find thin pages is to look for pages with low word count.
Ideally you shouldn’t link to poor pages in the first page, keeping your site structure efficient and organizing it to feature your most important pages via your navigation and other links. If a page is of no value, and there are no quality backlinks to it, consider removing it.
It’s a myth that 404s are bad for SEO. If the page was not useful in first place, it is perfectly fine to remove it and have it return a 404 Not Found Status Code as long as there are no external and internal links to it.
Improper Tagging
We just talked about the canonical and noindex tags. Misuse of these tags can lead to indexation problems as well. One classic case that I have run into is the situation where all of a site’s pages are canonicalized to the home page.
This is the equivalent of telling Google that you want your home page to rank for all of your keywords, which won’t work very well. You don’t want to do this.
A Chrome extension that lets you easily see the meta robots and the canonical tag of a page is SEO Meta in 1 Click.
Prioritizing Your Technical SEO Issues
To prioritize your technical SEO issues, evaluate the extent of the impact each issue has on the overall website. The larger the impact, the higher up the priority list the issue should be.
If Google can not completely process your pages, for whatever reason, be it that it is blocked from crawling or because all your links are obfuscated by JavaScript; these issues are of highest urgency to fix.
A page that isn’t crawled properly typically doesn’t get indexed and certainly won’t rank.
That doesn’t mean that the other issues that I have listed above aren’t also important, it’s just that if your site’s pages can’t get past their crawling and rendering issues, the reality is that not much else matters.
Once your pages are crawlable and render properly, then issues like orphan pages, thin pages and internal linking are next on the list to evaluate. If these issues impact enough pages, or even just your most important pages, then they too will be an obstacle for your SEO performance.
