The All in One SEO WordPress plugin is a very popular plugin. Many install it so that they have the ability to customize their post’s page title. This is very useful to do from the SEO perspective. Your post title should be written for your reader and be catchy, your page title for your post should be more oriented towards keywords.
But the plugin has other settings that require attention. Out of the box the plugin configuration will noindex your category pages but not your tag pages. This isn’t necessarily what you want and I’ve had to fix a couple of blogs that had the default settings.
What does noindex do? It’s a special meta tag placed into your source code as an instruction to the search engine spider. To be accurate the tag created by the plugin is actually a noindex,follow. The follow bit is important, you want that.
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
noindex,follow tells the search engine spider to not place the page into it’s index (which feeds the search engine results pages) but to go ahead and crawl and follow any links on the page.
Why on earth would you not want your category (and tags, archive pages) in the search engine’s index? The more the better right? Well, no, not really. Many category, tags and archive pages just repeat all or part of the content that is already on the post pages. To Google and the other search engines, this is content that is duplicated and not interesting. Duplicate content within a single domain is a bad thing. It’s better to have 10 pages of unique content for Google to rank rather than 30 pages of duplicated content.
To add insult to injury, the way people use tags makes it worse. The ease of tagging causes many tags to be created (I’m just as guilty – just check out my tag cloud). I’ve seen blogs where no tag was ever reused, people treat them almost like a shortened form of a title and will tag each post with 4 or 5 unique words. The problem with this is that this creates one page per tag with one post on it, repeated 4 or 5 times. If your blog has links to your tag pages (ie. through a tag cloud or shows them on your posts) and you left them indexed then the poor spider has to crawl through all these duplicates (/tag/banana … /tag/bread … /tag/recipe .. get the idea?). It will just give up and go away.
With one exception my recommendation is to noindex all three aggregate pages: category, tags and archives.
The exception is if you have written unique manual excerpts for your posts (they are different than your your main post) AND the excerpt is shown (many themes don’t show them). Then I would consider indexing your category pages (as they tend to be fewer of those than tags). At this point you should do some keyword research and use that to name your categories.
For a while I was hesitant about noindexing all three, it seems wrong to not give the spiders a “bridge” page to add context to your posts, but after some study, I checked all three boxes a while back. I’ve given some thought about implementing excerpts for this blog and turning off noindexing for categories. But I’m lazy, and I might just build some keyword rich static pages, which will probably rank better anyway.
I am looking everywhere for a decent article which discusses the merits and results of what to index and what to noindex. This article was promising until I realised that you haven’t waited to see the results of noindexing all three?
What happened? Did your posts win more traffic? Did you change it back? At current all this tells the reader is that you decided to noindex all three, but not whether your site benefited from the change?
Any chance of an edit or follow up which updates us as to the positive or negative results of such a change?
Good question, yes, I should do a follow up to this post. I didn’t keep track of when I made the change exactly. Since I published this post traffic has been flat, but I know I changed it a few weeks prior. In the last 2 months my traffic has jumped up quite a bit but I wouldn’t attribute it to just this one change.
Good article.
I have a site where each post is an individual product, categorized by categories and tags. I hadn’t changed any of the default settings of the All in One SEO plugin until reading this article.
I have unchecked all noindex’s today so will keep an eye on this and provide some feedback if successful or not.
Thanks for the comment, I’ll be interested in your findings. I recently posted a summary of the changes/activities that has increased traffic to my blog. Noindex my tags was one of them.
Hi, I had the same issue and question for my website. As many guide suggested at the beginning I choose no index for category, tags and archive, but I was struggling to rank for certain combinations of keywords which I only could get by allowing index for category. I did it, and I am actually seeing, even if the change is recent, that organic traffic for that category pages is already growing considerably.
I know I am facing a duplicate content issue doing that, but I also though:
1) every website, when growing, has almost no chance to avoid 100% duplication content
2) if I consider duplication of content the content I show to user in category pages, then I should noindex the home page as well, because what I show ther is just a snippet of my latest 5 entries!
So, my opinion is, according to your site architecture, to pick ONE between category OR tags and allow indexation if you need certain content to be displayed in a certain way and crewled for ranking purposes.
My 2 cents,
lavoroinspagna.com
Definitely I would agree that you shouldn’t have both the tag and categories pages indexed.
Maybe I don’t get it. Isn’t that the reason that the plugin has the option “Canonical URLs”?
It could be an alternative. What would you set the canonical tag to? The home page? Keep in mind that the canonical tag is a hint (although Google definitely heeds it, Bing less so) not a directive like noindex so it’s a less strong signal.
I want to know when category pages and tags created duplicates and you say they should not be indexed why then all the big news sites and popular blogs index all of their tags?
I can name http://technorati.com/blogs/top100 as an example, why their ranking in Google is good, is it useful for them too?
Yours Sincerely
The one exception is when that page byitself is considered a viable resource for users. A top 100 blog listing page on technorati.com fits that criteria and undoubtedly has a lot of links to it so yes it should be left as indexed. If you put some work into jazzing up your category pages and make them viable landing pages then keep them indexed. Thanks for the question it’s a good one..!
Hi
Thanks for answering my previous question
I have my new WordPress tags indexed and removed them from Google index.
But with all the tags off my Google rate is 300 IP which was 3000 IP before.
I wanted to know whether this is normal and my visits from Google would increase again or what?
My site is
http://www.apam.ir
Best regards
I’m not sure what you mean by “IP” here. Your Google traffic might increase, it might not, unfortunately there are so many other factors involved.
ip = visitors
hi – i trye test on my website – noindex and follow for tags, meta, rss comment , subages homepage – where it’s duplicate – i have only one website with hard trafic from g – – soo i write there what’s hange be with trafic – sorry my english it’s not good =o/
Hi, site architecture is just one part of SEO. You have to write posts/pages with keywords that match what the searcher is looking for – hopefully with keywords that aren’t that competitive. And you need to promote via social media and linkbuilding. Hope that helps.
hey – thanks for the reply – says the translator for this may appear słowinctwie problems – so after a few days I noticed that the drop site – nromalne – but also decreased the rate of visits so I gave index and follow – which tests for – the latest update animal you probably know what – I fell in the results and it’s very / first on the long tail key phrases then, I’m not spamming my own facilities, but also the difficult themes presented in my country (finance) have changed as far struktóre inbound links – I’m waiting for further results: (movement lost the noindex back
Nice explanation and advice.
many thanks for the article
best explanation for noindex option in All in one SEO Pack
i had traffic drop last month , then check the plugin , i saw that options that i didnt know what they do.
now i read your article and check three box : Use noindex for Tag Archives , archive , categories.
hope it affect on rising traffic.
thank you
Thanks! I was not sure about no-indexing general informational and navigation type pages. I will now do this on our site. Good luck in 2017.
Which is better index or nonindex…if i am using nonindex showing error in search engine console
like Submitted URL marked ‘noindex’
Index the pages that you think are ideal for searchers to land on. Noindex the pages that are duplicate, thin or otherwise not good for searchers to see as the first page of your site. The new Google Search Console will show you pages that are not indexed because of the noindex tag. This is not an error but rather information for you to evaluate whether it is appropriate that these pages are not in the Google index.
Hello Kathy,
I must say I like the simplicity of this post. Your writing style makes it easy to understand the subject at hand.
I’ve been reading on best practices regarding noindex and nofollow in general, and I’m still somewhat torn about not indexing categories, but, you make a very good point on making them a valuable landing place, so I believe I might do that with keywords.
Keep up the good info!
Yes, if you are willing to put some SEO love into your category pages then leave them indexed. Thanks for your kind words on my writing style!
Good Article but still ,my issue is not fixed. I have a wordpress theme, my problem is that, my home page is indexing but other of my pages are not indexing and they are showing “NoIndex, Follow” tag in the header. I don’t now how to fix it, Please Help me to solve out this.
Ok I looked and on your website design the tag is set to index.
how do i category page index in to wordpress platform?
It depends on the plugin (and some cases theme) you are using. Look for settings that control the indexation of the page types.
A lot of tags on my website are indexed and when i search for my website link on Google, i see all tags no articles. I just noindexed my tags after reading your article.
Another master’s content about Pinterest SEO.
Thank You.