The All in One SEO WordPress plugin is a very popular plugin. Many install it so that they have the ability to customize their post’s page title. This is very useful to do from the SEO perspective. Your post title should be written for your reader and be catchy, your page title for your post should be more oriented towards keywords.
But the plugin has other settings that require attention. Out of the box the plugin configuration will noindex your category pages but not your tag pages. This isn’t necessarily what you want and I’ve had to fix a couple of blogs that had the default settings.
What does noindex do? It’s a special meta tag placed into your source code as an instruction to the search engine spider. To be accurate the tag created by the plugin is actually a noindex,follow. The follow bit is important, you want that.
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
noindex,follow tells the search engine spider to not place the page into it’s index (which feeds the search engine results pages) but to go ahead and crawl and follow any links on the page.
Why on earth would you not want your category (and tags, archive pages) in the search engine’s index? The more the better right? Well, no, not really. Many category, tags and archive pages just repeat all or part of the content that is already on the post pages. To Google and the other search engines, this is content that is duplicated and not interesting. Duplicate content within a single domain is a bad thing. It’s better to have 10 pages of unique content for Google to rank rather than 30 pages of duplicated content.
To add insult to injury, the way people use tags makes it worse. The ease of tagging causes many tags to be created (I’m just as guilty – just check out my tag cloud). I’ve seen blogs where no tag was ever reused, people treat them almost like a shortened form of a title and will tag each post with 4 or 5 unique words. The problem with this is that this creates one page per tag with one post on it, repeated 4 or 5 times. If your blog has links to your tag pages (ie. through a tag cloud or shows them on your posts) and you left them indexed then the poor spider has to crawl through all these duplicates (/tag/banana … /tag/bread … /tag/recipe .. get the idea?). It will just give up and go away.
With one exception my recommendation is to noindex all three aggregate pages: category, tags and archives.
The exception is if you have written unique manual excerpts for your posts (they are different than your your main post) AND the excerpt is shown (many themes don’t show them). Then I would consider indexing your category pages (as they tend to be fewer of those than tags). At this point you should do some keyword research and use that to name your categories.
For a while I was hesitant about noindexing all three, it seems wrong to not give the spiders a “bridge” page to add context to your posts, but after some study, I checked all three boxes a while back. I’ve given some thought about implementing excerpts for this blog and turning off noindexing for categories. But I’m lazy, and I might just build some keyword rich static pages, which will probably rank better anyway.