Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear

-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://moz.rainyclouds.online/learn/seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Strange landing page in Google Analytics
Hello MOZ Community, The website in question is https://x-y.com/ When i looked at the landing pages report in GA , x-y.com is appended at the end of every URL like this. https://x-y.com/x-y.com When i open the above URL in GA interface, it shows page not found. This is obvious as there is no such URL.
Reporting & Analytics | | Johnroger
The metrics like sessions, Users, Bounce rate all look good. In the property settings, The default URL is written like this http:// cell-gate.com (Please note that s is missing in property settings). But how is traffic tracked correctly How do i solve this problem. What settings should we change to make the landing pages report look ok Thanks0 -
How to exclude traffic for a specific mobile device in Google Analytics view?
Hi, Need help on how to exclude traffic for a specific mobile device in Google Analytics view. I have been searching and the only information available is excluding IP address of internal traffic. Is there any way to exclude traffic through a mobile MAC address?
Reporting & Analytics | | Khadija_K0 -
% Change - Google analytics - how to calculate?
Hi All,
Reporting & Analytics | | JohnPalmer
I have two dates with two different numbers I want to calculate the "% Change" like google analytics, The numbers of June 2015 - 127,931 sessions
The numbers of June 2014 - 90914 sessions please tell me what is the %Change. Best. J0 -
Google Analytics Goal/Event/SOMETHING to show only Wordpress "Posts", not pages, etc
Hi all, Our site is build on Wordpress and formerly the post URL's had the typical date format at the beginning. This made it easy for me to look at, for example, all search traffic to the blog. I would just view URL's containing /2014/ and /2015/ and boom. We have since removed the dates from the URL's with proper redirects etc, which is great, but now I can't figure out a way to look at ONLY the blog in GA. I like to track a KPI of 'search visits to blog posts' and I can't figure out how to now. Can I set up a GA event that only fires when the post type template for blog posts loads? Some other solution? I'm lost here, and there's gotta be a good way to do it...
Reporting & Analytics | | 3DR0 -
Referral Traffic from Google
Hello, I have a question about my company's new website. I've worked in SEO and studied Google Analytics results for a few years now but have never really come across something like this. I started in this position in January of this year and when I started breaking down the traffic sources in Google Analytics, I noticed most of the traffic was coming from Google.com as a referral source. I had never seen Google.com as a referral source before so I looked into options for what it could be. It was not a paid ad and our organic traffic was coming through in Analytics, Before I could get any further, our new website was launched (we switched CRM's to WordPress) and the referral traffic from google went from 2,966 in January of 2015 to 22 in February 2015. for more comparison, in February of 2014, the referral traffic from Google was 2,496. I expected a drop when we switched CRM's but we correctly re-directed all pages and created a new sitemap and our organic traffic is up since the switch (not enough to cover drop in referral). I thought at first this had to do with our Google sellers account being de-activated when we made the switch, but I quickly fixed this over a month ago and no change. I'm wondering if anyone has ever seen Google.com come through as a referral source in Google Analytics and if they we're able to figure out what it actually was. This would be a great help! Thank you, Alex
Reporting & Analytics | | RASEO1 -
How can I track my rankings on Google Images?
I noticed a small amount of traffic coming from a particular very generic keyword. Being pleasantly surprised that we are ranking for this, and after some digging, I found that we are actually ranking in Google images, rather than in the web results. How can I track whether other keywords are ranking in Google images? I use Rank Checker to track keywords in the main web results, but this doesn't have a function for Google Images. Help please - thanks.
Reporting & Analytics | | TheJewelleryEd0 -
How can I see what Google sees when it crawls my page?
In other words, how can see the text and what not it sees from start to finish on each page. I know there was a site, but I can't remember it.
Reporting & Analytics | | tiffany11030 -
What is s.ytimg.com in google analytics?
My clients GA reports 273 visits from s.ytimg.com. I go to the site, it doesn't exist. I googled it, there were some code with s.ytimg.com in it, but nothing I could understand. Anybody have an idea where this comes from?
Reporting & Analytics | | endlessrange0