No Index PDFs

[MonicaOConnor]

Our products have about 4 PDFs a piece, which really inflates our indexed pages. I was wondering if I could add REL=No Index to the PDF's URL? All of the files are on a file server, so they are embedded with links on our product pages. I know I could add a No Follow attribute, but I was wondering if any one knew if the No Index would work the same or if that is even possible. Thanks!

[MonicaOConnor]

The files aren't duplicate. I am familiar with using the XRobots tag. I was really just curious if my theory would work.

Thanks for all your input.

[Alick300]

Hi Monica,

I presume you already check all the options before posting this question. I have concluded this by seeing your others posts/reply in this community.

Now here is my answer

To prevent your PDF file (or any non HTML file) from being listed in search results, the only way is to use the HTTP X-Robots-Tag response header, e.g.:

X-Robots-Tag: noindex

robots.txt does not prevent your page from being listed in search results.

What it does is stop the bot from crawling your page, but if a third party links to your PDF file from their website, your page will still be listed.

If you stop the bot from crawling your page using robots.txt, it will not have the chance to see the X-Robots-Tag: noindex response tag. Therefore, never ever ever disallow a page in robots.txt if you employ the X-Robots-Tag header.

I hope it helps but not very sure.

Thanks

[OlegKorneitchouk]

1. If you want to deindex all PDF files, I recommend using the x-robots-tag in .htaccess - https://yoast.com/x-robots-tag-play/

2. If the PDFs are pdf versions of existing pages, I would set canonicals to point to the URL you do want indexed (#2 on http://moz.rainyclouds.online/blog/htaccess-file-snippets-for-seos )

[DirkC]

If the pdf's are in a separate folder on your site - you could mark that folder as noindex in robots.txt

As far as I know, it's not possible to add a noindex to a link.

rgds

Dirk