Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Hreflang tags and canonical tags - might be causing indexing and duplicate content issues
-
Hi,
Let's say I have a site located at https://www.example.com, and also have subdirectories setup for different languages. For example:
https://www.example.com/es_ES/
https://www.example.com/fr_FR/
https://www.example.com/it_IT/
My Spanish version currently has the following hreflang tags and canonical tag implemented:
My robots.txt file is blocking all of my language subdirectories. For example:
User-agent:*
Disallow: /es_ES/
Disallow: /fr_FR/
Disallow: /it_IT/
This setup doesn't seem right. I don't think I should be blocking the language-specific subdirectories via robots.txt
What are your thoughts?
Does my hreflang tag and canonical tag implementation look correct to you? Should I be doing this differently?
I would greatly appreciate your feedback and/or suggestions.
-
Hi... I'm sorry to tell you that the answer offered by Gaston is not totally correct.
So, in your Spanish page you have these hreflang and canonical annotations:
This is not correct because you are not adding also the self-referential hreflang annotation
Google is very precise about this, and it states its need in the help pages as well in many Googlers tweets and webmaster office hangouts.
The rel="canonical" is correct. Remember that the self-referential and the alternative href URLs must always be canonicals.
Finally, regarding the subfolders blocked via robots.txt, yes! that's totally incorrect:
if you're blocking Googlebot from accessing the Spanish, French and Italian subfolders, then Googlebot won't be able to parse the code of their pages, hence it won't be able to see also the hreflang annotations... with obvious erroneous consequences.
-
that's corect.
-
Yes. example.com/en-US/ redirects 301 to example.com
So, when referencing that version in hreflang will it look like this?
Is this correct?
-
So, every page of example.com/en-US/ redirects 301 to example.com/ ?
If that's the case, then there is no reason in having that folder (/en-US/), just when configuring Hreflang for en-US use the URL without that folder
-
What do you mean by "exactly the same then there should not be 2 sites"?
My https://www.example.com/en-US/ 301 redirects to https://www.example.com
Thoughts?
-
Yeap, those implementations are correct.
in order to avoid duplicate content between different language/countries websites, in each page that is present on each site, there should be their corresponding hreflang tag.
In your case:
And for a sample page: https://www.smarttechMKT.com/es-ES/gastonriera-espanolNOTE, in the case that site.com and site.com/en-us/ are exactly the same then there should not be 2 sites. Just the one without the folder and hreflang tag with en-US should point to that
Hope it helps.
Best luck.
GR -
Thank you for the response, Gaston! I really appreciate it.
So you are certain that my hreflang and canonical tags are implemented correctly? For example, my Spanish version (https://www.example.com/es_ES/

Is this implementation correct?
Also, will I have any duplicate content issues with these different language versions?
-
Hello there,
Watch out your question, there is a site there. If its your clients, edit it.
Canonical and hreflang seems OK.
Blocking other languages/countries is wrong. There you are allowing google to see ONLY the us version.For further reading, take a look at these articles:
Hreflang:
Multi-regional and multilingual sites - Google Search Console
International checklist - Moz Blog
Using the correct hreglang tag - Moz Blog
Guide to international website expansion - Moz Blog
Tool for checking hreflang anotations - Moz Blog Canonicals:
SEO Best Practices for Canonical URLs + the Rel=Canonical Tag - Whiteboard Friday Consolidate duplicate URLs - Google Search Console HelpHope it helps.
Best Luck.
GR.Hope it helps.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
"Duplicate without user-selected canonical” - impact to SERPs
Hello, we are facing some issues on our project and we would like to get some advice. Scenario
International SEO | | Alex_Pisa
We run several websites (www.brandName.com, www.brandName.be, www.brandName.ch, etc..) all in French language . All sites have nearly the same content & structure, only minor text (some headings and phone numbers due to different countries are different). There are many good quality pages, but again they are the same over all domains. Goal
We want local domains (be, ch, fr, etc.) to appear in SERPs and also comply with Google policy of local language variants and/or canonical links. Current solution
Currently we don’t use canonicals, instead we use rel="alternate" hreflang="x-default": <link rel="alternate" hreflang="fr-BE" href="https://www.brandName.be/" /> <link rel="alternate" hreflang="fr-CA" href="https://www.brandName.ca/" /> <link rel="alternate" hreflang="fr-CH" href="https://www.brandName.ch/" /> <link rel="alternate" hreflang="fr-FR" href="https://www.brandName.fr/" /> <link rel="alternate" hreflang="fr-LU" href="https://www.brandName.lu/" /> <link rel="alternate" hreflang="x-default" href="https://www.brandName.com/" /> Issue
After Googlebot crawled the websites we see lot of “Duplicate without user-selected canonical” in Coverage/Excluded report (Google Search Console) for most domains. When we inspect some of those URLs we can see Google has decided that canonical URL points to (example): User-declared canonical: None
Google-selected canonical: …same page, but on a different domain Strange is that even those URLs are on Google and can be found in SERPs. Obviously Google doesn’t know what to make of it. We noticed many websites in the same scenario use a self-referencing approach which is not really “kosher” - we are afraid if we use the same approach we can get penalized by Google. Question: What do you suggest to fix the “Duplicate without user-selected canonical” in our scenario? Any suggestions/ideas appreciated, thanks. Regards.0 -
Using same URL for both "en" and "en-us" hreflang tags
Hi,I have a question. Is it okay if I use the same URL for both "en" and "en-us" hreflang tags? For example, for my en-us page: Is this okay with Google? What are your thoughts on this?
International SEO | | Avid_Demand0 -
How do hreflang attributes affect ranking?
We have a site in English. We are considering translating the site into Dutch. If we use a hreflang attribute does that mean we have to create a duplicate page in Dutch for each English page, or does Google auto-translate? How would duplicate pages, even if they are in a different language, affect ranking?
International SEO | | Substance-create0 -
HELP: Incorrect Meta Tag description showing for the wrong search results
Hi Guys I'm stuck here! I have update the hreftags, updated the sitemaps. I have 3 top level domains and my zenory.com site is showing for the home page the wrong meta tag description, as you can see in the attachement the meta tag is showing the new zealand site meta tag description which is for zenory.co.nz Anyone know what might be going on here? I have also fetched the home page through WMT as well and its still returning the same results any advice would be much appreciated! Thanks
International SEO | | edward-may0 -
Is there any reason to get a massive decrease on indexed pages?
Hi, I'm helping on SEO for a big e-commerce in LatAm and one thing we've experienced during the last months is that our search traffic had reduced and the indexed pages had decreased in a terrible way. The site had over 2 Million indexed pages (which was way too much, since we believe that around 10k would be more than enough to hold the over 6K SKUs) but now this number has decreased to less than 3K in less than 2 months. I've also noticed that most of the results in which the site is still appearing are .pdf or .doc files but not actual content on the website. I've checked the following: Robots (there is no block, you can see that on the image as well) Webmaster Tools Penalties Duplicated content I don't know where else to look for. Can anyone help? Thanks in advance! cpLwX1X
International SEO | | mat-relevance0 -
What is the proper way to setup hreflang tags on my English and Spanish site?
I have a full English website at http://www.example.com and I have a Spanish version of the website at http://spanish.example.com but only about half of the English pages were translated and exist on the Spanish site. Should I just add a sitemap to both sites with hreflang tags that point to the correct version of the page? Is this a proper way to set this up? I was going to repeat this same process for all of the applicable URLs that exist on both versions of the website (English and Spanish). Is it okay to have hreflang="es" or do I need to have a country code attached as well? There are many Spanish speaking countries and I don't know if I need to list them all out. For example hreflang="es-bo" (Bolivia), hreflang="es-cl" (Chile), hreflang="es-co" (Columbia), etc... Sitemap example for English website URL:
International SEO | | peteboyd
<url><loc>http://www.example.com/</loc></url> Sitemap example for Spanish website URL:
<url><loc>http://spanish.example.com/</loc></url> Thanks in advance for your feedback and help!0 -
Showing different content according to different geo-locations on same URL
We would like our website to show different content according to different Geo-locations (but in the same language). For example, if www.mywebsite.com is accessed from the US, it would show text (in English) appealing to North Americans, but, if accessed from Japan, it would show text (also in English) that appeals more to Japanese people. In the Middle East, we would like the website to show different images than those shown in the US and Asia. Our main concern is that we would like to keep the same URL. How will Google index these pages? Will it index the www.mywebsite.com (Japan version) in its Asia archives and the www.mywebsite.com (US version) in its North American archives? Will Google penalise us for showing different content across Geo-locations on the same URL? What if a URL is meant to show content only in Japan? Are there any other issues that we should be looking out for? Kindest Regards L.B.
International SEO | | seoec0 -
Massive jump in pages indexed (and I do mean massive)
Hello mozzers, I have been working in SEO for a number of years but never seen anything like a jump in pages indexed of this proportion (image is from the Index Status report in Google Webmaster Tools: http://i.imgur.com/79mW6Jl.png Has anyone has ever seen anything like this?
International SEO | | Lina-iWeb
Anyone have an idea about what happened? One thing that sprung to mind might be that the same pages are now getting indexed in several more google country sites (e.g. google.ca, google.co.uk, google.es, google.com.mx) but I don't know if the Index Status report in WMT works like that. A few notes to explain the context: It's an eCommerce website with service pages and around 9 different pages listing products. The site is small - only around 100 pages across three languages 1.5 months ago we migrated from three language subdomains to a single sub-domain with language directories. Before and after the migration I used hreflang tags across the board. We saw about 50% uplift in traffic from unbranded organic terms after the migration (although on day one it was more like +300%), especially from more language diversity. I had an issue where the 'sort' links on the product tables were giving rise to thousands of pages of duplicate content, although I had used the URL parameter handling to communicate to Google that these were not significantly different and only to index the representative URL. About 2 weeks ago I blocked them using the robots.txt (Disallow: *?sort). I never felt these were doing us too much harm in reality although many of them are indexed and can be found with a site:xxx.com search. At the same time as adding *?sort to the robots.txt, I made an hreflang sitemap for each language, and linked to them from an index sitemap and added these to WMT. I added some country specific alternate URLs as well as language just to see if I started getting more traffic from those countries (e.g. xxx.com/es/ for Spanish, xxx.com/es/ for Spain, xxx.xom/es/ for Mexico etc). I dodn't seem to get any benefit from this. Webmaster tools profile is for a URL that is the root domain xxx.com. We have a lot of other subdomains, including a blog that is far bigger than our main site. But looking at the Search Queries report, all the pages listed are on the core website so I don't think it is the blog pages etc. I have seen a couple of good days in terms of unbranded organic search referrals - no spike or drop off but a couple of good days in keeping with recent improvements in these kinds of referrals. We have some software mirror sub domains that are duplicated across two website: xxx.mirror.xxx.com and xxx.mirror.xxx.ca. Many of these don't even have sections and Google seemed to be handling the duplication, always preferring to show the .com URL despite no cross-site canonicals in place. Very interesting, I'm sure you will agree! THANKS FOR READING! 79mW6Jl.png0