| Alert Properties | Alert Description |
|---|---|
| Alert Name | Page allowed by robots.txt but restricted by nofollow attribute |
| Code | P16 |
| Description | Web Pages found with links allowed by robots.txt for search engines to crawl but restricted by nofollow attribute in tags |
| Level | Warning |
If you want search engines like Google not to crawl links or content present in a page, the nofollow meta tag is quite useful for the purpose. But this attribute will also guide search engines to unfollow internal links present on the page, which ruins SEO efforts. If you want to notify search engines to not follow or crawl specific links or page use, then refrain from using nofollow attributes on <a> tag. Instead, use the robots.txt Disallow rule to disallow search engines from following a link to a page on your own site.
robots.txt
User-agent: *
Disallow: /products/
Learn more about nofollow attribute and its proper use from these articles here:
Qualify your outbound links to Google
Create a robots.txt file
Be careful who you link to
Evolving “nofollow” – new ways to identify the nature of links