Google wants to replace the link attribute Nofollow (rel = “nofollow”) for the purpose of concretization partly by the new attributes Sponsored and UGC. As part of the presentation of the new attributes of the search engine giant made clear that the setting of Nofollow future would no longer be understood as an instruction to the crawler, but only as a hint.
No binding Meta-Tag Nofollow anymore
Now, Gary Illyes from Google’s Webmaster Trends team adds that the new interpretation is also a meta-tag for Nofollow. This is a change in the way things have gone before, as Google has always conveyed in its official documents that a nofollow meta tag would discourage the crawler from indexing the appropriately flagged pages. Yes, however, makes it very clear that even setting the meta-tag in the future will only be considered a hint. The same should apply to the meta-tag Noindex. This also does not prevent crawling, but merely ensures that content marked in this way does not appear in search results. Here it is important to note the difference between indexing and crawling. Namely, while Google had always officially excluded Nofollow’s crawl, Noindex only excluded indexing. So Google has an externally visible, searchable page pool and another, known only internally to the machine. This makes sense from the point of view of the search engine because the calculation of the relevance of contributions can be run over both areas in order to arrive at a ranking decision. Webmasters who really want to exclude that Google not only does not index certain content and page areas but does not even crawl it will only be able to access robots.txt in the root directory of the respective website.