Guides, insights and how-tos on technical seo from The Cogent team.
What is an XML sitemap in SEO? An XML sitemap is a file that lists the pages, videos, images and other content on your website so that search engines can more easily crawl your site. While a sitemap will help to discover your…
22 min readWhat is robots.txt in SEO? It's a plain text file at the root of your website that tells search engine crawlers which URLs they may crawl and which…
13 min read“Crawled – currently not indexed” means Google has visited and read your page but decided, for now, not to add it to the search index. It is not a…
9 min readRobots.txt or noindex: Which one actually keeps a page out of Google? The honest answer is that they do two different jobs, and mixing them up is one…
8 min read
