Robots.txt or noindex: Which one actually keeps a page out of Google? The honest answer is that they do two different jobs, and mixing them up is one of the…