Google may expand its unsupported robots.txt rules list using HTTP Archive data and could broaden how it handles common misspellings of disallow.
The DOJ Antitrust memorandum opinion from September 2025 mentions clicks as a “raw signal” that Google uses. It also ...