List of websites excluded from the Wayback Machine

From Archiveteam
Revision as of 13:48, 26 April 2019 by ATrescue (talk | contribs) (Added MediaFire.com at the bottom. Should be sorted into the list automatically by user:JAABot. + adding introduction.)
Jump to navigation Jump to search

There are two ways webmasters keep the Wayback Machine out of their website: Through robots.txt ia_archiver exclusion (“user-agent:ia_archiver disallow:/”) or through a manual exclusion request.

While the first, more common way of exclusion shows “This page cannot be crawled or displayed due to Robots.txt” when trying to access it through the Wayback Machine, the second way displays “This page has been excluded from the Wayback Machine”.