Frequently Asked Questions

From Archiveteam
Jump to: navigation, search

How can I help?

See Who We Are, Deathwatch, and Category:Projects_status. These pages describe our projects and the things you can do to help.

Is the Archive Team affiliated with the Internet Archive (

No. A few members are affiliated, but majority of Archive Team members are volunteers who help while not busy at work or school.

Why is ArchiveTeam crawling my site / disrespecting robots.txt?

A detailed manifesto is located at Robots.txt. Please read it first and contact us through IRC (described in a answer below) before making harsh actions. We cooperate!

If you notice the crawler's user-agent is "ArchiveBot", please see ArchiveBot.

How should I go about backing things up?

What would you like to back up? If you want to mirror/backup a website, the de facto tool is Wget (but there's lots more, see Software!). WARC files are highly recommended as they can be ingested by the Wayback Machine.

If you want to back up your personal files, "List of backup software" at Wikipedia is an extensive list of backup software. See Backup Tips as well!

Where do all the saved files go?

Files are ultimately uploaded to Internet Archive on the Archive Team collection. Archive Team relies on Internet Archive for storing the files.

What are these WARC files in the Internet Archive? How do I extract files from a WARC file?

WARC files are de facto medium of digital preservation of the web. These WARC files are ingested by the Wayback Machine. WARC files are not simple zip files; they're designed to record metadata.

There is a growing number of tools that can manipulate WARC files in The WARC Ecosystem.


halp pls halp

I uploaded a WARC file but why doesn't it show up in Wayback Machine?

To ensure content integrity, items with WARC files must have the mediatype set to "web" and be under the Archive Team collection in order for it to be ingested by the Wayback Machine.

I think there is a web site that's going to shut down / sun set / end its incredible journey. Can you save it?

Yes, do tell us a soon as possible! Don't just Tweet about it; do something. Get things done. Talk to us on IRC to let us know. There is also a /r/shutdown sub-Reddit.

For small websites, see ArchiveBot. For large websites, the Warrior may need to be deployed. Please take a look at Dev for learning how the Warrior works.

I saved/archived some stuff from a website / ftp server. Do you want it? Where should I upload it?

Great, yes, and! Tag your items with the subject keyword "archiveteam" and let us know so we can move it under the ArchiveTeam collection.

P.S. Creating an account on Internet Archive is free and should be the first thing that comes to mind for archive files. File hosting sites like Putfile, Megaupload, etc are not suitable for hosting archives!

I lost my stuff from Geocities/Tabblo/Posterous/some web host! Where can I get it back?

Try searching the wiki for a page about the specific website to find out more about what happened. Typically, there are several ways of recovering files from the Internet Archive:

  • A specially crafted username lookup page created by Archive Team
    • Allows you to search by your username and will present the relevant materials. Only a small set of projects have this feature.
  • The Internet Archive's Wayback Machine
    • This method is the easiest for most users but some web pages take months to show up in the Wayback Machine.
  • Individual WARC Files uploaded to the Internet Archive
    • This method is the most accurate but requires power user skills with working with WARC files. As well, WARC files produced by the Internet Archive are not publicly available (but the ones by Archive Team are always available).

For details, see Restoring.

I need help running the Warrior or scripts. I think it's broken.

See the FAQ in the Warrior wiki page.

We Are Not The Internet Archive

How do I upload something to the Internet Archive (

You can either use the HTML5/Flash uploader or use an alternative method.

Can someone remove or fix something on the Internet Archive?

Possibly. Keep in mind that majority of Archive Team are volunteers who are not affiliated with the Internet Archive and requests should go to staff instead. If strenuous circumstances arise, please see the question about contacting Archive Team below.

How do I get the original file on the Wayback Machine?

Add id_ after the date in the URL. For example:

How do I search the contents of the Wayback Machine?

You can't unfortunately. However, the Internet Archive provides API access (designed for programmers and power users) to the Wayback Machine and to the CDX database.

Why does the Wayback Machine follow robots.txt in a way that I don't like?

Because it makes the lawyers go away. We're not the Internet Archive. Don't ask us, ask them.

Can I upload $COPYRIGHTED_THING to the Internet Archive?

Although the Internet Archive prefers freely-redistributable content, they also accept still-in-copyright things. If there's a valid complaint/DMCA takedown request, they'll simply make the item private, but they will not delete the data. Having said that, the Internet Archive is not The Pirate Bay, so please don't treat it as such.

Can I save big files with the Save Now feature?

No, files larger than 200 MB will not be saved correctly.

How redundant is Archiveteam?

Is there a backup of the data on the website? If so where can I download it?

Two sets of backups of this wiki are available. There are backups done by the hosting provider (several, going back days and weeks as well as hours), which use the storage capability of the shared hosting to keep them automatically (no tape or disk backups being done as most people would think of them). There are similarly copies of the database kept going back months.

Additionally, an XML dump of the Mediawiki database (which can be imported into any MediaWiki software) is accessible at New backups are currently pushed out once a week (and will be increased if changes on the site require it). All images are also wrapped into a images.tar.gz file.[1] Our entire images directory is available at

Dumps of the ArchiveTeam Wiki are generated with WikiTeam tools and uploaded to the Internet Archive quite regularly. You can find them here and newer ones here.

Is there a mirror of the website?

There are no mirrors we know of, although we encourage our more paranoid or protective readers to maintain one based on the above dumps.

Who are y'all?

Does Archive Team have any social media accounts?

Follow us on Twitter: @archiveteam, @at_warrior, /r/archiveteam and like us on Facebook. These accounts are run by selected volunteers and may not be monitored for questions.

(There is a Google ArchiveTeam group but it is not used.)

Who's the administrator?

For a list of administrators, see Tracker#People which has a table at the bottom of the page.

I went through the wiki and I still have a question! How do I contact the Archive Team?

Join us on IRC! (All channels and info listed here.) For general inquiries, visit #archiveteam on EFNet. Email can be sent to

This wiki is not monitored for questions.

If a FAQ should appear here, please add it.


  1. Doesn't seem to be working any more.

Recommended ReadingFrequently Asked QuestionsMain Page

[view]  [edit]                   Archive Team                  
Current events Alive... OR ARE THEY · Deathwatch · Projects · Download available archives
Archiving projects · BetaArchive · Internet Archive · It Died · · · OSBetaArchive · TEXTFILES
The Dead, the Dying & The Damned · UK Web Archive · WebCite
Blogs/website hosts Angelfire · · Blogger · Blogster · · EtherPad · FortuneCity · Free ProHosting · Fuelmyblog · GeoCities (patch) · Google Business Sitebuilder · Google Sites · ISP Hosting · Jux · LiveJournal · My Opera · Open Diary · Ownlog · Posterous · · Proust · Roon · Splinder · Swipnet · Tripod · Verizon Mysite · Verizon Personal Web Space · Vox · Windows Live Spaces · · Xanga · Yahoo! Blog · Zapd
Cloud hosting/file sharing services AnyHub · Box · Dropbox · Google Drive · Google Groups Files · iCloud · MediaCrush · MediaFire · Mega · MegaUpload · MobileMe · OneDrive · RapidShare
Corporations Apple · IBM · Google · Microsoft · Yahoo!
Events Arab Spring · Occupy movement · Spanish Revolution
Font Repos Google Web Fonts · GNU FreeFont · Fontspace
Image hosting services · Camera+ · Cameroid · Flickr · Fotopedia · Geograph Britain and Ireland · ImageShack · Imgur · · Madden GIFERATOR · MLKSHK · Microsoft Clip Art · Nokia Memories · Odysee · Panoramio · Photobucket · Picasa · Picplz · Ptch · · · Snapjoy · TwitPic · Wallbase · Webshots · Wikimedia Commons
Knowledge/Wikis arXiv · Citizendium · Edit.This · Encyclopedia Dramatica · Everything2 · infoAnarchy · GeoNames · GNUPedia · Google Books · Insurgency Wiki · Knol · Nupedia · OpenCourseWare · OpenStreetMap · Orain · · Project Gutenberg · Puella Magi · Referata · SongMeanings · ShoutWiki · The Internet Movie Database · The Pirate Bay · TropicalWikis · Urban Dictionary · Webmonkey · Wikia · Wikidot · WikiHow · Wikkii · WikiLeaks · Wikipedia (Simple English Wikipedia) · Wikispaces · · Wiki-Site · WikiTravel
Microblogging Heello · · Jaiku · Plurk · Sina Weibo · Tumblr · Twitter · TwitLonger
Music/Audio · digCCmixter · · exfm · Free Music Archive · Gogoyoko · Indaba Music · Jamendo · · MOG · PureVolume · Reverbnation · ShareTheMusic · SoundCloud · Soundpedia ·
People Aaron Swartz · Michael S. Hart · Steve Jobs · Mark Pilgrim · Dennis Ritchie · Len Sassaman Project
Q&A Askville · Answerbag · · · Askalo · Baidu Knows · Blurtit · ChaCha · Experts Exchange · Formspring · GirlsAskGuys · Google Answers · Google Questions and Answers · JustAnswer · MetaFilter · Quora · StackExchange · The AnswerBank · The Internet Oracle · Uclue · WikiAnswers · Yahoo! Answers
Social bookmarking Addinto · Backflip · Balatarin · BibSonomy · Bkmrx · Blinklist · BlogMarks · BookmarkSync · CiteULike · Connotea · Delicious · Digg · Diigo · · Evernote · Excite Bookmark · Faves · Favilous · folkd · Freelish · Getboo · · Gnolia · Google Bookmarks · Hacker News · HeyStaks · IndianPad · Kippt · Knowledge Plaza · Licorize · Linkwad · Menéame · Microsoft Developer Network · Microsoft TechNet · Mister Wong · My Web · Mylink Vault · Newsvine · Oneview · Pearltrees · Pinboard · Pocket · Reddit · · Scloog · Scuttle · Simpy · SiteBar · Slashdot · Squidoo · StumbleUpon · Twine · Vizited · Yummymarks · Xmarks · Yahoo! Buzz · ZipList · Zootool · Zotero
Social networks Bebo · BlackPlanet · · Cyworld · deviantART · Dogster · Dopplr · douban · Ello · Facebook · Flixster · Friendster · Gaia Online · Google+ · Habbo · hi5 · Hyves · LinkedIn · Miiverse · mixi · MyHeritage · MyLife · Myspace · Netlog · Odnoklassniki · Orkut · Plaxo · Qzone · Renren · Skyrock · · Tagged · Upcoming · Viadeo · Vkontakte · WeeWorld · Weibo · Wretch · Yahoo! Groups · Yahoo! Neighbors · Yahoo! Stars India · more sites...
Software/code hosting services Android Development · Alioth · Assembla · BerliOS · Betavine · Bitbucket · BountySource · Codecademy · CodePlex · Freepository · Free Software Foundation · GNU Savannah · GitHub · GitHub Downloads · Gitorious · Gna! · Google Code · ibiblio · · JavaForge · KnowledgeForge · Launchpad · LuaForge · Maemo · mozdev · · OW2 Consortium · Openmoko · · Ovi Store · Project Kenai · RubyForge · · SourceForge · TestFlight · · Transifex · TuxFamily · Yahoo! Downloads
Video hosting services Academic Earth · · Epic · Google Video · · Nokia Trailers · TED Talks · Ustream · Viddler · Viddy · Vimeo · Vstreamers · Yahoo! Video · YouTube
Other 4chan · Akoha · · April Fools' Day · Amplicate · Circavie · Club Nintendo · Cobook · Co.mments · Countdown · Distill · Dmoz · Easel · Electronic Frontier Foundation · Emulation Zone · ESPN Forums · FanFiction.Net · Feedly · Ficlets · FriendFeed · · · FurAffinity · Game Developer Magazine · Gopher · Google Books Ngram · Google Helpouts · Google Reader · Halo · HeavenGames · IFTTT · isoHunt · Jajah · Lulu Poetry · Mailman · · Mochi Media · Mozilla Firefox · MyBlogLog · NBII · Neopets · Pastebin · · Quantcast · Quizilla · RadioShack · Salon Table Talk · Scoop · Slidecast · SOPA blackout pages · · Windows Technical Preview · World Wide Web
Teams Bibliotheca Anonoma · LibreTeam · URLTeam · Yahoo Video Warroom · WikiTeam
About Archive Team Introduction · Philosophy · Who We Are · Why Back Up? · Software · Films and documentaries about archiving · Formats · Cheap storage · Storage Media · Recommended Reading · FAQ
Personal tools