Difference between revisions of "WikiTeam"
m (→Wiki dumps: Table cleanup) |
(→Wiki dumps: Colorized the table. More knowledgeable people might want to doublecheck correctness) |
||
Line 41: | Line 41: | ||
* Backup your favorite wikis or leave the URL [[Talk:WikiTeam|here]]. | * Backup your favorite wikis or leave the URL [[Talk:WikiTeam|here]]. | ||
{| class="wikitable" | |||
| style="background: lightgreen" | | |||
| Good | |||
|- | |||
| style="background: lightyellow" | | |||
| Could be better | |||
|- | |||
| style="background: lightcoral" | | |||
| Bad | |||
|- | |||
| | |||
| Unknown | |||
|- | |||
|} | |||
{| class="wikitable" border=1 width=99% style="text-align: center;" | {| class="wikitable" border=1 width=99% style="text-align: center;" | ||
! Wiki !! Wiki is online? !! Dumps available? (official or home-made) !! Comments/Details !! Saved by us? Who? Where? | ! Wiki !! Wiki is online? !! Dumps available? (official or home-made) !! Comments/Details !! Saved by us? Who? Where? | ||
|- | |- | ||
| [http://s23.org/wikistats/anarchopedias_html.php Anarchopedias] || Yes || Official: no. Home-made: [http://www.mediafire.com/file/t73az9cwhzco2wb/Anarchopedia_Jun2011.7z Yes] || - || idiolect | | [http://s23.org/wikistats/anarchopedias_html.php Anarchopedias] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://www.mediafire.com/file/t73az9cwhzco2wb/Anarchopedia_Jun2011.7z Yes] || - || idiolect | ||
|- | |- | ||
| [http://archiveteam.org Archive Team Wiki] || Yes || Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=archiveteam yes] || - || WikiTeam | | [http://archiveteam.org Archive Team Wiki] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=archiveteam yes] || - || WikiTeam | ||
|- | |- | ||
| Bulbapedia || Yes || Official: no. Home-made: no || - || dr-spangle is working on it with a self-built PHP downloader | | Bulbapedia || style="background: lightgreen" | Yes || style="background: lightcoral" | Official: no. Home-made: no || - || dr-spangle is working on it with a self-built PHP downloader | ||
|- | |- | ||
| [[Citizendium]] || Yes || Official: [http://en.citizendium.org/wiki/CZ:Downloads daily] (no full history). Home-made: [[Citizendium|yes]], April 2011 || No image dumps available || - | | [[Citizendium]] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: [http://en.citizendium.org/wiki/CZ:Downloads daily] (no full history). Home-made: [[Citizendium|yes]], April 2011 || style="background: lightyellow" | No image dumps available || - | ||
|- | |- | ||
| [http://s23.org/wikistats/editthis_html.php EditThis] || Yes || Official: no. Home-made: in progress || - || - | | [http://s23.org/wikistats/editthis_html.php EditThis] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: in progress || - || - | ||
|- | |- | ||
| enciclopedia.us.es || Yes || Official: no. Home-made: no || Sysop sent me page text sql tables || emijrp | | enciclopedia.us.es || style="background: lightgreen" | Yes || style="background: lightcoral" | Official: no. Home-made: no || style="background: lightyellow" | Sysop sent me page text sql tables || emijrp | ||
|- | |- | ||
| [[Encyclopedia Dramatica]] || No || Official: no. Home-made: partial || WebEcology Project Article Dump (~9000 Articles)<br />Most of the Images probably Lost || - | | [[Encyclopedia Dramatica]] || style="background: lightcoral" | No || style="background: lightyellow" | Official: no. Home-made: partial || style="background: lightyellow" | WebEcology Project Article Dump (~9000 Articles)<br />Most of the Images probably Lost || - | ||
|- | |- | ||
| [[Encyclopedia Dramatica|Encyclopedia Dramatica.ch]]<br />(new ED) || Yes || Official: ? Home-made: ? || Slowly being rebuilt from old sources.<br />Should be up for a while but for who knows how long? || - | | [[Encyclopedia Dramatica|Encyclopedia Dramatica.ch]]<br />(new ED) || style="background: lightgreen" | Yes || Official: ? Home-made: ? || style="background: lightyellow" | Slowly being rebuilt from old sources.<br />Should be up for a while but for who knows how long? || - | ||
|- | |- | ||
| [http://s23.org/wikistats/gentoo_html.php Gentoo wikis] || Yes || Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=gentoo yes] || - || WikiTeam | | [http://s23.org/wikistats/gentoo_html.php Gentoo wikis] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=gentoo yes] || - || WikiTeam | ||
|- | |- | ||
| GNUpedia || No || Official: no. Home-made: no || No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed || - | | GNUpedia || style="background: lightcoral" | No || style="background: lightcoral" | Official: no. Home-made: no || style="background: lightcoral" | No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed || - | ||
|- | |- | ||
| [[MeatBall]] || Yes || Official: no. Home-made: [http://mirrors.sdboyd56.com/WikiTeam/meatball_wiki-20110706.7z yes] ([http://code.google.com/p/wikiteam/downloads/detail?name=meatball_wiki-20110706.7z mirror]) || No histories, no xml format || SDBoyd | | [[MeatBall]] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://mirrors.sdboyd56.com/WikiTeam/meatball_wiki-20110706.7z yes] ([http://code.google.com/p/wikiteam/downloads/detail?name=meatball_wiki-20110706.7z mirror]) || style="background: lightyellow" | No histories, no xml format || SDBoyd | ||
|- | |- | ||
| [http://s23.org/wikistats/metapedias_html.php Metapedia] || Yes || Official: ?. Home-made: no || - || - | | [http://s23.org/wikistats/metapedias_html.php Metapedia] || style="background: lightgreen" | Yes || style="background: lightcoral" | Official: ?. Home-made: no || - || - | ||
|- | |- | ||
| [http://s23.org/wikistats/scoutwiki_html.php Neoseeker] aka Scout wikis || Yes || Official: ?. Home-made: no || - || - | | [http://s23.org/wikistats/scoutwiki_html.php Neoseeker] aka Scout wikis || style="background: lightgreen" | Yes || style="background: lightcoral" | Official: ?. Home-made: no || - || - | ||
|- | |- | ||
| [[Nupedia]] || No || Official: ?. Home-made: Yes, saved from IA || - || - | | [[Nupedia]] || style="background: lightcoral" | No || style="background: lightyellow" | Official: ?. Home-made: Yes, saved from IA || - || - | ||
|- | |- | ||
| OmegaWiki || Yes || Official: [http://www.omegawiki.org/Development daily] || - || - | | OmegaWiki || style="background: lightgreen" | Yes || style="background: lightgreen" | Official: [http://www.omegawiki.org/Development daily] || - || - | ||
|- | |- | ||
| OpenStreetMap || Yes || Official: Yes. Home-made: no || - || - | | OpenStreetMap || style="background: lightgreen" | Yes || Official: Yes. Home-made: no || - || - | ||
|- | |- | ||
| [http://s23.org/wikistats/opensuse_html.php OpenSUSE wikis] || Yes || Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=opensuse in progress] || - || - | | [http://s23.org/wikistats/opensuse_html.php OpenSUSE wikis] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=opensuse in progress] || - || - | ||
|- | |- | ||
| OSDev || Yes || Official: [http://wiki.osdev.org/OSDev_Wiki:About weekly] || - || Not yet | | OSDev || style="background: lightgreen" | Yes || style="background: lightgreen" | Official: [http://wiki.osdev.org/OSDev_Wiki:About weekly] || - || Not yet | ||
|- | |- | ||
| [http://tvtropes.org TV Tropes] || Yes || Official: No Unofficial: In progress || No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" || DoubleJ | | [http://tvtropes.org TV Tropes] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: No Unofficial: In progress || style="background: lightyellow" | No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" || DoubleJ | ||
|- | |- | ||
| [http://s23.org/wikistats/uncyclomedia_html.php Uncyclomedias] || - || - || - || - | | [http://s23.org/wikistats/uncyclomedia_html.php Uncyclomedias] || - || - || - || - | ||
|- | |- | ||
| Wikanda || Yes || Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=wikanda yes] || - || emijrp | | Wikanda || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: no. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=wikanda yes] || - || emijrp | ||
|- | |- | ||
| [[Wikia]] || Yes || Official: [http://wiki-stats.wikia.com/ on demand] || No image dumps available || Not yet | | [[Wikia]] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: [http://wiki-stats.wikia.com/ on demand] || style="background: lightyellow" | No image dumps available || Not yet | ||
|- | |- | ||
| [http://wikifur.com WikiFur] || Yes || Official: [http://dumps.wikifur.com/ yes] || No image dumps available || Not yet | | [http://wikifur.com WikiFur] || style="background: lightgreen" | Yes || style="background: lightgreen" | Official: [http://dumps.wikifur.com/ yes] || style="background: lightyellow" | No image dumps available || Not yet | ||
|- | |- | ||
| WikiHow || - || - || - || - | | WikiHow || - || - || - || - | ||
|- | |- | ||
| [[Wikimedia Commons]] || Yes || Official: [http://dumps.wikimedia.org/commonswiki/latest/ periodically] || No image dumps available || Not yet | | [[Wikimedia Commons]] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: [http://dumps.wikimedia.org/commonswiki/latest/ periodically] || style="background: lightyellow" | No image dumps available || Not yet | ||
|- | |- | ||
| [[Wikipedia]] || Yes || Official: [http://dumps.wikimedia.org/backup-index.html periodically] || No image dumps available. English Wikipedia dump uses to be very old || Not yet | | [[Wikipedia]] || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: [http://dumps.wikimedia.org/backup-index.html periodically] || style="background: lightyellow" | No image dumps available. English Wikipedia dump uses to be very old || Not yet | ||
|- | |- | ||
| [http://s23.org/wikistats/wikisite_html.php Wiki-site.com] || - || - || - || - | | [http://s23.org/wikistats/wikisite_html.php Wiki-site.com] || - || - || - || - | ||
|- | |- | ||
| WikiTravel || Yes || Official: [http://wikitravel.org/en/Wikitravel:Database_dump not yet]. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=wikitravel yes], another of [http://dl.dropbox.com/u/63233/Wikitravel/Complete%20zip/WikitravelComplete14-June-2010.7z 2010-06-14] || - || WikiTeam | | WikiTravel || style="background: lightgreen" | Yes || style="background: lightyellow" | Official: [http://wikitravel.org/en/Wikitravel:Database_dump not yet]. Home-made: [http://code.google.com/p/wikiteam/downloads/list?can=1&q=wikitravel yes], another of [http://dl.dropbox.com/u/63233/Wikitravel/Complete%20zip/WikitravelComplete14-June-2010.7z 2010-06-14] || - || WikiTeam | ||
|- | |- | ||
| WikiWikiWeb || Yes || Home-made: [http://www.multiupload.com/BGGCFUHOE7 yes] || - || Ca7 | | WikiWikiWeb || style="background: lightgreen" | Yes || style="background: lightyellow" | Home-made: [http://www.multiupload.com/BGGCFUHOE7 yes] || - || Ca7 | ||
|- | |- | ||
| [http://co-forum.de/ (o:forum] || Yes || No || - || Not yet, to figure out how | | [http://co-forum.de/ (o:forum] || style="background: lightgreen" | Yes || style="background: lightcoral" | No || - || Not yet, to figure out how | ||
|- | |- | ||
| [http://www.wikiwiki.de/newwiki/pmwiki.php WikiWiki.de] || Yes || No || - || Not yet, to figure out how | | [http://www.wikiwiki.de/newwiki/pmwiki.php WikiWiki.de] || style="background: lightgreen" | Yes || style="background: lightcoral" | No || - || Not yet, to figure out how | ||
|- | |- | ||
| [http://www.wikiservice.at/gruender/wiki.cgi?action=HomePage GruenderWiki] || Yes || No || - || Not yet, to figure out how | | [http://www.wikiservice.at/gruender/wiki.cgi?action=HomePage GruenderWiki] || style="background: lightgreen" | Yes || style="background: lightcoral" | No || - || Not yet, to figure out how | ||
|} | |} | ||
Line 116: | Line 130: | ||
A feed of BitTorrent downloads is available for the latest files posted to the [http://code.google.com/p/wikiteam/downloads/list WikiTeam Google Code Downloads]. | A feed of BitTorrent downloads is available for the latest files posted to the [http://code.google.com/p/wikiteam/downloads/list WikiTeam Google Code Downloads]. | ||
* [http://pipes.yahoo.com/lobstor/google_code_torrent?_render=rss&project=wikiteam WikiTeam Torrent Feed] (pipes.yahoo.com) | * [http://pipes.yahoo.com/lobstor/google_code_torrent?_render=rss&project=wikiteam WikiTeam Torrent Feed] (pipes.yahoo.com) | ||
Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in | Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows). | ||
== Closing/In danger == | == Closing/In danger == |
Revision as of 08:45, 1 October 2011
100 wikis saved to date
WikiTeam | |
WikiTeam, a set of tools for wiki preservation and a repository of wikis | |
URL | http://code.google.com/p/wikiteam |
Status | Online! |
Archiving status | In progress... |
Archiving type | Unknown |
IRC channel | #wikiteam (on hackint) |
Welcome to WikiTeam. A wiki is a website that allows the creation and editing of any number of interlinked web pages, generally used to store information on a specific subject or subjects. This is done with a day-to-day web browser using a simplified markup language (HTML as an example) or a WYSIWYG (what-you-see-is-what-you-get) text editor.
Examples of huge wikis:
- Wikipedia - arguably the largest and one of the oldest Wikis on the planet. It offers public backups: http://dumps.wikimedia.org
- Wikimedia Commons - a Wiki of media files available for free usage. It offers public backups: http://dumps.wikimedia.org
- But there is no image dump available, only the image descriptions
- Wikia - a website that allows the creation and hosting of wikis. It offers public backups: http://wiki-stats.wikia.com
Most of the wikis don't offer public backups. How bad!
Tools and source code
- WikiTeam Google Code repository
- dumpgenerator.py to download MediaWiki wikis: python dumpgenerator.py --api=http://archiveteam.org/api.php --xml --images
- wikipediadownloader.py to download Wikipedia dumps from download.wikimedia.org: python wikipediadownloader.py
- There are many wiki engines, the most famous is MediaWiki. So, the tools must be ready to read data from almost every wiki engine to saved them all.
- Scripts of a guy who saved Wikitravel
- OddMuseWiki backup
- UseModWiki: use wget/curl and raw mode (might have a different URL scheme, like this)
Wiki dumps
For a more detailed list, visit the download section on Google Code.
There is another site of MediaWiki dumps located here on Scott's Website.
TODO lists:
- WikiTeam/Sites using MediaWiki (English)
- WikiTeam/Sites using MediaWiki (Multilingual)
- Backup your favorite wikis or leave the URL here.
Good | |
Could be better | |
Bad | |
Unknown |
Wiki | Wiki is online? | Dumps available? (official or home-made) | Comments/Details | Saved by us? Who? Where? |
---|---|---|---|---|
Anarchopedias | Yes | Official: no. Home-made: Yes | - | idiolect |
Archive Team Wiki | Yes | Official: no. Home-made: yes | - | WikiTeam |
Bulbapedia | Yes | Official: no. Home-made: no | - | dr-spangle is working on it with a self-built PHP downloader |
Citizendium | Yes | Official: daily (no full history). Home-made: yes, April 2011 | No image dumps available | - |
EditThis | Yes | Official: no. Home-made: in progress | - | - |
enciclopedia.us.es | Yes | Official: no. Home-made: no | Sysop sent me page text sql tables | emijrp |
Encyclopedia Dramatica | No | Official: no. Home-made: partial | WebEcology Project Article Dump (~9000 Articles) Most of the Images probably Lost |
- |
Encyclopedia Dramatica.ch (new ED) |
Yes | Official: ? Home-made: ? | Slowly being rebuilt from old sources. Should be up for a while but for who knows how long? |
- |
Gentoo wikis | Yes | Official: no. Home-made: yes | - | WikiTeam |
GNUpedia | No | Official: no. Home-made: no | No database. This "wiki encyclopedia" was only HTML pages. Only ~3 articles were sent to the mailing list. After that, the project was closed | - |
MeatBall | Yes | Official: no. Home-made: yes (mirror) | No histories, no xml format | SDBoyd |
Metapedia | Yes | Official: ?. Home-made: no | - | - |
Neoseeker aka Scout wikis | Yes | Official: ?. Home-made: no | - | - |
Nupedia | No | Official: ?. Home-made: Yes, saved from IA | - | - |
OmegaWiki | Yes | Official: daily | - | - |
OpenStreetMap | Yes | Official: Yes. Home-made: no | - | - |
OpenSUSE wikis | Yes | Official: no. Home-made: in progress | - | - |
OSDev | Yes | Official: weekly | - | Not yet |
TV Tropes | Yes | Official: No Unofficial: In progress | No dump mechanism, using wget -nc -r -p -l 0 -np -w 45 -E -k -T 10 -nv -x "http://tvtropes.org" | DoubleJ |
Uncyclomedias | - | - | - | - |
Wikanda | Yes | Official: no. Home-made: yes | - | emijrp |
Wikia | Yes | Official: on demand | No image dumps available | Not yet |
WikiFur | Yes | Official: yes | No image dumps available | Not yet |
WikiHow | - | - | - | - |
Wikimedia Commons | Yes | Official: periodically | No image dumps available | Not yet |
Wikipedia | Yes | Official: periodically | No image dumps available. English Wikipedia dump uses to be very old | Not yet |
Wiki-site.com | - | - | - | - |
WikiTravel | Yes | Official: not yet. Home-made: yes, another of 2010-06-14 | - | WikiTeam |
WikiWikiWeb | Yes | Home-made: yes | - | Ca7 |
(o:forum | Yes | No | - | Not yet, to figure out how |
WikiWiki.de | Yes | No | - | Not yet, to figure out how |
GruenderWiki | Yes | No | - | Not yet, to figure out how |
Tips
Some tips:
- When downloading Wikipedia/Wikimedia Commons dumps, pages-meta-history.xml.7z and pages-meta-history.xml.bz2 are the same, but 7z use to be smaller (better compress ratio), so use 7z.
BitTorrent downloads
A feed of BitTorrent downloads is available for the latest files posted to the WikiTeam Google Code Downloads.
- WikiTeam Torrent Feed (pipes.yahoo.com)
Files under 1 MB are blocked on the service generating these torrents (Burnbit.com), so not every file is available as a torrent. There may be some delay after a file is uploaded before the torrent appears on the feed. You can subscribe to this feed in your BitTorrent client for automatic downloads (this has been tested successfully in µTorrent on Windows).
Closing/In danger
- Gentoo wikis: Error 503 Service Unavailable as of 2011-04-06 http://s23.org/wikistats/gentoo_html.php
Offline wikis and wikifarms
elwiki.com
- 2011
- wik.is
- 2010
- 2009
- 2008
- Scribblewiki (wikifarm)
External links
- http://wikiindex.org - A lot of wikis to save
- http://wiki1001.com/
- http://meta.wikimedia.org/wiki/List_of_largest_wikis
- http://s23.org/wikistats/
- http://en.wikipedia.org/wiki/Comparison_of_wiki_farms
- http://en.wikipedia.org/wiki/User:Emijrp/Wikipedia_Archive
- http://blog.shoutwiki.com/
- http://wikiheaven.blogspot.com/
- List of largest wikis in the world
- Dump of nostalgia, an ancient version of Wikipedia from 2001, dump
- http://code.google.com/p/wikiteam/downloads/list?can=1 many dumps