Difference between revisions of "Everything2"

From Archiveteam
Jump to navigation Jump to search
(updated)
Line 11: Line 11:
Since everything2.com is both nonprofit, and shows no signs of shutting down in the near future, I limited wget to one process, and one page a second. Since I started on December 21st, I should be at the two million mark by January 13th. As of 1809h on the 22nd, I'm at node 71000.
Since everything2.com is both nonprofit, and shows no signs of shutting down in the near future, I limited wget to one process, and one page a second. Since I started on December 21st, I should be at the two million mark by January 13th. As of 1809h on the 22nd, I'm at node 71000.


02010/12/31 0725h: node 920964
== Progress ==
* 02010/12/31 0725h: node 920964


Since everything2.com URLs are of the form http://everything2.com/index.pl?node_id=NUMBER, it's fairly easy to increment NUMBER, and thus download everything on the site without having to follow links.
Since everything2.com URLs are of the form http://everything2.com/index.pl?node_id=NUMBER, it's fairly easy to increment NUMBER, and thus download everything on the site without having to follow links.


{{Navigation box}}
{{Navigation box}}

Revision as of 22:47, 31 December 2010

EVERYTHING2.COM
URL http://everything2.com/
Status Online!
Archiving status In progress...
Archiving type Unknown
IRC channel #archiveteam-bs (on hackint)

EVERYTHING2.COM is a kind of proto-wiki, dating from 1999. Never as popular as wikipedia, it still has about 2 million pages, many of which show up nowhere else in the internet.

Since everything2.com is both nonprofit, and shows no signs of shutting down in the near future, I limited wget to one process, and one page a second. Since I started on December 21st, I should be at the two million mark by January 13th. As of 1809h on the 22nd, I'm at node 71000.

Progress

  • 02010/12/31 0725h: node 920964

Since everything2.com URLs are of the form http://everything2.com/index.pl?node_id=NUMBER, it's fairly easy to increment NUMBER, and thus download everything on the site without having to follow links.