Difference between revisions of "Forums.starwars.com"
(→Forums.StarWars.Com Shutdown: Added #archivewars info) |
|||
Line 21: | Line 21: | ||
</blockquote> | </blockquote> | ||
== ArchiveWars on IRC == | |||
Please use channel [irc://irc.efnet.org/archivewars #archivewars ] for project coordination. ([http://efnet.org EFNet] network.) | |||
== Preliminary Project Scope == | |||
{| class="wikitable" | {| class="wikitable" | ||
Line 50: | Line 57: | ||
|colspan="3"|Note: these are the main categories found from a quick scrape.<br />Possible repetition : File type 'messages' might be the same as 'thread message' | |colspan="3"|Note: these are the main categories found from a quick scrape.<br />Possible repetition : File type 'messages' might be the same as 'thread message' | ||
|} | |} | ||
== '''Profile''' Range Signup Sheet == | == '''Profile''' Range Signup Sheet == |
Revision as of 04:33, 11 May 2011
FORUMS.STARWARS.COM | |
URL | http://forums.starwars.com |
Status | Closing on 2011-06-03 announcement tf.n report |
Archiving status | Not saved yet |
Archiving type | Unknown |
IRC channel | #archiveteam-bs (on hackint) |
Forums.StarWars.Com Shutdown
StarWars.Com announced the closure of their forums on 03 June 2011. (Forum will lock on 3rd May 2011) tf.n report
"The StarWars.com forums have been online since October 2001 and have featured conversations with various Star Wars VIPs. Lucas Licensing's Sue Rostoni had an ongoing dialog with Del Rey customers where she responded to fan questions and concerns. The forums received a facelift in July 2010 that gave users several new features."
"Update: Due to a forums outage over the weekend of April 22-24th, we've extended the time before the forums will be locked into read-only mode. The new date for that is Tuesday, May 3rd."
ArchiveWars on IRC
Please use channel #archivewars for project coordination. (EFNet network.)
Preliminary Project Scope
Page Type | File Name | Preliminary Count Estimate |
---|---|---|
Announcements : | ann.jspa?annID=# | Uncertain (probably under 10) |
Category : | category.jspa?categoryID=# category.jspa?categoryID=#&start=## (## = 0, 15, 30, 45, etc.) |
1 thru 20 each with multiple start values |
Forum : | forum.jspa?forumID=# forum.jspa?forumID=#&start=## |
1 thru ~193 (don't seem to be sequential) 3,500 estimate (One page for every 15 threads) [Highly used forumID=61 has at least 1102 pages start=16515] |
Messages : | message.jspa?messageID=# | ~2 million : according to stats on main forum page [quick scrap found a high value of 17965717] |
Profiles : | profile.jspa?userID=# | ???? : quick scrape found a high value of 9782310 [Seem to be sequential. Earlier # have earlier creation date] [Random numbers do find blank error-500.jsp pages] |
RSS : | rss.jspa?feed=rss%2Frssmessages.jspa?forumID=# | [Please confirm if these are scrape worthy] |
Tag : | tag.jspa?tagName=__NAME__ | ???? : Quantity Unknown [Main Star Wars terms each have their own __NAME__ tag] |
Thread Message : | thread.jspa?messageID=# | ???? : quick scrape found a high value of 17966647 [Similar to 'message'. Maybe redundant.] |
Thread Thread : | thread.jspa?threadID=# | 50,574 according to stats of main forum page ???? : quick scrape found a high value of 275287 |
Other : | Folder 'dwf', 'resources' & 'scripts' have JavaScript (.js) Folders 'images' & 'share' have .gifs File types 'index' and a few other misc. types | |
Note: these are the main categories found from a quick scrape. Possible repetition : File type 'messages' might be the same as 'thread message' |
Profile Range Signup Sheet
We're going to break up the Profile ids into ranges and let individuals claim a range to download. Use this table to mark your territory:
Start | End | Status | Size (Uncompressed) | Claimant |
---|---|---|---|---|
0000001 | 0009999 | Downloaded | 102.3MB | none295 |
0010000 | 0019999 | Downloaded | 24.8MB | none295 |
0020000 | 0099999 | Downloaded | 412.9MB | none295 |
0100000 | 0199999 | Downloaded | 939.2MB | none295 |
0200000 | 0299999 | Downloaded | 1.01GB | none295 |
0300000 | 0399999 | Downloaded | 1.09GB | none295 |
0400000 | 0499999 | Downloaded | __MB | underscor & none295 |
0500000 | 0599999 | Pool | ||
0600000 | 0699999 | Pool | ||
0700000 | 0799999 | Pool | ||
0800000 | 0899999 | Pool | ||
0900000 | 0999999 | Pool | ||
1000000 | 1999999 | Claimed | none295 | |
2000000 | 2099999 | Pool | ||
2100000 | 2199999 | Pool | ||
2200000 | 2299999 | Pool | ||
2300000 | 2399999 | Pool | ||
2400000 | 2499999 | Pool | ||
2500000 | 2599999 | Pool | ||
2600000 | 2699999 | Pool | ||
2700000 | 2799999 | Pool | ||
2800000 | 2899999 | Pool | ||
2900000 | 2999999 | Pool | ||
3000000 | 9999999 | Pool | Please split as required. |
Please try and claim 100,000 id blocks at this time, or more if your system has adequate space.
Example Profile List Generator:
perl -le 'print "http://forums.starwars.com/profile.jspa?userID=$_" for 2000000..2099999' > forums.starwars.com-profile_2000000-2099999