Ispygames
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
Gamespy, IGN, 1up, ugo | |
URL | http://www.gamespy.com & many others |
Project status | Closing |
Archiving status | In progress... |
Project source | Unknown |
Project tracker | Unknown |
IRC channel | #ispygames |
The News
IGN hit with layoffs, 1UP, UGO and GameSpy shutting down
1UP, UGO and GameSpy to be shut down
The Problems
- Once you start digging around these sites you find it to be a mess of inconsistent url schemes and content everywhere.
- Some files are being hosted on MediaFire.
- Based on tests the larger and older a site is the more that is missed by a wget crawl due to the url scheme.
What we know
- We already have a list of almost all the domains involved
- A clean list with dups and bad domains is already being process and will be posted here when complete.
- Most of the sites are not that big, but a few are huge.
The plan
- Save the sites and related content
- Backup the twitter feeds for any associated accounts. All my tweets just takes a username and returns the max tweets possible.
wget test command
This if for the gamespy sites.
USER_AGENT="Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)" SAVE_HOST="http://planetdoom.gamespy.com" WARC_NAME="warc_name" wget -e robots=off --mirror --page-requisites \ --waitretry 5 --timeout 60 --tries 5 --wait 2 \ --warc-header "operator: Archive Team" --warc-cdx --warc-file="$WARC_NAME" \ -U "$USER_AGENT" "$SAVE_HOST" \ --span-hosts --domains=$SAVE_HOST,pcmedia.gamespy.com,pnmedia.gamespy.com,pspmedia.gamespy.com,oystatic.ignimgs.com
Try this for the ign, ugo sites.
USER_AGENT="Mozilla/5.0 (Windows; U; MSIE 9.0; Windows NT 9.0; en-US)" SAVE_HOST="http://ve3d.ign.com" WARC_NAME="warc_name" wget -e robots=off --mirror --page-requisites \ --waitretry 5 --timeout 60 --tries 5 --wait 2 \ --warc-header "operator: Archive Team" --warc-cdx --warc-file="$WARC_NAME" \ -U "$USER_AGENT" "$SAVE_HOST"
IGN domains
In progress
- http://aavault.ign.com - grabbed, checking for completeness
- http://autoassaultvault.ign.com/ - grabbed, checking for completeness
- http://beacon.snowball.com - already closed
- http://bombergirl.ign.com - closed contest page in flash
- http://o.cubemedia.ign.com - dead end
- http://actionunleashed.ign.com - grabbed, checking for completeness
- http://dndvault.ign.com - grabbed, checking for completeness
- http://vault.ign.com - grabbing
- http://ac2vault.ign.com - Smiley - done
- http://acvault.ign.com - Smiley - done
- http://aion.ign.com - Smiley Grabbing
- http://wowvault.ign.com - Smiley grabbing
- http://atvault.ign.com - Smiley grabbing
- http://alli.ign.com - Smiley done
- http://antis.ign.com - Smiley grabbing
- http://doa.ign.com - Smiley grabbing
- http://aovault.ign.com - siliconvalleypark - grabbed
- http://aevans.dev.m.au.ign.com - dead
- http://aevans.dev.m.ca.ign.com - dead
- http://aevans.dev.m.ie.ign.com - dead
- http://aevans.dev.m.ign.com - dead
- http://aevans.dev.m.uk.ign.com - dead
- http://cdyi.dev.m.ign.com - dead
- http://ve3d.ign.com - grabbed, checking for completeness
- http://flashpoint.ign.com - siliconvalleypark - grabbed
- http://beaterator.ign.com - siliconvalleypark - grabbed
- http://aivlev.dev.m.au.ign.com - password protected dev site
- http://aivlev.dev.m.ie.ign.com - password protected dev site
- http://aivlev.dev.m.ign.com - password protected dev site
- http://aivlev.dev.m.uk.ign.com - password protected dev site
- http://apassey.dev.m.ign.com - password protected dev site
- http://bestof.ign.com - Tracer grabbing
- http://www.blockbuster.ign.com - already dead
- http://kaneandlynch.ign.com/ - flash based
- http://wishvault.ign.com - siliconvalleypark grabbing
- http://witchervault.ign.com - siliconvalleypark - grabbed
- http://www.supersmashbros.ign.com - Smiley grabbing
- http://www.championshipgamingseries.com - Smiley grabbing
- http://code.ign.com - Smiley grabbing
- http://mevault.ign.com - siliconvalleypark grabbing
- http://beacon.ign.com - Smiley, done
- http://blockbuster.ign.com - 404 not found
- http://broadband.ign.com - Smiley
- http://browserthemes.ign.com - Smiley
- http://ces2009.ign.com - Smiley, all broken links
- http://championshipgamingseries.ign.com - Smiley
- http://championsonline.ign.com - Smiley
- http://cohvault.ign.com - Smiley
- http://comiccon.ign.com - dead
- http://corp.ign.com - Smiley
- http://covvault.ign.com - Smiley, redirects to cohvault
- http://adtools.ign.com - blank
- http://crossassault.ign.com - Smiley, done
- http://design.ign.com - Smiley, done
- http://dragonica.ign.com - Smiley, done
- http://eq2vault.ign.com - Smiley, done with some timeouts
- http://eqvault.ign.com - Smiley, finished with some timeouts
- http://esports.ign.com - Smiley, done
- http://evevault.ign.com - Smiley
- http://feeds.ign.com - Smiley, done
- http://ffvault.ign.com - Smiley, finished with some timeouts
- http://findit.ign.com - Smiley, done
- http://gamechanger.ign.com - Smiley, done
- http://gamesites.ign.com - Smiley, done
- http://gamestore.ign.com - Smiley, done
- http://gamingworldrecord.ign.com - Smiley, done
- http://gbartone.dev.m.au.ign.com - not found
- http://gbartone.dev.m.ca.ign.com - not found
- http://gbartone.dev.m.ie.ign.com - not found
- http://gbartone.dev.m.ign.com - not found
- http://gbartone.dev.m.uk.ign.com - not found
- http://grandtheftautohood.ign.com -> grandtheftauto.ign.com
- http://grandtheftauto.ign.com - Smiley, done
- http://gtahood.ign.com -> grandtheftauto.ign.com
- http://gta.ign.com - Smiley, done
- http://www.blockbuster.ign.com - already dead
- http://guitarhero3.ign.com - Smiley, done
- http://gwvault.ign.com - Smiley, finished with some timeouts
- http://halo.ign.com - Smiley, done
- http://hls.gbartone.dev.m.ign.com -> hls.gbartone.dev.m.uk.ign.com, done, smiley
- http://horizonsvault.ign.com - Smiley, done
- http://ie.bestof.ign.com -> uk.bestof.ign.com (localized version of sites?
- http://ie.top100.ign.com - Smiley, (localized version redirected to .uk.)
- http://uk.bestof.ign.com - Smiley grabbing
- http://uk.corp.ign.com - Smiley grabbing
- http://uk.retro.ign.com - Smiley grabbing -> broken by + in url
- http://uk.sports.ign.com - Smiley grabbing -> uk.ign.com
- http://uk.top100.ign.com - Done
- http://iplstore.ign.com - dead, uploaded, Smiley
- http://jloijens.dev.m.ign.com - Smiley, redirected to .uk. version, done
- http://jtai.dev.m.ign.com - Smiley, dead
- http://kaneandlynch.ign.com - Smiley, done
- http://kjaniak.dev.m.ign.com - blocked by auth, Smiley
- http://l2vault.ign.com - Smiley
- http://labs.ign.com - Smiley, done
- http://lanoire.ign.com - Smiley, done
- http://links.em.ign.com - Smiley, done
- http://littlebigplanet.ign.com - Smiley, done
- http://live.ign.com - Smiley, done
- http://lotrovault.ign.com - Smiley
- http://mag.ign.com - Smiley, done
- http://mail2.ign.com - Smiley, done
- http://mcraft.ign.com - Smiley, done
- http://memoviedia.ign.com - Smiley, dead
- http://mevault.ign.com - Smiley, done
- http://m.ign.com -> m.uk.ign.com, Smiley, done
- http://minecraft.ign.com - Smiley, done
- http://uk.video.ign.com - Smiley grabbing -> broken by redirect
- http://niboppub.ign.com - Smiley, done
- http://nwvault.ign.com - Smiley, done
- http://m.uk.ign.com - Smiley
- http://musichub.ign.com - Smiley, done
- http://mxovault.ign.com - Smiley, done with some timeouts
- http://nchandra.dev.m.uk.ign.com - Smiley, done
- http://o.rpgvaultarchive.ign.com - Smiley, done
- http://potbsvault.ign.com - Smiley, done with some timeouts
- http://rotavault.ign.com - Smiley
- http://rpgvaultarchive.ign.com - Smiley
- http://ryzomvault.ign.com - Smiley, done, some timeouts
- http://sbvault.ign.com - Smiley, dead, done
- http://starcraft2.ign.com - Smiley done
- http://opt-out.emailpreferences.ign.com -> mail.ign.com, Smiley, done
- http://overlord.ign.com - Smiley, done
- http://pawong.dev.www.ign.com - Smiley, auth failed
- http://planetelderscrolls.ign.com - Smiley, done
- http://play.ign.com -> gamespyarcade.com Smiley, done
- http://gamespyarcade.com - Smiley, done
- http://podcast.ign.com -> ign.com/index/podcasts.html, Smiley, done
- http://podcasts.ign.com -> ign.com/index/podcasts.html, Smiley, done
- http://primeblog.ign.com - Smiley, done
- http://promotions.ign.com - Smiley, done
- http://promotools.ign.com - Smiley
- http://publish.ign.com - Smiley, done
- http://rift.ign.com - Smiley
- http://rmcadams.dev.m.ca.ign.com - Smiley, auth failed
- http://rmcadams.dev.m.ign.com - Smiley, auth failed
- http://rmcadams.dev.www.ign.com - Smiley, auth failed
- http://rsullivan.dev.m.ca.ign.com - Smiley, auth failed
- http://rsullivan.dev.m.ign.com - Smiley, auth failed
- http://rsullivan.dev.www.ign.com - Smiley, auth failed
- http://share.affiliation.com - Smiley, dead
- http://share.ign.com - Smiley, done
- http://shootmania.ign.com -> ign.com/ipl/shootmania, Smiley
- http://s.insiderdownloads.ign.com -> http://login.ign.com/prime/landing - Need IGN login subscriber, Smiley
- http://skate2.ign.com - Smiley, done
- http://smashbros.ign.com -> supersmashbros.ign.com, Smiley, done
- http://supersmashbros.ign.com - Smiley
- http://smcnabb.dev.www.ign.com - Smiley, auth failed
- http://sovault.ign.com -> vault.ign.com, Smiley, done
- http://strangleholdcentral.ign.com - Smiley, done
- http://strategyvault.ign.com -> vault.ign.com, Smiley, done
- http://sts.ign.com - Smiley, done
- http://studio.ign.com - Smiley, done
- http://swgvault.ign.com - Smiley
- http://swvault.ign.com - Smiley, done with errors
- http://tabularasavault.ign.com - Smiley, done
- http://tdu.ign.com - Smiley, done, broken
- http://tford.dev.m.ign.com - not found, Smiley
- http://tford.dev.www.ign.com - not found, Smiley
- http://thhe.ign.com - Smiley, done
- http://ticket.ign.com - Smiley, done
- http://tickets.ign.com - Smiley, done
- http://titanquestvault.ign.com - Smiley, done
- http://tjohnson.dev.uk.ign.com - 404, Smiley
- http://touch.ign.com - Smiley, done
- http://tqvault.ign.com -> titanquestvault.ign.com, Smiley, done
- http://trvault.ign.com -> tabularasavault.ign.com, Smiley, done
- http://twoworldsvault.ign.com - Smiley
- http://uovault.ign.com -> http://vault.ign.com/uovault.html, Smiley, done
Ready to grab
- http://au.bestof.ign.com
- http://au.retro.ign.com
- http://au.sports.ign.com
- http://wiki.stg.www.ign.com
untested
- http://au.microsites.ign.com
- http://au.top100.ign.com
- http://au.video.ign.com
- http://crose.dev.m.ca.ign.com
- http://crose.dev.m.ign.com
- http://m.au.ign.com
- http://m.ca.ign.com
- http://m.ie.ign.com
- http://mobile.ign.com
- http://nchandra.dev.m.au.ign.com
- http://nchandra.dev.m.ie.ign.com
- http://nchandra.dev.m.ign.com
- http://nchandra.dev.www.ign.com
- http://o.mobile.ign.com
- http://open.em.ign.com
- http://speedtv.ign.com
- http://sslvpn.ign.com
- http://staging-api.ign.com
- http://store.ign.com
- http://tjohnson.dev.au.ign.com
- http://tjohnson.dev.ca.ign.com
- http://tjohnson.dev.ie.ign.com
- http://tjohnson.dev.www.ign.com
- http://top100.ign.com
- http://v3-api.stg.ie.ign.com
- http://v3-api.stg.m.au.ign.com
- http://v3-api.stg.m.ie.ign.com
- http://v3-api.stg.www.ign.com
- http://vanguardvault.ign.com - Smiley, done
- http://vgu.stg.www.ign.com
- http://viashoka.dev.m.ign.com
- http://viashoka.dev.www.ign.com
- http://videopatch1.atio.dev.uk.ign.com
- http://videoplayer.phantom.stg.www.ign.com
- http://video.stg.www.ign.com
- http://vnboards.ign.com
- http://warhammervault.ign.com
- http://wikihub.stg.www.ign.com
- http://wiki.stg.www.ign.com
- http://wishvault.ign.com
- http://witchervault.ign.com
- http://www.antis.ign.com
- http://www.championshipgamingseries.com
- http://www.ipl.ign.com
- http://www.kaneandlynch.ign.com
- http://www.mevault.ign.com
- http://xboxlive.ign.com
These might be asset only hosting sites
- http://au.media.ign.com
- http://carsmedia.ign.com
- http://carsmovies.ign.com
- http://codesmedia.ign.com
- http://cubemedia.ign.com
- http://dcmedia.ign.com
- http://entertainmentmedia.ign.com
- http://faqsmedia.ign.com
- http://faqsmovies.ign.com
- http://ffmedia.ign.com
- http://formenmedia.ign.com
- http://guidesmedia.ign.com
- http://ie.media.ign.com
- http://insdermedia.ign.com
- http://insiderdownloads.ign.com
- http://insidermedia.ign.com
- http://macmedia.ign.com
- http://macmovies.ign.com
- http://media.ign.com
- http://moviemedia.ign.com
- http://moviesmovies.ign.com
- http://o.faqsmedia.ign.com
- http://o.faqsmovies.ign.com
- http://o.ffmovies.ign.com
- http://o.guidesmedia.ign.com
- http://o.insidermedia.ign.com
- http://o.media.ign.com
- http://o.moviesmovies.ign.com
- http://o.pcmedia.ign.com
- http://o.pocketmedia.ign.com
- http://o.ps2media.ign.com
- http://o.xboxmedia.ign.com
- http://xboxlivemedia.ign.com
- http://xboxlivemovies.ign.com
- http://xboxmedia.ign.com
- http://xboxmovies.ign.com
- http://pcmedia.ign.com
- http://pocketmedia.ign.com
- http://ps2media.ign.com
- http://psxmedia.ign.com
- http://psxmovies.ign.com
- http://scifimedia.ign.com
- http://uk.media.ign.com
- http://vaultmedia.ign.com
- http://wiremedia.ign.com
- http://wiremovies.ign.com
- http://c.ffmovies.ign.com
- http://s.faqsmovies.ign.com
- http://s.ffmovies.ign.com
- http://s.moviesmovies.ign.com
- http://s.psxmovies.ign.com
- http://s.xboxmovies.ign.com
Redirects
- http://cars.ign.com -> http://www.ign.com/
- http://guides.ign.com -> http://www.ign.com/wikis
- http://answers.ign.com -> http://www.ign.com/boards/
- http://ve3dboards.ign.com -> http://www.ign.com/boards/
- http://ffmovies.ign.com -> http://ffmedia.ign.com
- http://ddovault.ign.com -> http://dndvault.ign.com/
- http://bigworldvault.ign.com -> http://vault.ign.com
- http://911.ign.com -> http://tickets.ign-inc.com/
- http://bestofe3.ign.com -> http://games.ign.com/bestofe3.html
- http://dsi.ign.com -> http://ds.ign.com/dsi/
- http://www.ipl.ign.com -> http://www.ign.com/ipl/
- http://xboxlive.ign.com -> http://www.ign.com/xbox-live
- http://download.ign.com - redirect to fileplanet
- http://downloads.ign.com - redirect to fileplanet
- http://dsvault.ign.com - redirect to planetdungeonsiege.com
- http://emailpreferences.ign.com - redirect to mail.ign.com
- http://guidesarchive.ign.com -> http://uk.ign.com/wikis
- http://hjvault.ign.com - redirects to vault.ign.com
- http://ipl.ign.com -> ign.com/ipl
- http://mac.ign.com -> http://uk.ign.com/games/reviews?platformSlug=mac
- http://o.guidesarchive.ign.com -> uk.ign.com/wikis
- http://starcraft.ign.com - Smiley done - redirect to starcraft2.ign.com
Gamespy Domains
Ready to grab
In Progress
- http://lanoirepc.d2gstore.gamespy.com - Smiley, done
- http://gamespyarcade.com - Smiley, done
- http://planetthemovies.gamespy.com - Smiley
- http://planetelderscrolls.gamespy.com - Smiley
- http://planetcnc.gamespy.com - grabbed, checking for completeness
- http://planetthesims.gamespy.com - grabbed, checking for completeness
- http://planetfrontlines.gamespy.com - grabbed, checking for completeness
- http://planetcivilization.gamespy.com - grabbed, checking for completeness
- http://planethalflife.gamespy.com - grabbed, checking for completeness
- http://planettransformers.gamespy.com - grabbed, checking for completeness
- http://planetcoh.gamespy.com - grabbed, checking for completeness
- http://planetbattlefield.gamespy.com - grabbed, checking for completeness
- http://planetresidentevil.gamespy.com - grabbed, checking for completeness
- http://planetxmen.gamespy.com - grabbed, checking for completeness
- http://planetquake.gamespy.com - grabbed, checking for completeness, grabbing again.
- http://planetgrandtheftauto.gamespy.com - grabbed, checking for completeness
- http://planettonyhawk.gamespy.com - grabbed, checking for completeness
- http://planetunreal.gamespy.com - grabbed, checking for completeness
- http://planetfallout.gamespy.com - grabbed, checking for completeness
- http://planetageofempires.gamespy.com - grabbed, checking for completeness
- http://planetgearsofwar.gamespy.com - grabbed, checking for completeness
- http://planetcallofduty.gamespy.com - grabbed, checking for completeness
- http://classicgaming.gamespy.com - grabbed, checking for completeness
- http://planetdoom.gamespy.com - grabbed, checking for completeness
- http://planetwwe.gamespy.com - grabbed, checking for completeness
- http://pc.gamespy.com - grabbed, checking for completeness
- http://psp.gamespy.com - grabbed, checking for completeness
- http://ds.gamespy.com - grabbed, checking for completeness
- http://xbox360.gamespy.com - grabbed, checking for completeness
- http://planetfarcry.gamespy.com - grabbed, checking for completeness
- http://planetcrysis.gamespy.com - grabbing omf_
- http://planetmedalofhonor.gamespy.com - grabbing omf_
- http://xbox.gamespy.com - grabbing Smiley
- http://norad.gamespy.com - grabbing Smiley
- http://bf2142portalservices.gamespy.com - grabbing Smiley
- http://arena.gamespy.com - grabbing Smiley
- http://www.gamespy.com - grabbing Smiley
- http://bugsubmit.gamespy.com - grabbing Smiley
- http://ps3.gamespy.com - Smiley, done with 503's at the end.
Redirects
- http://forumplanet.gamespy.com -> ign.com/boards
- http://forums.gamespy.com -> ign.com/boards/categories/gamespy