Difference between revisions of "Windows Live Spaces"

From Archiveteam
Jump to navigation Jump to search
m (Added the Navigation box.)
(Did some cleaning up, added hotlists.)
Line 8: Line 8:
}}
}}


On March 16, 2011 will definitely shut down the platform of Windows Live Spaces and since September last year Microsoft has been notifying every user who had a Space active to migrate it to Wordpress using your Windows Live ID, save it to your hard drive or remove. Still, there are many long-abandoned blogs that have not yet been migrated so that they may not survive. For these reasons I decided to create this tutorial to know who wants to save some Spaces.
Microsoft announced that it would shut down Windows Live Spaces on March 16, 2011.  That day has been and gone, and as of March 20, 2011, Windows Live Spaces is still running.  [[User:NovaKing|NovaKing]] is currently scraping Bing for more profiles, so we'll need all the help we can get as soon as he can start producing lists.
 
Since September last year Microsoft has been notifying every user who had a Space active to migrate it to Wordpress using your Windows Live ID, save it to your hard drive or remove. Still, there are many long-abandoned blogs that have not yet been migrated so that they may not survive. For these reasons I decided to create this tutorial to know who wants to save some Spaces.


Any correction is welcome and any questions or suggestions about it must be raised in [[Talk:Windows Live Spaces|the discussion of the article]].
Any correction is welcome and any questions or suggestions about it must be raised in [[Talk:Windows Live Spaces|the discussion of the article]].


== Pending ==
== Pending Spaces ==
*See [[Spaces of Windows Live Spaces pending for download]] to know that Spaces still remain active.
We will need a lot of help to download all the Spaces.
*Autodetect Spaces that no longer exist, because they were closed for a parts or is private<ref>With any link checker appears the HTTP status code 200 so this Spaces type are difficult to detect (see [[wikipedia:List of HTTP status codes]] for details)</ref>, and show a notification that more or less says:
* [[User:Swicher|Swicher]] is currently using HTTrack to download [[Spaces of Windows Live Spaces pending for download|his list]].
  Sorry, you can not watch this space
* [[User:Auguste|Auguste]] and [[User:Dr-spangle|Dr-Spangle]] are using Wget to download an additional copy of that same list (see Hotlists).
The space you are trying to visit is private or does not exist.
 
*Periodically review the list of Spaces to see if anyone was deleted/closed/redirected to WordPress.com.<ref>[http://pastebin.com/wk0xcQp4 From here] you can download a python script written by the user Swicher to check various sites at once, but it is experimental, prone to failure and is in Spanish</ref>
===Hotlists===
*Find a way to detect active Spaces (there are no official Indexes for these sites).
This is a list of all Spaces hotlists and their current status. If you would like to take ownership of one, speak to Auguste on IRC.  If you have taken ownership, please update this table as soon as you are finished, or let Auguste know if you are unable to complete it.
 
These are intended to be downloaded with Perl/Wget, but if you can't use that, do what you can.  We don't have a lot of time, and something is better than nothing.
 
{| border="1" width="100%"
!Filename
!URL
!Owner
!Status
|-
|wls 0001-1000.txt
|http://pastebin.com/FMJh3vAa
|Auguste, Dr-Spangle
|In progress
|-
|wls 1001-2000.txt
|http://pastebin.com/xrXfPbL4
|Auguste, Dr-Spangle
|In progress
|-
|wls 2001-2202.txt
|http://pastebin.com/KAVYAW3c
|Auguste, Dr-Spangle
|In progress
|-
|wls 2203-3000.txt
|http://pastebin.com/pygEEHBr
|Hydruh
|In progress
|-
|wls 3001-4000.txt
|http://pastebin.com/LS8nvgdN
|Unassigned
|N/A
|}
 
 
== Tools ==
There are several ways to download Windows Live Spaces, but using Wget/Perl is recommended.


== HTTrack (graphic version)==
=== HTTrack (graphic version)===
I will explain what is the procedure to download one or more Spaces using HTTrack graphic version (WinHTTrack in Windows and in Linux is called WebHTTrack).
I will explain what is the procedure to download one or more Spaces using HTTrack graphic version (WinHTTrack in Windows and in Linux is called WebHTTrack).


Line 43: Line 83:
  Googlebot/2.1 (+ http://www.googlebot.com/bot.html)
  Googlebot/2.1 (+ http://www.googlebot.com/bot.html)


==LSSaver==
===LSSaver===
<ref>Some of the descriptions in this section was taken from http://www.softsea.com/review/LSSaver.html</ref>
<ref>Some of the descriptions in this section was taken from http://www.softsea.com/review/LSSaver.html</ref>
LSSaver is a Windows Freeware software to save an Windows Live Space blog to your local disk. It saves useful informations such as, blog title, content and comments. It is able to save the pictures included in the blog to local disk also.
LSSaver is a Windows Freeware software to save an Windows Live Space blog to your local disk. It saves useful informations such as, blog title, content and comments. It is able to save the pictures included in the blog to local disk also.
Line 56: Line 96:
*The names of the images are stored as 000001, 000002, etc. thus avoiding that the original can be found on the Internet (this refers to the images of external sites linked in a blog) or recognize the file format.
*The names of the images are stored as 000001, 000002, etc. thus avoiding that the original can be found on the Internet (this refers to the images of external sites linked in a blog) or recognize the file format.


== Using Perl/Wget ==
=== Using Perl/Wget ===
I ([[User:Auguste|Auguste]]) have written a couple of small Perl scripts to simplify the task of archiving Windows Live Spaces.  Unfortunately, it will only grab any dependencies within windows.live.com (i.e. next to nothing).  Due to lack of planning and lack of time, we haven't been able to write anything more thorough, but HTML is better than nothing at all.
I ([[User:Auguste|Auguste]]) have written a couple of small Perl scripts to simplify the task of archiving Windows Live Spaces.  Unfortunately, it will only grab any dependencies within windows.live.com (i.e. next to nothing).  Due to lack of planning and lack of time, we haven't been able to write anything more thorough, but HTML is better than nothing at all.


Line 68: Line 108:
That's it in a nutshell.  I haven't worked on it much, so feel free to write improvements.
That's it in a nutshell.  I haven't worked on it much, so feel free to write improvements.


==Auguste old script==
===Auguste's unfinished script===
The following script written by [[User:Auguste|Auguste]] parse a profile's friend list for get more profiles, then output them to a textfile and remove duplicates. Unfortunately this script has not been completed because Auguste could not to dedicate enough time to work due to their occupations, but he also public the script [http://pastebin.com/2JjZvWuZ here] in case anyone else wants to finish.
[[User:Auguste|Auguste]] began to write [http://pastebin.com/2JjZvWuZ this script] to parse a profile's friend list for more profiles, then output them to a textfile and remove duplicates. Unfortunately, it wasn't completed.  Anyone is welcome to use it or finish it off, but [[User:NovaKing|NovaKing]] is currently using a superior scraper to grab profiles from Bing.
  #!/usr/bin/perl
use warnings;
use strict;
use HTML::TokeParser;
use WWW::Mechanize;
#This is an unfinished Perl script to extract a list of Windows Live Spaces profile URLs from a given profile.
#It's poorly written and barely working, and also gets stuck in an infinite loop during the &getMaxPages subroutine.
#Feel free to take and modify as you please.
#SpaceInvader.pl
#Usage: SpaceInvader.pl PROFILEURL [OUTFILE]
#  e.g. SpaceInvader.pl http://foobar.spaces.live.com foobar_output.txt
my $url = "$ARGV[0]";
my $outFile;
if ($ARGV[1])
{
$outFile = "$ARGV[1]";
}
else
{
$outFile = "output.txt";
}
open(OUTFILE, ">$outFile");
my $mech = WWW::Mechanize->new();
my $page;
&updatePage($url);
&updatePage(&findFriendList);
my $cid = &getCid;
my $i;
my $pages = 1;
for ($i = 1; $i <= $pages; $i++)
{
my $currentPage = "http://cid-" . $cid . ".profile.live.com/friends/all/?page=$i";
&updatePage($currentPage);
&findFriends;
$pages = &getMaxPages;
}
print("Ok, we're all done. $i pages total.\n");
close(OUTFILE);
sub updatePage()
{
if ($_[0])
{
$url = $_[0];
}
print("Loading page $url...\n");
$mech->get("$url");
$page = HTML::TokeParser->new(\$mech->{content});
}
sub findFriendList()
{
print("Locating the friend list...\n");
while (my $tag = $page->get_tag("a"))
{
if ($tag->[1]{title} and $tag->[1]{title} eq "View friends")
{
print("Found the friend list...\n\n");
return $tag->[1]{href};
}
}
}
sub findFriends()
{
print("Producing a list of friends for CID $cid from page $i...\n");
while (my $tag = $page->get_tag("a"))
{
if ($tag->[1]{id} and $tag->[1]{id} =~ /ic[\d|\w]+_frame_clip/)
{
unless ($tag->[1]{href} =~ /$cid/)
{
my $profilePage = $tag->[1]{href};
print("  $profilePage\n");
print(OUTFILE "$profilePage\n");
}
}
}
print("Finished page $i - moving on.\n\n");
}
sub getCid()
{
my $temp = $url;
$temp =~ /cid-([\w|\d]+)/;
return $1;
}
sub getMaxPages()
{
$mech->get("$url");
$page = HTML::TokeParser->new(\$mech->{content});
print("Checking if there's another page...\n");
print("$url\n");
my $maxPages;
while (my $tag = $page->get_tag("a"))
{
if ($tag->[1]{id} and $tag->[1]{id} e


== Trivia ==
== Trivia ==

Revision as of 01:31, 20 March 2011

Windows Live Spaces
Windows Live Spaces homepage (after logging in), as at 18 March 2011
Windows Live Spaces homepage (after logging in), as at 18 March 2011
URL http://spaces.live.com
Status Closing on 2011-03-16[1]
Archiving status In progress...
Archiving type Unknown
IRC channel #archiveteam-bs (on hackint)

Microsoft announced that it would shut down Windows Live Spaces on March 16, 2011. That day has been and gone, and as of March 20, 2011, Windows Live Spaces is still running. NovaKing is currently scraping Bing for more profiles, so we'll need all the help we can get as soon as he can start producing lists.

Since September last year Microsoft has been notifying every user who had a Space active to migrate it to Wordpress using your Windows Live ID, save it to your hard drive or remove. Still, there are many long-abandoned blogs that have not yet been migrated so that they may not survive. For these reasons I decided to create this tutorial to know who wants to save some Spaces.

Any correction is welcome and any questions or suggestions about it must be raised in the discussion of the article.

Pending Spaces

We will need a lot of help to download all the Spaces.

Hotlists

This is a list of all Spaces hotlists and their current status. If you would like to take ownership of one, speak to Auguste on IRC. If you have taken ownership, please update this table as soon as you are finished, or let Auguste know if you are unable to complete it.

These are intended to be downloaded with Perl/Wget, but if you can't use that, do what you can. We don't have a lot of time, and something is better than nothing.

Filename URL Owner Status
wls 0001-1000.txt http://pastebin.com/FMJh3vAa Auguste, Dr-Spangle In progress
wls 1001-2000.txt http://pastebin.com/xrXfPbL4 Auguste, Dr-Spangle In progress
wls 2001-2202.txt http://pastebin.com/KAVYAW3c Auguste, Dr-Spangle In progress
wls 2203-3000.txt http://pastebin.com/pygEEHBr Hydruh In progress
wls 3001-4000.txt http://pastebin.com/LS8nvgdN Unassigned N/A


Tools

There are several ways to download Windows Live Spaces, but using Wget/Perl is recommended.

HTTrack (graphic version)

I will explain what is the procedure to download one or more Spaces using HTTrack graphic version (WinHTTrack in Windows and in Linux is called WebHTTrack).

I assume that the reader should be familiarized with the use of WinHTTrack (or WebHTTrack) so I'll just explain that you need configure (in the Option Panel of the program) to download a Space of Windows Live Spaces.[1]

In the section "Scan Rules" must be added the following lines: 

+*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
+*.7z
+*.pdf +*.doc +*.mid +*.3gp +*.djvu +*.amr +*.mp4 +*.ogg +*.ogv +*.ogm
+*.mov +*.mpg +*.mpeg +*.avi +*.asf +*.mp3 +*.mp2 +*.rm +*.wav +*.vob +*.qt +*.vid +*.ac3 +*.wma +*.wmv
+*.zip +*.tar +*.tgz +*.gz +*.rar +*.z
+*.arj +*.dar +*.lzh +*.lz +*.lza +*.arc
+*.gif +*.jpg +*.png +*.tif +*.bmp
-*.entry#comment
+*.profile.live.com/Lists/*
+*.byfiles.storage.live.com/*
+*.photos.live.com
+*.spaces.live.com

Line 1 to 7 indicate what types of files are downloaded from a Space (if the program finds one these and this lines can be modified to suit the user), the line 8 is because the program tries to capture the comments any post of a blog on Windows Live Spaces and this action generates errors (in addition to a waste of time when exploring a site), line 9 and 12 are used to capturing Spaces of the list of "friends" who might have the Space user which is capturing at that time (these lines are optional), and lines 10 and 11 are to capture the files and photos[2] that the user can have uploaded there.

Finally add in the field Browser "Identity" (from the section Browser ID) the following User Agent: 

Googlebot/2.1 (+ http://www.googlebot.com/bot.html)

LSSaver

[3] LSSaver is a Windows Freeware software to save an Windows Live Space blog to your local disk. It saves useful informations such as, blog title, content and comments. It is able to save the pictures included in the blog to local disk also.

LSSaver is very simple to use, so that its operation is:

  • First, you need to enter a Microsoft Live Space username.
  • Then, you click on the "Get" button to retrive all blog entries[4], when a blog entry is retrieved, it's title will appear in the tree which is the left part of the window. Wait until all titles are trieved. Then you can browse your blog titles by fold/unfold tree, check those you want to save. Once a blog entry is checked, it's content will appear on the right part of the window, check all blogs you want to save and wait until all of them appear.
  • To save the selected blogs, you simply click the Save button, a file selection window will open, select where the files will be saved and give a file name and click the Save button on the window, after a while, all the selected blogs are saved. The saved file is a HTML file, you can open it with a browser.

The program works as it should but we must take into consideration some details that differentiate it from any web site downloader:

  • As explained before, when the program save a blog all the articles (and comments) are crammed into an HTML file (which could become a problem if the blog has a lot of content).
  • The names of the images are stored as 000001, 000002, etc. thus avoiding that the original can be found on the Internet (this refers to the images of external sites linked in a blog) or recognize the file format.

Using Perl/Wget

I (Auguste) have written a couple of small Perl scripts to simplify the task of archiving Windows Live Spaces. Unfortunately, it will only grab any dependencies within windows.live.com (i.e. next to nothing). Due to lack of planning and lack of time, we haven't been able to write anything more thorough, but HTML is better than nothing at all.

You can use the following files to download Spaces:

If you're brave and know Wget pretty well, take a look at wls_experimental.pl, and see if you can get it working properly. It uses a few different Wget parameters in attempt to download offsite images and dependencies, but it ends up crawling other peoples Spaces too and downloading a lot of useless index pages. If you are able to correct this, please do. Otherwise, stick to the original wls.pl.

That's it in a nutshell. I haven't worked on it much, so feel free to write improvements.

Auguste's unfinished script

Auguste began to write this script to parse a profile's friend list for more profiles, then output them to a textfile and remove duplicates. Unfortunately, it wasn't completed. Anyone is welcome to use it or finish it off, but NovaKing is currently using a superior scraper to grab profiles from Bing.

Trivia

  • Existed 7.000.000 active sites[5], but until March 2 only 1.000.000 have been moved to WordPress[6], that is, only 14% of total.
  • Here you can see some statistics from user Swicher on their progress to download some Spaces.

Notes

  1. ↑ If you do not know how to use this program you can check this tutorial (in English) or this one (in Spanish)
  2. ↑ I'm not sure if which is allocated in *. photos.live.com will continue to exist after March 16, then I take the opportunity to save the section "Photos" of Spaces (if the user of Space has this section) so that the line 11 is also optional
  3. ↑ Some of the descriptions in this section was taken from http://www.softsea.com/review/LSSaver.html
  4. ↑ This operation may take up to several minutes depending on the number of entries that contains a blog, as well the user connection
  5. ↑ Over 500,000 Windows Live Spaces blogs migrated to WordPress.com
  6. ↑ Over one million new blogs on WordPress.com, but time is running out

External links

Retrieved from "https://wiki.archiveteam.org/index.php?title=Windows_Live_Spaces&oldid=3285"