Difference between revisions of "FriendFeed"

From Archiveteam
Jump to navigation Jump to search
(→‎How can I help?: hid discovery warrior project, as content grab needs the real power)
(16 intermediate revisions by 8 users not shown)
Line 5: Line 5:
| description = FriendFeed is the easiest way to share online
| description = FriendFeed is the easiest way to share online
| URL = https://friendfeed.com/
| URL = https://friendfeed.com/
| project_status = {{closing}}
| project_status = {{offline}}
| archiving_status = {{inprogress}}
| archiving_status = {{partiallysaved}}
| irc = humancentifeed
| irc = humancentifeed
| tracker = [http://tracker.archiveteam.org/friendfeeddisco/ friendfeeddisco], [http://tracker.archiveteam.org/friendfeed/ friendfeed]
| tracker = [https://tracker.archiveteam.org/friendfeeddisco/ friendfeeddisco], [https://tracker.archiveteam.org/friendfeed/ friendfeed]
| source = [https://github.com/ArchiveTeam/friendfeed-discovery friendfeed-discovery], [https://github.com/ArchiveTeam/friendfeed-grab friendfeed-grab]
| source = [https://github.com/ArchiveTeam/friendfeed-discovery friendfeed-discovery], [https://github.com/ArchiveTeam/friendfeed-grab friendfeed-grab]
}}
}}


'''FriendFeed''' aggregates [[RSS]] data for a user from the latest hip sites and puts it all in one place. On 2009-08-10, it was announced that FriendFeed had been acquired by [[Facebook]].
'''FriendFeed''' was a standalone social network, also aggregated [[RSS]] data for a user from the latest hip sites and put it all in one place. On 2009-08-10, it was announced that FriendFeed had been acquired by [[Facebook]]. On 2015-03-09 it was announced that FriendFeed would be shut down on 2015-04-09. After a long vigil FriendFeed got actually cut off on 2015-04-10 around 21:00 UTC.


== Vital Signs ==
== Shutdown notice ==


<blockquote>
<blockquote>
Line 30: Line 30:


</blockquote>
</blockquote>
== Discovery ideas ==
*Scrape profiles/groups. Parse them for new IDs. Repeat.
*A user has suggested using the search engine: http://friendfeed.com/search.


== Archiving ==
== Archiving ==
Line 40: Line 35:
=== ArchiveTeam ===
=== ArchiveTeam ===


It got thrown into [[ArchiveBot]] (job ident 7v7q0bcx27e43a527mdhd765f). A [[Warrior]] project might pop up depending on how well ArchiveBot crawls it. '''UPDATE:''' Warrior project popped up.
It got thrown into [[ArchiveBot]] ({{Job|7v7q0bcx27e43a527mdhd765f}}). But later a Warrior project also popped up.


For the newcomers: ArchiveTeam downloads – usually – publicly available content into [[WARC]] format, uploads them to the [[Internet Archive]] and makes them accessible through the [[Wayback Machine]]. At the moment ArchiveTeam is not planning downloading private feeds of FriendFeed, but please read further.
ArchiveTeam was able to discover nearly 1.5 million public accounts and groups, but in lack of time (we had about a week) we couldn't download every single post from everyone. Instead, what was saved for everyone is the first twelve pages of their posts. However, in the beginning, and after that upon request, we also downloaded quite a few accounts in its entirety (or, in fact, up to last 10,000 posts, as accessing more was not possible even using the API).


=== Other efforts ===
Size of archives (compressed): 2.4 TB. The [[WARC]] files can be found in the {{IA item|archiveteam_friendfeed}} collection of the [[Internet Archive]].


ArchiveTeam has been informed that a group of FriendFeed users, led by user [http://friendfeed.com/jimminy Jimminy] is working on exporting FriendFeed accounts (even private ones) of those who go to his site and submit their user name and API key. The archive would be hosted by him and would be accessible and browsable with the username and key.
=== FreeFeed.net ===


Using API keys, one is also able to download pages with index higher than 100, which is otherwise not possible.  
A group of Russian-speaking FriendFeed.com users self-organised in an effort to avoid a dead end. A non-profit entity '''FreeFeed.net MTÜ''' was registered in Estonia and a number of volunteers joined efforts to enable users to archive and later migrate their accounts to a new service.


Many users are migrating to the Italian FriendFeed clone project [http://frenf.it frenf.it].
User account data archiving was done using [https://github.com/zverok/clio Clio] backup script by [https://github.com/zverok Victor Shepelev]. Script was quickly banned by Facebook and a huge effort to bypass the ban using multiple nodes was needed to make data archiving possible. Both private and public accounts and groups were backed up for every user who opted in to back their data. All posts available for the user to access: all data in private groups, all posts and comments including those by users that did not opt-in into the backup - were saved and data archiving was running in real time up to the last minute of FreeFeed.com lifetime.


Discussion: http://friendfeed.com/zeigen/20c8cd8b/testing-waters-of-frenf-it-italian-friendfeed
An open-source FriendFeed.com clone [https://github.com/pepyatka Pepyatka] was used initially as a near-ready platform for the service. When it's architecture proved to be unfeasible, '''FreeFeed.net''' team organized a crowdfunding campaign on [https://www.indiegogo.com/projects/freefeed-v-1#/ Indiegogo] to help volunteers re-develop the system. Campaign was successful, team raised $11,700 and the beta version of it was at [https://beta.freefeed.net/].


ArchiveTeam tries to cooperate with this group. What ArchiveTeam basically could make use of are IDs of public feeds/groups, and possibly the API functionality allowing browsing feeds of more than 100 pages.
[https://freefeed.net/ FreeFeed.net] is currently an [https://github.com/FreeFeed Open Source] (MIT licence) project, democratically run by the community of its active users.


== How can I help? ==
=== Other efforts ===


=== Content discovery ===
Many users have migrated to the Italian FriendFeed clone project [https://www.frenf.it/ frenf.it].


==== Recommending accounts ====
Discussion: http://friendfeed.com/zeigen/20c8cd8b/testing-waters-of-frenf-it-italian-friendfeed
 
'''If you have, or know about, a public feed/group (profile/room) and want to make sure it will be saved by ArchiveTeam, and even help us discover new accounts as we would scrape that, let us know.''' For example, comment on this thread: http://friendfeed.com/friendfeed-feedback/60b14903/archive-team-needs-your-help-to-find-and-spider (preferred way). Alternatively, you can reach us in IRC channel #humancentifeed on EFNet.
 
<!-- I hide this, at least for now, as grab needs way more people!!
 
==== Running a Warrior ====
 
You can start up a [[Warrior]] and there select ''FriendFeed Discovery''. (If you don't really care what you are archiving, select ''ArchiveTeam's Choice'' instead, as at some points ArchiveTeam may priorize another project.)
 
'''Note:''' keep an eye on the Warrior in case of this project. For some yet unknown reason, the Warrior tends to hang up sometimes. If it does, please reboot it.
 
==== Running the script manually ====
 
If you use Linux and you're a bit familiar with it, you can try running the script directly.
 
The instructions can be found at [https://github.com/ArchiveTeam/friendfeed-discovery github.com/ArchiveTeam/friendfeed-discovery].
 
{| class="mw-collapsible mw-collapsed" style="text-align:left;"
! Some additional information
|-
| Don't forget to replace YOURNICKHERE with your nickname.
 
The number after <code>--concurrent</code> determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, bandwidth) are sufficient. However, if you constantly see messages about rate limiting, there is no need to increase the concurrency.
 
If you want to stop the script, please do it gracefully if possible. To do so, create an empty file named '''STOP''' in the folder of the script (terminal command: <code>touch STOP</code>). The script finishes the current item(s) and stops only after that. (If you kill the script immediately, the items get broken, and they will need to be reassigned to another user.) – Before starting the script again, don't forget to remove the STOP file.
 
If you see "Project code is out of date", kill the script, go to its folder (<code>cd friendfeed-discovery</code>) and issue <code><nowiki>git pull https://github.com/ArchiveTeam/</nowiki>friendfeed-discovery</code>. After the updating has finished, re-launch the script.
|}
 
'''Note:''' Running the script with quite high concurrency it's possible that you'll get banned for like 10-15 minutes. In such case please wait a bit and then you can restart again, with some lowered concurrency.
 
-->
 
=== Content grab ===
 
==== Running a Warrior ====
 
You can start up a [[Warrior]] and there select ''FriendFeed''. (If you don't really care what you are archiving, select ''ArchiveTeam's Choice'' instead, as at some points ArchiveTeam may priorize another project.)
 
'''Note:''' keep an eye on the Warrior in case of this project. For some yet unknown reason, the Warrior tends to hang up sometimes. If it does, please reboot it.
 
==== Running the script manually ====
 
If you use Linux and you're a bit familiar with it, you can try running the script directly.
 
The instructions can be found at [https://github.com/ArchiveTeam/friendfeed-grab github.com/ArchiveTeam/friendfeed-grab].
 
{| class="mw-collapsible mw-collapsed" style="text-align:left;"
! Some additional information
|-
| Don't forget to replace YOURNICKHERE with your nickname.
 
The number after <code>--concurrent</code> determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, bandwidth) are sufficient. However, if you constantly see messages about rate limiting, there is no need to increase the concurrency.
 
If you want to stop the script, please do it gracefully if possible. To do so, create an empty file named '''STOP''' in the folder of the script (terminal command: <code>touch STOP</code>). The script finishes the current item(s) and stops only after that. (If you kill the script immediately, the items get broken, and they will need to be reassigned to another user.) – Before starting the script again, don't forget to remove the STOP file.
 
If you see "Project code is out of date", kill the script, go to its folder (<code>cd friendfeed-grab</code>) and issue <code><nowiki>git pull https://github.com/ArchiveTeam/</nowiki>friendfeed-grab</code>. After the updating has finished, re-launch the script.
|}
 
'''Note:''' Running the script with quite high concurrency it's possible that you'll get banned for like 10-15 minutes. In such case please wait a bit and then you can restart again, with some lowered concurrency.
 
=== Donating to the Internet Archive ===
 
Content downloaded by the ArchiveTeam will be uploaded to the [[Internet Archive]], where it will be stored and be available – hopefully – forever. However, storing it costs thousands of dollars in the long run. So, if you can afford, please consider donating to the Internet Archive, so that this piece of history can be kept for us all. http://archive.org/donate


=== Do you like our cause? ===
Although ArchiveTeam was informed only about the Italian clone project, there must be quite a few others out there.


If you want to help in other projects, want to learn more about ArchiveTeam, or even help in development in general, navigate to the [[Main Page]] of this wiki, from there you can reach a lot of information. The Team consists of volunteers working on the projects in their free time, so helping hands (and resources) are always welcome.
Some users may have got json-exports of FriendFeed accounts. Although they differ from ArchiveTeam's preferred format (that is WARC) and won't be ingestable by the Wayback Machine, they are more than welcome in the [[Internet Archive]]. (If it's a private account, though, don't share its content without the user's consent!) Upload them – preferably compressed – to an item (or several items) to the Internet Archive, add metadata (description, tags) so that they can be found by search, and also let us know, maybe on IRC channel #archiveteam in the EFNet IRC network, so that they can be moved to the corresponding collection.


== References ==
== References ==

Revision as of 04:59, 1 February 2020

FriendFeed
FriendFeed logo
FriendFeed is the easiest way to share online
FriendFeed is the easiest way to share online
URL https://friendfeed.com/
Status Offline
Archiving status Partially saved
Archiving type Unknown
Project source friendfeed-discovery, friendfeed-grab
Project tracker friendfeeddisco, friendfeed
IRC channel #humancentifeed (on hackint)

FriendFeed was a standalone social network, also aggregated RSS data for a user from the latest hip sites and put it all in one place. On 2009-08-10, it was announced that FriendFeed had been acquired by Facebook. On 2015-03-09 it was announced that FriendFeed would be shut down on 2015-04-09. After a long vigil FriendFeed got actually cut off on 2015-04-10 around 21:00 UTC.

Shutdown notice

Posted by Benjamin on Monday, March 09, 2015

Dear FriendFeed community,

We wanted to let you know that FriendFeed will be shutting down soon. We've been maintaining the service since we joined Facebook five years ago, but the number of people using FriendFeed has been steadily declining and the community is now just a fraction of what it once was. Given this, we've decided that it's time to start winding things down.

Beginning today, we will no longer accept new signups. You will be able to view your posts, messages, and photos until April 9th. On April 9th, we'll be shutting down FriendFeed and it will no longer be available.

We want to thank you all for being such a terrific and enthusiastic community. We're proud of what we built so many years ago, and we recognize that it would have never been possible without your support.

- The FriendFeed team[1]

Archiving

ArchiveTeam

It got thrown into ArchiveBot (job:7v7q0bcx27e43a527mdhd765f). But later a Warrior project also popped up.

ArchiveTeam was able to discover nearly 1.5 million public accounts and groups, but in lack of time (we had about a week) we couldn't download every single post from everyone. Instead, what was saved for everyone is the first twelve pages of their posts. However, in the beginning, and after that upon request, we also downloaded quite a few accounts in its entirety (or, in fact, up to last 10,000 posts, as accessing more was not possible even using the API).

Size of archives (compressed): 2.4 TB. The WARC files can be found in the archiveteam_friendfeed collection of the Internet Archive.

FreeFeed.net

A group of Russian-speaking FriendFeed.com users self-organised in an effort to avoid a dead end. A non-profit entity FreeFeed.net MTÜ was registered in Estonia and a number of volunteers joined efforts to enable users to archive and later migrate their accounts to a new service.

User account data archiving was done using Clio backup script by Victor Shepelev. Script was quickly banned by Facebook and a huge effort to bypass the ban using multiple nodes was needed to make data archiving possible. Both private and public accounts and groups were backed up for every user who opted in to back their data. All posts available for the user to access: all data in private groups, all posts and comments including those by users that did not opt-in into the backup - were saved and data archiving was running in real time up to the last minute of FreeFeed.com lifetime.

An open-source FriendFeed.com clone Pepyatka was used initially as a near-ready platform for the service. When it's architecture proved to be unfeasible, FreeFeed.net team organized a crowdfunding campaign on Indiegogo to help volunteers re-develop the system. Campaign was successful, team raised $11,700 and the beta version of it was at [1].

FreeFeed.net is currently an Open Source (MIT licence) project, democratically run by the community of its active users.

Other efforts

Many users have migrated to the Italian FriendFeed clone project frenf.it.

Discussion: http://friendfeed.com/zeigen/20c8cd8b/testing-waters-of-frenf-it-italian-friendfeed

Although ArchiveTeam was informed only about the Italian clone project, there must be quite a few others out there.

Some users may have got json-exports of FriendFeed accounts. Although they differ from ArchiveTeam's preferred format (that is WARC) and won't be ingestable by the Wayback Machine, they are more than welcome in the Internet Archive. (If it's a private account, though, don't share its content without the user's consent!) Upload them – preferably compressed – to an item (or several items) to the Internet Archive, add metadata (description, tags) so that they can be found by search, and also let us know, maybe on IRC channel #archiveteam in the EFNet IRC network, so that they can be moved to the corresponding collection.

References