Difference between revisions of "Yahoo! Groups"
Jump to navigation
Jump to search
Switchnode (talk | contribs) |
Switchnode (talk | contribs) |
||
| Line 13: | Line 13: | ||
It's been stable for a long time (since the late 90s), long enough for some specialised software to be developed to do backups of it. (Not many other websites can say ''that''.) | It's been stable for a long time (since the late 90s), long enough for some specialised software to be developed to do backups of it. (Not many other websites can say ''that''.) | ||
== | == Statistics == | ||
As of 2019-10-16 the [https://groups.yahoo.com/neo/dir directory] lists 5619351 groups. 2752112 of them have been discovered. 1483853 (54%) have public message archives with an estimated number of 2.1 billion messages (1389 messages per group on average so far). 1.8 billion messages (86%) have been archived as of 2018-10-28. | |||
The following graphs are slightly outdated: | |||
[[File:Yahoo_groups_date_created.png]] | |||
[[File:Yahoo_groups_messages_per_group.png]] | |||
[[File:Yahoo_groups_post_date.png]] | |||
== | == Private groups of interest == | ||
{| class="wikitable" | |||
! Group | |||
! Notes | |||
! Admin contact attempted? | |||
|- | |||
| [https://groups.yahoo.com/neo/groups/numberactivation/info numberactivation] | |||
| see [https://trendingpress.com/some-of-the-uks-phone-number-infrastructure-relies-on-yahoo-groups-which-is-shutting-down/ all] [https://reclaimthenet.org/ofcom-oftel-uk-phone-numbers-yahoo-groups/ the] [https://www.axios.com/yahoo-groups-ofcom-cell-phone-number-porting-51949f81-446e-4b4b-82eb-26790146e9a0.html press] [https://techupdatess.com/some-of-the-uks-phone-number-infrastructure-relies-on-yahoo-groups-the-verge/ coverage] | |||
| Not yet; [https://www.whatdotheyknow.com/request/all_data_held_in_yahoo_groups_us FOI request] made | |||
|- | |||
| [https://groups.yahoo.com/neo/groups/hpslash/info hpslash] | |||
| see [https://fanlore.org/wiki/Hpslash_%28mailing_list%29 Fanlore page] | |||
| Not yet | |||
|} | |||
Potentially relevant: [https://fanlore.org/wiki/Category:Yahoo!_Groups List of groups with Fanlore pages] (contains both private and public groups) | |||
== Site structure == | |||
== Site | |||
There’s a convenient JSON API. May require logging in and joining a group to use all endpoints: | There’s a convenient JSON API. May require logging in and joining a group to use all endpoints: | ||
| Line 61: | Line 69: | ||
Note that all paginated responses are limited to the first 500 results and do not return anything new beyond that. | Note that all paginated responses are limited to the first 500 results and do not return anything new beyond that. | ||
== | == Python Yahoo! Group archivers == | ||
* [https://github.com/IgnoredAmbience/yahoo-group-archiver/network/members yahoo-group-archiver] scrapes a group using the JSON API and (for private endpoints) the two cookies Yahoo uses to verify a logged-in user. Relevant forks include [https://github.com/Frankkkkk/yahoo-group-archiver Frankkkkk] and [https://github.com/nsapa/yahoo-group-archiver nsapa]. Needs merging. Various branches have support (largely untested) for file attachments, photos, links, folders, and events. | |||
[ | * [https://github.com/andrewferguson/YahooGroups-Archiver YahooGroups-Archiver] is similar, but scrapes only messages (not files or any other data). It is not currently under active development. | ||
* [https://github.com/csaftoiu/yahoo-groups-backup yahoo-groups-backup] scrapes a group using Selenium, storing message info and metadata (both rendered message body and raw email) into a Mongo database. It also provides a script to dump its data to static HTML pages that can be viewed in the browser. | |||
* [https:// | |||
== | == Other archivers == | ||
* [http://sourceforge.net/projects/grabyahoogroup/ Yahoo Group Archiver], | * [http://sourceforge.net/projects/grabyahoogroup/ Yahoo Group Archiver]: Perl, defunct. | ||
* Is there a Windows thing out there? | |||
== External Links == | == External Links == | ||
Revision as of 10:45, 19 October 2019
| Yahoo! Groups | |
| |
| |
| URL | http://groups.yahoo.com/ |
| Status | Online! |
| Archiving status | In progress... |
| Archiving type | Unknown |
| IRC channel | #yahoosucks (on hackint) |
Yahoo! Groups is Yahoo's email service; it's the result of the acquisition of eGroups and some other Yahoo! stuff.
It's been stable for a long time (since the late 90s), long enough for some specialised software to be developed to do backups of it. (Not many other websites can say that.)
Statistics
As of 2019-10-16 the directory lists 5619351 groups. 2752112 of them have been discovered. 1483853 (54%) have public message archives with an estimated number of 2.1 billion messages (1389 messages per group on average so far). 1.8 billion messages (86%) have been archived as of 2018-10-28.
The following graphs are slightly outdated:
Private groups of interest
| Group | Notes | Admin contact attempted? |
|---|---|---|
| numberactivation | see all the press coverage | Not yet; FOI request made |
| hpslash | see Fanlore page | Not yet |
Potentially relevant: List of groups with Fanlore pages (contains both private and public groups)
Site structure
There’s a convenient JSON API. May require logging in and joining a group to use all endpoints:
- Group Information: https://groups.yahoo.com/api/v1/groups/concatenative/
- List of Messages: https://groups.yahoo.com/api/v1/groups/concatenative/messages?count=100
- Specific Message: https://groups.yahoo.com/api/v1/groups/concatenative/messages/1/
- Raw Message Content: https://groups.yahoo.com/api/v1/groups/concatenative/messages/1/raw – note that there seems to be a message encoding problem
- List of Topics: https://groups.yahoo.com/api/v1/groups/concatenative/topics?count=100
- Specific Topic: https://groups.yahoo.com/api/v1/groups/concatenative/topics/1
- List of Tables: https://groups.yahoo.com/api/v1/groups/a_furrys_world/database
- Specific Table: https://groups.yahoo.com/api/v1/groups/a_furrys_world/database/1/
- Table Content: https://groups.yahoo.com/api/v1/groups/a_furrys_world/database/1/records
- List of Files: https://groups.yahoo.com/api/v1/groups/a_furrys_world/files
- List of Attachments: https://groups.yahoo.com/api/v1/groups/a_furrys_world/attachments
- List of Polls: https://groups.yahoo.com/api/v1/groups/a_furrys_world/polls?count=100
- Specific Poll: https://groups.yahoo.com/api/v1/groups/a_furrys_world/polls/3549106
- List of Photos: https://groups.yahoo.com/api/v1/groups/a_furrys_world/photos
- List of Albums: https://groups.yahoo.com/api/v1/groups/a_furrys_world/albums
- Specific Album: https://groups.yahoo.com/api/v1/groups/a_furrys_world/albums/1841906391
- List Moderators: https://groups.yahoo.com/api/v1/groups/a_furrys_world/members/moderators
- Members With Incorrect Emails: https://groups.yahoo.com/api/v1/groups/a_furrys_world/members/bouncing
- List of Links: https://groups.yahoo.com/api/v1/groups/a_furrys_world/links
- Search: https://groups.yahoo.com/api/v1/search/groups?offset=0&maxHits=20&sortBy=&query=abcdef – sort can be one of OLDEST, RELEVANCE, MEMBERS, LATEST_ACTIVITY, NEWEST
- Categories: https://groups.yahoo.com/api/v1/dir/categories/0/?start=0
Note that all paginated responses are limited to the first 500 results and do not return anything new beyond that.
Python Yahoo! Group archivers
- yahoo-group-archiver scrapes a group using the JSON API and (for private endpoints) the two cookies Yahoo uses to verify a logged-in user. Relevant forks include Frankkkkk and nsapa. Needs merging. Various branches have support (largely untested) for file attachments, photos, links, folders, and events.
- YahooGroups-Archiver is similar, but scrapes only messages (not files or any other data). It is not currently under active development.
- yahoo-groups-backup scrapes a group using Selenium, storing message info and metadata (both rendered message body and raw email) into a Mongo database. It also provides a script to dump its data to static HTML pages that can be viewed in the browser.
Other archivers
- Yahoo Group Archiver: Perl, defunct.
- Is there a Windows thing out there?