Difference between revisions of "Gna!"

From Archiveteam
Jump to navigation Jump to search
(Some quantities. Found Arch hosting.)
(More quantities.)
Line 16: Line 16:
** All subversion repos available via anonymous rsync: rsync://svn.gna.org/svn/ (ref: bottom of every project's svn page e.g. [https://gna.org/svn/?group=admin]): ~43Gbyte in ~1m files
** All subversion repos available via anonymous rsync: rsync://svn.gna.org/svn/ (ref: bottom of every project's svn page e.g. [https://gna.org/svn/?group=admin]): ~43Gbyte in ~1m files
** Ditto CVS, it looks like: rsync://svn.gna.org/cvs/ : ~8Gbyte in ~200k files
** Ditto CVS, it looks like: rsync://svn.gna.org/cvs/ : ~8Gbyte in ~200k files
** Arch/tla [https://gna.org/cookbook/?func=detailitem&item_id=101]: rsync://download.gna.org/arch/ : sizes FIXME
** Arch/tla [https://gna.org/cookbook/?func=detailitem&item_id=101]: rsync://download.gna.org/arch/ : ~332Mbyte in ~91k files
** There's also a ViewVC web front-end to browse code.
** There's also a ViewVC web front-end to browse code.
* '''Ticket tracking'''
* '''Ticket tracking'''
Line 25: Line 25:
*** Attached files are allocated global increasing integer IDs, e.g. [https://gna.org/bugs/download.php?file_id=29845 file #29845]. It's probably possible to scrape all public files by varying 'bugs' and the ID.
*** Attached files are allocated global increasing integer IDs, e.g. [https://gna.org/bugs/download.php?file_id=29845 file #29845]. It's probably possible to scrape all public files by varying 'bugs' and the ID.
** Individual tickets can be private. (Maybe files too?)
** Individual tickets can be private. (Maybe files too?)
* '''File hosting''' at http://download.gna.org/
* '''File hosting''' at http://download.gna.org/ : ~125Gbyte in ~130k files
* '''Project websites''' on home.gna.org
* '''Project websites''' on home.gna.org
** e.g. http://home.gna.org/freeciv/
** e.g. http://home.gna.org/freeciv/
** These are managed via Subversion [https://gna.org/cookbook/?func=detailitem&item_id=107], so grabbing svn by rsync as above should also save website data + history
** These are managed via Subversion [https://gna.org/cookbook/?func=detailitem&item_id=107], so grabbing svn by rsync as above should also save website data + history
** ~7Gbyte in ~177k files (via rsync)? See below
* '''Project documentation''': FIXME
* '''Project documentation''': FIXME
* '''Mailing lists''' using [[Mailman]]
* '''Mailing lists''' using [[Mailman]]
Line 69: Line 70:


  $ rsync --bwlimit 100K --stats --dry-run -avHS rsync://svn.gna.org/cvs/ stoat/
  $ rsync --bwlimit 100K --stats --dry-run -avHS rsync://svn.gna.org/cvs/ stoat/
...
  Number of files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
  Number of files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
  Number of created files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
  Number of created files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
Line 83: Line 85:
  Total bytes received: 5,028,112
  Total bytes received: 5,028,112


Arch: FIXME
Arch:
 
$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/arch/ stoat/
...
Number of files: 91,447 (reg: 74,294, dir: 17,152, link: 1)
Number of created files: 91,447 (reg: 74,294, dir: 17,152, link: 1)
Number of deleted files: 0
Number of regular files transferred: 74,294
Total file size: 332,910,762 bytes
Total transferred file size: 332,910,749 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 1,771,746
File list generation time: 0.058 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 294,204
Total bytes received: 3,546,901


=== File hosting ===
=== File hosting ===
Line 90: Line 108:


  $ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/download/ stoat/
  $ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/download/ stoat/
...
  Number of files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
  Number of files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
  Number of created files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
  Number of created files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
Line 104: Line 123:
  Total bytes received: 5,977,965
  Total bytes received: 5,977,965


====
www (including svn working files):
 
$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/www/ stoat/
...
Number of files: 176,823 (reg: 150,915, dir: 25,887, link: 21)
Number of created files: 176,823 (reg: 150,915, dir: 25,887, link: 21)
Number of deleted files: 0
Number of regular files transferred: 150,915
Total file size: 6,840,609,373 bytes
Total transferred file size: 6,840,609,074 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 6,450,464
File list generation time: 0.083 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 559,763
Total bytes received: 9,045,749


== References ==
== References ==


<references />
<references />

Revision as of 14:59, 25 February 2017

Gna!
URL http://www.gna.org
Status Closing
Archiving status In progress...
Archiving type Unknown
IRC channel #gnarm (on hackint)

Gna! is a centralized location where software developers can develop, distribute and maintain free (GPL-compatible) software. It is an instance of the Savane code-hosting platform[1].

Hosted data

As of 2017-02 it claimed to have 1458 hosted projects. (Many are probably abandoned and will not be saved by their project admins before shutdown.)

  • Code hosting using CVS, Subversion, and Arch
    • All subversion repos available via anonymous rsync: rsync://svn.gna.org/svn/ (ref: bottom of every project's svn page e.g. [1]): ~43Gbyte in ~1m files
    • Ditto CVS, it looks like: rsync://svn.gna.org/cvs/ : ~8Gbyte in ~200k files
    • Arch/tla [2]: rsync://download.gna.org/arch/ : ~332Mbyte in ~91k files
    • There's also a ViewVC web front-end to browse code.
  • Ticket tracking
    • Up to 4 trackers per project: 'bugs', 'patch', 'task', 'support'
    • Project admins (only) can set up XML export of their own ticket text/metadata ("Export" item on tracker admin menu).
      • Only option for third parties looks like web scraping.
    • There's no supported interface for grabbing issue attachments (such as patches) even for project admins though.
      • Attached files are allocated global increasing integer IDs, e.g. file #29845. It's probably possible to scrape all public files by varying 'bugs' and the ID.
    • Individual tickets can be private. (Maybe files too?)
  • File hosting at http://download.gna.org/ : ~125Gbyte in ~130k files
  • Project websites on home.gna.org
    • e.g. http://home.gna.org/freeciv/
    • These are managed via Subversion [3], so grabbing svn by rsync as above should also save website data + history
    • ~7Gbyte in ~177k files (via rsync)? See below
  • Project documentation: FIXME
  • Mailing lists using Mailman
    • Which means public archives are available in mbox format (albeit with email addresses mangled). e.g. [4]
    • Some mailing lists are private.
  • Project metadata: groups, users, news etc. Probably only available via web scraping.
  • Usage stats at http://stats.gna.org/

Shutdown Notice

  • A notice of pending shutdown / request for takeover was first announced in Nov 2016[2] suggesting a time frame of six months
  • A news item about shutdown was posted to the front page 2017-01-31 linking to the above.
  • Exact shutdown timeline is currently (2017-02) unclear.

Detailed quantities

From 2017-02-25 (just counting, files not transferred):

Code hosting

SVN:

$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://svn.gna.org/svn/ stoat/
...
Number of files: 979,943 (reg: 961,966, dir: 17,976, link: 1)
Number of created files: 979,943 (reg: 961,966, dir: 17,976, link: 1)
Number of deleted files: 0
Number of regular files transferred: 961,966
Total file size: 43,448,178,925 bytes
Total transferred file size: 43,448,178,924 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 12,891,196
File list generation time: 1.371 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 2,961,672
Total bytes received: 18,378,594

CVS:

$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://svn.gna.org/cvs/ stoat/
...
Number of files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
Number of created files: 191,592 (reg: 169,489, dir: 22,101, link: 2)
Number of deleted files: 0
Number of regular files transferred: 169,489
Total file size: 8,032,418,393 bytes
Total transferred file size: 8,032,418,367 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 1,912,404
File list generation time: 0.338 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 600,574
Total bytes received: 5,028,112

Arch:

$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/arch/ stoat/
...
Number of files: 91,447 (reg: 74,294, dir: 17,152, link: 1)
Number of created files: 91,447 (reg: 74,294, dir: 17,152, link: 1)
Number of deleted files: 0
Number of regular files transferred: 74,294
Total file size: 332,910,762 bytes
Total transferred file size: 332,910,749 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 1,771,746
File list generation time: 0.058 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 294,204
Total bytes received: 3,546,901

File hosting

Download area:

$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/download/ stoat/
...
Number of files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
Number of created files: 129,722 (reg: 117,260, dir: 8,461, link: 4,001)
Number of deleted files: 0
Number of regular files transferred: 107,011
Total file size: 124,814,979,354 bytes
Total transferred file size: 118,372,817,722 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 3,965,140
File list generation time: 0.378 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 1,302,870
Total bytes received: 5,977,965

www (including svn working files):

$ rsync --bwlimit 100K --stats --dry-run -avHS rsync://download.gna.org/www/ stoat/
...
Number of files: 176,823 (reg: 150,915, dir: 25,887, link: 21)
Number of created files: 176,823 (reg: 150,915, dir: 25,887, link: 21)
Number of deleted files: 0
Number of regular files transferred: 150,915
Total file size: 6,840,609,373 bytes
Total transferred file size: 6,840,609,074 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 6,450,464
File list generation time: 0.083 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 559,763
Total bytes received: 9,045,749

References