Difference between revisions of "Dev/Source Code"
< Dev
Jump to navigation
Jump to search
(→Misc: add archiveteam-dev-env) |
(→URLTeam code: add links to new tracker code) |
||
| Line 32: | Line 32: | ||
URLTeam code is independent from the tracker and warrior. | URLTeam code is independent from the tracker and warrior. | ||
Old: | |||
'''[https://github.com/ArchiveTeam/tinyback tinyback]''' | '''[https://github.com/ArchiveTeam/tinyback tinyback]''' | ||
| Line 37: | Line 39: | ||
'''[https://github.com/ArchiveTeam/tinyarchive tinyarchive]''' | '''[https://github.com/ArchiveTeam/tinyarchive tinyarchive]''' | ||
: The server code for the tracker. | : The server code for the tracker. | ||
New: | |||
'''[https://github.com/ArchiveTeam/terroroftinytown-client-grab terroroftinytown-client-grab]''' | |||
: A pipeline shim to run the code. | |||
'''[https://github.com/ArchiveTeam/terroroftinytown terroroftinytown]''' | |||
: The code for both the client library and tracker. | |||
== Misc == | == Misc == | ||
Revision as of 17:42, 14 September 2014
Fork me on GitHub! File issues, fix bugs, refactor code, submit pull requests… all welcome!
The warrior uses the following repos:
Client code
Client code includes code that the Warrior executes.
warrior-preseed - shell
- For constructing the warrior virtual appliance image
warrior-code2 - shell
- Bootstrap code that is pulled from GitHub by the appliance
seesaw-kit - Python
- Library that helps build grab scripts, the web interface, and pipeline engine for the warrior. The name "seesaw" comes from its original behavior: download, upload, and repeat.
Projects
Projects are in separate repositories typically with the name -grab as a suffix.
Server code
Server code includes code that the Tracker executes.
universal-tracker - Ruby
- The server of which the Seesaw contacts
warrior-hq - Ruby
- The server of which the warrior appliances contact for project metadata
archiveteam-megawarc-factory - shell
- The scripts that bundles the WARC files.
URLTeam code
URLTeam code is independent from the tracker and warrior.
Old:
- The client code that scrapes the shortlinks. It includes a pipeline shim to run the code.
- The server code for the tracker.
New:
- A pipeline shim to run the code.
- The code for both the client library and tracker.
Misc
- Dockerfile that runs the warrior inside a Docker container.
ArchiveBot - Ruby, Python, Lua
- An IRC bot for archiving websites.
wget-lua - C, Lua
- A patched version of Wget for web crawling.
standalone-readme-template - Markdown
- A template for readme files included in grab repositories.
archiveteam-dev-env - Shell
- Ubuntu preseed for a developer environment for ArchiveTeam projects.