Difference between revisions of "Dev/Warrior"

From Archiveteam
< Dev
Jump to navigation Jump to search
m
(Add more details and latest updates)
 
(8 intermediate revisions by 4 users not shown)
Line 3: Line 3:
== Packages ==
== Packages ==


The [https://github.com/ArchiveTeam/warrior-preseed Warrior image] is built off Debian 6.0.5 (squeeze). Here are the basics:
The Warrior image ([https://github.com/ArchiveTeam/Ubuntu-Warrior version 3], [https://github.com/ArchiveTeam/warrior4-vm version 4]) is built off Alpine Linux 3.6.2 (VM version 3.0), Alpine Linux 3.12.0 (VM version 3.1), Alpine Linux 3.13.2 (VM version 3.2 before automatic update), Alpine Linux 3.19.0 (VM version 3.2 after automatic update), Alpine Linux "latest-stable" (VM version 4.0):


* kernel 2.6.32-5-686 (released 2009-03-12)
* kernel 4.9.32 (VM version 3.0), kernel 5.4.43-1virt (VM version 3.1), kernel 5.10.16-0-virt (VM version 3.2 before automatic update), kernel 6.6.7-0-virt or later (VM version 3.2 after automatic update), kernel 6.1.39-0-virt (VM version 4.0)
* Python 2.6.6, pip 1.1
* the Warrior 3.x virtual machine image is prepared using the <code>[https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/master/stage.sh stage.sh]</code> script and contains a pre-installed <code>[https://raw.githubusercontent.com/ArchiveTeam/Ubuntu-Warrior/master/boot.sh /root/boot.sh]</code> script that downloads and boots the warrior.
* Perl v5.10.1, cpan 1.9402 (still needs config)
* Warrior 4 is prepared using a [https://github.com/ArchiveTeam/warrior4-vm/blob/main/dev.md different method]
* gcc 4.4.5, make 3.81, bash 4.1.5
 
* nano 2.2.4 with color syntax highlighting
The warrior itself runs in a [https://github.com/ArchiveTeam/warrior-dockerfile Docker container] running Debian 12 Bookworm that contains Python 3.9, NodeJS, wget-at, and [https://github.com/ArchiveTeam/grab-base-df/blob/master/Dockerfile numerous other dependencies].
* curl 7.21.0
 
Warrior 3.2 will [https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/30056e7b50a7f0ed4fc68dbc221663380e7025eb/startup.sh#L4-L56 automatically update itself] from Alpine Linux version 3.13.2 to Alpine Linux version 3.19 upon boot in order to install a more compatible version of Docker that works with Debian Bookworm. The terminal login prompt will still display "Welcome to Alpine Linux 3.13" after this update, but you can check the kernel version or the contents of <code>/etc/apk/repositories</code> to see if this update has been applied.
 
<strike>Warrior VM versions 3.0 and 3.1 ship with a [https://store.docker.com/community/images/archiveteam/warrior-dockerfile Docker container] running Ubuntu 16.04 that contains
 
* Python 3.5.2, pip 8.1.1
* Perl v5.22.1
* gcc 5.4.0, make 4.1, bash 4.3.48
* curl 7.47.0</strike>
 
<strike>VM versions 3.0 and 3.1 will automatically be updated use the new, updated container configuration the next time they are booted up and connected to the internet.</strike>
 
VM versions 3.0, 3.1, and 3.2-beta will automatically display an EOL message and reboot weekly the next time they are booted up and connected to the internet. Automatically updating these VMs does not appear to be possible, though contributions are welcomed. (Warrior 3.0 segfaults during the upgrade attempt. Warrior 3.1 and Warrior 3.2-beta also segfault during upgrade but that can be fixed by upgrading apk-tools to the latest version for Alpine Linux 3.12 beforehand. However, 3.1 and 3.2-beta still freeze on boot after the upgrade.)


== Bootup ==
== Bootup ==
Line 18: Line 30:
# Start the virtual machine
# Start the virtual machine
# Linux boots
# Linux boots
# The user <code>warrior</code> is automatically logged in.
# <code>boot.sh</code> downloads and launches <code>[https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/master/startup.sh /root/startup.sh]</code>
# <code>/etc/inittab</code> kicks off <code>/home/warrior/warrior-code2/boot.sh</code>.
# <code>startup.sh</code> prepares and runs a Docker container with the warrior runner
## This will <code>git pull https://github.com/ArchiveTeam/warrior-code2</code> into <code>/home/warrior/warrior-code2/</code>.
## <code>/home/warrior/warrior-code2/warrior-runner.sh</code> sets up a process which monitors <code>/dev/shm/ready-for-warrior</code> and launches <code>run-warrior</code> when the state changes.
# <code>boot.sh</code> launches <code>/home/warrior/warrior-code/boot-part-2.sh</code>
# <code>boot-part-2.sh</code> is a short script that does the following:
## <code>./warrior-install.sh</code>
### install/update seesaw, check branch, version
#### installs seesaw-kit into <code>/home/warrior/warrior-code2/src</code>
### install framebuffer support, DNS caching
## <code>sudo ./make-data-disk.sh</code> (the second virtual disk)
### cleans up
### it creates and prepares the data partition
### mounts the partition under <code>/data</code>
### <code>mkdir -p /home/warrior/projects</code>
## <code>touch /dev/shm/ready-for-warrior</code>
### triggers the launch of <code>/usr/local/bin/run-warrior</code> which launches <code>/home/warrior/warrior-code2/src/seesaw/run-warrior</code>
### contacts warriorhq.archiveteam.org and requests the <code>projects.json</code> file. This file contains the projects you see in the Available Projects page.
## <code>./say-hello.sh</code>
### setup vmware port forwarding
### show splash screen
# Point your web browser to http://localhost:8001 and go.
# Point your web browser to http://localhost:8001 and go.


The code for each project is stored in <code>/home/warrior/projects/<PROJECTNAME>/</code>
On Warrior 3.2, <code>boot.sh</code> also checks <code>repo_prefix.txt</code> and <code>branch.txt</code> to determine if <code>startup.sh</code> should be downloaded from an alternate location. See the <code>boot.sh</code> source code for details including URL format.
 
== Viewing container logs ==
Starting with VM version 3.2, you can view basic logs from the virtual machine console: press ALT+F2 for Warrior logs, press ALT+F3 for automatic updater logs, and press ALT+F1 to return to the splash screen.


== Logging into the Warrior ==
== Logging into the Warrior ==
Line 47: Line 43:
To log into the warrior,
To log into the warrior,


# Press Alt+F3 (or press Alt+Right).
# Press Alt+F4 (make sure you capture your keyboard in the VM first or instead press Alt+Right several times).
# The username is <code>root</code> and the password is <code>archiveteam</code>
# The username is <code>root</code> and the password is <code>archiveteam</code>
# You are now logged in as root.
# You are now logged in as root.
# To log in as <code>warrior</code>, enter: <code>sudo -u warrior -i</code>
# Check the docker container with <code>docker ps</code>. This will give you docker container identifier, among others. If you are using VM version 3.2, there should be containers named <code>warrior</code> (the Warrior itself), <code>watchtower</code> (automatic updater, runs every hour), and <code>instantwtower</code> (automatic updater, runs once on VM startup). VM versions 3.0 and 3.1 will automatically be updated use these new, named containers the next time they are booted up and connected to the internet.
# Enter the inside of the docker container with <code>docker exec -it ''identifier'' /bin/bash</code>


== Testing Core Warrior Code ==
== Testing Core Warrior Code ==
''This part may be outdated and refer to Warrior version 2''


Since the Warrior pulls from GitHub, it is important to commit only stable changes into the <code>master</code> branch. Recommended Git branching practices use a development branch.
Since the Warrior pulls from GitHub, it is important to commit only stable changes into the <code>master</code> branch. Recommended Git branching practices use a development branch.
Line 66: Line 65:


By the same route you can return your warrior to the <code>master</code> branch.
By the same route you can return your warrior to the <code>master</code> branch.
The code for each project is stored in <code>/home/warrior/projects/<PROJECTNAME>/</code>


{{devnav}}
{{devnav}}

Latest revision as of 21:31, 14 December 2023

The Warrior is a virtual machine appliance used by volunteers to participate in projects.

Packages

The Warrior image (version 3, version 4) is built off Alpine Linux 3.6.2 (VM version 3.0), Alpine Linux 3.12.0 (VM version 3.1), Alpine Linux 3.13.2 (VM version 3.2 before automatic update), Alpine Linux 3.19.0 (VM version 3.2 after automatic update), Alpine Linux "latest-stable" (VM version 4.0):

  • kernel 4.9.32 (VM version 3.0), kernel 5.4.43-1virt (VM version 3.1), kernel 5.10.16-0-virt (VM version 3.2 before automatic update), kernel 6.6.7-0-virt or later (VM version 3.2 after automatic update), kernel 6.1.39-0-virt (VM version 4.0)
  • the Warrior 3.x virtual machine image is prepared using the stage.sh script and contains a pre-installed /root/boot.sh script that downloads and boots the warrior.
  • Warrior 4 is prepared using a different method

The warrior itself runs in a Docker container running Debian 12 Bookworm that contains Python 3.9, NodeJS, wget-at, and numerous other dependencies.

Warrior 3.2 will automatically update itself from Alpine Linux version 3.13.2 to Alpine Linux version 3.19 upon boot in order to install a more compatible version of Docker that works with Debian Bookworm. The terminal login prompt will still display "Welcome to Alpine Linux 3.13" after this update, but you can check the kernel version or the contents of /etc/apk/repositories to see if this update has been applied.

Warrior VM versions 3.0 and 3.1 ship with a Docker container running Ubuntu 16.04 that contains

  • Python 3.5.2, pip 8.1.1
  • Perl v5.22.1
  • gcc 5.4.0, make 4.1, bash 4.3.48
  • curl 7.47.0

VM versions 3.0 and 3.1 will automatically be updated use the new, updated container configuration the next time they are booted up and connected to the internet.

VM versions 3.0, 3.1, and 3.2-beta will automatically display an EOL message and reboot weekly the next time they are booted up and connected to the internet. Automatically updating these VMs does not appear to be possible, though contributions are welcomed. (Warrior 3.0 segfaults during the upgrade attempt. Warrior 3.1 and Warrior 3.2-beta also segfault during upgrade but that can be fixed by upgrading apk-tools to the latest version for Alpine Linux 3.12 beforehand. However, 3.1 and 3.2-beta still freeze on boot after the upgrade.)

Bootup

The virtual machine is self-updating. It does the following:

  1. Start the virtual machine
  2. Linux boots
  3. boot.sh downloads and launches /root/startup.sh
  4. startup.sh prepares and runs a Docker container with the warrior runner
  5. Point your web browser to http://localhost:8001 and go.

On Warrior 3.2, boot.sh also checks repo_prefix.txt and branch.txt to determine if startup.sh should be downloaded from an alternate location. See the boot.sh source code for details including URL format.

Viewing container logs

Starting with VM version 3.2, you can view basic logs from the virtual machine console: press ALT+F2 for Warrior logs, press ALT+F3 for automatic updater logs, and press ALT+F1 to return to the splash screen.

Logging into the Warrior

To log into the warrior,

  1. Press Alt+F4 (make sure you capture your keyboard in the VM first or instead press Alt+Right several times).
  2. The username is root and the password is archiveteam
  3. You are now logged in as root.
  4. Check the docker container with docker ps. This will give you docker container identifier, among others. If you are using VM version 3.2, there should be containers named warrior (the Warrior itself), watchtower (automatic updater, runs every hour), and instantwtower (automatic updater, runs once on VM startup). VM versions 3.0 and 3.1 will automatically be updated use these new, named containers the next time they are booted up and connected to the internet.
  5. Enter the inside of the docker container with docker exec -it identifier /bin/bash

Testing Core Warrior Code

This part may be outdated and refer to Warrior version 2

Since the Warrior pulls from GitHub, it is important to commit only stable changes into the master branch. Recommended Git branching practices use a development branch.

To test core Warrior code, you can switch from the master branch to the development branch. The Warrior will fetch the corresponding seesaw-kit repository branch.

To change branches,

  1. Log in as root
  2. Execute cd /home/warrior/warrior-code2
  3. Execute sudo -u warrior git checkout development
  4. Execute reboot

By the same route you can return your warrior to the master branch.

The code for each project is stored in /home/warrior/projects/<PROJECTNAME>/


Developer Documentation