Difference between revisions of "Dev/Warrior"

From Archiveteam
< Dev
Jump to navigation Jump to search
m (add devnav)
(Add more details and latest updates)
 
(10 intermediate revisions by 6 users not shown)
Line 1: Line 1:
The warrior image is built off Debian 6.0.5 (squeeze). Here are the basics:
The '''[[Warrior]]''' is a virtual machine appliance used by volunteers to participate in projects.


* kernel 2.6.32-5-686 (released 2009-03-12)
== Packages ==
* Python 2.6.6, pip 1.1
 
* Perl v5.10.1, cpan 1.9402 (still needs config)
The Warrior image ([https://github.com/ArchiveTeam/Ubuntu-Warrior version 3], [https://github.com/ArchiveTeam/warrior4-vm version 4]) is built off Alpine Linux 3.6.2 (VM version 3.0), Alpine Linux 3.12.0 (VM version 3.1), Alpine Linux 3.13.2 (VM version 3.2 before automatic update), Alpine Linux 3.19.0 (VM version 3.2 after automatic update), Alpine Linux "latest-stable" (VM version 4.0):
* gcc 4.4.5, make 3.81, bash 4.1.5
 
* nano 2.2.4 with color syntax highlighting
* kernel 4.9.32 (VM version 3.0), kernel 5.4.43-1virt (VM version 3.1), kernel 5.10.16-0-virt (VM version 3.2 before automatic update), kernel 6.6.7-0-virt or later (VM version 3.2 after automatic update), kernel 6.1.39-0-virt (VM version 4.0)
* curl 7.21.0
* the Warrior 3.x virtual machine image is prepared using the <code>[https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/master/stage.sh stage.sh]</code> script and contains a pre-installed <code>[https://raw.githubusercontent.com/ArchiveTeam/Ubuntu-Warrior/master/boot.sh /root/boot.sh]</code> script that downloads and boots the warrior.
* Warrior 4 is prepared using a [https://github.com/ArchiveTeam/warrior4-vm/blob/main/dev.md different method]
 
The warrior itself runs in a [https://github.com/ArchiveTeam/warrior-dockerfile Docker container] running Debian 12 Bookworm that contains Python 3.9, NodeJS, wget-at, and [https://github.com/ArchiveTeam/grab-base-df/blob/master/Dockerfile numerous other dependencies].
 
Warrior 3.2 will [https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/30056e7b50a7f0ed4fc68dbc221663380e7025eb/startup.sh#L4-L56 automatically update itself] from Alpine Linux version 3.13.2 to Alpine Linux version 3.19 upon boot in order to install a more compatible version of Docker that works with Debian Bookworm. The terminal login prompt will still display "Welcome to Alpine Linux 3.13" after this update, but you can check the kernel version or the contents of <code>/etc/apk/repositories</code> to see if this update has been applied.
 
<strike>Warrior VM versions 3.0 and 3.1 ship with a [https://store.docker.com/community/images/archiveteam/warrior-dockerfile Docker container] running Ubuntu 16.04 that contains
 
* Python 3.5.2, pip 8.1.1
* Perl v5.22.1
* gcc 5.4.0, make 4.1, bash 4.3.48
* curl 7.47.0</strike>
 
<strike>VM versions 3.0 and 3.1 will automatically be updated use the new, updated container configuration the next time they are booted up and connected to the internet.</strike>
 
VM versions 3.0, 3.1, and 3.2-beta will automatically display an EOL message and reboot weekly the next time they are booted up and connected to the internet. Automatically updating these VMs does not appear to be possible, though contributions are welcomed. (Warrior 3.0 segfaults during the upgrade attempt. Warrior 3.1 and Warrior 3.2-beta also segfault during upgrade but that can be fixed by upgrading apk-tools to the latest version for Alpine Linux 3.12 beforehand. However, 3.1 and 3.2-beta still freeze on boot after the upgrade.)


== Bootup ==
== Bootup ==
The virtual machine is self-updating. It does the following:


# Start the virtual machine
# Start the virtual machine
# Linux boots
# Linux boots
# The user <code>warrior</code> is automatically logged in.
# <code>boot.sh</code> downloads and launches <code>[https://github.com/ArchiveTeam/Ubuntu-Warrior/blob/master/startup.sh /root/startup.sh]</code>
# <code>/etc/inittab</code> kicks off <code>/home/warrior/warrior-code2/boot.sh</code>.
# <code>startup.sh</code> prepares and runs a Docker container with the warrior runner
## This will <code>git pull https://github.com/ArchiveTeam/warrior-code2</code> into <code>/home/warrior/warrior-code2/</code>.
## <code>/home/warrior/warrior-code2/warrior-runner.sh</code> sets up a process which monitors <code>/dev/shm/ready-for-warrior</code> and launches <code>run-warrior</code> when the state changes.
# <code>boot.sh</code> launches <code>/home/warrior/warrior-code/boot-part-2.sh</code>
# <code>boot-part-2.sh</code> is a short script that does the following:
## <code>./warrior-install.sh</code>
### install/update seesaw, check branch, version
### install framebuffer support, DNS caching
### sets up <code>/data</code>
## <code>sudo ./make-data-disk.sh</code>
### cleans up
### creates and prepares the partition
### <code>mkdir -p /home/warrior/projects</code>
## <code>touch /dev/shm/ready-for-warrior</code>
### triggers the launch of <code>/usr/local/bin/run-warrior</code> which launches <code>/home/warrior/warrior-code2/src/seesaw/run-warrior</code>
### contacts warriorhq.archiveteam.org and requests the <code>projects.json</code> file. This file contains the projects you see in the Available Projects page.
## <code>./say-hello.sh</code>
### setup vmware port forwarding
### show splash screen
# Point your web browser to http://localhost:8001 and go.
# Point your web browser to http://localhost:8001 and go.


The code for each project is stored in <code>/home/warrior/projects/<PROJECTNAME>/</code>
On Warrior 3.2, <code>boot.sh</code> also checks <code>repo_prefix.txt</code> and <code>branch.txt</code> to determine if <code>startup.sh</code> should be downloaded from an alternate location. See the <code>boot.sh</code> source code for details including URL format.
 
== Viewing container logs ==
Starting with VM version 3.2, you can view basic logs from the virtual machine console: press ALT+F2 for Warrior logs, press ALT+F3 for automatic updater logs, and press ALT+F1 to return to the splash screen.
 
== Logging into the Warrior ==
 
To log into the warrior,
 
# Press Alt+F4 (make sure you capture your keyboard in the VM first or instead press Alt+Right several times).
# The username is <code>root</code> and the password is <code>archiveteam</code>
# You are now logged in as root.
# Check the docker container with <code>docker ps</code>. This will give you docker container identifier, among others. If you are using VM version 3.2, there should be containers named <code>warrior</code> (the Warrior itself), <code>watchtower</code> (automatic updater, runs every hour), and <code>instantwtower</code> (automatic updater, runs once on VM startup). VM versions 3.0 and 3.1 will automatically be updated use these new, named containers the next time they are booted up and connected to the internet.
# Enter the inside of the docker container with <code>docker exec -it ''identifier'' /bin/bash</code>
 
== Testing Core Warrior Code ==
 
''This part may be outdated and refer to Warrior version 2''
 
Since the Warrior pulls from GitHub, it is important to commit only stable changes into the <code>master</code> branch. Recommended Git branching practices use a development branch.


== Testing pre-production code ==
To test core Warrior code, you can switch from the <code>master</code> branch to the <code>development</code> branch. The Warrior will fetch the corresponding seesaw-kit repository branch.


(Don't do this unless you really need or want to.) If you are developing a warrior script, you can test it by switching your warrior from the <code>master</code> branch to the <code>development</code> branch or create another branch. The warrior will fetch the corresponding seesaw repo branch.
To change branches,


<ol>
# Log in as root
<li>Start the warrior.</li>
# Execute <code>cd /home/warrior/warrior-code2</code>
<li>Press Alt+F3 and log in as <code>root</code> , password <code>archiveteam</code></li>
# Execute <code>sudo -u warrior git checkout development</code>
<li><code>cd /home/warrior/warrior-code2</code></li>
# Execute <code>reboot</code>
<li><code>sudo -u warrior git checkout development</code></li>
<li><code>reboot</code></li>
</ol>


By the same route you can return your warrior to the <code>master</code> branch.
By the same route you can return your warrior to the <code>master</code> branch.
The code for each project is stored in <code>/home/warrior/projects/<PROJECTNAME>/</code>


{{devnav}}
{{devnav}}
{{Navigation box}}

Latest revision as of 21:31, 14 December 2023

The Warrior is a virtual machine appliance used by volunteers to participate in projects.

Packages

The Warrior image (version 3, version 4) is built off Alpine Linux 3.6.2 (VM version 3.0), Alpine Linux 3.12.0 (VM version 3.1), Alpine Linux 3.13.2 (VM version 3.2 before automatic update), Alpine Linux 3.19.0 (VM version 3.2 after automatic update), Alpine Linux "latest-stable" (VM version 4.0):

  • kernel 4.9.32 (VM version 3.0), kernel 5.4.43-1virt (VM version 3.1), kernel 5.10.16-0-virt (VM version 3.2 before automatic update), kernel 6.6.7-0-virt or later (VM version 3.2 after automatic update), kernel 6.1.39-0-virt (VM version 4.0)
  • the Warrior 3.x virtual machine image is prepared using the stage.sh script and contains a pre-installed /root/boot.sh script that downloads and boots the warrior.
  • Warrior 4 is prepared using a different method

The warrior itself runs in a Docker container running Debian 12 Bookworm that contains Python 3.9, NodeJS, wget-at, and numerous other dependencies.

Warrior 3.2 will automatically update itself from Alpine Linux version 3.13.2 to Alpine Linux version 3.19 upon boot in order to install a more compatible version of Docker that works with Debian Bookworm. The terminal login prompt will still display "Welcome to Alpine Linux 3.13" after this update, but you can check the kernel version or the contents of /etc/apk/repositories to see if this update has been applied.

Warrior VM versions 3.0 and 3.1 ship with a Docker container running Ubuntu 16.04 that contains

  • Python 3.5.2, pip 8.1.1
  • Perl v5.22.1
  • gcc 5.4.0, make 4.1, bash 4.3.48
  • curl 7.47.0

VM versions 3.0 and 3.1 will automatically be updated use the new, updated container configuration the next time they are booted up and connected to the internet.

VM versions 3.0, 3.1, and 3.2-beta will automatically display an EOL message and reboot weekly the next time they are booted up and connected to the internet. Automatically updating these VMs does not appear to be possible, though contributions are welcomed. (Warrior 3.0 segfaults during the upgrade attempt. Warrior 3.1 and Warrior 3.2-beta also segfault during upgrade but that can be fixed by upgrading apk-tools to the latest version for Alpine Linux 3.12 beforehand. However, 3.1 and 3.2-beta still freeze on boot after the upgrade.)

Bootup

The virtual machine is self-updating. It does the following:

  1. Start the virtual machine
  2. Linux boots
  3. boot.sh downloads and launches /root/startup.sh
  4. startup.sh prepares and runs a Docker container with the warrior runner
  5. Point your web browser to http://localhost:8001 and go.

On Warrior 3.2, boot.sh also checks repo_prefix.txt and branch.txt to determine if startup.sh should be downloaded from an alternate location. See the boot.sh source code for details including URL format.

Viewing container logs

Starting with VM version 3.2, you can view basic logs from the virtual machine console: press ALT+F2 for Warrior logs, press ALT+F3 for automatic updater logs, and press ALT+F1 to return to the splash screen.

Logging into the Warrior

To log into the warrior,

  1. Press Alt+F4 (make sure you capture your keyboard in the VM first or instead press Alt+Right several times).
  2. The username is root and the password is archiveteam
  3. You are now logged in as root.
  4. Check the docker container with docker ps. This will give you docker container identifier, among others. If you are using VM version 3.2, there should be containers named warrior (the Warrior itself), watchtower (automatic updater, runs every hour), and instantwtower (automatic updater, runs once on VM startup). VM versions 3.0 and 3.1 will automatically be updated use these new, named containers the next time they are booted up and connected to the internet.
  5. Enter the inside of the docker container with docker exec -it identifier /bin/bash

Testing Core Warrior Code

This part may be outdated and refer to Warrior version 2

Since the Warrior pulls from GitHub, it is important to commit only stable changes into the master branch. Recommended Git branching practices use a development branch.

To test core Warrior code, you can switch from the master branch to the development branch. The Warrior will fetch the corresponding seesaw-kit repository branch.

To change branches,

  1. Log in as root
  2. Execute cd /home/warrior/warrior-code2
  3. Execute sudo -u warrior git checkout development
  4. Execute reboot

By the same route you can return your warrior to the master branch.

The code for each project is stored in /home/warrior/projects/<PROJECTNAME>/


Developer Documentation