FTP

From Archiveteam
Revision as of 19:16, 13 June 2014 by SN4T14 (talk | contribs) (Fixed whitespace and script.)
Jump to navigation Jump to search
FTP
Threeplaces.jpg
Status Online!
Archiving status Not saved yet
Archiving type Unknown
Project source https://github.com/ArchiveTeam/ftp-nab
IRC channel #effteepee (on hackint)

Archiving a whole public FTP host/mirror is easy:

SketchCow> I use wget -r -l 0 -np -nc ftp://ftp.underscorporn.com
tar cvf 2014.01.ftp.underscorporn.com.tar ftp.underscorporn.com
tar tvf 2014.01.ftp.underscorporn.com.tar > 2014.01.ftp.underscorporn.com.tar.txt

OR, use this handy dandy function to put in your .bashrc file, you can also remove the first and last line to turn it into a fancy bash script. Made by SN4T14

ftp-grab(){
    target="$1"
    wget -r -l 0 -np -nc "$target"
    if "$target" =~ ^ftp://.*$ 
        then
        target="$(echo "$target" | cut -d '/' -f 3)"
        echo "ftp"
        echo "$target"
    fi
    tar cvf $(date +%Y).$(date +%m)."$target".tar "$target"
    tar tvf $(date +%Y).$(date +%m)."$target".tar > $(date +%Y).$(date +%m)."$target".tar.txt
}


Now zip/tar it up and send to the spacious Internet Archive![1] (If you're short on space: tar --remove-files deletes the files shortly after adding them to the tar, not waiting for it to be complete, unlike zip -rm.)

The Project

Who is grabbing what?
Midas ftp.tu-chemnitz.de
Midas ftp.uni-muenster.de
Midas gatekeeper.dec.com
Midas ftp.uni-erlangen.de
Midas ftp.warwick.ac.uk

Uni FTP's are massive, currently only grabbing DEC and Sweex.

External Links