Fileplanet/uploadingfilestoia

Brainstorming on how we can upload the files to IA items.

Remember not to upload the ftp2 stuff publically!

x-archive-meta-title: "Fileplanet_Path_with_underscores". Without the ftp1/2/3 bit, so eg "ftp1/102011/Yes_Man_Dynamic_Theme.7z" would become the item "102011_Yes_Man_Dynamic_Theme.7z", "ftp1/fpnew/patches/prorally2001_v11.exe" would be "fpnew_patches_prorally2001_v11.exe". This might need some care about special characters that IA does not support for item names.

x-archive-meta-description: For now just put the path there. So basically the item name but with special characters intact. TODO later is to add the real metadata we got, shaqfu has a sqlite db with it, otherwise use the fileinfo item: http://archive.org/details/FileplanetFiles_fileinfo_pages_images

x-archive-meta-date: Not sure what format it supports, but this should be the file's original timestamp.

x-archive-meta-year: See above.

x-archive-meta-subject: gaming;software;gaming software;fileplanet;gamespy;ign;planetnetwork

x-archive-meta-collection:archiveteam-fileplanet no idea if this would work with s3 upload

x-archive-meta-mediatype:software (not always correct, maybe check for filename extensions like avi, mov, mp4 etc to set it to movies for those)

Remember not to upload the ftp2 stuff publically!

script draft

 #!/bin/bash
 
 #traverse through subdirectories, generate metadata
 
 commonheaders=' --add-header x-archive-auto-make-bucket:1
 --add-header "x-archive-meta-subject: gaming;software;gaming software;fileplanet;gamespy;ign;planetnetwork"
 --add-header "x-archive-meta-collection:archiveteam-fileplanet"
 --add-header "x-archive-meta-mediatype:software" ' 
 # (not always correct, maybe check for filename extensions like avi, mov, mp4 etc to set it to movies for those?)
 
 echo $commonheaders
 
 tempfile="/tmp/fileplanet_ListOfFiles"
 
 # generate a list of files
 find -type f > ${tempfile}
 
 while read file
 do
 	echo ${file} # has a ./ in front
 	
 	datetime=$(ls -l --time-style=long-iso "${file}" | awk '{print $6" "$7}')
 	#echo ${datetime}
 	year=$(echo ${datetime} | grep -Eo '[0-9]{4}')	
 	echo "--add-header x-archive-meta-date:\"${datetime}\""
 	echo "--add-header x-archive-meta-year:\"${year}\""
 	
 	file=$(echo ${file}| sed 's/\.\///') # remove ./
 	filename=$(basename ${file})
 	ftppath=$(dirname ${file})
 	ftppath=$(echo "${ftppath}/")
 	echo "--add-header x-archive-meta-title:\"Fileplanet Archive: ${filename}\""
 	echo "--add-header x-archive-meta-description:\"${filename}, mirrored from its original location in ${ftppath}\""
 	
 	# from famicoman
 	# IA supports alphanum and _-.
 	itemname=$(echo "Fileplanet_${filename}" | tr ' ' '_' | tr -d '[{}(),\!:?~@#$%^&*+=;<>|]' | tr -d "\'" | sed 's/\//_/g')
 	echo "s3://${itemname}"
 	
 	echo "#################"
 done < ${tempfile}
 
 rm ${tempfile}