Creating a Local PCLinuxOS Repository
by Robert Stahl (Hairyplotter)
Some might ask, "why keep a local repository when everything you need is available on the internet?" This is a valid question, and there are a couple of reasons why someone might want to create one. First, if you have more than one computer with PClinuxOS installed, it makes updating, especially from a fresh install, very fast. If you store your repository on an external hard drive, you can install and update machines that might not be connected to the internet. Or if you're like me, you do it just to say that you have it.
I am not going to get into specific hardware configurations. You can store your repository on a spare computer as long as its connected to the internet, on an external hard drive or even your desktop system. The key to automating the process of synchronizing your repository is that the system you use is running, connected to the internet and the partition you choose to store your files is mounted when cron runs the update script (see below).
Step 1
Plan out the location of your repository and make sure you have enough free disk space. You can determine the amount of free disk space you have on all of your currently mounted partitions by opening konsole then typing df <enter>.
Filesystem Size Used Avail Use% Mounted on /dev/sda1 12G 4.7G 6.6G 42% / /dev/sda6 40G 18G 22G 46% /home /dev/hda 550M 550M 0 100% /media/cdrom0 /dev/sdb1 459G 176G 260G 41% /media/Seagate
Since I have 260Gb of free space on my external hard drive (/dev/sdb1), I will put the repository there. That will also allow me to update other computers when necessary.
WARNING: Make sure your repository is away from any other files and directories. Bury it under a few directories as I did with /media/Seagate/Repository/PCLinux/. The rsync command (see below) will delete any files not found in the repository you sync with. If you place your repository in /, it will delete ALL of your files.
If all you want are the standard repository sections, (main extra nonfree kde4 gnome), then you will need around 15g of free disk space. If you want to add SRPM (source rpms), then you should have between 30-40g free.
Step 2
Choose a repository close to your geographical location. Downloading the initial repository is the longest part of the setup, so I choose to do it in an interactive session. Doing it this way allows me to monitor its progress and interrupt it if necessary. The following is one command, with no line breaks.
rsync -aP --no-motd --stats --exclude=*testing --exclude=SRPM* spout.ussg.indiana.edu::pclinuxos/pclinuxos/apt/pclinuxos/2010/ /home/ftp/pub/linux
- Make sure to replace the bold text with the location you have chosen to house your repository.
- Remove the --exclude=SRPM* if you want the source rpms.
Once this part is finished, I wrote the following script and put it in /usr/local/bin, then created a link to it in /etc/cron.daily* to keep the repository updated daily and maintain a detailed log of file transfers.
* ln -s /usr/local/bin /etc/cron.daily
#!/bin/bash
#--------------[ Change Log ]--------------
# 3/10/2010 Added --delete to the rsync command that was omitted by mistake.
# Added $REPFILE variable to hold the location of the repository.
#
#
#
#
#
#
#--------------[ To Do List ]--------------
#--
#--
#--
#--
#--
#----------------------------------------------------------
# Function to perform some housekeeping before exiting.
#----------------------------------------------------------
function cleanup {
#
# Did we create a temporary file?
if [ -f $TMPFILE ]; then
# YES, then delete it before we exit.
rm $TMPFILE
fi
# Terminate the script with a return code of 0 (normal termination) or any other number (abnormal termination).
exit $RETVAL
}
# name the file we are going to log our output to.
REPFILE="/media/Seagate/Repository/PCLinux"
LOGFILE="/media/Seagate/Repository/rsync.log"
# Create a temporary file.
TMPFILE=`mktemp -t rsync.XXXXXXXXXX`
# Was the temporary file created without errors?
if [ $? -ne 0 ]; then
# NO, print a message to our log file then terminate.
printf "\n********** `date +'[%A %b %d %Y] - [%r]'` **********\n" >> $LOGFILE
printf "Repository update FAILED: Unable to create temporary file\n" >> $LOGFILE
RETVAL=1
cleanup
fi
# Set our file counter to 0
COUNT=0
# Call rsync to sync our repository with spout.ussg.indiana.edu.
# -a: This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything.
# -P: Show progress. This isn't very useful in a non-interactive session, but its nice when we want to sync interactively.
# --no-motd: Most servers have an motd, we don't need to see it so have rsync ignore it.
# --stats: Give us some extra file transfer stats. Good during an interactive session.
# --exclude=<pattern>: Ignore directories matching <pattern>. In this case, testing, kde4 and SRPM (source rpms).
# --log-file=: Log output to the temporary file we created above.
rsync -aP --no-motd --stats --delete --exclude=*testing --exclude=*kde4 --exclude=SRPM* --log-file=$TMPFILE spout.ussg.indiana.edu::pclinuxos/pclinuxos/apt/pclinuxos/2007/ /home/ftp/pub/linux 2>/dev/null
# get the return value from rsync and assign it to RETVAL for later use.
RETVAL=$?
# Did rsync terminate without errors?
if [[ $RETVAL -ne 0 ]]; then
# NO, there was a problem with rsync. Write a FAILED notice to our log file, then exit.
printf "\n********** `date +'[%A %b %d %Y] - [%r]'` **********\n" >> $LOGFILE
case $RETVAL in
1) REASON="Syntax or usage error";;
2) REASON="Protocol incompatibility";;
3) REASON="Errors selecting input/output files, dirs";;
4) REASON="Requested action not supported";;
5) REASON="Error starting client-server protocol";;
6) REASON="Daemon unable to append to log-file";;
10) REASON="Error in socket I/O";;
11) REASON="Error in file I/O";;
12) REASON="Error in rsync protocol data stream";;
13) REASON="Errors with program diagnostics";;
14) REASON="Error in IPC code";;
20) REASON="Received SIGUSR1 or SIGINT";;
21) REASON="Some error returned by waitpid()";;
22) REASON="Error allocating core memory buffers";;
23) REASON="Partial transfer due to error";;
24) REASON="Partial transfer due to vanished source files";;
25) REASON="The --max-delete limit stopped deletions";;
30) REASON="Timeout in data send/receive";;
35) REASON="Timeout waiting for daemon connection";;
*) REASON="Undefined error";;
esac
printf "Repository update FAILED with error code $RETVAL: $REASON\n" >> $LOGFILE
RETVAL=2
cleanup
else
# YES, rsync finished without errors.
# get the number of package files transferred.
TRANSFERRED=`cat $TMPFILE | grep -c .rpm`
# Were any files transferred durning this update?
if [ $TRANSFERRED -gt "0" ]; then
# YES, we had some files transferred.
# Add a note to the logfile detailing the current time and the number of files transferred.
printf "\n********** `date +'[%A %b %d %Y] - [%r]'` **********\n" >> $LOGFILE
printf "Repository update SUCEEDED with $TRANSFERRED files transferred.\n" >> $LOGFILE
printf "Number\tSection\t\tFilename\n" >> $LOGFILE
echo "-------------------------------------------" >> $LOGFILE
else
# NO, there were no files transferred durning this update.
# No need to add a note to the log file.
# Call the cleanup function which will terminate this script after some housekeeping.
RETVAL=0
cleanup
fi
# Since we had some files transferred, we will parse our temp file line by line to get the filenames.
cat $TMPFILE | while read line
do
# Does this line have .rpm in it (denoting a package file)
FILE=`echo $line | grep .rpm`
if [ ! -z "$FILE" ]; then
# YES, lets increment our counter by 1 then write that number along with the section and filename transferred to our log file.
COUNT=$(($COUNT+1))
FILE=`echo $FILE | awk '{ print $NF }'`
SECTION=`echo $FILE | cut -d'/' -f3`
FILENAME=`basename $FILE`
printf " $COUNT:\t$SECTION\t$FILENAME\n" >> $LOGFILE
fi
done
fi
RETVAL=0
cleanup
Step 3
Now that I had the repository downloaded and ready to go, I needed to set up an ftp server to make the files available to all of my machines.
I started by installing vsftpd from the repositories.
After doing some reading on the vsftpd configuration file, I opened /etc/vsftpd/vsftpd.conf with kwrite and added:
*Change the text in bold to the location of your FTP.
# Allow anonymous FTP? (Beware - allowed by default if you comment this out). # I want a read only anonymous ftp. anonymous_enable=YES anon_root=/home/ftp anon_upload_enable=NO anon_mkdir_write_enable=NO anon_other_write_enable=NO anon_world_readable_only=YES cmds_allowed=PASV,PWD,LIST,TYPE,MKD,CWD,EXIT,QUIT,HELP,RETR pasv_min_port=10450 pasv_max_port=10500 pam_service_name=vsftpd userlist_enable=YES tcp_wrappers=YES use_localtime=YES
I set vsftpd to start when the computer is turned on through PCC -> System -> Manage system services by enabling or disabling them. I opened Synaptic on my other machines, and went to "settings" -> "repositories", unchecked the default repository, then clicked "New."
URI: 192.168.1.105/pub/linux Distribution: pclinuxos/2010 sections: main extra nonfree kde gnome
Click "Ok", then "Reload." You should now have your very own PCLinuxOS repository that keeps itself up to date.
*NOTE: 192.168.1.105 is the IP address I manually assigned to my spare machine. I used FTP as my means of serving the repository, but its not the only method available.
