How to save your rpm's for future reuse in Mandriva. By Glenn Waller 28th April 2010.
Why did I decide to reuse the downloaded rpm's (Redhat Package Management)?
To save time, connectivity problems (Internet) Bandwidth and embarrassment after I broke the system. I have added my scripts with explanations to the end of this paper but they are customised for my system, so read and edit them before trying to use them. ;-)
How I started reusing the saved rpm's.
Including, Mandriva installation choice of copying the data to HD, before the package selection process, was saved in /var/ftp/pub/mandriva(ver)...
Mostly, back then it was for large packages that I would reuse, such as the kernel-source (50Mb, approx) and the plf (Mandriva's Penguin Liberation Front) rpm's for video codecs.
Then I found genhdlist and soon after genhdlist2, and wrote a bunch of commands to a text-file and made a shell script out of it.
I partitioned my hard-drives to give me some space that would not be formatted during an install, Mandy generally requires formatting of /usr and / (root). This maybe a GNU/Linux requirement across the board of the many distributions (distros)
After I found genhdlist2 was I able to usefully add my local repositions (repos) to drakRPM, to make them available when using smart-packager and urpm(i), And reuse them at will, regardless of when I broke it and, and keep moving forward.
Using the no-clean hack in the GUI has never been reliable on my system, as it seems to reset it's self after each invocation of the program. So I took a different track, I wanted to save the rpms while at the same time being able to view the selected packages and dependencies (deps) before committing to the changes.
So, I would painstakingly copy and paste the confirmation text from Drakrpm to a plain text file and remove the new-line and dash+space at the beginning of each new line to create a space separated list that urpmi could work with. After the copy and paste and before running the list with urpmi –noclean I would close Drakrpm cancelling the process, but my list of packages was safe, and usable.
I still have not figured out how to keep the signatures for the packages, although I have tried using some guides from the web. But I guess if they were fine when initially downloaded, they'll still be fine unless I get a hard-drive error, which has not occurred for quite some time (Fingers crossed).
The list was generally discarded after use, generally because of the gross amount of redundant files I would end up with, sometimes it would take upto an hour to edit the list (1340 odd packages, lines) so I've been hunting for a way to make this more expedient and reliable. And I found sed.
Editing the list was accomplished from the bottom to the top, so I was not continually working at the end of the line, but the beginning of the line to save scrolling, with the mouse to high-light the characters I wanted/needed to remove and the space bar.
“Two hands, two guns!” (sifu Malcolm Sue)
Now with all of that it only takes a second to convert the list and I can enjoy my time doing what ever else I like.
I hope someone else finds this helpful, cheers and kind regards, Glenn (GlennsPref@gmail.com)
Scripts with explanations....
The first was a backup script which only collected the downloaded rpms from /var/cache /urpmi/rpms to a directory safe within /home/glenn/....I have many partitions, don't know why, Maybe just because I can, lol.
glenn@GamesBox:~$ df
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 20G 1.8G 17G 10% /
/dev/sda1 92M 59M 28M 68% /boot
/dev/sda14 383G 232G 151G 61% /home
/dev/sdc7 33G 176M 33G 1% /home/glenn/local/Music
/dev/sdd5 52G 180M 52G 1% /home/glenn/local/Video
/dev/sdb8 73G 61G 13G 84% /home/glenn/local/spare
/dev/sdc1 56G 180M 55G 1% /home/glenn/local/stuff
/dev/sdb7 34G 31G 2.9G 92% /home/glenn/local/vm1
/dev/sdb9 74G 51G 23G 69% /home/glenn/stuff
/dev/sdb1 22G 13G 9.4G 57% /media/win_c
/dev/sdd6 31G 14G 18G 44% /media/win_c2
/dev/sdb6 29G 19G 9.8G 66% /media/win_d
/dev/sdd7 63G 23G 41G 36% /media/win_d2
/dev/sda11 5.0G 138M 4.6G 3% /opt
/dev/sda8 8.0G 157M 7.4G 3% /tmp
/dev/sda7 9.9G 5.3G 4.1G 57% /usr
/dev/sda10 5.0G 139M 4.6G 3% /usr/local
/dev/sdc6 20G 520M 18G 3% /usr/src
/dev/sda6 20G 911M 18G 5% /var
/dev/sdc5 3.5G 72M 3.3G 3% /var/ftp
/dev/sda13 5.0G 139M 4.6G 3% /var/spool
/dev/sda12 2.0G 56M 1.9G 3% /var/www
glenn@GamesBox:~$
That is all it did.
Of course to simplify the editing of this script, it calls other scripts that do most of the work,
and it's still a “work in progress”(wip). It has some text to put up on the screen to give some
feedback to the user of what stage the process is up to, and although some of them are duplicated in the other scrips, it works quite well (IMHO).
#! /bin/zsh
#echo "/home/glenn/bin/backup-rpms"
echo "/home/glenn/bin/backup-rpms"
echo
sudo sh /home/glenn/build/rpms/organise-rpms
echo "call organise-rpms....."
echo "Backup-rpms"
#echo "generating new hdlist.cz for local archives........"
sudo sh /home/glenn/build/rpms/gen-hdlist2
echo
echo "Backup-rpms"
echo "rebuilding local rpm and file databases... this might take a while ........"
sudo rpm --rebuilddb && sudo updatedb
echo
echo "Backup-rpms"
echo "local database up to date ........"
echo
echo
sudo sh /home/glenn/build/rpms/update-rpm-database
echo "run rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt, maybe upto 15kb text."/n
sudo rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt
echo "run rpm -Va | grep miss & send to /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt"
sudo rpm -Va | grep miss > /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt
echo "clean rpm cache"
sudo smart clean
sudo rm -rf /var/cache/urpmi/rpms/*rpm
echo ""
echo "all repositories are now up to date, any errors? see above!"
date && whoami
The first thing it does is call organise-rpms from ~/build/rpms...
sudo sh /home/glenn/build/rpms/organise-rpms
This script finds rpms where they maybe stored by me, and the programs DrakRPM and
Smart. (I really like Smart, it is a synaptic-like package manager although it has been changed over
the years, and is now much more useful, but harder to hack, like saving the rpms (Grief)).[/code]
#! /bin/zsh
#echo "/home/glenn/build/rpms/organise-rpms"
echo "organise-rpms"
echo "Called by, ~/bin/backup-rpms"
echo "organising saved rpm cache for local archives....."
#1st Level
sudo mv -v /var/cache/urpmi/rpms/*2010.0*rpm /home/glenn/local/Archive/2010.0/noarch/
sudo mv -v /var/cache/urpmi/rpms/*2010.1*rpm /home/glenn/local/Archive/2010.1/noarch/
sudo cp -v /var/lib/smart/packages/*2010.0*rpm /home/glenn/local/Archive/2010.0/noarch/
sudo cp -v /var/lib/smart/packages/*2010.1*rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /var/cache/urpmi/rpms/*rpm /home/glenn/local/Archive/2010.1/noarch/
sudo cp -v /var/lib/smart/packages/*rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/Download/*rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/Download/packagesNsource/*rpm
echo "filter for 2010.1 packages"
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.0.x86_64.rpm /home/glenn/local/Archive/2010.0/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*plf2010*.rpm /home/glenn/local/Archive/2010.1/plf/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*plf2010*.rpm /home/glenn/loca/Archive/2010.1/plf/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/kde430/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/kde430/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.0/i586/*mdv2010.1.i586.rpm /home/glenn/local/Archive/2010.1/i586/
echo "cleanup-stagglers"
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.1.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.1.i586.rpm /home/glenn/loca/Archive/2010.1/i586/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*plf2010.1.x86_64.rpm /home/glenn/local/Archive/2010.1/plf/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*plf2010.0.x86_64.rpm /home/glenn/local/Archive/2010.0/plf/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*mdv2010.0.noarch.rpm /home/glenn/local/Archive/2010.0/noarch/
sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*2010.0.noarch.rpm /home/glenn/local/Archive/2010.0/noarch/
sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.0/contrib/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/contrib/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/plf/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/i386/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.1/i586/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/
sudo mv -v /home/glenn/local/Archive/2010.0/contrib/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/
date && whoami
This script also calls another file to change the permissions so I can use them with out errors
such as read-only.
#! /bin/zsh
echo "Backup-rpms"
echo "~/build/rpms/rpms-chmod"
sudo chmod -R 755 /home/glenn/local/Archive/2009.0/
sudo chmod -R 755 /home/glenn/local/Archive/2009.1/
sudo chmod -R 755 /home/glenn/local/Archive/2010.0/
sudo chmod -R 755 /home/glenn/local/Archive/2010.1/
date && whoami
Next the script calls for genhdlist2 to recreate the files required for DrakRPM and Smart
package data-bases, hdlist.cz
#! /bin/zsh
echo "Backup-rpms"
echo "generating new hdlist.cz for local archives........"
echo ""
echo "Backup-rpms"
echo "generating new hdlist.cz for local archives........"
# echo "Backup-rpms local/Archive/2009.0/i386"
# sudo genhdlist2 --xml-info -v --allow-empty-media /home/glenn/local/Archive/2009.0/i386
# echo "Backup-rpms local/Archive/2009.1/noarch"
# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/noarch
# echo "Backup-rpms local/Archive/2009.1/i586"
# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/i586
# echo "Backup-rpms local/Archive/2009.1/i586/contrib"
# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/i586/contrib
# echo "Backup-rpms local/Archive/2009.1/i386"
# sudo genhdlist2 --xml-info -v --allow-empty-media /home/glenn/local/Archive/2009.1/i386
# echo "Backup-rpms local/Archive/2009.1/x86_64"
# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/x86_64
# echo "Backup-rpms local/Archive/2009.1/kde430"
# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/kde430
# echo "Backup-rpms local/Archive/2009.1/x86_64/contrib"
# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/x86_64/contrib
echo "Backup-rpms 2010.0"
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/x86_64
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/noarch
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/i586
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/i386
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/contrib
echo "Backup-rpms 2010.1"
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/x86_64
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/noarch
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/i586
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/i386
sudo genhdlist2 --clean --xml-info -v --allow-empty-media /home/glenn/local/Archive/2010.1/contrib
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/kde
echo "Backup-rpms plf"
# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.0/plf
# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/plf
sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/plf
sudo genhdlist2 --clean --xml-info -v --allow-empty-media /home/glenn/local/Archive/2010.1/plf
echo "GenHdList2 up to date!"
date && whoami
As new versions of Mandy come out, new lines are added and the old ones removed, as seen
above (#) commented out...
If genhdlist finds a corrupted package it stops and shows and error, naming the package,
thanks!
Next it refreshes the rpm database of installed packages, and the find data-base too.
rpm --rebuilddb && sudo updatedb
This takes a while....
The next task is to update the external rpm repo data-bases....
#! /bin/zsh
#echo "/home/glenn/bin/backup-rpms"
echo "/home/glenn/build/rpms/update-rpm-database"
echo "Called from, ~/bin/backup-rpms"
echo
echo
echo
echo
echo "updating urpmi rpm database....."
echo
echo
echo
#echo "urpmi.update local repos"
#sudo urpmi.update local
#echo "uncomment lines to run update all, skipped for now, GW"
echo "urpmi.update all"
sudo urpmi.update -a
#echo "smart update"
#sudo smart update
echo
echo "Backup-rpms"
#echo "local repositories are up to date .. GamesBox.GlennsPref.net........Have a nice day!"
echo "urpmi is up to date .. GamesBox.GlennsPref.net........Have a nice day!"
date && whoami
smart update is no longer required, as it reads the URPM data-bases
The last things to run is a clean up, and checking for missing packages and a catalogue of
installed rpms for the system.
sudo rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt
echo "run rpm -Va | grep miss & send to /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt"
sudo rpm -Va | grep miss > /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt
echo "clean rpm cache"
sudo smart clean
sudo rm -rf /var/cache/urpmi/rpms/*rpm
rpm -Va | grep miss, takes a while to finish, but can be well worth the wait, before rebooting and
having a crash or cli only system because of a missing dep or lib.
The output is redirected to a file and the file name is created with the date and hour it was
written.
Now the reason I written this is because of a sed trick I found that converts the list to
something urpmi can work with, This process is not completed yet, more things will occur to me as time rolls on,
#!/bin/zsh
echo "Create a space-separated-list from top-down list with preceeding chars."
cd ~/build
echo "remove old files"
rm -f ~/build/file2.txt
rm -f ~/build/file3.txt
echo "Remove - (dash space) from lead of lines using sed...."
echo "sed 's/^[-\ ]*//' file1.txt (from file1.txt in pwd dir)"
echo "redirect output to new file...file2.txt"
sed 's/^[-\ ]*//' file1.txt > file2.txt
echo "Now replace carriage returns with a space... using file2.txt, create file3.txt"
sed ':a;N;$!ba;s/\n/ /g' file2.txt > file3.txt
First it removes the old files, to save stopping for errors, file exists.
Then it removes the dash space from the head of the lines...
And lastly it replaces the newline char with a space, bringing all the list to one line, sweet!
I'd like to reference where most of my info came from,
MDE repo http://www.mde.djura.org/index.html and
SoS repo http://seerofsouls.com/ (website down atm), help files...
Few years old now but still very useful.
http://sed.sourceforge.net/sed1line.txt
And that's IT!
Any questions? Did I miss something? Please ask!
Regards Glenn.
No comments:
Post a Comment