26 November 2010

The Evolution of MandY

The evolution of Mandriva, or is that Mandrake...? I'm not sure, But I know I find it easy to use and easy to configure. http://mageia.org/en/
From web browsing, file managing, database and/or presentations, to spread-sheets with graphs. It's all here. "Back-up able" and "recoverable". tip; use a filesystem that is recoverable, encrypted or not.
You can decide on a media centric install, or a web-media (rss-wget-rsync) centric, maybe a science centric install......?GlennsPref@gmail.com

Works for me ;) GW

04 November 2010

Mandriva security and sudo

Mandriva security and sudo, MSEC. GlennsPref@gmail.com

Hi, I've read some discussions here and elsewhere about Mandriva security and the use of sudo.

As well as editing /etc/sudoers, and making yourself ($USER) part of the wheel group, you may require a tighter security than that of the default Mandriva system setup.

Warning! This is a one user system, so some selections may not apply to your system, be careful, if you don't know what a setting does Leave it at the default.

I've been using Mandriva for a few years now, and I generally do these tasks mentioned here, with every update of sudo and the MSEC tool packages.

To be honest I don't know where the config file is for MSEC, and that may be just as well. ;-)
[code]/etc/security/msec[/code]
So to ease my fingers and kb, I have snapshots (.png) of the settings I have made.

After you've done all of this, a reboot is required to put the settings in place, ( like sudo
and group changes).

/etc/sudoers
===========================================================================
# sudoers file.
#
# This file MUST be edited with the 'visudo' command as root.
# Failure to use 'visudo' may result in syntax or file permission errors
# that prevent sudo from running.
#
# See the sudoers man page for the details on how to write a sudoers file.
#

# Host alias specification

# User alias specification

# Cmnd alias specification

# Defaults specification

# Runas alias specification

# User privilege specification
root ALL=(ALL) ALL

# Uncomment to allow people in group wheel to run all commands
%wheel ALL=(ALL) ALL

# Same thing without a password
%wheel ALL=(ALL) NOPASSWD: ALL

# Samples
%users ALL=/sbin/mount /cdrom,/sbin/umount /cdrom
%users localhost=/sbin/shutdown -h now

=================================================================
snapshot0-MSEC-system-security-and-audit.png


snapshot1-Basic+security.png



snapshot2-System-security-top.png



snapshot3-System-security-bottom.png



snapshot4-Network-security.png



snapshot5-Periodic-checks.png


snapshot6-Periodic-checks-bottom.png


snapshot7-Exceptions(none).png


snapshot8-Permissions(all).png



snapshot9-Users-and-Groups.png



snapshot10-Users-and-Groups-bottom.png



snapshot11-Groups-bottom.png



snapshot12-Groups-top.png


Don't forget to reboot for permission changes for groups and sudo.

I hope this helps you, Regards Glenn :-)

28 May 2010

What I'm reading

electonics docs (.pdf)... http://www.analogzone.com/tech_fram.htm

Understanding Qigong
What is Qigong?
The Human Qi Circulatory System
by Dr. Yang, Jwing-Ming

Phylosophy... http://www.vl-site.org/taoism/index.html

in particular.... http://www.vl-site.org/taoism/ttcstan1.html

Your most basic Human Rights

Seems I've said, or restated, this else where, time it was here too.


Your most basic Human Rights.

You have the right to...

be treated with respect,
make mistakes,
say NO and not feel guilty,
experience and express your feelings,
take time to slow down and think,
change your mind,
do less than you are humanly capable of,
ask for information,
feel good about your self.

;-)

14 May 2010

How to recover lost mp3's for free

Glenn Waller May 2010
How to recover lost mp3's for free.
suitable for one user system users, administrators and technicians
time to do? 45Min approx.
programs used,
GNU/Linux (Mandriva 2010.1) Kernel 2.6.32.3-2mnb_x86_64
Photorec partition data recovery tool
find to list and sort files
EasyTag to recover file names from tags

You'll need root access to use Photorec, su or sudo to root in your shell, type in the root password if required. I think it's because of access to raw (unmounted) file system data. The files will be written as (user and group) root, so we fix the permission changes during this tutorial (you need root access to do this too) back to user-name group and user before doing too much more. This way we can use user-land instead of having to stay as root for long periods of time.

Scenario
Msec was deleting unowned files in the /home partition

moved mp3's off etx3 partition to fat32 partition to share whilst working within windows xp land
and out of Msec's long hands. Eventually I had no access to the files permissions from
GNU/Linux. During previous attempts to fix the permission problem, I had copied the data off the
disk, changed the attributes under ext4 (this time), formatted the disk and copied the...data.....ack.
anyway, it didn't work.

The task begins.
Being a long time advocate for Photorec in the past, (recovering music from old damaged cdroms) I knew I could (probably) almost certainly recover the music files from the ext4 formatted partition. There were also a lot of other files on there too, like a backup of HalfLife2 CC:S downloads and games, maps, etc, about 8.5 to 12 gig, and much more.
First things first, you need a partition equal to or greater than in size of the partition you wish to recover. Next you need to run Photorec from the directory from where you wish to store the files Photorec finds. So, make a directory on the partition you are using for the recovered files and cd to the recovery directory, like this (you'll need to change the absolute address to match your folder names)
[code]
mkdir /home/glenn/storage/recovery
cd /home/glenn/storage/recovery

Unmount the drive the data is being recovered from, (/home/glenn/stuff in my case)

/etc/fstab...
# Entry for /dev/sdb9 :
UUID=ef444615-fcc7-438d-8711-57fc1571b4c1 /home/glenn/stuff ext4 relatime 1 2

check /etc/fstab in a text file reader, like cat. # Entry for /dev/sdb9 :

sudo umount /dev/sdb9

If you can't unmount the drive you may be able to use a bootable GNU/Linux flash-stick, like slax and knoppix live distributions.

I pulled up Photorec on the screen, cli mode, type 'photorec' at the prompt. (reminder, you need to be root)
What is displayed on the screen is a list of physical drives on the system, navigate with the cursor keys and select(hit enter to continue) the drive the data you want to recover may be found (Last Know Address). Make sure you highlight the drive selection, with the up and down arrows and select continue or abort with the left or right arrows, to save unintentional aborting.
The next screen list some details about the drive, continue to the next section about the partition table types, I have a PC and selected the Intel platform. (please don't ask me about the other partition types, I don't know) Now here it gets to the partition level, a note of warning...
Be Careful Here it's easy to begin the recovery of ALL the data on the partition, but I only want songs mp3, wav, ogg, and pictures bmp, jpg, gif, png.
So if you only want certain file types, select 'options' down near the bottom of the partition list (we'll select the partition after we set the file extension filter)
Here you can set the paranoia mode, low ram, and other setting better suited to Forensic file recovery. I usually leave this at the defaults, we just want to do a late backup.
Hit enter to 'quit'.
Scroll across to the 'File Opt' tab and enter into the extension list. Using the arrow keys and space bar to mark the grid for the types you want Photorec to search for.
“Press s for default selection, b to save the settings”
Saving this creates a file that may be called again when you next use Photorec (as root).
Once you had a good search through the list and marked the file types you want return to the partition page and use the arrows to scroll down to the partition you want, in my case .../sdb9.
Photorec is now ready to start, if you are ready highlight the 'search' function and hit enter. You should see a list of file types appear on the screen, they appear as they are found, so don't abort if you don't see the file types you selected in 'File Opt' as soon as it starts. It should also display how long it will take to complete the search. The files still remain on the old partition too in case you need to search harder. Photorec took 21 min to scan 54gig of data spread over a 73Gb partition. And save them neatly in the present working directory (/home/glenn/storage/recovery)

This is the end screen of a “second scan” without 'File Opt' being edited, approx 28minutes

PhotoRec 6.11, Data Recovery Utility, April 2009
Christophe GRENIER
http://www.cgsecurity.org

Disk /dev/sdb - 250 GB / 232 GiB (RO) - ATA WDC WD2500AAKS-0
Partition Start End Size in sectors
9 L Linux 20859 1 1 30400 254 63 153292167 [stuff]


17647 files saved in /home/glenn/storage/recovery/recup_dir directory.
Recovery completed.
txt: 8728 recovered
mp3: 3345 recovered
mpg: 2351 recovered
gif: 1513 recovered
jpg: 944 recovered
flv: 524 recovered
mov: 91 recovered
png: 77 recovered
ogg: 60 recovered
gpg: 6 recovered
others: 8 recovered

you need to quit each page until you are back in the bare shell.

Post-recovery clean-up
Prior to sorting and renaming, and the last job we need to do as root, is change the file permissions to my user and group. In my case, user glenn and group glenn with chown and chmod to make them readable,writeable and executable.
[code]
sudo chown -hR glenn:glenn /home/glenn/storage/recovery
#change user and group to glenn
sudo chmod -R 775 /home/glenn/storage/recovery
#make writeable by a program and user

…..type exit and hit enter to quit the shell as root.

Using find to list and sort files
ref. Shawn Hermans
I wanted to copy all the mp3 files to a directory and also filter off the files I thought would be too small to be useful as an mp3 with any quality, I set it for 2Mb 2048kb's and also made a dir small for the small files that I would not bother renaming.
[code]
mkdir /home/glenn/storage/recovery/mp3
mkdir /home/glenn/storage/recovery/small


find does not use the standard sequence off directives in GNU/Linux
use 'man find' and 'man xargs' to get the lowdown on these tools. Type q to quit at any time from the man page.

Excerpt from man find...find uses 'find source arguments expression destination'...

NON-BUGS
$ find . -name *.c -print
find: paths must precede expression
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]

This happens because *.c has been expanded by the shell resulting in find actually receiving a
command line like this:

find . -name bigram.c code.c frcode.c locate.c -print

That command is of course not going to work. Instead of doing things this way, you should
enclose the pattern in quotes or escape the wildcard:
$ find . -name '*.c' -print
$ find . -name \*.c -print

The xargs manpage reveals more on how the find command works....
man xargs excerpt...

EXAMPLES
find /tmp -name core -type f -print | xargs /bin/rm -f

Find files named core in or below the directory /tmp and delete them. Note that this will
work incorrectly if there are any filenames containing newlines or spaces.

find /tmp -name core -type f -print0 | xargs -0 /bin/rm -f

Find files named core in or below the directory /tmp and delete them, processing filenames in
such a way that file or directory names containing spaces or newlines are correctly handled.

find /tmp -depth -name core -type f -delete

Find files named core in or below the directory /tmp and delete them, but more efficiently
than in the previous example (because we avoid the need to use fork(2) and exec(2) to launch
rm and we don't need the extra xargs process).

Oh, and the i means [-i[replace-str]] Replace string


Because of type setting and formatting of text I tried to use a back-slash (\) here to continue code lines, It's not very effective in the my zshell, bash may be different. You may need to copy the lines to a text editor like kate to edit/join them up, just one space is enough, and no spaces at the end. ;-)

find /home/glenn/storage/recovery/ -name "*.mp3" | xargs -i mv {} /home/glenn/storage/recovery/mp3/
find /home/glenn/storage/recovery/ -name "*.ogg" | xargs -i mv {} /home/glenn/storage/recovery/mp3/
find /home/glenn/storage/recovery/ -name "*.wav" | xargs -i mv {} /home/glenn/storage/recovery/mp3/

This code moves files from the mp3 directory that are less than 2048kb (2Mb) Most 2.5 to 3 minute mp3s are about +2.5Mb

minimum file size...

find /home/glenn/storage/recovery/mp3/ -name "*.mp3" -size -2048k | xargs -i mv {} /home/glenn/storage/recovery/small/
find /home/glenn/storage/recovery/mp3/ -name "*.ogg" -size -2048k | xargs -i mv {} /home/glenn/storage/recovery/small/
find /home/glenn/storage/recovery/mp3/ -name "*.wav" -size -2048k | xargs -i mv {} /home/glenn/storage/recovery/small/

Now, we have sorted the files we want to work with from the files we don't want to bother with.

Next we want to get most of our file names back, For this I found EasyTag, a a tag search like mp3info with a very handy bulk scanning and renaming tool with a graphical user interface.

Example of reading data tags with mp3info...

mp3info -F /home/glenn/storage/recovery/mp3/f81018974.mp3
File: /home/glenn/storage/recovery/mp3/f81018974.mp3
Title: The River Track: 6
Artist: Bruce Springsteen & The E Stre
Album: Live In New York City (Disc 1) Year: 2001
Comment: Genre: Other [12]


Retrieving Filenames
ref. http://easytag.sourceforge.net/ EasyTag 2.1.6
http://en.wikipedia.org/wiki/EasyTag

Using Mandriva 'urpmi easytag' will install the package.

Dependencies....
Checking the sourceforge site for EasyTag, the dependent libraries are listed as such

The program depends on the following libraries:
GTK 2.4,
id3lib to handle MP3 files,
libogg,
libvorbis for Ogg Vorbis files,
libFLAC (with libogg and libvorbis) for FLAC Vorbis files,
mpeg4ip for MP4 files.


urpmi installed the package EasyTag with no questions, but due to previous experience I went ahead and checked to see if I could find them. I did not find them all, some of the library names have changed and some were plain missing. I used a combination of locate, urpmq and urpmf to search for the libs and subsequent dependants. And also “Install and Remove Software” to see the info on the package I could not find. Here are my findings... an explanation follows...


EasyTag deps....

#GTK 2.4,
#id3lib to handle MP3 files, (optional, except to write ID3v2.3 tags in MP3 files),
#libogg,
#libvorbis forsupport for Ogg Vorbis and flac files,
#libFLAC (with libogg and libvorbis) support for FLAC Vorbis files,
#mpeg4ip support MP4/ACC file
#libid3tag(required)
#libxcb-xlib
#lib64amrnb3
#lib64arts1
#lib64mpeg4ip0
#lib64xvid4
#end deps list (may not be complete)glib-devel?
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

libs
# libavcodec.so.51
#locate libavcodec.so.5
#copy and rename
#/usr/lib/libavcodec.so.52
#/usr/lib/libavcodec.so.52.66.0
#copy and rename
sudo cp /usr/lib64/libavcodec.so.52 /usr/lib64/libavcodec.so.51
#/usr/lib64/libavcodec.so.52.66.0
# package-lib64ffmpeg52

# libdirect-1.0.so.0
#copy and rename
#locate libdirect-1
#/usr/lib64/libdirect-1.2.so.0
#/usr/lib64/libdirect-1.2.so.0.8.0
sudo cp /usr/lib64/libdirect-1.4.so.0 /usr/lib64/libdirect-1.0.so.0
#/usr/lib64/libdirect-1.4.so.0.3.0

# package-lib64directfb1.4, install previous versions of the 1.4 packages first.
#because the latest ver of the package only has the shared files, not the libs.
#libdirectfb-1.0.so.0 locate libdirectfb-1
#copy and rename
sudo cp /usr/lib64/libdirectfb-1.4.so.0 /usr/lib64/libdirectfb-1.0.so.0 #/usr/lib64/libdirectfb-1.2.so.0.8.0
#/usr/lib64/libdirectfb-1.4.so.0
#/usr/lib64/libdirectfb-1.4.so.0.3.0

# libfusion-1.0.so.0
#locate libfusion-1
sudo cp /usr/lib64/libfusion-1.2.so.0 /usr/lib64/libfusion-1.0.so.0
#/usr/lib64/libfusion-1.2.so.0.8.0
#/usr/lib64/libfusion-1.4.so.0
#/usr/lib64/libfusion-1.4.so.0.3.0

# libmp4v2.so.0
#locate libmp4v2
#copy and rename
sudo cp /usr/lib64/libmp4v2.so.1 /usr/lib64/libmp4v2.so.0
#/usr/lib64/libmp4v2.so.1.9.1

# libxcb-xlib.so.0, libxcb-res0 replaces this package
#copy and rename
#locate libxcb-res
#/usr/lib64/libxcb-res.la
#/usr/lib64/libxcb-res.so
sudo cp /usr/lib64/libxcb-res.so.0 /usr/lib64/libxcb-xlib.so.0
#/usr/lib64/libxcb-res.so.0.0.0



The majority of the problem libraries were version labels, except one that I found had a replacement with a new name, so effectively these libs just needed renaming to allow the program EasyTag to work. I've left the required lines uncommented(#). You will need root access to change these files. I thought about linking the files rather than copying them with new names and decided that at the rate of development with Mandriva and GNU/Linux in general it would make little difference to my system, but feel free to experiment (ln and ln -s instead of cp) This way, at least, the system should be able to find and use the libs when called by the program.
In short...


sudo cp /usr/lib64/libavcodec.so.52 /usr/lib64/libavcodec.so.51
sudo cp /usr/lib64/libdirect-1.4.so.0 /usr/lib64/libdirect-1.0.so.0
sudo cp /usr/lib64/libdirectfb-1.4.so.0 /usr/lib64/libdirectfb-1.0.so.0
sudo cp /usr/lib64/libfusion-1.2.so.0 /usr/lib64/libfusion-1.0.so.0
sudo cp /usr/lib64/libmp4v2.so.1 /usr/lib64/libmp4v2.so.0
sudo cp /usr/lib64/libxcb-res.so.0 /usr/lib64/libxcb-xlib.so.0


Using the EasyTag program
When first started the program offers little until a directory has been scanned. Make sure the directory is writeable by the user (you) and the select the folder in the browser section of the window. EasyTag will now scan the files for the tag information of all the available files in that directory, don't worry it won't do anything else yet. Now the buttons for options appear usable. The directory I used (/home/glenn/storage/recovery/mp3) has 2775 files that were recovered from the formatted disk, some of which were deleted by MSEC.
Go to the 'scanner' menu and select 'Rename file/s and directory', this shows a dialogue where you can type or select from the drop down list what tags to search for (and in what order) and eventually use for renaming. Check the website for details. Also the settings for the scanner and re-namer are available here too, and by selecting the 'Scanner Options' icon, the settings panel appears. I have not changed much here, except to replace the %20 strings with a space. A helpful information comment appears when the mouse cursor is over an option to help understanding. I have chosen create directory-name/file-name, %a - %b/%n - %t. (%a = Artist, %b = Album, %n = Track number, %t = Title) On revision, %a/%b/%n-%t, may suit better, artist/album/track-filename. ;-) That's what I use now. :-)
This process takes much longer, but is required to read the tags from the files. Once it has finished, do not go to the 'file' menu and 'select all', It Crashes the program, Instead go to the 'file' menu and select 'sort by tag', 'browse results by artist'. Select a group of files by one artist then select 'save'. This should rename the files and create directories for the files, from the tags. Repeat this action for all of the individual artists. That tool care of 80% of my files and saved me a lot of time. To clean up the rest (about 600 files) I used Kde's Amarok. Here I listened to the files and renamed them, amazingly, Amarok had some of the tags right before I started. Cool, more time saved.
I'm still sorting through the list, many of the files are from Hl2/CS:S background sounds.

I hope this helps you too. :-)

Cheers and Regards Glenn

How to save your rpm's for future reuse in Mandriva.

How to save your rpm's for future reuse in Mandriva. By Glenn Waller 28th April 2010.


Why did I decide to reuse the downloaded rpm's (Redhat Package Management)?

To save time, connectivity problems (Internet) Bandwidth and embarrassment after I broke the system. I have added my scripts with explanations to the end of this paper but they are customised for my system, so read and edit them before trying to use them. ;-)

How I started reusing the saved rpm's.

Including, Mandriva installation choice of copying the data to HD, before the package selection process, was saved in /var/ftp/pub/mandriva(ver)...

Mostly, back then it was for large packages that I would reuse, such as the kernel-source (50Mb, approx) and the plf (Mandriva's Penguin Liberation Front) rpm's for video codecs.

Then I found genhdlist and soon after genhdlist2, and wrote a bunch of commands to a text-file and made a shell script out of it.

I partitioned my hard-drives to give me some space that would not be formatted during an install, Mandy generally requires formatting of /usr and / (root). This maybe a GNU/Linux requirement across the board of the many distributions (distros)

After I found genhdlist2 was I able to usefully add my local repositions (repos) to drakRPM, to make them available when using smart-packager and urpm(i), And reuse them at will, regardless of when I broke it and, and keep moving forward.

Using the no-clean hack in the GUI has never been reliable on my system, as it seems to reset it's self after each invocation of the program. So I took a different track, I wanted to save the rpms while at the same time being able to view the selected packages and dependencies (deps) before committing to the changes.

So, I would painstakingly copy and paste the confirmation text from Drakrpm to a plain text file and remove the new-line and dash+space at the beginning of each new line to create a space separated list that urpmi could work with. After the copy and paste and before running the list with urpmi –noclean I would close Drakrpm cancelling the process, but my list of packages was safe, and usable.

I still have not figured out how to keep the signatures for the packages, although I have tried using some guides from the web. But I guess if they were fine when initially downloaded, they'll still be fine unless I get a hard-drive error, which has not occurred for quite some time (Fingers crossed).

The list was generally discarded after use, generally because of the gross amount of redundant files I would end up with, sometimes it would take upto an hour to edit the list (1340 odd packages, lines) so I've been hunting for a way to make this more expedient and reliable. And I found sed.

Editing the list was accomplished from the bottom to the top, so I was not continually working at the end of the line, but the beginning of the line to save scrolling, with the mouse to high-light the characters I wanted/needed to remove and the space bar.

“Two hands, two guns!” (sifu Malcolm Sue)

Now with all of that it only takes a second to convert the list and I can enjoy my time doing what ever else I like.

I hope someone else finds this helpful, cheers and kind regards, Glenn (GlennsPref@gmail.com)


Scripts with explanations....

The first was a backup script which only collected the downloaded rpms from /var/cache /urpmi/rpms to a directory safe within /home/glenn/....I have many partitions, don't know why, Maybe just because I can, lol.

glenn@GamesBox:~$ df

Filesystem Size Used Avail Use% Mounted on

/dev/sda5 20G 1.8G 17G 10% /

/dev/sda1 92M 59M 28M 68% /boot

/dev/sda14 383G 232G 151G 61% /home

/dev/sdc7 33G 176M 33G 1% /home/glenn/local/Music

/dev/sdd5 52G 180M 52G 1% /home/glenn/local/Video

/dev/sdb8 73G 61G 13G 84% /home/glenn/local/spare

/dev/sdc1 56G 180M 55G 1% /home/glenn/local/stuff

/dev/sdb7 34G 31G 2.9G 92% /home/glenn/local/vm1

/dev/sdb9 74G 51G 23G 69% /home/glenn/stuff

/dev/sdb1 22G 13G 9.4G 57% /media/win_c

/dev/sdd6 31G 14G 18G 44% /media/win_c2

/dev/sdb6 29G 19G 9.8G 66% /media/win_d

/dev/sdd7 63G 23G 41G 36% /media/win_d2

/dev/sda11 5.0G 138M 4.6G 3% /opt

/dev/sda8 8.0G 157M 7.4G 3% /tmp

/dev/sda7 9.9G 5.3G 4.1G 57% /usr

/dev/sda10 5.0G 139M 4.6G 3% /usr/local

/dev/sdc6 20G 520M 18G 3% /usr/src

/dev/sda6 20G 911M 18G 5% /var

/dev/sdc5 3.5G 72M 3.3G 3% /var/ftp

/dev/sda13 5.0G 139M 4.6G 3% /var/spool

/dev/sda12 2.0G 56M 1.9G 3% /var/www

glenn@GamesBox:~$

That is all it did.

Of course to simplify the editing of this script, it calls other scripts that do most of the work,

and it's still a “work in progress”(wip). It has some text to put up on the screen to give some

feedback to the user of what stage the process is up to, and although some of them are duplicated in the other scrips, it works quite well (IMHO).

#! /bin/zsh

#echo "/home/glenn/bin/backup-rpms"

echo "/home/glenn/bin/backup-rpms"

echo

sudo sh /home/glenn/build/rpms/organise-rpms

echo "call organise-rpms....."

echo "Backup-rpms"

#echo "generating new hdlist.cz for local archives........"

sudo sh /home/glenn/build/rpms/gen-hdlist2

echo

echo "Backup-rpms"

echo "rebuilding local rpm and file databases... this might take a while ........"

sudo rpm --rebuilddb && sudo updatedb

echo

echo "Backup-rpms"

echo "local database up to date ........"

echo

echo

sudo sh /home/glenn/build/rpms/update-rpm-database

echo "run rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt, maybe upto 15kb text."/n

sudo rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt

echo "run rpm -Va | grep miss & send to /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt"

sudo rpm -Va | grep miss > /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt

echo "clean rpm cache"

sudo smart clean

sudo rm -rf /var/cache/urpmi/rpms/*rpm

echo ""

echo "all repositories are now up to date, any errors? see above!"

date && whoami

The first thing it does is call organise-rpms from ~/build/rpms...

sudo sh /home/glenn/build/rpms/organise-rpms

This script finds rpms where they maybe stored by me, and the programs DrakRPM and

Smart. (I really like Smart, it is a synaptic-like package manager although it has been changed over

the years, and is now much more useful, but harder to hack, like saving the rpms (Grief)).[/code]

#! /bin/zsh

#echo "/home/glenn/build/rpms/organise-rpms"

echo "organise-rpms"

echo "Called by, ~/bin/backup-rpms"

echo "organising saved rpm cache for local archives....."

#1st Level


sudo mv -v /var/cache/urpmi/rpms/*2010.0*rpm /home/glenn/local/Archive/2010.0/noarch/

sudo mv -v /var/cache/urpmi/rpms/*2010.1*rpm /home/glenn/local/Archive/2010.1/noarch/

sudo cp -v /var/lib/smart/packages/*2010.0*rpm /home/glenn/local/Archive/2010.0/noarch/

sudo cp -v /var/lib/smart/packages/*2010.1*rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /var/cache/urpmi/rpms/*rpm /home/glenn/local/Archive/2010.1/noarch/

sudo cp -v /var/lib/smart/packages/*rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/Download/*rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/Download/packagesNsource/*rpm

echo "filter for 2010.1 packages"

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.0.x86_64.rpm /home/glenn/local/Archive/2010.0/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*plf2010*.rpm /home/glenn/local/Archive/2010.1/plf/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*plf2010*.rpm /home/glenn/loca/Archive/2010.1/plf/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/kde430/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/kde430/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*mdv2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.0/i586/*mdv2010.1.i586.rpm /home/glenn/local/Archive/2010.1/i586/

echo "cleanup-stagglers"

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.1.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*2010.1.i586.rpm /home/glenn/loca/Archive/2010.1/i586/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*4.3.8*mdv2010.1.x86_64.rpm/home/glenn/local/Archive/2010.1/kde/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*plf2010.1.x86_64.rpm /home/glenn/local/Archive/2010.1/plf/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*plf2010.0.x86_64.rpm /home/glenn/local/Archive/2010.0/plf/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*mdv2010.0.noarch.rpm /home/glenn/local/Archive/2010.0/noarch/

sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*2010.0.noarch.rpm /home/glenn/local/Archive/2010.0/noarch/

sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*2010.1.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.0/contrib/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/contrib/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/plf/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/i386/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.1/i586/*.*mnb*.x86_64.rpm /home/glenn/local/Archive/2010.1/x86_64/

sudo mv -v /home/glenn/local/Archive/2010.0/contrib/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.0/noarch/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.0/x86_64/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.1/noarch/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

sudo mv -v /home/glenn/local/Archive/2010.1/x86_64/*.*mnb*.noarch.rpm /home/glenn/local/Archive/2010.1/noarch/

date && whoami

This script also calls another file to change the permissions so I can use them with out errors

such as read-only.


#! /bin/zsh

echo "Backup-rpms"

echo "~/build/rpms/rpms-chmod"

sudo chmod -R 755 /home/glenn/local/Archive/2009.0/

sudo chmod -R 755 /home/glenn/local/Archive/2009.1/

sudo chmod -R 755 /home/glenn/local/Archive/2010.0/

sudo chmod -R 755 /home/glenn/local/Archive/2010.1/

date && whoami


Next the script calls for genhdlist2 to recreate the files required for DrakRPM and Smart

package data-bases, hdlist.cz


#! /bin/zsh

echo "Backup-rpms"

echo "generating new hdlist.cz for local archives........"

echo ""

echo "Backup-rpms"

echo "generating new hdlist.cz for local archives........"

# echo "Backup-rpms local/Archive/2009.0/i386"

# sudo genhdlist2 --xml-info -v --allow-empty-media /home/glenn/local/Archive/2009.0/i386

# echo "Backup-rpms local/Archive/2009.1/noarch"

# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/noarch

# echo "Backup-rpms local/Archive/2009.1/i586"

# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/i586

# echo "Backup-rpms local/Archive/2009.1/i586/contrib"

# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/i586/contrib

# echo "Backup-rpms local/Archive/2009.1/i386"

# sudo genhdlist2 --xml-info -v --allow-empty-media /home/glenn/local/Archive/2009.1/i386

# echo "Backup-rpms local/Archive/2009.1/x86_64"

# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/x86_64

# echo "Backup-rpms local/Archive/2009.1/kde430"

# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/kde430

# echo "Backup-rpms local/Archive/2009.1/x86_64/contrib"

# sudo genhdlist2 --xml-info -v /home/glenn/local/Archive/2009.1/x86_64/contrib

echo "Backup-rpms 2010.0"

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/x86_64

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/noarch

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/i586

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/i386

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/contrib

echo "Backup-rpms 2010.1"

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/x86_64

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/noarch

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/i586

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/i386

sudo genhdlist2 --clean --xml-info -v --allow-empty-media /home/glenn/local/Archive/2010.1/contrib

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.1/kde

echo "Backup-rpms plf"

# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.0/plf

# sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2009.1/plf

sudo genhdlist2 --clean --xml-info -v /home/glenn/local/Archive/2010.0/plf

sudo genhdlist2 --clean --xml-info -v --allow-empty-media /home/glenn/local/Archive/2010.1/plf

echo "GenHdList2 up to date!"

date && whoami


As new versions of Mandy come out, new lines are added and the old ones removed, as seen

above (#) commented out...


If genhdlist finds a corrupted package it stops and shows and error, naming the package,

thanks!

Next it refreshes the rpm database of installed packages, and the find data-base too.


rpm --rebuilddb && sudo updatedb


This takes a while....

The next task is to update the external rpm repo data-bases....


#! /bin/zsh

#echo "/home/glenn/bin/backup-rpms"

echo "/home/glenn/build/rpms/update-rpm-database"

echo "Called from, ~/bin/backup-rpms"

echo

echo

echo

echo

echo "updating urpmi rpm database....."

echo

echo

echo

#echo "urpmi.update local repos"

#sudo urpmi.update local

#echo "uncomment lines to run update all, skipped for now, GW"

echo "urpmi.update all"

sudo urpmi.update -a

#echo "smart update"

#sudo smart update

echo

echo "Backup-rpms"

#echo "local repositories are up to date .. GamesBox.GlennsPref.net........Have a nice day!"

echo "urpmi is up to date .. GamesBox.GlennsPref.net........Have a nice day!"

date && whoami



smart update is no longer required, as it reads the URPM data-bases


The last things to run is a clean up, and checking for missing packages and a catalogue of

installed rpms for the system.


sudo rpm -qa | sort | less > /home/glenn/build/note-rpm-installed-list-`date +%Y-%m-%d-%H`.txt

echo "run rpm -Va | grep miss & send to /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt"

sudo rpm -Va | grep miss > /home/glenn/build/anote-rpm-Va-grep-miss-list-`date +%Y-%m-%d-%H`.txt

echo "clean rpm cache"

sudo smart clean

sudo rm -rf /var/cache/urpmi/rpms/*rpm


rpm -Va | grep miss, takes a while to finish, but can be well worth the wait, before rebooting and

having a crash or cli only system because of a missing dep or lib.

The output is redirected to a file and the file name is created with the date and hour it was

written.

Now the reason I written this is because of a sed trick I found that converts the list to

something urpmi can work with, This process is not completed yet, more things will occur to me as time rolls on,


#!/bin/zsh

echo "Create a space-separated-list from top-down list with preceeding chars."

cd ~/build


echo "remove old files"

rm -f ~/build/file2.txt

rm -f ~/build/file3.txt


echo "Remove - (dash space) from lead of lines using sed...."

echo "sed 's/^[-\ ]*//' file1.txt (from file1.txt in pwd dir)"


echo "redirect output to new file...file2.txt"

sed 's/^[-\ ]*//' file1.txt > file2.txt


echo "Now replace carriage returns with a space... using file2.txt, create file3.txt"

sed ':a;N;$!ba;s/\n/ /g' file2.txt > file3.txt


First it removes the old files, to save stopping for errors, file exists.

Then it removes the dash space from the head of the lines...


And lastly it replaces the newline char with a space, bringing all the list to one line, sweet!


I'd like to reference where most of my info came from,

MDE repo http://www.mde.djura.org/index.html and

SoS repo http://seerofsouls.com/ (website down atm), help files...


Few years old now but still very useful.


http://sed.sourceforge.net/sed1line.txt

http://linux.dsplabs.com.au/rmnl-remove-new-line-characters-tr-awk-perl-sed-c-cpp-bash-python-xargs-ghc-ghci-haskell-sam-ssam-p65/

And that's IT!

Any questions? Did I miss something? Please ask!

Regards Glenn.