Spacewalk Update Failure

Today I decided to update my Spacewalk server to the latest version (0.8 on today’s nightly build). In case you have never heard of Spacewalk before, “Spacewalk is an open source (GPLv2) Linux systems management solution. It is the upstream community project from which the Red Hat Network Satellite product is derived”. Personally I use it as a central update server for all my CentOS server (more than 40 systems at the moment). I follow spacewalk since version 0.2, so one can easy understand that this is my sixth update and I have not faced any serious issues until now. The only problem is the fact that Spacewalk scripts take for granted the fact that the Apache server running on the spacewalk server does not serve any other sites. Unfortunately this is not my case so I have to reconfigure apache all the time after a spacewalk upgrade and I have to notice that in every single update the required changes are different!

This time things were not so easy as Oracle was not friendly! I followed the documented procedure and while the schema upgrade I received the following error:

SQL> update rhnPackageFile p
2     set checksum_id = lookup_checksum('md5', md5);
update rhnPackageFile p
       *
ERROR at line 1:
ORA-30036: unable to extend segment by 8 in undo tablespace 'UNDO'

It was obvious a problem with the UNDO tablespace hitting its extend limit. I verified that I had enough hard disk space available and then I executed the sqlplus utility to verify that the tablespace has hit its limit.

SQL> select file_name, bytes, autoextensible, maxbytes
from dba_data_files where tablespace_name='UNDO';

FILE_NAME
----------------------------------------------------------------------
BYTES AUT   MAXBYTES
---------- --- ----------
/usr/lib/oracle/xe/oradata/XE/undo.dbf
524288000 YES  524288000

Then I extend it genersouly!

SQL> alter database datafile '/usr/lib/oracle/xe/oradata/XE/undo.dbf'
autoextend on next 100m maxsize 2000m;

Database altered.

SQL> select file_name, bytes, autoextensible, maxbytes
from dba_data_files where tablespace_name='UNDO';

FILE_NAME
----------------------------------------------------------------------
BYTES AUT   MAXBYTES
---------- --- ----------
/usr/lib/oracle/xe/oradata/XE/undo.dbf
524288000 YES 2097152000

Unfortunately I could not use any more the automated script spacewalk-schema-upgrade in order to complete the schema upgrade. The solution was again to use the sqlplus utility and execute all sql scripts against the database. To do that I wrote the following bash script:

#!/bin/bash
cd /etc/sysconfig/rhn/schema-upgrade/
cd spacewalk-schema-0.7-to-spacewalk-schema-0.8/
for sqlscript in `ls -1`
do
echo "****"
echo "Executing script $sqlscript"
echo "****"
sqlplus dbuser/dbpass <<EOF
@$sqlscript;
EOF
done

I carefully noticed the output of the script for errors. As expected the first scripts, which have already been executed, displayed several warnings and errors but after the script 012-2-rhnPackageFile-data.sql which initialy failed, everything went smoothly!

UPDATE:
Later I have found that I had to manually update the rhnVersionInfo table in order for new spacewalk upgrades to work. From a shell execute the command:

rpm -q spacewalk-schema

and you will get something like “spacewalk-schema-0.8.9-1.el5.noarch.rpm”. Then login to sqlplus interface and execute the following query (alter accordingly to fit your enviroment):

SQL> INSERT INTO rhnVersionInfo
( label, name_id, evr_id, created, modified )
VALUES ('schema', lookup_package_name('spacewalk-schema'),
lookup_evr(null, '0.8.9', '1.el5' ), sysdate, sysdate );
SQL> COMMIT;

ORIGINAL POST: http://www.soumplis.com/2010/02/09/spacewalk-update-adventures/

Leave a Comment

Your email address will not be published.