36.2. Patching Oracle9i R2 to 9.2.0.6.0 on Red Hat Enterprise Linux 4.. 138 37. Starting and Shutting down the Oracle9i Database 38. Oracle Installation Errors 39. Reference List A. Revision History 151 153
Part I. Tuning and Optimizing Red Hat Enterprise Linux for Oracle Database 9i and 10g

Chapter 1.

Introduction
This guide covers optimizations for Red Hat Enterprise Linux x86 (32 bit) and x86_64 (64 bit) platforms running Oracle 9i R2 (32 bit or 64 bit) and Oracle 10g R1/R2 (32 bit or 64 bit) standalone and RAC databases. This white paper covers Red Hat Enterprise Linux Advanced Server 2.1, 3, 4 and 5. Various workarounds covered in this article are due to the 32 bit address limitations of the x86 platform. However, many steps described in this document also apply to x86-64 platforms. Sections that do not specifically say that its only applicable to 32bit or 64bit apply to both platforms. If you think that a section is not very clear on that, let me know.For supported system configurations and limits for Red Hat Enterprise Linux releases, see http://www.redhat.com/rhel/details/limits/. For instructions on installing Oracle 10g and 9i databases on Red Hat Enterprise Linux, see Appendix A and Appendix B respectively.

Chapter 2.

Hardware Architectures and Linux Kernels

2.1. General

When it comes to large databases the hybrid x86-64 architecture platform is strongly recommended over the 32 bit x86 platform. 64 bit platforms can access more than 4GB of memory without workarounds. With 32 bit platforms there are several issues that require workaround solutions for databases that use lots of memory, for example refer to Chapter 17, Using Very Large Memory (VLM). If you are not sure whether you are on a 32 bit or 64 bit hardware, run dmidecode or cat / proc/cpuinfo. Running uname -a can be misleading since 32 bit Linux kernels can run on x86-64 platforms. But if uname -a displays x86-64, then you are running a 64 bit Linux kernel on a x86_64 platform.
2.2. 32 bit Architecture and the hugemem Kernel
In Red Hat Enterprise Linux 3, 4 or 5 the smp kernel can be used on systems with up to 16 GB of RAM. The hugemem kernel is required in order to use all the memory on systems that have more than 16GB of RAM up to 64GB. However, it is recommend to use the hugemem kernel even on systems that have 8GB of RAM or more due to the potential issue of "low memory" starvation (see next section) that can happen on database systems with 8 GB of RAM. The stability you get with the hugemem kernel on larger systems outperforms the performance overhead of address space switching. With x86 architecture the first 16MB-896MB of physical memory is known as "low memory" (ZONE_NORMAL) which is permanently mapped into kernel space. Many kernel resources must live in the low memory zone. In fact, many kernel operations can only take place in this zone. This means that the low memory area is the most performance critical zone. For example, if you run resource-intensive applications or have a lot of memory installed and are simultaneously running resource-intensive applications, memory addresses below 896MB can become constrained. This happens because more kernel structures must be allocated to these low memory addresses. Low memory starvation happens when LowFree in /proc/meminfo becomes very low accompanied by a sudden spike in paging activity. To free up memory in the low memory zone, the kernel bounces buffers aggressively between low memory and high memory which becomes noticeable as paging (do not confuse it with paging to the swap partition). If the kernel is unable to free up enough memory in the low memory zone, then the kernel can hang the system. Paging activity can be monitored using the vmstat command or using the sar command (option 'B') which comes with the sysstat RPM. Since Linux tries to utilize the whole low memory zone, a low LowFree in /proc/meminfo does not necessarily mean that the system is out of low memory. However, when the system shows increased paging activity when LowFree gets below 50MB, then the hugemem kernel should be installed. The stability you gain from using the hugemem kernel makes up for any performance impact resulting from the 4GB-4GB kernel/user memory split in this kernel (a classic 32 bit x86 system splits the available 4 GB address space into 3 GB virtual memory space for user processes and a 1 GB space for the kernel). To see some allocations in the low memory zone, refer to /proc/meminfo and slabtop(1) for more information. Note that Huge Pages would free up memory in the low memory zone since the system has less bookkeeping to do for that part of virtual memory, see Chapter 17, Using Very Large Memory (VLM).

title Red Hat Enterprise Linux Server (2.6.18-8.el5) root (hd0,0) kernel /vmlinuz-2.6.18-8.el5 ro root=/dev/sda2 elevator=deadline initrd /initrd-2.6.18-8.el5.img
This entry tells the 2.6.18-8.el5 kernel to use the Deadline scheduler. Make sure to reboot the system to activate the new scheduler.
http://www.redhat.com/magazine/008jun05/features/schedulers/

Chapter 5.

Memory Usage and Page Cache
5.1. Checking the Memory Usage
To determine the size and usage of memory, you can enter the following command:
grep MemTotal /proc/meminfo
You can find a detailed description of the entries in /proc/meminfo at http://www.redhat.com/advice/ tips/meminfo.html. Alternatively, you can use the free(1) command to check the memory:
$ free total Mem: 4040360 -/+ buffers/cache: Swap: 4200956 $ used 12184 free 4188772 shared 0 buffers 176628 cached 3571348
In this example the total amount of available memory is 4040360 KB. 264224 KB are used by processes and 3776136 KB are free for other applications. Do not get confused by the first line which shows that 28160KB are free! If you look at the usage figures you can see that most of the memory use is for buffers and cache. Linux always tries to use RAM to speed up disk operations by using available memory for buffers (file system metadata) and cache (pages with actual contents of files or block devices). This helps the system to run faster because disk information is already in memory which saves I/O operations. If space is needed by programs or applications like Oracle, then Linux will free up the buffers and cache to yield memory for the applications. If your system runs for a while you will usually see a small number under the field "free" on the first line.
5.2. Tuning the Page Cache
Page Cache is a disk cache which holds data of files and executable programs, for example pages with actual contents of files or block devices. Page Cache (disk cache) is used to reduce the number of disk reads. To control the percentage of total memory used for page cache in Red Hat Enterprise Linux 3, the following kernel parameter can be changed:
# cat /proc/sys/vm/pagecache 30
The above three values are usually good for database systems. It is not recommended to set the third value very high, around 100, as it used to be with older Red Hat Enterprise Linux 3 kernels. This can cause significant performance problems for database systems. If you upgrade to a newer kernel like 2.4.21-37, then these values will automatically change to "30" unless its set to different values in /etc/sysctl.conf. For information on tuning the pagecache kernel parameter, see Understanding Virtual Memory. Note this kernel parameter does not exist in Red Hat Enterprise Linux 4. The pagecache parameters can be changed in the proc file system without reboot:

Never add a permanent swap file to the system due to the performance impact on the file system layer. Swap Size Recommendations According to Oracle9i Installation Guide Release 2 a minimum of 512MB of RAM is required to install Oracle9i Server. According to Oracle Database Installation Guide 10g Release 2 at least 1024MB of RAM is required for 10g R2. For 10g R2, Oracle gives the following swap space requirement: RAM 1 GB - 2 GB 2 GB - 8 GB Greater than 8GB Swap Space 1.5 times the size of RAM Equal to the size of RAM 0.75 times the size of RAM
Table 6.1. Recommended Swap Space Requirements for 10g R2

Chapter 6. Swap Space

6.2. Checking Swap Space Size and Usage
You can check the size and current usage of swap space by running the command: grep SwapTotal /proc/meminfo Swap usage may slowly increase as shown above but should stop at some point. If swap usage continues to grow steadily or is already large, then one of the following choices may need to be considered: Adding more RAM. Reducing the size of the SGA. Increasing the size of the swap space. If you see constant swapping, then you need to either add more RAM or reduce the size of the SGA. Constant swapping should be avoided at all cost. You can check current swap activity using the following commands:
$ vmstat 3 procs r b swpd free 944932 memory swap buff cache si so bi 705 io bo 0 in 1501 system cpu cs us sy id wa 87
The fields si and so show the amount of memory paged in from disk and paged out to disk, respectively. If the server shows continuous swap activity then more memory should be added or the SGA size should be reduced. To check the history of swap activity, you can use the sar command. For example, to check swap activity from Oct 12th:
# ls -al /var/log/sa | grep "Oct 12" -rw-r--r-1 root root 2333308 Oct 12 23:55 sa12 -rw-r--r-1 root root 4354749 Oct 12 23:53 sar12 # sar -W -f /var/log/sa/sa12 Linux 2.4.21-32.0.1.ELhugemem (rac01prd) 10/12/2005 12:00:00 12:05:00 12:10:00 12:15:00 12:20:00 12:25:00 12:30:00. AM AM AM AM AM AM AM pswpin/s pswpout/s 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
The fields pswpin and pswpout show the total number of pages brought in and out per second, respectively. If the server shows sporadic swap activity or swap activity for a short period time at certain intervals, then you can either add more swap space or RAM. If swap usage is already very large, do not confuse very large swap usage with constant swapping, then more RAM is recommended.

Chapter 7.

Setting Shared Memory
Shared memory allows processes to access common structures and data by placing them in shared memory segments. It is the fastest form of inter-process communication available since no kernel involvement occurs when data is passed between the processes. In fact, data does not need to be copied between the processes. Oracle uses shared memory segments for the Shared Global Area (SGA) which is an area of memory that is shared by Oracle processes. The size of the SGA has a significant impact to Oracle's performance since it holds database buffer cache and much more. To see all shared memory settings, execute:

$ cat /etc/redhat-release Red Hat Enterprise Linux AS release 3 (Taroon Update 6) $ egrep "shm|tmpfs|ramfs" /proc/filesystems nodev tmpfs nodev shm nodev ramfs $
Chapter 17. Using Very Large Memory (VLM)
$ cat /etc/redhat-release Red Hat Enterprise Linux AS release 4 (Nahant Update 2) $ egrep "shm|tmpfs|ramfs" /proc/filesystems nodev tmpfs nodev ramfs $
This means that if you try to mount a shmfs file system in Red Hat Enterprise Linux 4 or 5, you will get the following error message:
mount: fs type shm not supported by kernel
The difference between shmfs and tmpfs is you do not need to specify the size of the file system if you mount a tmpfs file system.
17.2. Configuring Very Large Memory (VLM)
The following example shows how to use the RAM disk ramfs to allocate 8 GB of shared memory for the Oracle 10g database buffer cache on a 32 bit Red Hat Enterprise Linux 3, 4 or 5 systems (hugemem kernel). If this setup is performed on a server that does not have enough RAM, then Linux will appear to hang and the kernel will automatically start killing processes due to memory shortage (ramfs is not swappable). Furthermore, ramfs is not backed by Huge Pages and therefore the Huge Pages pool should not be increased for database buffers, see Chapter 6, Swap Space. In fact, if there are too many Huge Pages allocated, then there may not be enough memory for ramfs. Since ramfs is not swappable, it is by default only usable by root. If you put too much on a ramfs file system, you can easily hang the system. To mount the ramfs file system and to make it usable for the Oracle account, execute:
# umount /dev/shm # mount -t ramfs ramfs /dev/shm # chown oracle:dba /dev/shm
When Oracle starts it will create a file in the /dev/shm directory that corresponds to the extended buffer cache. Ensure to add the above lines to /etc/rc.local. If ointall is the primary group of the Oracle account, use chown oracle:oinstall /dev/shm instead. For security reasons you do not want to give anyone write access to the shared memory file system. Having write access to the ramfs file system allows you to allocate and pin a large chunk of memory in RAM. In fact, you can kill a machine by allocating too much memory in the ramfs file system. To enable VLM, set the Oracle parameter use_indirect_data_buffers to true:

Install the libaio-0.3.96 RPM or a newer version, otherwise the OUI prerequisite check will fail. To check the RPMs, run:
rpm -q binutils compat-db control-center gcc gcc-c++ glibc glibc-common \ gnome-libs libstdc++ libstdc++-devel make pdksh sysstat xscreensaver libaio
For 10g R2 (32 bit) on Red Hat Enterprise Linux 3 x86, the document Oracle Database Installation Guide 10g Release 2 (10.2) for Linux x86 lists the following required package versions or higher:
make-3.79.1 gcc-3.2.3-34 glibc-2.3.2-95.20 compat-db-4.0.14-5 compat-gcc-7.3-2.96.128 compat-gcc-c++-7.3-2.96.128 compat-libstdc++-7.3-2.96.128 compat-libstdc++-devel-7.3-2.96.128 openmotif21-2.1.30-8 setarch-1.3-1
libaio-0.3.96 RPM or a newer version is required for the OUI to work successfully.
rpm -q make gcc glibc compat-db compat-gcc compat-gcc-c++ compat-libstdc++ \ compat-libstdc++-devel openmotif21 setarch libaio
For Oracle 10g R1 (64 bit) on Red Hat Enterprise Linux 3 x86-64, the document Oracle Database Installation Guide 10g Release 1 (10.1.0.3) for Linux x86-64 lists the following required package versions or higher:
make-3.79.1 gcc-3.2.3-34 glibc-2.3.2-95.20 glibc-devel-2.3.2-95.20 glibc-devel-2.3.2-95.20 compat-db-4.0.14-5

(32 bit)

compat-gcc-7.3-2.96.128 compat-gcc-c++-7.3-2.96.128 compat-libstdc++-7.3-2.96.128 compat-libstdc++-devel-7.3-2.96.128 gnome-libs-1.4.1.2.90-34.1 (32 bit) openmotif21-2.1.30-8 setarch-1.3-1 libaio-0.3.96-3 libaio-devel-0.3.96-3
To check if you are running the AMD64 or Intel 64(x86-64) kernel on an AMD64 or Intel 64 platform, run:
rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' \ make gcc glibc glibc-devel compat-db compat-gcc compat-gcc-c++ \ compat-libstdc++ compat-libstdc++-devel gnome-libs openmotif21 setarch \ libaio libaio-devel
It is important to have the right x86 and x86-64 RPMs installed. The above command will list the architecture of each binary package. And as you can see in the above list, glibc-devel and other RPMs are listed twice. This means that you have to install packages for both architectures, x86 and x86-64. For Oracle 10g R1 (32 bit) on Red Hat Enterprise Linux 3 x86, the document Oracle Database Installation Guide 10g Release 1 (10.1) for UNIX Systems lists the following required package versions or higher:

\ \ \ \ \ \ \

You also need to install the following i386 and x86-64 RPMs if not already installed, otherwise you will get various different error messages. For a detailed list of error messages, read Chapter 29, Oracle 10g and Linux Installation Errors and Troubleshooting.
rpm -Uvh xorg-x11-deprecated-libs-6.8.2-1.EL.13.20.i386.rpm \ xorg-x11-libs-6.8.2-1.EL.13.20.i386.rpm xorg-x11-Mesa-libGL-6.8.2-1.EL.13.20.i386.rpm expat-1.95.7-4.i386.rpm fontconfig-2.2.3-7.i386.rpm freetype-2.1.9-1.i386.rpm zlib-1.2.1.2-1.2.i386.rpm

\ \ \ \ \

Chapter 20. Installing Required Software Packages
rpm -Uvh libaio-0.3.103-3.x86_64.rpm rpm -Uvh compat-libstdc++-33-3.2.3-47.3.x86_64.rpm rpm -Uvh glibc-devel-2.3.4-2.13.i386.rpm libgcc-3.4.4-2.i386.rpm
If you are yet to install Update 3 or later, do not forget to install an updated binutils RPM from 1 https://rhn.redhat.com/ or from http://oss.oracle.com/. This command will install the newer version of binutils, remember to change the version number if the one you downloaded is newer.
rpm -Uvh --force binutils-2.15.92.0.2-13.0.0.0.2.x86_64.rpm
If you do not install a newer binutil RPM from Oracle or RHN, then you will get the following error message:
/usr/bin/ld: /u01/app/oracle/oracle/product/10.2.0/db_1/lib//libirc.a(fast_memcpy.o): relocation R_X86_64_PC32 against `_memcpy_mem_ops_method' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status
For more information on this bug, see Bugzilla Bug 679. Oracle lists the control-center and xscreensaver RPMs as a requirements. But you should not have any problems when these RPMs are missing. But if you want to install them, you may have to install many additional RPMs in order to satisfy dependencies:
rpm -Uvh gcc-3.4.4-2.x86_64.rpm \ gcc-c++-3.4.4-2.x86_64.rpm \ libstdc++-devel-3.4.4-2.x86_64.rpm \ cpp-3.4.4-2.x86_64.rpm \ glibc-devel-2.3.4-2.13.x86_64.rpm \ glibc-headers-2.3.4-2.13.x86_64.rpm \ glibc-kernheaders-2.4-9.1.98.EL.x86_64.rpm rpm -Uvh control-center-2.8.0-12.rhel4.2.x86_64.rpm \ xscreensaver-4.18-5.rhel4.9.x86_64.rpm \ eel2-2.8.1-2.x86_64.rpm \ gail-1.8.0-2.x86_64.rpm \ gnome-desktop-2.8.0-5.x86_64.rpm \ gnome-icon-theme-2.8.0-1.el4.1.3.noarch.rpm \ libgail-gnome-1.1.0-1.x86_64.rpm \ libxklavier-1.02-3.x86_64.rpm \ metacity-2.8.6-2.8.x86_64.rpm \ nautilus-2.8.1-4.x86_64.rpm \ startup-notification-0.7-1.x86_64.rpm \ xloadimage-4.1-34.RHEL4.x86_64.rpm \ xorg-x11-Mesa-libGLU-6.8.2-1.EL.13.20.x86_64.rpm \ at-spi-1.6.0-3.x86_64.rpm \ desktop-backgrounds-basic-2.0-26.2.1E.noarch.rpm \ eog-2.8.1-2.x86_64.rpm \ gnome-panel-2.8.1-3.3E.x86_64.rpm \ gnome-vfs2-smb-2.8.2-8.2.x86_64.rpm \ hicolor-icon-theme-0.3-3.noarch.rpm \ libexif-0.5.12-5.1.x86_64.rpm \ librsvg2-2.8.1-1.x86_64.rpm \ nautilus-cd-burner-2.8.3-6.x86_64.rpm \

Now when you try to run any GUI tool on the remote server, it should automatically be relinked to your local desktop. If this is not working, verify that the ForwardX11 setting is not set to "no" in /etc/ ssh/ssh_config on your remote server:
If you use newer Red Hat Enterprise Linux versions as your desktop and you want to install the database on another machine, then you need to set the DisallowTCP entry in /etc/X11/gdm/gdm.conf for the GNOME Display Manager to read: DisallowTCP=false After that you need to restart your X server. You can do this with the init command:
su - root init 3 init 5 If you are using telnet you will have to set the DISPLAY variable manually.
Recovery Manager(rman) hangs
You are probably running the wrong rman binary which belongs to the XFree86-devel RPM:
$ which rman /usr/X11R6/bin/rman
During the Oracle 10g installation you probably provided a password for the Oracle database accounts that started with a digit number. Ignore this error message and change the password when you are done with the Oracle 10g installation.

./runInstaller Crashes

The Oracle installer./run Installer crashes with the
Exception java.lang.UnsatisfiedLinkError: /tmp/OraInstall2005-06-15_07-36-25AM/jre/1.4.2/lib/ i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory occurred. java.lang.UnsatisfiedLinkError: /tmp/OraInstall2005-06-15_07-36-25AM/jre/1.4.2/lib/i386/ libawt.so: libXp.so.6: cannot open shared object file: No such file or directory at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1560) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1477)
You may receive this error message on Red Hat Enterprise Linux 3 x86-64, Red Hat Enterprise Linux 4 x86-64, and on other systems. Even though you most probably have /usr/X11R6/lib64/ libXp.so.6 installed on your system, this error messages is complaining that it can not find the libXp.so.6 shared library for i386:
/tmp/OraInstall2005-06-15_07-36-25AM/jre/1.4.2/lib/i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory

$ su - root # cat /proc/sys/kernel/shmmax 33554432 # echo `expr 1024 \* 1024 \* 1024` > /proc/sys/kernel/shmmax # cat /proc/sys/kernel/shmmax 1073741824
It is recommended to increase the shmmax setting permanently for Oracle. For more information on optimizing shared memory settings for Oracle databases on Linux, see Chapter 7, Setting Shared Memory. These parameters apply to all Red Hat Enterprise Linux versions.
30.4. Examining Temporary(/tmp) Space
The Oracle Universal Installer requires up to 400 MB of free space in the /tmp directory. To find out the space in /tmp, run:
If you do not have enough space in the /tmp directory, you can temporarily create a tmp directory in another file system. Here are the steps for doing it:
su - root mkdir /<AnotherFilesystem>/tmp chown root.root /<AnotherFilesystem>/tmp chmod 1777 /<AnotherFilesystem>/tmp export TEMP=/<AnotherFilesystem> export TMPDIR=/<AnotherFilesystem>
The TEMP=/<AnotherFilesystem> file is used by Oracle. The TMPDIR=/<AnotherFilesystem> file is used by Linux programs like the linker "ld". When you are done with your Oracle installation, shut down Oracle and remove the temporary directory:
30.5. Sizing Oracle Disk Space
You will need about 2.5 GB for the database software. If you perform a typical database installation and not a customized database installation, then you will need about 3.5 GB of disk space.

Chapter 31.

Verifying Required Packages(RPMs)
You will need certain RPM development packages for the Oracle installer to build Oracle modules, otherwise you will get error messages similar to this one:
Error in invoking target ntcontab.o of makefile /u01/app/oracle/product/9.2.0/network/lib/ins_net_client.mk
Always ensure you use the latest stable versions of RPM.
See the Oracle9i Release Notes Release 2 (9.2.0.4.0) for Linux x86 for the list of required RPMs.
31.1. Required Packages for Red Hat Advanced Server 2.1
Ensure the following development packages are installed:
rpm -q gcc cpp compat-libstdc++ glibc-devel kernel-headers binutils
Most of these packages will be missing on Red Hat Advanced Server 2.1 if the "Software Development" package was not selected during the OS install. If these RPMs are missing, execute the command below, remember your versions may be newer. If they are not newer than those seen below update to a more recent version.

\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
If you are using the Red Hat Network(RHN), you can run: up2date gnome-libs gnome-libs-devel You can use the up2date command for any packages. It takes care of dependencies by installing all required packages automatically. To install the compat-oracle-rhel4 and compat-libcwait packages you have to download the patch 4198954 from http://metalink.oracle.com. Make sure to select the Linux x86 platform. To unzip the downloaded p4198954_21_LINUX.zip file, run:
$ unzip p4198954_21_LINUX.zip Archive: p4198954_21_LINUX.zip creating: 4198954/ inflating: 4198954/compat-oracle-rhel4-1.0-5.i386.rpm inflating: 4198954/compat-libcwait-2.0-2.i386.rpm inflating: 4198954/README.txt
The compat-oracle-rhel4 and compat-libcwait packages require the xorg-x11deprecated-libs and xorg-x11-deprecated-libs-devel packages, see above. To install the two RPMs from the 4198954 patch, run: # rpm -Uvh 4198954/compat-oracle-rhel4-1.0-5.i386.rpm 4198954/ compat-libcwait-2.0-2.i386.rpm

Chapter 32.

Setting Up a Working Environment for Oracle
This chapter covers the creation of user accounts, directories and Oracle environments for your Oracle Database before you install.
32.1. Creating Oracle User Accounts
This section covers the command necessary to activate the required accounts to install an Oracle Database. Create the group of users to be granted with SYSDBA system privilege

su - root groupadd dba

Now create a group owner for Oracle files
groupadd oinstall useradd -c "Oracle software owner" -g oinstall -G dba oracle passwd oracle
For more information on the "oinstall" group account, see When to use OINSTALL group during install of oracle.
32.2. Creating Oracle Directories
Make sure that the Oracle file systems, in this example /u01, is large enough, see Section 30.5, Sizing Oracle Disk Space for more information.
su - root #mkdir -p /u01/app/oracle/product/9.2.0 #chown -R oracle.oinstall /u01 #mkdir /var/opt/oracle #chown oracle.dba /var/opt/oracle #chmod 755 /var/opt/oracle
32.3. Setting Oracle Environment Variables
Make sure to set the following Oracle environment variables before you execute runInstaller. As the oracle user execute the following commands: # Make sure to set the LD_ASSUME_KERNEL environment variable for Red Hat Enterprise Linux 3 and 4 !! # Use the "Linuxthreads with floating stacks" implementation instead of NPTL:

Click ignore. This will be fixed after you apply the patch 3119415 after the 9.2.0.4 patchset has been applied. You can not apply the patch 3119415 at this time since the file /u01/app/oracle/oraInventory/ ContentsXML/comps.xml does not exist yet. We will show you how to apply patch 3119415 later. "Error in invoking target install of makefile /u01/app/oracle/product/9.2.0/ctx/lib/ ins_ctx.mk. " The /u01/app/oracle/product/9.2.0/install/make.log file reads:
/usr/bin/ld: ctxhx: hidden symbol `stat' in /usr/lib/libc_nonshared.a(stat.oS) is referenced by DSO collect2: ld returned 1 exit status make: *** [ctxhx] Error 1
Click ignore. This will be fixed when you apply the 9.2.0.4 patch set.
35.2. Patching Oracle9i to 9.2.0.4.0 on Red Hat Enterprise Linux 3
To patch Oracle9i R2, download the Oracle9i Release 2 Patch Set 3 Version 9.2.0.4.0 for Linux x86 from http://metalink.oracle.com. Download the p3095277_9204_LINUX.zip file to /tmp and run the following command:
su - oracle $ cp p3095277_9204_LINUX.zip /tmp $ cd /tmp $ unzip p3095277_9204_LINUX.zip Archive: p3095277_9204_LINUX.zip inflating: 9204_lnx32_release.cpio inflating: README.html inflating: patchnote.css $ $ cpio -idmv < 9204_lnx32_release.cpio Disk1/stage/locks Disk1/stage/Patches/oracle.apache.isqlplus/9.2.0.4.0/1/DataFiles/bin.1.1.jar Disk1/stage/Patches/oracle.apache.isqlplus/9.2.0.4.0/1/DataFiles/lib.1.1.jar.
To patch the runInstaller, execute:
su - oracle $ echo $LD_ASSUME_KERNEL 2.4.1 $ cd /tmp/Disk1/ $./runInstaller
- Welcome Screen: Click Next - File Locations: Use default values - Available Products: Select "Oracle Universal Installer 2.2.0.18.0" - Components Locations: Accept default values and click Next - Summary: Start the Install - At the end of the installation, you must exit runInstaller To patch Oracle9i R2, execute:
su - oracle $ echo $LD_ASSUME_KERNEL 2.4.1 $ cd $ORACLE_HOME/bin $./runInstaller
Patching Oracle9i to 9.2.0.4.0 on Red Hat Enterprise Linux 3
- Welcome Screen: Click Next - File Locations: Use default values - Available Products: Select "Oracle9iR2 Patch Set 3 9.2.0.4.0 !" - Summary: Start the Install - At the end of the installation, exit runInstaller You may get the error: "Error in invoking target install of makefile /u01/app/oracle/ product/9.2.0/network/lib/ins_oemagent.mk". The /u01/app/oracle/product/9.2.0/install/make.log file reads:

$ su - oracle $ which opatch /tmp/OPatch/opatch $
Please note, the patch needs "fuser" which should be located in /sbin, if it is not you will need to install it. To apply now the patch, run:
$ $ $ $ $ su - oracle unzip p3238244_9204_LINUX.zip cd 3238244 export PATH=$PATH:/sbin opatch apply
Now you need to relink dbsnmp. This is the binary that crashed when running agentctl start. To find which makefile handles the linking of dbsnmp, you can run:
$ su - oracle $ find $ORACLE_HOME -name "*.mk" | xargs grep -l dbsnmp /u01/app/oracle/product/9.2.0/network/lib/ins_oemagent.mk /u01/app/oracle/product/9.2.0/network/lib/env_oemagent.mk $
To relink dbsnmp and all associated executable files which are maintained by the ins_oemagent.mk makefile, run:
Patching Oracle Intelligent Agent on Red Hat Enterprise Linux 3
$ su - oracle $ cd $ORACLE_HOME/network/lib $ make -f ins_oemagent.mk install
Now you should be able to start the agent:
$ su - oracle $ agentctl start
Do not forget to undo the links to /usr/bin/gcc and /usr/bin/g++ if you do not need them any more. You may also want to undo the changes in /etc/ld.so.preload file.

Chapter 36.

Installing Oracle9i R2 (9.2.0.6.0) on Red Hat Enterprise Linux 4
In order to install Oracle9i Release 2 (9.2.0.6), the 9.2.0.6 patch set must be applied for the Oracle database server (patch number 3948480) after the Oracle9i Release 2 (9.2.0.4) installation. For more information, see Oracle9i Release Notes Release 2 (9.2.0.4.0) for Linux x86 - Red Hat Enterprise Linux 4 Certification Update.
36.1. Installing Oracle9i R2 (9.2.0.4.0) on Red Hat Enterprise Linux 4
Before you continue, ensure all the required RPMs are installed, see Checking Packages (RPMs). Also ensure LD_ASSUME_KERNEL is set to 2.4.19 (see Setting Oracle Environments):
$ su - oracle $ echo $LD_ASSUME_KERNEL 2.4.19 $
Now execute runInstaller:
$ su - oracle $ echo $LD_ASSUME_KERNEL 2.4.19 $ /media/cdrom/runInstaller
- Welcome Screen: Click Next - Inventory Location: Click OK - Unix Group Name: Use "oinstall" and click Next. When asked to run /tmp/orainstRoot.sh enter the following into the command line

Always check first the error logs for 9.2.0 in /tmp/OraInstall*, where "*" will be the date of the install, for example /tmp/OraInstall2002-07-04_09-50-19PM. When you problems with make, see also the $ORACLE_HOME/install/make.log file.

make Problems

First ensure that gcc is installed on your system by executing:

$ which gcc /usr/bin/gcc

Here is the command to find the RPM package name for /usr/bin/gcc:
$ rpm -qf /usr/bin/gcc gcc-2.96-98
Verify that your error is not one of the other error messages below. See Chapter 31, Verifying Required Packages(RPMs) for more information on whether you have the correct packages.
Error in invoking target install of makefile/u01/app/oracle/product/9.2.0/ctx/lib/ ins_ctx.mk
You may see the following errors in $ORACLE_HOME/install/make.log:
/lib/libdl.so.2: /lib/libdl.so.2: /lib/libdl.so.2: /lib/libdl.so.2: /lib/libdl.so.2: undefined undefined undefined undefined undefined reference reference reference reference reference to to to to to `_dl_addr@GLIBC_PRIVATE' `_dl_open@GLIBC_PRIVATE' `_dl_close@GLIBC_PRIVATE' `_dl_sym@GLIBC_PRIVATE' `_dl_vsym@GLIBC_PRIVATE'
This error comes up when the following step is executed:
/usr/bin/make -f ins_ctx.mk install ORACLE_HOME=/u01/app/oracle/product/9.2.0
Edit the file $ORACLE_HOME/ctx/lib/env_ctx.mk, add "$(LDLIBFLAG)dl" to the "INSO_LINK =" line. The updated line should with the added "$(LDLIBFLAG)dl" flag, should look like this:
INSO_LINK = -L$(CTXLIB) $(LDLIBFLAG)m $(LDLIBFLAG)dl $(LDLIBFLAG)sc_ca $(LDLIBFLAG)sc_fa $(LDLIBFLAG)sc_ex $(LDLIBFLAG)sc_da $(LDLIBFLAG)sc_ut $(LDLIBFLAG)sc_ch $(LDLIBFLAG)sc_fi $(LLIBCTXHX) $(LDLIBFLAG)c -Wl,-rpath,$(CTXHOME)lib $(CORELIBS) $(COMPEOBJS)
After that hit Retry in the error pop-up. If this did not fix the problem, try the following solution: Edit the file $ORACLE_HOME/ctx/lib/env_ctx.mk again, go to "INSO_LINK =", remove the above entry you made and add "`cat $(LIBHOME)/sysliblist`" to the line and save it. This is what the updated line, with the added "`cat $(LIBHOME)/sysliblist`" string, look like:

Errors in dbca

If you receive the error:

$ dbca SIGSEGV

11* segmentation violation stackbase=0x453da000, stackpointer=0x453d9d5c Full thread dump: "AWT-EventQueue-0" (TID:0x411d1e20, sys_thread_t:0x453d9e0c, state:R) prio=5 *current thread* java.lang.Object.wait(Object.java) java.awt.EventQueue.getNextEvent(EventQueue.java:126).
If this happens, try the following:
$ su - root touch /etc/rac_on
Now try to restart dbca. Another option is to edit $ORACLE_HOME/bin/dbca and to put the following lines under comment except the line not preceded by a hash:
# if [ -f /etc/rac_on ]; then # Run DBCA $JRE_DIR/bin/jre -native -DORACLE_HOME=$OH. # else # Run DBCA # $JRE_DIR/bin/jre -DORACLE_HOME=$OH. # fi

Now try to restart dbca.

gcc errors in Red Hat Enterprise Linux 3
When installing or running Oracle on Red Hat Enterprise Linux 3 you may encounter this gcc error.
gcc -o /u01/app/oracle/product/9.2.0/rdbms/lib/oracle \ -L/u01/app/oracle/product/9.2.0/rdbms/lib/.. /usr/bin/ld: /u01/app/oracle/product/9.2.0/rdbms/lib/oracle: hidden symbol `__fixunssfdi' in / usr/lib/gcc-lib/i386-redhat-linux/3.2.3/libgcc.a(_fixunssfdi.oS) is referenced by DSO collect2: ld returned 1 exit status make: *** [/u01/app/oracle/product/9.2.0/rdbms/lib/oracle] Error 1 /usr/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/u01/app/oracle/product/9.2.0
To fix the linking problem, execute the following commands:
# # # # mv mv ln ln /usr/bin/gcc /usr/bin/gcc323 /usr/bin/g++ /usr/bin/g++323 -s /usr/bin/gcc296 /usr/bin/gcc -s /usr/bin/g++296 /usr/bin/g++
Now you should be able to relink the oracle binary again. Once you are done, you may want to undo the changes you have performed above:
# mv /usr/bin/gcc323 /usr/bin/gcc # mv /usr/bin/g++323 /usr/bin/g++
./runInstaller: line 58:./runInstaller: cannot execute binary file
You are probably trying to run a 64 bit Oracle version on a 32 bit Linux system. Make sure you downloaded the right Oracle version for your Linux system. To check if runInstaller is a 32 bit binary or a 64 bit binary, run the following command:

2.5.1. rpmbuild

A user account for building, e.g., rpmbuild should be created to allow several administrators to share the build environment and the GPG key. It is recommended that this non-privileged user is used when building RPMs. The home directory for the user, /home/rpmbuild by default, should contain the file.rpmmacros with at least the following content (where _gpg_name must match the name for the GPG key used for signing RPMs):
%_topdir %_signature %_gpg_name
%(echo $HOME)/rpmbuild %gpg rpmbuild <rpmbuild@example.com>
The directory listing for the defined top level directory (/home/rpmbuild/rpmbuild in the example above) must have the same directory layout that is present under /usr/src/redhat. Appendix B contains a very basic RPM spec file which can be used a basis when starting to craft real spec files needed locally.
2.5.2. GPG Key for Signing RPMs
The following commands will initiate GPG key creation and export it in a format suitable for distributing to client systems. The created key should be stored safely and backed up, and its passphrase should be known only by trusted administrators.
mkdir -p ~/.gnupg gpg --gen-key gpg --list-keys --fingerprint gpg --export --armor "rpmbuild <rpmbuild@example.com>" > EXAMPLE-RPM-GPG-KEY
To import this key to the RPM database to allow RPM origin and integrity verification, the following command must be run as root on all target systems (naturally this should happen automatically during client installations): 13
rpm --import EXAMPLE-RPM-GPG-KEY
Once an RPM has been created it must be signed with the GPG key and uploaded to a correct channel:
rpm --resign package.rpm rhnpush --server=http[s]://satellite.server/APP package.rpm --channel=custom-channel-name
The following commands will verify an RPM package located in the current directory:
rpm qip pakcage.rpm rpm -K package.rpm
2.5.3. RPM Spec File Example
The following is a basic example of an RPM spec file. When building, it should be located in the directory SPECS under the _topdir as defined in user's.rpmmacros file and the corresponding source and patch files should in the SOURCES directory.

Name: foo Summary: The foo package does foo Version: 1.0 Release: 1 License: GPL Group: Applications/Internet URL: http://www.example.org/ Source0 : foo-1.0.tar.gz Buildroot: %{_tmppath}/%{name}-%{version}-%{release}-root Requires: pam BuildPrereq: coreutils %description This package performs the foo operation. %prep %setup -q %build %install mkdir -p %{buildroot}/%{_datadir}/%{name} cp -p foo.spec %{buildroot}/%{_datadir}/%{name} %clean rm -fr %{buildroot} %pre # Add user/group here if needed %post /sbin/chkconfig --add food %preun if [ $1 = 0 ]; then # package is being erased, not upgraded /sbin/service food stop > /dev/null 2>&1 /sbin/chkconfig --del food fi %postun if [ $1 = 0 ]; then # package is being erased # Any needed actions here on uninstalls else # Upgrade /sbin/service food condrestart > /dev/null 2>&1 fi

Kickstart

%files %defattr(-,root,root) %{_datadir}/%{name} %changelog * Mon Jun Some One <one@example.com> - fixed the broken frobber (#86434)

2.6. Kickstart

System kickstarting is essential part of automated installation and efficient system provisioning. PXE combined with remote hardware management provides fully automated solution for kickstarting. However, parts of the network might not have PXE/DHCP functionality available so also CD/USB booting might be needed in some cases. The following sections provide step by step instructions on how to create customized boot images for kickstarting clients with CD/PXE/USB using standard Linux utilities from the syslinux package. Some of the steps (beginning with #) expects that they are run either with sudo or as root. Appendix C provides an example of boot menus for different kickstart targets that can be interactively selected when provisioning a system.
2.6.1. Preparing Custom Boot Media
First, locate the standard RHEL CD boot image boot.iso, create work directories for different boot images, and populate them with RHEL boot files:
mkdir -p temp cd/isolinux pxe/pxelinux.cfg usb/extlinux usb/temp mount -o loop boot.iso temp cp -a temp/isolinux/* cd/isolinux/ cp -a temp/isolinux/* pxe/ cp -a temp/isolinux/* usb/extlinux/ umount temp chmod -R u+rw cd pxe usb
Follow the instructions below to create needed boot images.
2.6.2. Creating CD Boot Media
Creating bootable CD image can done with the following commands after the earlier preparations (note that the backslash "\" represents a continuation of one line at the shell prompt):
cd./cd cp -p /usr/lib/syslinux/menu.c32 isolinux vi isolinux/isolinux.cfg mkisofs -o./custom-boot.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot \ -boot-load-size 4 -boot-info-table -J -l -r -T -v -V "Custom RHEL Boot".
Customize any boot parameters and targets in isolinux.cfg as needed for CD booting.

Figure 3.2. Create New Organization 1. Input the Organization Name in the provided text box. The name should be between 3 and 128 characters. 2. Create an administrator for the organization: a. Enter a Desired Login for the organization administrator, which should be between 3 and 128 characters long. b. Create a Desired Password and Confirm the password. c. Type in the Email address for the organization administrator.
d. Enter the First Name and Last Name of the organization administrator. 3. Click the Create Organization button to complete the process. 20
Managing Entitlements Once the new organization is created, the Organizations page will display with the new organization listed.
Satellite Administrators should consider reserving the organization 1 Organization Administrator account for themselves to have the option of logging into this organization for various reasons. If your Satellite is configured for PAM authentication, avoid using PAM accounts for the one Satellite administrative organization administrator account in new organizations. Instead, create a Satellite-local account for organization administrators and reserve PAM-authenticated accounts for Satellite logins with less elevated privileges in order to discourage users to frequently log into the Satellite with elevated privileges, as the potential for making mistakes is higher using these accounts. Additionally, consider creating a login name for the Organization Administrator account that describes (for example, orgadmin-mktg or eng-dept-admin), to match admin login names with the organization.
3.3. Managing Entitlements
One important task after creating a new organization is to assign management entitlements to the new organization. Management system entitlements are a base requirement for an organization to function on the Satellite. The number of management entitlements allocated to an organization is equivalent to the maximum number of systems that may register to that organization on the Satellite, regardless of the number of software entitlements available. For example, if there are 100 Red Hat Enterprise Linux Client entitlements but only 50 management system entitlements to an organization, only 50 systems are able to register to that organization. You should also grant RHN Tools software channel entitlements to each organization. The RHN Tools channel contains various client software required for extended Satellite functionality, such as clients necessary for configuration management and kickstart support as well as the rhn-virtualization package, which is necessary for the entitlements of Xen virtual guests to be counted correctly corresponding to the number of Red Hat Enterprise Linux subscriptions to which they are associated. Access the Subscriptions tab by clicking Admin -> Organizations -> Details -> Subscriptions. The Subscriptions tab has two subtabs for managing the software channel and system entitlements for the organization.

3.4. Configuring Systems in an Organization
Now that an organization has been created and requisite entitlements assigned to it, you can then assign systems to each organization. There are two basic ways to register a system against a particular organization: 22
Admin -> Users 1. Registering Using username and Password If you provide a username and password created for a specified organization, the system will be registered to that organization. For example, if user-123 is a member of the Central IT organization on the Satellite, the following command on any system would register that system to the Central IT organization on your Satellite:
rhnreg_ks --username=user-123 --password=foobaz
The --orgid (for Red Hat Enterprise Linux 4 and 5) and --orgpassword (in RHEL 4) parameters in rhnreg_ks are not related to Satellite registration or RHN Satellite's multiple organizations support.
2. Registering Using An Activation Key You can also register a system to an organization using an activation key from the organization. Activation keys will register systems to the organization in which the activation key was created. Activation keys are a good registration method to use if you want to allow users to register systems into an organization without providing them login access to that organization. If you want to move systems between organizations, you may also automate the move with scripts using the activation keys.
Activation keys have a new format since RHN Satellite 5.1.0, so the first few characters of the activation key are used to indicate which organization (by ID number) owns the key.

3.5. Admin -> Users

The Users Across Satellite page contains a list of all users on the Satellite, throughout all organizations.
You are only able to modify the details of organization users if you are logged in as that Organization Administrator.
Clicking the Username displays the User Details page.
3.5.1. Admin -> Organizations -> Details -> Users
The Users subtab lists the users assigned to the organization, including their real names, email address, and a check mark indicating that the user is an administrator of the organization. If you are the Organization Administrator, you can click the username to display the User Details page 23 for the user.
You must be logged in as the Organization Administrator to edit the User details for an organization. The Satellite Administrator cannot edit user details for organization users.

Chapter 4.

Organizational Trusts
IT deployments are not one-dimensional. Within any organization, the IT infrastructure truly needs to be managed in a multitenant / multi-organizational structure. Whether the division is bureaucratic (organizational departments, offices, subsidiaries) or functional (Web apps, databases, desktops, business processes) Red Hat Network Satellite enables administrators to divide their deployments into organized containers. These containers (or organizations) assist in maintaining clear separation of purpose and ownership of systems and the content deployed to those systems. Red Hat Network Satellite 5.1 introduced the creation and management of multiple organizations within one Satellite installation, allowing for the division of systems, content, and subscriptions across these different organizations, departments, and other specified groups. Red Hat Network Satellite 5.3 expanded multiple organization support to include organizational trusts that allow sharing custom channel content and migration of systems between organizations. This guide will help you understand and make the most of these new capabilities.

4.1. Establishing an Organizational Trust
A Satellite Administrator can create a trust between two or more organizations. To do this, perform the following steps: 1. Click the Organizations link on the side menu on the Admin main page. 2. Click the name of one of the organizations and within the Details page, click the Trusts subtab. 3. On the Trusts subtab, there is a listing of all the other trusts on the RHN Satellite. Here you may use the Filter by Organization text box to narrow down a long list of organizations to a specific subset.
Figure 4.1. Organizational Trusts
Chapter 4. Organizational Trusts 4. Click the checkbox next to the names of the organizations you want to be in the organizational trust with the current organization 5. Click the Modify Trusts button
4.2. Organization Trust Relationships
Organizations can share their resources with each other by establishing an organizational trust relationship. Organizational trusts are defined by the Satellite Administrator and implemented by the Organization Administrator. An organizational trust is bi-directional, meaning that once a Satellite Administrator establishes a trust between two or more organizations, the Organization Administrator from each organization is free to share as much or as little of their resources as they need to. It is up to each Organization Administrator to determine what resources to share, and what shared resources from other organizations in the trust relationship to use. Each individual relationship is unique and mutually exclusive from other org's trust relationship. For example: if the Accounting Org trusts the Finance Org , and the Finance Org trusts the Facilities Org, then the Accounting Org will not trust the Facilities Org unless a separate trust relationship is defined between Accounting and Facilities organizations.
4.3. Sharing Custom Channels
Once an organizational trust has been established, organizations can now share custom software channels with the other organizations in the trust. There are three levels of channel sharing that can be applied to each channel for finer-grained channel access control: Private Make the channel private so that it cannot be accessed by any organizations except the owning organization. Protected Allow the channel to be accessed by specific trusted organizations of your choice. Public Allow all organizations within the trust to access the custom channel. Trusted Organizations that are granted access to the custom content using the protected or public access modes can allow their client systems to install and update packages from the shared channel. When a Organization Administrator changes the access of a shared channel to a access level that is not allowed for a consuming organization, the systems subscribed to the shared channel will lose subscription access. Systems that are subscribed to shared content can be unsubscribed automatically when the following actions occur: Trust Relationship is removed by the Satellite Administrator Organization Administrator changes channel access to private Organization Administrator changes channel access to public and does not include the subscribed system's organization in the protected list Organization Administrator deletes the shared channel directly Organization Administrator deletes the parent channel of a shared child channel All Red Hat software channels are managed through entitlements. Organization Administrators cannot share Red Hat Channels because they are available to all organizations that have entitlements to those channels. The Satellite administrator is responsible for assigning Red Hat software channel entitlements to each organization. 26

System Migration

4.4. System Migration
In addition to sharing software channels, organizations in a trust can migrate systems to other trusted organizations by using a utility, migrate-system-profile, that is executed from the command-line using the systemID and orgID as arguments to specify the system migration and its destination organization. The Satellite Administrator can migrate a system from one trusted organization to any other in the trust. However, Organization Administrators can only migrate a system from their own organization to another in the trust. The migrate-system-profile command requires the spacewalk-utils package to be installed. spacewalk-utils is installed on Satellite 5.3 by default but may also be installed on other clients that are subscribed to the rhn-tools channel. When an organization migrates a system with the migrate-system-profile command, the system does not carry any of the previous entitlements or channel subscriptions from the source organization. However, the system's history is preserved, and can be accessed by the new Organization Administrator in order to simplify the rest of the migration process, which includes subscribing to a base channel and granting entitlements. The usage from the command line is the following:
migrate-system-profile --satellite {SATELLITE HOSTNAME OR IP} --systemId={SYSTEM ID} --toorg-id={DESTINATION ORGANIZATION ID}
For example, if the Finance department (created as an organization in RHN Satellite with OrgID 2) wants to migrate a workstation (with SystemID 10001020) from the Engineering department, but the Finance Organization Administrator does not have shell access to the RHN Satellite server. The RHN Satellite hostname is satserver.example.com. The Finance Organization Administrator would type the following from a shell prompt:
migrate-system-profile --satellite satserver.example.com --systemId=10001020 --to-org-id=2
The Finance Organization Administrator is then prompted for their username and password (unless they specified it using --username= and --password= at the command-line). The Finance Organization Administrator would then be able to see the system from the Systems page when logged into the RHN Satellite web interface. The Finance Organization Administrator can then finish the migration process by assigning a base channel and granting entitlements to the client as he would any other system registered to his organization, which is avaiable from the system's History page in the Events subtab.

Chapter 4. Organizational Trusts
Figure 4.2. System History
The Satellite Administrator can migrate a system from one trusted organization to any other in the trust. However, Organization Administrators can only migrate a system from their own organization to another in the trust.
Satellite Administrators that need to migrate several systems at once can use the --csv option of migrate-system-profile to automate the process using a simple comma-separated list of systems to migrate. A line in the CSV file should contain the ID of the system to be migrated as well as destination organization's ID in the following format:

systemId,to-org-id

The systemId, for example could be 1000010000, while the to-org-id could be 3. An example CSV would look like the following:
1000010000,3 1000010020,1 1000010010,4
System Migration For more information about using migrate-system-profile, refer to the manual page by typing man migrate-system-profile or for a basic help screen type migrate-system-profile -h at the command line.
Part II. Satellite Provisioning
This part describes how to manage, allocate resources, and deploy servers and systems using Satellite, including kickstarting physical and virtual server systems, and provisioning supported systems that have no previous operating system installed (also called bare metal provisioning).

Chapter 5.

Provisioning with Satellite
All organizations need simple, yet powerful tools to deploy Red Hat Enterprise Linux systems. For many years, Red Hat Network Satellite has empowered companies to build repeatable, predictable and reliable deployment processes to ensure rapid repurposing of Linux servers and desktops. Whether you have 10 systems or 10,000 systems, RHN Satellite can help you achieve this goal in a disciplined fashion. Now, after significant investment, RHN Satellite 5.3 has dramatically boosted the flexibility and power of its signature provisioning capabilities. This document contains concise details and instructions for use of the kickstart provisioning functionality in Red Hat Network Satellite.

5.1. Requirements

To use the new provisioning functionality, you need one or more target machines either physical, bare metal computer system(s) or virtual machine host(s). If you want to use Satellite's virtual machine provisioning functionality, your virtual machine host(s) should be configured with either the Xen or KVM virtualization technologies. Note that RHEL 5.4 and newer support KVM virtualization at this time.

5.8. Kickstarting a Machine

5.8.1. Bare Metal

Satellite provides three mechanisms by which you can provision bare metal machines machines that have no operating system or that have the wrong operating system installed: 1. Boot Anaconda-style operating system installation disk 2. PXE boot 3. Boot Cobbler boot disk
5.8.1.1. Booting from an Anaconda Style Installation Disk
Simply boot the selected system using an installation disc that matches your kickstart. For example, if your kickstart was configured to use the ks-rhel-i386-server-5-u2 kickstart tree, you must boot with the Red Hat Enterprise Linux 5.2 i386 installation disc. When the boot prompt comes up, simply type: 43
linux ks=http://satellite.example.com/path/to/kickstart
The system will boot, download the kickstart, and re-install itself.

5.8.1.2. PXE Booting

PXE booting is a very convenient method of installing and reinstalling your physical systems, but does come with a few requirements: You must have a DHCP server, even if your systems are to be configured statically after installation. As DHCP does not normally cross network (router) boundaries, you will need to make special provision to ensure that all of your machines can connect to your DHCP server in the event your machines reside on multiple networks. Options here include multi-homing your DHCP server (either real or trunked vlan) and configuring your routers or switches to pass DHCP across network boundaries. You must be able to configure your DHCP server to point to the PXE server (the Satellite server), by setting the next-server address for the systems you want to be managed by Satellite. Each system you have must support PXE booting at the BIOS level. Nearly all recent hardware should be able to do this. You must have the TFTP service on and running.
5.8.1.2.1. Configuring an Existing DHCP Server
If you have a DHCP server deployed on another system on the network, you will need administrative access to the DHCP server in order to to edit the DHCP configuration file so that it points to the Cobbler server and PXE boot image. As root on the DHCP server, edit the /etc/dhcpd.conf file and append a new class with options for performing PXE boot installation. For example:
allow booting; allow bootp; class "PXE" { match if substring(option vendor-class-identifier, 0, 9) = "PXEClient"; next-server 192.168.2.1; filename "pxelinux.0"; }
Following each action step-by-step in the above example: 1. The administrator enables network booting with the bootp protocol. 2. Then, the administrator creates a class called PXE, which, if a system that is configured to have PXE first in its boot priority, identifies itself as PXEClient. 3. Then DHCP server then directs the system to the Cobbler server at 192.168.2.1. 4. Finally, the DHCP server refers to the boot image file (in this case, at /var/lib/tftpboot/ pxelinux.0. 44

/sbin/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 69 -j ACCEPT /sbin/iptables -A INPUT -m state --state NEW -m udp -p udp --dport 69 -j ACCEPT

For HTTPD:

/sbin/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT /sbin/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT

For Cobbler:

/sbin/iptables -A INPUT -m state --state NEW -m tcp -p udp --dport 25150 -j ACCEPT

For Koan:

/sbin/iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 25151 -j ACCEPT
Once those firewall rules are entered, be sure to save the firewall configuration:

/sbin/iptables-save

5.8.1.2.4. Syncing and Starting the Cobbler Service
Once all the prerequisites specified in cobbler check are met, you can now start the Cobbler service. First, ensure that the configuration files are all synchronized by running the following command:

cobbler sync

Then, start the Satellite server:
/usr/sbin/rhn-satellite start
Do not start or stop the cobblerd service independent of the Satellite service, as doing so may cause errors and other issues. Always use /usr/sbin/rhn-satellite to start or stop RHN Satellite.
5.8.1.2.5. Cobbler configuration
Cobbler is already set up to generate PXE configurations, but you may want to adjust the pxe_just_once configuration option depending on how your machines BIOSes are configured, for the best possible PXE workflow. A common setup has PXE occur first in the BIOS order, effectively not booting off the local disk unless the PXE server instructs the system to do so remotely. By having pxe_just_once: 1 (enabled) in / etc/cobbler/settings, it will prevent "boot loops" where the system continually reinstalls. What happens is that the $kickstart_done macro in the kickstart templates will expand into a directive that indicates to the cobbler server that the system will then boot locally, instead of booting from the network. Then, to reinstall the system, the netboot-enabled flag on the system can be toggled back on via the Satellite GUI or Cobbler. Once enabled, the next time the system power cycles it will PXE install instead of booting locally. At the end of each install, the server will trip the netboot-enabled flag back to off again to tell the system to boot to the local hard drive the next time it powers up. Note that if your kickstart is missing the $kickstart_done line in %post, this will not work, and boot loops will occur. With pxe_just_once set to 0, the netboot enabled flag will not be disabled after an install, so if PXE is first in your BIOS boot order, the system will loop indefinitely. If you have the BIOS of the system set up to boot to local hard drives first, though, there is no need to set pxe_just_once enabled, but to re-PXE a system it is then neccessary to zero out the MBR of that system.

5.8.1.2.6. Cobbler System Record
Cobbler system records are objects within cobbler that keep track of a system and its associated kickstart profile. To do PXE kickstarting you'll need to ensure that a Satellite kickstart profile is tied to Cobbler system records corresponding to the machines you intend to PXE kickstart to that profile. To make this association: 1. Visit the System details page of each system in question and click on the Provisioning link 2. Select the kickstart profile you want to associate it with 3. Click the Create Cobbler System Record button. Once you've made this association, it will remain in place forever unless you have set pxe_just_once to true in cobbler for any given machine. In that case the association will be broken after a successful kickstart. Without this association, a machine that PXE bootstraps to a Satellite server will be presented with a menu of kickstart profiles which requires manual interaction.
5.8.1.3. Cobbler Boot ISO
The Cobbler boot ISO is a disk image that can be built on your Satellite server and burned to a CD or DVD. You can then boot any system with it. When you do you will see a menu of available kickstarts similar to the one you would see if you PXE boot a machine off a Cobbler server without a system 47
Chapter 5. Provisioning with Satellite record. Simply select the kickstart you want, and the system will start to install itself. Any time you add a kickstart within Satellite, you will need to recreate the ISO and re-burn it to an optical disc. To create a boot ISO, log in to your Satellite server as root and run cobbler buildiso. The ISO will contain all kernel/initrd images stored in your Satellite along with all associated kernel argument settings. Kickstart files will be sourced remotely. This means that changes to the kickstart templates can be made without having to re-burn the CD. If you create a new kickstart profile and want to use it via the cobbler boot ISO, you will need to recreate a fresh disc.
Due to issues with the version of syslinux shipped with Red Hat Enterprise Linux 4, this command will not work unless the Satellite is running on Red Hat Enterprise Linux 5. Also since syslinux is not available for s390x, it is not possible to use this on a satellite running on s390x.

5.8.2. Re-Provisioning

Re-provisioning is the act of reinstalling an existing system. It could be reinstalled to the same version and release, or to a completely new version. When you re-provision through the Satellite web interface your system will use the same system profile that it had before it was re-provisioned. This can be useful as much of the information and settings about the system such as its history will be preserved. You can schedule a re-provision from the provisioning tab while viewing a system. If you would like to configure additional options click on the Advanced Options page. On this page you can configure details such as kernel options, networking information, and package profile synchronizations. The Kernel Options section pertains to kernel options used during kickstart. Post Kernel Options are the kernel options that will be used after the kickstart is complete and the system is booting for the first time. For example: If you want to establish a VNC connection so you can monitor the kickstart remotely, include vnc vncpassword=PASSWORD in the Kernel Options line If you want the kernel of the resulting system to boot with the noapic kernel option, add noapic to the Post Kernel Options line Note that this requires a system that is accessible over your network and already registered to Satellite. If you are reinstalling a system that is not registered to Satellite, there are several options: PXE Use cobbler buildiso Install koan and spacewalk-koan on the system and use the koan command line tool, pointing at the Satellite server, to install a named profile Koan is covered in a later section.

kickstart.keys

kickstart.profile

kickstart.profile.keys

Cobbler On the Command Line XML-RPC Namespace kickstart.profile.software kickstart.profile.system Usage manipulate the list of packages associated to a kickstart profile. manage file preservations, manage cryptography keys, enable/ disable config management and remote commands, setup partitioning schemes, and setup locale information associated to a given kickstart profile.
Additionally, the following API methods calls may be used to re-provision a host and schedule guest installs. system.provision_system system.provision_virtual_guest For more information on these API calls and others refer to the API documentation available on https://sat FQDN/rhn/rpc/api replacing sat FQDN with your Satellite server.
5.9.2. Cobbler On the Command Line
Satellite uses Cobbler to facilitate provisioning. When the kickstart profiles, trees (distributions) and systems for provisioning are updated in satellite, they are synchronized to the Cobbler instance on the Satellite host. This means that you can use cobbler directly to manage their provisioning if you prefer. To get a list of profiles run the following command in a shell on host where the satellite is installed:
sudo cobbler profile list
To get a list of kicktstart trees (and kernels, ramdisks, and other options) run:

sudo cobbler distro list

To get a list of system records (which are created when a kickstart is scheduled) run:

sudo cobbler system list

To show more detailed output about a specific object, use the "report" command:
sudo cobbler profile report --name=profile-name sudo cobbler system report --name=system-name
Various parameters can be tweaked just as with the Satellite Web UI, for instance, asking that each virtualized install of a given profile get 1 GB of RAM:
sudo cobbler profile edit --name=profile-name --virt-ram=1024
5.9.3. Cobbler Command Line: Next Steps
Setting a system (see pxe_just_once above) to be reinstalled at next reboot:

Breaking this down into the 4 steps you have:
1) mkdir -p /usr/share/rhn/
Creating a directory to house the custom SSL cert used by the RHN Satellite
2) wget http://satellite.example.com/pub/RHN-ORG-TRUSTED-SSL-CERT -O /usr/share/rhn/RHN-ORGTRUSTED-SSL-CERT
Fetch the SSL certificate to use during registration
3) perl -npe 's/RHNS-CA-CERT/RHN-ORG-TRUSTED-SSL-CERT/g' -i /etc/sysconfig/rhn/*
Search/replace the SSL certificate strings from the rhn-register configuration files.
4) rhnreg_ks --serverUrl=https://satellite.example.com/XMLRPC --sslCACert=/usr/share/rhn/RHNORG-TRUSTED-SSL-CERT --activationkey=1-c8d01e2f23c6bbaedd0f6507e9ac079d
Register to the RHN Satellite with the SSL certificate and an activation key. Every Kickstart Profile includes an Activation Key that assures that the system is assigned the correct base and child channels, gets the proper System Entitlements, and is associated with the previous System Profile if you are re-provisioning an existing system. If the rhnreg_ks command fails you may see errors in the ks-post.log indicating:
ERROR: unable to read system id.
And calls to rhn_check return the error above you know the system failed to register to the RHN Satellite. The best way to troubleshoot this is to view the kickstart file and copy-paste the four steps from above into the shell prompt and run them after the system comes back from kickstarting. Generally rhnreg_ks will produce usable error messages which should help you figure out what is failing during registration. 58
Directory structure for Kickstarts and Snippets
5.10.7. Directory structure for Kickstarts and Snippets
Kickstarts The base path where the kickstart files are stored is /var/lib/rhn/kickstarts/. Within this directory, raw (non-wizard generated) kickstarts reside in the subdirectory upload while wizard generated ones are in the wizard subdirectory, thus:
Raw Kickstarts: /var/lib/rhn/kickstarts/upload/$profile_name--$org_id.cfg Wizard Kickstarts: /var/lib/rhn/kickstarts/wizard/$profile_name--$org_id.cfg
Snippets Cobbler Snippets are stored in /var/lib/rhn/kickstarts/snippets. Cobbler accesses snippets in this structure via a symbolic link in /var/lib/cobbler/snippets callled spacewalk thus /var/lib/cobbler/snippets/spacewalk. Satellite's RPMs expect Cobbler's kickstart and snippet directories to be in their default locations it is not recommended to alter them.

Snippets:

/var/lib/rhn/kickstarts/snippets/$org_id/$snippet_name

Part III. Working with multiple Satellites
This part describes how to work with multiple Satellite instances, including managing Satellite interaction with Red Hat Network Hosted, the remote network from which Satellite receives supported errata and content updates officially from Red Hat, Inc.

Chapter 6.

Inter-Satellite Sync (ISS)
As an organization grows, so does complexity of the deployed systems. Red Hat Network Satellite deployment can grow with your IT infrastructure. One capability that assists with managing scale and complexity is the deployment of multiple RHN Satellites configured to communicate with each other. Each Satellite manages a segment of an organization's deployment, minimizing the inefficiencies of both massive scaling and dispersed data centers. With multiple Satellites, an organization can infinitely scale upward, allow semi-autonomous management of IT segments, and ensure maximum management performance of systems closest to the managing Satellite while enforcing content standardization and life-cycle best practices. RHN Satellite 5.3 supports synchronization between two Satellites. This synchronization, also called Inter-Satellite Sync, allows administrators to simplify the process of coordinating content from one RHN Satellite source to another or several others.
6.1. Required Environment for ISS
The following are the basic requirements for Inter-Satellite Sync. At least two RHN Satellite 5.3 servers At least one RHN Satellite populated with at least one channel Master RHN Satellite SSL certificate available on each of the slave RHN Satellites for secure connection
6.2. Recommended Models for Inter-Satellite Sync
The Inter-Satellite Sync feature for Satellite provides facilities for synchronizing content between two or more Satellites. The following are some of the more typical uses that show the possibilities of Inter-Satellite Sync and help guide you in determining how to make the most of this feature in your environment.
If you are not sure if the Inter-Satellite Sync feature is right for your organization, please note that you can continue to use RHN Satellite 5.3 in the typical manner. Installing or upgrading to RHN Satellite 5.3 does not require that you make use of this feature.
Figure 6.1. Staging Satellite In this example, the Stage Satellite is used to prepare the content and perform quality assurance (QA) work to make sure that packages are fit for production use. After content is approved to go to production, the Production Satellite will then synchronize the content from the Stage Satellite.