Installing Oracle 11g R2 on Enterprise Linux 5.5 with ASM

Installing Oracle 11g R2 on Enterprise Linux 5.5 with ASM

 

Note: for any installation, you should check the Release Notes documenation before taking

any practical step.

Installation Environment

 Emulation software: VMWare Workstation 7 for Windows

 OS: Oracle Enterprise Linux 5.5 for x86 64-bit

Required Software

 Oracle Database 11g Release 2 for Linux x86 64-bit

 Oracle Database 11g Release 2 Grid Infrastructure (11.2.0.1.0) for Linux x86 64-bit

Used Hardware

 In the VMWare: create one virtual machine with the following specs:

o 2.5 GB RAM

o One ethernet card configured as bridged or host-only in VMware

o CPU Count: 2

o Disk1: 34 GB on SCSI 0:0 used to install the OS and software

o Disk2: of 12 GB. It will be used for +Data. Set it on controller SCSI 1:1

o Disk3: of 6 GB. It will be used for +Flash. Set it on controller SCSI 1:2

Installation Plan

1. Preinstallation tasks

 Hardware requirements

 Software requirements

 Environment configuration

2. Oracle Grid Infrastructure installation

3. Oracle Grid Infrastructure Patching

4. Oracle Database 11g R2 Software Installation

5. Oracle Database 11g R2 Software Patching

6. Install EM Agent in cluster nodes (if required)

7. ASM Diskgroups Creation

8. Database Creation

9. Complete postinstallation tasks

10. Useful postinstallation tasks

1. Preinstallation tasks

 Install Oracle Enterprise Linux in the first local hardisk. Install nothing in the remaining

disks.

Note: for a production system, consider becoming an Oracle Unbreakable Linux customer

and register your server on the Unbreakable Linux Network.

o Configure the swap area in the local hardisk to have 6 GB disk space.

o Give the ethernet card IP 192.168.4.105 the hostname srv07

o Insall the following packages:

 Desktop Environments

o GNOME Desktop Environment

 Applications

o Graphical Internet (optional)

o Editors (optional)

 Development

o Development Libraries

o Development Tools

 Servers

o Do not select anything in this group.

 Base System

o Administration Tools

o System Tools

 Add the package 'sysstat' by clicking on the Details link

and selecting "sysstat - The sar an iostat system

monitoring commands." from the Optional Packages list.

 X Window System

 Complete the installation

 Install further packages:

# to know distribution and version of Linux

cat /etc/issue

# to know kernel version (and its errata level) (2.6.18-194.el5)

uname -r

# to list missed packages:

rpm -q --qf '%{NAME}-%{VERSION}-%{RELEASE} (%{ARCH})\n' binutils \

compat-libstdc++-33 \

elfutils-libelf \

elfutils-libelf-devel \

gcc \

gcc-c++ \

glibc \

glibc-common \

glibc-devel \

glibc-headers \

ksh \

libaio \

libaio-devel \

libgcc \

libstdc++ \

libstdc++-devel \

make \

sysstat \

unixODBC \

unixODBC-devel

# for missed packages, install them:

rpm -Uvh elfutils-libelf-0.137-3.el5.i386.rpm

rpm -ivh elfutils-libelf-devel-static-0.137-3.el5.i386.rpm elfutils-libelfdevel-

0.137-3.el5.i386.rpm

rpm -Uvh kernel-headers-2.6.18-194.el5.x86_64.rpm

rpm -Uvh glibc-headers-2.5-49.x86_64.rpm

rpm -Uvh glibc-devel-2.5-49.i386.rpm

rpm -Uvh glibc-devel-2.5-49.x86_64.rpm

rpm -Uvh libgomp-4.4.0-6.el5.x86_64.rpm

rpm -Uvh gcc-4.1.2-48.el5.x86_64.rpm

rpm -Uvh gcc-c++-4.1.2-48.el5.x86_64.rpm

rpm -Uvh libstdc++-devel-4.1.2-48.el5.x86_64.rpm

rpm -Uvh gcc-c++-4.1.2-48.el5.x86_64.rpm

rpm -Uvh libaio-devel-0.3.106-5.i386.rpm

rpm -Uvh libaio-devel-0.3.106-5.x86_64.rpm

rpm -Uvh unixODBC-2.2.11-7.1.i386.rpm

rpm -Uvh unixODBC-2.2.11-7.1.x86_64.rpm

rpm -Uvh unixODBC-devel-2.2.11-7.1.i386.rpm

rpm -Uvh unixODBC-devel-2.2.11-7.1.x86_64.rpm

# Oracle ASM Libaray and drivers can be downloaded from here

# to know the kernel verion: uname -rm

# In this case we need:

# library and tools

rpm -Uvh oracleasm-support-2.1.3-1.el5.x86_64.rpm

rpm -Uvh oracleasm-2.6.18-194.el5-2.0.5-1.el5.x86_64.rpm

rpm -Uvh oracleasmlib-2.0.4-1.el5.x86_64.rpm

# SELINUX must be disabled

cat /etc/selinux/config | grep SELINUX=

vi /etc/selinux/config

SELINUX=disabled

shutdown -h now -r

 Check the hardware requirements

# Hardware Requirements (in cluster nodes)

# At least 1.5 GB of physical memory but practically 1.5 is not fine

grep MemTotal /proc/meminfo

# swap space: same as the amount of physical memory

grep SwapTotal /proc/meminfo

# to display swap and memory in one command:

free

# if you don't have enought swap,

# you can add swap space by creating a temporary swap file.

# let's say about 500MB:

dd if=/dev/zero of=tempswap bs=1k count=500000

chmod 600 tempswap

mke2fs tempswap

mkswap tempswap

swapon tempswap

# 1 GB disk space in /tmp

df -h /tmp

# 8 GB of disk space for Oracle software

df

The size of the shared memory should be at least the greater of

MEMORY_MAX_TARGET and MEMORY_TARGET for each Oracle instance on the computer.

To determine the amount of shared memory available, enter the following

command:

df -h /dev/shm/

# to adjust the shared memory file system size:

umount tmpfs

mount -t tmpfs shmfs -o size=1200m /dev/shm

 Create the required network configuration:

ping srv07

# Network names Resolution

cat /etc/hosts

127.0.0.1 srv07 localhost.localdomain localhost

 Create and configure the required OS users and groups

Note: When I tried using differenct OS users and groups for Grid Infrastrucutre and ASM,

I got error later in DBCA.

# Grid Infrastructure (GI) and the Oracle RDBMS home will

# be installed using different users:

/usr/sbin/groupadd -g 501 oinstall

/usr/sbin/groupadd -g 502 dba

/usr/sbin/groupadd -g 504 asmadmin

/usr/sbin/useradd -u 502 -g oinstall -G dba,asmadmin oracle

# set passwords

passwd oracle

# make sure nobody user exists (if not there, create it useradd nobody)

id nobody

# define the env variables for oracle user

vi /home/oracle/.bash_profile

# Oracle Settings

export EDITOR=vi

TMP=/tmp; export TMP

TMPDIR=$TMP; export TMPDIR

ORACLE_HOSTNAME=srv07; export ORACLE_HOSTNAME

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME

ORACLE_SID=oradb; export ORACLE_SID

ORACLE_TERM=xterm; export ORACLE_TERM

PATH=/usr/sbin:$PATH; export PATH

PATH=$ORACLE_HOME/bin:$PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jdk/jre/lib/:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib;

export CLASSPATH

# shell startup file

vi /etc/profile

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

umask 022

fi

# for C shell

vi /etc/csh.login

if ( $USER = "oracle" || $USER = "grid" ) then

limit maxproc 16384

limit descriptors 65536

endif

 Configure kernel parameters and shell limits

Note: If you make a mistake with a parameter setting and your system does not start,

then you must start Linux in the single-user runlevel (runlevel 1). At this runlevel, the

/etc/sysctl.conf file is not run.

# Kernel Parameters

# to tune thme, refer to metalink document 169706.1

# Append the following to the /etc/sysctl.conf file as the root user:

vi /etc/sysctl.conf

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

fs.aio-max-nr = 1048576

fs.file-max = 6815744

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

# to take immediate effect

/sbin/sysctl -p

# User Shell Limits

# memlock is used to increase the per-process max locked memory

vi /etc/security/limits.conf

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

vi /etc/pam.d/login

session required pam_limits.so

 Create partitions in the disks:

Note: On a real life storage, you would create a single whole-disk partition with exactly 1 MB

offset on each LUN to be used as ASM Disk. In fdisk: u (to change units from cylinder to

sectors), n, p, 1, 2048, w.

# as a root, for the disks /dev/sdb .. /dev/sdd

# confirm they are seen:

ls /dev/sd*

#partition the disks:

fdisk /dev/sdb

# answers: "n", "p", "1", "Return", "Return", "p" and "w"

Note: if the following message appears after the "w" command:

WARNING: Re-reading the partition table failed with error 16: Device or

resource busy, then you can avoid restarting the machine by the following

command: partprobe

# to make sure partions are created

ls -lX /dev/sd*

 Configure ASM drivers:

# as root (to be done in all nodes)

oracleasm configure -i

Default user to own the driver interface []: oracle

Default group to own the driver interface []: asmadmin

Start Oracle ASM library driver on boot (y/n) [n]: y

Fix permissions of Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

# Load the kernel module using the following command:

/usr/sbin/oracleasm init

# If you have any problems, make sure you have the correct

# version of the driver (may require Internet connection):

/usr/sbin/oracleasm update-driver

# mark the shared disks: (one node)

/usr/sbin/oracleasm createdisk DISK1 /dev/sdb1

/usr/sbin/oracleasm createdisk DISK2 /dev/sdc1

# check the disks are marked and seen:

/usr/sbin/oracleasm listdisks

#If you need to unmark a disk that was used in a createdisk command:

/usr/sbin/oracleasm deletedisk DISK1

/usr/sbin/oracleasm deletedisk DISK2

 Create the required directories for the Oracle software:

# to know if there is an existing oracle inventory

# from its output, ORACLE_BASE will be parent of oraInventory

more /etc/oraInst.loc

# to identify existing Oracle home directories

more /etc/oratab

# Oracle Inventory Directory

# as a root

mkdir -p /u01/app/oraInventory

chown -R oracle:oinstall /u01/app/oraInventory

chmod -R 775 /u01/app/oraInventory

# Grid Infrastructure Home Directory

mkdir -p /u01/11.2.0/grid

chown -R oracle:oinstall /u01/11.2.0/grid

chmod -R 775 /u01/11.2.0/grid

# Oracle Base Directory

mkdir -p /u01/app/oracle

chown -R oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/app/oracle

#needed to ensure that dbca is able to run after the rdbms installation

mkdir /u01/app/oracle/cfgtoollogs

chown -R oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/app/oracle

# Oracle RDBMS Home Directory

mkdir -p /u01/app/oracle/product/11.2.0/db_1

chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/db_1

chmod -R 775 /u01/app/oracle/product/11.2.0/db_1

2. Oracle Grid Infrastructure installation

# software staging folder

mkdir -p /u01/app/stage/ora11gr2gridinfra

chown -R oracle:oinstall /u01/app/stage/ora11gr2gridinfra

chmod -R 775 /u01/app/stage/ora11gr2gridinfra

mkdir -p /u01/app/stage/ora11gr2db

chown -R oracle:oinstall /u01/app/stage/ora11gr2db

chmod -R 775 /u01/app/stage/ora11gr2db

# as oracle: copy Grid software files to /u01/app/stage/ora11gr2db

# as root:

chmod -R 775 /u01/app/stage/ora11gr2gridinfra

# lunch OUI from the clusterware ( as oracle)

cd /u01/app/stage/ora11gr2gridinfra

./runInstaller

Installation Option

>Select radio button 'Install and Configure Grid Infrastructure for a

Standalone Server'

>Next

Product Language

>Accept 'English' as language'

>Next

Creat ASM Disk Group

>Disk Group Name: Data (12GB disk: DISK1)

>Redundancy: external

>Next

NOTE: If you see an empty screen for you candidate disks it is likely that

ASMLib has not been properly configured or installed. Try reconfigure them.

If you are sure that ASMLib has been properly configured click on 'Change

Discovery Path' and provide the correct destination.

ASM Password

>Specify and conform the password you want to use

>Next

Privileged OS Groups

>Assign the correct OS groups for OS authentication (mostly default is OK)

>Next

Installation Location

>ORACLE_BASE: /u01/app/oracle

Software location: /u01/11.2.0/grid

>Next

Create Inventory

>Specify the locations: /u01/app/oraInventory

>Next

Perform Prerequisite Checks

>Check that status of all checks is Succeeded

>Next

Summary

>Finish

Execute Configuration Scripts

>Run the scripts as instructed in the screen

>OK

>Next

Message: The installation of the Grid Infrastructure was successfull.

>Close

3. Oracle Grid Infrastructure Patching

Apply patch set, if there is any.

4. Oracle Database 11g R2 Software Installation

# as oracle: copy DB software files into /u01/app/stage/ora11gr2db

# as root:

chmod -R 775 /u01/app/stage/ora11gr2db

-- start OUI

su - oracle

cd /u01/app/stage/ora11gr2db

./runInstaller

Follow the steps.

Install database software only.

When executing root.sh, select "y" for ovewriting questions.

5. Oracle Database 11g R2 Software Patching

6. Install EM Agent in cluster nodes (if required)

7. ASM Diskgroups Creation

# as grid user: start the ASM Configuration Assistant (ASMCA)

su - oracle

cd /u01/11.2.0/grid/bin

./asmca

>Disk Groups tab

>Create button

>Disk Group Name: FLASH

>Redundancy: External

>DISK2

>OK

>Exit

>Yes

8. Database Creation

# as oracle

cd /u01/app/oracle/product/11.2.0/db_1/bin

./dbca

follow the steps to create a database with sid: oradb

The DBCA may pop up the following message in the end of db creation:

"Error securing Database control, Database Control has been brought up in nonsecure

mode. To secure the Database Control execute the following commands:

... "

# check Oracle processes:

ps -eo pid -o command | grep ora_ | grep -v grep

9. Postinstallation tasks

# backup the root.sh script

cp /u01/app/oracle/product/11.2.0/db_1/root.sh ~/root.sh.bak

cont>>

10. General Useful Postinstallation Tasks in Linux

 Consider using rlwrap utility with SQL*Plus and RMAN:

o Using rlwrap Utility with RMAN in Unix-Based Systems

o Using rlwrap Utility with SQL*Plus in Unix-Based Systems

/* Make crs_stat -t more readable */

/* copy the following script into ~/scripts/crstat.sh */

#!/usr/bin/ksh

#

# Sample 10g CRS resource status query script

#

# Description:

# - Returns formatted version of crs_stat -t, in tabular

# format, with the complete rsc names and filtering keywords

# - The argument, $RSC_KEY, is optional and if passed to the script, will

# limit the output to HA resources whose names match $RSC_KEY.

# Requirements:

# - $ORA_CRS_HOME should be set in your environment

# suggested scrip name: crstat.sh

RSC_KEY=$1

QSTAT=-u

AWK=/usr/bin/awk # if not available use /usr/bin/awk

# Table header:echo ""

$AWK \

'BEGIN {printf "%-45s %-10s %-18s\n", "HA Resource", "Target", "State";

printf "%-45s %-10s %-18s\n", "-----------", "------", "-----";}'

# Table body:

$ORA_CRS_HOME/bin/crs_stat $QSTAT | $AWK \

'BEGIN { FS="="; state = 0; }

$1~/NAME/ && $2~/'$RSC_KEY'/ {appname = $2; state=1};

state == 0 {next;}

$1~/TARGET/ && state == 1 {apptarget = $2; state=2;}

$1~/STATE/ && state == 2 {appstate = $2; state=3;}

state == 3 {printf "%-45s %-10s %-18s\n", appname, apptarget, appstate;

state=0;}'

# then add the following in the .bashrc of oracle user

# if the file was saved in ~/scripts/crstat.sh

alias crstat='~/scripts/crstat.sh'

/* Easy Acces to crs and db homes */

# it is common to access bin directories in clusterware and db homes

# add the following to .bashrc of oracle user

alias db='cd /u01/app/oracle/product/11.2.0/db_1/bin'

# add the following to .bashrc of grid user

alias crs='cd /u01/app/oracle/crs/bin'