How to Install Oracle RAC 12c (12.2) on Linux with ASM

Background & Overview

The following documentation provides instructions for building an Oracle 12c database with ASM residing on RedHat. 12c comes with a number of new features and architecture changes. Flex ASM and Container Databases to name just two major changes. This guide shows how to install a standard clustered database, the type used in 11g. It does not make use of Flex ASM or Container Databases.

Further reference documentation about Oracle 12c and Oracle Linux is detailed in the Other References section below.

You must also have sufficient disk space in the software installation locations to store the Oracle software, as described in the following table.

Location	Amount	Purpose
Grid home directory	At least 8 GB	Software installation of Oracle Clusterware and Oracle Automatic Storage Management (Oracle ASM)
Grid home directory	100 GB is recommended	Additional disk space for the associated log files and patches
Oracle base of the Oracle Grid Infrastructure installation owner (Grid user)	At least 10 GB	Oracle Clusterware and Oracle ASM log files and for diagnostic collections generated by Trace File Analyzer (TFA) Collector
Oracle home	At least 6.4 GB	Oracle Database software binaries

Steps

First Step: Configure Operating System
Second Step: Install Grid Infraestructure
Third Step: Install the Database Software
Fourth Step: Create ASM Disk Group For DATA and FRA
Fifth Step: Create the Database
Extra Information
Very good Links

First Step: Configure Operating System

Ensure the Oracle Linux version is 6 or higher.

cat /etc/*release*

Ensure the Oracle Linux Kernel version for both nodes is 2.6.32 or higher.

uname -r

Set the kernel parameters in /etc/sysctl.conf for all nodes as follows. NOTE If the current value for any parameter is higher than the value listed below, do not change the value of that parameter.

Parameter	Value
shmmax	Half the server memory
shmmni	4096 (or greater)
fs.file-max	6815744 (or greater)
fs.aio-max-nr	1048576 (or greater)
net.core.rmem_default	262144 (or greater)
net.core.rmem_max	4194304 or greater)
net.core.wmem_default	262144 (or greater)
net.core.wmem_max	1048576 (or greater)
sem	250 32000 100 128 (or greater)
net.ipv4.ip_local_port_range	9000 65535
vm.nr_hugepages=512	Needs to be set to the total SGA size of the DB and ASM plus 750MB. (Each page is 2048)
panic_on_oops	1
randomize_va_space	0
exec-shield	0
vm.min_free_kbytes	512mb
rp_filter	0 or 1

Add the following lines to the "/etc/sysctl.conf" file, or in a file called "/etc/sysctl.d/98-oracle.conf".

# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

# oracle-database-preinstall-19c setting for fs.file-max is 6815744
fs.file-max = 6815744

# oracle-database-preinstall-19c setting for kernel.sem is '250 32000 100 128'
kernel.sem = 250 32000 100 128

# oracle-database-preinstall-19c setting for kernel.shmmni is 4096
kernel.shmmni = 4096

# oracle-database-preinstall-19c setting for kernel.shmall is 1073741824 on x86_64
kernel.shmall = 1073741824

# oracle-database-preinstall-19c setting for kernel.shmmax is 4398046511104 on x86_64
kernel.shmmax = 4398046511104

# oracle-database-preinstall-19c setting for kernel.panic_on_oops is 1 per Orabug 19212317
kernel.panic_on_oops = 1

# oracle-database-preinstall-19c setting for net.core.rmem_default is 262144
net.core.rmem_default = 262144

# oracle-database-preinstall-19c setting for net.core.rmem_max is 4194304
net.core.rmem_max = 4194304

# oracle-database-preinstall-19c setting for net.core.wmem_default is 262144
net.core.wmem_default = 262144

# oracle-database-preinstall-19c setting for net.core.wmem_max is 1048576
net.core.wmem_max = 1048576

# oracle-database-preinstall-19c setting for net.ipv4.conf.all.rp_filter is 2
net.ipv4.conf.all.rp_filter = 2

# oracle-database-preinstall-19c setting for net.ipv4.conf.default.rp_filter is 2
net.ipv4.conf.default.rp_filter = 2

# oracle-database-preinstall-19c setting for fs.aio-max-nr is 1048576
fs.aio-max-nr = 1048576

# oracle-database-preinstall-19c setting for net.ipv4.ip_local_port_range is 9000 65500
net.ipv4.ip_local_port_range = 9000 65535

Add the following lines to a file called "/etc/security/limits.d/oracle-database-server-12cR2-preinstall.conf" file.

oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    16384
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768
oracle   hard   memlock    134217728
oracle   soft   memlock    134217728

Run one of the following commands to change the current kernel parameters, depending on which file you edited.

/sbin/sysctl -p

Upgrade Packages

On each node

yum upgrade

yum install oracleasm-support

Selinux Disable

On each node

You can do disabled or permissive

vi /etc/selinux/config

SELINUX=disabled

Firewall Close and Disable

On each node

You can open after installation,but you have to give permission to necessery ports.

systemctl stop firewalld.service

systemctl disable firewalld.service

Ensure oracle account and the following groups exist on all nodes: oinstall, dba, asmdba

Create Users and Groups on each node

oracle:x:1000:oracle
onboard:x:1001:
asmdba:x:1002:

oinstall:x:54321:oracle
dba:x:54322:oracle
oper:x:54323:
backupdba:x:54324:
dgdba:x:54325:
kmdba:x:54326:
racdba:x:54330:

groupadd -g 1003 asmadmin
groupadd -g 1004 asmoper
groupadd -g 501 oinstall
groupadd -g 502 dba
groupadd -g 503 oper
groupadd –g 505 asmdba

useradd –u 1000 –g oinstall –G dba,asmdba,oper oracle
passwd oracle

useradd -u 1002 -g oinstall -G asmadmin,asmoper,asmdba grid
passwd grid

Configure Bash Profile of Users

Connect as Oracle On Node 1 and:
vi .bash_profile

# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=wfrac0081.us.oracle.com
export ORACLE_UNQNAME=RACTEST
export ORACLE_BASE=/oracle/db/12.2.0.1
export DB_HOME=$ORACLE_BASE/db_home
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=RACTEST1
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias home='cd /oracle/db/12.2.0.1/db_home'

Connect as Oracle On Node 2 and:
vi .bash_profile

# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=wfrac0082.us.oracle.com
export ORACLE_UNQNAME=RACTEST
export ORACLE_BASE=/oracle/db/12.2.0.1
export DB_HOME=$ORACLE_BASE/db_home
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=RACTEST2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias home='cd /oracle/db/12.2.0.1/db_home'

Connect as Grid user bash_profile on node 1

# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=wfrac0081.us.oracle.com
export ORACLE_UNQNAME=RACTEST
export ORACLE_BASE=/oracle/gridbase/12.2.0.1
export GRID_BASE=/oracle/gridbase/12.2.0.1
export GRID_HOME=/oracle/grid/12.2.0.1/grid_home
export ORACLE_HOME=/oracle/grid/12.2.0.1/grid_home
export ORACLE_SID=+ASM1
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias home='cd /oracle/grid/12.2.0.1/grid_home'

Connect as Grid user bash_profile on node 2

# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_HOSTNAME=wfrac0082.us.oracle.com
export ORACLE_UNQNAME=RACTEST
export ORACLE_BASE=/oracle/gridbase/12.2.0.1
export GRID_BASE=/oracle/gridbase/12.2.0.1
export GRID_HOME=/oracle/grid/12.2.0.1/grid_home
export ORACLE_HOME=/oracle/grid/12.2.0.1/grid_home
export ORACLE_SID=+ASM2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
alias home='cd /oracle/grid/12.2.0.1/grid_home'

Create Directories on each Node as root

mkdir -p /oracle/grid/12.2.0.1/grid_home
mkdir -p /oracle/grid/12.2.0.1/log
mkdir -p /oracle/gridbase/12.2.0.1/
mkdir -p /oracle/db/12.2.0.1/db_home
chown -R oracle.oinstall /oracle/
chown -R grid.oinstall /oracle/grid*
chmod -R 775 /oracle/

Configure ASM Disks on each Node as root

# oracleasm configure -i

Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: done

# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Create partition.

    I will arrive to this:

    ASM1 --> /dev/xvdb 53.6 GB
    ASM2 --> /dev/xvdc 53.6 GB
    DATA --> /dev/xvdd 214.7 GB

fdisk /dev/xvdb
fdisk /dev/xvdc
fdisk /dev/xvdd

oracleasm createdisk ASM1 /dev/xvdb
oracleasm createdisk ASM2 /dev/xvdc
oracleasm createdisk DATA /dev/xvdd

oracleasm scandisks
    Reloading disk partitions: done
    Cleaning any stale ASM disks...
    Scanning system for ASM disks...
    Instantiating disk "ASM1"
    Instantiating disk "ASM2"
    Instantiating disk "DATA"

oracleasm listdisks
    ASM1
    ASM2
    DATA

You have to see disks on each node with oracleasm listdisks command.

ll /dev/oracleasm/disks/
    total 0
    brw-rw----. 1 grid asmadmin 202, 17 Jun 23 12:46 ASM1
    brw-rw----. 1 grid asmadmin 202, 33 Jun 23 12:46 ASM2
    brw-rw----. 1 grid asmadmin 202, 49 Jun 23 12:46 DATA

oracleasm querydisk -d DATA
    Disk "DATA" is a valid ASM disk on device [202,33]

oracleasm querydisk -d ASM1
    Disk "ASM1" is a valid ASM disk on device [202,33]

oracleasm querydisk -d ASM2
    Disk "ASM2" is a valid ASM disk on device [202,33]

How to Configure, Create, Scan, List, Query, Rename, Delete OracleASM disk in Linux
https://www.2daygeek.com/create-scan-list-query-rename-delete-configure-oracleasm-disk-linux/

Configure Passwordless SSH Connection (recommended)

You can configure ssh passwordless connection. Grid installation step can make this step, but sometimes you can get an error.

Create SSH Key on First Node as Grid User:
mkdir ~/.ssh
chmod 700 ~/.ssh
/usr/bin/ssh-keygen -t dsa

Generating public/private rsa key pair.
Enter file in which to save the key (/home/grid/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/grid/.ssh/id_dsa.
Your public key has been saved in /home/grid/.ssh/id_dsa.pub.
The key fingerprint is:

cd /home/grid/.ssh/
cat id_dsa.pub >> authorized_keys

In the .ssh directory, you should see the id_dsa.pub keys that you have created, and the file authorized_keys

scp authorized_keys node2:/home/grid/.ssh/

Your output should be similar to the following, where xxx represents parts of a valid IP address:
[grid@node1 .ssh]$ scp authorized_keys node2:/home/grid/.ssh/
The authenticity of host 'node2 (xxx.xxx.173.152) can't be established.
DSA key fingerprint is 7e:60:60:ae:40:40:d1:a6:f7:4e:zz:me:a7:48:ae:f6:7e.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node1,xxx.xxx.173.152' (dsa) to the list
of known hosts
grid@node2's password:
authorized_keys 100% 828 7.5MB/s 00:00

Using SSH, log in to the node where you copied the authorized_keys file. Then change to the .ssh directory, and using the cat command, add the DSA keys for the second node to the authorized_keys file, clicking Enter when you are prompted for a password, so that passwordless SSH is set up:
[grid@node1 .ssh]$ ssh node2
[grid@node2 grid]$ cd .ssh
[grid@node2 ssh]$ cat id_dsa.pub >> authorized_keys

Repeat these steps from each node to each other member node in the cluster.
When you have added keys from each cluster node member to the authorized_keys file on the last node you want to have as a cluster node member, then use scp to copy the authorized_keys file with the keys from all nodes back to each cluster node member, overwriting the existing version on the other nodes.
To confirm that you have all nodes in the authorized_keys file, enter the command more authorized_keys, and determine if there is a DSA key for each member node. The file lists the type of key (ssh-dsa), followed by the key, and then followed by the user and server. For example:
Copyssh-dsa AAAABBBB . . . = grid@node1
Copyssh-dsa AAAABBBB . . . = grid@node2

The grid user's /.ssh/authorized_keys file on every node must contain the contents from all of the /.ssh/id_dsa.pub files that you generated on all cluster nodes.

Chrony NTP Configuration

Chrony plugin came with Oracle Linux 7 version for ntp sync. You can write your ntp server information in /etc/chrony.conf file.

On each node as root:

vi /etc/chrony.conf
server ntp.yourdomain
systemctl restart chronyd.service
systemctl enable chronyd.service

ifconfig

eth0:   inet 10.22.91.164 netmask 255.255.224.0 broadcast 10.22.95.255

eth1:   inet 192.168.0.81 netmask 255.255.255.0 broadcast 192.168.0.255

virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
        inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255

cat /etc/resolv.conf
nameserver 10.209.76.198
nameserver 10.209.76.197

Configure Hosts File

Your public IP address and your Virtual should be on the same subnet.
On each Node as root:

vi /etc/hosts

#Public
10.22.91.164 wfrac0081.us.oracle.com    wfrac0081
10.22.91.165 wfrac0082.us.oracle.com    wfrac0082

#Virtual
10.22.91.166 wfrac0081-vip.us.oracle.com   wfrac0081-vip
10.22.91.167 wfrac0082-vip.us.oracle.com   wfrac0082-vip

#Private
192.168.0.81 wfrac0081-priv.us.oracle.com    wfrac0081-priv
192.168.0.82 wfrac0082-priv.us.oracle.com    wfrac0082-priv

#SCAN
10.22.91.168 wfrac008182-r.us.oracle.com wfrac008182-r
10.22.91.169 wfrac008182-r.us.oracle.com wfrac008182-r
10.22.91.170 wfrac008182-r.us.oracle.com wfrac008182-r

DNS Register (optional)

If you do not want to take an warning at last step of installation, you can add your alias to your domain server.
Also, you can configure dns information of nodes in /etc/resolv.conf file.

Preinstallation Package Intallation

On each node as root:

yum install oracle-database-server-12cR2-preinstall
reboot

Second Step: Install Grid Infrastructure

Copy and Unzip Grid Files as root

cp linuxx64_12201_grid_home.zip /oracle/grid/12.2.0.1/grid_home
cd /oracle/grid/12.2.0.1/grid_home
chown grid.oinstall linuxx64_12201_grid_home.zip
su - grid
cd /oracle/grid/12.2.0.1/grid_home
unzip linuxx64_12201_grid_home.zip

Install the following package from the grid home as the "root" user on all nodes.
su -
# Local node.
cd /oracle/grid/12.2.0.1/grid_home/cv/rpm
rpm -Uvh cvuqdisk*

# Remote node.
scp ./cvuqdisk* root@node2:/tmp
ssh root@node2 rpm -Uvh /tmp/cvuqdisk*
exit

Display Config and Start Grid Installation

su –
export DISPLAY=:0.0
xhost +

su – grid
export DISPLAY=:0.0

xhost +
cd $GRID_HOME
./gridSetup.sh

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - Cluster Node Information Step 4 of 160MracDD81 by Oracle Grid ORACLE 12 GRID INFRASTRUCTURE Infrastructure with their Public Hostname Virtual Hostname wfrac0081-vigL wfrac0082-vignuyoracIexom be managed Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storage Option Grid Infrastructure Managem Create ASM Disk Group ASM Password Operating System Groups Installation Location Root script execution Prerequisite Checks Install Product Provide the list of nodes to and Virtual Hostname Public wfrac008Luy wfrac0082uy Hostname oraclecom oraclecom grid Test Remove Setup Use Cluster Configuration File_ OS Password: SH connectivity OS Username: User home is Reuse private shared by the selected nodes and public keys existing in the user home$

GIMR came with Oracle 12c version. Every Oracle Standalone Cluster and Oracle Domain Services Cluster contains a Grid Infrastructure Management Repository (GIMR), or the Management Database (MGMTDB).

The Grid Infrastructure Management Repository (GIMR) is a multitenant database with a pluggable database (PDB) for the GIMR of each cluster. The GIMR stores the following information about the cluster:

Real time performance data the Cluster Health Monitor collects
Fault, diagnosis, and metric data the Cluster Health Advisor collects
Cluster-wide events about all resources that Oracle Clusterware collects
CPU architecture data for Quality of Service Management (QoS)
Metadata required for Rapid Home Provisioning

You have to give at least 40 GB size to GIMR data. I gave 100 GB this system and gave separate disk area for GIMR db.

On the next screen you can say NO or YES to Create the GIMR

If NO:

Select the "No" option, as we don't want to create a separate disk group for the GIMR in this case. Click the "Next" button:

Set the redundancy to "External", click the "Change Discovery Path" button and set the path to "/dev/oracleasm/disks/*". Return to the main screen and select your disks. In this example I selected all, but you can select 1 of them, later using ASMCA you can create more GROUPS over the rest of disks.

Uncheck the "Configure Oracle ASM Filter Driver" option, then click the "Next" button.

If YES

Select YES

Change the Name to GRID, click on "Change Discovery Path" and select /dev/oracleasm/disks/*

Then all the disks will be displayed:

Machine generated
alternative text: Select Disks Disk Path
Idev/oracIeasm/disks/ASMI Idev/oracIeasm/disks/ASM2
Idev/oracIeasm/disks/DATA Show Candidate/Provisioned Disks Size
(in MB) Status 51199 Provisioned 51199 Provisioned 204799
Provisioned Change Discovery Path-L

Enter the credentials and click the "Next" button.

Accept the default IPMI option by clicking the "Next" button.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - Failure Isolation Support Step 10 of 180wfracDD81 ORACLE 12 GRID INFRASTRUCTURE Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana em Create ASM Disk Grouo ASM Password Failure Isolation Choose Use To one of the following Failure Isolation Support optiony Intelligent Platform Management Interface (IPMI) ensure successful installation with IPMI enabled. ensure your IPMI drivers are properly installed and enabled Password Do not use Intelligent Platform Management Interface (IPMI)$

Don't register with EM. Click the "Next" button.

Define the proper groups:

Accept the default inventory directory (it was defined on the .bash_profile) by clicking the "Next" button.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer Specify Installation Location step 13 of 18@wfracD081 ORACLE 12 GRID INFRASTRUCTURE Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana em Create ASM Disk Grouo Specify the Oracle Grid Infrastructure for a Cluster Oracle base ay default. Oracle Grid Infrastructure is installed in a path indicating the Oracle Grid Infrastructure release and grid infrastructure software owner Oracle base: loracIe/gridbase/122ffI This software directory is the Oracle Grid Infrastructure home directory Software location: loracIe/grid/122ffI/grid home Browse_$

If you want the root scripts to run automatically, enter the relevant credentials. I prefer to run them manually. Click the "Next" button.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - Step 15 of IgavfracDD81 ORACLE 12 GRID INFRASTRUCTURE Root script execution Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana Create ASM Disk Grouo ASM Password Failure Isolation Manaaement Ootions Ogeratina Svstem Grouos Installation Location configuration During the software configuration. certain operations have to be performed as "root" user You can choose to have the installer perform these operations automatically by specifying inputs for one of the options belowL The input specified will also be used by the installer to perform additional prerequisite check& Automatically run configuration scripts @ Use "root" user credential Password C) Use sudo Program path Password lusr/bin/sudo$

Wait while the prerequisite checks complete. If you have any issues use the "Fix & Check Again" button.

Fix & Check Again button creates fix scripts to resolve warnings.

You have to run this scripts on both nodes to resolve warnings.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - Perform Prerequisite Checks Step 16 of I gavfracDD81 ORACLE GRID INFRASTRUCTURE Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana Create ASM Disk Grouo ASM Password Failure Isolation Manaaement Ootions Ogeratina Svstem Grouos Installation Location Verification Result Some of the minimum requirements for installation are not completed Review and fix the issues listed in the following table. and recheck the systenm All Nodes Status Ignore All Check Again Checks Swap Size Show Failed Checks Hard Limit: maximum open file descriptors Soft Limit: maximum stack size Daemon "avahi-daemon" not configured and running Device Checks for ASM Device Checks for ASM ASM device sharedness check ASM device sharedness check Shared Storage Accessibility:/dev/oracIeasm/disks/DATA./dev/oracIeasrrWarning$

$Machine generated alternative text: Fix up ScriptOMracDD81 Some of the prerequisites have failed on the following nodey Installer has generated a fixup script that needs to be run as a privileged user (root) on the listed nodey Script: Nodes: /tmp/GridsetupActions2020-06-so 08-36-00AM/CVU 12 2ffL0 grid/runfixup.h wfrac0082 wfrac0081 OK" to continue To execute the fixup script: I Open a terminal window 2 Login as "root" S Run the script 4 Return to this window and click "$

Once possible fixes are complete, check the "Ignore All" checkbox and click the "Next" button. It is likely the "Physical Memory" and "Network Time Protocol (NTP)" tests will fail for this type of installation. This is OK.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - Perform Prerequisite Checks Step 16 of IgavfracDD81 ORACLE 12 GRID INFRASTRUCTURE Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana Create ASM Disk Grouo ASM Password Failure Isolation Manaaement Ootions Ogeratina Svstem Grouos Review and fix the issues listed in the following table. and recheck the Ignore All Status Verification Result Some of the minimum requirements for installation are not completed systenm Check Again Checks Fix & Check Again Checks Swap Size Device Checks for ASM Device Checks for ASM ASM device sharedness check ASM device sharedness check Shared Storage Ignored Ignored$

If you are happy with the summary information, click the "Install" button.

$Machine generated alternative text: Oracle Grid Infrastructure 12c Release 2 Installer - step 17 of ORACLE 12 GRID INFRASTRUCTURE asmadmin (OSASM) (l Summary Confiauration Ootion Cluster Confiauration Grid Plua and Plav Cluster Node Information Network Interface Usaae Storaae Ootion Grid Infrastructure Mana Create ASM Disk Grouo ASM Password Failure Isolation Manaaement Ootions Ogeratina Svstem Grouos Installation Location Create Inventorv Root scriot execution Prereauisite Checks Su mma ry Oracle Grid Infrastructure 12c Release 2 Installer Global Settings Config Option: Configure Oracle Grid Infrastructure for a New Cluster (l Oracle base for Oracle Grid Infrastructure: loracIe/gridbase/122ffI (l Grid home: loracIe/grid/122ffI/grid home Privileged Operating System Groups: asmdba (OSDBA). asmoper (OSOPER). Root script execution configuration: Manual configuration (l Inventory information Inventory location: loracle/gridbase/oralnventory (l Central inventory (oralnventory) group: oinstall (l Management information Management method: None (l Grid Infrastructure Settings Cluster Configuration: Standalone Cluster (l Cluster Name: wfrac-cluster (l Hub nodes: wfrac0081.wfrac0082 (l Single Client Access Name (SCAN): wfrac008182-ruyoracIexom (l SCAN Port: 1521 Public Interface(s): etho (l ASM & Private Interface(s): ethl (l$

When prompted, run the configuration scripts on each node. Please do it one at a time

$Machine generated alternative text: Execute Configuration ScriptsavfracDD81 The following configuration scripts need to be executed as the "root" user on each listed cluster node Each script in the list below is followed by a list of nodes on which it has to be executed loracIe/gridbase/oraInventoMorainstRootsh loracIe/grid/122ffI/grid home/rootsh To execute the configuration scripts: L Open a terminal window 2 Login as "root" Run the scripts wfrac0081. wfrac0081. Nodes wfrac0082 wfrac0082 Return to this window and click "OK" to continue Run the script on the local node first After successful completion. you can start the script in parallel on all other nodey$

This process takes some time

When this is done, You can check your grid status with crs_stat -t and crsctl stat res -t commands with grid user.

crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora....SM.lsnr ora....er.type ONLINE ONLINE wfrac0081

ora.DATA.dg ora....up.type ONLINE ONLINE wfrac0081

ora....ER.lsnr ora....er.type ONLINE ONLINE wfrac0081

ora....AF.lsnr ora....er.type OFFLINE OFFLINE

ora....N1.lsnr ora....er.type ONLINE ONLINE wfrac0082

ora....N2.lsnr ora....er.type ONLINE ONLINE wfrac0081

ora....N3.lsnr ora....er.type ONLINE ONLINE wfrac0081

ora.MGMTLSNR ora....nr.type ONLINE ONLINE wfrac0081

ora.asm ora.asm.type ONLINE ONLINE wfrac0081

ora.chad ora.chad.type ONLINE ONLINE wfrac0081

ora.cvu ora.cvu.type ONLINE ONLINE wfrac0081

ora.mgmtdb ora....db.type ONLINE ONLINE wfrac0081

ora....network ora....rk.type ONLINE ONLINE wfrac0081

ora.ons ora.ons.type ONLINE ONLINE wfrac0081

ora.proxy_advm ora....vm.type OFFLINE OFFLINE

ora.qosmserver ora....er.type ONLINE ONLINE wfrac0081

ora.scan1.vip ora....ip.type ONLINE ONLINE wfrac0082

ora.scan2.vip ora....ip.type ONLINE ONLINE wfrac0081

ora.scan3.vip ora....ip.type ONLINE ONLINE wfrac0081

ora....81.lsnr application ONLINE ONLINE wfrac0081

ora....081.ons application ONLINE ONLINE wfrac0081

ora....081.vip ora....t1.type ONLINE ONLINE wfrac0081

ora....82.lsnr application ONLINE ONLINE wfrac0082

ora....082.ons application ONLINE ONLINE wfrac0082

ora....082.vip ora....t1.type ONLINE ONLINE wfrac0082

crsctl stat res -t

--------------------------------------------------------------------------------

Name Target State Server State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.ASMNET1LSNR_ASM.lsnr

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.DATA.dg

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.LISTENER.lsnr

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.chad

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.net1.network

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.ons

ONLINE ONLINE wfrac0081 STABLE

ONLINE ONLINE wfrac0082 STABLE

ora.proxy_advm

OFFLINE OFFLINE wfrac0081 STABLE

OFFLINE OFFLINE wfrac0082 STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE wfrac0082 STABLE

ora.LISTENER_SCAN2.lsnr

1 ONLINE ONLINE wfrac0081 STABLE

ora.LISTENER_SCAN3.lsnr

1 ONLINE ONLINE wfrac0081 STABLE

ora.MGMTLSNR

1 ONLINE ONLINE wfrac0081 169.254.10.200 192.1

68.0.81,STABLE

ora.asm

1 ONLINE ONLINE wfrac0081 Started,STABLE

2 ONLINE ONLINE wfrac0082 Started,STABLE

3 OFFLINE OFFLINE STABLE

ora.cvu

1 ONLINE ONLINE wfrac0081 STABLE

ora.mgmtdb

1 ONLINE ONLINE wfrac0081 Open,STABLE

ora.qosmserver

1 ONLINE ONLINE wfrac0081 STABLE

ora.scan1.vip

1 ONLINE ONLINE wfrac0082 STABLE

ora.scan2.vip

1 ONLINE ONLINE wfrac0081 STABLE

ora.scan3.vip

1 ONLINE ONLINE wfrac0081 STABLE

ora.wfrac0081.vip

1 ONLINE ONLINE wfrac0081 STABLE

ora.wfrac0082.vip

1 ONLINE ONLINE wfrac0082 STABLE

--------------------------------------------------------------------------------

Third Step: Install the Database Software

You can unzip database file under /tmp/ and you have to give permission oracle user.

mkdir /tmp/database

chown oracle.oinstall –R /tmp/database/

cp linuxx64_12201_database.zip /tmp/database/

cd /tmp/database/

unzip linuxx64_12201_database.zip

cd database

./runInstaller

Select the "Install database software only" option, then click the "Next" button.

$Machine generated alternative text: Oracle Database 12c Release 2 Installer - Select Installation Option Step 2 of gavfracDD81 ORACLE 12 DATABASE Confiaure Securitv Ugdates Installation Option Database Installation O tion Typical Installation Prerequisite Checks Select any of the following install optiony Create and configure a database Install database software onl Upgrade an existing database$

Accept the "Oracle Real Application Clusters database installation" option by clicking the "Next" button.

$Machine generated alternative text: Oracle Database 12c Release 2 Installer - Step 3 of gawfracDOeI Select Database Installation Option ORACLE 12 DATABASE Confiaure Securitv Ugdates Installation Ootion Database Installation Op Install Tvoe Typical Installation Prerequisite Checks Select the type of database installation you want to perfornm Single instance database installation Oracle Real Application Clusters database installation Oracle RAC One Node database installation$

Make sure both nodes are selected. Also setup SSH Connectivity if needed

$Machine generated alternative text: Oracle Database 12c Release 2 InstallerawfracDOeI Successfully established passwordless SSH connectivity between the selected nodey$

Then click the "Next" button.

$Machine generated alternative text: Oracle Database 12c Release 2 Installer - Select List of Nodes Step 4 of 100MracDD81 ORACLE 12 DATABASE Select nodes (in addition to the local node) in the cluster where the installer should install Oracle Confiaure Securitv Ugdates RAC or Oracle RAC One Deselect all Installation Ootion Database Installation O tion Nodes Selection Install Tvoe Typical Installation Prerequisite Checks Install Product wfrac0081 wfrac0082 SH connectivity oracle OS Username: Node name Select all OS Password: Setup User home is shared by the selected nodes Reuse private and public keys existing in the user home Test$

Select the "Enterprise Edition" option, then click the "Next" button.

Define the locations , then click the "Next" button.

Select the desired operating system groups, then click the "Next" button.

$Machine generated alternative text: Oracle Database 12c Release 2 Installer - Step 7 of I I avfracDD81 Privileged Operating System groups ORACLE 12 DATABASE Confiaure Securitv Ugdates Installation Ootion Database Installation O tion Nodes Selection Database Edition Installation Location Operating System Groups Prereauisite Checks Install Product SYS privileges are required to create a database using operating system (OS) authenticatiorm Membership in OS Groups grants the corresponding SYS privilege. membership in OSDBA grants the SYSDBA privilege Database Administrator (OSDBA) group: Database Operator (OSOPER) group (Optional): Database Backup and Recovery (OSBACKUPDBA) group: Data Guard administrative (OSDGDBA) group: Encryption Key Management administrative (OSKMDBA) group: Real Application Cluster administrative (OSRACDBA) group:$

Wait for the prerequisite check to complete. If there are any problems either click the "Fix & Check Again" button, or check the "Ignore All" checkbox and click the "Next" button.

$Machine generated alternative text: Oracle Database 12c Release 2 Installer - Step 8 of I I avfracDD81 Perform Prerequisite Checks ORACLE 12 DATABASE Confiaure Securitv Ugdates Installation Ootion Database Installation O tion Nodes Selection Database Edition Installation Location Ogeratina Svstem Grouos Prerequisite Checks Install Product Verification Result Some of the minimum requirements for installation are not completed Review and fix the issues listed in the following table. and recheck the systenm Fix & Check Again Checks Checks Swap Size Status$

Machine generated alternative text: Oracle Database 12c
Release 2 InstallerOMracDD81 (INS-IS0161 You have chosen to
ignore some of the prerequisites for this installatiorL This may
impact product configuratiorL Are you sure you want to continue
? Details

If you are happy with the summary information, click the "Install" button.

Execute the scripts as root when needed:

cd /oracle/db/12.2.0.1/db_home/

sh root.sh

$Machine generated alternative text: Execute Configuration ScriptsOMracDD81 The following configuration scripts need to be executed as the "root" user on each listed cluster node Each script in the list below is followed by a list of nodes on which it has to be executed loracIe/db/122ffI/db home/rootsh Nodes wfrac0081. wfrac0082$

Please close when done:

Machine generated alternative text: Oracle Database 12c
Release 2 Installer Step I I of 1 ORACLE 12 DATABASE The
installation of Oracle Database was successful Configure
Security Updates Installation Option Database Installation
Option Nodes Selection Database Edition

Fourth Step : Create ASM Disk Group For DATA and FRA

If you need to create more GROUPS, follow these steps (example, I didn't use it):

Create ‘DATA’ disk group with External Redundancy by selecting appropriate candidate disks

Machine generated alternative text: Creat e Disk AS"
Disk Disk 12 See CANO I DATE CANDIDATE .ppex _oatazp apw,'C
_oaTa RECY32pl 2SS999 2SS999 102399 102399 102399 204799

Create two ‘REDO’ disk groups – REDO1 and REDO2 - with External Redundancy by selecting at least one candidate disk per REDO disk group

Machine generated alternative text: Creat e Disk ASH Disk
Nam. _FRAI CANDIDATE 12 MRAsmucruR€ 102399 102399 20.799

Create ‘FRA’ disk group with External Redundancy by selecting appropriate candidate disks

Machine generated alternative text: Cre ate Disk Group ASH
Disk ORAcL_e 12 appe 20"99

Create ‘TEMP’ disk group with External Redundancy by selecting appropriate candidate disks

Machine generated alternative text: Create Disk Group
AS" Create Disk CD CD C) Disk CAW'OATE 12 eat'.. 204799

Verify all required disk groups and click Exit to close from ASMCA utility

Machine generated alternative text: Disk Groups ASH 999
19989 ORACLE' 12 MO _MEO I 89

Change ASM striping to fine-grained for REDO, TEMP and FRA diskgroups as a grid user using below commands

SQL> ALTER DISKGROUP REDO ALTER TEMPLATE onlinelog ATTRIBUTES (fine)

SQL> ALTER DISKGROUP TEMP ALTER TEMPLATE tempfile ATTRIBUTES (fine)

SQL> ALTER DISKGROUP FRA ALTER TEMPLATE onlinelog ATTRIBUTES (fine)

Fifth Step : Create the Database

The following steps are applicable for node 1 of your cluster environment, unless otherwise specified:

Login as oracle user
From $, run the DBCA utility by typing:

cd ORACLE_HOME

dbca

In the Select Database Operation window, select Create Database and click Next

Select the "Advanced Configuration" option.

Review the Defaults and select Next

In the Select List of nodes window select nodes and click Next

In the Specify Database Identification Details window:

In the Storage Options Window select Datafiles storage location and Unselect Use Oracle- Managed Files (OMF) and click Next:

Accept the Warning:

Machine generated alternative text: Database Configuration
AssistantOMracDD81 (DST-110031 ASM storage option is selected
for datafile locatiorL However Oracle Managed Files (OMF) option
is not selected Are you sure you want to continue ? Details

In the Select Fast Recovery Option window, select Specify Fast Recovery Area location and define the size. Thenk click Next

In the Select Oracle Data Vault Config Option window, select default values and click Next

In the Specify Configuration Options window put required SGA and PGA values and click Next

Machine generated alternative text: Use Automatic Shared
Memory Managemen SGA size: PGA Size: 5724 7632 1908 Use Manual
Shared Memory Management Shared pool size: Buffer cache size:
Total memory for database O MB Use Automatic Memory Management

In the Specify Management Options window select "Run Cluster Verification periodically"and click Next

In the Specify Database User Credentials window input password and click Next

In the Select Database Creation Options window, Click on Customize Storage Locations

Then Click on Redo Log Groups and Increase their size to , at least, 800 MB

Machine generated alternative text: Customize
StorageOMracDD81 Storage Summary of Redo Log Groups Update All
Control Files Datafiles Redo Log Groups Name Change All
Size(MB): Size(Ma)

Also Click on each Redo Log File to define its Redo Disk Group Location and multiplex each one of them Click on "Update All" and "Apply". Then click Next

In the Summary window, click Finish to create database

Click Close on the Finish window after the database creation is complete. The RAC database creation is now complete.

Extra Information

Step-By-Step Guide - Configure ASM

1. Create the extra Disk Groups

. oraenv

ORACLE_SID = [oracle] ? +ASM1

sqlplus / as sysasm

SQL> CREATE DISKGROUP DATA01 EXTERNAL REDUNDANCY DISK ‘/dev/asm-data0*' ATTRIBUTE ‘COMPATIBLE.ASM’=’12.1.0.0.0′,’COMPATIBLE.RDBMS’=’12.1.0.0.0′,’AU_SIZE’=’4M';

SQL> CREATE DISKGROUP FRA01 EXTERNAL REDUNDANCY DISK ‘/dev/asm-fra0*' ATTRIBUTE ‘COMPATIBLE.ASM’=’12.1.0.0.0′,’COMPATIBLE.RDBMS’=’12.1.0.0.0′,’AU_SIZE’=’4M';

SQL> CREATE DISKGROUP REDO01 EXTERNAL REDUNDANCY DISK ‘/dev/asm-redo0[1,3,5,7]′ ATTRIBUTE ‘COMPATIBLE.ASM’=’12.1.0.0.0′,’COMPATIBLE.RDBMS’=’12.1.0.0.0′,’AU_SIZE’=’4M';

SQL> CREATE DISKGROUP REDO02 EXTERNAL REDUNDANCY DISK ‘/dev/asm-redo0[2,4,6,8]′ ATTRIBUTE ‘COMPATIBLE.ASM’=’12.1.0.0.0′,’COMPATIBLE.RDBMS’=’12.1.0.0.0′,’AU_SIZE’=’4M';

2. Update parameter file with new Diskgroups

SQL> ALTER SYSTEM SET ASM_DISKGROUPS=‘DATA01′,‘FRA01′,‘REDO01′,‘REDO02′ SCOPE=BOTH SID=’*’

SQL> ALTER DISKGROUP VOTE01 SET ATTRIBUTE ‘compatible.rdbms’ = ‘12.1.0.0.0’

SQL> quit

srvctl start diskgroup -g DATA01

srvctl start diskgroup -g FRA01

srvctl start diskgroup -g REDO01

srvctl start diskgroup -g REDO02

3. Configure ASM to use huge pages

. oraenv

ORACLE_SID = [oracle] ? +ASM1

SQL> sqlplus / as sysasm

SQL> alter system set memory_max_target=0 scope=spfile sid=’*’;

SQL> alter system set memory_target=0 scope=spfile sid=’*’;

SQL> alter system set sga_target=1088M scope=spfile sid=’*’;

SQL> alter system set use_large_pages=’ONLY’ scope=spfile sid=’*’;

SQL> quit

sudo /u01/app/12.1.0/grid/bin/crsctl stop cluster -all

sudo /u01/app/12.1.0/grid/bin/crsctl start cluster -all

Step-By-Step – Database Configuration (Optional and Site Specific)

1. Log on as Oracle

2. Update the oratab

ssh node1

. oraenv

vi /etc/oratab

Add the following line

MYRACDB01:/u01/app/oracle/product/12.1.0/dbhome_1:N

ssh node2

. oraenv

vi /etc/oratab

Add the following line

MYRACDB02:/u01/app/oracle/product/12.1.0/dbhome_1:N

3. Set some database options and rename the spfile in ASM

. oraenv

ORACLE_SID = [oracle] ? RACTEST1

SQL> sqlplus / as sysdba

SQL> alter database flashback on;

SQL> alter database force logging;

SQL> alter system set cursor_sharing=force scope=spfile sid=’*';

SQL> alter system set control_file_record_keep_time =14 scope=spfile sid=’*’;

SQL> alter system set use_large_pages =’ONLY’ scope=spfile sid=’*’;

SQL> BEGIN;

SQL> exec dbms_auto_task_admin.disable(client_name => ‘auto optimizer stats collection’, operation => NULL, window_name => NULL);

SQL> alter system set undo_retention=54000 scope=spfile sid=’*';

SQL> alter system set inmemory_query=’DISABLE’ scope=spfile sid=’*';

SQL> quit

srvctl stop cluster -all

ssh myhost1

cd $ORACLE_HOME/dbs

mv initRACTEST1.ora /u01/app/oracle/admin/RACTEST/pfile/initRACTEST1.ora

ln -s /u01/app/oracle/admin/RACTEST/pfile/initRACTEST1.ora

ssh myhost2

cd $ORACLE_HOME/dbs

mv initRACTEST2.ora /u01/app/oracle/admin/RACTEST/pfile/initRACTEST2.ora

ln -s /u01/app/oracle/admin/RACTEST/pfile/initRACTEST2.ora

srvctl start cluster -all

rman

RMAN> connect target /

RMAN> configure controlfile autobackup on;

RMAN> quit

3. Implement DCD on the databases.

On each node perform the following

cd $TNS_ADMIN

vi sqlnet.ora

Add the following line to the sqlnet.ora file.

SQLNET.EXPIRE_TIME = 10

Security Compliance (Recommended, but optional)

The following steps should be performed once the database has been built to ensure that the environment conforms to Oracle's Best Practices and also ensures that it doesn't fall foul of Internal Audit.

1. Remove the GRANT EXECUTE TO PUBLIC privilege from potentially harmful packages.

sqlplus '/ as sysdba'

SQL> revoke execute on utl_file from public;

SQL> revoke execute on dbms_random from public;

SQL> revoke execute on utl_http from public;

SQL> revoke execute on utl_smtp from public;

SQL> revoke execute on utl_tcp from public;

SQL> exit

2. Ensure the following user-ids' passwords, where applicable, have been changed on ASM and the Database:

SYS

SYSTEM

DBSNMP

ASMSNMP

3. Adjust the database DEFAULT TABLESPACE settings so that the SYSTEM tablespace is not used inappropriately.

sqlplus '/ as sysdba'

SQL> alter database default tablespace users;

SQL> alter user sys default tablespace system;

SQL> alter user system default tablespace system;

4. Correct any users that already have SYSTEM as their DEFAULT TABLESPACE

sqlplus '/ as sysdba'

SQL> select username, default_tablespace from dba_users where default_tablespace = 'SYSTEM';

SQL> alter user <user name> default tablespace <non-SYSTEM tablespace>;

SQL> exit

5. Lock and expire potentially harmful built-in user-ids.

sqlplus '/ as sysdba'

SQL> alter user &user_idpassword expire account lock;

Repeat for each user as required.

SQL> exit

Very Good Links

https://www.fatihacar.com/blog/oracle-12c-r2-12-2-0-1-rac-installation-steps-on-oracle-linux-7-3/
https://oracle-base.com/articles/12c/oracle-db-12cr2-rac-installation-on-oracle-linux-7-using-virtualbox
https://www.dell.com/support/article/en-us/how16671/how-to-deploy-oracle-12c-release-2-grid-and-rac-database-on-rhel-7-x?lang=en
https://oracledbwr.com/step-by-step-guide-to-install-oracle-19c-rac-installation-using-virtualbox/
https://rene-ace.com/category/rac/rac-attack/
https://onlinedbalearning.blogspot.com/2018/12/step-by-step-oracle-rac-12c-release-2.html
http://dbaora.com/oracle-rac-12-1-0-1-on-windows-2008-using-oracle-virtual-box/
http://appsdbaworkshop.blogspot.com/2014/05/oracle-12c-rac-installation-on-linux.html