If you want to learn more or contribute to the project please visit their official Wiki Page.
In addition I have inserted some relevent guidelines regarding Oracle 12c on RHEL 7 from RedHat’s website (Deploying Oracle RAC Database 12c on Red Hat Enterprise Linux 7 - Best Practices)
1. Setup VirtualBOX
2. Create the first node
3. Create shared ASM devices
- Configuring Device Mapper Multipathing
- Using udev Rules instead of Oracle ASMLib
4. Clone node
5. Install Grid Infrastructure 12c Release 1
6. Install Database Software 12c Release 1
7. Create Database
8. Enjoy and experiment
RAM : min 8 GB - actual 10 GB
DISK: min 50 GB - actual 500 GB
* RedHat free Evaluation Subscription can be done by creating an account with a business email address (here).
Oracle software - 1- Login to: http://edelivery.oracle.com/ with your Oracle Account
2- On the Media Pack Search page, select:
Product pack: Oracle Database Platform: Linux x86-64
3- Click GO then Select and download the following products :
Grid Infrastructure 12c Release 1 (184.108.40.206.0) and Oracle Database 12c Release 1 (220.127.116.11.0)
Disk space consumed for Oracle Grid Infrastructure Home and Oracle Database Home Enterprise Edition may vary.
Swap Space (Recommended) :
Filesystem layout to properly install the Oracle Grid Infrastructure and Oracle Database software
1. Setup VirtualBOX Go to Top
a. Network configuration : I have used the following network setup. Feel free to adjust to your needs:
- Hostnames london1,london2 Domaine: Evilcorp.com ( Mr Robot fan here…)
- Public IPs 192.168.78.0/24 (51,52)
- Interconnect 172.16.100.0/24 (51,52)
o Private IPs available from RAC nodes only
o Name => london-cluster-scan.evilcorp.com
o IPs => 192.168.78.251/252/253
1. From the main VBOX screen, choose File > Preferences > Network.
2. Double click on VirtualBox Host-Only Ethernet Adapter. Update the settings as shown and click Ok twice.
IPv4 Address: 192.168.78.1
IPv4 Network Mask: 255.255.255.0
3. Create a virtual hard drive now >
VDI (VirtualBox Disk Image)> Dynamically allocated> Location /London> size 30G
4. Disable USB
5. Allocate the minimum physical memory required ( 3.5G<Memory<4G )
o Storage: Delete the Empty disk under the IDE Controller
> Add CD/DVD Device
> Locate and open the rhel-server-7.1-x86_64- dvd.iso file.
o Network: Adapter 1: Host-only Adapter.
Adapter 2: Internal Network (Name: rac-priv) .
Adapter 3: NAT .
Shared folder 12gR2 : on shared folder link the diretory containing all the installation files and name it 12gR2
You can mount it later using the following command mount -t vboxsf 12gr2 /mnt
2. Create the first node Go to Top1). Linux RedHat Enterprise installation :
A. Click on Start to boot the virtual machine. If asked to select a start-up disk, choose RHEL7 iso.
o Skip the CD testing by pressing the right arrow key and press enter.
o From now on you can use the mouse to navigate. To unlock the mouse and keyboard from the VM, hit the right Ctrl or
B. Under LOCALIZATION
- Adapt or leave the timezone settings as they are and exit.
o KEYBOARD : Leave U.S. English selected and exit (unless you have another preferred keyboard layout).
C. Under INSTALLATION DESTINATION Leave Basic Storage Devices selected and exit.
D. Under NETWORK AND HOSTNAME
In the hostname field, type london1.evilcorp.com .
D.1 Highlight EnpOs3 interface and click the Configure... button.
- Click the IPv4 Settings tab.
Click the Connect automatically checkbox. Click the Method: dropdown and select Manual. Click Add and type 192.168.78.51 for the address. Leave Netmask at 24. In the DNS servers: box, type 192.168.78.51, 192.168.78.52. In the Search domains: box, evilcorp.com then Click the Save .
D.2 Highlight EnpOs8 Interface and click the Configure... button.
Click the IPv4 Settings tab. Click the Connect automatically checkbox. Click the Method: dropdown and select Manual. Click Add and type 172.16.100.51. Change Netmask to 24. Click the Save button.
- Click the IPv4 Settings tab.
E. Under SOFTWARE SELECTION
Click the Connect automatically checkbox. Click the Method: dropdown and select Automatic (DHCP) addresses only. Click the Save button. Click Close to close the NETWORK & HOST NAME menu and click Next.
o Choose infrastructure Server: Select DNS Name Server and Compatibility Libraries Add-Ons then exit.
o Enter the Root Password as racattack (Click Use Anyway when warned about the weak password).
o Click Begin Installation
o When Installation is complete,Click Reboot.
2). Linux post installation configuration:
- We will first have to change the network interface naming from EnpOs3/8/9 to Eth0/1/2 through this *** Tutorial>>.
To connect to the new virtual machine, create a connection in Putty Called london1 with the IP Address of 192.168.78.51. Optionally, create another connection for london2 with the IP Address of 192.168.78.52 that will be created later as a clone of london1.
Open the london1 connection by clicking Open.
- Enter root at the login as: prompt and racattack at the password: prompt.
- Remove special character occuring when hitting backspace on the terminal
o Verify the network configuration via network manager user interface
o In case the hostname wasn't set durring the installation run the following
o Confirm host configuration
- Shared memory allocation
*New in REDHAT 7: tmpfs is a file system that keeps all files and folders in the virtual memory of the operating system.
Here are some of the known purposes of tmpfs:
|/dev contains the special device files for all the devices.|
|/dev/shm contains shared memory allocation|
|/run used for system logs|
|/sys/fs/cgroup used for cgroups, a kernel feature to limit, police and account the resource usage of certain processes|
o Resize swap and dev/shm to 2G (shared memory allocation):
o Setting & Mounting Shared Memory File System on Linux (I actually assigned 3G )
o To enable it at boot time, edit /etc/fstab to include the following entry:
o Check the new swap size
- OS Configuration
Install Required Linux Packages for Oracle Grid Infrastructure and Oracle Database 18.104.22.168 :
o Add a local DVD repository
Add Virtual Box guest addition plugin
- Note: Due to some headers mismatch check and possible execution errors, few rpms may need to be installed first :
- Make the VirtualBox guest additions available to the OS by clicking Devices>Install Guest Additions
o Run the below commands
Note : Some of the Install Guide requirements will already be present from the "default-RPMs" foundation of Linux that you started with (including gcc’s installed above) . The rest is located in the DVD except the compat_libstdc++-33.
- compat-libstdc++-33-3.2.3-72.el7.x86_64 needs to be downloaded from redhat repository (30 days trial) or can be found as Oracle linux equivalent on https://yum.oracle.com/repo/OracleLinux/OL7 or rpm.pbone.net
- Make the linux RHEL7 DVD available to the OS
- Run the below commands to install the required rpms
o Additional rpms for other tools (i.e VNC)face
o Note: The reason behind the non-availability of xorg-x11-twm package is because it has been discontinued in RHEL 7 and replaced by metacity.
Solution : Install the below packages :
o Next , just download and install xorg-x11-twm from your favorite CentOS 6 mirror
o For maximum flexibility you can add the following CentOS 7 packages
o Finally, verify that all the rpms are installed by typing the below rpm commands
o System parameter change : Here is the list of system parameters to be adapted :
Virtual Memory: Shared Memory (SHMMAX, SHMALL, SHMMNI) Semaphores (SEMMSL, SEMMNI, SEMMNS) Ephemeral Network Ports Optimizing Network Settings Setting NOZEROCONF Increasing synchronous I/O Requests Increasing File Handles Kernel Panic On OOPS Parameter Disabling the avahidaemon service (N/A)
Note: Prior to making any changes to the /etc/sysctl.conf, create a backup as follows:
o Adapt the content of sysctl.conf with the bellow lines if they are different or not present.
o For the changes to take effect immediately, run the following command
o Set NOZEROCONF on /etc/sysconfig/network file
Modify this line to ensures that the route 169.254.0.0/16 is not added to the routing table. If the file is empty skip this stepce
o Disable SElinux
SELinux can prevent to establish a correct SSH communication between two hosts, or prevent the creation of ASM disks with the oracleasm tool.To disable SELinux, Edit the file /etc/selinux/config as follow:
o Disable the avahidaemon service if loaded:
o Turn off and disable the Firewall:
If the firewall option is operating, you will need to first manually disable UDP ICMP rejections
*Note: RHEL 7 introduced firewalld in replacement of iptables which makes /etc/sysconfig/iptables no longer relevant.
o Turn UDP ICMP rejections off for all subsequent server reboots (which should always be turned off).
o Add Users and groups
Note : vboxsf group allows access to the shared folders
o Setting Shell Limits for the Grid and Oracle Users
- Or : Alternatively,you can update the resource limits in the /etc/security/limits.conf configuration file for the Oracle installation owner by adding the following lines.
o As the root, create a shell script labeled oracle-grid.sh within /etc/profile.d/ to create the appropriate ulimits
for the oracle and grid user .
o Add the following lines to the /etc/pam.d/login file to use limits, if it does not already exist.
*Note : If you want to reference /lib/security as default library directory create the folder and run the following
o Add two modified .bash_profile to /home/oracle & /home/grid directories respectively
“.bash_oracle_profile” and “.bash_grid_profile”
(rename each of them to .bash_profile and assign a value to ORACLE_SID accordingly )
o Enable ssh: Run as oracle /and grid
o Create Grid oracle Directories
o Create Oracle inventory Directory
o Setup oracle root profiles
o Add following lines to /root/.bashrc
3). RAC DNS configuration
o Edit named.conf : empty-zones-enable no; within options setion will allow one reverse zone for all networks
o Edit forward zone (evilcorp.com)
o Edit reverse Zone (evilcorp.com.rev)
o Generate the rndc.key file. alternative to writing the rndc.conf file ( automatic rndc configuration)
o Restart named service and Enable at boot
*Note: Forwarding requires to add named.service simlink to run 'systemctl enable named.service'.
o Finally check if the DNS configuration is well set
3. Create shared ASM devices Go to Top
o Execute the following command to create 5 disks for ASM (+DATA diskgroup will use 3 ). Each command is in a single line.
Create and attach vm disks
through GUI if necessary)
o Create partition from above added disks
o Configure the option -g for the scsi_id command to expect an UUID from the shared devices.
o Prepare the file /etc/udev/rules.d/99-oracle-asmdevices.rules by running the following script:
- Note: Choose carefully the owner of the asm disks here (OWNER=grid if the installation uses grid user and OWNER=oracle if installation will be done through oracle user).
The file content should look like the below:
o Check the result
4. Clone the node Go to Top
Copy london1 VBOX hdrive rename it to london2 and start a new vm with a name london2 using existing london2.vdi.
- Add virtual machine network configuration as
- Adapter 1 (host only).
- Adapt2 : internal network rac-priv.
- Adapter3 (NAT).
- Add shared file 12rc_RAC
- Attach 4 previous ASM disks
o Startup london2 node and run the network manager tool
- eth0 public IP to 192.168.78.52
- eth1 Private IP to 172.16.100.52
- eth2 DHCP address only
- Configure new device> dns configuration
- Use london2.evilcorp.com as hostname
o Remove the udev network rules file. It will be regenerated on the next reboot
with the new MAC addresses.
o Remove the udev network rules file. It will be regenerated on the next reboot with the new MAC addresses.
o Remove the HWADDR and UUID lines in the network adapter configuration files.
o Complete DNS setup
o Confirm that the asm disk are present on node london2
o Setup the VNC Server on london1 and unzip the installation files
o Use the following command to install the cvuqdisk package on both nodes (this allows Cluster Verification Utility to find the shared disks during installation)
o Adjust the cvu_config to avoid the pdksh "missing package" pre-req failure
5. Install grid infrastructure 12c release 1 Go to Top
o Run Grid Cluster installer
- If the Candidate Disks list is empty
- This will change the two following lines
o Execute configToolAllCommands as grid user on first node:
6. Install the database software 12c release 1 Go to Top
o INSTALL the DBMS software
Warning : SWAP too small+ /dev/shm not permanently mounted
7. Create the 12c database on the cluster Go to Top
o Create +FRA asm diskgroup first
o Run the DBCA:
Click Finish to start the installation
The use of Automatic Memory Management (AMM) is absolutely incompatible with HugePages.
3- Reboot the system to ensure the huge pages setting takes effect properly.
7. Enjoy and experiment Go to Top
For a better preview of the RAC 12c features I invite you to explore it in my next post >> Enjoy and experiment