본문 바로가기
Appz Knowledge

RHEL6.8/Oracle RAC 설치 무작정 따라하기-1

by 스쳐가는인연 2016. 10. 31.

Oracle 12c 환경에서 이슈가 있어 테스트 환경 구성차 설치해보았다.
(오라클 설치하는 엔지니어 분들 존경!!! 어려워도 너~ 무~~ 어려운 ㅡ_ㅡ;;)

 

테스트 환경
Model : DL360 Gen9(Intel E5-2630 v3 CPU x 1ea, 16GB x 1ea, P440ar(300GB HDD x 2ea, R1), 96W SSB, 82Q x 1ea) x 2 nodes

Shared Storage Space : 10GB x 3ea LUN on 3PAR Storage
OS : RHEL 6.8

SPP: SPP 2016.04
DB : ORACLE 12c

---------------------------------------------------------------------------

http://technet.oracle.com

Download > Database - Database 12c Enterprise/Standard Editions

 

http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

 

Accept License Agreement

 

 

Oracle Database 12c Release 1 (12.1.0.2.0) for Linux x86-64

 

  Download linuxamd64_12c_database_1of2.zip (1,673,544,724 bytes) (cksum - 839029806) 

  Download linuxamd64_12c_database_2of2.zip (1,014,530,602 bytes) (cksum - 1187131466) 

 

 

Oracle Database 12c Release 1 Grid Infrastructure (12.1.0.2.0) for Linux x86-64

 

  Download linuxamd64_12c_grid_1of2.zip (1,747,043,545 bytes) (cksum - 1194876808) 

  Download linuxamd64_12c_grid_2of2.zip (646,972,897 bytes) (cksum - 2519919927)  

---------------------------------------------------------------------------

 

하드웨어 구성도

 

 

작업은 공유 볼륨 관련 설정을 제외하고는 기본적으로는 각 멤버 노드에서 동일하게 수행해야 하는 것 ...

 

System pre-requisites

login as: root

# as root

$ as grid

 

Memory requirement

[root@node1 ~]# grep MemTotal /proc/meminfo
MemTotal:       16202064 kB


[root@node1 ~]# grep SwapTotal /proc/meminfo
SwapTotal:       8167420 kB


[root@node1 ~]# swapon -s
Filename Type Size Used Priority
/dev/dm-1                               partition 8167420 0 -1


[root@node1 ~]# free -g
             total       used       free     shared    buffers     cached
Mem:            15          1         14          0          0          0
-/+ buffers/cache:          0         14
Swap:            7          0          7

 

Check the temporary space available

[root@node1 ~]# df -h /tmp
Filesystem      Size  Used Avail Use% Mounted on
/tmp             50G  7.8G   39G  17% /tmp

 

Check for the kernel release

[root@node1 ~]# uname -m
x86_64


[root@node1 ~]# more /etc/redhat-release
Red Hat Enterprise Linux Server release 6.8 (Santiago)

 

Check for the newly presented shared LUNs

[root@node1 ~]# ls /sys/class/fc_host/host1 host2
[root@node2 ~]# echo 1 >/sys/class/fc_host/host1/issue_lip[root@node2 ~]# echo 1 >/sys/class/fc_host/host2/issue_lip[root@node2 ~]# echo - - - >/sys/class/scsi_host/host1/scan[root@node2 ~]# echo - - - >/sys/class/scsi_host/host2/scan

 

Set the kernel parameters
[root@node1 etc]# sysctl -pnet.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1kernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 68719476736kernel.shmall = 4294967296fs.aio-max-nr = 1048576fs.file-max = 6815744kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 1048586
[root@node2 ~]# sysctl -pnet.ipv4.ip_forward = 0net.ipv4.conf.default.rp_filter = 1net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1kernel.msgmnb = 65536kernel.msgmax = 65536kernel.shmmax = 68719476736kernel.shmall = 4294967296fs.aio-max-nr = 1048576fs.file-max = 6815744kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 1048586

 

Check the necessary packages
[root@node2 /]# rpm -q make-*
make-3.81-23.el6.x86_64


[root@node2 /]# rpm -qa|grep make
make-3.81-23.el6.x86_64
automake-1.11.1-4.el6.noarch
cmake-2.8.12.2-4.el6.x86_64
makebootfat-1.4-10.el6.x86_64
imake-1.0.2-11.el6.x86_64


[root@node2 /]# yum install libaio-devel*
Loaded plugins: aliases, changelog, kabi, ovl, presto, product-id, refresh-packagekit, search-disabled-repos, security,
              : subscription-manager, tmprepo, verify, versionlock
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Loading support for Red Hat kernel ABI
Setting up Install Process
Update                                                                                                | 4.1 kB     00:00 ... 
Update/primary_db                               100% [=====================================]  0.0 B/s | 496 kB     --:-- ETA
Update/primary_db                               100% [=====================================] 992 kB/s | 1.1 MB     --:-- ETA
Update/primary_db                               100% [=====================================] 1.1 MB/s | 1.9 MB     --:-- ETA
Update/primary_db                               100% [=====================================] 1.1 MB/s | 2.6 MB     --:-- ETA
Update/primary_db                                                                                     | 3.1 MB     00:01 ...
Package libaio-devel-0.3.107-10.el6.x86_64 already installed and latest version
Nothing to do

 

About the IP addressing requirement
 
Cluster Name : Oracle
[root@node1 ~]# vim /etc/hosts# Public10.254.251.19 node110.254.251.40 node2
# Private192.168.1.1 node1priv192.168.1.2 node2priv
# Virtual10.254.251.248 node1vip10.254.251.249 node2vip
# SCAN10.254.251.251 ora12c10.254.251.252 scan210.254.251.253 scan3 

 

Setting Network Time Protocol for Cluster Time Synchronization
[root@node2 /]# vim /etc/ntp.conf
[root@node2 /]# service ntpd startStarting ntpd:    [  OK  ]
[root@node2 /]# vim /etc/sysconfig/ntpdOPTIONS="-u ntp:ntp -p /var/run/ntpd.pid –g -x"
[root@node2 /]# ntpq -p 

 

Check the SELinux setting
[root@node1 etc]# vim /etc/selinux/configSELINUX=disabled
[root@node1 etc]# setenforce 0
[root@node1 etc]# service iptables stop 

 

Create the grid and oracle users and groups
[root@node1 etc]# /usr/sbin/groupadd -g 504 asmadmin[root@node1 etc]# /usr/sbin/groupadd -g 505 asmdba[root@node1 etc]# /usr/sbin/groupadd -g 506 asmoper[root@node1 etc]# /usr/sbin/groupadd -g 507 dba[root@node1 etc]# /usr/sbin/groupadd -g 508 oper[root@node1 etc]# /usr/sbin/groupadd -g 509 oinstall[root@node1 etc]# useradd -g oinstall -G dba,asmdba,oper -s /bin/bash -u \ 501 oracle[root@node1 etc]# useradd -g oinstall -G asmadmin,asmdba,asmoper,dba -u 502 grid
[root@node1 etc]# id oracleuid=501(oracle) gid=509(oinstall) groups=509(oinstall),505(asmdba),507(dba),508(oper)
[root@node1 etc]# id griduid=502(grid) gid=509(oinstall) groups=509(oinstall),504(asmadmin),505(asmdba),506(asmoper),507(dba)
[root@node1 etc]# passwd oracleChanging password for user oracle.New password: BAD PASSWORD: it is based on a dictionary wordRetype new password: passwd: all authentication tokens updated successfully.
[root@node1 etc]# passwd gridChanging password for user grid.New password: BAD PASSWORD: it is based on a dictionary wordRetype new password: passwd: all authentication tokens updated successfully.
[root@node2 /]# /usr/sbin/groupadd -g 504 asmadmin[root@node2 /]# /usr/sbin/groupadd -g 505 asmdba[root@node2 /]# /usr/sbin/groupadd -g 506 asmoper[root@node2 /]# /usr/sbin/groupadd -g 507 dba[root@node2 /]# /usr/sbin/groupadd -g 508 oper[root@node2 /]# /usr/sbin/groupadd -g 509 oinstall[root@node2 /]# useradd -g oinstall -G dba,asmdba,oper -s /bin/bash -u \ 501 oracle[root@node2 /]# useradd -g oinstall -G asmadmin,asmdba,asmoper,dba -u 502 grid
[root@node2 /]# id oracleuid=501(oracle) gid=509(oinstall) groups=509(oinstall),505(asmdba),507(dba),508(oper)
[root@node2 /]# id griduid=502(grid) gid=509(oinstall) groups=509(oinstall),504(asmadmin),505(asmdba),506(asmoper),507(dba)
[root@node2 ~]# passwd oracle[root@node2 ~]# passwd grid 

 

Configure the secure shell service

[root@node2 /]
# su – grid[root@node2 ~]
$ ./sshUserSetup.sh -user grid -hosts "10.254.251.40 10.254.251.19"[root@node2 ~]
$ ./sshUserSetup.sh -user oracle -hosts "10.254.251.40 10.254.251.19"[root@node1 /]
# su – grid[root@node1 ~]
$ ./sshUserSetup.sh -user grid -hosts "10.254.251.19 10.254.251.40"[root@node1 ~]
$ ./sshUserSetup.sh -user oracle -hosts "10.254.251.19 10.254.251.40"
[oracle@node1 .ssh]$ scp id_rsa.pub 10.254.251.40:/home/grid/.ssh/rsa@10.254.251.19
[oracle@node2 sshsetup]$ cat rsa@10.254.251.19>>authorized_keys

 

Note
The important point here is there is no password requested.
 
Set the limits
vim /etc/security/limits.conf
grid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536grid soft stack 10240grid hard stack 32768grid soft memlock 41984000grid hard memlock 41984000oracle soft memlock 41984000oracle hard memlock 41984000oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536oracle soft stack 10240oracle hard stack 32768

 

Installing the cvuqdisk RPM for Linux

[root@node1 ~]# rpm -qa|grep cvuqdiskcvuqdisk-1.0.9-1.x86_64 

 

Storage configuration
 
LUN 1 - /dev/sdb
LUN 2 - /dev/sdc
LUN 3 - /dev/sdd
 
# fdisk /dev/sdb# fdisk /dev/sdc# fdisk /dev/sdd 


# pvcreate /dev/sdb1
# vgcreate vgvoting /dev/sdb1
# lvcreate --extents 100%FREE -n voting vgvoting


# pvcreate /dev/sdc1
# vgcreate vgdata01 /dev/sdc1
# lvcreate --extents 100%FREE -n data01 vgdata01


# pvcreate /dev/sdd1
# vgcreate vgfra01 /dev/sdd1
# lvcreate --extents 100%FREE -n fra01 vgfra01

 

To manage the right level of permission to the shared volume.
# pwd/etc/udev/rules.d
# cp 60-raw.rules 99-oracle.rules
# vim 99-oracle.rules 

# Oracle Configuration RegistryKERNEL=="mapper/vgvoting-voting", OWNER="root", GROUP="oinstall", MODE="640"# Voting DisksKERNEL=="mapper/vgdata01-data01", OWNER="oracle", GROUP="dba", MODE="660"KERNEL=="mapper/vgfra01-fra01", OWNER="oracle", GROUP="dba", MODE="660" 

 

Install the ASMLib support library

# yum install kmod-oracleasm-2.0.8-13.el6_8.x86_64.rpm oracleasmlib-2.0.4-1.el6.x86_64.rpm oracleasm-support-2.1.8-1.el6.x86_64.rpm 
Loaded plugins: aliases, changelog, kabi, ovl, presto, product-id, refresh-packagekit, search-disabled-repos, security, subscription-              : manager, tmprepo, verify, versionlockThis system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.Loading support for Red Hat kernel ABISetting up Install ProcessExamining kmod-oracleasm-2.0.8-13.el6_8.x86_64.rpm: kmod-oracleasm-2.0.8-13.el6_8.x86_64Marking kmod-oracleasm-2.0.8-13.el6_8.x86_64.rpm to be installedExamining oracleasmlib-2.0.4-1.el6.x86_64.rpm: oracleasmlib-2.0.4-1.el6.x86_64Marking oracleasmlib-2.0.4-1.el6.x86_64.rpm to be installedExamining oracleasm-support-2.1.8-1.el6.x86_64.rpm: oracleasm-support-2.1.8-1.el6.x86_64Marking oracleasm-support-2.1.8-1.el6.x86_64.rpm to be installedResolving Dependencies--> Running transaction check---> Package kmod-oracleasm.x86_64 0:2.0.8-13.el6_8 will be installed---> Package oracleasm-support.x86_64 0:2.1.8-1.el6 will be installed---> Package oracleasmlib.x86_64 0:2.0.4-1.el6 will be installed--> Finished Dependency ResolutionkABI data files are missing or corrupted - kABI checking disabled
Dependencies Resolved
======================================================================= Package                        Arch            Version            Repository                                         Size=======================================================================Installing: [1mkmod-oracleasm     (B[m x86_64     2.0.8-13.el6_8     /kmod-oracleasm-2.0.8-13.el6_8.x86_64    117 k [1moracleasm-support  (B[m x86_64     2.1.8-1.el6         /oracleasm-support-2.1.8-1.el6.x86_64       216 k [1moracleasmlib           (B[m x86_64     2.0.4-1.el6         /oracleasmlib-2.0.4-1.el6.x86_64                27 k
Transaction Summary=======================================================================Install       3 Package(s)
Total size: 360 kInstalled size: 360 kIs this ok [y/N]: yDownloading Packages:Running rpm_check_debugRunning Transaction TestTransaction Test SucceededRunning TransactionWarning: RPMDB altered outside of yum.Installing : kmod-oracleasm-2.0.8-13.el6_8.x86_64 [############################################################################# ] 1/3Installing : kmod-oracleasm-2.0.8-13.el6_8.x86_64                                                                        1/3 Installing : oracleasmlib-2.0.4-1.el6.x86_64 [                                                                               ] 2/3Installing : oracleasmlib-2.0.4-1.el6.x86_64 [############################################################################# ] 2/3Installing : oracleasmlib-2.0.4-1.el6.x86_64                                                                                  2/3 Installing : oracleasm-support-2.1.8-1.el6.x86_64 [############################################################################# ] 3/3Installing : oracleasm-support-2.1.8-1.el6.x86_64                                                                          3/3 
Update/productid                                                                                             | 1.6 kB     00:00 ... Verifying  : kmod-oracleasm-2.0.8-13.el6_8.x86_64                                                                       1/3 Verifying  : oracleasm-support-2.1.8-1.el6.x86_64                                                                         2/3 Verifying  : oracleasmlib-2.0.4-1.el6.x86_64                                                                                 3/3 
Installed:  kmod-oracleasm.x86_64 0:2.0.8-13.el6_8         oracleasm-support.x86_64 0:2.1.8-1.el6         oracleasmlib.x86_64 0:2.0.4-1.el6        
Complete!
# /usr/sbin/oracleasm init# /etc/init.d/oracleasm configure
Default user to own the driver interface []: gridDefault group to own the driver interface []: asmadminStart Oracle ASM library driver on boot (y/n) [n]: yScan for Oracle ASM disks on boot (y/n) [y]: y

[root@node2 ~]# oracleasm createdisk VOTING /dev/mapper/vgvoting-voting[root@node2 ~]# oracleasm createdisk DATA01 /dev/mapper/vgdata01-data01[root@node2 ~]# oracleasm createdisk FRA01 /dev/mapper/vgfra01-fra01
[root@node2 ~]# oracleasm listdisksDATA01FRA01VOTING
[root@node1 ~]# oracleasm scandisks[root@node1 ~]# oracleasm listdisks
# vim /etc/sysconfig/oracleasm
# ORACLEASM_SCANORDER: Matching patterns to order disk scanningORACLEASM_SCANORDER="/dev/mapper"# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scanORACLEASM_SCANEXCLUDE="sd"
# chkconfig --list oracleasm 

 

Check the available disk space

# mkdir -p /u01/app/oracle/oraInventory# chown -R grid:oinstall /u01/app/oracle/oraInventory# chmod -R 775 /u01/app/oracle/oraInventory
# mkdir -p /u01/app/grid/12c# chown -R grid:oinstall /u01/app/grid# chmod -R 775 /u01/app/grid
# mkdir -p /u01/app/base# chown -R grid:oinstall /u01/app/base# chmod -R 775 /u01/app/base
# mkdir /u01/app/oracle/12c# chown -R oracle:oinstall /u01/app/oracle# chmod -R 775 /u01/app/oracle 

 

Setting the disk I/O scheduler on Linux

# echo deadline > /sys/block/sdb/queue/scheduler# echo deadline > /sys/block/sdc/queue/scheduler# echo deadline > /sys/block/sdd/queue/scheduler
# cat /sys/block/sdb/queue/scheduler
# vim /etc/grub.conf 

"elevator=deadline" 추가

 

Determining root script execution plan

# visudo
## Allow root to run any commands anywhereroot ALL=(ALL) ALLgrid ALL=(ALL) NOPASSWD: ALLoracle ALL=(ALL) NOPASSWD: ALL 

 

 

RHEL 6.8/Oracle RAC 설치 무작정 따라하기 ---------------

01. System pre-requisites

http://infoages.tistory.com/1735

 

02. Oracle Clusterware installation

http://infoages.tistory.com/1736

 

03. Oracle RAC 12c database installation

http://infoages.tistory.com/1737

-----------------------------------------------------------------

 

참고자료

4AA4-8504ENW Technical white paper

Oracle RAC 12c on HP blade servers running Red Hat Enterprise Linux 6 Update 4

Installation cookbook

반응형