ORACLE 11.2.0.3 RAC FOR
REHL6.3(64BIT)安装文档
作者：魔心之泪
1、安装环境 ..................................................................................................................................... 2
2、安装前系统配置 .......................................................................................................................... 2
2.1 存储规划........................................................................................................................................ 2
2.2 ip 地址规划 .................................................................................................................................... 2
2.3 创建用户、组(所有节点) ............................................................................................................. 3
2.4 配置各节点环境变量 .................................................................................................................... 3
2.5 关闭所有节点防火墙 .................................................................................................................... 4
2.6 配置 NTP 服务(所有节点) ............................................................................................................. 4
2.7 系统参数设置 ................................................................................................................................ 5
2.8 存储分区........................................................................................................................................ 5
2.9 UDEV 绑定裸设备 ......................................................................................................................... 6
3、安装 GRID .................................................................................................................................... 7
4、创建 ASM 磁盘组 ...................................................................................................................... 22
5、安装数据库 ................................................................................................................................ 24
6、建库 ........................................................................................................................................... 29
1、安装环境
[root@db1 ~]# uname -a
Linux db1 2.6.32-279.el6.x86_64 #1 SMP Wed Jun 13 18:24:36 EDT 2012
x86_64 x86_64 x86_64 GNU/Linux
[root@db1 ~]# lsb_release
LSBVersion: :core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0
-noarch:printing-4.0-amd64:printing-4.0-noarch
2、安装前系统配置
2.1 存储规划
名称
OCR_VOTING
Flash_Recovery
DATA
映射裸设备
/dev/raw/raw1
/dev/raw/raw2
/dev/raw/raw3
/dev/raw/raw4
/dev/raw/raw5
/dev/raw/raw6
/dev/raw/raw9
/dev/raw/raw10
/dev/raw/raw11
/dev/raw/raw12
/dev/raw/raw13
/dev/raw/raw14
/dev/raw/raw15
/dev/raw/raw16
/dev/sdb1
/dev/sdb2
/dev/sdb3
/dev/sdb5
/dev/sdb6
/dev/sdb7
/dev/sdc1
/dev/sdd1
/dev/sde1
/dev/sdf1
/dev/sdg1
/dev/sdh1
/dev/sdi1
/dev/sdj1
2.2 IP 地址规划
主机名称
db1
db2
eth0 IP(PUBLIC)
10.238.78.34
10.238.78.35
eth0 IP(VIP)
10.238.78.40
10.238.78.41
eth1(PRIV)
192.168.0.1
192.168.0.2
SCAN IP
SID
ORACLE 版本
10.238.78.42
partner1
partner2
11.2.0.3
大小
1G
1G
1G
5G
5G
5G
536.9G
536.9G
536.9G
536.9G
536.9G
536.9G
536.9G
536.9G
修改所有节点 hosts 文件
[root@db1 ~]# cat /etc/hosts
127.0.0.1
localhost localhost.localdomain localhost4 localhost4.localdomain4
10.238.73.135
websv1
10.238.78.34
db1
10.238.78.35
db2
10.238.78.40
db1-vip
10.238.78.41
db2-vip
192.168.0.1
db1-priv
192.168.0.2
db2-priv
10.238.78.42
rac-scan
2.3 创建用户、组(所有节点)
groupadd -g 1000 oinstall
groupadd -g 1200 asmadmin
groupadd -g 1201 asmdba
groupadd -g 1202 asmoper
groupadd -g 1300 dba
groupadd -g 1301 oper
useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper grid
useradd -m -u 1101 -g oinstall -G dba,oper,asmdba oracle
mkdir -p /oracle/grid
chown -R grid:oinstall /oracle/grid
chown -R grid:oinstall /oracle/grid/11.2.0.3
mkdir -p /oracle/db
chown -R oracle:oinstall /oracle/db
chmod -R 775 /oracle
passwd grid
passwd oracle
2.4 配置各节点环境变量
修改 grid 用户.bash_profile
export ORACLE_SID=+ASM1#节点 2 为+ASM2
export ORACLE_BASE=/oracle/oracle11g
export ORACLE_HOME=/oracle/grid/11.2.0.3
export PATH=$ORACLE_HOME/bin:$PATH:/usr/local/bin/:.
export TEMP=/tmp
export TMP=/tmp
export TMPDIR=/tmp
umask 022
修改 oracle 用户.bash_profile
export ORACLE_SID=partner1#节点 2 为 partner2
export ORACLE_UNQNAME=partner
export ORACLE_BASE=/oracle/db
export ORACLE_HOME=/oracle/db/11.2.0.3
export PATH=$ORACLE_HOME/bin:$PATH:$ORA_CRS_HOME/bin
export TEMP=/tmp
export TMPDIR=/tmp
umask 022
2.5 关闭所有节点防火墙
Service iptables status
Service iptables stop
chkconfig iptables off
chkconfig iptables –list
设置/etc/selinux/config 文件，将 SELINUX 设置为 disabled
[root@db1 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#
enforcing - SELinux security policy is enforced.
#
permissive - SELinux prints warnings instead of enforcing.
#
disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#
targeted - Targeted processes are protected,
#
mls - Multi Level Security protection.
SELINUXTYPE=targeted
2.6 配置 NTP 服务(所有节点)
LINUX 客户端使用
ntpdate 172.30.218.114
来向 NTP 服务器同步自己的时间
其它 LINUX 如果仅作为只客户端的话,则不能启动 ntpd 服务!否则无法运行
ntpdata 服务器地址 来同步时间
之后可以使用 cron 或修改 crontab 文件定期向 NTP 服务器更新时间,并用
# hwclock --systohc
将系统时间设置为硬件时间
[root@db2 ~]# crontab -l
*/15 * * * * ntpdate 172.30.218.114
2.7 系统参数设置
/etc/security/limits.conf 文件添加如下内容：
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
/etc/pam.d/login 添加：
session
required
pam_limits.so
2.8 存储分区
所有规划的设备进行分区，例如在节点 1 执行：
[root@db1 ~]# fdisk /dev/sde
WARNING: DOS-compatible mode is deprecated. It's strongly recommended
to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): p
Disk /dev/sde: 536.9 GB, 536870912000 bytes
255 heads, 63 sectors/track, 65270 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000d60dc
Device Boot
Start
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-65270, default 1):
Using default value 1
End
Blocks
Id System
Last cylinder, +cylinders or +size{K,M,G} (1-65270, default 65270):
Using default value 65270
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
所有节点同步分区
Partprobe
2.9 UDEV 绑定裸设备
vi /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw
/dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1",
RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
#for ocr_voting
ACTION=="add",KERNEL=="sdb1",RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add",KERNEL=="sdb2",RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add",KERNEL=="sdb3",RUN+="/bin/raw /dev/raw/raw3 %N"
#for Flash_Recovery
ACTION=="add",KERNEL=="sdb5",RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add",KERNEL=="sdb6",RUN+="/bin/raw /dev/raw/raw5 %N"
ACTION=="add",KERNEL=="sdb7",RUN+="/bin/raw /dev/raw/raw6 %N"
#for data
ACTION=="add",KERNEL=="sdc1",RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add",KERNEL=="sdd1",RUN+="/bin/raw /dev/raw/raw8 %N"
ACTION=="add",KERNEL=="sde1",RUN+="/bin/raw /dev/raw/raw9 %N"
ACTION=="add",KERNEL=="sdf1",RUN+="/bin/raw /dev/raw/raw10 %N"
ACTION=="add",KERNEL=="sdg1",RUN+="/bin/raw /dev/raw/raw11 %N"
ACTION=="add",KERNEL=="sdh1",RUN+="/bin/raw /dev/raw/raw12 %N"
ACTION=="add",KERNEL=="sdi1",RUN+="/bin/raw /dev/raw/raw13 %N"
ACTION=="add",KERNEL=="sdj1",RUN+="/bin/raw /dev/raw/raw14 %N"
KERNEL=="raw[1-9]*", OWNER="grid", GROUP="asmadmin", MODE="660"
重启 UDEV：
# /sbin/start_udev
以上内容需要在所有节点执行
验证：
[root@db1 rules.d]# ll /dev/raw
total 0
crw-rw---- 1 grid asmadmin 162, 1 Jul 11 15:19 raw1
crw-rw---- 1 grid asmadmin 162, 10 Jul 11 15:19 raw10
crw-rw---- 1 grid asmadmin 162, 11 Jul 11 15:19 raw11
crw-rw---- 1 grid asmadmin 162, 12 Jul 11 15:19 raw12
crw-rw---- 1 grid asmadmin 162, 13 Jul 11 15:19 raw13
crw-rw---- 1 grid asmadmin 162, 14 Jul 11 15:19 raw14
crw-rw---- 1 grid asmadmin 162, 2 Jul 11 15:19 raw2
crw-rw---- 1 grid asmadmin 162, 3 Jul 11 15:19 raw3
crw-rw---- 1 grid asmadmin 162, 4 Jul 11 15:19 raw4
crw-rw---- 1 grid asmadmin 162, 5 Jul 11 15:19 raw5
crw-rw---- 1 grid asmadmin 162, 6 Jul 11 15:19 raw6
crw-rw---- 1 grid asmadmin 162, 7 Jul 11 15:19 raw7
crw-rw---- 1 grid asmadmin 162, 8 Jul 11 15:19 raw8
crw-rw---- 1 grid asmadmin 162, 9 Jul 11 15:19 raw9
crw-rw---- 1 root disk
162, 0 Jul 11 15:19 rawctl
3、安装 GRID
节点 1 用 GRID 用执行：
[grid@db1 grid]$ export DISPLAY=10.238.73.144:0.0
[grid@db1 grid]$ export LANG=en_US.utf-8
[grid@db1 grid]$ ./runI*
这里没有配置 DNS,使用/etc/hosts 文件来解析 SCAN
这里可以使用自动配置节点直接 ssh 互信，也可手动配置
用 grid、oracle 用户在所有节点执行
ssh-keygen -t dsa
ssh-keygen -t rsa
cd .ssh
touch authorized_keys
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys
#grid 用户执行
db1 执行
ssh db2 cat /home/grid/.ssh/id_rsa.pub >> authorized_keys
ssh db2 cat /home/grid/.ssh/id_dsa.pub >> authorized_keys
db2 执行
ssh db1 cat /home/grid/.ssh/id_rsa.pub >> authorized_keys
ssh db1 cat /home/grid/.ssh/id_dsa.pub >> authorized_keys
#oracle 执行
db1 执行
ssh db2 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
ssh db2 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys
db2 执行
ssh db1 cat /home/oracle/.ssh/id_rsa.pub >> authorized_keys
ssh db1 cat /home/oracle/.ssh/id_dsa.pub >> authorized_keys
grid、oracle 用户验证
ssh db1 date
ssh db2 date
ssh db1-priv date
ssh db2-priv date
如果仍然需要输入密码，需检查相关目录权限
chmod 644 authorized_keys
chmod 755 /oracle/grid
chmod 700 /oracle/grid/.ssh
小提示：如果有的设备没有显示，则该设备可能被使用过，使用 DD 命令清除：
dd if=/dev/zero of=/dev/raw/raw1 bs=10M count=10
grid 安装包中找到 cvuqdisk-1.0.9-1.rpm 后运行(所有节点)：
[root@db2 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm
Preparing...
###########################################
[100%]
Using default group oinstall to install package
1:cvuqdisk
###########################################
[100%]
安装 oracleasm,可在官网下载
[root@db2 rpm]# rpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm
warning: oracleasm-support-2.1.8-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID
ec551f03: NOKEY
Preparing...
###########################################
[100%]
package oracleasm-support-2.1.8-1.el6.x86_64 is already installed
使用/etc/hosts 解析 scan 可忽略此错误
先在节点 1 和节点 2 执行：
/oracle/oraInventory/orainstRoot.sh
其次：
/oracle/grid/11.2.0.3/root.sh
执行结果：
[root@db1 oracle]# /oracle/grid/11.2.0.3/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /oracle/grid/11.2.0.3
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /oracle/grid/11.2.0.3/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'db1'
CRS-2676: Start of 'ora.mdnsd' on 'db1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'db1'
CRS-2676: Start of 'ora.gpnpd' on 'db1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'db1'
CRS-2672: Attempting to start 'ora.gipcd' on 'db1'
CRS-2676: Start of 'ora.cssdmonitor' on 'db1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'db1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'db1'
CRS-2672: Attempting to start 'ora.diskmon' on 'db1'
CRS-2676: Start of 'ora.diskmon' on 'db1' succeeded
CRS-2676: Start of 'ora.cssd' on 'db1' succeeded
ASM created and started successfully.
Disk Group OCRVOTING created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 14bd8fb4dff14f05bf40fca499a9a11f.
Successful addition of voting disk 25c05e68002f4f6bbf722c09f8c7b068.
Successful addition of voting disk 0231b788ad924fb3bf7eab84fd1de9ac.
Successfully replaced voting disk group with +OCRVOTING.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##
STATE
-- -----
File Universal Id
-----------------
File Name Disk group
--------- ---------
1. ONLINE
14bd8fb4dff14f05bf40fca499a9a11f (/dev/raw/raw1) [OCRVOTING]
2. ONLINE
25c05e68002f4f6bbf722c09f8c7b068 (/dev/raw/raw2) [OCRVOTING]
3. ONLINE
0231b788ad924fb3bf7eab84fd1de9ac (/dev/raw/raw3) [OCRVOTING]
Located 3 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'db1'
CRS-2676: Start of 'ora.asm' on 'db1' succeeded
CRS-2672: Attempting to start 'ora.OCRVOTING.dg' on 'db1'
CRS-2676: Start of 'ora.OCRVOTING.dg' on 'db1' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@db2 rpm]# /oracle/oraInventory/orainstRoot.sh
Changing permissions of /oracle/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /oracle/oraInventory to oinstall.
The execution of the script is complete.
You have new mail in /var/spool/mail/root
[root@db2 rpm]# /oracle/grid/11.2.0.3/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /oracle/grid/11.2.0.3
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /oracle/grid/11.2.0.3/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS
daemon on node db1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
You have new mail in /var/spool/mail/root
验证：
4、创建 ASM 磁盘组
用 gird 用户登录
$asmca
5、安装数据库
用/etc/hosts 解析 SCAN 可忽略此错误
安装完成后所有节点执行
[root@db2 db2]# /oracle/db/11.2.0.3/root.sh
Performing root user operation for Oracle 11g
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /oracle/db/11.2.0.3
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
6、建库
Oracle 用户执行 dbca
7、验证服务
8、相关问题
8.1
ORA-12545
【现象】
【解决】
[oracle@db2 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Thu Jul 18 09:14:46 2013
Copyright (c) 1982, 2011, Oracle.
All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> alter system set
local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=10.238.78
.40)(PORT=1521))))' sid='partner1';
System altered.
SQL> alter system set
local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=10.238.78
.41)(PORT=1521))))' sid='partner2';
System altered.