RAC 11g useful commands
Check the status of the cluster
[oracle@rac1 ~]$ crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online
[oracle@rac1 ~]$ crsctl check cluster CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online
[oracle@rac1 ~]$ crsctl check cluster -all ************************************************************** rac1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online **************************************************************
Check the status of the services on the cluster
[oracle@rac1 ~]$ crsctl status resource -t
NAME TARGET STATE SERVER STATE_DETAILS Local Resources ---------------------------------------------------------------------------- ora.DATA.dg ONLINE ONLINE rac1 ora.LISTENER.lsnr ONLINE ONLINE rac1 ora.asm ONLINE ONLINE rac1 ora.gsd OFFLINE OFFLINE rac1 ora.net1.network ONLINE ONLINE rac1 ora.ons ONLINE ONLINE rac1 ora.registry.acfs ONLINE ONLINE rac1 --------------------------------------------------------------------------------- Cluster Resources --------------------------------------------------------------------------------- ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE rac1 ora.cvu 1 ONLINE ONLINE rac1 ora.oc4j1 ONLINE ONLINE rac1 ora.rac1.vip1 ONLINE ONLINE rac1 ora.rac2.vip1 OFFLINE OFFLINE ora.scan1.vip1 ONLINE ONLINE rac1 ora.testrac.db1 OFFLINE OFFLINE Instance Shutdown
Stop crs on all nodes (only one node up in this example)
[root@rac1 bin]# ./crsctl stop cluster -all CRS-2673: Attempting to stop ‘ora.crsd’ on ‘rac1’ CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘rac1’ CRS-2673: Attempting to stop ‘ora.registry.acfs’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.DATA.dg’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.cvu’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.oc4j’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.LISTENER_SCAN1.lsnr’ on ‘rac1’ CRS-2677: Stop of ‘ora.LISTENER.lsnr’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.rac1.vip’ on ‘rac1’ CRS-2677: Stop of ‘ora.LISTENER_SCAN1.lsnr’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.scan1.vip’ on ‘rac1’ CRS-2677: Stop of ‘ora.scan1.vip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.rac1.vip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.cvu’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.registry.acfs’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.oc4j’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.DATA.dg’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.asm’ on ‘rac1’ CRS-2677: Stop of ‘ora.asm’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.ons’ on ‘rac1’ CRS-2677: Stop of ‘ora.ons’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.net1.network’ on ‘rac1’ CRS-2677: Stop of ‘ora.net1.network’ on ‘rac1’ succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘rac1’ has completed CRS-2677: Stop of ‘ora.crsd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.evmd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.asm’ on ‘rac1’ CRS-2677: Stop of ‘ora.asm’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘rac1’ CRS-2677: Stop of ‘ora.evmd’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.ctssd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.cssd’ on ‘rac1’ CRS-2677: Stop of ‘ora.cssd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘rac1’ CRS-2677: Stop of ‘ora.diskmon’ on ‘rac1’ succeeded
Stop crs on specifc node
[root@rac1 bin]# ./crsctl stop has CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘rac1’ CRS-2673: Attempting to stop ‘ora.crsd’ on ‘rac1’ CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘rac1’ CRS-2673: Attempting to stop ‘ora.cvu’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.registry.acfs’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.DATA.dg’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.LISTENER_SCAN1.lsnr’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.oc4j’ on ‘rac1’ CRS-2677: Stop of ‘ora.LISTENER.lsnr’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.rac1.vip’ on ‘rac1’ CRS-2677: Stop of ‘ora.rac1.vip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.LISTENER_SCAN1.lsnr’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.scan1.vip’ on ‘rac1’ CRS-2677: Stop of ‘ora.scan1.vip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.cvu’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.oc4j’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.registry.acfs’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.DATA.dg’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.asm’ on ‘rac1’ CRS-2677: Stop of ‘ora.asm’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.ons’ on ‘rac1’ CRS-2677: Stop of ‘ora.ons’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.net1.network’ on ‘rac1’ CRS-2677: Stop of ‘ora.net1.network’ on ‘rac1’ succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘rac1’ has completed CRS-2677: Stop of ‘ora.crsd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.crf’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.evmd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.asm’ on ‘rac1’ CRS-2677: Stop of ‘ora.asm’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘rac1’ CRS-2677: Stop of ‘ora.evmd’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.crf’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.mdnsd’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.ctssd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.cssd’ on ‘rac1’ CRS-2677: Stop of ‘ora.cssd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘rac1’ CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘rac1’ CRS-2677: Stop of ‘ora.gipcd’ on ‘rac1’ succeeded CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘rac1’ CRS-2677: Stop of ‘ora.diskmon’ on ‘rac1’ succeeded CRS-2677: Stop of ‘ora.gpnpd’ on ‘rac1’ succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘rac1’ has completed CRS-4133: Oracle High Availability Services has been stopped.
Disable CRS from starting on reboot
[root@rac1 bin]# ./crsctl disable has CRS-4621: Oracle High Availability Services autostart is disabled.
Enable CRS to start on reboot
[root@rac1 bin]# ./crsctl enable has CRS-4622: Oracle High Availability Services autostart is enabled.
Start crs on specifc node
[root@rac1 bin]# ./crsctl start has CRS-4123: Oracle High Availability Services has been started.
Check database status
[oracle@rac1 ~]$ srvctl status database -d RACDB Instance RACDB1 is running on node rac1 Instance RACDB2 is running on node rac2
Stop an instance
[oracle@rac1 ~]$ srvctl stop instance -i RACDB1 -d RACDB [oracle@rac1 ~]$ srvctl status database -d RACDB Instance RACDB1 is not running on node rac1 Instance RACDB2 is running on node rac2
Start an instance
[oracle@rac1 ~]$ srvctl start instance -i RACDB1 -d RACDB [oracle@rac1 ~]$ srvctl status database -d RACDB Instance RACDB1 is running on node rac1 Instance RACDB2 is running on node rac2
Stop the database
[oracle@rac1 ~]$ srvctl stop database -d RACDB [oracle@rac1 ~]$ srvctl status database -d RACDB Instance RACDB1 is not running on node rac1 Instance RACDB2 is not running on node rac2
Start the database
[oracle@rac1 ~]$ srvctl start database -d RACDB [oracle@rac1 ~]$ srvctl status database -d RACDB Instance RACDB1 is running on node rac1 Instance RACDB2 is running on node rac2
Stop the listener on a Node
[oracle@rac1 ~]$ srvctl status listener -n rac1 Listener LISTENER is enabled on node(s): rac1 Listener LISTENER is not running on node(s): rac1
Start the listener on a Node
[oracle@rac1 ~]$ srvctl status listener -n rac1 Listener LISTENER is enabled on node(s): rac1 Listener LISTENER is running on node(s): rac1
Check the voting disk
[oracle@rac1 ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -------------------------------------------------------------------------------------- 1. ONLINE 91b18024f9674f83bf5117d3c0996a05 (ORCL:VOL1) [DATA] Located 1 voting disk(s).
Query the network addresses
[oracle@rac1 ~]$ oifcfg getif eth0 192.168.245.0 global public eth1 192.168.126.0 global cluster_interconnect
Show OCR backup – the voting file (disk) is also backed up with the OCR backup
[root@rac1 bin]# ./ocrconfig -showbackup rac1 2017/01/17 07:47:17 /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr rac1 2017/01/17 03:47:15 /u01/app/11.2.0/grid/cdata/rac-cluster/backup01.ocr rac1 2017/01/16 23:47:13 /u01/app/11.2.0/grid/cdata/rac-cluster/backup02.ocr rac1 2017/01/16 19:47:11 /u01/app/11.2.0/grid/cdata/rac-cluster/day.ocr rac1 2017/01/16 19:47:11 /u01/app/11.2.0/grid/cdata/rac-cluster/week.ocr rac1 2017/01/23 11:30:19 /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20120323_113019.ocr
Take a manual backup of the OCR
[root@rac1 bin]# ./ocrconfig -manualbackup rac1 2017/01/17 10:08:28 /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20120717_100828.ocr rac1 2017/01/17 10:07:39 /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20120717_100739.ocr rac1 2017/01/23 11:30:19 /u01/app/11.2.0/grid/cdata/rac-cluster/backup_20120323_113019.ocr
STOP SEQUENCE
COMMAND DESCRIPTION ------------------------------------------------------------------------------------------------------------------------------------------ srvctl stop database -d ORCL STOP DATABASE srvctl stop instance -d ORCL -i ORCL1 STOP first INSTANCE (skip it if running 'stop database' as that will stop both instances) srvctl stop instance -d ORCL -i ORCL2 STOP second INSTANCE (skip it if running 'stop database' as that will stop both instances) srvctl stop asm -n NODE1 STOP ASM INSTANCES on NODE 1 (In 11G , we have OCR on ASM so we cannot stop ASM, but if you have OCR in NON-ASM you should stop it) srvctl stop asm -n NODE2 STOP ASM INSTANCES on NODE 2 (In 11G , we have OCR on ASM so we cannot stop ASM, but if you have OCR in NON-ASM you should stop it) srvctl stop nodeapps -n NODE1 STOP NODEAPPS on NODE 1 srvctl stop nodeapps -n NODE2 STOP NODEAPPS on NODE 2 /etc/init.d/init.crs stop STOP CRS PROCESSES (AS ROOT USER)
START SEQUENCE
COMMAND DESCRIPTION ------------------------------------------------------------------------------------------- /etc/init.d/init.crs start START CRS PROCESS (AS ROOT USER) srvctl start asm -n NODE1 START ASM INSTANCE on node 1 srvctl start asm -n NODE2 START ASM INSTANCE on node 2 srvctl start database -d ORCL START DATABASE srvctl start instance -d ORCL -i ORCL1 START first INSTANCE (skip it if running 'start database' as that will start both instances) srvctl start instance -d ORCL -i ORCL2 START second INSTANCE (skip it if running 'start database', as taht will start both instances) srvctl start nodeapps -n NODE1 START NODEAPPS on NODE1 srvctl start nodeapps -n NODE2 START NODEAPPS ON NODE2
OTHER USEFUL COMMANDS
COMMAND DESCRIPTION ---------------------------------------------------------------------------------------------------------------------------- crsctl status resource -t Clusterware Resource Status Check srvctl status database -d ORCL STATUS OF DATABASE srvctl stop listener -l LISTENER_NAME STOP A LISTENER srvctl start listener -l LISTENER_NAME START A LISTENER crsctl stop has stop all the clusterware services/ resources on specific node (including DB and listener) (run as root) crsctl start has start all the clusterware services/ resources on specific node (including DB and listener) (run as root) crsctl stop cluster -all to stop csr services on all nodes of clusterware (run as root) crsctl start cluster -all to start crs services on all nodes of clusterware (run as root) crsctl check has to check if ohasd is running/ stopped (run as root) crsctl enable has enable Oracle High Availability Services autostart (run as root) crsctl disable has disable Oracle High Availability Services autostart (run as root) crsctl config has check if Oracle High Availability Services autostart is enabled/ disabled (run as root) srvctl status nodeapps to check the status of services on all nodes crsctl stop crs stop all the clusterware services/ resources ON THAT NODE! (run as root) crsctl start crs start all the clusterware services/ resources ON THAT NODE! (run as root) cluvfy comp scan -verbose Verifying scan status scan_listener srvctl config scan_listener Verifying scan port srvctl relocate scan -i 1 -n NODE1 Relocate scan listener 1 to the mentioned node
[oracle@rac1 grid]$ crs_stat -t
Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE rac1 ......... .........
[oracle@rac1 grid]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE rac1 ora....ER.lsnr ora....er.type ONLINE ONLINE rac1 ora....N1.lsnr ora....er.type ONLINE ONLINE rac2 ora....N2.lsnr ora....er.type ONLINE ONLINE rac1 ora....N3.lsnr ora....er.type ONLINE ONLINE rac1 ora.MGMTLSNR ora....nr.type ONLINE ONLINE rac1 ora.asm ora.asm.type ONLINE ONLINE rac1 ora.cvu ora.cvu.type ONLINE ONLINE rac1 ora.mgmtdb ora....db.type ONLINE ONLINE rac1 ora....network ora....rk.type ONLINE ONLINE rac1 ora.oc4j ora.oc4j.type ONLINE ONLINE rac1 ora.ons ora.ons.type ONLINE ONLINE rac1 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 .... ....
[oracle@rac1 grid]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [oracle@rac1 grid]$
srvctl -h
#srvctl start database -d <db_unique_name> [-o <start_options>] [-n <node>]
srvctl start database -d sha1p
#srvctl stop database -d <db_unique_name> [-o <stop_options>] [-f]
srvctl stop database -d sha1p -o immediate
#srvctl start instance -d <db_unique_name> {-n <node_name> [-i <inst_name>] | -i <inst_name_list>} [-o <start_options>]
srvctl start instance -d sha1p -i sha1p1
srvctl stop instance -d <db_unique_name> {-n <node_name> | -i <inst_name_list>} [-o <stop_options>] [-f]
srvctl status instance -d <db_unique_name> {-n <node_name> | -i <inst_name_list>} [-f] [-v]
srvctl start service -d <db_unique_name> [-s "<service_name_list>" [-n <node_name> | -i <inst_name>] ] [-o <start_options>]
srvctl stop service -d <db_unique_name> [-s "<service_name_list>" [-n <node_name> | -i <inst_name>] ] [-f]
srvctl status database -d <db_unique_name> [-f] [-v]
srvctl enable database -d <db_unique_name> [-n <node_name>]
srvctl convert database -d <db_unique_name> -c RAC [-n <node>]
srvctl convert database -d <db_unique_name> -c RACONENODE [-i <inst_name>] [-w <timeout>]
srvctl status nodeapps
srvctl start diskgroup -g <dg_name> [-n "<node_list>"]
srvctl stop diskgroup -g <dg_name> [-n "<node_list>"] [-f]
srvctl config listener [-l <lsnr_name>] [-a]
srvctl start listener [-l <lsnr_name>] [-n <node_name>]
srvctl stop listener [-l <lsnr_name>] [-n <node_name>] [-f]
srvctl status listener [-l <lsnr_name>] [-n <node_name>] [-v]
srvctl enable listener [-l <lsnr_name>] [-n <node_name>]
srvctl disable listener [-l <lsnr_name>] [-n <node_name>]
srvctl config scan [-i <ordinal_number>]
srvctl start scan [-i <ordinal_number>] [-n <node_name>]
srvctl stop scan [-i <ordinal_number>] [-f]
srvctl relocate scan -i <ordinal_number> [-n <node_name>]
srvctl status scan [-i <ordinal_number>] [-v]
srvctl enable scan [-i <ordinal_number>]
srvctl start scan_listener [-n <node_name>] [-i <ordinal_number>]
srvctl stop scan_listener [-i <ordinal_number>] [-f]
srvctl relocate scan_listener -i <ordinal_number> [-n <node_name>]
srvctl status scan_listener [-i <ordinal_number>] [-v]
srvctl status service -d
Voting Disk
| adding | crsctl add css votedisk <file> |
| deleting | crsctl delete css votedisk <file> |
| querying | crsctl query css votedisk |
SELECT THREAD#,MAX(SEQUENCE#) FROM V$ARCHIVED_LOG GROUP BY THREAD# order by THREAD#;
select thread#,max(sequence#) from v$archived_log where applied=’YES’ group by thread#;
THREAD# MAX(SEQUENCE#) ---------- -------------- 1 59056 2 44181 SELECT THREAD#,MAX(SEQUENCE#) FROM V$ARCHIVED_LOG GROUP BY THREAD# order by THREAD#; THREAD# MAX(SEQUENCE#) ---------- -------------- 1 59056 2 44181
set pages 300 lines 300
col MACHINE format a40
col USERNAME format a20
col OSUSER format a20
col SERVICE_NAME format a30
select count(*),username,status,machine,osuser,INST_ID,SERVER, service_name from gv$session where program not like 'ORACLE%' group by username,status,machine, osuser,INST_ID,SERVER,service_ name order by username; select count(*),username,status, machine,osuser,INST_ID,SERVER, service_name from gv$session where program not like 'ORACLE%' and machine like '%#####%' group by username,status,machine, osuser,INST_ID,service_name order by 4,2;
Please kill sessions if required
select 'alter system kill session ''' ||sid|| ',' || serial#|| ''' immediate;' from v$session where status='INACTIVE'
select ' alter system kill session ''' || sid|| ',' || serial# || ',@' || inst_id || ''' immediate; ' from gv$session where type='USER' and username not in ('SYS','SYSTEM');
CRCONFIG: — OCR (Oracle Cluster Registry) CONFIGuration tool
#ocrconfig -help or ocrconfig -h #ocrconfig -showbackup [auto|manual] -- default location is $ORA_CRS_HOME/cdata/cluster_name #ocrconfig -showbackup #ocrconfig -backuploc dir_name -- change OCR autobackup directory location #ocrconfig -backuploc /u02/backups #ocrconfig -manualbackup -- Oracle RAC 11g command, to perform OCR backup manually #ocrconfig -restore backup_file.ocr -- recovering from autobackup file #ocrconfig -restore /u02/backups/backup00.ocr #ocrconfig -export file_name.dmp [-s online] -- exports OCR content to a file #ocrconfig -export /tmp/ocr_exp #ocrconfig -import file_name.dmp -- recover OCR logically, must be done on all nodes #ocrconfig -import /tmp/ocr_exp #ocrconfig -replace ocr [file_name] -- adding/replacing an OCR file #ocrconfig -replace ocrmirror [file_name] #ocrconfig -repair ocr file_name #ocrconfig -repair ocrmirror file_name #ocrconfig -repair -replace current_OCR_location -replacement target_OCR_location #ocrconfig -upgrade [user [group]] -- upgrades OCR #ocrconfig -downgrade [-version version_string] -- downgrades OCR #ocrconfig -overwrite #ocrconfig –local –import file_name #ocrconfig –local –manualbackup #ocrconfig -local -backuploc new_olr_backup_path #ocrconfig -local -restore file_name #ocrconfig -add +new_disk_group #ocrconfig -delete +unused_disk_group #ocrconfig -add file_location #ocrconfig -add /dev/sdd1 #ocrconfig -delete old_storage_location #ocrconfig -delete /dev/raw/raw2 Log file will be $ORACLE_HOME/log/node_name/client/ocrconfig_pid.log Debugging can be controlled through $ORA_CRS_HOME/srvm/admin/ocrlog.ini
How to take backup of OCR file?
#ocrconfig -manualbackup
#ocrconfig -export file_name.dmp
How to recover OCR file?
#ocrconfig -restore backup_file.ocr
#ocrconfig -import file_name.dmp