Oracle RAC Architecture 10g and 11g

Architecture Diagram:

This is a basics architecture of Oracle RAC, We will discuss about new features/additions in 11g in each sections.

RAC components

• Shared Disk System
• Oracle clusterware
• Cluster interconnects
• Oracle Kernel Components

Shared Disk System

Below are the three major type of shared storage which are using in RAC

Raw volumes: A raw logical volume is an area of physical and logical disk space that is under the direct control of an application such as database or partition rather than under the direct control of the operating system or a file system. This as the only available option in 9i.

Cluster File system: This option is not widely used and here the cluster file system such as Oracle Cluster file system (OCFS) for MS Windows and Linux holding the all datafiles of RAC database

Automatic Storage Management (ASM): Oracle recommended storage option which is optimized for cluster file system for Oracle database files introduced in Oracle 10g.

Oracle Clusterware

Clusterware is the mandatory piece of software which is required to run Oracle RAC option,it provides the basic clustering support at OS level and enables oracle software to run in cluster mode.

Clusterware enables nodes to communicate each other and forms the cluster that make the nodes work as a single logical server.

Clusterware is managed by cluster ready services (CRS) using the Oracle Cluster registry (OCR) and voting disk

Cluster Ready Services (CRS)

Oracle clusterware uses CRS for interaction between the OS and the Database.CRS have five components - namely Process Monitor daemon (OPROCd: obsolete in 11gR2),CRS daemon(CRSd),Oralce Custer Synchronization Service daemon(OCSSd),Event Volume Manager daemon(EVMd) and the Oracle Notification Service (ONS)

CRS is installed and run from ORA_CRS_HOME in 10g and GRID_HOME in 11g

Starting and Stopping CRS

crsctl stop crs      # Stops Oracle clusterware

crsctl start crs     # Starts Oracle clusterware

crsctl enable crs    # Enables Oracle clusterware

crsctl disable crs   # Disables Oracle clusterware

Note: These commands should be used in root user (Superuser)

Oracle Cluster Synchronization Services

OCSSd provides synchronization between the nodes. Failure of OCSSd causes the machine to reboot to avoid split-brain situation.OCSSd runs as oracle user.

Event Manager Process

This runs to daemon process EVMd. The daemon process spawns a permanent child process called evmlogger and generate the events when things happens.

Oracle Notification Services

Whenever state of cluster resource changes ONS process in each node will communicate with each other and exchange the high availability information.

Oracle Process Monitor

The oracle process monitor daemon(OPROCd) identifies the potential cluster node hang and rebooting the hang node.

Oracle Cluster Registry (OCR)

OCR uses the cluster registry to keep the configuration information. This should be a shared storage and should be accessible  to all the nodes in the cluster simultaneously.This shared storage is known as Oracle cluster registry (OCR)  

Oracle Logical Registry(11gR2)

Similar to OCR, introduces in 11gR2 but it only stores information about the local node. It is not shared by other nodes of cluster and used by OHASd while starting or joining a cluster.

Voting Disk

A voting disk is a shared diks that will be accessed by all the member of the nodes in the cluster.It is used as   central referece for all the nodes and keeps the heartbeat information between the nodes.If any of the node is unable to ping the voding disk,cluster immediatly recogonize the comminucation failure and envicts the node from the cluster.

Cluster interconnect

It is the communication path used by the cluster for the synchronization of resources and it is also used in some cases for transfer of data from one instance to another.Typically, the interconnect is a network connections that is dedicated to the server nodes of a cluster (thus is sometimes referred as private interconnect)

Single Client Access Name (SCAN)-11gR2

SCAN is a single network name that resolves in three different IPs registered in DNS or GNS.

Prior the 11g R2 if we add or remove a node from cluster it was required to changes the connection setting in client . By configuring the connection using SCAN name this problem is eliminated.

Oracle Kernel Components 

Set of additional background process in each instance is known as oracle kernel components in RAC environment.Since buffer and shared pool became global in RAC , special handling is required to manage the resources  to avoid conflicts and corruption.Additional background process (for RAC) and single instance background process works together and achieved this.

Global Cache and Global Enqueue Services

In Oracle RAC each instance will have its own cache but it is required for an instance to access the data blocks currently residing in  another instance cache.This management and data sharing is done by Global Cache services (GCS). Blocks other than data such as locks, enqueue details and shared across the instances are known as Global Enqueue Services (GES).

Global Resource Directory

In a cluster group, then centre repository formed by all the resources is known as Global Resource Directory (GRD).GRD is managed by GCS and GES.

Oracle RAC Background Processes

• LMS       Global Cache Service Process
• LMON   Global Enqueue Service Monitor
• LMD      Global Enqueue service Daemon
• LCK0     Instance Enqueue Process 

LMS: Global Cache Services Process

LMS- Lock Manager Server Process is used in Cache Fusion.It enebles consistent copies of blocks to be transfered from a holding instance's buffer cache to a requesting instance bufer cache with out a disk write under certian conditions.

It rollbacks any uncommitted transactions for any blocks that are being requested for a consistent read by the remote instance.

LMON:Global Enqueue Services Monitor

LMON-Lock Monitor Process is responsible to manage Global Enqueue Services (GES).It maintain consistency of GCS memory in case of any process death. LMON is also responsible for the cluster reconfiguration when an instance joins or leaves the cluster. It also check for the instance death and listens for local manages.

LMD: Global Enqueue Services Daemon

LMD-Lock Manager Daemon process manages Enqueue manager service requests for GCS. It also handles deadlock detection and remote resource requests.

LCK0:Instance Enqueue Process

LCK-Lock Process manages instance resource requests and cross-instance call operations for shared resources.

DIAG Process

Separate process using for diagnostic purpose, PMON will start incase this process dies.DIAG process will not affect any other background process.

================================

Reference: Oracle Real Application Cluster Handbook by K.Gopalakrishnan