Step By Step: Configuring Automatic Storage Management (ASM) on Standalone Server with Oracle Enterprise Linux (OEL with Oracle 11g R2 Installed) using External Storage Managed by Openfiler

1) Cygwin

• Installation of Cygwin (v1.7.16-1) 

2) Oracle Enterprise Linux (v5.7 64 bit) & Oracle DB 11g R2

• Installation of OEL
• Creation of Oracle Enterprise Linux Virtual Machine
•  Installation of Oracle Enterprise Linux
• Connection between Host and Guest
• Oracle Linux pre-installation steps for Oracle DB 11g R2
• Installation of Oracle DB 11g R2

3) Openfiler (v2.99)

• Openfiler Overview
• Download Openfiler
• Installation of Openfiler  

4) Creation and Configuration of ASM

• Add three hard disks to Openfiler virtual machine
• Creation of new physical volume
• Enable all openfiler services  
• Register Oracle Linux at "Network Access Configuration" under "System" tab 
• Creation of logical volumes asmdisk01, asmdisk02, asmdisk03
• Assign iSCSI target to newly created volumes
• Configure iSCSI disk devices for Oracle ASM with ASMLib

5) Installation of Grid Infrastructure


6) Creation of Database


Installation of Cygwin
Cygwin is a collection of tools which provide a Linux look environment for Windows. Since we are going to install Oracle 11g R2 on OEL without GUI, Oracle DB needs an X-terminal to display user interfaces and Cygwin helps us display them on Windows host by setting the DISPLAY environment variable point to Windows host. For the download & installation of Cygwin please refer to Step By Step: Installing Oracle 11g R2 on Oracle Enterprise Linux without GUI (Graphical User Interface).

Installation of OEL & Oracle DB 11g R2
Oracle Enterprise Linux is Oracle's Linux distribution based on Red Hat Enterprise Linux. Please refer to Step By Step: Installing Oracle 11g R2 on Oracle Enterprise Linux without GUI (Graphical User Interface) for:

• Creation of Oracle Enterprise Linux Virtual Machine
• Installation of Oracle Enterprise Linux
• Connection between Host and Guest
• Oracle Linux pre-installation steps for Oracle DB 11g R2
• Installation of Oracle DB 11g R2 (Note: During "Create and Configure a Database" check "Install Database Software Only". We are going to create and configure database later.) 

Openfiler Overview
Openfiler takes the pain out of deploying and managing networked storage. The hardware, any industry standard server will do, is combined with Openfiler software and the result is a powerful networked storage solution that exports data via a full suite of industry standard storage networking protocols. Openfiler lowers deployment and maintenance costs for networked storage without compromising functionality or performance. 

Openfiler is a network storage operating system, fronted by a web based management user interface. With the features built into Openfiler, everyone can take advantage of file-based Network Attached Storage and block-based Storage Area Networking functionality in a single cohesive framework. 

Any industry standard x86 or x86/64 server can be converted into a powerful multi-protocol network storage appliance, replete with an intuitive browser-based management interface, in as little as 15 minutes. File-based storage networking protocols such as CIFS and NFS ensure cross-platform compatibility in homogeneous networks - with client support for Windows, Linux, and Unix. Fibre channel and iSCSI target features provide excellent integration capabilities for virtualization environments such as Xen and VMware.

iSCSI target functionality is especially useful for enterprise applications such as Microsoft Exchange server integration, Oracle 11g RAC backend storage or video surveillance and disk-to-disk backup - taken from Openfiler website.

Download Openfiler
Openfiler can be download directly from openfiler website
http://www.openfiler.com/community/download/ -> Openfiler NAS/SAN Appliance, version 2.99 (Final Release) - 507 MB

Installation of Openfiler
Open VMware and start creating a new virtual machine

Select Custom (advanced) to create virtual machine with advanced options

Next

Select Installer disk image file (iso) and navigate to the openfiler downloaded file. It will display a warning that could not detect which operation system is on disk image. Do not worry we will choose other linux kernel later.

Select linux under guest operating system and choose Other linux 2.6.x kernel 64-bit

  

Specify a name for virtual machine and the location where it will be stored.

Specify number of processors for this virtual machine

Specify the amount of memory you would like to use for this virtual machine

Select use bridge networking as type of network connection

Select LSI Logic (Recommended)

Select create a new virtual disk

  
Select SCSI (Recommended)

Specify how much you want this disk to be and select split virtual disk into 2 GB files

 Specify the location of storage

  
Review your configuration and after checking power on this virtual machine after creation click finish

Click Enter to install using graphical mode

Next

Select U.S. English as appropriate language for the keyboard

  
Click YES to format the data on virtual disk 

Leave defaults and continue to next step

Click YES to remove all partitions on /dev/sda

Configure manually IPv4 (192.168.1.8/24) and disable IPv6

  
Specify hostname (home.test.al) and if you do not have any DNS server configured in your home environment leave other fields empty 

 Click continue and specify time zone (Tirane/Albania)

Specify root password

Click next to start the installation

  
The installation will start and it won't take much time

  
After the installation is completed, reboot the system

Last screen from installation

   
Once the system boots up, start configuring Openfiler by pointing your browser at the host name or IP address of the Openfiler system. The interface is accessible from https port 446. e.g.. https://192.168.1.8:446

Management Interface: https://<ip of openfiler host>:446
Administrator Username: openfiler
Administrator Password: password

Add three hard disks to Openfiler virtual machine 
Shutdown openfiler virtual machine first by shutdown -h now command.

Click Edit Virtual Machine Settings and press Add button

 Select Hard Disk and press Next

 

Choose Create a new virtual disk and press Next

 

Choose SCSI (Recommended) and press Next

 

Specify Maximum disk size as 5GB and choose Split virtual disk into 2GB files. Press Next

 

  Choose Openfiler OS and press Finish

 

Add two more disks by following the above procedure. At the end it must look like the below prt sc

Creation of new physical volume

Log in to openfiler administration website and go to Volume -> Create new physical volumes

 

 

The below prt sc should appear. Choose /dev/sdb.

 

Go at section Create a partition in /dev/sdb and by leaving the defaults press create. /dev/sdb1 partition will be created. By clicking on Block Device (right panel), do the same for /dev/sdc and /dev/sdd

 

Enable all openfiler services  

Log in to openfiler web app and go to services tab. Enable all of them

Register Oracle Linux at "Network Access Configuration" under "System" tab 

Like the following prt sc oracle linux must be registered. Just enter the Name (Oracle Linux), IP (192.168.1.7) and NetmaskMask (255.255.255.0) and press update

Creation of logical volumes asmdisk01, asmdisk02, asmdisk03

Go to "Volume Group" in the right panel. Enter asmdisk03 in "Volume group name (no spaces)" text box and choose a physical volume, in my case /dev/sdd1. Press "Add volume group".

Go to "Add Volume" and fill the appropriate values for asmdisk03. Note that file system type should be iSCSI.

Assign iSCSI target to newly created volumes

Click "iSCSI target" in the right panel. Enter the name for target IQN (iqn-asmdisk01), leave settings as defaults. In "LUN Mapping" locate asmdisk03 and press Map. R/W Mode should be write-thru. In Network ACL allow Oracle Linux to write to this disk. Do the same for asmdisk01 and asmdisk02.

Return to Openfiler OS and restart iscsi-target with the following command:  

service iscsi-target restart

Return to Oracle Linux OS, restart iscsi service and make it start automatically every time the system is restart by following commands:

chkconfig --level 35 iscsi on

service iscsi restart

Again in Oracle Linux OS make iscsi discover manually and then automatically new created LUNs by the following commands:

iscsiadm -m discovery -t sendtargets -p 192.168.1.8

iscsiadm -m node -T iqn-asmdisk03 -p 192.168.1.8 –l 

iscsiadm -m node -T iqn-asmdisk03 -p 192.168.1.8 --op update -n node.startup -v automatic

If Oracle Linux can not discover new created LUNs try comment everything in Opnefiler:/etc/initiators.deny

Configure iSCSI disk devices for Oracle ASM with ASMLib

Partition the Disk Devices from Oracle Linux. Format these disks to contain a single primary partition to represent it at the time of creating ASM disk using oracleasm.Command is: 

fdisk /dev/sdb->n->1->p->default->default->w

fdisk /dev/sdc->n->1->p->default->default->w

fdisk /dev/sdd->n->1->p->default->default->w

Verified that the below RPMs are installed before configuring the ASM driver. (Refer to operating system package section at Step By Step: Installing Oracle 11g R2 on Oracle Enterprise Linux without GUI (Graphical User Interface) )

oracleasm-2.6.18-194.el5-2.0.5-1.el5 

oracleasm-support-2.1.3-1.el5 

oracleasm-2.6.18-194.el5xen-2.0.5-1.el5

oracleasm-2.6.18-194.el5PAE-2.0.5-1.el5

oracleasm-2.6.18-194.el5debug-2.0.5-1.el5

oracleasmlib-2.0.4-1.el5

Oracleasmlib-2.0.4-1.e15 is not found in downloaded iso file so it must be download explicitly (It can be download from this link).

Loaded the ASMLib Module:

oracleasm init

Created the ASM disks using oracleasm:

oracleasm createdisk DSK01 /dev/sdb1

oracleasm createdisk DSK02 /dev/sdc1

oracleasm createdisk DSK03 /dev/sdd1

oracleasm scandisks

oracleasm listdisks       

[root@home oracle]# oracleasm listdisks
DSK01
DSK02
DSK03

Installation of Grid Infrastructure  

In the Oracle 11g Release 2, the Grid Infrastructure is required to use Automatic Storage Management (ASM).

Copy the grid executables to Oracle Linux. This can be done by using Cygwin with the following command:

$ cd H:\\Meine\\Apps\\Oracle (Point to the location of grid in Windows host)

$ scp linux.x64_11gR2_grid.zip oracle@192.168.1.7:/home/oracle (Copy to Oracle Linux /home/oracle path)

In Oracle Linux unzip the file by below command:

unzip linux.x64_11gR2_grid.zip

Log in to Oracle Linux as oracle user. Set environmental variable DISPLAY=192.168.1.100:0.0 (windows host ip) at Oracle Linux. Start Cygwin x-window by startXwin command and add Oracle Linux to control list by issue xhost + 192.168.1.7. Then from Linux point to /home/oracle/grid folder and run ./runInstaller.

In the first window select  Install and Configure Grid Infrastructure for a Standalone Server

Selected language 

In the next window create ASM disks should be appear. Specify Disk Group Name (DATA), select redundancy as External and choose ORCL:DSK01

Use the same passwords for these accounts

Leave Operation System Groups as default and press Next

Leave Installation Location as default

 

Continue to Next after the reach of Finish button.

Log in as root and execute the below script

 

Last window from grid installation

  

Creation of Database

Log in to Oracle Linux as oracle user. Set environmental variable DISPLAY=192.168.1.100:0.0 (windows host ip) at Oracle Linux. Start Cygwin x-window and add Oracle Linux to control list by issue xhost + 192.168.1.7. Then from Linux point to Oracle bin folder and run ./netca for first configuring a listener and then run ./dbca for creation of database.

In step 6 of 12, as storage type must be choose Automatic Storage Management (ASM) as below:

Under UseOracle-Managed Files click Browse and then choose DATA in available disk groups

In next window do not specify flash recovery area. 

In next window check Sample Schema

Leave Initialization Parameters as default

Below is the summary

Create Database - Summary 

Database Configuration Summary 

Global Database Name: orcl.test.al 

Database Configuration Type: Oracle Restart Enabled Single Instance 

SID: orcl 

Management Option Type: Database Control Storage Type: Automatic Storage Management (ASM) Memory Configuration Type: Automatic Memory Management

Database is being creating\

 

If we open Enterprise Manager we see that database is up and running.

Hope you find these article helpful