OCFS Oracle Cluster Filesystem

See 10g RAC administration, VMWare
Refer to official guide ocfs2_users_guide.pdf, short and practical guide
Concepts
For every problem you must see:
tail -100 /var/log/messages

The IP used must be the public, not the virtual, not the private

Version 1
Datafiles, control files, redo log, archived log, OCR, CSS voting disk

Version 2
Version 1 + Compatible with 9i and 10g, manages Oracle binaries
 
Oracle Cluster File System
- NFS not supported
- Compatible with 9i and 10g
- Can manage binaries in RAC in a shared location
- Relies on hardware for HA
- Eliminates the need for raw partitions
- Cannot be created on Logical Volume Manager LVM

The correct rpm!
They are shipped with the dvd used to install the OS, in my example:
/media/cdrom/Enterprise/RPMS/ocfs2-2.6.9-89.0.0.0.1.ELsmp-1.2.9-1.el4.i686.rpm

Install following the order: 
ocfs2-tools-1.2.7-1.el5.i386.rpm
ocfs2-2.6.18-92.el5xen-1.2.9-1.el5.i686.rpm
ocfs2console-1.2.7-1.el5.i386.rpm
rpm -Uvh ocfs2*.rpm 

build rpm from source - UNTESTED
./configure --prefix=/usr
make
make install

Make a device available to ocfs
1) create a partition with fdisk, example: fdisk /dev/sdg
2) start ocfs services(see below) and launch the graphic console with root (ocfs2console)
3) select the "Format" menu...
 
[/etc/ocfs2/cluster.conf]
#configuration file, should be the same on each node, use ocfs2console
cluster:
        node_count = 2
        name = ocfs2

node:
        ip_port = 7777
        ip_address = 192.168.1.15
        number = 0
        name = node1.fadalti.com
        cluster = ocfs2

node:
        ip_port = 7777
        ip_address = 192.168.1.16
        number = 1
        name = node2.fadalti.com
        cluster = ocfs2
 
OCFS service
chkconfig --list o2cb
o2cb            0:off   1:off   2:on    3:on    4:on    5:on    6:off

/etc/init.d/o2cb status       #Check status
/etc/init.d/o2cb load/unload  #Load modules 
/etc/init.d/o2cb online ocfs2 #Online cluster
/etc/init.d/o2cb configure  

Check the status using
service o2cb status
Module "configfs": Not loaded
Filesystem "configfs": Not mounted
Module "ocfs2_nodemanager": Not loaded
Module "ocfs2_dlm": Not loaded
Module "ocfs2_dlmfs": Not loaded
Filesystem "ocfs2_dlmfs": Not mounted

Load
/etc/init.d/o2cb load

Start driver on boot
/etc/init.d/o2cb configure

Stop/start ocfs2 cluster service
/etc/init.d/o2cb start ocfs2

Restart all
#You need to unmount all ocfs2 disk using
umount /u03 
#Otherwise you get
Unable to stop cluster as heartbeat region still active
/etc/init.d/o2cb restart ocfs2
 
Mount
#datavolume implies a direct I/O. This is an absolute requirement

[/etc/fstab] entry
#_netdev waits for network to be up, datavolume for Oracle files
/dev/sdg1               /mnt/ocfs               ocfs2   _netdev,datavolume,nointr 0 0

Use the -o datavolume for Oracle files
mount -t ocfs2 -o datavolume /dev/sdal /u02
 
Format a partition
mkfs.ocfs to format the filesytem

#Create the mount point
mkdir /u03
#The following command creates an ocf s2 file system on device /dev/sdal with the label "/u03": 
mkfs.ocfsa -L "/u03" /dev/sdal 
or
mkfs.ocfs2 -b 4K -C 32K -N 4 -L /u03 /dev/sdc1

Known issues
I got:
Writing backup superblock: mkfs.ocfs2: Bit does not exist in bitmap range while backing up superblock.
---
use --no-backup-super
See http://oss.oracle.com/projects/ocfs2/dist/documentation/ocfs2_faq.html
 
ocfs2console                  #start graphic console

ocfstool(GUI) or resizeocfs/tuneocfs(CLI) 
 - specify volume size
 - resize the OCFS filesystem volumes
 - change fs metadata(permissions)
 - list and clear node configuration slots
 
 Set volume size for filesystem
 tuneocfs -S 210M /dev/sda8
 
 List node configuration slots on the filesystem(every node is registered on the filesystem)
 tuneocfs -l /dev/sda8
 Delete node slots 2 on the filesystem
 tuneocfs -N 2 /dev/sda8
 
fsck.ocfs
check an OCFS file system on a specified partition, read only by default
fsck.ocfs [options] /dev/sda8
  -n No heartbeat check
  -w Writeable, fix errors, file system mnust be unmounted by all nodes
  -V Version 
  -v verbose 
  -q Quiet 

mounted.ocfs2 /dev/sdal
Check which nodes are currently mounting a specific device

ocfsextfinder  print a list of unused extents

 

debugocfs
The following example dumps the volume header for the OCFS file system on device /dev/sdal: 
debugocfs -h /dev/sdal 

Dump the voting sector using the following command: 
debugocfs -v /dev/sdal 

dump a list of all files using 
debugocfs -l /dev/sdal 

dumps the file entry structure for the file /oradata/RAC/ tsOl.dbf on device /dev/sdal: 
debugocfs -f /oradata/RAC/tsOl.dbf /dev/sdal 

 

Tracing to /var/log/messages
To enable OCFS tracing, use 
echo -1 > /proc/sys/kernel/ocfs/debug_level 
echo -l > /proc/sys/kernel/ocfs/debug_context 

To disable OCFS tracing again, use 
echo 0 > /proc/sys/kernel/ocfs/debug_level 
echo 0 > /proc/sys/kernel/ocfs/debug_context 

may be permanently set in /etc/sysctl.conf