How to add a new node in a running Veritas Cluster instance

Steps for adding a new node in a running Veritas Cluster

1) Install the VCS depots on the new node with ‘-installonly’ option

# ./installvcs –installonly

2) Rename the startup scripts on all nodes as a precaution:

# mv /etc/rc2.d/S70llt /etc/rc2.d/s70llt
# mv /etc/rc2.d/S92gab /etc/rc2.d/s92gab
# mv /etc/rc3.d/S99vcs /etc/rc3.d/s99vcs

3) Modify /etc/llthosts, /etc/gabtab, /etc/llttab and /etc/VRTSvcs/conf/config/main.cf as required to show the new node. These modifications should be done on all nodes.

#vi /etc/llthosts

0 ggnqinta1

1 ggnqinta2

2 ggnqinta3

#vi /etc/llttab

Node1
set-node ggnqinta1

set-cluster 13

link lan1 /dev/lan:1 - ether - -

link lan2 /dev/lan:2 - ether - -

Node2

set-node ggnqinta2

set-cluster 13

link lan1 /dev/lan:1 - ether - -

link lan2 /dev/lan:2 - ether - -

Node3

set-node ggnqinta3

set-cluster 13

link lan1 /dev/lan:1 - ether - -

link lan2 /dev/lan:2 - ether - - 

# vi /etc/gabtab

/sbin/gabconfig -c -n 3

Caution : make sure the number of nodes is correctly reflected in the gabtab and 'set-node' is correct in /etc/llttab of each host or you will have problems.

4) Start GAB and LLT on the new node

# /sbin/rc2.d/S70llt start

# /sbin/rc2.d/S92gab start

5) Start the cluster on new node

# hastart

6) Add the new added system into configuration

 # hasys –add ggnqinta3

7) Save the configurtion

# haconf –dump makero

8) Move back the start scripts

# mv /etc/rc2.d/s70llt /etc/rc2.d/S70llt
# mv /etc/rc2.d/s92gab /etc/rc2.d/S92gab
# mv /etc/rc3.d/s99vcs /etc/rc3.d/S99vcs

9) The newly added node needs to be added in servicegroup configuration

# hagrp -modify SG SystemList -add ggnqinta3

# hagrp -modify SG SystemList -add ggnqinta3

# hagrp -modify SG SystemList -add ggnqinta3

# hagrp -modify SG AutoStartList -add ggnqinta3

# hagrp -modify SG AutoStartList -add ggnqinta3

# hagrp -modify SG AutoStartList -add ggnqinta3

How to recover from a I/O error in VxVM

While working as a Unix Admin , many a times i have experienced critical issues with Veritas Volume Manager. Most of the times its a heart stopping moment when you hear something is wrong with your disk Storage subsystem. After all , your data sits on Veritas. I am sure any Unix Admin worth his salt will agree with me.

But the beauty of Veritas is its simplicity. It stores objects in a very structured and systematic fashion and if you work out the problem logically you can solve it.

Here i present a typical issue which is encountered with Veritas Volumes i.e filesystems becoming stale or unmountable after a sudden power-off or a sudden loss of connectivity from SAN or HBA card. Moreover this is a very useful question for interviews as well.

Veritas throws I/O error , dont worry vxmend comes to your rescue ...... just unmount the filesystem forcefully with a '-f' and run below commands in sequence.

Vxmend......
Force the plex to go off ( its already stale)

# vxmend -o force off <plex>

Bring it on again ....

# vxmend on <plex>

Clean the plex of any errors as shown below:-

# vxmend fix clean <plex>

Start the volume.

# vxvol start<volume>

Run a fsck as usual

# fsck -F vxfs/dev/vx/rdsk/<diskgroup>/<volume>

Now mount the filesystem, it should be back up in a sane manner.

# mount -F vxfs /dev/vx/dsk/<diskgroup>/<volume> /mountpoint

I have always found Veritas to be very robust and powerful. Just keep learning without fear and worries. Cheers ............................