Troubleshooting the lost diskset on Sun Cluster

In middle of night I check my cluster labs, and it show my apache resource group is not running… I re-check it, and I found that the cluster node didn’t mount the webds /global/web. My webds diskset is gone, I don’t know the root cause of this problem… 😀 maybe I’m doing another lab in the same node, and did not consciously change its configuration.

if i check the metaset of the webds status, it show no node that own this diskset

bash-3.00# metaset -s webds

Set name = webds, Set number = 2

Host                Owner

Mediator Host(s)    Aliases

Driv Dbase

d6   Yes

and if I running the metastat status for webds it coming up with error :

bash-3.00# metastat -s webds
metastat: clnode-01: webds: must be owner of the set for this command

The resolution is simple, below my troubleshooting :
1. boot your node-1 & node-2 in non cluster mode
2. comment out the share device at /etc/vfstab
3. boot your node-1 & node-2 in cluster mode
4. on node-2 :
force purge the lost disket :

metaset -s <setname> -P -f

5. on node-1 :
force purge the lost disket :

metaset -s <setname> -P -f

re-recreate your metaset disket :

metaset -s <setname> -a -h NodeA NodeB
metaset -s <setname> -a <diskpath0> <diskpath1> ... <diskpathN>
metaset -s <setname> -a -m NodeA NodeB

(should show new set and ownership)

Note : because my webds disket is set of the svm disk, I re-create the soft partition on it..

bash-3.00# metainit -s webds d1 1 1 /dev/did/rdsk/d6s0
webds/d1: Concat/Stripe is setup
bash-3.00# metainit -s webds d200 -p d1 3g
d200: Soft Partition is setup
bash-3.00# metastat -s webds
webds/d200: Soft Partition
    Device: webds/d1
    State: Okay
    Size: 6291456 blocks (3.0 GB)
        Extent              Start Block              Block count
             0                       32                  6291456

webds/d1: Concat/Stripe
    Size: 10457088 blocks (5.0 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        d6s0            0     No            Okay   No

Device Relocation Information:
Device   Reloc  Device ID
d6   No         -

testing mount, and ls direcory :

bash-3.00# mount /dev/md/webds/dsk/d200 /global/web
bash-3.00# ls -l /global/web
total 24
drwxr-xr-x   2 root     root         512 May 31 21:55 bin
drwxr-xr-x   2 root     bin          512 May 25 06:22 cgi-bin
drwxr-xr-x   2 root     root         512 May 31 21:59 conf
drwxr-xr-x   2 root     bin         1024 May 25 06:22 htdocs
drwx------   2 root     root        8192 May 31 20:56 lost+found

in theory, And you should be happy, your cluster resource group is running again.

bash-3.00# clrg status

=== Cluster Resource Groups ===

Group Name    Node Name             Suspended   Status
----------    ---------             ---------   ------
nfs-rg        clnode-01             No          Online
              clnode-02             No          Offline

apache-rg     clnode-01:webapp-01   No          Online
              clnode-01:webapp-02   No          Offline

bash-3.00# clrs status

=== Cluster Resources ===

Resource Name      Node Name             State     Status Message
-------------      ---------             -----     --------------
nfs-res            clnode-01             Online    Online - Service is online.
                   clnode-02             Offline   Offline

nfs-stor           clnode-01             Online    Online
                   clnode-02             Offline   Offline

mycluster-nfs      clnode-01             Online    Online - LogicalHostname online.
                   clnode-02             Offline   Offline

apache-res         clnode-01:webapp-01   Online    Online - Service is online.
                   clnode-01:webapp-02   Offline   Offline

apache-stor        clnode-01:webapp-01   Online    Online
                   clnode-01:webapp-02   Offline   Offline

mycluster-webapp   clnode-01:webapp-01   Online    Online - LogicalHostname online.
                   clnode-01:webapp-02   Offline   Offline

reboot your node if needed. 🙂

Using Centos DVD as Yum Repository

Centos coming with dvd/cd as installation media, in some case you want to use it as yum repository. In this tech tips posting, I used centos 5.6 dvd.

1. Mount your dvd media under one of this directory as default :


I mount it under /media/CentOS directory, if not exist you should create it

# mount /dev/cdrom-hdc /media/CentOS

2. edit the configuration :

#vi /etc/yum.repos.d/CentOS-Media.repo

make sure enabled = 1

name=CentOS-$releasever - Media

Testing the repo:

3. If you want to use your dvd repo along another repo :

# yum --enablerepo=c5-media install package-name


# yum --enablerepo=c5-media install nmap

4. If you want to use only your dvd repo :

# yum --disablerepo=\* --enablerepo=c5-media install package-name

Example :

# yum --disablerepo=\* --enablerepo=c5-media install nmap
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * c5-media: 
c5-media                                                                                                | 1.3 kB     00:00     
c5-media/primary                                                                                        | 1.2 MB     00:00     
c5-media                                                                                                             3539/3539
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package nmap.x86_64 2:4.11-1.1 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

 Package                    Arch                         Version                          Repository                      Size
 nmap                       x86_64                       2:4.11-1.1                       c5-media                       680 k

Transaction Summary
Install       1 Package(s)
Upgrade       0 Package(s)

Total download size: 680 k
Is this ok [y/N]: y
Downloading Packages:
warning: rpmts_HdrFromFdno: Header V3 DSA signature: NOKEY, key ID e8562897
c5-media/gpgkey                                                                                         | 1.5 kB     00:00     
Importing GPG key 0xE8562897 "CentOS-5 Key (CentOS 5 Official Signing Key) <>" from /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : nmap                                                                                                    1/1 

  nmap.x86_64 2:4.11-1.1                                                                                                       


Java, Solaris and Mysql Certification Updates

Effective June 1, 2011 – Java, Oracle Solaris, MySQL, and NetBeans certification exams are being delivered through our new test delivery vendor — Pearson VUE — and are no longer available through Prometric. This consolidates all Oracle Certification exams within the operations of a single testing vendor, allowing us to best streamline our processes and service. Pearson VUE currently offers certification testing in more than 5,000 test centers worldwide, and we believe this will provide excellent service and global testing coverage for these Oracle certification exams.

Candidates can now visit Pearson VUE to find more information, register for an exam or locate a testing center in their area. Note that existing Oracle exam vouchers will continue to be valid as long as they have not expired.

It is important that you create a Pearson VUE account now so that you can continue to register for Oracle Certification exams:

  • Go to MyAccount on Pearson VUE and select Create A New Web Account and follow the prompts to create a profile and enter your contact information. See our tutorial video as well as these detailed steps for reference.
    • Please note that your Prometric exam history will not show in your Pearson VUE account. Sun Certification history will soon be available to candidates at CertView. Please visit the Oracle Certification Blog for continued updates.
  • IMPORTANT: Be sure to enter your profile and contact information on the next screens as it exists in your Oracle profile at CertManager, as the system will perform an automated process to match the record you are creating with the record that was uploaded from Oracle.
    • If Pearson VUE is not able to match your record, you will be placed in a queue and will be contacted by Pearson VUE within 72 hours for verification.

Also, if you schedule a Java, Oracle Solaris (Cluster), MySQL, or NetBeans exam with Pearson VUE between May 16, 2011 and August 1, 2011, you will receive access to a set of practice questions at no cost. For more information, view our full announcement and Frequently Asked Questions online at

Reference :