Wednesday, November 24, 2010

Physical Disk, Cell Disk, Grid Disk, and ASM Disk in Exadata

Exadata is the Ferrari of the technology world. It’s considered the technology for the select few, but everyone dreams about it. There is very little information available on the Internet right now about the architecture and the administration of Exadata, though it’s a huge subject in itself, which demand numerous high quality books, dedicated blogs, websites and special interest groups.

This blog post just mentions some of very important components of Exadata Storage Server, which are physical disks, cell disks, grid disks and ASM disks and their correspondence.

The Exadata Storage Server contains 12 physical disks.

There is one to one relationship between a physical disk and the cell disk. One physical disk corresponds to a single cell disk.

Then a Grid disk comprises of many cell disks.

And a Grid disk corresponds to one ASM disk.

An ASM diskgroup comprises of many ASM disks.

On the Exadata Storage Server, We can use cellcli command line utility in Exadata to see the information about physical disks, cell disks, grid disks and the cell.

The cellcli utility works from the root, celladmin and cellmonitor (read-only) users. The best practice is actually to run it from the last two less-privileged users, and not from the root user.


Now let’s have look at some of the disk management commands using cellcli utility.

1[root@mycell-net0 ~]# cellcli
2CellCLI: Release 11.2.1.3.1 - Production on Fri Oct 29 07:47:26 GMT 2010
3 
4Copyright (c) 2007, 2009, Oracle. All rights reserved.
5Cell Efficiency Ratio: 140

Just to give you an idea about what cellcli has on offer, look at the output of the help command:
01CellCLI> help
02 
03HELP [topic]
04Available Topics:
05ALTER
06ALTER ALERTHISTORY
07ALTER CELL
08ALTER CELLDISK
09ALTER GRIDDISK
10ALTER IORMPLAN
11ALTER LUN
12ALTER THRESHOLD
13ASSIGN KEY
14CALIBRATE
15CREATE
16CREATE CELL
17CREATE CELLDISK
18CREATE FLASHCACHE
19CREATE GRIDDISK
20CREATE KEY
21CREATE THRESHOLD
22DESCRIBE
23DROP
24DROP ALERTHISTORY
25DROP CELL
26DROP CELLDISK
27DROP FLASHCACHE
28DROP GRIDDISK
29DROP THRESHOLD
30EXPORT CELLDISK
31IMPORT CELLDISK
32LIST
33LIST ACTIVEREQUEST
34LIST ALERTDEFINITION
35LIST ALERTHISTORY
36LIST CELL
37LIST CELLDISK
38LIST FLASHCACHE
39LIST FLASHCACHECONTENT
40LIST GRIDDISK
41LIST IORMPLAN
42LIST KEY
43LIST LUN
44LIST METRICCURRENT
45LIST METRICDEFINITION
46LIST METRICHISTORY
47LIST PHYSICALDISK
48LIST THRESHOLD
49SET
50SPOOL
51START

Let’s see the output of some of the commands listed above:

01CellCLI> list physicaldisk detail
02 
03name:                   [5:3:2:0]
04diskType:               FlashDisk
05id:                     00000200000000000000
06luns:                   1_2
07makeModel:              "MARVELL SD88SA02"
08physicalFirmware:       D20R
09physicalInsertTime:     2010-10-04T21:05:46+00:00
10physicalInterface:      sas
11physicalSerial:         0000000000000000000
12physicalSize:           22.8880615234375G
13slotNumber:             "PCI Slot: 1; FDOM: 2"
14status:                 normal
15
01CellCLI> list cell detail
02 
03name:                   mycellnet0
04bmcType:                IPMI
05cellVersion:            OSS_11.2.0.1.0_LINUX.X64_100818.1
06cpuCount:               16
07fanCount:               12/12
08fanStatus:              normal
09id:                     0000X00000
10interconnectCount:      3
11interconnect1:          bond0
12iormBoost:              0.0
13ipaddress1:
14kernelVersion:          2.6.18-194.3.1.0.2.el5
15makeModel:              SUN MICROSYSTEMS SUN FIRE X4275 SERVER SAS
16metricHistoryDays:      7
17notificationMethod:     snmp
18notificationPolicy:     critical,warning,clear
19offloadEfficiency:      47,485.1
20powerCount:             2/2
21powerStatus:            normal
22smtpFrom:
23smtpFromAddr:
24smtpPort:
25smtpServer:
26smtpToAddr:
27smtpUseSSL:
28snmpSubscriber:
29status:                 online
30temperatureReading:     27.0
31temperatureStatus:      normal
32upTime:                 16 days, 14:01
33cellsrvStatus:          running
34msStatus:               running
35rsStatus:               running
36
01CellCLI> list celldisk detail
02 
03name:                   FD_15_mycellnet0
04comment:
05creationTime:           2010-07-14T02:05:06+00:00
06deviceName:             /dev/sdy
07devicePartition:        /dev/sdy
08diskType:               FlashDisk
09errorCount:             0
10freeSpace:              0
11id:                     00000000-0000-0000-0000-000000000000
12interleaving:           none
13lun:                    5_3
14size:                   22.875G
15status:                 normal
16
01CellCLI> list griddisk detail
02 
03name:                   CD_15_mycellnet0
04availableTo:
05cellDisk:               CD_15_mycellnet0
06comment:
07creationTime:           2010-07-14T02:12:55+00:00
08diskType:               FlashDisk
09errorCount:             0
10id:                     00000000-0000-000-0000-000000000000
11offset:                 528.734375G
12size:                   29.125G
13status:                 active

In the future posts, I will be touching the administration of celldisks, as how to perform operations like (import/export/create/drop/alter) on the celldisks.

The post was originally published by me at Pythian's Blog.

No comments: