raid
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
raid [2009/08/27 09:17] – 172.26.0.166 | raid [2009/11/02 14:20] – 172.26.0.166 | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | === HPC RAID array === | + | ===== RAID ===== |
- | The storage | + | We have two RAIDs on the HPC |
+ | * Linux kernel software | ||
+ | * 3mware hardware RAID | ||
+ | ==== Drive numbering ==== | ||
- | It is currently reporting a degraded array (27 Aug 2009): | + | If you're looking at the front of the HPC you'll see four rows of drives. |
+ | * Rows 0 - 2 are SATA, connected to the hardware 3ware RAID card | ||
+ | * Row 3 are IDE | ||
+ | ===== Software RAID ===== | ||
+ | The Linux kernel has the '' | ||
- | < | + | Here is information on their configuration: |
- | Personalities : [raid0] [raid1] | + | |
- | md1 : active raid1 hda1[0] | + | < |
- | | + | /dev/md0 on / type ext3 (rw) |
+ | /dev/md3 on /boot type ext3 (rw) | ||
+ | /dev/md2 on /scratch type ext3 (rw) | ||
+ | /dev/md1 on /export type ext3 (rw) | ||
+ | # df -h | grep md | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | / | ||
+ | |||
+ | It should be noted that ''/ | ||
+ | < | ||
+ | Filename Type Size Used Priority | ||
+ | / | ||
+ | |||
+ | A snapshot of the software RAID's health: | ||
+ | |||
+ | < | ||
+ | Personalities : [raid1] [raid0] | ||
+ | md3 : active raid1 hdd1[1] | ||
+ | | ||
| | ||
- | md3 : active raid1 hdc3[1] hda3[0] | + | md1 : active raid1 hdd3[1] hda3[0] |
- | | + | |
| | ||
- | md2 : active | + | md2 : active |
- | | + | |
| | ||
- | md0 : active | + | md4 : active raid1 hdd6[1] hda6[0] |
- | | + | 2168640 blocks [2/2] [UU] |
+ | |||
+ | md0 : active | ||
+ | | ||
| | ||
unused devices: < | unused devices: < | ||
+ | |||
+ | ==== Repair RAID ==== | ||
+ | |||
+ | Setting a disk faulty/ | ||
+ | |||
+ | # mdadm --fail /dev/md0 /dev/hdc1 | ||
+ | |||
+ | DO NOT run this every on a raid0 or linear device or your data is toasted! | ||
+ | |||
+ | Removing a faulty disk from an array: | ||
+ | |||
+ | # mdadm --remove /dev/md0 /dev/hdc1 | ||
+ | Clearing any previous raid info on a disk (eg. reusing a disk from another decommissioned raid array) | ||
+ | |||
+ | # mdadm --zero-superblock /dev/hdc1 | ||
+ | Adding a disk to an array | ||
+ | |||
+ | # mdadm --add /dev/md0 /dev/hdc1 | ||
+ | |||
=== To Do list: === | === To Do list: === | ||
Line 24: | Line 73: | ||
Prepare written instructions on how to repair disk arrays. | Prepare written instructions on how to repair disk arrays. | ||
+ | |||
+ | What disks to we have? | ||
Add extra spare disks? | Add extra spare disks? | ||
Line 29: | Line 80: | ||
How do you know which physical disk is broken to replace it? | How do you know which physical disk is broken to replace it? | ||
+ | |||
+ | ===== Hardware RAID ===== | ||
+ | |||
+ | A 3ware 9500S SATA RAID card using the 3w-9xxx kernel module. | ||
+ | |||
+ | ==== Physical Disk Layout ==== | ||
+ | |||
+ | We have one RAID controller, ' | ||
+ | |||
+ | | Port 8 | Port 9 | Port 10 | Port 11 | | ||
+ | | Port 4 | Port 5 | Port 6 | Port 7 | | ||
+ | | Port 0 | Port 1 | Port 2 | Port 3 | | ||
+ | |||
+ | ==== Repairing ' | ||
+ | |||
+ | There is a utility, tw_cli, which can be used to control/ | ||
+ | |||
+ | Study the output of '' | ||
+ | * Which controller is active? (c0, c1, etc) | ||
+ | * Which unit is degraded? (u0, u1, u2, etc) | ||
+ | * Which port is inactive or missing? (p1, p5, etc) | ||
+ | |||
+ | <note warning> | ||
+ | |||
+ | Remove the faulty port: | ||
+ | < | ||
+ | Insert a new drive and rescan: | ||
+ | < | ||
+ | Rebuild the degraded array: | ||
+ | < | ||
+ | |||
+ | Check the status of the rebuild by monitoring ''/ | ||
+ | |||
+ | < | ||
+ | 3w-9xxx: scsi1: AEN: INFO (0x04: | ||
+ | |||
+ | This sucks: | ||
+ | |||
+ | < | ||
+ | 3w-9xxx: scsi1: AEN: INFO (0x04: | ||
+ | 3w-9xxx: scsi1: AEN: ERROR (0x04: | ||
+ | |||
+ | < | ||
+ | Password: | ||
+ | // | ||
+ | |||
+ | Unit UnitType | ||
+ | ------------------------------------------------------------------------------ | ||
+ | u0 RAID-5 | ||
+ | u1 RAID-5 | ||
+ | u2 RAID-5 | ||
+ | |||
+ | Port | ||
+ | --------------------------------------------------------------- | ||
+ | p0 | ||
+ | p1 | ||
+ | p2 | ||
+ | p3 | ||
+ | p4 | ||
+ | p5 | ||
+ | p6 | ||
+ | p7 | ||
+ | p8 | ||
+ | p9 | ||
+ | p10 OK | ||
+ | p11 OK | ||
+ | |||
+ | Looks like another drive failed. |
raid.txt · Last modified: 2010/09/19 23:58 by aorth