Posts filed under 'ZFS'
Sun x4500 Thumper: Mapping logical drives to physical
The Sun x4500 has 48 disk slots, numbered 0 to 47. However on Solaris, drives are named according to their controller/target location. I was wondering how you work out how to go from the logical naming, to the physical one.
Well the answer lays on the x4500 Tools & Drivers CD. On it is a nifty package named "SUNWhd-1.07.pkg", which plonks a utility called "hd" at "/opt/SUNWhd/hd/bin/hd". Running spits out the serial numbers of the disks, their temperature, and at the end, it finally spits out some ASCII art depicting the layout:
---------------------SunFireX4500------Rear---------------------------- 36: 37: 38: 39: 40: 41: 42: 43: 44: 45: 46: 47: c4t3 c4t7 c3t3 c3t7 c6t3 c6t7 c5t3 c5t7 c1t3 c1t7 c0t3 c0t7 ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: 35: c4t2 c4t6 c3t2 c3t6 c6t2 c6t6 c5t2 c5t6 c1t2 c1t6 c0t2 c0t6 ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: c4t1 c4t5 c3t1 c3t5 c6t1 c6t5 c5t1 c5t5 c1t1 c1t5 c0t1 c0t5 ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ 0: 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: c4t0 c4t4 c3t0 c3t4 c6t0 c6t4 c5t0 c5t4 c1t0 c1t4 c0t0 c0t4 ^b+ ^b+ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ ^++ -------*-----------*-SunFireX4500--*---Front-----*-----------*----------
Rather funky, and useful!
Add comment November 16th, 2008
Sun Fire x4500 Thumper: Recommended ZFS Zpool Layout
The x4500 comes with 48 disks, two of which you typically use as a mirrored ZFS pair for the host OS, leaving 46 drives for data. One of the questions you’re faced with, is how to efficiently lay out your zpool configuration to balance performance, reliability and capacity.
For the particular workload we’ll be using the x4500 for, we want a balance across all 3. No particular factor wins out over the others - they’re all equally important. To further complicate matters, the box has six 8-channel SATA controllers, so you want to spread your workload across the controllers in an intelligent fashion.
There are many differing opinions on this. I sparked a debate on #solaris on Freenode posing the question, with some suggesting a single zpool with collection of mirrors if databases are involved, 1 drive per controller. Others suggested lots of small raidz2 sets in a single zpool.
After expirementing, musing, and researching on the web, we finally settled on the following configuration, which provides a fair balance:
pool: zpool01
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
zpool01 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4t1d0 ONLINE 0 0 0
c3t1d0 ONLINE 0 0 0
c6t1d0 ONLINE 0 0 0
c5t1d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c0t1d0 ONLINE 0 0 0
c3t0d0 ONLINE 0 0 0
c6t0d0 ONLINE 0 0 0
c5t0d0 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c0t0d0 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4t3d0 ONLINE 0 0 0
c3t3d0 ONLINE 0 0 0
c6t3d0 ONLINE 0 0 0
c5t3d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
c0t3d0 ONLINE 0 0 0
c3t2d0 ONLINE 0 0 0
c6t2d0 ONLINE 0 0 0
c5t2d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c0t2d0 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4t5d0 ONLINE 0 0 0
c3t5d0 ONLINE 0 0 0
c6t5d0 ONLINE 0 0 0
c5t5d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c0t5d0 ONLINE 0 0 0
c3t4d0 ONLINE 0 0 0
c6t4d0 ONLINE 0 0 0
c5t4d0 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c0t4d0 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4t7d0 ONLINE 0 0 0
c3t7d0 ONLINE 0 0 0
c6t7d0 ONLINE 0 0 0
c5t7d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
c0t7d0 ONLINE 0 0 0
c3t6d0 ONLINE 0 0 0
c6t6d0 ONLINE 0 0 0
c5t6d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c0t6d0 ONLINE 0 0 0
spares
c4t2d0 AVAIL
c4t6d0 AVAIL
This gives 2 spares, and 11 drives across 4 raidz2 groups. The chances of 3 drives failing in a 11 disk raidz2 pool before the spares finish rebuilding are (hopefully!) fairly low. In the unlikely event that 3 drives did fail, they’d more than likely be spread across the 4 raidz2 groups. It’s all about managing risk.
The command to create this would be:
zpool create zpool01 raidz2 c{4,3,6,5,1,0}t1d0 c{3,6,5,1,0}t0d0
zpool add zpool01 raidz2 c{4,3,6,5,1,0}t3d0 c{3,6,5,1,0}t2d0
zpool add zpool01 raidz2 c{4,3,6,5,1,0}t5d0 c{3,6,5,1,0}t4d0
zpool add zpool01 raidz2 c{4,3,6,5,1,0}t7d0 c{3,6,5,1,0}t6d0
zpool add zpool01 spare c4t2d0 c4t2d6
Finally, this inspiration for this configuration came from the Joyent web blog. Those guys know their stuff and have been using ZFS in production for longer than most.
Add comment November 16th, 2008