External RAID configurations

Royce: Yes, you are corrected. One can utilize the dynamic disks by spanned and striped (R0) multiple disks into one volume per Windows. That’s involve using windows software raid built in dynamic disk feature, which also take us back to Greg’s question.

“Can I partition these into 2 x 1.5TB partitions?”

No you can’t but you can do this

To get greater than 2TB in one physical disk or single array from a raid controller has to be sliced into smaller arrays at raid controller, then stripped it using dynamic disk function at windows.

One : You will need 2 separate R5 arrays at raid controller level, waste another disk (Number of Drives - 1)
Two : Involved windows SW raid to stripped both arrays into one dynamic stripped volume. Basically, turn both of that R5 (<2TB) into a dynamic R0 (>2TB) with windows raid.

These can get complicated and raise a questionable recovery process in the future.

As for expanding the volume in the array, yes you can.

It’s a common feature found on a dedicated raid controller. The term is Online Capacity Expansion (OCE). I’ve done that several times with different R5 array in the past. It's rather a small job for nowadays raid controller but nerve-wrecking and time consuming. It takes hours to days depend on your controller speed and array volume to expand the array into a fully working status. Some even support cross cards single array expansion, that's a single array spanning from multiple raid cards. I tried that one once and lost 2TB of data in a single click. Recovery a break-out raid array this size was a life time experience.

This is what I did with all my R5 setup before i use one.

Test you station for stability ran 24/7 for a week.
After setting up the R5, fill up half the volume, simulate the failed drive then test the repair.
If you have plan for expanding the capacity, test the Online Capacity Expansion. (First add the disk, then expand physical array. Next, use disk management in windows to expand the volume)
That usually take day or weeks (depend on raid controller speed), you’ll be glad yours step-up is up to the task or you're familiar with the steps when it come to recovery process.

Greg,

I'm getting into this discussion pretty late, but I would suggest you look at the Apple XRaid system. IT solves a number of issues for you...

1) With the RAID 5 controller in the system, you have the capability of running 7 drives in the RAID configuration... if you start with let's say 4, you can add a drive to the RAID config at any time.

2) Apple supports the system on Macs or Windows OS systems

3) The 2 Gb/s connection will give you the highest transfer speed direct to your desktop, and you can share the drives over the network to your laptop or other computers.

4) The system continues to grow with you, allowing a total of 14 drives in the enclosure controlled by 2 separate controllers, ethernet buses, and fibre channel connections + dual power supplies.

Greg, if you want a really simple external raid solution you might look at ReadyNAS NV+ here is a link: http://www.infrant.com/products/product ... S%20NVPlus

This is what I have been using with XRaid - you can hot swap or add disks of any size at any time and the device will automatically update everything - it couldn't be easier for someone like myself who isn't really up to speed on RAID devices.

Ron

Daniel: This is where the web info ambiguity is coming in with what I've been reading. There are multiple things going on with with Windows NTFS volumes >2TB, and people writing docs are evidently not being very clear which of several variables they are assuming when they claim it can be done. Having just finished a lot more digging, it looks like you are correct except in one case, Case #3 below.

• Case #1: 32-bit Windows XP is limited to addressing 2^32 blocks of storage. Since the standard block size is 512 bytes, this is where the 2TB limit comes from. If a RAID controller supports >2TB volumes, it would still appear to WinXP as several sequential chunks of space each of 2TB, except the final one which picks up whatever is left over. Converting these to dynamic disks in the disk manager, they could then be concatenated together using Windows' software RAID to format a single apparent NTFS volume >2TB. This is the scenario you lined out.
  Case #2: WinXP 64-bit, as well as Win2K3 SP1 and all Win Vista, support the GPT option we talked about before. Again, if a RAID controller can create a volume >2TB, it can be converted to a GPT volume under these versions of Windows and formatted as a single NTFS volume with >2TB. But since Greg has WinXP 32-bit, this option is out for him.
  Case #3: This is the one that was confusing me, since I was definitely seeing references to people running WinXP 32-bit with a single NTFS volume >2TB, based on a RAID controller supporting >2TB. Having poked into this further, it looks like this option has nothing to do with the RAID controller handling 64-bit LBA mode (which requires WinXP 64-bit, Win2K3 SP1, or Win Vista to be used). Rather it has to do with the RAID controller supporting a 32-bit Windows "hack", which exports a single RAID volume >2TB by changing the block size from 512 to 4096 bytes. This now permits a max volume size of 16TB and allows the WinXP disk manager to see & format a single NTFS volume >2TB as a basic disk, i.e. without converting it to a dynamic disk or using Windows RAID to concatenate multiple volumes together. However because a 4096 block size isn't standard, there may or may not be issues with this approach.

I found this link at Tom's Hardware, scrolling down a guy ("SomeJoe7777") lays out the different options quite clearly. Again the accuracy of info on the web may be suspect , but it seems to agree & make sense with what you're saying above so I think this is how it is.
http://www.tomshardware.com/forum/22936 ... ger-arrays

Just poking around, I found the WiebeTech RT5e eSATA 5-disk RAID enclosure. It claims compatibility with WinXP, and states it supports single volumes greater that 2TB. The product docs don't explicitly state that single volumes >2TB can be directly used by WinXP 32-bit -- there's that documentation ambiguity again. But if they can be, then I infer the WiebeTech RAID controller is using the 4096 byte sector hack from Case #3. If so then perhaps such an approach is stable, since WiebeTech has a good rep and I can't imagine they would use an unstable hack that would be liable to destroy more than 2TB of data if some software didn't work properly. But I don't know...

Greg: The bottom line now seems that if you are pinned to 32-bit Windows XP, it might be safest to break your single large RAID storage up into at least two NTFS volumes <2TB apiece. The alternative would be to use Case #1 above as Daniel described, or Case #3 with a RAID controller that supports a "Windows "hack mode" for volumes >2TB. I personally wouldn't be keen to do Case #1 because of adding the extra complexity, performance hit and failure risk of Windows volume management over top of the hardware RAID.

If I thought Case #3 was reliable, I might be happy to go that route. Daniel is correct that larger & larger volumes do create larger & larger backup & recovery matters to deal with. But we're all sort of stuck in that boat anyway as data capacity increases. Whether your 5-disk RAID box is mapped to a single volume or multiple volumes, if something takes out the box likely you will have a lot of recovery time to spend to get things back up & running.

This is why I personally always keep two RAID boxes paired with each other, one used as a primary storage server (mine are all NAS) and the second one backing up the primary. If the primary ever dies a horrible death I can immediately flip everything to point to the secondary while I take however many hours are necessary to get the primary running again. Then it becomes the secondary and life goes on. This costs $ but seems a reasonable cost to me to minimize potential downtime of my critical storage.

When you format the NTFS volume just select the larger cluster size like 64KB. The default is 4KB.
This can easily be done in Disk Manager in Windows.


From Wikipedia:
Maximum Volume Size
In theory, the maximum NTFS volume size is 2 raised to the power of 64-1 clusters. However, the maximum NTFS volume size as implemented in Windows XP Professional is 2 raised to the power of 32-1 clusters. For example, using 64 KiB clusters, the maximum NTFS volume size is 256 TB minus 64 KB.
Using the default cluster size of 4 KB, the maximum NTFS volume size is 16 TB minus 4 KB. Because partition tables on master boot record (MBR) disks only support partition sizes up to 2 TB, dynamic or GPT volumes must be used to create bootable NTFS volumes over 2 TB.

My head hurts.

Greg: Computers... they can make some things easier. But computer companies like to make things harder so that they can sell more computers to make them easier again...

Andreas: That's what I was saying earlier. But from Daniel's info and other materials it looks like in reality it's not that straightforward, at least not with WinXP 32-bit edition which is what Greg has. If I had a spare 3TB RAID box sitting around with some different controllers I would try this directly but unfortunately I don't...

Can I use FAT32? I want to in case I switch to a MAC and so I can use the volume (with a USB adapter) with my MacBookPro. Can I do what Andreas is suggestion with a FAT32 partition and, assuming I can, do I lose anything by going with larger clusters?

Also, I can and will consider a Vista upgrade if need be...

Thanks for the education. When I get it up and running I should write about my experience.

I wouldn't recommend FAT32 for several reasons for multi-TB storage, one of which IMO is reliability of the file system. Also it maxes out at 2TB on all platforms I believe. So again you'd have to create multiple partitions. You would gain direct access between Mac & Windows, which no other file system will do for you.

Still, if you're verging into using large storage systems with multiple computers including two or more operating systems, to me this begins to weigh more in favor of going the NAS route. NAS is designed for multi-OS multi-system use, while no single file system right now available for large capacity & direct access is designed for that. But you're aware of the performance trade-off with NAS and wish to steer clear of that, which I can understand.

My personal approach is to use NAS for secondary storage, and smaller but faster local disks for working storage. It's a bit more complicated on the surface of things, but really gives the best of both worlds with minimal mucking about.

IMO most people don't really need direct, instantaneous access to many TB of data. Rather what they need is fast access to a smaller working set of data, and the ability to relatively quickly swap in & out of different working sets. This is how I've configured my setup and it works well for the purpose:

• - 2TB NAS boxes as master storage where I keep master copies of everything, but classed as "secondary storage" because they're not used for primary "online" work for any of my workstations
  - NAS boxes accessed over gigabit ethernet to upload & download working sets; takes relatively little time compared to the time I spend actually working on the stuff
  - Fast 500GB working disks (or whatever is appropriate) in every major workstation
  - Some portable 500GB disks for when I need to move big hunks of stuff between several systems, some of which are not networked

If you're willing to consider a Vista upgrade, that would improve the storage situation if you still elect not to go the NAS route. The GPT style of partitions that Daniel & I have been talking about would then become an option with Vista, and should permit straightforward creation & use of >2TB NTFS volumes.

This doesn't help whatsoever with Mac access however. Nothing will help with that except for going to a multi-platform NAS or using smaller FAT32 volumes...

Greg, there is an issue with FAT32 and that is that Windows, XP and Vista can't format drives larger then 32 Gb.
There are third party products available that can format up to 8TB though.
Once formatted the drives can be mounted in Windows, although it is not supported by Microsoft according to this bulletin.

http://support.microsoft.com/kb/314463

My 500Gb WD external HD is formated with FAT32 and it works just fine.

Cluster size is the minimum allocation area that the OS can assign.
So if you have a file smaller then 64KB it will fit in one cluster but if it grows beyond it will need to allocate additional clusters to accomodate the file you are trying to store. If the file is smaller then 64Kb the additional area in the cluster is not used meaning that a lot of small files will generate a lot of unused storage area.

edit: pasted in the wrong link

Ok, then forget FAT 32. In the rare case I need to access data directly to my laptop (remember I can have the drive networked and access it that way anyway) I will use NTFS and understand the read only limitation on a MAC.

Royce, great works putting all the points together. I digged around 64 bit LBA support as well but nothing conclusive about what you already find.

But i came upon this tech article, option 3.
http://www.carltonbale.com/2007/05/how- ... tem-limit/

I think it's refer to Raid controller that support splitting >2TB array into smaller volumes. That also means what i said earlier is not nessaccery truth. Greg can partition >2TB single array into 2 or more at Raid controller level using provided raid management application without going tho windows sw raid. That if the raid controller has that feature. The resulting R5 array will have mutiple volumes but physically.. it's still one single array sob.

For instance, using auto Craving with 3ware raid controller.

http://freebsd.so14k.com/en/3DM_2_OLH-2-4.html

Then i look into varies Raid controller support site. In fact, one can define any size of an array when it was first created. So if Greg should be ok if he can live it 2 partitions.

Although it's the same known issues we already addressed but these step-by-step with screen captures guide line could be helpful to see how windows disk management managing >2TB partition.

http://www.webconnexxion.com/raid/info_ ... ages_id=43

Greg: Accessing the RAID storage over the network by sharing it from your Windows machine effectively is going the NAS route. Just not with a dedicated NAS box. I think this approach, either with Case #1 or Case #3 described previously, or upgrading to Win Vista to take on Case #2, are the only options on the table. These all assume that you must have all of the storage show as a single volume. If you are willing to live with 2 or more volumes you can of course create several <2TB and use them that way.

Daniel: Yes, I found that carltonbale.com reference as well. Many RAID controllers appear to do this 2TB carving thing automatically for >2TB arrays, unless the "Windows hack mode" is activated using 4KB sectors (such as could be done wiht the Areca RAID controller, as shown in the bottom link you posted). This auto-carving makes a single RAID array appear to Windows as several separate volumes. This is basically my Case #1. It appears that most (all?) RAID controllers supporting 64-bit LBA can create arrays >2TB and then make them available to Windows XP in 2TB chunks.

Of course, I realize it would simply be the same as NAS for other computers beyond the host but it is the host computer where I do my heavy editing. Any access by other computers would be to access a file here and there but not to work with them to any degree - so speed would be much less of an issue. My goal in staying away from NAS for the host editing computer is for speed - I will have the best of both worlds in this scenario. I just need the host to be running (which obviously is not the most efficient) when accessing from other computers.

If I can do it I have no problem splitting up the space via partitions.

I will say that Addonics has been very helpful and their tech support has told me that they will walk me through all the configuration. You don't get that from Amazon.

Greg, I think you're close to liftoff then.

Same as NAS "for the other computers" is what I meant to say. With 2 NTFS partitions each <2TB, fast performance on the main machine, and access over the network for other machines, that may be your best option over all.

Greg, I got an Addonics RAID card which I used to control four drives in an external eSATA box from PC PitStop: http://www.pc-pitstop.com/sata_enclosures/scsat44x.asp. I began having problems with data loss. Windows XP would tell me that various drives were "dirty" at reboot. Windows would then go through a process of trying to recover the data. I was running the drives as independent drives (JBOD), not a RAID configuration.

One probable error I was making was letting my PC and drives "go to sleep" at night. THat might not work with these RAID cards. I stopped doing that, but I was so worried about data loss that I bought a more expensive RAID card (ca. $300) from RocketRaid. (Sorry, the model numbers are at the office.) I'm still running the disks as four independent, "pass-through" disks, but the problems never recurred, so I'm happy.

I never determined whether the source of my problems was the Addonics card, or letting the PC go to sleep. I haven't been willing to experiment with terabytes of data now that it's working.

One final thing I discovered is that "hot-swappable" doesn't necessarily work. I thought I'd be able to swap out drives while the system was running. There may be a way to do that safely in my configuration, but I decided it wasn't worth it, so I power down the PC and then the drives completely if I want to physically swap drives.

The bottom line is that I'd recommend having everything well backed up on separate disconnected drives (or DVDs), and run your new configuration for a while with reboots, and do various checks of your files, to make sure it's working well. Not that it's easy to through 50,000 raw files to verify their integrity.

I'm not disparaging Addonics, because I have no idea if that card was causing any problems. But I thought you might be interested in my experiences.

I sure love having all those disks! In my own situation, I haven't seen the need for putting them in a RAID array, although I may try that with the working drive on my new system.