I should have not assumed that gkfahnbulleh was building a RAID 6. You can certainly use a RAID 5 with an additional hot spare.
One thing is the number of drives does not make the RAID, it is the formatting and method to save the data.
I do not disagree with your set up gkfahnbulleh, I think you have a particular attention to security and data safety but I am not sure that the average user would need that much. I am set up in a 1.5 provider (1/2 time NP) with 3 additional users on the local network. Just a simple RAID 1 with good back up is enough for me for my data. The hospital I work for provides to the server services, we attach to the domain remotely (which I do not like).
If one of your RAID 5 drives fails then you are at risk for data loss since any additional loss of drives will be catastrophic. You should rebuild your array right away with that loss of the drive. If you continue to use the array without restoring it you are asking for trouble. Assuming that the hard drive failed because it was bad or defective then continuing to use array may be OK. However if the failure occurred because the controller is bad, case kicked over, coffee spilled on the case or the power supply spiked the hard drive then continuing to use the array is inviting disaster. You have to look at the server and make sure that everything is OK. You do not just continue to use the server without knowing what is going on. You can rebuild while in use, it is just matter of what risk you want to take.
As for the throughput of the RAID implementation a RAID 0 will have faster performance but not necessarily application performance. This is a theoretical discussion. With RAID you certainly have to compare apples to apples. A RAID performance is separate from the hard drive performance. If you want ultimate read results get a solid state drive. I am more use to the consumer desktop arena where a RAID 0 works faster. In smaller transactional data writes like to a database then the performance of a RAID 5 will suffer compared to a RAID 1 set up. I wonder if the choice of a RAID 5 will impact the performance of the server to more significant degree than the use of a RAID 1. From my brief reading on the subject smaller transactional data writes will suffer in a RAID 5 set up because of the parity writes. Of course a high performance RAID controller may negate some of these problems.
I do not have enough knowledge or experience to recommend one over the other but I would probably just use a RAID 1 set up with a very strong back up. The problem is that the more hard drives you use the more likely a failure is to occur. Also the use of 6 drives in a single machine certainly draws a lot of power and heat that is not properly managed will damage the equipment over time. I my opinion the average single office system with a couple of providers and staff 10-15 would not need such an enterprise set up with RAID 1 on the OS and RAID for data. If I were to do that I would attach a separate server with the data on the RAID 5 rather than one monolithic server.
The choice of RAIDs is not as much as performance issue but rather a fault tolerance one and how much you want to place your data risk for failure.
Geoff
