c3

Members
  • Posts

    1175
  • Joined

  • Last visited

Everything posted by c3

  1. c3

    Multi Hundred TBs

    It all comes down to how valuable the data is. I do not store data on less than m=3 and that is a test array of (3,3). Everything else is 4 or higher. Without knowing the value of the data it is hard to say what is reasonable or good enough.
  2. c3

    Multi Hundred TBs

    How valuable is the data? This is the only real question. From that truth you can build the required durability. It is best to get a good understanding of that before trying to build the storage system. It is very possible you will have one dataset which is of a different value than another, such as personally derived works (family albums) and backup copies of DVDs. By definition the DVD backups are backup and probably do not need the yet another backup, depending on the amount of work the curator has done, and potentially be lost. Personally derived works should not be stored as a single copy, regardless of durability, due to locality, use an offsite backup. As far as the math of RAID is concerned, basic RS can be used for an unlimited number of parity disks. But practically, the process needs optimizations, and so RS is replaced with others, Hamming, etc. A RAID stripe is the data words (n) and parity set (m). The current maximums for unRAID is the sum of n+m being 30, or (28,2) and (29,1). Each increase in n reduces durabilty, and each increase in m increases durability. The fewer data disks the better durability, the more parity disks the better durability. As mentioned above the best/easy way to get more than 2 parity disks is backup. Doing so not only increases durability, but can done to also increase availability. In the possible unRAID combinations, (29,1) would be the lowest durability, and (1,2) would be the highest. I think there is way to run unRAID without parity, but that would be worse. All this durability comes at a price, assuming the disks are fully utilized, the ratio of (29,1) to (1,2) is (30X)/(87X). It costs 2.9 times as much to store data at the highest durability vs the lowest. A dataset stored on RAID is a single copy, with some level of durability. It is not a backup. Since dual parity is often implemented without RS, adding a third parity is complicated. Single parity can be done with simple addition (XOR). Row Diagonal Parity allows for optimal computations for dual parity, again using only XOR. Triple Parity is available, raidz3, etc. These higher level of durability are filed under the section titled erasure codes.
  3. I am using lots of 10G now, but it is never full. However it is often beyond the 1G mark. Isn't that the real question? Not will 10G be filled, but has the 1G limit been reached? Similarly, the use of (2) 10G (or 1G) is not about capacity, but availability. Everything possible is connected via mlag.
  4. I am surprised this is news to anyone. While the video may be fun, the actual data includes the performance loss after fan failure. The typical response of a server from the big box suppliers is to run all remaining fans at max. Like screaming this can extend seeks times, unlike screaming it can last for days. Even the solution is years old https://www.geek.com/chips/noctua-creates-worlds-first-cpu-cooler-with-active-noise-cancellation-1559029/ But remains unshipped https://smallformfactor.net/news/noctua-computex-2017 There was also a DIY project done by/at a university, but I can not find the link just now. It was basically the parts from ANC headset broken out and placed in the computer case. It helped, some. The drive manufacturers have also advanced their products with this data.
  5. If the PSU has the power you need, consider breaking out the soldering iron. And if your soldering/custom cabling making is weak, the power splitters / converters mentioned above are the source of problems, but often work for long periods before needing to be reseated, and then work for long period again.
  6. There is no question that the math supports statements like yours about odds of a URE during reconstruction, but you should also include the stripe width as a factor. Using larger drives on the same size dataset reduces the number of drives. So, it is not the drive density but the amount of data trying to be protected. (20) 4TB drives protected by single parity is more risky than (10) 8TB protected by single parity. Either may make you nervous. Changing to dual parity greatly increases the data durability. Some will be comforted by (20) 4TB protected by dual parity, but even more with (10) 8TB protected by dual parity. As far as experiencing URE, the occurrence during rebuild is similar to the occurrence during parity check. I have not seen many reports of spurious errors during monthly checks.
  7. Yes, there in lies the problem. You think you are leaving an airgap as (2) 7mm laptop drives fit in a 3.5in space (nominal z is 1in=25mm), but (2) of the 12.5mm or 15mm do NOT fit or leave air gap. All 2.5in drives are not laptop drives, and may not fit in the planned space.
  8. Today, many laptops only fit 7mm (your link is from 2010). This drive is 15mm Seagate 4TB BarraCuda ST4000LM024 This drive is 7mm Western Digital 1TB Blue WD10SPZX
  9. FYI: 2.5 inch drives (like 3.5 inch drives) are not all the same height (z). This may foul your plans to put x many in y space. The large capacity drives are not actually laptop drives, and dont fit in many laptops. Just something to watch for.
  10. Naw, I am still thinking the config needs to be changed to remove the old directory names, and include the new if you want them.
  11. Wow, the same thing as you reported in March 1, 2017, might want to try the same thing this year and see if it works.
  12. c3

    Harnessing Heat

    I think you mean Seebeck
  13. I suggest you locate a local IT resource (or become that resource). unRAID may be a good fit, but like any other IT infrastructure, it is not magic and will require some level of attention. unRAID is not known for great write performance. If write performance is important to your application, you'll want to use a cache drive(s).
  14. SATA is rated in Gb/sec and PCI in GB/sec SATA III, version 3.x, SATA 6Gb/sec has 600MB/sec throughput. PCI Express Gen 3, most numbers like 32GB/s are full duplex and twice any theoretical read or write rate.
  15. Ah, there is your misconception. All heads can read at once. The all heads read all the data in the track on all platters without moving. The whole cylinder can be read without a seek. This is throughput optimization disk have been using for sequential reads. It is pretty hard to beat. But as density grows, it may be time to give up some of that for more IOPs. This is also why throughput drops so much for non sequential reads. Mechanically it is too easy to read all the data around a single random request, and dump it. So, read ahead always has lots of data to try a second guess the next request, or next or next... A single head is far slower than the whole cylinder, and that includes all the encoding overhead, so the yield is well below those nice sequential burst speeds seen on parity checks. Most disk explanations leave out the real world case of multiple platters. Here is a class on disk basics from UT. Spinning disks have been optimized and tweak may times over for decades. They are far more than just a simple read head at the end of a single actuator. The idea of multiple actuators is not new, and this is not the first time disks have been built with more than one. Us longer term people will recall the Chinook from the last century (or the IBM 3340 for those even longer in the tooth). Back then we talked of RAID as redundant array of inexpensive disks vs SLED single large expensive disk. But that was after we hole punched the diskette and flipped it for doublesided use
  16. OK, let me rephrase that. This dual actuator does not change improve the number of read heads, or pretty much any other throughput factor. The current design is throughput focused. This change does not improve on that (nor it is meant to). The amount of data under the heads does not change due to dual actuators. It may change due to platter count or platter density, but not actuator count. The time to seek and settle does not change due to actuator count. Changing the data is written would change from the current throughput optimized, thus lowering throughput, in favor of increased IOPs. Most of the work shows that dual actuators can be implemented without reducing throughput. There is potential for improved throughput with the multiple read head design of BPMR, which may be used in conjunction with dual actuators. But dial actuators is not the factor. Of course BPMR is meant to improve platter density and that should improve throughput as well. There are several advancing technologies, most are meant to increase density, which often has the easy side effect of improved throughput.
  17. While this is a nice advancement, I do not see it changing things like parity check. Parity check is reading all the data. This dual actuator does not change the number of read heads, or pretty much any other throughput factor. This will address a different problem being seen with high density drives, the IOPs/TB. This is a huge problem seen in large data sets. The current density path put archive disks on a collision path with tape. But tape keeps moving that target and keeping price very low. The problem with keeping a large data set on disk, it has to be useful. Using data means using IOPs. But the IOPs per drive hasn't changed while the TB per drive has grown. This has created a straw to access the pool/lake of data.
  18. yes, it is possible to run the backhaul without a dedicated radio, and in this case the performance wont be hurt. Typically the dedicated radio is all about getting huge speed numbers. Orbi has a very nice dedicated backhaul with 1.7G bandwidth! But Orbi is not a mesh, and suffers the sticky client problem. Velop is a mesh and has a dedicated radio. My beginning research on Ubiquiti Networks' Amplifi shows they don't have dedicated backhaul radios. They use their AC1200 class radios for both backhaul and client device connection. But this will be acceptable if they can solve the problem. More detail to learn about Ubiquiti... Unifi seems to require internet to be functional due to some functions not be done local, this is an extra tax on the limited internet bandwidth available.
  19. Unfortunately, the device list is huge and uncontrollable, but I'll certainly look into SWIFI for myself
  20. Which unifi APs? They don't really mention a wireless backhaul link on the website? The UAP PRO seems to be the only comparable, but it has no third radio for the backhaul.
  21. Trying to solve the wifi everywhere problem in a house with a huge stone chimney wall dividing the house. Locate WAP on the larger west side, works great on the west, but very bad on the east (kitchen). Add second WAP on the east, phones connected to west don't roam, still have poor connection. Cycle the wifi and connection is good until moving back to the west. Does Velop really solve this?
  22. c3

    how is cache used?

    This keeps files under 90days old on RAID0, not a good idea.
  23. unRAID is not like other RAID systems, even with dual parity, loss of three drives, unRAID does NOT lose everything. All the remaining data drives are still valid independent filesystems. In a traditional RAID with data striping, the data loss is amplified by stripe size. XFS has xfs_repair which is very good for recovering from corruption. There are some dangerous options, which may be the cause of your concern. Typically, the filesytem must be unmounted, but xfs_repair has the "dangerously" -d option to allow a mounted filesystem to be repaired. Also the -L option which zeroes the log, losing transactions. XFS is getting enhanced corruption detection with the addition of checksum for data, currently only metadata has checksum (added v5, kernel 3.15). And +1 to making sure notifications is working. Mostly of my storage has more than three parity blocks, up to 14.
  24. Can never run too many blockers, for ads or other, especially now with crypto mining in web browsers. I run pihole as well.
  25. c3

    Drive Temperatures

    At idle, they power down. Worse case is parity check where low power drives may be in the 30C and 7200rpm drives at or over 35C.