c3

Recently, Lime Tech has done a great job of keeping drivers updated. And you can always ask to get a driver added/updated if you find a need.

Considering disk drives never read a single sector, and unRAID places the contents of a single file on a single disk, it is going to very hard to beat read performance with any kind of reconstructed read. Typically the "next" read is already in cache on the disk drive, limited by track size and file fragmentation. This not like RAID 10 at all. Split or dual reads on duplicated data is one disk operation verse one disk operation. Reconstructed read is one disk operations verse the slowest of N disk operations. Every disk operation in the reconstructed read has to finish faster than the single direct read. Like the famous bear race, the single direct read only has to finish second to last to win.

Remember "tape is dead", and yet the amount of tape grows each year. The roadmap for incoming disk drive technology continues for many years into the future. Helium, SMR, TDMR, HAMR, BPMR, HDMR are technologies to increase density. Each costs time and money, and that needs to be recouped. Having a demand is key to these technologies fruition. The demand for client HDD (sub 2TB) is fading so fast, best to think of it as gone. Client HDD still ship more than capacity HDD, but no new technology is needed. The capacity HDD is now the only source of funding for future HDD technologies/implementations. While capacity SSDs do exist, they are far from price competitive. This lifts the price pressure on capacity HDD. The loss of unit count from client HDD is rippled into cost per capacity. If/When the roadmap technologies become available in the market, pressure from competing tape and SSD will determine pricing. While SSD and HDD density advances have fluctuated between 10% and 40% per year, tape has been remarkably stable at 30% per year.

dd if=/dev/zero of=/dev/sdX bs=512 This will not be fast, but should write new data to all sectors. Another is badblocks -wsv /dev/sdX You could run an extended smart test to try and locate which sector(s) are having trouble reading. That would allow you to focus on just the pending.

I did also cover a possible cause for non-fatal, persistent slow block, and products for remediation. Your concern for both the reported errors and the measured performance is valid. If the manufacturer/source will replace it, that is a great plan.

Yes, I have used MHDD, and many others like it. I just don't spend a lot of time trying to predict failure or performance of single components. I work to achieve data durability and performance under all conditions at scale.

Yes, I did notice it said <500ms and >500ms, and 30,000ms is a long way from 500ms. TLER is 7,000ms. If response time is being used to determine drive health, the environment needs to be carefully controlled. Disk drives do not report when they re-calibrate for temperature, vibration, or even noise abatement. Which are possible sources for the >500ms response time, without being indicative of drive health. These are transitive conditions. Repeating the test would show different results. It is possible the sector was written as near off track, and deliver persistent slow performance. It can be corrected by rewriting the sector. And there are products... Or the sector could be difficult to read due to the media being scuffed, particles flying about, etc. Most of which lead to permanent drive failure. Another direction to take is issue additional reads. Drive drives fail, and the data must survive. So data is protected by additional writes. By issues additional reads, the failure or performance degradation is mitigated. If the data is protect single or double parity, a stripe read can be used to respond. Classically this was done serially, if the first read was slow or an error, then more reads were issued. Things like TLER were put in place to limit the performance impact. But the additional reads can be issued in parallel. Then the data is returned by the first of A) the block returned from a single target disk, or B) the block rebuilt as soon as enough stripe is returned. Either of which should be accomplished well below the performance threshold.

Unless you have changed something, 500msec should not be a read error. It is slow, but the timeout is measured in seconds and defaults to 30 seconds. This is the reason for TLER, the 30 seconds is wasted time when the array can rebuild from other drives far faster.

Since I buy plenty of the external enclosure drives, using USB to test/preclearing eases the risk of shucking a DOA/early failure drive. It is much slower, but often the testing finishes long before I get time to get back to work on them. Weekdays can be hard to get time for home projects. So I use USB, but not because no slots/bays, just to get some testing before breaking plastic The convection cooling is not what you really want, even on the bare drive. I do add a bit of forced air cooling, as everyone has said. Mine is cut cardboard boxes, and wonderful blue tape.

This is old. But now, the exploit has been released. https://www.helpnetsecurity.com/2017/04/11/exploit-vulnerability-routers/ Back in 2013, problems with UPnP were discovered. https://community.rapid7.com/docs/DOC-2150 Cisco even fixed some/all of the issues. But the problems were wide spread, requiring not only a fix from many device makers, but firmware updates done by customers. The first is hard, the second is practically impossible. I know I have vulnerable devices around my house, not plugged in, just collecting dust in the just in case pile, never getting firmware updates...

http://ask.adaptec.com/app/answers/detail/a_id/17414/~/support-for-sata-and-sas-disk-drives-with-a-size-of-2tb-or-greater I would not recommend that card as it is primarily a RAID controller, but versions(updates) do support large drives.

What about the rest of the data? You are thinking correctly, without raid1 the cache drive has the potential to fail and you lose data changed since the last mover run. If you are fine with that, all good.

Since the phone is the hotspot, the data plan and hotel wifi are out of the picture.

Yes, this limited free space will cause issues. My advise, buy an additional drive so the archive drives are not forced to garbage collect so much.

What model/version are you finding inside?

Over time, the parity drive is given more writes than any other drive. If your array is 3 drives, D1, D2, and P1. When a file is written to D1, P1 gets writes too. And when a file is written to D2, P1 also gets writes. If you fill D1 with 2TB, and then fill D2 with 2TB (unRAID default behavior), P1 will have written 4TB, the same 2TB written twice. The flash usage in the hybrid is optimized towards reads to boost boot and application load performance, and avoid the issues with overwrites. Most unRAID user reads are serviced by data drives. Only in a degraded state is the parity drive used for user reads. Combined, the questionable best position would be a data drive.

I am not sure we agree on who/what is a paper boy. In my poor model, that is the ISP/carrier. They don't enter homes without permission, but all data entering or leaving is closely inspected, as required by governance under the banner of security. And then let go to profit. Loyalty programs are another example. That grocery discount card, yup nice big data on your diet, health, spending, income, location, travel, etc. But you get a discount, and they are covered by the FTC. Anyone getting a discount from Comcast because they sniff all your packets?

+1 on the backup You'll find that pretty everyone agrees you need a backup for anything you want. There are just too many things that can go wrong when you only have one copy.

No need for separate server, can use a separate share.

I am going to try to avoid being political, if I fail, please call me out so I can edit. I am not a politician or legal expert. The thoughts, ideas, and opinion are mine, not my employer. I do work for a company in the wider field. There are at least three things going on. First the collection of data, second the FCC rules and this week's congressional action, and then the FTC exclusion. This article in WP may provide some background. The first part was the growing data collection happening at the ISP level. Twenty-five years ago, ISP had their hands full trying to keep the physical infrastructure working. To do that they collected a lot of data, think packet loss. ISPs also collected data on usage, because they wanted to be profitable. ISPs need this data to accurately price services. If their average customer used a line 8 hours a day and or peak demand was 1000 modems or 10mbit. In order to modify the average customer, price was the single knob. That is not a very good model. So caps were put in place, caps on the connected hours, caps on the data rate, and caps on the data transfer. More knobs made for a better model. Roll forward to this century, always on, fiber to the neighborhood/house, etc, some of these knobs became less useful/pointless. But more/better data collection enabled a better understanding of the traffic flows, and the model was improved. The improve was to look at both sides of the traffic flow as sources of revenue, peering became a knob. Leading to net neutrality thing. Traffic flow shifted to data flow (deeper packet inspection), potentially ballooning the data collection, privacy concerns. Second part, rules on what must be collected, can be collected, who does the collecting, how the collected can be used. What is neutral? What is privacy? All valid discussions. See that first part, must be collected, Twenty-five years ago, ISPs single knob, money was the only data must be collected and that was for paying taxes. But more recently, much more became must be collected, for security. The recent action, as mentioned above, can be claimed to address some unfairness between business models. The mentioned difference between Facebook(content provider) and Comcast(ISP/carrier) is far greater to me than mentioned above. One I pay money to, the other I don't. So, fundamentally, they must be different business models. One is doing business with me, and the other is not. The line between ISP and carrier is often crossed in these discussions, who must/can do what. I am not going to try and split them correctly, just a poor model. The carrier was simple, a means of delivery this to there, a river, or man made, a road. The ISP, more like a news paper boy doing home delivery. They didn't read the paper, they threw them. But what if the paper was dangerous, seditious, treason? The boy was asked where he got it and off to the source. But the source and destination pay the paper boy not to read, or answer questions about the delivery. The boy is in a bind. Governance is raised to require the boy (users of public roads) to know what he is delivering and be forth coming with details. Now, bill of laden is required at source and available as required along the way. When the requirement scales to deep packet inspection and record retention by the ISP, the cost of delivery goes up. Rather than raise prices the paper boy wants to sell the details of his delivery to news paper for headline news. Now public security is being used for financial gain. Entities outside this realm had different rules, and different business models. The reasons for collection are completely different, the scope is completely different, and the service provided is completely different. Why do these business models need to the same rules? But do they? Non carriers have to follow the privacy rules of the FTC, carriers are excluded. The repeal is not a repeal of the data collection, but a repeal of the privacy rules on carriers.

I am going to necromancer this thread due to the recent actions in the US by congress I figure more than a few people will be seeking a VPN provider recommendation.

I am sorry for the delay, I was busy working on things like data checksum on the XFS roadmap. Everyone there understood why, it was just the timing and who would do the work. I did take time to have discussions with several disk drive manufacturers about the ECC performance, which remains RS. Two of them indicated I might get my hands on the details under NDA, of a non current device (like from 2TB). We spent a lot of time talking about the impact adding read heads (TDMR) will have on this whole process. There was a pretty good joke about going from helium to vacuum drives would help stabilize the head, but then how to fly in vacuum, maglev. I guess you had to be there. Since I was told RS is still being used, Lemma 4 ends with (emphasis is mine); "If that occurs, then Lemma 4 implies that there must have been more than e errors. We cannot correct them, but at least we have detected them. Not every combination of more than e errors can be detected, of course—some will simply result in incorrect decoding". This is the foundation of UDE(paywalled), which drives the need for filesystem level checksum. UDE is a larger set of conditions, especially when talking spinning rust. You can see where TDMR will help. To improve the chance of avoid anything that might have my ideas, work, or decisions; use Microsoft products, but only the newer filesystems. In other news, quietly slipped into unRAID 6.3.2 was f2fs. Which is very exciting (or worrisome), especially post 4.10, probably the next roll of unRAID. f2fs now has SMR support (took 2 years). But the stuff I work on takes a long time to do and even longer to get rid of. SMR is just the beginning, but the whole concept of decoupling the write head from the read head/track width was fundamental to driving density. Other filesystems will follow, and/or use dm-zoned. Doing so will have benefits for flash write durability. Best of luck avoiding. But things like filesystem checksum will be needed outside the spinning rust world, and the OP is probably grateful.

If all you are looking for is weakness in ECC, there are plenty of papers on solomon-reed (and it's replacements). It comes down to a game of show me your ECC method and there is a dataset which will be undetected. As mentioned early on, the more recent case of this is SHA. Widely used and thought to be collision "impossible", until 2005 when a non brute force method was published. Now the non brute force method has been improved and weaponized. It is important to note the "impossible" was always really just impractical. The time/probability was so large people just ignored it. Some organizations, like CERN, work at larger scale and understand that these can not be ignored, but measured at scale. If the data you are trying to protect is x bits long, protecting is with fewer bits y, leaves a gap. In the case of the 512byte sector, the ECC is (40) 10 symbol fields. So, 512bytes is protected by 50 bytes. This is all done at the DSP level so the bits to bytes, baud rates etc apply. Now in the 4k sector the ECC is 100 bytes. Yeah, more data protected with a smaller portion allocated to protection. And remember, unlike SHA, this 100bytes has to not only detect errors, but correct errors. This double edge sword is played against the manufacturer's goal of quality vs cost. Each quality improvement becomes an opportunity to reduce cost. When the spindle motor wobble/vibration is improved, the track width is narrowed. The error rates goes down with one and up with the other. The current range is 10^14 to 10^17. Some will take that as impossible. I am sorry for the old papers. I'll stop doing that. They are from when I worked for an enterprise storage manufacturer. I have since moved up the stack to OS and filesystems, where we flat out tell disk drive manufacturers there is no expectation of perfect, and work to build robust systems on imperfect hardware.

I have read it, more than once.

While everyone wants to talk about Annual Failure Rate (AFR), there are plenty of other measures, some more important depending on how the device is used. SSDs have a lower AFR than HDD. SSDs also have higher uncorrected errors. This means SSDs lose data more often, but keep working. You can read the study from last year. Many years ago the BER number from drive manufacturers were confirmed by CERN. Pretty much everyone agrees, there is no guarantee that data will be written and read without corruption. CERN found 500 errors in reading 5x10^15 bits. "Silent data corruption is an unrecognized threat that may affect as many as 1 in 90 SATA drives." from NEC as an enterprise storage provider.