January 19, 201115 yr Note: You could have also used the button on the unMEU "User-Scripts" page to check for updates and then install them. There is no need to use the command line if you don't want to (the buttons just invoke the unmenu_install command for you) Thanks. That's even better.
January 19, 201115 yr Looks like I won't be able to get to this testing as soon as I thought, as all my dev machines are either down or tied up in other projects.
January 20, 201115 yr You should throw a poll on this thread along the lines of testing/production/not upgrading etc just to see what people are doing. Peter
January 20, 201115 yr Author You should throw a poll on this thread along the lines of testing/production/not upgrading etc just to see what people are doing. Peter You can't add a poll to a thread, but anyone can create a poll thread. 4.7b1 has been the most solid "first beta release" I've seen, but I would still recommend users carefully review all test results (and scope of tests performed) before deciding to move their production array. The release is still quite new. I waited about 2 months to jump onto 5.0b2. But if you do move to 4.7b1, run it for a while before adding any sector 64 offset partitioned drives. Because once you do that, it will be hard to go back to the release version.
January 23, 201115 yr Amp1 Custom Script (new array test w/ EARS and F4) - Preclear the four disks with the new preclear script (use "-A" option). Carefully review all output to make sure drives are good.(Y) The third preclear cycle of the last of the four test drives finished yesterday afternoon. - Add two of them to the array (1 EARS as parity and 1 F4 as disk1) Done. - Set them at 4K aligned partition in unRAID (won't matter since preclear will already have done it, but good to set it correctly) Done. - Start array (parity computes), format disk Array started, data disk formatted, parity computed. - Confirm that drives are using sector 64 offset. You can use myMain (I just completed an update that displays the partition alignment), or use the comand fdisk -lu /dev/sdX, where sdX is the device name. The partitions start at sector 64: root@Unraid2:~# fdisk -lu /dev/sda Disk /dev/sda: 2000.3 GB, 2000398934016 bytes 1 heads, 63 sectors/track, 62016336 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 64 3907029167 1953514552 83 Linux Partition 1 does not end on cylinder boundary. root@Unraid2:~# fdisk -lu /dev/sdc Disk /dev/sdc: 2000.3 GB, 2000398934016 bytes 1 heads, 63 sectors/track, 62016336 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdc1 64 3907029167 1953514552 83 Linux Partition 1 does not end on cylinder boundary. root@Unraid2:~# - Add 2nd F4 to the array (it is precleared so should be quick). Format it. Confirm data is still on the existing F4 (i.e., format only formatted the new F4) Done. - Confirm alignment again. The partition starts at sector 64: root@Unraid2:/# fdisk -lu /dev/sdd Disk /dev/sdd: 2000.3 GB, 2000398934016 bytes 1 heads, 63 sectors/track, 62016336 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdd1 64 3907029167 1953514552 83 Linux Partition 1 does not end on cylinder boundary. root@Unraid2:/# - Copy a bunch of data and beat the living crap out of the F4s, verifying that they are trustworthy. (You might want to research situations that have caused data loss with the F4s and the old firmware, and try to do some of those things and make sure you 110% trust them with your most precious data.) I'm about to start on this test. I'll start by copying over a 24GB directory of TV shows to /dev/sdc while simultaneously running this in a shell: watch -n 1 "hdparm -I /dev/sdc | head" That sort of activity is reported to have triggered data corruption on the old F4 firmware. I'll calculacte md5sums for all files copied over and make sure nothing was changed in writing them to /dev/sdc. Then I'll do the same thing but copying a different set of files to drive /dev/sdd. I'll report on these tests later today.
January 23, 201115 yr I've successfully replaced my two F4's with two new EARS drives that were precleared with -A. Each took 26 hours. Preclear confirmed 64 alignment Then ran dd command and preclear -A on the two F4's (both already had firmware upgrade). Each took 27 hours. By the way love the new preclear format. Preclear confirmed 64 alignment Then used one F4 to upgrade a 500GB drive and rebuilt with no issues, and the other F4 was added to the last slot in the array (fully populated now) Writing to both F4's and EARS anywhere from 27-33 MB/s to protected array, user and disk shares from win 7 with teracopy Writing to cache drive at 65MB/s, very constant 20 data disks and one cache
January 23, 201115 yr Writing to both F4's and EARS anywhere from 27-33 MB/s to protected array, user and disk shares from win 7 with teracopy Writing to cache drive at 65MB/s, very constant 20 data disks and one cache Great feedback and very nice example of a big array with 4 of the 4k aligned disks successfully added. Can I assume Terracopy is confirming the tranfers? It appears you've got most of the A tests, the B3 test and the C1 and C3 tests covered here. Peter
January 23, 201115 yr yes confirmed with terracopy. Also all data on F4's is stored on a drive on my windows machine until I feel 100% comfortable with them. Really wish samsung would issue a new firmware
January 23, 201115 yr Great. Can you list the test points from the first page you are confident about so bjp999 can add you to the results and make it official. I do agree about the f4's, releasing a new firmware with the same number was a dumb move that doesn't inspire confidence at all. Peter
January 23, 201115 yr A1 A2 A3 A5 A6 B3 - with 64 aligned disk, not 63 C1 C3 D1 D2 D3 E1 Data disk only, never had a cache drive with 4.6. Speeds same to data disks
January 23, 201115 yr - Copy a bunch of data and beat the living crap out of the F4s, verifying that they are trustworthy. (You might want to research situations that have caused data loss with the F4s and the old firmware, and try to do some of those things and make sure you 110% trust them with your most precious data.) It won't be much longer before I'm at the above test. I'm running the new firmware on the Samsung F4s, they're aligned (partition starts at sector 64), what is on these F4s is copies of files that are on two other servers, and I'm perfectly willing to lose these third copies of files during testing. Could someone provide links to message threads about specific things to test previous problems on F4s?
January 23, 201115 yr I ran these tests on the F4's http://sourceforge.net/apps/trac/smartmontools/wiki/SamsungF4EGBadBlocks Was able to reproduce error before firmware updates and could not after updates as they did with their tests
January 23, 201115 yr I ran these tests on the F4's http://sourceforge.net/apps/trac/smartmontools/wiki/SamsungF4EGBadBlocks Was able to reproduce error before firmware updates and could not after updates as they did with their tests Josh -- Thanks for the pointer. It seems like if I run watch -n 1 "hdparm -I /dev/sdc | head" while I'm copying a few gigabytes of data to /dev/sdc, then if I'd been running with old F4 firmware I should be able to trigger the corrupted written data problem, but shouldn't be able to with the new F4 firmware. That's assuming that I'm testing while disk write cache is enabled and NCQ is enabled. For that drive, both write caching and NCQ are enabled, so I should be able to just run the hdparm command once a second on that drive while copying lots of data to the drive. Do I have that right? Here's the hdparm output for the F4 drive I'll be testing that shows write caching and NCQ are both on: root@Unraid2:~# hdparm -I /dev/sdc /dev/sdc: ATA device, with non-removable media Model Number: SAMSUNG HD204UI Serial Number: S2HGJ1BZ803377 Firmware Revision: 1AQ10001 Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6 Standards: Used: unknown (minor revision code 0x0028) Supported: 8 7 6 5 Likely used: 8 Configuration: Logical max current cylinders 16383 16383 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors: 16514064 LBA user addressable sectors: 268435455 LBA48 user addressable sectors: 3907029168 Logical Sector size: 512 bytes Physical Sector size: 512 bytes device size with M = 1024*1024: 1907729 MBytes device size with M = 1000*1000: 2000398 MBytes (2000 GB) cache/buffer size = unknown Form Factor: 3.5 inch Nominal Media Rotation Rate: 5400 Capabilities: LBA, IORDY(can be disabled) Queue depth: 32 Standby timer values: spec'd by Standard, no device specific minimum R/W multiple sector transfer: Max = 16 Current = ? Advanced power management level: disabled Recommended acoustic management value: 254, current value: 0 DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns Commands/features: Enabled Supported: * SMART feature set Security Mode feature set * Power Management feature set * Write cache * Look-ahead * Host Protected Area feature set * WRITE_BUFFER command * READ_BUFFER command * NOP cmd * DOWNLOAD_MICROCODE Advanced Power Management feature set Power-Up In Standby feature set * SET_FEATURES required to spinup after power up SET_MAX security extension Automatic Acoustic Management feature set * 48-bit Address feature set * Device Configuration Overlay feature set * Mandatory FLUSH_CACHE * FLUSH_CACHE_EXT * SMART error logging * SMART self-test * General Purpose Logging feature set * 64-bit World wide name * WRITE_UNCORRECTABLE_EXT command * {READ,WRITE}_DMA_EXT_GPL commands * Segmented DOWNLOAD_MICROCODE * Gen1 signaling speed (1.5Gb/s) * Gen2 signaling speed (3.0Gb/s) * Native Command Queueing (NCQ) * Phy event counters * NCQ priority information DMA Setup Auto-Activate optimization Device-initiated interface power management * Software settings preservation * SMART Command Transport (SCT) feature set * SCT Long Sector Access (AC1) * SCT LBA Segment Access (AC2) * SCT Error Recovery Control (AC3) * SCT Features Control (AC4) * SCT Data Tables (AC5) Security: Master password revision code = 65534 supported not enabled not locked not frozen not expired: security count supported: enhanced erase 330min for SECURITY ERASE UNIT. 330min for ENHANCED SECURITY ERASE UNIT. Logical Unit WWN Device Identifier: 50024e9003e49b88 NAA : 5 IEEE OUI : 0024e9 Unique ID : 003e49b88 Checksum: correct root@Unraid2:~#
January 23, 201115 yr I assume that will work as long as you verify files after your copy. That's how i've been doing it for the last month with zero errors. But before that I did the dd command and badblocks command from that site. You can't do that test if the drive is in the array
January 23, 201115 yr OK, when I do the test copies I'll get on md5sum for each file on the source system (a Mac) and the target (Unraid) and make sure the sums are identical. The drives are in an array, which is why I was looking for a different way to test other than just blasting a bunch of /dev/zero to the raw disk. Thanks for your help with the testing.
January 24, 201115 yr Author Thanks amp1, Josh, and rest of testers and contributors to this testing effort! I have updated the OP with the results posted (please help me double check that I didn't miss anything). We have a successful test showing that the F4s are working well with 4.7b1! Remaining tests that I'd like to see ... 1 - NFS Test 2 - Remove jumper from EARS test (I think there may have been some people doing this, but not in this thread. Please post a link and I will mark that test as completed). I think people are having trouble with the drive hanging when trying this, needing to zero the MBR. Keep posting progress and I'll keep updating (although I won't be adding any more people for A1, A2, A3, A5, A6, A7, D1, D2, D3. Those are completed with multiple positives. If anyone has any problems with any of those tests, I will post failed situations.) Thanks everyone!
January 24, 201115 yr Amp1 Custom Script (new array test w/ EARS and F4) - Copy a bunch of data and beat the living crap out of the F4s, verifying that they are trustworthy. (You might want to research situations that have caused data loss with the F4s and the old firmware, and try to do some of those things and make sure you 110% trust them with your most precious data.) Done. Repeated copies from another server to F4s worked fine. Repeated copies between the two F4s on the test system also worked fine. Comparing md5sums for copied files on source server and test server shows no changes in file contents when writing to F4s. I ran watch -n 1 "hdparm -I /dev/sdc | head" during some copies to /dev/sdc and the equivalent hdparm command for /dev/sdd while copying to /dev/sdd in an attempt to reproduce the corruption that has been reported with old F4 firmware but md5sums of copied files showed no corruption. I'm not sure how else to trigger corruption short of pulling power or SATA data cables during writes (I didn't try that). - Run parity check. Make sure no parity errors. Done. No parity errors were found. - Stop array, go to devices page and unassign one of the F4s, start the array. Verify you can access the missing disk (unRAID will be simulating it). Done. Files on the unassigned/missing disk were still available. - Stop array, go to devices page, and assign the remaining EARS to the removed F4s slot. Set drive to sector 64 partition alignment. Start the array. Drive should rebuild onto the new EARS. Verify EARS restores properly. Done -- EARS drive assigned in place of "missing" F4 for disk2. Array started. Tested that data for the missing drive was still available to clients while disk2 was being rebuilt onto the new EARS drive (it was). Unraid successfully rebuilt disk2 onto the new EARS drive. - Confirm alignment. Alignment is correct: root@Unraid2:/var/log# fdisk -lu /dev/sdb Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes 1 heads, 63 sectors/track, 62016336 cylinders, total 3907029168 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 64 3907029167 1953514552 83 Linux Partition 1 does not end on cylinder boundary. root@Unraid2:/var/log# - Run speed tests of F4 compared to EARS Unless someone has specific suggestions on how I can compare speeds in more meaningful way than recording and comparing the preclear times (see http://lime-technology.com/forum/index.php?topic=10190.msg97771#msg97771 where the F4s took 54.4% longer to preclear compared to the EARSs), I'm going to move on to the remaining tests. I've got no PC to do speed testing with, just the test 4.7b1 Unraid server, another 4.6 sever, and several Macs.
January 24, 201115 yr Amp1 Custom Script (new array test w/ EARS and F4) - Run speed tests of F4 compared to EARS Later today I'll probably do the speed testing. I've already clocked the time it takes to preclear two different EARS and two different F4s (see earlier message in this thread for preclear times). Any suggestions on how to measure performance of an F4 drive and and EARS drive? What I have available to install measurement tools on and run performance tests from is the Unraid 4.7b1 test server itself, an Unraid 4.6 production server, several Macs, one of which is connected via a gigabit Ethernet switch to test server, and other Macs connected via 100MB Ethernet and 802.11n wireless.
January 25, 201115 yr Author I split off the discussion about 4.7 and Samsung drives into its own thread. See 4.7 Discussion (Split from Testing Thread)
January 25, 201115 yr With 4.7 final having been released, all but one remaining test on the amp1 custom list being repeats of earlier tests, and the last unique test being to compare speeds of EARSs and F4s, I'm tempted to drop out of the testing. But if someone can suggest a useful way for me to systematically compare speeds of EARSs and F4s, I'll give that a go.
January 25, 201115 yr Author Maybe someone will have an idea for something better, but you could just copy a collection of file (say 10G worth) from your workstation to one and then to the other. You can just time it with a stopwatch or even a wrist watch. You can then compute MB/sec and compare the 2 results. The results won't be comparable if one disk is almost empty and the other almost full because disks are faster writing to the outer (lower) sectors than to the inner (higher) sectors. But if the two disks are recently added, without extensive data copied to either, the results should be comparable.
January 25, 201115 yr Author I want to thank everyone who contributed to the testing effort of 4.7b1. Now that it has been offically released as 4.7, I am going to declare this testing cycle complete. Together you guys tested almost all of the test cases, and 4.7b1 passed with flying colors. You have shown that the F4s, with the firmware update, are reliable with unRAID. The efforts here helped prove 4.7b1 was safe and reliable, and gave users confidence to install it. I think it contributed to the quick promotion to an official release. The only anomally (so far ) that was found in 4.7, was on a disk that contained an HPA (which was not one of the things we attempted to test for) will get corrupted if sector 64 offsets are used. Lime-Tech has said that HPA disks are not supported, so it is not being treated as a defect. Anyone reading this, especially those with a Gigabyte motherboard (famous for the HPAs), should be very careful that HPA is disabled and no disks added to their arrays have an HPA. (see THIS POST) So what do you think - should we try something similar with 5.0b3?
January 25, 201115 yr I will be of more help for 5.0 beta 3. I have 5.0b2 running everywhere and did not want to "downgrade" to check things. I also have a few Mac's in the house so should be able to test AFP related stuff pretty easily.
Archived
This topic is now archived and is closed to further replies.