jademonkee

Hello again johnnie.black Since we last talked, I've added the Samsung drive to my array, and am checking parity for the first time today. As you predicted, the parity check is chugging along at a woeful 25MB/s - no doubt due to the Samsung drive (the last time I checked parity it completed with an average speed of ~152MB/s - although this was before I enabled data caching in BIOS). Sitting next to my server is a Dell Perc H310 controller that I will soon install and flash to IT mode. It was going to be for my SSD cache (currently only 1x disk, but was hoping to expand to 3x), but I would consider moving the HDD cage to that card if it would fix this problem. Only, I don't know if the card in IT mode is just the same as the motherboard 'card' in AHCI mode, and therefore will cause the same slow parity builds? Any idea? Thanks for your help.

FWIW, the quick comparison run by WinMerge shows all the files as being identical, so it seems that there have been no corruptions. Further, I can't find any files that have different time stamps, so I have NO IDEA why SyncToy insists on syncing extra files. I shall try running FreeFile Sync now and see if it also thinks that files need to be synced. Thanks itimpi for the tip. EDIT: FreeFileSync works a treat! Looks like a great (and fast) program. Also handy that it runs from the command line, and on Mac. Thanks again, itimpi!

Thanks, I'll look into that. I'm going to wait for WinMerge's file comparison to complete first, though.

An excellent idea. Just purchased it. Thanks.

I ran consld8 to consolidate my music directory, and now SyncToy thinks heaps of files have changed that shouldn't have. Can you confirm that the time modified stamp doesn't change when using consld8? I've posted my problem - in excruciating detail - here: https://lime-technology.com/forum/index.php?topic=50458.0 Your help is appreciated.

Hi there. Make yourself comfortable, this is a long and curious tale... I recently bought an HP MicroServer gen8 and have loaded it up with unRAID Basic. I used to use an ARM-based Cubietruck (with 2TB internal HDD) running Armbian (based on Debian) to store my music and control playback around the house using Logitech Media Server (LMS). I also have my music stored on my Windows 7 Pro PC, where I manage its metadata. I would use SyncToy to sync the music from my PC to the Cubietruck whenever I bought a new album. I thought the same thing would work with unRAID, however SyncToy insists on syncing albums that haven't been modified - even for 5 or so years. Some background: I set up my unRaid with 1x 4TB Parity disk and 1x 4TB storage disk, then copied all my movies and TV shows over from my PC to an unRAID user share ('video') using robocopy. I then removed the 2TB HDD from my PC (which used to contain my movies etc), pre-cleared it, and added it to unRAID. I then used SyncToy to copy the entire contents (ie from the root of the drive) of my 2TB 'audio' HDD over to another unRAID user share ('audio' - about 1.4TB of data). I then installed a 500GB SSD cache drive, and set up Plex and LMS Dockers, storing the Docker image on the root of the cache drive, and creating a special cache only 'appdata' share for the configs. However, I noticed that when I used LMS that both the 4TB (disk1) and the 2TB (disk3 - there is no disk2, as I'm keeping that bay free for a future upgrade) disk spun up, as my albums were spread across both, due to my disk allocation settings (high water, automatically split any directory as required). So I used consld8 from the diskmv utilities to move all of the music I access using LMS onto disk 1. Gigabytes of tunes poured over to disk1. I also arranged my other miscellaneous audio bits n pieces (things that I don't access through LMS) onto disk3 (my 2TB HDD) to free up some room on disk1. It was about then that I realised how much musical junk I had stored, so I went through the HDD on my PC and decided to clean out a heap of the music I no longer listen to, and ran SyncToy in Synchronisation mode to propagate those changes and deletions to unRAID. After SyncToy was running for a while, I noticed that it was replacing a heap of music on my PC's HDD - even though none of those files had been edited. I freaked out and hit stop. I spent most of the day yesterday trying to get the two to happily sync, and even ran WinMerge. What was curious about WinMerge (I set the folder Compare Method to 'Modified Date' as it would take more than a day to do a full contents comparison) is that it only saw the expected differences - ie those files I had edited since I last synced, not the ones SyncToy wanted to move and which hadn't been touched for years. So I manually aligned the two folders by deleting anything I had deleted in my clean up, and copying over anything new. A manual sync, in effect (WinMerge made this surprisingly easy, as I could compare the folders between the two locations). After this manual sync, unRAID had once again spread the files across multiple disks (I neglected to change the share settings), so I ran consld8 again, and then created a new user share ('mymusic') for the albums that LMS uses, forcing it to only use disk1. I then used midnight commander (KB and monitor plugged directly into unRAID, logged in as root) to move the files between the two user shares (almost instantaneous). I then renamed the old 'audio' share to 'miscaudio'. I then ran SyncToy on the 'mymusic' user share, expecting it to say that the two folders were 'in sync' but it just started copying files over to the PC again. So I decided to run it in Echo mode (meaning that it would only copy from the PC to unRAID), hoping that it would just align the two and work from then on. It copied gigabytes and gigabytes of old audio files from my PC to unRAID (those that I think were moved from unRAID to my PC during the Sync I ran after cleaning up my music folders? It's hard to know - these aren't files that should have been modified in any way). When it finished, I thought I'd then run it in sync (ie two-way) mode again (to see if the two disks were now truly 'in sync'), and curiously, it started copying unchanged files back over from unRAID to my PC, just like it had that morning. I can't figure out what has changed on those files that makes SyncToy think that they've been modified, and I can't figure out what process caused that change. Most importantly, I can't figure out how to get SyncToy to work properly again. Worse, I fear that maybe the files have been damaged during some copy (consld8, maybe?), and that's why SyncToy thinks they've been modified. As such, I'm not too interested in running a sync from unRAID > PC, just in case they are somehow corrupt. I now only have one backup of these files around (a Mac-encrypted external HDD - stupidly, I ran a sync on my ARM-based server yesterday, propagating the modified PC files to it, too), and it's a few weeks out of date/currency. I'm currently running a Quick compare using WinMerge, to see if there really are differences between my unRAID copies and those on my PC, but it's going to take hours to run. Some possible causes for SyncToy to confuse unmodified files for modified ones: Consld8 changed the modified times of the files, causing SyncToy to think that they'd changed (even though the author said that all attributes are maintained). The Mover changed the modified times of the files, causing SyncToy to think they'd changed. Having SyncToy run on a User share, rather than a disk share (which I don't have turned on), causes something funny in the way SyncToy looks at files. Note that after fiishing copying any files to unRAID (whether the manual sync I did or a SyncToy run), I would always run The Mover before running another sync. I guess I'm posting here to find out if anyone else has had a similar problem with SyncToy, or can shed light onto why SyncToy thinks the files have changed, or can offer a way to get SyncToy to work again (I really need a reliable method to sync from my PC to unRAID - so I'm open to alternative syncing methods, too).

Oh wow. I'm really glad you posted that, thanks.

Ok, so I cancelled the preclear around 75 hours in, and rebooted to BIOS to enable write cache. The preclear speed is now up at ~74MB/s I know it's naughty doing this without a UPS, so I shall buy one very soon. If I don't use it to power my main PC, I can get away with a £65 one (CyberPower VALUE 600EILCD, which I believe is supported natively in unRAID).

Good to know, thanks. FWIW, I only periodically open the GUI to check on the progress.

Hrmmm just found this: https://lime-technology.com/forum/index.php?topic=24992.0 Thoughts on if this applies to me? I'm 72 hours in to my preclear, while 7% (@ 5 MB/s) through the Post-Read stage, so I'm unsure if I should cancel it now to check, or wait the next three or so days for this step to finish - I odn't want to have to start the process from scratch again... Do I wait? Do I start again?

I thought I'd run my first parity check since building the array, and even while preclearing an old 2TB disk (which is also progressing very slowly: see https://lime-technology.com/forum/index.php?topic=50293.0), I'm getting an estimated speed of 195.5 MB/sec. Not too shabby.

I initially intended to run the preclear three times, as per the recommendation, however given it takes this long... maybe just twice. It's now up to the post-read stage, and it's doing even worse: Preclear in progress... Post-Read (1 of 3). 3% @ 5 MB/s (67:24:28) Curiously, in the preclear log it says it's reading at 10.5MB Seems you're right: 5 Reallocated sector count 0x0033 252 252 010 Pre-fail Always Never 0

Pending sectors are bad news. I suggest you do a long SMART self test on it and post your results. Will do mate. It's now zeroing @ 97 MB/s (58:50:42), so will let it finish before I do. Notably, the SMART error has now cleared itself: 2016-07-11 23:24 unRAID device sdd SMART message [197] Notice [PERCY] - current pending sector returned to normal value SAMSUNG_HD203WI_S1UYJDWZ609551 (sdd) normal And from the SMART info screen: 197 Current pending sector 0x0032 252 100 000 Old age Always Never 0

And here are my anonymised diagnostics if they're of help, too. percy-diagnostics-20160711-1459.zip

Hi there, I removed a 2TB drive (SAMSUNG_HD203WI_S1UYJDWZ609551) from my desktop PC over the weekend, and stuck it in my (new) unRAID box (HP MicroServer gen8 w/ a Celeron G1610T @ 2.30GHz and 16GB ECC RAM). The disk is a few years old. I started the preclear from the GUI before I went out on Saturday, and it wasn't until today that I saw most speeds for preclear were up near or above 100MB/s, while mine has been crawling along at about 10MB/s since I started the process. It's currently at 88% Pre-read (after 49h:35m), so I think I'll let it finish, but does anyone have any idea why this is so much slower? I've attached my syslog. Also, FWIW, I also had slow speeds when building the parity disk, which I posted about here: https://lime-technology.com/forum/index.php?topic=50194.msg481766 I have a 4TB Seagate NAS as parity (ST4000VN000-2AH166_WDH00MXQ sdb), and another identical disk as storage (ST4000VN0002AH166_WDH00GP7 - sdc). I didn't use the preclear script/GUI to preclear these disks, so can't compare how fast they were. Your help and insight is appreciated, thanks. percy-syslog-20160711-1436.zip

Thanks for your help. I'm reckon it was just some weird thing going on with the Seagate NAS Disk firmware during the intial parity. Write speeds are fine now. I have an SSD coming in the mail

What should I look for? Is AHCI the only setting that should be changed from default? EDIT: Have booted to BIOS. System Options > SATA Controller Options Embedded SATA Configuration: Enable SATA AHCI Support Drive Write Cache: Disabled

Was there ever a solution to this? Or did you have to return the NAS drives? Also, I'm not sure what to search for in the logs? I searched for "ide" but found no instances. I'm not sure that's the problem, anyway, as I ran diskspeed and got the following results: /dev/sdb (Disk 2): 173 MB/sec avg /dev/sdc (Disk 1): 159 MB/sec avg I'm currently copying across the network at about 80MB/s (large files).

Holy hell. I was considering using Plex for music (for remote streaming; I use Logitech Media Server for music around the house), but I can't imagine the size of the library with ~35k songs. Perhaps I shall opt for the 500GB cache drive, then. Good tip. If this is a local cache of internet files, then I should be ok as I have a good, fast, reliable internet connection here.

Thanks so much for the info! I thought that 120GB seemed rather excessive for a Plex install... Looking at the price difference between the 750 and the 850, and the fact that the 850 comes with a 5 year warranty, I reckon I'll go the 850 EVO. Will take a look at my budget and decide if I can afford the extra £££ for the 500GB model or not and buy accordingly.

Sorry to butt in here and not start a new thread, but I'm also considering a cache drive for my new unRAID as a place to store my future Dockers (Plex, LMS, sickbeard, sabnzbd, transmission, maybe more) [note that I only have 2x drives (1x storage, 1x parity) in there at the moment, so speed isn't yet a problem]. I'm leaning toward a Samsung 750 EVO or Intel 535 (just on brand strength alone - I know nothing about the drive - any other recommendations are appreciated) but am unsure of what size is best - 240GB or 500GB. It's unlikely I'll ever transfer close to even 100GB in a single day, but I'm concerned about how much space the Dockers will take up (and VMs, should I go down that path later). How much space should I expect? I saw on the Plex forums that someone mentioned that Plex ate through their 120GB cache drive pretty quickly, and don't want that to happen to me. See: https://forums.plex.tv/discussion/comment/855869/#Comment_855869 Note that in a few months' time (perhaps as much as 6 or more), I will likely be replacing my desktop PC, so will have 2x 120GB SSDs available to add to the cache, too, but I will need to purchase a mounting bracket (US$50 after shipping http://homeservershow.com/hp-microserver-gen8-drive-bracket.html) and a PCI-E > SATA card to attach them to my server, so may never end up doing it. Also note that my server is still being set up - I'm yet to install any apps/dockers/VMs on it at all (thus my confusion on how much space I should allocate to it). Thanks all for your help.

Thanks for the info, but I've had the browser closed for the majority of the time, and it doesn't look to change the speed at all. I think it might be something in the Seagate NAS drive's firmware. To quote the marketing material: "supports error correction via customised error recovery controls" so there might be something going on there. There's still 16 hours left for the parity build, but I shall run some speed tests once it's done and report back.

In BIOS, you mean? I have set the card to AHCI (Press F9 during boot -> System Options -> SATA Controller Options -> Embedded SATA Configuration -> Enable SATA AHCI Support) Or did you mean in an unRAID setting (or somewhere else entirely)?

Thanks again for the help and the info. The friend who recommended me unRAID warned me of the slow speeds, so I'm expecting them. What I didn't know was that it's faster with just the 1+1 disks. I was looking to add two SSDs for cache some time in the near future, but it's good to know that I won't have to worry about that until I add in another storage disk. Thanks again.

Ok, just ran the test and got the following results: /dev/sdb (Disk 2): 173 MB/sec avg /dev/sdc (Disk 1): 159 MB/sec avg sdb is the previous parity drive. FWIW, I just copied a 400 MB collection of lossless audio files over to sdc (containing my empty media share) at about 95-100MB/s. Any idea what else may be causing the slow speeds? I shall try rebuilding the parity now, and report back. EDIT: Same speeds building the parity as before I guess I'll just have to wait the day until it completes, then test to see what write speeds are. Maybe there's something funny going on with the controller causing slow speeds when reading/writing at the same time. I dunno. I just hope it doesn't mean that writes over the network will be 35MB/s...