EDIT:  solution was utilize FolderMatch by SaltyBrine's Directory Compare (\\Tower\Share vs \\Tower2\Share).  Note that you need to setup the compare options from FileName/Date to FileContents, which is a Binary check.

 

I just updated all the disks in tower1 from RFS to XFS.  The D525 was not powerful enough to run rsync -nrcvPX /mnt/disk2/ /mnt/disk1/ after rsync -avPX /mnt/disk2/ /mnt/disk1/.

 

All of these files are on Tower2, which has read access to Tower1.  On Tower2 I have the shares passed through to an ubuntu VM (Tower1_Doc and Tower2_Doc)  --- I would like to treat the Tower2 files as Master to confirm the Tower1 files are OK

 

Will the following command on Tower2 work?  Note that I have about 10 subdirectory levels under each share

 

for file in Tower2_Doc/*.*; do diff "$file" "Tower1_Doc/${file##*/}"; done

 

Thanks!

The D525 was not powerful enough to run rsync -nrcvPX /mnt/disk2/ /mnt/disk1/ after rsync -avPX /mnt/disk2/ /mnt/disk1/.

 

What error did rsync give you?

 

giving rsync the c flag does make it take a whole heck of a lot longer.

the server would use 100% cpu, and after a few hours hard lock.  The first disk I let sit overnight only to wake up to a server which had turned itself off.

 

As such, all disks have been moved to XFS, but the validation step was skipped

 

Out of curiosity ...

 

(a)  What version of UnRAID are you running on the D525?

(b)  How much memory does the D525 system have?

©  What plugins/Dockers are you running on that system?

 

A. 6.1.7

B 4 GB

C no dockers, no VM.  While rysnc, disabled plugins other than shell in a box/nerdpack.  Recycle bin, preclear disks, power down, cache dirs, open files, and hopefully soon file integrity.

And post conversion, ssd trim

Did you compare your CPU usage (or perhaps just the "feel" of the system's responsiveness) between v5 and v6 ??

 

I know v6 is notably more CPU intensive than v5, although the plugins you're using shouldn't cause any appreciable issues except the file integrity plugin, which is very CPU intensive.    I compute checksums on everything BEFORE I copy it to my server; and do integrity checks by simply right-clicking on the share I want to check and selecting "Verify Checksums".    The actual computations are done from within my Windows box (which has a i7-4790), so UnRAID isn't stressed at all.

 

Before we wander too far from the path1, back to the question at hand ->

Will the following command on Tower2 work?  Note that I have about 10 subdirectory levels under each share

 

for file in Tower2_Doc/*.*; do diff "$file" "Tower1_Doc/${file##*/}"; done

And to continue our meandering ->

I did not compare the CPU usage between 5 and 6.  I know that before I could have 3 streams and a parity check going, now more than 1 stream during a parity check causes the streams to … stutter/drop.  I can live with this for now.

 

I do know that write speed decreased by 15-20 MB/S with the move to 6.0.  The move to the SSD was prompted by prepping my HDD Cache Warm Spare for Dual Parity.  A little bummed that the write speed did not change.

 

The move from RFS to XFS was prompted by the desire to run the Dynamix File Integrity Plugin

 

 

 

I don't know about the command you asked about.

 

As for the loss of performance in v6 on lower-powered systems ... I'd already noticed that when I upgraded my media server [a Pentium E6300] to v6.  I DO like v6 a lot ... but clearly it's not a good fit for lower-end systems.

 

My D525 system is used purely as a NAS, and is exceptionally reliable running v5.0.6.  I had been considering giving v6 a try on this, but your experience has definitely convinced me my inclination to just leave well enough alone is the right approach for that system.    My E6300 scores 1706 on PassMark, a D525 scores 700 ... not a good match for the demands of v6.    I use it purely as a NAS ... Clean PowerDown, UPS support, and UnMenu are the only extras installed on it -- and all I need.    So there's no reason to move it to v6.

 

My desktop systems all have CPU's that score over 10,000 on PassMark ... the same will be true for all of my future UnRAID setups 

 

Think I would be better off just doing a copy (overwriting) from server2 to server1 using Window's TeraCopy (after setting up SMB permissions on server2) for a 1 time piece of mind as TeraCopy does a checksum of the 2 datasources post copy.  If so, best to disable the parity drive and cache for the moves (about 11 TB of data)?

 

I Love... well everything about 6... however to do it again I would leave  5.0.6/RFS with the same plugins you mentioned on my D525.  6 works great on my 2500K machine with VMs/Dockers/MANY Plugins/XFS/Etc

 

 

Copying with TeraCopy isn't a bad idea.    I wouldn't disable parity, however -- whether it takes 2 days or 4 days to copy your 11TB likely doesn't really matter  (once you start the copy, it's all "computer time" -- not "your time") and I'd think it best to have the files fault tolerant immediately after they're written.

 

I agree with your assessment of v6 -- I like it a lot; but it's not a good match for a D525-based system.  [i would have almost certainly tried it on mine if it wasn't for your experience ... but won't be doing so now.]

 

BTW, if you haven't actually copied your data back to the D525 server, you COULD still re-do it with 5.0.6 ... you'd have to reformat all the disks as RFS, but that's a very quick process (as you know).    I don't know just how much different the "feel" is vis-à-vis performance, but it may be worth doing that if it's sluggish enough.

 

 

I will set the shares to not use the cache drive for the Copy to overwrite all existing files for now... given I am only getting 40 mb/s even with the cache drive no reason to burden the SSD

 

My wife and daughter are growing frustrated over not having file save access and not being able to watch their shows (my wife is expecting delivery of our second child SOON)... so I would like to limit downtime of the D525 server... and while I do not trust for certain that the RFS to XFS conversion left all the files untouched I do not want to lose the copy and rely on my backup server having the only copy of the data.

 

In the past my only gripe with the D525 was lack of ECC... I am considering swapping the D525 for something a bit more robust and combining my gaming rig into this (not in a big rush).  I am used to a i7-3770K with an AMD 290 and an optical drive...

 

The SUPERMICRO MBD-X10SRA-O would be PERFECT if only it had IMPI.  http://www.newegg.com/Product/Product.aspx?Item=N82E16813182958

2 of these: MEM-DR416L-SL01-ER21

Then again... this is a little painful: E5-1650 v3

 

 

The SUPERMICRO MBD-X10SRA-O would be PERFECT if only it had IMPI. 

 

This this must be perfection 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

 

Then again... this is a little painful: E5-1650 v3

 

Interesting ... I've been looking at the same board/CPU combo [except the IPMI version I just linked to].    I've learned many times over the years that spending a bit more to get what you really want isn't a bad idea ... the extra $$ are quickly forgotten, and you won't be "wishing" you had those extra cores, higher performance, buffered RAM, etc.

 

I am used to a i7-3770K with an AMD 290 and an optical drive...

 

I'm not a gamer, so I don't have a high-end graphics card, but both of my HTPCs, my wife's PC, and my primary desktop all have i7-4790's ... I definitely don't like to skimp on CPU performance 

 

My UnRAID D525 system was built as a pure NAS, and it does a superb job of that ... but UnRAID has now evolved to the point where low-performance CPU's are not appropriate.    Anything I build in the future will not have any such problem      I've been looking very seriously at building a system with the E5-1650v3 that you mentioned ... pop in 32GB of RAM and it'd be a VERY nice virtualization box with PLENTY of "horsepower" => and with buffered RAM it could easily be bumped up to 64GB, 128GB, or even more with no concerns about memory bus loading.

 

 

 

This this must be perfection 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

 

Wow I never knew they made a 10port Sata board. Now you got my attention. lol

 

This this must be perfection 

http://www.newegg.com/Product/Product.aspx?Item=N82E16813182959

 

 

Wow I never knew they made a 10port Sata board. Now you got my attention. lol

X10SL7-F-O. 14 SATA ports, IPMI, etc. Not as good for virtualization as the e5 board referenced above though.

GREAT catch on the IPMI board Gary!  I FULLY agree with you on all points just above

I know if I do move forward with this and do not go for the 6 core 12 thread cpu I will regret it.

 

I BELIEVE the chipset supports 10 SATA3 native!

 

Of course… in a few months we will have a Linus video showing off an Octet of Pascals wrangled into a Skylake 8 thread / 16 core machine with 256 GB of ECC

 

I am considering just swamping in the board/cpu/ram for now... think my current PSU is sufficient?  It works fine for the i5-2500 (which has 5 7200 RPM HGST 4 TB Drives)

 

E5?1650V3 22nm 6/12 3.8 GHz 140W

i5-2500K 32nm 4/4 3.7 GHz 95W

 

SeaSonic X Series X650 Gold

5x 4 TB Seagate ST4000DM (5400rpm)

1x 850 Pro SSD

 

... what about after adding in a GPU: AMD 290?

I think the X650 would be plenty of power for the E5-1650v3, even with your Radeon 290.  It's a superb power supply, and you're only boosting the max power up by 45w ... I'm sure you're not coming that close to the PSU's limits now.

 

 

thanks for the sanity check

 

I mean when spending $1K on a MOBO, CPU, and ECC... why drop $100 on a new PSU, right

thanks for the sanity check

 

I mean when spending $1K on a MOBO, CPU, and ECC... why drop $100 on a new PSU, right

 

:)

 

A high quality PSU like the Seasonic X series would likely be over $100 ... but regardless of the cost, I agree there's no reason to replace the PSU when you already have a very high quality unit that will work fine.

This D525 and trying to verify file integrity is driving me nutso...  I REALLY should have kept this on v5 until the hardware upgrade...

 

Now I am attempting to run squids' checksum suite on both Tower1/ShareA and Tower2/ShareA https://lime-technology.com/forum/index.php?topic=43396.0

 

Tower2 is a i5-2500 with a mechanical Cache.  The Checksum Suite is writing the .HASH files to the cache at ~ 115 MB/s and~30% CPU

Tower1 is an Atom with a SSD Cache (850 P).  The Checksum Suite is writing the .HASH files to the cache at ~ 20 MB/s and 75% CPU

 

Once Done I will move the Hash files and folder structures to Windows with:

ROBOCOPY \\Tower2\ShareA C:\Compare\Tower2\ShareA *.hash /E

ROBOCOPY \\Tower1\ShareA C:\Compare\Tower1\ShareA *.hash /E

.... Then compare with Corz/Checksum to look for any missing, extra, or different checksums...

 

 

how I got to this point:

Apparently Copying Large amounts of files from Tower2 to Tower1 ends up failing without a Cache Drive (100% CPU utilization causes the D525 to lock up after about 10 hours).... as the original plan was to just copy all of Tower2 to Tower1 using TeraCopy and over-writing the files... then relying upon TeraCopies lovely CRC check.  The same holds true for robocopy -is...

 

...

 

EDIT:  solution was to remove the .hash files using a Robocopy command and utilize FolderMatch by SaltyBrine's Directory Compare (\\Tower\Share vs \\Tower2\Share).  Note that you need to setup the compare options from FileName/Date to FileContents, which is a Binary check.

 

$600 E5-1650V3 22nm 6/12 3.8 GHz 140W Haswell-EP/4th Gen

$335 SUPERMICRO MBD-X10SRA-F-O (with IPMI)

$140 MEM-DR480L-SL01-ER21 (16 GB DDR4-2133 ECC Registered)

$140 MEM-DR480L-SL01-ER21 (16 GB DDR4-2133 ECC Registered)

$1,200 Total

 

Is 2 sticks of RAM ok or do I NEED 4?

What heatsink?  Will any LGA2011 work? 

Not being lazy... just not certain if the server class setup has restrictions/considerations that a desktop class system would not...

 

***

 

Or wait for 2016 Q2 rumored release of Broadwell-EP/5th Gen E5?

https://linustechtips.com/main/topic/462578-xeon-e5-broadwell-and-purley-platform-base-xeon-e5-skylakecannonlake-38-hedt/

http://www.itworld.com/article/2985214/hardware/intels-xeon-roadmap-for-2016-leaks.html

 

From what I am reading it appears we will go from 6->8 cores and a much lower TDP at the same price point.  Ram ups from DDR4-2133 to DDR4-2400 (no noticeable difference here).  It also appears that existing C610 motherboard should be fine

 

***

 

Skylake/6th Gen E5 Xeons look to be out in Mid 2017... while a major overhaul I am not certain I WANT to wait this long

 

That is a SUPERB system => if I was ordering parts for a new system today it's exactly what I'd buy.

 

I think 32GB is fine ... I'd at least start with that.    It's true that if you buy 4 modules the RAM will run in quad channel mode, but with 2 it will still run in dual channel mode.  Not likely you'll even notice the difference.

 

Any good 2011 heatsink will work.

 

Thanks Gary