funksta88

I saw on Midnight Commander how a lot of the files being transferred that were slow happened to be in the .apple* folders. I ran this find . -name \.AppleDouble -exec rm -rf {} \; and it deleted most of the problem files. Then I copied each application folder one by one. Probably averaged 45MBps overall because of the small files clogging up the system. Everything has been copied over and the drives have been successfully swapped over. Now deleting everything off the old WD Green cache drive to add it to the array.

I have updated it to RC15a and tried copying again. On certain files it transfers around 75MBps and on others it can be 300KBps. I think it has something to do with how midnight commander handles small files. The larger files seem to transfer easily whilst the small files appear to bog down the system.

Here is the hdparam of the relevant drives. Input is hdparm -Tt /dev/sd(letter of drive) Cache WD Green /dev/sda: Timing cached reads: 23836 MB in 1.99 seconds = 11980.64 MB/sec Timing buffered disk reads: 344 MB in 3.00 seconds = 114.54 MB/sec Future cache (not in array but mounted) WD Red /dev/sdk: Timing cached reads: 23300 MB in 1.99 seconds = 11680.86 MB/sec Timing buffered disk reads: 420 MB in 3.01 seconds = 139.56 MB/sec Drive currently in array also used to check speed WD Red /dev/sdj: Timing cached reads: 21662 MB in 2.00 seconds = 10857.32 MB/sec Timing buffered disk reads: 456 MB in 3.00 seconds = 151.92 MB/sec

So I bought a WD Red 3TB to replace an older WD Green 3TB as the cache drive. I read up on how to transfer from cache drive to a precleared drive for an easy swap out. I have programs running on the cache drive and would like them to move to the new drive including sabnzbd, sickbeard, couchpotato, dropbox, etc. I tried using midnight commander as I thought it would be the easiest option but for some reason it is very slow to copy. Max transfer I saw was 2.2MBps and It constantly goes down to the mid KBps range. I want to copy about 1.4TB worth of data but at these speeds it seems not worth waiting for. I tried copying the relevant folders directly to the new drive (drive has been precleared and has reiserfs) and also to an existing drive in the array but the same slow speeds are there. Even tried mounting them on a computer with ubuntu for a direct copy. The permissions in ubuntu won't allow the copy so I don't want to mess around with that. I have also copied about 700GB worth of data over the network in a few hours to an external portable drive and the speeds there were fine (almost 90MBps) Is there anything I can do to speed it up because obviously this is not normal to have such a slow movement of data within the server itself? syslog-2013-06-26.txt.zip

Found out it was an unmenu script for making a swap file on the cache drive that was keeping it busy. Unmounted the cache drive properly and started the reiser check. This is what has happened so far. (will update this post as it continues) Zero bit found in on-disk bitmap after the last valid bit. Checking internal tree.. finished Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs. Checking Semantic tree: finished 2 found corruptions can be fixed when running with --fix-fixable Input this reiserfsck --fix-fixable /dev/sda1 Output Replaying journal: Done. Reiserfs journal '/dev/sda1' in blocks [18..8211]: 0 transactions replayed Zero bit found in on-disk bitmap after the last valid bit. Fixed. Checking internal tree.. finished Comparing bitmaps..vpf-10630: The on-disk and the correct bitmaps differs. Will be fixed later. Checking Semantic tree: finished No corruptions found There are on the filesystem: Leaves 543058 Internal nodes 3709 Directories 485772 Other files 830449 Data block pointers 349248906 (16535493 of them are zero) Safe links 0 ########### Started the array and everything has returned to normal. Adding the new disks now.

Running version 5.0-rc12a Tried to run this command reiserfsck --check /dev/sda1 (Yes) and then this shows up Partition /dev/sda1 is mounted with write permissions, cannot check it Just did this as well dd if=/dev/sda1 count=195 | od -c -A d | sed 30q 195+0 records in 195+0 records out 99840 bytes (100 kB) copied, 0.000464767 s, 215 MB/s 0000000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 0065536 i 024 252 + J 201 314 027 6 X 261 \r 022 \0 \0 \0 0065552 \0 \0 \0 \0 \0 \0 \0 \0 004 \0 \0 316 212 , 0 0065568 204 003 \0 \0 036 \0 \0 \0 \0 \0 \0 \0 \0 020 314 003 0065584 214 002 002 \0 R e I s E r 2 F s \0 \0 \0 0065600 003 \0 \0 \0 005 \0 U W 002 \0 \0 \0 240 G 306 001 0065616 001 \0 \0 \0 305 374 ^ 312 z - H 352 204 211 2 4 0065632 337 031 } 201 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0065648 \0 \0 \0 \0 L \0 036 \0 : 204 325 O \0 N 355 \0 0065664 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 0065728 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 001 \0 \0 \0 0065744 Q 245 # \0 U 245 # \0 _ 245 # \0 ` 245 # \0 0065760 | 245 # \0 } 245 # \0 220 245 # \0 222 245 # \0 0065776 224 245 # \0 227 245 # \0 230 245 # \0 232 245 # \0 0065792 233 245 # \0 235 245 # \0 375 245 # \0 376 245 # \0 0065808 024 246 # \0 025 246 # \0 026 246 # \0 030 246 # \0 0065824 034 246 # \0 036 246 # \0 246 # \0 ! 246 # \0 0065840 " 246 # \0 # 246 # \0 % 246 # \0 ' 246 # \0 0065856 ) 246 # \0 + 246 # \0 . 246 # \0 / 246 # \0 0065872 R 246 # \0 T 246 # \0 c 246 # \0 g 246 # \0 0065888 336 246 # \0 337 246 # \0 343 246 # \0 345 246 # \0 0065904 y 257 # \0 z 257 # \0 300 262 # \0 301 262 # \0 0065920 325 262 # \0 326 262 # \0 341 262 # \0 342 262 # \0 0065936 347 262 # \0 350 262 # \0 031 265 # \0 033 265 # \0 0065952 a 266 # \0 e 266 # \0 g 266 # \0 i 266 # \0 0065968 j 266 # \0 l 266 # \0 213 266 # \0 214 266 # \0 0065984 g 267 # \0 i 267 # \0 202 267 # \0 203 267 # \0 0066000 002 304 # \0 003 304 # \0 337 304 # \0 340 304 # \0

Adding 3 new hard drives to the array and Putting them in their slots within the web gui. For some reason I decided to change the cache drive slot to nothing and then put it back into cache drive position. I thought there wouldn't be any danger in that as nothing would change unless i selected to apply it. So I went to start the array using the web gui and discovered that now the cache drive was showing as unformatted with the new drives being added. It is still the same drive and it is still in the cache slot according to the web gui. Checked the cache drive settings and found it was showing this Partition format: GPT: 4K-aligned File system type: unknown When I check unmenu, i discover that it shows the cache drive as having a reiserfs filesystem. WDC_WD30EZRX-00_WD-WMAWZ0427163 37°C 3T /dev/sda partition (2,930,266,535 blocks): 3T /dev/sda1 /mnt/cache reiserfs Not sure where to go from here. I just want the array to recognise it as it was before as nothing should have changed.

Excellent, thanks for the reply. Typed it in and it displayed CPU0 CPU1 CPU2 CPU3 My chip doesn't support hyper threading so i guess all the cores are fired up. Another question is do you know what shfs is? It uses about 50% of my CPU and is run as root.

I wish I had of seen this earlier. Worked out that plex stopped working on average every 15 hours and finally found out about the hidden folders bug and cache only user shares. Everything is fine with the cache drive now.

I'm sorry but I am not the best with Linux and terminology but what do you mean log into boxes? Do you mean type that into the unraid server through telnet or is there a button or link I am not seeing?

I built a very nice unraid server a couple weeks ago and I am very happy with it after adding in the extra plugins like sabnzbd, couchpotato, sickbeard, Plex and simple features. The server itself is built with a bit of overkill in mind; Intel i5 3GHz quad core with 16Gb RAM. The only problem is that when I use simple features and go to utils, system profiler and check on the CPU info, everything looks fine apart from the amount of cores used on the processor. Version: Intel® Core i5-2320 CPU @ 3.00GHz Voltage: 1.0 V External Clock: 100 MHz Max Speed: 3800 MHz Current Speed: 3109 MHz Status: Populated, Enabled Upgrade: Other Serial Number: To Be Filled By O.E.M. Asset Tag: To Be Filled By O.E.M. Part Number: To Be Filled By O.E.M. Core Count: 4 Core Enabled: 1 Characteristics: 64-bit capable Im thinking that this led to my problems when trying to watch films over Ethernet using Plex Transcoding. The film plays really well in 720 and even 1080, but when sabnzbd has to unpack a rar or move a job on the cache drive the movie stops and starts similar to YouTube buffering. The only way to overcome this is to pause the film and wait until sabnzbd has finished using up resources then play the movie. I'm pretty sure that a quad core should be able to handle both actions at once because I used to do the same thing from my intel core 2 duo 2.4GHz with 8GB RAM running a full graphical OS and that worked fine. Even gone to BIOS and made sure everything is sorted ( chose super duper power user where the computer part picture was flaming because it was working so fast setting). Is this just an error showing the amount of cores used? Is there a way to tell unraid to use more cores if it only using one? Could you think of any other reason this is happening? Is there any other way to check amount of cores utilised in unraid? Thanks in advance.

It took me a while but I found out about using -r in the command and it went well. cp -r /mnt/user/Plex/.TEMP /mnt/cache/.TEMP

HAHA, so simple. Thanks mate you're a champ. Also wondering if it is possible to move the files from their current location in /mnt/user/Plex/.TEMP to /mnt/cache/.TEMP I tried to copy them using cp /mnt/user/Plex/.TEMP /mnt/cache/.TEMP but it says cp :omitting directory '/mnt/user/Plex/.TEMP' Is there anyway to copy over hidden folders?

So i'm fairly new to unRAID but i am loving it so far when it does actually work. I recently set up a cache drive to hold my sabnzbd, couchpotato, plex and sickbeard and i followed what the GAQ was telling me to do when i put a period in front of the folder to make the mover script skip miving files from the cache to the array. So now my folders are like this: /mnt/cache/Plex/.TEMP/(seperate folders for programs) I have it set up so that once the files are downloaded and extracted, the files are moved to user share: /mnt/user/Plex/Media/(type of media) and that works great. Thought i could sleep tight last night knowing i could wake up and have whatever files that were made moved to the array and have the programs work continuously as that was happening. That did not end up happening as when i tried to log into my tower it wouldn't access it. Had to push reset and after cancelling the programs from auto starting i managed to start the array and find that the many of the files behind the .TEMP had been moved??? Found them all sitting in the array and not in the cache which was not good at all. I am not sure what is going on at all or maybe this is a once off. I'm not even sure how to copy the folders and files back in place using commands so if i could at least do that it would be a start. Is there anything I am doing wrong? Should I edit the mover script? Where and how do I edit the script? How do I copy the folders that were moved in the array back to the cache? Thankyou in advance for helping. I am completely lost on this topic.

Ok, so far it looks all good then?

Hey everyone. After reading a lot about cord cutting and media servers over the past few months i have decided to try and find an all in one solution for my house. I have an idea in mind where i could start off with about 3TB of storage and eventually add more and more drives as needed until I hit about 38TB. Not entirely sure on the final figure but i guess it can be changed later down the track (a few years?). I want it to be able to run the usual sickbeard, sabnzbd, couchpotato, as well as plex so i can automatically get everything i want and have it available on my apple tv and ipad which are both running plex clients. Probably use green drives as i don't think i would benefit much from high speed 7200rpm drives. I'm hoping to keep it around $1000 AUD/USD (or less) So far I have been searching and Raj's Design seems the most simple to follow in terms of buying the Gear needed but i'm not sure whether the beast in question could do everything i needed it to do. Essentially, i need it to run the above programs but I don't know how you would configure such a thing or even if it is possible to do so. Do I log in with http like i do now to change settings? This is the build I was looking at: 20 Drive Beast (tower) Mobo: SUPERMICRO MBD-X8SIL-F-O CPU: Intel i3 540 Clarkdale (the i3 530 and i3 550 are fine too, choose based on price and availability) RAM: Kingston 4GB DDR3 1333 currently searching for replacement 4 GB RAM, but here's 8 GB RAM PSU: CORSAIR Enthusiast Series TX650 V2 650W Case: Antec 1200 Hot Swap Drive Bays: Norco SS-500 x 4 (The Norco SS-500 drive cages are currently unavailable in the US. I list two replacement options below. Both have their pros and cons. You can read my review of all three cages here.) Icy Dock MB455SPF-B x 4 (replacement option 1) Supermicro CSE-M35T-1B x 4 (replacement option 2) Fans: Coolink SWiF2-801 80mm x 4 *optional (Norco SS-500/Icy Dock MB455SPF-B fan upgrade if quieter performance is desired) GELID Solutions FN-PX09-20 92mm x 4 *optional (Supermicro CSE-M35T-1B fan upgrade if quieter performance is desired) SATA Expansion Cards: Supermicro AOC-SASLP-MV8 8-Port SAS/SATA Add-on Card x 2 Cables:0.5m 30AWG Internal Mini SAS 36pin (SFF-8087) Male w/ Latch to SATA 7pin Female (x4) Forward Breakout Cable x 4 No SATA Cables required (6 included with motherboard) Hard Drives: Green drives (5400 rpm or 5900 rpm), such as WD Green, Hitachi CoolSpin, Samsung EcoGreen, and Seagate LP Cost: $1150 (without the optional fans/fanplate) + shipping + cost of hard drives (price is obsolete due to Norco SS-500 drive cage and RAM unavailability) Capacity: 38 TB Expandability: None. I got it from this page about halfway down. I am aware that the RAM will change but that should be an easy swap when i search. edit: Changed the build to an updated design. If i didn't put enough information here, please ask and you shall receive.