Hi Folks,

 

I am currently performing a major overhaul of my system(s) and after all backups

were fine, I decided to test all used and new drives before importing the backups again.

 

Now, I have 16 drives (8x 2TB + 8x 3TB, mixed models) that need a preclear but I can only manage

to start the script on 4 drives at a time.

 

Whenever I run more preclear processes, everything gets very slow and I see someting like this

flooding the syslog:

Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: attempting task abort! scmd(ec41de40)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 0c 08 a5 70 00 01 00 00
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: handle(0x000a), sas_address(0x5001e67544b55fe0), phy(0)
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: enclosure_logical_id(0x5001e67544b55fff), slot(4)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: task abort: SUCCESS scmd(ec41de40)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: attempting task abort! scmd(ea09b540)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 0c 08 a6 70 00 01 00 00
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: handle(0x000a), sas_address(0x5001e67544b55fe0), phy(0)
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: enclosure_logical_id(0x5001e67544b55fff), slot(4)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: task abort: SUCCESS scmd(ea09b540)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: attempting task abort! scmd(d9a38900)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: [sdo] CDB: cdb[0]=0x28: 28 00 0d 0a ed 20 00 01 00 00
Dec 30 20:15:57 Tower kernel: scsi target0:0:14: handle(0x0018), sas_address(0x5001e67544b55fee), phy(14)
Dec 30 20:15:57 Tower kernel: scsi target0:0:14: enclosure_logical_id(0x5001e67544b55fff), slot(14)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: task abort: SUCCESS scmd(d9a38900)

 

Here is my setup:

 

-I have a new 24 bay case, technically identical to a NORCO 4224, in which I have

currently connected/used four of the backplanes for my drives.

 

- unRAID 5.0-rc5-r8168

- preclear_disk.sh v1.13

- Mobo: X8SIL-F with XEON L3426 and 16GB ECC-R

- HBA: M1015 flashed to IT with latest P15 firmware

- Intel SAS Expander RES2CV240

- PSU is a 600W SuperFlower 80plus-Platinum - single 12V rail good for 48A

- all multilane cables are new

 

The mobo, HBA, 8x2TB disks  and PSU are receycled from my old case.

Expander, Multilane cables and 8x 3TB disks are new.

 

I am pulling my hair out...

The problem seems not to be connected to a specific backplane; I can test with

different combinations of any 4 drives...everything is fine for less than 5 drives.

 

I can run a ZFS array of all 16 drives and copy my backups over, giving full load to all disks without problems

...but this would not allow me to preclear the disks 

 

Any hints are very much appreciated!

 

TIA,

  Ford

Hi Folks,

 

I am currently performing a major overhaul of my system(s) and after all backups

were fine, I decided to test all used and new drives before importing the backups again.

 

Now, I have 16 drives (8x 2TB + 8x 3TB, mixed models) that need a preclear but I can only manage

to start the script on 4 drives at a time.

 

Whenever I run more preclear processes, everything gets very slow and I see someting like this

flooding the syslog:

Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: attempting task abort! scmd(ec41de40)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 0c 08 a5 70 00 01 00 00
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: handle(0x000a), sas_address(0x5001e67544b55fe0), phy(0)
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: enclosure_logical_id(0x5001e67544b55fff), slot(4)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: task abort: SUCCESS scmd(ec41de40)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: attempting task abort! scmd(ea09b540)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: [sda] CDB: cdb[0]=0x28: 28 00 0c 08 a6 70 00 01 00 00
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: handle(0x000a), sas_address(0x5001e67544b55fe0), phy(0)
Dec 30 20:14:23 Tower kernel: scsi target0:0:0: enclosure_logical_id(0x5001e67544b55fff), slot(4)
Dec 30 20:14:23 Tower kernel: sd 0:0:0:0: task abort: SUCCESS scmd(ea09b540)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: attempting task abort! scmd(d9a38900)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: [sdo] CDB: cdb[0]=0x28: 28 00 0d 0a ed 20 00 01 00 00
Dec 30 20:15:57 Tower kernel: scsi target0:0:14: handle(0x0018), sas_address(0x5001e67544b55fee), phy(14)
Dec 30 20:15:57 Tower kernel: scsi target0:0:14: enclosure_logical_id(0x5001e67544b55fff), slot(14)
Dec 30 20:15:57 Tower kernel: sd 0:0:14:0: task abort: SUCCESS scmd(d9a38900)

 

Here is my setup:

 

-I have a new 24 bay case, technically identical to a NORCO 4224, in which I have

currently connected/used four of the backplanes for my drives.

 

- unRAID 5.0-rc5-r8168

- preclear_disk.sh v1.13

- Mobo: X8SIL-F with XEON L3426 and 16GB ECC-R

- HBA: M1015 flashed to IT with latest P15 firmware

- Intel SAS Expander RES2CV240

- PSU is a 600W SuperFlower 80plus-Platinum - single 12V rail good for 48A

- all multilane cables are new

 

The mobo, HBA, 8x2TB disks  and PSU are receycled from my old case.

Expander, Multilane cables and 8x 3TB disks are new.

 

I am pulling my hair out...

The problem seems not to be connected to a specific backplane; I can test with

different combinations of any 4 drives...everything is fine for less than 5 drives.

 

I can run a ZFS array of all 16 drives and copy my backups over, giving full load to all disks without problems

...but this would not allow me to preclear the disks 

 

Any hints are very much appreciated!

 

TIA,

  Ford

Either

 

A. do not try to do 4 at a time

or

B. Use the "-r" "-w" and -"b" option to limit the resources used so you do not run out of memory when running many in parallel.

 

Yours is not an unusual situation...  Many have reported that with today's larger disks they cannot preclear as many drives in parallel.  The OS simply runs out of resources.

 

preclear_disk.sh -r 65536 -w 65536 -b 200 /dev/sdX

should do it.

Thanks Joe L. for the fast response !

 

Either

 

A. do not try to do 4 at a time

 

 

Well, that is what I am trying to do...I want to do 16 at a time with 3 cycles each 

 

or

B. Use the "-r" "-w" and -"b" option to limit the resources used so you do not run out of memory when running many in parallel.

 

Yours is not an unusual situation...  Many have reported that with today's larger disks they cannot preclear as many drives in parallel.  The OS simply runs out of resources.

 

preclear_disk.sh -r 65536 -w 65536 -b 200 /dev/sdX

should do it.

 

Thanks, I will try as soon as my first set of 4 disks has passed...

The system has enough RAM and a large processor (4 cores, 8 with HT) compared to unRAID "standards".

When running 16 preclear processes, "only" approx 10GB of RAM was in use...with the 4 disks currently processed, RAM usage is at 1GB and load is at 4.5 to 5.5...no sweat (CPU temp is low, with fan on low speed).

...update:

 

It appears that preclearing of up to 9 disks simultaneously is working with the parameters provided by JoeL.

 

Thanks a lot! This saved me 3-6 days for a third and fourth lot/round.

 

Edit: looks like the nasty effects re-appear after some hrs....tested with 6 concurrent processes/drives but still coming back....I am now down to 4 drives at a time, which passed previously.