• [6.7.0-rc2] Reading all disks when writing to a single one


    hawihoney
    • Retest Minor

    After upgrade from 6.6.6 stable to 6.7.0-rc2 I see unusual reads whenever I write to a single disk.

     

    E.g. In this example I write/copy to \\tower2\disk21 from my Windows 10 machine (SMB). During the whole copy all other disks are spun up and are read at low speed. In the example shown in the picture a 40GB file is written. disk21 and parity/parity2 show the usual write activity. But the other disks are spun up and read as well.

     

    After the file is written reading the other disks stopps as well.

     

    Diagnostics and image attached.

     

    *** Edit: The Main page shows that same read activity for the flash drive as well. Forgot to mention that.

     

     

    tower2-diagnostics-20190127-1031.zip

    Clipboard01.jpg




    User Feedback

    Recommended Comments



    Thanks, will check. That server has no Dockers, no VMs. Only two plugins (User scripts, Fix common problems). Bet it will show up in Safe Mode as well.

     

    Link to comment

    I'm also noticing some unusual disk activity

    Updated from rc1 to rc2 and now all my disks is spinning mostly all the time. before it was normal with one or two disk spinning. 

    Disks are set to spin down after 3 hours.  

    Link to comment

    Did some tests in Safe Mode and without Safe Mode.

     

    1.) What I see is, that writes are delayed. What I don't know is if this is the GUI lagging or a real diskwrite lag. E.g.: Copy a file between two individual disks with all disks spun-down. MC reports a successful copy. After that (!!!) the GUI shows that both parity disks and that individual disk are spinning up. This does not happen with 6.6.6.

     

    2.) Copying like mentioned in my first post did not spin up all disks (neither Safe Mode nor without Safe Mode).

     

    3.) But while writing this post I can see in a second window that all disks begin to spin up. This is true for Safe Mode and without Safe Mode. Something is waking up the disks. This does not happen with 6.6.6.

     

    There's one special disk in the array. Here I write backups nightly. This disk is ony used in the night, it's not used for anything else. Even this disk did spin up with all the other disks.

     

    Link to comment

    no turbo write here. same settings as on 6.6.6 and 6.7.0rc1 

    Tunable (md_write_mehtod) is set to auto

     

    Tried to manual spin down all disks and after a few minutes all disks are now spinning.

    Only activity is watching a tv show from one of the disks 

    Edited by Handl3vogn
    Link to comment
    12 minutes ago, trurl said:

    You might try turning on turbo write and then turning it back off.

    Turbo write is "Tunable (md_write_mehtod)" right? 

    And I'm only reading from disk no writhing. Still all disks spins up (maybe some read/write to cache but this should not spin up disks)

    Edited by Handl3vogn
    Link to comment

    No, I don't use something like Turbo Write. In fact I don't even know what it is.

     

    Are these Tunable Values?

     

    I changed md_num_stripes 4096, md_sync_window 2048 and md_sync_thresh 2000. All others are default.

     

    ***Edit*** These three Tunable Values mentioned above have been changed a year ago. I didn't change anything between 6.6.6 and 6.7.0-rc2.

     

     

     

     

    Edited by hawihoney
    Link to comment
    16 minutes ago, Handl3vogn said:

    Turbo write is "Tunable (md_write_mehtod)" right? 

    Yes.

     

    And one of you is writing and the other isn't. So it's not clear that you are experiencing the same thing.

     

    Other things to consider might be the Cache-Dirs plugin. Or anything else that might scan your disks such as plex. You could try to eliminate these factors to see if it changes.

     

    2 hours ago, johnnie.black said:

    Can't reproduce this, does the same happen in safe mode?

    Not happening to me either even without safe mode.

     

    Link to comment

    I'm shure there is writing to cashe (SSD) but this should not spin up other disks. And this worked fine on 6.6.6 and 6.7.0rc1.

    Only noticed it after updating to rc2 that  all my disks is spinning when normal behavior is none at night and 1-2 disks during daytime.

     

    Will try to change turbo write and some other settings to see if that helps

    Edited by Handl3vogn
    Link to comment

    After more testing turning turbo write on and off did nothing.

    Seems to be exactly the same problems as op. When writing to my array I get some reads from the rest of my disks resulting in all disks spinning up.

     

    The reads from the other disks seems to begin at the same time the transfer speed drops 10-15GB into the transfer. (most of the times)

     

    Seems to happen when I write to cache only share also 10-15GB into the transfer. (gets reads from all drives except parity this time)

     

    Also getting reads from flash device just like op said.

     

    unraid.PNG.2e0a60c433ea4413e230c6bd942be5b9.PNGshare.thumb.PNG.58c622fc398c7abcc745772f05a37306.PNG1784129187_disksettings.thumb.PNG.086d3edfadb5810d14197de5f9980f72.PNG

    Edited by Handl3vogn
    Link to comment

    Update:

    Tried to rollback to 6.7.0rc1 same problem, so it seems this was introduced in the rc1

    Went back to 6.6.6 and now the problem is gone.

    Link to comment

    Before I deep dive into this issue... maybe something is trying to read disk temperatures or other SMART info, which will spin up most HDD's, do you have a browser window open on the Dashboard?

     

    Second, are these writes to disk shares or user shares?

    Link to comment
    10 minutes ago, limetech said:

    Before I deep dive into this issue... maybe something is trying to read disk temperatures or other SMART info, which will spin up most HDD's, do you have a browser window open on the Dashboard?

     

    Second, are these writes to disk shares or user shares?

    Only having the Main page open to look at read/write values.

     

    I dont think there are anything else trying to read disk info.

    and I get reads even on boot flash drive.. This seems to happen at the same time the write speed goes down when writing.  And you can see my share is a user share that is just permitted to use disk 3. 

     

    After going back to 6.6.6 all these problems is gone and my unused disks can be spin downed

    Edited by Handl3vogn
    Link to comment

    I'm the OP and in my case I only write to individual disks. User shares are defined but I don't use them.

     

    The server has no Dockers, no VMs. The only two plugins are User scripts and Fix common problems.

     

    During a copy to disk21 all disks were spun up and showed read requests during the whole copy activity (around 10 minutes). As soon as copy to disk21 ended all read activity to the other disks stopped as well.

     

    One of the spun up and read disks is disk17. This disk is for backup purposes only. It's not used from anything but backup jobs beginning at 04:00 every night and running for approx. an hour. This disk was spun up and read for 10 minutes as well during that copy operation.

     

    This reminds me of one of the previous releases. I came late into the RCs, did mention something like "all disks spin up", had to read lots of "not happening here" and with the final release a patch to SMB was introduced that fixed that behaviour.

     

    I'm not at home currently, will look for this old thread tomorrow.

     

     

     

     

     

    Link to comment

    For me, OP, I can no longer reproduce that unusual behaviour on 6.7.0-rc2.

     

    I went back to 6.6.6 and did re-test the complete scenario. Here, everything was as expected.

     

    Then I did install 6.7.0-rc2 as before and did re-test that whole scenario again. And this time Unraids behaviour was as expected.

     

    As this machine is mainly a backup and read-only machine with no dockers, no VMs and just two plugins I can't say what's the reason for this. I will stay with 6.7.0-rc2 on that machine and test a little bit further.

     

    Thanks for listening.

     

    Link to comment

    Argh, it's happening again. I'm reading disk17 and all disks spin up and have low read activity. There's no hint in lsof or syslog. When looking at the server case I can see very minimal blinks of the activity LEDs. While disk17 ist constantly lit, the activity LEDs on all other disks circulate very fast (disk1, disk2, ...). That happens every 5-10 seconds.

     

    Sorry, was to fast with my previous post. Never seen that behaviour an 6.6.6

     

    Clipboard01.jpg

    Edited by hawihoney
    Link to comment
    26 minutes ago, hawihoney said:

    I'm reading disk17 and all disks spin up and have low read activity.

    Do you have cache dirs plugin installed?

    Link to comment
    19 hours ago, hawihoney said:

    I'm the OP and in my case I only write to individual disks. User shares are defined but I don't use them.

    Since you are not using User Shares, then turn this off.  Settings/Global Share Settings.

    Part of status polling which occurs generally every second, is to look at each disk for user shares (top level directory names).  Most of the time this does not cause spinup because these directory 'inodes' are cached in RAM.

    But maybe something  about this kernel 4.19 and your i/o pattern is conspiring together to cause those idodes to get ejected.

    Link to comment
    2 hours ago, jonathanm said:

    Do you have cache dirs plugin installed?

    No. Only User scripts and Common problems.

     

     

     

    Edited by hawihoney
    Link to comment
    1 hour ago, limetech said:

    Since you are not using User Shares, then turn this off.

    ...

    But maybe something  about this kernel 4.19 and your i/o pattern is conspiring together to cause those idodes to get ejected.

    Thanks, will do.

     

    ***Edit*** Wait, it can't be that easy. The small read request to the other disks always end with the read or write request to the single disk.

    Edited by hawihoney
    Link to comment

    I have the exact same issue.

     

    With 6.6.6, the drives would spin down, upon testing 6.7.0-rc1/rc2 I've noticed the system would behave exactly like the original poster's system.

     

    The system is running Plex (Docker), Cache Dirs, and a couple of plugins.

     

    I've stopped all the docker images, enabled and disabled the cache dirs with various settings (changed cache pressure to 1, enabled and disabled scanning of user shares, etc).  I use SMB Time Machine on 6.7.0-rc, and I've disabled it as well, with no avail.

     

    I have also tried going into the disk settings and changed the Tunable (md_write_method): from auto, to "read/modify/write".

     

    The only thing that seems to fix it is downgrading it to 6.6.6, however I'd like to be on the 6.70 branch since I'm using the SMB+TimeMachine.

     

    Chris

     

     

    Link to comment



    Join the conversation

    You can post now and register later. If you have an account, sign in now to post with your account.
    Note: Your post will require moderator approval before it will be visible.

    Guest
    Add a comment...

    ×   Pasted as rich text.   Restore formatting

      Only 75 emoji are allowed.

    ×   Your link has been automatically embedded.   Display as a link instead

    ×   Your previous content has been restored.   Clear editor

    ×   You cannot paste images directly. Upload or insert images from URL.


  • Status Definitions

     

    Open = Under consideration.

     

    Solved = The issue has been resolved.

     

    Solved version = The issue has been resolved in the indicated release version.

     

    Closed = Feedback or opinion better posted on our forum for discussion. Also for reports we cannot reproduce or need more information. In this case just add a comment and we will review it again.

     

    Retest = Please retest in latest release.


    Priority Definitions

     

    Minor = Something not working correctly.

     

    Urgent = Server crash, data loss, or other showstopper.

     

    Annoyance = Doesn't affect functionality but should be fixed.

     

    Other = Announcement or other non-issue.