Jump to content

casperse

Members
  • Content Count

    395
  • Joined

  • Last visited

Posts posted by casperse


  1. Hi Everyone

     

    I needed more NIC's and until I get 10gbe card I wanted to use one more 4xNIC intel card!

    Have I configured this wrong?

     

    My cards:

    image.png.060288d44fde142d9b6830b195dc3ec6.png

     

    The 4 from the "Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)" works fine!

    But only the first two of the: "Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)" works?

     

    I have created the Stub like this:

    image.png.e8b5e2f011ed8d038758ef57b8802b92.png

     

    It's like the last 2 NIC's are deactivated? I can see a short connection flash orange but no green activity led?

    It's almost like I only have half of the card? - Could this be some PCi Express limitation of my MB?

    Shared lanes? or did I just configure this wrong? "8086:10e8,8086:1521"

     

     

     


  2. 17 minutes ago, JorgeB said:

    Don't need to change anything in the BIOS, you don't even need a BIOS.

    So I need to do nothing...?

    Wouldn't it just be the right thing to do? (Setting boot : Disabled skipping this every time I reboot the server)

    And is hoot-swap working if I don't set the "removable media support" to YES?

    And do I get all the drives listed during boot if I don't set any info

    Sorry I was just sure I did something on the old controller?


  3. Hi All

     

    PLEASE HELP!!!

    I now have the opportunity to borrow a LSI SAS9305-24i card (JUST FOR TESTING) and replace it with my LSI Logic SAS 9201-16i

    (I currently use this 16i + 8 x sata on the MB cabled to = 2 x  Mini-SAS 36 Pin SFF-8087 on the back plane)

     

    So this should have native IT mode (As standard) and be able to replace the existing SATA ports currently used on my MB (8 x sata)

    But I had to see if the Firmware was updated and looks like it is...

    image.thumb.png.44eda34e20862da90f8b5f4e97c65294.png

     

    BUT I can not remember if I need to change anything else in the config tool?

    I was looking to remove any boot options to drives? - Unraid don't need this (Just a bunch of drives) - BIOS is just access to the Bios of the card right?

    image.thumb.png.06518dd2a90dfa4a55e73f4659daf499.png

     

    Also there was some options to display during boot of the drives....

    image.png.f07791c7e1b88721bb72073e2a548ed0.png

     

    And do I have to set "Removeable Media Support" to get Hot Swap functionality of my drives like I have today?

    image.thumb.png.7a09466b0eff80a6d8e3b23e4f9340a9.png

     

    Sorry this is so long time I tinkered with a LSI card (They just work!)

     


  4. 1 hour ago, mgutt said:

    Maybe it's possible to monitor the traffic of the disks or use the Plex API to spindown all unused drives after several minutes instead of waiting for the usual time. But of course this would cause delay if someone starts s second movie from a different disk. What do you think?

    Since this only monitor the Plex service I think its ok to spin-up all drives up for a person accessing the system (Its a VIP service ;))

    1 hour ago, mgutt said:

    After they spin-up the usual timeout should keep them in this state or what do you mean?

    I was wondering what the timeout of this "Spin-up state" was?

    Since I can see accessing the Plex service and any library folder in Plex will draws +10% CPU every time!

    Even just accessing my Tautulli app on my iphone would push the script to activate wich is ok and by design!

     

    But what happens after you have accessed the library and you start to scroll through titles (They are all on the cache and loads to the browser cache) so the %CPU would go down again - how long time using "browsing titles" before it would spin down again?

     

    I use your test script to monitor activity of the script (4%):

    while true; do
        plex_cpu_load=$(docker stats --no-stream | grep -i plex | awk '{sub(/%/, "");print $3}')
        echo $plex_cpu_load
        if awk 'BEGIN {exit !('$plex_cpu_load' > 4)}'; then
            echo "Container's CPU load exceeded threshold"
        fi
    done

     It's a great way to see how and when it triggers


  5. Finally -  Thanks for all your help! - I think it's finally where I can use this script without spinning my drives up 24/7 :D

    But maybe I should make it around 4-5% - I still get some spikes after 3 min:
    3.71 - Container's CPU load exceeded threshold

    And I think Plex activity of any user going into any media library will go way above 4%...even 10% in my testing

    Could be interesting to see if this actual have any noticeable impact on my electric bill since all 22 drives (Not including the 2 x parity drives)  would be spin up during any playback of any movie. I actually did the separate UAD drive for frequent access in order not to do spin ups but that was 24/7 I think this is reasonable trade off... Thanks again

    image.thumb.png.e004193b9666ab710d76dff06dee8915.png

     

    Shouldn't it keep them spinning 2-3min between getting threshold again? keeping the drives spin-up?


  6. WOw = 3 min!

     

    Script Starting Oct 24, 2020 11:59.04
    Full logs for this script are available at /tmp/user.scripts/tmpScripts/Plex Preloader v0.9/log.txt
    Available RAM in Bytes: 11931406300
    Amount of Videos that can be preloaded: 195

    ....*.mkv*.srt 

    tr: write error: Broken pipe
    tr: write error
    cut: write error: Broken pipe
    Script Finished Oct 24, 2020 12:02.09

     

    Would like to do a test of the impact doing this but struggling how to do accomplish it?

    Needs to be "remote" for transcoding to kick in, how did you go about this during your dev/testing?

    Also some people have "remote transcode settings" fixed, would they overrule this? like 720P 4Mb/s and make the Preloader obsolete? 

    image.png.10421978575890a67e9733171ef8b2ee.png

    Sorry for the many Q's again great work! your work in so many ways are really moving the "Media tweak" post forward!


  7. Oh yes Stuff happens especially when we like us - go and do special tweaks (Stuff also happened for me, but the backup script from @mgutt is great and I use that for both emby and Plex (Appdata is huge! incremental backup is great) and of course the VM/Docker backup apps from the Unraid app community)

     

    This issue got me thinking, what would happen if you have the appdata folder as:

    image.thumb.png.05648be2aa6454beed59b05a93a8635f.png

    And you have all your Appdata path in the docker changed to /cache/

    image.png.fddbe47415c0d25844d21e7f3579a87c.png

    But you do have the global share settings set:

    image.thumb.png.ddd02d35e055014b73e93892c669cf46.png

    And you have a overflow of Plex metadata to appdata? (Prefer setting!)

    Would it write to the array?

    The path from the docker would only see the cache drive, so new data would not be available?

    How would this work? (Don't really want to test this in practice :))

    Sorry if this is a stupid Q...

     

     

     

     

     


  8. 33 minutes ago, mgutt said:

    First process is SHFS. Something is still writing to / reading from "/mnt/user/".

     

    1.) Your docker.img path uses /mnt/cache?

    Yes I pretty much have /Cache/ written everywhere now :-)

    image.png.bf05320959283447a321d729e51fc406.png 

    33 minutes ago, mgutt said:

    2.) Did you disable Heartbeat? (The Link contains a command to monitor writes to the docker.img as well)

    Funny I said that it looked like a heartbeat didn't know there actually was one! (I will disable it when there is no traffic and test again)

    Thanks!


  9. 3 hours ago, mgutt said:

    No problem as the Preloader default setting is 50% of the free RAM. This means there are still 50% left for the subtitles. Except you set a much higher value or you have much less of free RAM. But you are right, the best would be to parse only those folders, which contain preloaded video files. Will think about that.

    Maybe it's not the size of the srt files but the fact that they are located across 24 drives? 

    Anyway your code is solid my server doesn't crash and it still run perfectly! just take some time LOL (I kept it at default 50%)


  10. On 10/21/2020 at 10:50 PM, mgutt said:

    Are you sure? SRT files are loaded AFTER all clips haven been preloaded. And yes, it loads ALL SRT files into the RAM (as they are small and I'm too lazy to write a function that checks dependencies between clips and SRT files ^^).

    All of the srt files? for Movies & TV shows?! WOAW

    I have a average of 3-4 srt files (Nordic TV sometime also german close to DK - for each TV show and most movies) how does it have enough space left for loading the right subtitles?

    Just did a count on TV shows alone and I have around +40K srt files - so now I understand why it did run "forever"

    Average srt file is 30-45K = 1G Ram (TV only)

     


  11. So I did the change to all appdata path to /cache/

    (If I ever update my cache drive I need to remember to change this back before moving it back to the array! LOL)

     

    I then started to shutdown every docker! and every VM down! so only Plex docker was left running:

    image.thumb.png.a6fcca1e3755e38bd144bcc07bf71cdd.png

     

    And I keep getting these Bump as a heartbeat? So only thing left is Unraid Apps (Can not stop them, but I can start to uninstall them?)

    Or its the Plex inc Docker? that somehow are different than the others? (I could try another one?)

    Almost given up now! :-) 

     


  12. 2 minutes ago, mgutt said:

    Maybe this will solve it. Because it does not make sense to have I/O wait on an NVMe. An NVMe has a huge bandwith and allows (instead to SATA) parallel access. Or do you use a low-grade NVMe with QLC flash / non SLC Cache / no RAM?

     

    I got this one: Samsung SSD 970 EVO Plus 2TB the fastest I could get around a year ago! (Plex poster scroll loading :D)

    So this should help I/O wait


  13. 20 hours ago, mgutt said:

    Do you already use direct disk access (replace /mnt/user/... against /mnt/cache/...) for Plex appdata, docker.img and other docker appdata containers, using the cache only?

     

    I see what you mean!

    I only went into the dockers of Plex/Emby and changed the path to /cache/ I should do that for all of the running dockers /appdata/ path!

    I come back and report my findings when that's done  ;)


  14. Hmm yes I do have allot of things running on the server!

    But I did manage to get individual Sleep of drives working by having separate UAD for download and other things (There is a nice FAQ about this setup in the forum) - and things I don't want to have on the array

    And yes I have allot of apps running 24/7

     

    I think the cache is the problem!

    Plex and all appdata is located on the CACHE drive NVME drive and its the same drive that is used as cache creating the high I/O?

     

    So I cant use Plex %CPU because everything is on my cache drive - But I still experience "slow" playback from Plex at strange hours because the other disk is in sleep state - I would need to find another trigger, to bad you cant set Plex user web activity = Spin-up 🙂 

     

     

     


  15. Hi mgutt

     

    Just wanted to let you know that I just tried your script, and I think its working 100%

     

    Logfile:

    Preload /mnt/user/movies/Movies/Friends with Kids (2012)/Friends with Kids (2012) [Bluray.720p.x264.8Bit.DTS.5.1].mkv (6.835s)
    Preload /mnt/user/movies/Movies/Transylvania 6-5000 (1985)/Transylvania 6-5000 (1985) [Bluray.720p.x264.8Bit.DTS.2.0].mkv (0.337s)
    Script Starting Oct 21, 2020 20:20.51
    
    Full logs for this script are available at /tmp/user.scripts/tmpScripts/Plex Preloader v0.7/log.txt
    
    Script Finished Oct 21, 2020 20:20.51

    Update: Did a test added "TV Shows" and I could see srt files being loaded (Running Unraid 6.8.3)

    So how to get 1TB of memory!!! LOL

    Great work, you have really made a big impact on my Plex server!

     


  16. 5 minutes ago, mgutt said:

    Really strange. Maybe the container CPU load includes I/O wait? Because your top results show permanent wait (1.4, 0.7, 0.5 and 1.0). This means something is reading / writing data from/to HDD/SSD and the volume answers not fast enough.

     

    Please repeat the test, but this time check all processes:

    
    top -c

    I'll bet its unraid's SHFS process ^^

    image.thumb.png.4e34ac980fa63bd6f4a0ab861ecc84bf.png

     

    I can't see what it is, hope you can :-)


  17. 22 hours ago, mgutt said:

    When this happens execute the following to see all Plex processes

     

     top -b -c -d 5 | grep -i -E "(top -|tasks:|%cpu|mem:|swap:|plex)"

     

    I guess Plex:

    - rescans the library

    - Plex generates Thumbnails

    - Plex tries to find tv show intros (so they can be skipped)

     

     

     

    So I did run both script at the same time:

    image.png.53de2bc1085f02ff29c9ab85336f63bc.png

     

    But below I don't see the CPU go above 1%

    image.thumb.png.66c0273985078b6bf21f288a99370fc4.png

    Not a single time!

    This is really strange

     

     


  18. I think I found the solution you have to add "ata-" then it works

        <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='writeback'/>
          <source dev='/dev/disk/by-id/ata-ST10000NE0004-1ZF101_ZA21DG4A'/>
          <target dev='hdd' bus='sata'/>
          <address type='drive' controller='1' bus='0' target='0' unit='3'/>
        </disk>

     


  19. Ok so I started changing my drives:

        <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='writeback'/>
          <source dev='/dev/disk/by-id/ST10000NE0004-1ZF101_ZA21DG4A'/>
          <target dev='hdd' bus='sata'/>
          <address type='drive' controller='1' bus='0' target='0' unit='3'/>
        </disk>

    Serial from the dashboard ID:

    image.png.0038d4487a32952ca812591627427cec.png

     

    And trying to start I get:

    image.png.c479719a7470b20b4d0ded9bf48e253e.png

    Update tried the serial number I could save the VM but it couldnt start

     

    Sorry @itimpi this doesn't seem to work?