Jump to content

Falcosc

Members
 View Profile  See their activity

  • Posts

    91

  • Joined

  • Last visited

  • Days Won

    1

 Content Type 

Everything posted by Falcosc

  1. Falcosc
    This did bother me, too. You see it on the circle, but a progress bar is much better. So I added it: monitor verifications and all manual bunker calls https://github.com/bergware/dynamix/pull/47 At some point we lost our documentation, it was still available but kind of hidden. I add the help text toggle to the disc table and button bar to give new users a more easy start: fix inline help for disk table and commands https://github.com/bergware/dynamix/pull/49 Which process did not appear at dynamix.file.integrity/logs? For now, I did add a small non-downloadable watch errors popup which can be accessed during progress but only appears if there are at least 1 skip/warning/error: show errors of progress: https://github.com/bergware/dynamix/pull/50
  2. Falcosc
    I use 500GB test data with 180.000 files and I run through that about 20 times already to get all these improvements tested and benchmarked. I got 2 builds with 1 missing key each. And it was not the same file. So b2sum did just not return a key and I don't know the condition at which this happened. I don't terminate my test runs, so I was really unsure what is happening. But the user responses related to truncated files did prove that something is unstable. Did you notice that all truncated file reports did use blake2? Maybe just a coincidence, but maybe a clue.
  3. Falcosc
    I don't know why b2sum does not return a key in very rare cases because I did not log stderr and sigcode of all 800.000 b2sum calls. But it happened twice on my host and because b3sum is still without any calculation failure after 1.200.000 calls on 200.000 different files, I am confident that our b2sum binary is a bit unstable and may need an update. Testing such rare issue is complicated because my 1.200.000 hashes needed to be validated as well. If anybody want to still use the old blake2, we may have to check if the old b2sum cli got any kind of fixes. But blake3 is better than blake2 in performance anyway we don't take care of b2sum and if you need tools without blake3 support you should consider sha256 with hardware acceleration https://en.wikipedia.org/wiki/Intel_SHA_extensions It's cool that software calculated blake3 is even faster than current Intel SHA extensions That shows just how efficient blake3 really is.
  4. Falcosc
    I thought about it, but it is disruptive. Each hash has it's own field User.md5 User.blake2 and so on So for hashes we already on track But the following fields are shared: User.filesize (file size while hash creation) User.filedate (fie data during hash creation) User.scandate (last scan date) So if you change the hash method the plugin will get confused because the shared fields doesn't match to the hash creation. To solve it we would have to use this structure: User.blake2="hash filesize filedate scandate" I have tried this structure, it does improve creation time a bit because you need 3 syscalls less. But this change would invalidate existing attributes. I will consider this for a later release, for example by adding a migration script to avoid slow and complex bash script handling of backwards compatibility. For now, it is better to only use one hash method. By the way, b2sum is a bit unstable. It does return no key in one of 400.000 calls on my host. That's the reason why so many people have issues with truncated file names. If key is missing, the file name gets truncated by the length of the missing key. blake3 will fix this because it comes with a new CLI b3sum and I did already called b3sum 1.200.000 times without a single crash.
  5. Falcosc
    Community comes to the rescue! add blake3 support https://github.com/bergware/dynamix/pull/42 speed up bash by reduce expensive calls https://github.com/bergware/dynamix/pull/43 use check list to speed up check button https://github.com/bergware/dynamix/pull/44 avoid indexing of filesystem while watching progress https://github.com/bergware/dynamix/pull/45 avoid terminal wrap during progress https://github.com/bergware/dynamix/pull/46 monitor verifications and all manual bunker calls https://github.com/bergware/dynamix/pull/47 fix truncated file names if key is broken https://github.com/bergware/dynamix/pull/48 This weekend some or all of my pull requests get tested and some time later released I did these updates to speed up my process because the old tool can't handle small files: Old version: Average speed: 62 MB/s new Version: Average speed: 167 MB/s
  6. Falcosc
    I found a possible issue, something can get wrong during parsing the response from getting the extended file attributes. I can fix it if anybody of you is still around, you just need to execute the following commands to create something which can be manually checked: path=/mnt/disk1 hash=blake2 find "$path" -type f -name "*" -exec getfattr -n user.$hash --absolute-names "{}" 2>/dev/null 1>/tmp/debugfiles_raw.txt + sed -n "/^$/d;s/^# file: //;h;n;s/^user.$hash=\"//;s/\"$//;G;s/\n/ \*/;p" /tmp/debugfiles_raw.txt 1>/tmp/debugfiles_sed.txt In /tmp/debugfiles_sed.txt you will find your truncated line or an empty line or a line where the key is there but no file name after the key. I would like to see your result to fix it. I was unable to construct a broken file to get this to fail. Maybe there is an unexpected response from getfattr on stdout.
  7. Falcosc
    I can fix this issue. At which action did this happen?
  8. Falcosc
    Depending on how you use it you can get false positives. I got 2 as well but was able to fix them. If you are still interested, you can get in touch with me.
  9. Falcosc
    There are 2 find commands: is executed once a day is executed every 2 minutes to monitor active drives while watching the File Integrity Control page I had to fine tune the 2nd one because it was messing with my check performance. I could resolve your issue as well, could you try to describe when these did occur to remove them?
  10. Falcosc
    I would like to ask the question from December 2020 again. What about BLAKE3 integration? https://github.com/BLAKE3-team/BLAKE3/releases/tag/1.0.0 Is anybody working on that, or should I create a pull request for that? By now, enough people should have CPUs with AVX2 support to use it. Performance could be even better than ardware accelerated SHA https://www.phoronix.com/forums/forum/software/programming-compilers/1175724-blake3-cryptographic-hashing-function-sees-experimental-vulkan-implementation?p=1176083#post1176083 which is only supported on latest CPUS https://en.wikipedia.org/wiki/Intel_SHA_extensions
  11. Falcosc
    I found the reason of both of my issues: Sleep Plugin does only autostart if you have started an array because the startup is scripted in the file "disks_mounted" WOL options may get reset during PCI-E device order testing, maybe at on of my reboots it didn't found the nic or there is still an issue there. But didn't occur anymore on my host.
  12. Falcosc
    The power of mgutt. Why does your topic attract so many people, while I had to wait days until anybody even downloaded one of my logfiles I bet the logfile downloads are only related to your magic thread Thank you for this attention. Anyway, If nobody looks into bug reports, then I can post updates here: Unraid's ASPM management is not totally broken on my platform. Devices connected to CPU via the x16 Slot do automatically get ASPM Enabled like expected, even without boot option. Only devices connected to my Chipset have this issue. 0000:00:1c.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #7 [8086:a116] (rev f1) (prog-if 00 [Normal decode]) I don't know why Ubunutu doesn't have this issue. I tested the old ubuntu-20.04.2.0 with Kernel 5.8. I can do any test you like to have because I didn't clean up my test bench yet and still evaluating. New Unraid Builds, other Ubuntu Versions, Old Ubuntu with manual Kernel update. Just tell me what you need. @mgutt maybe link the bug report in your topic opening post because the test setup description is detailed enough to see that there is no setup issue and really a difference between Ubuntu and Unraid to hunt for.
  13. Falcosc
    setWol="g" gets lost after some reboot as well. After some reboot, I have to configure set WOL before sleep to g, but after each reboot I have to start the plugin by change it from sleep to disabled and then from disabled to sleep again. Is there a way to enable the plugin during boot?
  14. Falcosc
    How to keep plugin status S3 Sleep enabled after reboot or cold boot? It does only stays enabled the setting if you don't reboot the server. Currently, it always boots in disabled status, despite having it configured to execute sleep. cat /boot/config/plugins/dynamix.s3.sleep/dynamix.s3.sleep.cfg options="-C 1 -a -m 3 -n -e eth0 -N 1250 -l -w g -b /usr/local/emhttp/plugins/dynamix.s3.sleep/scripts/preRun -p /usr/local/emhttp/plugins/dynamix.s3.sleep/scripts/postRun -D 2" service="1" stopDay="" stopHour="" checkHDD="-a" hdparm="" timeout="3" checkTCP="-n" port="eth0" idle="1250" pingIP="" login="-l" setWol="g" preRun="printf+%22%5Cn%24%28date+%2B%27%25F+%25T%27%29%22+%3E%3E+%2Fboot%2Fcustom%2Fsleep.csv" postRun="printf+%22%2C%24%28date+%2B%27%25F+%25T%27%29%22+%3E%3E+%2Fboot%2Fcustom%2Fsleep.csv" forceGb="" dhcpRenew="" debug="2" I have no array configured because I'm currently evaluating Unraid.
  15. Falcosc
    Steps to reproduce: correctly configure bios to support ASPM Don't install any unnecessary components, no hard drives connected, no add-in cards installed boot a fresh Ubuntu 21.04 from USB check if ASPM is enabled lspci -vvvnnPPDq lspci_ubuntu.txt boot Unraid 6.9.2 check if ASPM is enabled lspci -vvvnnPPDq lspci_unraid.txt try to fix it with echo -n powersave > /sys/module/pcie_aspm/parameters/policy try to fix it with boot option add pcie_aspm=force to /boot/syslinux/syslinux.cfg reboot check if ASPM is enabled lspci_unraid_aspm.txt try to fix it with echo -n powersave > /sys/module/pcie_aspm/parameters/policy powertop --auto-tune veryfy if boot option was used pcie_aspm=force unraid_aspm_syslog.txt Expected Result: ASPM should be enabled like it is in Ubuntu to save 4W of power in idle Actual Result: It is still disabled lspci_unraid_aspm.txt What setting or which driver could prevent ASPM or does disable ASPM? I don't know how to compare Ubuntu with Unraid to check what is different. Workarround: I know that Unraid and my hardware is capable of enable ASPM. Not only because Ubuntu does it, but because I can force it with manually setting it in the register https://wireless.wiki.kernel.org/en/users/documentation/aspm#enabling_aspm_with_setpci After doing this hack for my root and ethernet interface, the power consumption gets reduced by 4W and powertop does tell that my CPU Package does now reach C7 instead of C2 Hardware Infos server-diagnostics-20210727-1604.zip About Me I don't know if this error did exist in the previous versions because I'm currently evaluating Unraid and would like to know how important power saving issues are for you because I want to use it to make it more easy to reduce my servers power consumption. Power is expensive at my location, I can easily afford server upgrades based on power savings. I would like to give the money to you for a license instead to my power provider
  16. Falcosc
    Since this topic is referenced in some related topics, I want to add my point of view in this over 10 years old topic. You don't need a Switch with IGMP snooping support, and you don't need a wake on ARP capable NIC. If you want a cheap solution, just setup a 10$ RaspberryPi with 1 W power consumption and convert the ARPs to Magic packages. This is possible because ARPs are broadcasted to all network clients anyway. You can even filter them to prevent unwanted wake-up calls from misbehaving sources. I do use this tool: https://github.com/nikp123/wake-on-arp I needed to extend it a bit to add monitoring and to exclude unwanted wake-up calls from the router. And I did share my config https://github.com/nikp123/wake-on-arp#how-to-install
×
×
  • Create New...