3 hours ago, JorgeB said:

It should be running this script, I dont see the the cache pool being trimmed with the manual command, I assume it still exists?

 

Also please post the complete diags just in case there's something out of the ordinary there.

 

Yea I'm not sure why it didn't show cache. When i ran it, it displayed the path, but then it seemed to overwrite that line with the deluge pool line.

I just ran it again, and it shows cache now. Cache has always existed and was never disconnected. So not sure why it wasn't listed.

 

 

Ive attached my diagnostic Zip.

 

 

@shorty_ I changed the trim schedule this past Sunday, to run it Tuesday instead of on Monday, so that should have taken care of that.

 

backup-diagnostics-20240703-1416.zip

12 hours ago, TurboStreetCar said:

I just ran it again, and it shows cache now.

I would expect that manual TRIM would run the same script as the scheduled TRIM, so don't get why the HDD is only trimmed on the scheduled run, let me see if I can find more info.

I'd just use the mount option workaround and be done with it.

11 hours ago, JorgeB said:

I would expect that manual TRIM would run the same script as the scheduled TRIM, so don't get why the HDD is only trimmed on the scheduled run, let me see if I can find more info.

 

Thats what i would expect too, but seems to be different. Maybe its running with a different level of permission or something? Ill admit when it comes to these things its beyond my depth. I can google some code stuff and get by, but ill admit im not sure how to diagnose whats going on.

 

 

10 hours ago, _Shorty said:

I'd just use the mount option workaround and be done with it.

 

It doesnt seem unraid has user settable mount options. How would i go about trying this?

 

 

 

I guess another question, If i have the drives set for "AutoTrim = ON", do i need to run a scheduled trim at all?

Edited July 4, 20242 yr by TurboStreetCar

Looking at it again, it seems you don't even need to modify any mount info.  Rather, you can just use a dummy file with a line that looks the same as what you'd find in /etc/fstab and use that to tell fstrim to skip /dev/sde1 for you.  Maybe go to /boot/config and create a dummy file called "fstrim-dummy-file" that just has this line in it:

 

/dev/sde1  /mnt/sde1  ext4  defaults,X-fstrim.notrim  0  0

 

And in /usr/local/emhttp/plugins/dynamix/scripts/ssd_trim change this:

 

write(_("TRIM operation started")."\n","\n","\n");
// trim btrfs, xfs
exec("findmnt -lnt btrfs,xfs -o target,source|awk '\$2!~\"\\\\[\"{print \$1,\$2}'",$mounts);
foreach ($mounts as $mount) {
  [$target,$source] = explode(' ',$mount);
  if (is_hdd($source)) continue;
  write("$target: ... <i class='fa fa-spin fa-circle-o-notch'></i>\r");
  $trim = exec("fstrim -v $target 2>/dev/null");
  if ($trim) write("$trim on $source\r","\n"); else write("\r");
}

 

to this:

 

write(_("TRIM operation started")."\n","\n","\n");
// trim btrfs, xfs
exec("findmnt -lnt btrfs,xfs -o target,source|awk '\$2!~\"\\\\[\"{print \$1,\$2}'",$mounts);
foreach ($mounts as $mount) {
  [$target,$source] = explode(' ',$mount);
  if (is_hdd($source)) continue;
  write("$target: ... <i class='fa fa-spin fa-circle-o-notch'></i>\r");
  $trim = exec("fstrim -v --listed-in /boot/config/fstrim-dummy-file $target 2>/dev/null");
  if ($trim) write("$trim on $source\r","\n"); else write("\r");
}

and I believe that should stop it from trimming that drive.  The only change is the one line to add the dummy file that lists the drive to skip.

 

$trim = exec("fstrim -v --listed-in /boot/config/fstrim-dummy-file $target 2>/dev/null");

 

Edited July 4, 20242 yr by _Shorty

On 7/4/2024 at 3:13 PM, _Shorty said:

Looking at it again, it seems you don't even need to modify any mount info.  Rather, you can just use a dummy file with a line that looks the same as what you'd find in /etc/fstab and use that to tell fstrim to skip /dev/sde1 for you.  Maybe go to /boot/config and create a dummy file called "fstrim-dummy-file" that just has this line in it:

 

/dev/sde1  /mnt/sde1  ext4  defaults,X-fstrim.notrim  0  0

 

And in /usr/local/emhttp/plugins/dynamix/scripts/ssd_trim change this:

 

"..."

to this:

"..."

and I believe that should stop it from trimming that drive.  The only change is the one line to add the dummy file that lists the drive to skip.

 

$trim = exec("fstrim -v --listed-in /boot/config/fstrim-dummy-file $target 2>/dev/null");

 

 

 

Hmm. I think before i do this, im going to try to add a debug line to the script to write an output to a file to make sure the scheduled trim is running the same script.

OK so i added some debug commands to the ssd_trim script found in /usr/local/emhttp/plugins/dynamix/scripts/ssd_trim.

This is the final function of the script now:

$myfile = fopen("ssd_trim.log", "a");
$timestamp = date("m/d/Y - H:i:s");										### Open Log File
fwrite($myfile,"Trim Started at $timestamp\n\r");						### Write Timestamp of beginning of Trim
write(_("TRIM operation started")."\n","\n","\n");
// trim btrfs, xfs
exec("findmnt -lnt btrfs,xfs -o target,source|awk '\$2!~\"\\\\[\"{print \$1,\$2}'",$mounts);
foreach ($mounts as $mount) {
  [$target,$source] = explode(' ',$mount);
  fwrite($myfile,"Checking Source: $source Target: $target\n\r");		### Write current $source and $target being checked
  if (is_hdd($source)) continue;
  fwrite($myfile,"$source Is HDD\n\r");									### Result of HDD Check
  write("$target: ... <i class='fa fa-spin fa-circle-o-notch'></i>\r");
  $trim = exec("fstrim -v $target 2>/dev/null");
  if ($trim) write("$trim on $source\r","\n"); else write("\r");		
  if ($trim) fwrite($myfile, "Trimming $source\n\r");
  else fwrite($myfile, "Not Trimming $source\n\r");						### Write to log file when drive is trimmed
}
// trim zfs
zfs_trim(true);
write(_("Finished")."\n",'_DONE_','');
fwrite($myfile, "Finished Trim \n\r\n\r\n\r");							### Write end of trimming operation
fclose($myfile);														### Close log file

 

 

When i run this in a terminal using the command:

root@Backup:/usr/local/emhttp/plugins/dynamix/scripts# php ssd_trim

 

The log file received this output:

 

Trim Started at 07/06/2024 - 13:47:42
Checking Source: /dev/md1p1 Target: /mnt/disk1
Checking Source: /dev/nvme0n1p1 Target: /mnt/cache
/dev/nvme0n1p1 Is HDD
Trimming /dev/nvme0n1p1
Checking Source: /dev/sde1 Target: /mnt/cctv
Checking Source: /dev/nvme1n1p1 Target: /mnt/deluge_pool
/dev/nvme1n1p1 Is HDD
Trimming /dev/nvme1n1p1
Checking Source: /dev/sdb1 Target: /mnt/plex
/dev/sdb1 Is HDD
Trimming /dev/sdb1
Checking Source: /dev/loop2 Target: /var/lib/docker
/dev/loop2 Is HDD
Trimming /dev/loop2
Checking Source: /dev/loop3 Target: /etc/libvirt
/dev/loop3 Is HDD
Trimming /dev/loop3
Finished Trim 

 

 

I received this output ALSO when i click the "Trim Now" button in the scheduler page.

HOWEVER........

 

I changed the schedule to run a trim at 1400 Hours (44 minutes ago, my local time) and it is taking WAY longer to run, AND im receiving "disk full" errors in my Frigate container.

SO i checked "system processes" on the tools page, and i see these entries related to SSD Trim:

root       888  0.0  0.0   3972  2832 ?        S    14:00   0:00 /bin/sh -c /usr/local/emhttp/plugins/dynamix/scripts/ssd_trim cron|logger &> /dev/null
root       911  0.0  0.0  95616 27620 ?        SL   14:00   0:00 /usr/bin/php -q /usr/local/emhttp/plugins/dynamix/scripts/ssd_trim cron
root       912  0.0  0.0   2588   900 ?        S    14:00   0:00 logger
root       913  0.0  0.0   3976  2880 ?        S    14:00   0:00 sh -c fstrim -va 2>/dev/null
root       914  0.0  0.0   3332  1068 ?        D    14:00   0:01 fstrim -va

 

Which leads me to believe, that its running the same script that im running both manually in terminal, and by hitting the Trim Now button.

So im not sure whats going on, but it definitely seems to be running a different script on the schedule vs pressing the trim now button, or running that same script in a terminal.

Is there anything i can do to more thoroughly check what is the difference between these two different methods of running (seemingly) the same script?

Edited July 6, 20242 yr by TurboStreetCar

if (is_hdd($source)) continue;
fwrite($myfile,"$source Is HDD\n\r"); ### Result of HDD Check

 

The "continue" command actually means "bail out here and don't execute any of the remaining commands in this side branch, and instead, continue back on the branch that called it."  So your fwrite is being executed only when a hard drive is NOT found.  Since you do not see your added line of text output when it looks at /dev/sde1 this tells you that it is indeed bailing out at that continue command.  What I find interesting about your list of processes that are running is the fstrim command is not the same as what's in the section of script code we're looking at.  The command listed there in the processes list is "fstrim -va", which means trim all drives.  I wonder where it is getting that command from, since the one we've been looking at doesn't have -va, but only -v.  Ah yes, right at the very start of the script.

 

// cron operation
if ($argc==2 && $argv[1]=='cron') {
  // trim btrfs, xfs
  echo shell_exec("fstrim -va 2>/dev/null");
  // trim zfs
  zfs_trim(false);
  exit(0);
}

 

This checks if it is running as a cron job, and if it is then it runs "fstrim -va 2>/dev/null" which trims ALL drives, and "zfs_trim(false);" to trim zfs pools, and then exits without running the rest of the script.  So the remainder of the code where it checks for which drive types there are, and only trims SSDs, never gets executed.  The two slashes "//" at the start of those lines are an indication that whatever else follows on that line is just commentary and is ignored as far as execution goes.  So if you just add two slashes to the start of the echo line and the exit line it should start behaving.

 

// cron operation
if ($argc==2 && $argv[1]=='cron') {
  // trim btrfs, xfs
  // echo shell_exec("fstrim -va 2>/dev/null");
  // trim zfs
  zfs_trim(false);
  // exit(0);
}

 

That will stop it from running fstrim on all drives here, and will also stop it from exiting here so that the remainder of the code will run.  I think that'll do the trick.

Edited July 6, 20242 yr by _Shorty

@shorty_ Ah Ha! I think thats the key right there!

I wonder why they have it check for to see if its running as a cron job, and then have it run differently? 

Maybe thats a bug?

I will make those changes and change the schedule to run it again later tonight to see if it runs properly and report back! 

Definitely a bug, as it isn't doing what it should be doing.  I would guess that this was an oversight that was forgotten after writing and testing the rest of the script.  It was probably meant to be commented out or removed once they got the rest of the script working how they wanted, and that removal part just never happened.

35 minutes ago, _Shorty said:

Definitely a bug, as it isn't doing what it should be doing.  I would guess that this was an oversight that was forgotten after writing and testing the rest of the script.  It was probably meant to be commented out or removed once they got the rest of the script working how they wanted, and that removal part just never happened.

 

Trim just ran on the new script with the cron check commented out as shown above. 

 

Script ran as i now expected it to run, gave me output to the ssd_trim.log file specified, and skipped the spinning disks.

Looks like its fixed. Marking the above post as the solution and going to make a post in bug reports.

Thanks so much!!!!

That's good.  You're welcome.

Figured i would put the whole script here for anyone else looking to make this change. One thing to note, the origional trim function is run with an echo, which outputs to the syslog information about what was trimmed.

The later trim function has no echo statement and will not output any data to the syslog. Ive added an echo statement that outputs that information to the syslog.

The updated script:
 

#!/usr/bin/php -q
<?PHP
/* Copyright 2005-2023, Lime Technology
 * Copyright 2012-2023, Bergware International.
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License version 2,
 * as published by the Free Software Foundation.
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 */
?>
<?
$docroot = $docroot ?? $_SERVER['DOCUMENT_ROOT'] ?: '/usr/local/emhttp';

require_once "$docroot/webGui/include/Wrappers.php";
extract(parse_plugin_cfg('dynamix',true));

// cron operation
if ($argc==2 && $argv[1]=='cron') {
  // trim btrfs, xfs
  //echo shell_exec("fstrim -va 2>/dev/null");
  // trim zfs
  zfs_trim(false);
  //exit(0);
}

// add translations
$_SERVER['REQUEST_URI'] = 'settings';
$login_locale = _var($display,'locale');
require_once "$docroot/webGui/include/Translations.php";

function write(...$messages){
  $com = curl_init();
  curl_setopt_array($com,[
    CURLOPT_URL => 'http://localhost/pub/plugins?buffer_length=1',
    CURLOPT_UNIX_SOCKET_PATH => '/var/run/nginx.socket',
    CURLOPT_POST => 1,
    CURLOPT_RETURNTRANSFER => true
  ]);
  foreach ($messages as $message) {
    curl_setopt($com, CURLOPT_POSTFIELDS, $message);
    curl_exec($com);
  }
  curl_close($com);
}

function is_hdd($disk) {
  $disk = explode('/',$disk);
  $disk = preg_replace('/^(sd[a-z]+|nvme[0-9]+n1)p?1$/','$1',end($disk));
  return file_get_contents("/sys/block/$disk/queue/rotational")==1;
}

function zfs_info($name) {
  $trim = preg_replace('/(.$)/',' $1',exec("zfs list -Ho used $name"))."iB";
  $bytes = exec("zfs list -Hpo used $name");
  exec("zpool list -vHP $name|grep -Po '^\s+\K/\S+'",$devs);
  foreach ($devs as &$dev) if (is_hdd($dev)) $dev = '';
  return "/mnt/$name: $trim ($bytes bytes) trimmed on ".implode(', ',array_filter($devs));
}

function zfs_trim($write) {
  if (!file_exists('/proc/spl/kstat/zfs/arcstats')) return;
  exec("zfs list -d0 -Ho name",$pools);
  foreach ($pools as $name) {
    if ($write) {
      write("/mnt/$name: ... <i class='fa fa-spin fa-circle-o-notch'></i>\r");
      if (exec("zpool trim -w $name 2>&1")=='') write(zfs_info($name)."\r","\n"); else write("\r");
    } else {
      if (exec("zpool trim -w $name 2>&1")=='') echo zfs_info($name)."\n";
    }
  }
}

write(_("TRIM operation started")."\n","\n","\n");
echo "Trim Operation Started\n";
// trim btrfs, xfs
exec("findmnt -lnt btrfs,xfs -o target,source|awk '\$2!~\"\\\\[\"{print \$1,\$2}'",$mounts);
foreach ($mounts as $mount) {
  [$target,$source] = explode(' ',$mount);
  if (is_hdd($source)) continue;
  write("$target: ... <i class='fa fa-spin fa-circle-o-notch'></i>\r");
  $trim = exec("fstrim -v $target 2>/dev/null");
  echo $trim,"\n";		//Echo information to syslog.
  if ($trim) write("$trim on $source\r","\n"); else write("\r");
}
// trim zfs
zfs_trim(true);
write(_("Finished")."\n",'_DONE_','');
echo "Trim Operation Finished\n";
?>

 

That whole cron section is just useless, actually, considering all that remains is zfs_trim(false), and zfs_trim(true) runs at the end anyway.  No need to run it twice.

1 hour ago, _Shorty said:

That whole cron section is just useless, actually, considering all that remains is zfs_trim(false), and zfs_trim(true) runs at the end anyway.  No need to run it twice.

I agree, but i didnt know if the ZFS part needed to be there or not, so i figured id leave it alone.

 

It won't hurt to run it twice, but it isn't doing anything useful by doing so.  The whole cron section can just be commented out without worry.

On 7/4/2024 at 7:52 AM, JorgeB said:

let me see if I can find more info.

Sorry, I forgot to post an update, LT confirmed that probably due to an oversight, the "TRIM now" option and the scheduled TRIM are running slightly different scripts, this should be fixed for the next release, but probably only for v7.0, it may not get backported to v6.12

57 minutes ago, JorgeB said:

Sorry, I forgot to post an update, LT confirmed that probably due to an oversight, the "TRIM now" option and the scheduled TRIM are running slightly different scripts, this should be fixed for the next release, but probably only for v7.0, it may not get backported to v6.12

Yes, as I mentioned earlier, I would bet this was just something that was only supposed to be there temporarily until the rest of the script was worked out.  And they just forgot, probably in the middle of doing 90 different things, hehe.  Commenting that first part out fixes it, as the rest of the script does what it should when it actually gets to run.

This issue will be fixed for 7.0.0-beta.2 and possibly also backported to v6.12.11