Defuse

Members
  • Posts

    20
  • Joined

  • Last visited

Everything posted by Defuse

  1. I have an SSD cache with spindle drives behind it. Network connection is 10g. What I am observing is that if I copy movie.mkv to unraid to a folder that already has movie.mkv over SMB and choose to replace the existing file the transfer will write the file to the share disk as opposed to the cache drive if the file did not exist already. I see the performance drop very quickly during the copy in this manner. From 500/MBs on cache to 70-100/MBs on share disk. This has been happening all along and I am just now getting around to asking about it. Usually what I have been doing is removing the old files first so that I can take advantage of the 10g network and SSD write performance. Being able to just say, yeah replace the existing would be even better.
  2. Turns out it was a bad SFP. Thanks for the help JorgeB
  3. Ahh good catch. The link is only running at 1 GB. Now I need to figure out how to fix that..
  4. I recently upgraded my server to a ROG STRIX X299-E Gaming and i9-7900X. The NIC I use is an Intel 82599ES 10Gig. I also upgraded the cache disks from a 500GiB setup to a 1 TiB setup. Still SATA based SSD's though. Samsung 870s to be specific. On my previous setup I was able to copy data from my workstation that also uses 10g network at 800-900 MBps without issue. Now after my server hardware changes I am only able to achieve 100 MBps throughput. I am having a hard time identifying what could be causing this.
  5. Yep that was it. Thanks for the help!
  6. On a few of my shares when I use the "Compute" option it shows that the share has 0KB of data on some disks. In the attached photo I am expecting this result but I am curious as to why it's showing 0KB on some disks but not all. Looking at other shares I see similar results. Is this a problem? Is there a way to clean it up so that if there is no data for a share on a disk it does not show that disk?
  7. Ah bummer. I figured it was a tall ask but thought it might be worth a shot. We do this kind of stuff at work with Isilon and Unity SAN's and wanted to have something similar at home. Thanks for the info though I appreciate it.
  8. I primarily use Unraid with Plex and Jellyfin docker containers to host my media collection. My primary media share that stores the Movies and TV shows is limited to only spindle disks as the capacity is around 40TB. Jellyfin can utilize NFO files for metadata which I am wanting to leverage for my collection. However, it does not have the ability to store the NFO based metadata in a location other than where the media file is. Since the application cannot support my desired configuration I am looking for a solution at the host (Unraid) level. Essentially I want to be able to present the Media share to Jellyfin so that it can read and assume that all of the media and metadata files are in the same place but on the backend host have the media files be on spindle drives and the metadata files on an SSD share. I "think" this is something that can be achieved with some kind of symbolic link but I have never actually worked with them before so I don't know what their benefits / limitations are. As a bonus if I am able to setup this configuration I would like to be able to set the media share that stores the actual content as read-only but allow the metadata share to be read/write which would allow Jellyfin to create new NFO based metadata as content is added. My current share / folder structure looks like this. Media Movies movie.mkv movie.nfo movieart.png I "think" what I will need to do to satisfy my configuration is something like this. Media <- Share that is hosted on SSD and is allowed as Read/Write Movies <- Linked share that is a read only reference to a spindle drive share Movie.mkv <- Hosted on linked share that is read only movie.nfo <- Hosted on Movies SSD share movieart.png <-Hosted on Movies SSH share
  9. Ah that is a good idea as I do plan to put the current 18TB parity drive back in to the mix as a data drive. Thank you for the tip!
  10. I am getting ready to make some hardware changes to my disk configuration and I am looking to for some tips. I currently have a 10 disk array with a single 18TB parity drive. My end state will be a 10 disk array with two 20TB parity drives. Two disks in my array are 4TB drives that I plan to also upgrade with 20TB drives. Essentially the goal here is to get all 4 20TB drives in place without losing any data. I "think" the appropriate plan of attack is to add a single 20TB drive and assign it in the secondary parity slot. Let the array do whatever it needs to do then replace the 18TB parity drive with a 20TB parity drive, then let the array do what it needs to do to be back to a normal state. For the two data drives I presume it's safer to replace one at a time but I "think" I could replace both at this point since I now have two parity disks correct? Also, for the actual hardware connections. I am also presuming that it is a good idea to separate the parity disks across controllers for redundancy. Is there a compelling reason not do this?
  11. It shows up in Settings > Scheduler. Interestingly. I started checking this out because I had an upcoming parity check scheduled and wanted to make sure that the settings were correct as this was the first time I was using your scheduling plugin. When I originally set it up I did not see any warnings. The parity check started, there was an update to the parity check scheduler plugin which I performed while the parity check was paused because of the schedule enforcement that I had set. After the update this warning presented itself. The parity check finished later that day but I never received a notification of that until I updated the plugin again earlier today. I thought that timing was a bit odd so I thought I would share. I have also attached a screenshot of the message as I see it.
  12. Hello. After the latest update I am getting a similar Warning as I was before. The location is a bit different but wanted to share what I am getting. Warning: session_start(): Cannot start session when headers already sent in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.helpers.php on line 115 This is on version 6.9.2
  13. Yep, looks good now. Thank you for the quick fix!
  14. Thanks for the response. If it helps I am running 6.9.2.
  15. Under Parity Check Tuning in settings I am getting this warning. Warning: session_start(): Cannot start session when headers already sent in /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.helpers.php on line 117 This is lines 116-118 of the referenced file. // add translations session_start(); if (!isset($_SESSION['locale'])) { This is the entirety of the file <?PHP /* * Helper routines used by the parity.check.tining plugin * * Copyright 2019-2021, Dave Walker (itimpi). * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License version 2, * as published by the Free Software Foundation. *oig * Limetech is given explicit permission to use this code in any way they like. * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. */ // useful for testing outside Gui $docroot = $docroot ?? $_SERVER['DOCUMENT_ROOT'] ?: '/usr/local/emhttp'; $parityTuningNotify = "$docroot/webGui/scripts/notify"; require_once "$docroot/webGui/include/Helpers.php"; // Set up some useful constants used in multiple files define('PARITY_TUNING_PLUGIN', 'parity.check.tuning'); define('EMHTTP_DIR' , '/usr/local/emhttp'); define('CONFIG_DIR' , '/boot/config'); define('PLUGINS_DIR' , CONFIG_DIR . '/plugins'); define('PARITY_TUNING_EMHTTP_DIR', EMHTTP_DIR . '/plugins/' . PARITY_TUNING_PLUGIN); define('PARITY_TUNING_PHP_FILE', PARITY_TUNING_EMHTTP_DIR . '/' . PARITY_TUNING_PLUGIN . '.php'); define('PARITY_TUNING_BOOT_DIR', PLUGINS_DIR . '/' . PARITY_TUNING_PLUGIN); define('PARITY_TUNING_FILE_PREFIX', PARITY_TUNING_BOOT_DIR . '/' . PARITY_TUNING_PLUGIN . '.'); define('PARITY_TUNING_CFG_FILE', PARITY_TUNING_FILE_PREFIX . 'cfg'); define('PARITY_TUNING_LOG_FILE', PARITY_TUNING_FILE_PREFIX . 'log'); define('PARITY_TUNING_PARTIAL_FILE',PARITY_TUNING_FILE_PREFIX . 'partial'); // Create when partial check in progress (contains end sector value) define('EMHTTP_VAR_DIR' , '/var/local/emhttp/'); define('PARITY_TUNING_EMHTTP_VAR_FILE', EMHTTP_VAR_DIR . 'var.ini'); define('PARITY_TUNING_EMHTTP_DISKS_FILE', EMHTTP_VAR_DIR . 'disks.ini'); define('PARITY_TUNING_DATE_FORMAT', 'Y M d H:i:s'); $dynamixCfg = parse_ini_file('/boot/config/plugins/dynamix/dynamix.cfg', true); $parityTuningTempUnit = $dynamixCfg['display']['unit'] ?? 'C'; // Use Celsius if not set // Handle Unraid version dependencies $parityTuningUnraidVersion = parse_ini_file("/etc/unraid-version"); $parityTuningVersionOK = (version_compare($parityTuningUnraidVersion['version'],'6.7','>') >= 0); $parityTuningRestartOK = (version_compare($parityTuningUnraidVersion['version'],'6.8.3','>') > 0); // Configuration information if (file_exists(PARITY_TUNING_CFG_FILE)) { $parityTuningCfg = parse_ini_file(PARITY_TUNING_CFG_FILE); } else { $parityTuningCfg = array(); } // Set defaults for any missing/new values setCfgValue('parityTuningLogging', '0'); setCfgValue('parityTuningLogTarget', '0'); setCfgValue('parityTuningIncrements', '0'); setCfgValue('parityTuningFrequency', '0'); setCfgValue('parityTuningUnscheduled', '0'); setCfgValue('parityTuningAutomatic', '0'); setCfgValue('parityTuningRecon', '0'); setCfgValue('parityTuningClear', '0'); setCfgValue('parityTuningRestart', '0'); setCfgValue('parityTuningResumeHour', '0'); setCfgValue('parityTuningResumeMinute', '15'); setCfgValue('parityTuningPauseHour', '3'); setCfgValue('parityTuningPauseMinute', '30'); setCfgValue('parityTuningResumeCustom', '15 0 * * *'); setCfgValue('parityTuningPauseCustom', '30 3 * * *'); setCfgValue('parityTuningHeat', '0'); setCfgValue('parityTuningHeatHigh','3'); setCfgValue('parityTuningHeatLow','8'); setCfgValue('parityTuningHeatShutdown', '0'); setCfgValue('parityTuningHeatCritical', '1'); setCfgValue('parityProblemType', 'sector'); setCfgValue('parityProblemStartSector', 0); setCfgValue('parityProblemStartPercent', 0); setCfgValue('parityProblemEndSector', 100); setCfgValue('parityProblemEndPercent', 0); setCfgValue('parityProblemCorrect', 'no'); $parityTuningCLI = (basename($argv[0]) == 'parity.check'); parityTuningLoggerTesting("CLI Mode: $parityTuningCLI"); // Set a value if not already set for the configuration file // ... and set a variable of the same name to the current value function setCfgValue ($key, $value) { $cfgFile = $GLOBALS['parityTuningCfg']; if (! array_key_exists($key,$cfgFile)) { $cfgFile[$key] = $value; } else { if (empty($cfgFile[$key]) || $cfgFile[$key]== ' ' ) { $cfgFile[$key] = $value; } // Next 2 lines handle migration of settings to new values - will be removed in future release. if ($cfgFile[$key] == "no") $cfgFile[$key] = 0; if ($cfgFile[$key] == "yes") $cfgFile[$key] = 1; if ($cfgFile[$key] == "daily") $cfgFile[$key] = 0; if ($cfgFile[$key] == "custom") $cfgFile[$key] = 1; } $GLOBALS['parityTuningCfg'][$key] = $cfgFile[$key]; // TODO: Not sure thir is actually needed any more $GLOBALS[$key] = $cfgFile[$key]; } // Multi-Language support code always required $plugin = 'parity.check.tuning'; $docroot = $docroot ?: $_SERVER['DOCUMENT_ROOT'] ?: '/usr/local/emhttp'; $translations = file_exists("$docroot/webGui/include/Translations.php"); if ($translations) { // add translations session_start(); if (!isset($_SESSION['locale'])) { parityTuningLoggerTesting("setting locale from dynamix setting"); $_SESSION['locale'] = $dynamixCfg['display']['locale']; } $_SERVER['REQUEST_URI'] = 'paritychecktuning'; require_once "$docroot/webGui/include/Translations.php"; parse_plugin('paritychecktuning'); parityTuningLoggerTesting("Multi-Language support active, locale: " . $_SESSION['locale']); } else { $noscript = true; // legacy support (without JavaScript) require_once "$docroot/plugins/parity.check.tuning/Legacy.php"; parityTuningLoggerTesting('Legacy Language support active'); } if (file_exists(PARITY_TUNING_EMHTTP_DISKS_FILE)) { $disks=parse_ini_file(PARITY_TUNING_EMHTTP_DISKS_FILE, true); $parityTuningNoParity = ($disks['parity']['status']=='DISK_NP_DSBL') && ($disks['parity2']['status']=='DISK_NP_DSBL'); } // load some state information. // (written as a function to facilitate reloads) function loadVars($delay = 0) { if ($delay > 0) { parityTuningLoggerTesting ("loadVars($delay)"); sleep($delay); } if (! file_exists(PARITY_TUNING_EMHTTP_VAR_FILE)) { // Protection against running plugin while system initializing so this file not yet created parityTuningLoggerTesting(sprintf('Trying to populate before %s created so ignored', PARITY_TUNING_EMHTTP_VAR_FILE)); return; } $vars = parse_ini_file(PARITY_TUNING_EMHTTP_VAR_FILE); $size = $vars['mdResyncSize']; $pos = $vars['mdResyncPos']; $GLOBALS['parityTuningVar'] = $vars; $GLOBALS['parityTuningCsrf'] = $vars['csrf_token']; $GLOBALS['parityTuningServer'] = strtoupper($vars['NAME']); $GLOBALS['parityTuningPos'] = $pos; $GLOBALS['parityTuningSize'] = $size; $GLOBALS['parityTuningAction'] = $vars['mdResyncAction']; $GLOBALS['parityTuningActive'] = ($pos > 0); // If array action is active (paused or running) $GLOBALS['parityTuningRunning'] = ($vars['mdResync'] > 0); // If array actimb on is running (i.e. not paused) $GLOBALS['parityTuningCorrecting'] = $vars['mdResyncCorr']; $GLOBALS['parityTuningErrors'] = $vars['sbSyncErrs']; } // Get the long text description of an array operation function actionDescription($action, $correcting) { $act = explode(' ', $action ); switch ($act[0]) { case 'recon': // TODO use extra array entries to decide if disk rebuild in progress or merely parity sync return _('Parity Sync') . '/' . _('Data Rebuild'); case 'clear': return _('Disk Clear'); case 'check': if (count($act) == 1) return _('Read-Check'); return (($correcting == 0) ? _('Non-Correcting') : _('Correcting')) . ' ' . _('Parity Check'); default: return sprintf('%s: %s',_('Unknown action'), $action); } } // test if partial parity check in progress // ~~~~~~~~~~~~~~~~~~~ function parityTuningPartial() { // ~~~~~~~~~~~~~~~~~~~ return file_exists(PARITY_TUNING_PARTIAL_FILE); } // Logging functions // Write message to syslog and also to console if in CLI mode // Change source according to whether doing partial check or not function parityTuningLogger($string) { $logTarget = $GLOBALS['parityTuningLogTarget']; parityTuningLoggerCLI ($string); $logName = parityTuningPartial() ? "Parity Problem Assistant" : "Parity Check Tuning"; if ($logTarget > 0) { $line = date(PARITY_TUNING_DATE_FORMAT) . ' ' . $GLOBALS['parityTuningServer'] . " $logName: $string\n"; file_put_contents(PARITY_TUNING_LOG_FILE, $line, FILE_APPEND | LOCK_EX); } $string = str_replace("'","",$string); if ($logTarget < 2) { $cmd = 'logger -t "' . $logName . '" "' . $string . '"'; shell_exec($cmd); } } // Write message to syslog if debug or testing logging active function parityTuningLoggerDebug($string) { if ($GLOBALS['parityTuningLogging'] > 0) parityTuningLogger('DEBUG: ' . $string); } // Write message to syslog if testing logging active function parityTuningLoggerTesting($string) { if ($GLOBALS['parityTuningLogging'] > 1) parityTuningLogger('TESTING: ' . $string); } function parityTuningLoggerCLI($string) { if ($GLOBALS['parityTuningCLI']) echo $string . "\n"; } // Useful matching functions function startsWith($haystack, $beginning, $caseInsensitivity = false){ if ($caseInsensitivity) return strncasecmp($haystack, $beginning, strlen($beginning)) === 0; else return strncmp($haystack, $beginning, strlen($beginning)) === 0; } function endsWith($haystack, $ending, $caseInsensitivity = false){ if ($caseInsensitivity) return strcasecmp(substr($haystack, strlen($haystack) - strlen($ending)), $haystack) === 0; else return strpos($haystack, $ending, strlen($haystack) - strlen($ending)) !== false; } ?>
  16. Yep. Reformatting the cache pool as encrypted took care of this. I am glad it was something obvious. Thanks everyone for the help.
  17. I suspected that might be the cause. Is the cache pool not being encrypted intentional/by design or is that something that I missed doing on initial setup? If the cache pool cannot be encrypted is there a best practice recommendation for dealing with that?
  18. Attached prism-diagnostics-20210427-0740.zip
  19. One of my shares is showing that not all of the files in the share are encrypted. I have not been able to find a way to determine which files are not encrypted or why they are not getting encrypted. This share does use a cache pool if that is relevant. Any pointers on how I can identify what is going on here?