Update Smartctl Database

[nickp85]

Is there any way to update smartctl manually? I've got 6.12.10 and it shows as smartctl 7.3/5601 however a brand new spinning disk I put in today says it is not in the smartctl database. It's a newer Seagate Ironwolf Pro ST16000NT001. I have 2 other Ironwolf Pro 16TB but they are detected, looks like Seagate made a revision change from ST16000NE000 to ST16000NT001 for the same drives.

 

Appears there is a Smartctl 7.4 dated August 2023. Other than version update, is there some way to update the database to detect newer drives?

[nickp85]

I found out how to update the database but when refreshing the GUI, the drive is still not in there

 

root@nicknas2:~# update-smart-drivedb
/usr/sbin/update-smart-drivedb: gpg: not found in '/usr/local/bin:/usr/bin:/bin' ('--no-verify' to ignore)
Try '/usr/sbin/update-smart-drivedb -h' for help
root@nicknas2:~# update-smart-drivedb --no-verify
/usr/share/smartmontools/drivedb.h 7.3/5601 updated to 7.3/5610 (NOT VERIFIED)

 

Not in smartctl database 7.3/5610

[bmartino1]

? not aware of any issues with theses drives and smartmon.

Looks more like your tying to update the unraid installed package.
https://slackware.pkgs.org/15.0/slackware-x86_64/smartmontools-7.2-x86_64-4.txz.html

Per Nerd Tools Plugin:
Unraid terminal:
cd /boot/extra
curl/wget the application and unzip the application....

reboot? - should have package.

Does the drive show up under main?

click the drive go to self test and click short test:


the database is empty as their is nothing on the disk to read or a test that would show any data?...

[nickp85]

The drive is there, and curiously enough is showing SMART info. The short test also passed. This is a new drive so it's currently running pre-clear. Could that be what's messing it up?

[nickp85]

1 hour ago, bmartino1 said:

? not aware of any issues with theses drives and smartmon.

Looks more like your tying to update the unraid installed package.
https://slackware.pkgs.org/15.0/slackware-x86_64/smartmontools-7.2-x86_64-4.txz.html

Per Nerd Tools Plugin:
Unraid terminal:
cd /boot/extra
curl/wget the application and unzip the application....

 

I downloaded the updated version here

https://slackware.pkgs.org/current/slackware-x86_64/smartmontools-7.4-x86_64-1.txz.html

I then tried to update it using upgradepkg --install-new smartmontools-7.4-x86_64-1.txz

 

It says I already have 7.4 but the Unraid GUI is showing 7.3

+==============================================================================
| Skipping package smartmontools-7.4-x86_64-1 (already installed)
+==============================================================================

[bmartino1]

1 hour ago, nickp85 said:

The drive is there, and curiously enough is showing SMART info. The short test also passed. This is a new drive so it's currently running pre-clear. Could that be what's messing it up?

IF the parity / format / etc is not done with it the drive is not fully mounted and ready to use as such the tools would not see it.

[nickp85]

2 hours ago, bmartino1 said:

IF the parity / format / etc is not done with it the drive is not fully mounted and ready to use as such the tools would not see it.

got it, I'll check it once the clear finishes tomorrow. Thank you!

[nickp85]

17 hours ago, bmartino1 said:

IF the parity / format / etc is not done with it the drive is not fully mounted and ready to use as such the tools would not see it.

still not in database with the drive in the array and formatted now

[bmartino1]

and you have rebooted after a completion?

 

[nickp85]

26 minutes ago, bmartino1 said:

and you have rebooted after a completion?

 

yes indeed, I've opened a ticket with smartctl as I don't think this model is in their database

[bmartino1]

per other post on form have you tried:

 

/usr/sbin/update-smart-drivedb

 

^ a Bitr dated and may not work but last thing i know of. even if smartctrl is not working, that done't mean smart on the drive isn't there. Which is why I asked if it was showing anything in the smart tab under main. There are mutiple smartmon tools to get this data.

Correct the next steps would to have the devs on smartctrl look into the HD type and model you have as the manufacture may have moved or changed how values are read.

smart exist on the physical disk and is not Manually writable. we can add another entry to smart and read previous smart data. this is done for 3rd party and re-used disk to check its age and if it still functions.

[urbaud]

On 5/28/2024 at 4:36 PM, nickp85 said:

yes indeed, I've opened a ticket with smartctl as I don't think this model is in their database

Did you ever get a resolution on this? I have a similar situation with two ironwolf pro 8TB drives in a Thunderbay 4 DAS. I can see the smart reports from the command line but not from unraid.

[nickp85]

On 9/9/2024 at 9:16 PM, urbaud said:

Did you ever get a resolution on this? I have a similar situation with two ironwolf pro 8TB drives in a Thunderbay 4 DAS. I can see the smart reports from the command line but not from unraid.

Same issue, newer model number not in smartctl database. Ticket hasn't gone anywhere

[urbaud]

How are your drives connected to your server? I was wondering whether it might be a problem with Thunderbolt 4 or my Thunderbay enclosure.

Where did you submit your ticket? I'll put one in also.

[bmartino1]

Just to confirm.

You made a ticket here:
https://www.smartmontools.org/report

as this is not a unraid issues, its a smartmon issue...

also, there have been updates and changes on how smart is done for non ata devices.

Thunder port / usb3 attached storage and how smart is done.

Review:

https://eclecticlight.co/2022/04/09/explainer-s-m-a-r-t-and-disk-health/

https://mjtsai.com/blog/2022/04/12/mac-s-m-a-r-t-support-usb-c-vs-thunderbolt/

https://kb.plugable.com/checking-drive-smart-values-in-linux

https://www.linuxquestions.org/questions/slackware-14/thunderbolt-control-is-there-a-slackbuild-for-bolt-boltd-or-similar-4175713129/
 

Mby beta 7 and a kernel addon is needed for theses..
"any third-party utility to be able to monitor storage connected by USB, the SAT SMART kernel extension has to be installed"

Remember, unraid is build on Slackware Linux...

This may also be a Linux OS issues with new technology... which what I have tested with, I don't see a problem with unassigned disk and thunderbolt/usb3 attached storage. 

How are you adding the disk to the system?

[urbaud]

I am using an AMD-based x86 Mini PC with a thunderbolt 4 Port, not a Mac. Unraid sees the discs just fine, and they are part of an array that is running fine. I can see the smart reports from the Linux command line, but not in unraid. 

 

The question is: how do I make unraid see the smart capability of the discs?

[SimonF]

10 minutes ago, urbaud said:

I am using an AMD-based x86 Mini PC with a thunderbolt 4 Port, not a Mac. Unraid sees the discs just fine, and they are part of an array that is running fine. I can see the smart reports from the Linux command line, but not in unraid. 

 

The question is: how do I make unraid see the smart capability of the discs?

Do you have to use options for device type from command line?

 

If you click on disk you can scroll down to smart values look at controller values

[urbaud]

I am running unraid as a VM under Proxmox. After a couple of hours with GPT-o1 today, I figured out that I have to pass through the drives as SATA instead of SCSI. That fixed it.

[bmartino1]

On 9/15/2024 at 8:25 PM, urbaud said:

I am running unraid as a VM under Proxmox. After a couple of hours with GPT-o1 today, I figured out that I have to pass through the drives as SATA instead of SCSI. That fixed it.

Knowing that would have helped...

please see forum post:

It is better to use a HBA and not pass disk by id... this has to do with how Proxmox sees the disk and how the VM seees the disk. They should not be able to interact with each other... This way Unraid has full access to the disk and the data...

[urbaud]

As a reminder, I'm using a thunderbolt connection to the discs, so I'm not sure HBA applies. Here's what GPT-o1 says about passing through the discs as SATA:

================================
 

Since you're passing the disks through to Unraid and not using ZFS, and SMART monitoring is working, that's great news. Your current setup appears to be effective, and there are no significant disadvantages if everything is functioning as intended.

 

Advantages of Your Current Setup

Direct Disk Access:

• Unraid Compatibility: Passing the disks directly to Unraid allows it to manage the disks as if it were running on bare metal. This is ideal because Unraid's storage system benefits from direct access to individual disks.
• Flexibility: Unraid can use its proprietary parity and storage management features without interference from the host.

SMART Monitoring:

• Disk Health Monitoring: Having SMART data accessible within Unraid enables you to monitor the health of your disks, predict failures, and take proactive measures.
• Maintenance: You can perform disk checks, schedule maintenance, and receive alerts directly within Unraid.

Performance:

• Efficiency: Passing through disks at the hardware level minimizes virtualization overhead, potentially offering near-native performance.
• Simplicity: This method avoids the complexity of setting up and managing additional layers like ZFS on the host.

No Significant Disadvantages

Given that:

• SMART Monitoring Works: You have full visibility into disk health.
• Performance Meets Needs: There are no performance bottlenecks impacting your use case.
• Stability Is Maintained: The system operates reliably without crashes or data corruption.

There are no significant disadvantages to your current approach. Passing the disks directly to Unraid is a common practice and is supported by both Proxmox and Unraid communities.

 

Conclusion

Your current method of passing the disks through to Unraid as SATA devices is effective and appropriate for your needs. Since SMART monitoring is operational and you're not experiencing any issues, there's no inherent disadvantage to this setup.

[bmartino1]

On 9/18/2024 at 11:52 AM, urbaud said:

As a reminder, I'm using a thunderbolt connection to the discs, so I'm not sure HBA applies. Here's what GPT-o1 says about passing through the discs as SATA:

================================
 

Since you're passing the disks through to Unraid and not using ZFS, and SMART monitoring is working, that's great news. Your current setup appears to be effective, and there are no significant disadvantages if everything is functioning as intended.

 

Advantages of Your Current Setup

Direct Disk Access:

• Unraid Compatibility: Passing the disks directly to Unraid allows it to manage the disks as if it were running on bare metal. This is ideal because Unraid's storage system benefits from direct access to individual disks.
• Flexibility: Unraid can use its proprietary parity and storage management features without interference from the host.

SMART Monitoring:

• Disk Health Monitoring: Having SMART data accessible within Unraid enables you to monitor the health of your disks, predict failures, and take proactive measures.
• Maintenance: You can perform disk checks, schedule maintenance, and receive alerts directly within Unraid.

Performance:

• Efficiency: Passing through disks at the hardware level minimizes virtualization overhead, potentially offering near-native performance.
• Simplicity: This method avoids the complexity of setting up and managing additional layers like ZFS on the host.

No Significant Disadvantages

Given that:

• SMART Monitoring Works: You have full visibility into disk health.
• Performance Meets Needs: There are no performance bottlenecks impacting your use case.
• Stability Is Maintained: The system operates reliably without crashes or data corruption.

There are no significant disadvantages to your current approach. Passing the disks directly to Unraid is a common practice and is supported by both Proxmox and Unraid communities.

 

Conclusion

Your current method of passing the disks through to Unraid as SATA devices is effective and appropriate for your needs. Since SMART monitoring is operational and you're not experiencing any issues, there's no inherent disadvantage to this setup.

AI is not definitive and doesn't use know anything outside of its trained data... It is incapable of a test and results vary on data input. use AI with a grain of salt and confirm it.

The problem. Proxmox can still touch the disk and will. when it does you have 2 OS touch disks at the same time. On Unraid its a problem as you lose the drive spin down features sets and other that you may or may not have been using. There is performance hits on disk drives running over usb/thuderbolt stopped vs scsi/sata speed..

I would strongly recommend you use a HBA to remove proxmox host abilty to touch the disks used in a VM. passing it via disk by id on other OS and how you have it setup is not meant for long term. even windows ntfs and prxomox have known issues ... to learn more on this ask and talk on proxmox form. 

Examples:

https://forum.proxmox.com/threads/disk-passthrough-proper-guide-on-how-to-passthrough-disks-to-vms.145759/

 

https://forum.proxmox.com/threads/proxmox-omv-hd-passthrough.135260/

Having tested this and used other tools while possible I don't recomend what your doing... I will only warn and say that this is a bad idea and you will kill that disk faster then its normal mean time as you have doubled reads from 2 OS. and can potenai kill the disk with how smart reports to it.

look into proxmox smart test and potency disabling it on thoese drives at the host level... let unraid handle smart as one example...

[urbaud]

How does it kill the disc?

HBA is not an option because I am running on a Mini PC with these specifications.

 

Proxmox Server Specification: Minisforum EM680

Processor

AMD Ryzen™ 7 6800U

8 Cores / 16 Threads

16M L3 cache, up to 4.7 GHz

Graphics

AMD Radeon™ 680M

Graphics Frequency: 2200 MHz

Memory

LPDDR5 Dual Channel (Onboard)

16GB, 6400 MHz

Storage

M.2 2230 PCIe 4.0 SSD ×1 (up to 2TB)

Storage Expansion

TF Card Slot ×1

Wireless Connectivity

Wi-Fi 6E, Bluetooth 5.3

Video Output

HDMI 2.1 (4K @ 60Hz) ×1

USB4 (8K @ 60Hz) ×2

Audio Output

HDMI 2.1 ×1

Audio Jack ×1

Ports & Buttons

USB 3.2 Gen2 Type-A Port ×3

USB4 Port ×2 (Alt PD)

TF Card Slot ×1

HDMI 2.1 ×1

Audio Jack ×1

DMIC ×1

Clear CMOS ×1

Power

65W GaN Type-C Power Delivery Adapter (Included)

Product Dimensions

80 × 80 × 43 mm

Edited September 20 by urbaud

[bmartino1]

the host proxmox make disk checks. one of these disk checks and scheduling also preforms a smart test. if you pass a disk by id from proxmox. The host Proxmox is still doing these disk checks. When the Virtual machine (similar to unraid, truenas and others that affect the Disk. they conflict and can write malformed data as the host wrote something while the vm couldn't flag in it. And vise versa, the vm did something and the host flagged it. look into other linux kernel options such as noatime. This is an example that is now duplicated. 
You have doubled your overall reads and writes. 

unraid web ui test. no dockers running, no vms, no lxc just the disk array started and samba with no connection.
Look at your disk reads and writes. they are constantly going up due to proxmox host checks and claims to the disk. This also cause extra strain on the disk and hurts performance. As unraid is also trying to spin down the disk...

There are tool you can run at the host and vm level to check... Average disk MTF is 3-5 years, overall time is cut in half and shorted as your setup is making unnecessary writes by default. Go look on proxmox forum and find all the settings... Unraid Devs have made it clear that they don't want their OS virtualized... some are bugs caused by the QEMU and how proxmox is default some may have been fixed by hook scripts... 

You will need to heavily edit proxmox on disk stuff and some settings on the VM... if you are unable to pass them any other way....
So lets look at VM proxmox usb/thunder side, as there may have been fixed... As I stopped when proxmox 6 became EOL and 8.1 was bleeding edge. They may have fixed some but I know the default configuration on these disk checks are on by default!... SMART is just one of many disk checks. (Especially more so if you are using ZFS passed this way)
Read full post here:

As I already explained most of it with another... 

The goal is to not have Proxmox touch the disk after a disk by id pass is made... HBA are one example. 
With thunderbolt you may need to look into passing the entire port(The entire PCI bus for it...) not just the disk to stop some of the checks. In proxmox with the VM on YOU SHOULD NOT SEE YOUR THUDERPORT DISK in proxmox web UI!... if you do this is a clear sign that your killing your disk and you need to look into it... (this is one of many checks...) you could be running hock script IDK... Again theses are better asking on proxmox form...

 

 

 

Edited September 21 by bmartino1
spelling / grammar

[urbaud]

Quote

Look at your disk reads and writes. they are constantly going up due to proxmox host checks and claims to the disk. This also cause extra strain on the disk and hurts performance. As unraid is also trying to spin down the disk...

There are tool you can run at the host and vm level to check.

What tools are you referring to?

[bmartino1]

PM...