[PLUGIN] Disk Location


Recommended Posts

5 minutes ago, Trites said:

I'm getting the same blank screen.

I removed the plugin, and removed the database but still getting a blank screen.

Running Version: 6.7.1-rc1 

 

Also the icon doesn't seem to load in the plugins screen and is using the default green "Puzzle Piece" icon.

If you open the terminal and type: 

ls -la /usr/local/emhttp/plugins/disklocation/*

Do you see anything there at all?

 

Expected results:

root@Odin:~# ls -la /usr/local/emhttp/plugins/disklocation/*
-rw-rw-rw- 1 root root 140 May 21 18:29 /usr/local/emhttp/plugins/disklocation/README.md
-rw-rw-rw- 1 root root 143 May 22 13:40 /usr/local/emhttp/plugins/disklocation/disklocation.page
-rw-rw-rw- 1 root root 124 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_config.page
-rw-rw-rw- 1 root root 268 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_devices.page
-rw-rw-rw- 1 root root 113 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_help.page
-rw-rw-rw- 1 root root 120 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_info.page

/usr/local/emhttp/plugins/disklocation/icons:
total 24
drwxrwxrwx 2 root root 160 May 21 18:29 ./
drwxrwxrwx 5 root root 220 May 21 18:29 ../
-rw-rw-rw- 1 root root 414 May 21 18:29 add.png
-rw-rw-rw- 1 root root 412 May 21 18:29 configuration.png
-rw-rw-rw- 1 root root 374 May 21 18:29 delete.png
-rw-rw-rw- 1 root root 359 May 21 18:29 devices.png
-rw-rw-rw- 1 root root 352 May 21 18:29 help.png
-rw-rw-rw- 1 root root 281 May 21 18:29 information.png

/usr/local/emhttp/plugins/disklocation/images:
total 4
drwxrwxrwx 2 root root  60 May 21 18:29 ./
drwxrwxrwx 5 root root 220 May 21 18:29 ../
-rw-rw-rw- 1 root root 661 May 21 18:29 disklocation.png

/usr/local/emhttp/plugins/disklocation/pages:
total 92
drwxrwxrwx 4 root root   220 May 21 18:29 ./
drwxrwxrwx 5 root root   220 May 21 18:29 ../
-rw-rw-rw- 1 root root 19119 May 21 18:29 config.php
-rw-rw-rw- 1 root root 11689 May 21 18:29 devices.php
-rw-rw-rw- 1 root root  5812 May 21 18:29 help.php
-rw-rw-rw- 1 root root  9276 May 21 18:29 info.php
-rw-rw-rw- 1 root root   527 May 21 18:29 locate.php
drwxrwxrwx 2 root root    80 May 21 18:29 script/
-rw-rw-rw- 1 root root  9234 May 21 18:29 sqlite_tables.php
drwxrwxrwx 2 root root   120 May 21 18:29 styles/
-rw-rw-rw- 1 root root 23302 May 21 18:29 system.php
root@Odin:~# 

 

Link to comment

I think I found the problem, and just because the power lines was shut down today Unraid had to be shutdown as well, I can check the Unraid from a clean state:

 

According to https://forums.unraid.net/topic/80001-unraid-os-version-67-available/ git is supposed to be included by default in Unraid 6.7.0 in the base operating system, and is only installed by 6.6.9 and earlier version of disklocation plugin.

 

But:

root@Odin:~# git help
-bash: git: command not found

 

I dunno if they forgot to include git? @limetech

 

 

Meanwhile you can install git from the NerdPack plugin until I know what's the idea here.

Link to comment

Update 2019.05.23

  • Commit #72 - BUG: Blank screen after (re-)installing plugin, git not included after all in 6.7.0/6.7.1-rc1. Will install git with the intended version before installing the plugin.

 

This will fix the problem with Unraid 6.7.0 and 6.7.1 only. I will watch future releases until I get final answer about what's the status of git in the base system.

Edited by olehj
Added info
Link to comment
8 hours ago, ijuarez said:

So you have to install git from the nerd pack before you install the a plug-in is that correct.

Sent from my SM-N960U using Tapatalk
 

Not anymore, I updated the last installation files to include the installation of git. So far git 2.21.0 enabled for 6.7.0 and 6.7.1, and 2.14.x for the 6.6 series. Newer versions will be enabled later until I know what's going on with the base OS.

Link to comment
On 5/22/2019 at 6:56 PM, olehj said:

If you open the terminal and type: 


ls -la /usr/local/emhttp/plugins/disklocation/*

Do you see anything there at all?

 

Expected results:


root@Odin:~# ls -la /usr/local/emhttp/plugins/disklocation/*
-rw-rw-rw- 1 root root 140 May 21 18:29 /usr/local/emhttp/plugins/disklocation/README.md
-rw-rw-rw- 1 root root 143 May 22 13:40 /usr/local/emhttp/plugins/disklocation/disklocation.page
-rw-rw-rw- 1 root root 124 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_config.page
-rw-rw-rw- 1 root root 268 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_devices.page
-rw-rw-rw- 1 root root 113 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_help.page
-rw-rw-rw- 1 root root 120 May 21 18:29 /usr/local/emhttp/plugins/disklocation/disklocation_info.page

/usr/local/emhttp/plugins/disklocation/icons:
total 24
drwxrwxrwx 2 root root 160 May 21 18:29 ./
drwxrwxrwx 5 root root 220 May 21 18:29 ../
-rw-rw-rw- 1 root root 414 May 21 18:29 add.png
-rw-rw-rw- 1 root root 412 May 21 18:29 configuration.png
-rw-rw-rw- 1 root root 374 May 21 18:29 delete.png
-rw-rw-rw- 1 root root 359 May 21 18:29 devices.png
-rw-rw-rw- 1 root root 352 May 21 18:29 help.png
-rw-rw-rw- 1 root root 281 May 21 18:29 information.png

/usr/local/emhttp/plugins/disklocation/images:
total 4
drwxrwxrwx 2 root root  60 May 21 18:29 ./
drwxrwxrwx 5 root root 220 May 21 18:29 ../
-rw-rw-rw- 1 root root 661 May 21 18:29 disklocation.png

/usr/local/emhttp/plugins/disklocation/pages:
total 92
drwxrwxrwx 4 root root   220 May 21 18:29 ./
drwxrwxrwx 5 root root   220 May 21 18:29 ../
-rw-rw-rw- 1 root root 19119 May 21 18:29 config.php
-rw-rw-rw- 1 root root 11689 May 21 18:29 devices.php
-rw-rw-rw- 1 root root  5812 May 21 18:29 help.php
-rw-rw-rw- 1 root root  9276 May 21 18:29 info.php
-rw-rw-rw- 1 root root   527 May 21 18:29 locate.php
drwxrwxrwx 2 root root    80 May 21 18:29 script/
-rw-rw-rw- 1 root root  9234 May 21 18:29 sqlite_tables.php
drwxrwxrwx 2 root root   120 May 21 18:29 styles/
-rw-rw-rw- 1 root root 23302 May 21 18:29 system.php
root@Odin:~# 

 

My screen matches what I see above from the ls command, but I still get a blank screen.

Link to comment
3 hours ago, smburns25 said:

My screen matches what I see above from the ls command, but I still get a blank screen.

Did you update to 2019.05.23?

If you still have a blank screen after this, then I suspect there's some oddities in the SMART data of one or more of the drives you have (at least for the readout in the plugin). Or they respond extremely slowly hence the timeout of the script.

I will add a debugging switch with some outputs which might help later when I get some time.

You can also paste your output from this, if this command returns "1" (run in terminal):

ls /boot/config/plugins/disklocation/disklocation.sqlite | wc -l

Then read disks database and paste the output:

sqlite3 /boot/config/plugins/disklocation/disklocation.sqlite "select id,device,devicenode,luname,model_name,smart_status,hash from disks WHERE status IS NULL";

 

Link to comment
17 hours ago, smburns25 said:

I ran the first command and it returned a 1.  I ran the query and it just returned me to the prompt.  I am assuming that a NULL status not returning is a good thing for the query, but not so good for solving my issue.

For sure there's something blocking the read from smartctl to inserting the parsed data into the database.

 

smartctl --version

And this reports version 7.0 I hope?

 

But yeah, just give me some time and I will add some debugging into the plugin. The upcoming days are a bit busy, so it will take some time until it's ready.

 

Meanwhile, do you have any drives lacking a serial number, or duplicate serial numbers? Or does some of the smart data contain any weird characters?

 

Check the simple info with each drive, maybe there's a drive corrupting the read/write in the plugin.

 

smartctl -i /dev/sdX

 

Link to comment

Update 2019.06.03

  • Commit #74 - IMPROVEMENT: Add debugging tools.

 

Edit system.php and enable debugging, in terminal:

nano /usr/local/emhttp/plugins/disklocation/pages/system.php

set "$debugging_active = 1;" (remove // in front of the variable for debugging).

From:

        // Comment out to enable debugging:
        //$debugging_active = 1;
...

To:

        // Comment out to enable debugging:
        $debugging_active = 1;
...

 

The output will be shown directly and stored appended in /boot/config/plugins/disklocation/debugging.html

 

@smburns25 You can try to enable the debugging and send me the debugging.html file after ~5 minutes. Hopefully something is stored in that file, the error messages should be displayed as they tick in, but there's some cache stuff going on with Unraid so it doesn't work as I want it to. But if it creates something in the debugging file, then hopefully that might help me solve the problem of yours.

 

Edited by olehj
added instructions for enabling the debugging
Link to comment
  • 2 weeks later...

First, thank you for your hard work! This looks very promising!

 

I'm running 6.6.7. I'm using default settings for the plugin after just installing it. I'm configuring the device locations and I'm experiencing very long save times (30-60s). Sometimes it seems to timeout and result in an empty white page. If I then immediately refresh the settings page an error for SQLite for being locked is being displayed. After a short while (~1 minute) the configuration page refreshes and the settings have seem to be saved properly.

Warning: SQLite3::querySingle(): Unable to execute statement: database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 258

Warning: SQLite3::exec(): database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 288
database is locked
Warning: SQLite3::exec(): cannot start a transaction within a transaction in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 321
cannot start a transaction within a transaction
Warning: SQLite3::query(): Unable to prepare statement: 5, database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 329

Fatal error: Uncaught Error: Call to a member function fetchArray() on boolean in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php:330 Stack trace: #0 /usr/local/emhttp/plugins/disklocation/pages/system.php(53): require_once() #1 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(418) : eval()'d code(2): require_once('/usr/local/emht...') #2 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(418): eval() #3 /usr/local/emhttp/plugins/dynamix/template.php(62): require_once('/usr/local/emht...') #4 {main} thrown in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 330

It seems like the SQLite is working very slowly and the above error is just the original save query still running and locking the db. 

 

Where is this SQLite database stored? Any ideas what could be causing this very slow save?

 

Edit: Also the initial loading of the plugin's front page (devices) takes a long time. Switching to other "tabs" is immediate.

Edited by henris
Link to comment

Wanted to install today, but wihle downloading I get this error:

plugin: installing: https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg
plugin: downloading https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg
plugin: downloading: https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg ... done
plugin: skipping: smartmontools-7.0-x86_64-1.txz - Unraid version too high, requires at most version 6.6.9
plugin: skipping: git-2.14.5-x86_64-1_slack14.2.txz - Unraid version too high, requires at most version 6.6.9
plugin: downloading: https://mirrors.slackware.com/slackware/slackware64-current/slackware64/d/git-2.21.0-x86_64-1.txz ... failed (Invalid URL / Server error response)
plugin: wget: https://mirrors.slackware.com/slackware/slackware64-current/slackware64/d/git-2.21.0-x86_64-1.txz download failure (Invalid URL / Server error response)

Updating Support Links

I'm on Nvidia unRAID 6.7

Link to comment
8 hours ago, abb0r said:

Wanted to install today, but wihle downloading I get this error:


plugin: installing: https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg
plugin: downloading https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg
plugin: downloading: https://raw.githubusercontent.com/olehj/disklocation/master/disklocation-master.plg ... done
plugin: skipping: smartmontools-7.0-x86_64-1.txz - Unraid version too high, requires at most version 6.6.9
plugin: skipping: git-2.14.5-x86_64-1_slack14.2.txz - Unraid version too high, requires at most version 6.6.9
plugin: downloading: https://mirrors.slackware.com/slackware/slackware64-current/slackware64/d/git-2.21.0-x86_64-1.txz ... failed (Invalid URL / Server error response)
plugin: wget: https://mirrors.slackware.com/slackware/slackware64-current/slackware64/d/git-2.21.0-x86_64-1.txz download failure (Invalid URL / Server error response)

Updating Support Links

I'm on Nvidia unRAID 6.7

Looks like git has changed the URL or version from the current tree. I will look into this soon. Meanwhile you can try to install "git" via NerdPack plugin if you have it installed.

Link to comment
On 6/14/2019 at 8:01 PM, henris said:

First, thank you for your hard work! This looks very promising!

 

I'm running 6.6.7. I'm using default settings for the plugin after just installing it. I'm configuring the device locations and I'm experiencing very long save times (30-60s). Sometimes it seems to timeout and result in an empty white page. If I then immediately refresh the settings page an error for SQLite for being locked is being displayed. After a short while (~1 minute) the configuration page refreshes and the settings have seem to be saved properly.


Warning: SQLite3::querySingle(): Unable to execute statement: database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 258

Warning: SQLite3::exec(): database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 288
database is locked
Warning: SQLite3::exec(): cannot start a transaction within a transaction in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 321
cannot start a transaction within a transaction
Warning: SQLite3::query(): Unable to prepare statement: 5, database is locked in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 329

Fatal error: Uncaught Error: Call to a member function fetchArray() on boolean in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php:330 Stack trace: #0 /usr/local/emhttp/plugins/disklocation/pages/system.php(53): require_once() #1 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(418) : eval()'d code(2): require_once('/usr/local/emht...') #2 /usr/local/emhttp/plugins/dynamix/include/DefaultPageLayout.php(418): eval() #3 /usr/local/emhttp/plugins/dynamix/template.php(62): require_once('/usr/local/emht...') #4 {main} thrown in /usr/local/emhttp/plugins/disklocation/pages/sqlite_tables.php on line 330

It seems like the SQLite is working very slowly and the above error is just the original save query still running and locking the db. 

 

Where is this SQLite database stored? Any ideas what could be causing this very slow save?

 

Edit: Also the initial loading of the plugin's front page (devices) takes a long time. Switching to other "tabs" is immediate.

It will/might use a lot of time because it does a double SMART query of each drive, do not refresh the page while it's running. The database and the script is still running in the background and causes the database to be locked.

If you have a lot of drives, this might take a very long time. And some drives are slower than others (in my experience shucked drives from external enclosures tends to be very slow). The drives must (usually) also be spun up before they post any information, so waiting for a drive to become ready and then query SMART will just add up some time.

Read through the forum posts for more information and debugging, you'll find a lot of information about database location and what you can do to debug ev. slow drives.

Switching tabs is fast because they're already loaded. All tabs are treated as "one common page"

Link to comment

Update 2019.06.19

  • Commit #78 - IMPROVEMENT: Major change on how this plugin behaves has been implemented. The drives will now be inserted in the database at install time, and updated hourly via cronjob for the active drives.
    Force SMART scan button is also available at "Configuration".
    The temperature unit is now configured globally from Unraid itself (Display Settings).
  • NB!!! Plugin does NOT install git automatically anymore, this is still required and must be installed manually or via "Nerd Pack" plugin (this does not apply to Unraid 6.6.x, the support from 6.6 series might end soon).

 

If you encounter slow drives/loading times, you can run the script manually which will give you an indication about time usage per drive. Type in the terminal after installation (you will also see this executed during install/update):

php -f /usr/local/emhttp/plugins/disklocation/pages/system.php cronjob

 

Link to comment

Update 2019.06.20

  • Commit #82 - IMPROVEMENT: Installer and cron script now checks for running instances before executing them.
  • Commit #81 - BUG: Forgot to add standby flag to smartctl. Disks should now be able to go back to standby/sleep/off mode again. Also made sure to not accidentally move drives to "lost" when they where just standby.
  • git requirement removed!
Link to comment

I have a small issue I just noticed...   I have been doing alot of preclears on hard drives (to test older hard drives and make sure they are empty) and I went to Disk Location and seen I have a whole list of drives that are no longer in my system. Are they going to always be there? Reason I'm asking is I still have about 30 more drives to check. It's not that big of a deal. It just seems like they just started staying there. I don't remember drives staying in the past..   

 

Thanks in advance for any information...  

Disk Location drives.PNG

Link to comment
8 hours ago, Melocco said:

I have a small issue I just noticed...   I have been doing alot of preclears on hard drives (to test older hard drives and make sure they are empty) and I went to Disk Location and seen I have a whole list of drives that are no longer in my system. Are they going to always be there? Reason I'm asking is I still have about 30 more drives to check. It's not that big of a deal. It just seems like they just started staying there. I don't remember drives staying in the past..   

 

Thanks in advance for any information...  

 

First, be sure to have 2019.06.20 version as it fixed some weird issues.

 

Second, click on "Force SMART reload", it should then remove all non-existing drives into the correct table. Please report back to me if it worked or not.

 

The reason why this is probably happening is because of the new way of handling SMART data, it runs in the background every hour and checks all spinning drives (active/idle) for data. Non-spinning drives (standby/sleep/off) will not be queried because we don't want to wake up drives from standby mode. And that's why this plugin can't go through with removing non-installed disks anymore automagically, but it will do so by running the "Force SMART reload" button - this will also wake up all drives one by one!

 

The benefit of the new system is that the plugin page will be very fast to load, second is that we now can have a dashboard widget.

 

I might rename the button in the future to something more appropriate and understandable.

Edited by olehj
Added more info.
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.