[Support] Linuxserver.io - diskover

[JDK]

So i had been using this container for a couple of years at least, then a few weeks (maybe months now?) ago it broke with some error in the logs that i didn't get to figure out at the time. Fast forward to this week and i finally get back to fixing this container on my server and see that we are now on v2. That seems to bring with it the fact that most of the great features are now locked behind pay editions. The v1.5 tags are still available, and i realize now that the initial issue that broke my container was likely due to an update because i can pull one of the more recent v1.5 tags and the same error is there, but going back a few versions (~JUN 2021) and no error.

But the template is no longer available..does anyone have v1.5 still running and mins posting a screenshot of the template so i can see which variables to set up again?

[gamerkonks]

Here's my template for v1.x

I might need some help getting v2.x set up. I'd like to give it a go.

[JDK]

Awesome, thanks. ill give this a poke tomorrow to see if i can get v1.5 back up.

 

v2 was really simple actually and fewer steps since it no longer seems to require Redis or a specific version of Elastic (i have it running against 7.10.2). That said there is now so much confusing information because so many struggled getting v1.5 up in the first place that there are a loads of loose bits of info that can throw you off course when v2 really doesnt need it.

 

If it helps you, these are the exact steps i followed while installing v2:

1. Install CA User Scripts
   • Create a new script named vm.max_map_count
   • Contents of script as follows: #!/bin/bash sysctl -w vm.max_map_count=262144
   • Set script schedule to At Startup of Array.
   • Add the docker container.
2. Install Elasticsearch container from CA (for me, 7.10.2 works just fine using d8sychain's release)
3. Install diskover container from CA
   • configure all your variables, do not start the container upon creation. if you did, no problem, just power it down.
   • i had to manually edit the config file (/appdata/diskover/diskover-web.conf.d/Constants.php) to set the ES_HOST variable. it does not seem to update from the docker template, resulting in failed connection to ES.
   • start the container
4. The diskover Application Setup documentation mentions the issue "The application doesn't start an index by default.", so i had to manually create the first index using below.
   • docker exec -u abc -d diskover python3 /app/diskover/diskover.py -i diskover-my_index_name /data
   • you can exclude the "-i diskover-my_index_name" and it will create an index using this pattern "diskover-data-211128210916".
   • the index may take a while to crawl. you can see its progress here: http://server:9200/_cat/indices?v where 9200 is your ES port.
5. optional: go back to CA User Scripts and create a new script using the command from previous step.
   • i may have missed something in the config, but the v2 container is not creating indices automatically like the v1.5 container did.

For me it took a few minutes for the first index to become accessible and i could actually use the diskover app. one thing to note is that you need to be careful to not delete the index you have selected in diskover ("Save selection" option). i did this and the app broke completely, not defaulting to one of the other indices. the only way i could fix this was to recreate the index using the command from step #4...at least the error gives you the name of the missing index.

 

[gamerkonks]

Thanks, I think I've got it working, just waiting on the first index to build.

 

Quote

i may have missed something in the config, but the v2 container is not creating indices automatically like the v1.5 container did.

 

I think it may be because the crontab file has the first line commented out.

It's defaulted to 3am everyday, remove the "#" and save it and check if it builds a new index at 3am.

[gamerkonks]

Disappointed by the locked features in v2, but can't manage to get v1.5 working again.

Had to remove security from ElasticSearch5 to get it working again, apparently that's a paid feature now?

And then had to run

docker exec -it diskover bash
pip3 install 'click==7.1.2' --force-reinstall
exit
docker restart diskover

To get diskover 1.5 to stop throwing errors in the log.

You need to reapply this every time you make a change to the docker template or if it updates though.

 

If I try and go to the Web UI for diskover 1.5, I get a server error 500.

[JDK]

You should be able to use one of the other elastic dockers and just specify the tag. for example i remember using FoxxMD's image. if you just look at the template you'll see where to change it to "elasticsearch:5.6.16". thats how i had v1.5 running before.

 

I won't get to try installing v1.5 for a couple of days, but ill update here with my progress once i do get back to it.

[OFark]

I created a docker app in CA just for Diskover. Search for OFark or ElasticSearch. It's been a while but I'm assuming it still works. 

[gamerkonks]

I'll try out your elasticsearch container.

I'm currently using the elasticsearch container with the tag elasticsearch:5.6.14.

Hopefully I'll be able to copy all my old indexes over.

[VladoB]

Hi, I installed latest available version of docker Diskover yesterday and it seems that diskover simply still ignores setting from GUI, am I right?

[gamerkonks]

9 hours ago, VladoB said:

Hi, I installed latest available version of docker Diskover yesterday and it seems that diskover simply still ignores setting from GUI, am I right?

 

Yeah, I had to update the values directly in the config file.

[Helmonder]

Installed the container but the userid/password settings do not seem to work ?  I can set them but the WEBGUI will not let me log on.. The edit page states that you can leave them empty if you do not want userid/pass but the edit page will not allow them to be empty..

[issabankz]

On 3/4/2022 at 12:29 PM, Helmonder said:

Installed the container but the userid/password settings do not seem to work ?  I can set them but the WEBGUI will not let me log on.. The edit page states that you can leave them empty if you do not want userid/pass but the edit page will not allow them to be empty..

i have the same issue, loging in is not possible!

[Mattti1912]

Hello 

 

I've seen an youtube video, where it was implied, that i can get back deleted data with this tool.. Is that correct??

thank you, regards.

[cam217]

13 minutes ago, Mattti1912 said:

Hello 

 

I've seen an youtube video, where it was implied, that i can get back deleted data with this tool.. Is that correct??

thank you, regards.

I don't think that, no.
What video are you talking about?

[oh-tomo]

Oops something went wrong 

 Unable to connect to Elasticsearch host localhost:9200. Error: Failed to connect to localhost port 9200 after 0 ms: Connection refused

Reload index page

Logout

 

Getting this after logging into webui.

 

[Squid]

localhost refers to the IP address of the container itself.  Try setting the elasticsearch variable to be the IP address of the server (and also set the applicable port of the container).  Assuming of course you have elasticsearch installed

[oh-tomo]

15 minutes ago, Squid said:

localhost refers to the IP address of the container itself.  Try setting the elasticsearch variable to be the IP address of the server (and also set the applicable port of the container).  Assuming of course you have elasticsearch installed

 

Thanks for the quick reply.  I've set elasticsearch variable to the IP. I didn't have elasticsearch installed.  Now it is installed and running.  Also set up the vm.max_map_count user script.  Stopped and started diskover and still see the same "Oops something went wrong, Unable to connect... localhost:9200" error.  Then stopped and restarted elasticsearch and stopped and restarted diskover.  Same error.

[oh-tomo]

9 minutes ago, oh-tomo said:

 

Thanks for the quick reply.  I've set elasticsearch variable to the IP. I didn't have elasticsearch installed.  Now it is installed and running.  Also set up the vm.max_map_count user script.  Stopped and started diskover and still see the same "Oops something went wrong, Unable to connect... localhost:9200" error.  Then stopped and restarted elasticsearch and stopped and restarted diskover.  Same error.

 

New error: "No completed indices found in Elasticsearch. Run a crawl and after it finishes reload select indices page."

[oh-tomo]

5 hours ago, oh-tomo said:

 

New error: "No completed indices found in Elasticsearch. Run a crawl and after it finishes reload select indices page."

 

Saw JDK's November post above suggesting to manually start indexing so ran this command:

 

docker exec -u abc -d diskover python3 /app/diskover/diskover.py /data

 

Then checked http://192.168.0.36:9200/_cat/indices?v but no activity present.    ????

 

I've also uncommented the crontab entry as suggested in an earlier post.

 

Maybe it would be easier to just upload everything to Google Drive and search there.   Google used to have a desktop search product from 2004-2011, but alas it went to the graveyard of useful products.  https://en.wikipedia.org/wiki/Google_Desktop

 

[aarontry]

Is it still possible to have a new v1.5 installation? I cannot find anywhere to get the auth token.

[dajinn]

Are there any good alternatives to this? It seems like most of the functionality is now behind a paywall. It's not immediately obvious but I don't see a way to actually do anything with the indexed data, such as look for duplicates, or take action against anything.

Edited September 14, 2022 by dajinn

[BoKKeR]

On 5/30/2022 at 11:49 AM, issabankz said:

i have the same issue, loging in is not possible!

from the config/diskover-web.conf.d/Constants.php file you can find 

 

    const USER = 'diskover';
    const PASS = 'darkdata';

 

Edited October 5, 2022 by BoKKeR

[BoKKeR]

On 8/2/2022 at 3:01 AM, oh-tomo said:

 

Saw JDK's November post above suggesting to manually start indexing so ran this command:

 

docker exec -u abc -d diskover python3 /app/diskover/diskover.py /data

 

Then checked http://192.168.0.36:9200/_cat/indices?v but no activity present.    ????

 

I've also uncommented the crontab entry as suggested in an earlier post.

 

Maybe it would be easier to just upload everything to Google Drive and search there.   Google used to have a desktop search product from 2004-2011, but alas it went to the graveyard of useful products.  https://en.wikipedia.org/wiki/Google_Desktop

 

 

This worked for me, but I had to switch to elasticsearch version 7.17.5, since 2.X edition only supports that one according to their page.

 

https://docs.diskoverdata.com/diskover_installation_guide/

 

I am not sure if 2.x is worth it, since all features are stripped away. I tried getting an essential license for home use, but I was met with having to schedule a call just to hear the price

 

Edited October 6, 2022 by BoKKeR

[dajinn]

On 10/6/2022 at 2:06 AM, BoKKeR said:

 

This worked for me, but I had to switch to elasticsearch version 7.17.5, since 2.X edition only supports that one according to their page.

 

https://docs.diskoverdata.com/diskover_installation_guide/

 

I am not sure if 2.x is worth it, since all features are stripped away. I tried getting an essential license for home use, but I was met with having to schedule a call just to hear the price

 

 

Same here. I tried asking about potentially getting a perpetual license for home use but was basically just ignored.

Edited October 29, 2022 by dajinn

[marco_polo]

On 5/30/2022 at 2:49 AM, issabankz said:

i have the same issue, loging in is not possible!

 

Hi, Same here any update when this can be resolved. 

Tried username = diskover and password=darkdata and did not work
 

Edited December 22, 2022 by marco_polo