Squid

A couple of days late here in documenting CA's update to 2015.09.29. A typo of mine caused some temporary files to remain in RAM. Basically, every time you went into the docker tab, a new temporary copy of the application feed was saved and not deleted. Individually each of these files take up about 70k, so depending upon your usage of the docker tab and the uptime of your server, the wasted RAM could have become significant. The update on 2015.09.29 fixed this issue, but in order to remove the already existing files, you either need to Reboot your server, or enter in the following into the terminal: rm /tmp/tmp*.url

Haven't looked that closely at it, but the 67G is probably the space that the container would take up if docker didn't leverage the COW (copy on write) features of btrfs

Then that's probably where its mounted. Like I said at work so couldn't check.

@squid, thanks for your help. This has been the most progress I've had in understanding what is going on in a week. At that location, I see a /tmp folder that contains dozens of files with names like tmp-1018085421.url which mc reports to be of size 714424 (I don't know what units this is in). However, I see that /community.applications is of size 60, so I'm guessing the other file is very much bigger. The oldest tmp file seems to correspond to the date I last rebuilt docker.img. Here is a list of what's in /tmp inside docker.img. root@Media:/tmp# ls -l total 50472 drwxrwxrwx 3 root root 60 Sep 23 07:29 community.applications/ drwx------ 2 root root 40 Sep 24 16:59 mc-root/ drwxr-xr-x 4 root root 80 Sep 23 07:27 notifications/ drwxr-xr-x 2 root root 200 Sep 28 23:16 plugins/ -rw-rw-rw- 1 root root 0 Sep 28 10:00 preclear_assigned_disks1 -rw-rw-rw- 1 root root 1680 Sep 28 10:00 preclear_report_sda -rw-rw-rw- 1 root root 168 Sep 28 10:00 preclear_stat_sda -rw-rw-rw- 1 root root 10 Sep 28 10:00 read_speedsda -rw-rw-rw- 1 root root 4571 Sep 28 10:00 smart_finish_sda -rw-rw-rw- 1 root root 4571 Sep 27 09:55 smart_mid_after_zero1_sda -rw-rw-rw- 1 root root 144 Sep 27 09:55 smart_mid_pending_reallocate_sda -rw-rw-rw- 1 root root 4571 Sep 26 23:02 smart_mid_preread1_sda -rw-rw-rw- 1 root root 4574 Sep 26 09:22 smart_start_sda -rw-rw-rw- 1 root root 705595 Sep 23 13:47 tmp-1001038311.url -rw-rw-rw- 1 root root 714424 Sep 27 14:46 tmp-1018085421.url -rw-rw-rw- 1 root root 711853 Sep 25 10:04 tmp-1022176830.url -rw-rw-rw- 1 root root 711853 Sep 25 09:58 tmp-1026715570.url -rw-rw-rw- 1 root root 714424 Sep 28 09:40 tmp-1037033344.url -rw-rw-rw- 1 root root 711853 Sep 27 05:01 tmp-1041957111.url -rw-rw-rw- 1 root root 711853 Sep 25 09:57 tmp-104351457.url -rw-rw-rw- 1 root root 705595 Sep 23 11:43 tmp-1079397565.url -rw-rw-rw- 1 root root 714424 Sep 27 15:08 tmp-1114330130.url -rw-rw-rw- 1 root root 714554 Sep 28 23:16 tmp-1115295570.url -rw-rw-rw- 1 root root 714424 Sep 27 15:09 tmp-1115779883.url -rw-rw-rw- 1 root root 714554 Sep 29 10:48 tmp-1184253690.url -rw-rw-rw- 1 root root 714938 Oct 1 08:22 tmp-1244214065.url -rw-rw-rw- 1 root root 714424 Sep 28 02:27 tmp-124625526.url -rw-rw-rw- 1 root root 705595 Sep 23 11:07 tmp-1257712804.url -rw-rw-rw- 1 root root 711853 Sep 26 22:48 tmp-1262417929.url -rw-rw-rw- 1 root root 705595 Sep 23 11:26 tmp-1266159711.url -rw-rw-rw- 1 root root 705595 Sep 23 11:07 tmp-1303099892.url -rw-rw-rw- 1 root root 705595 Sep 23 11:17 tmp-1304736983.url -rw-rw-rw- 1 root root 714424 Sep 27 15:07 tmp-1320279264.url -rw-rw-rw- 1 root root 714424 Sep 27 14:44 tmp-137839263.url -rw-rw-rw- 1 root root 711853 Sep 26 14:04 tmp-1394325280.url -rw-rw-rw- 1 root root 714424 Sep 27 14:47 tmp-1475139423.url -rw-rw-rw- 1 root root 714554 Sep 29 10:48 tmp-1478797206.url -rw-rw-rw- 1 root root 711853 Sep 26 22:58 tmp-1502409291.url -rw-rw-rw- 1 root root 714554 Sep 28 10:04 tmp-1527394120.url -rw-rw-rw- 1 root root 714554 Sep 28 23:16 tmp-1548075922.url -rw-rw-rw- 1 root root 711853 Sep 26 22:47 tmp-1565010257.url -rw-rw-rw- 1 root root 714554 Sep 28 10:05 tmp-1584110921.url -rw-rw-rw- 1 root root 705595 Sep 23 07:30 tmp-160169679.url -rw-rw-rw- 1 root root 714424 Sep 27 15:21 tmp-1661075416.url -rw-rw-rw- 1 root root 711853 Sep 25 10:04 tmp-1672534634.url -rw-rw-rw- 1 root root 705595 Sep 23 11:43 tmp-1701087208.url -rw-rw-rw- 1 root root 714554 Sep 28 15:39 tmp-170252017.url -rw-rw-rw- 1 root root 708172 Sep 24 22:10 tmp-1731846235.url -rw-rw-rw- 1 root root 714554 Sep 29 10:49 tmp-1776699683.url -rw-rw-rw- 1 root root 711853 Sep 25 10:04 tmp-1794736320.url -rw-rw-rw- 1 root root 705595 Sep 23 11:17 tmp-1798316926.url -rw-rw-rw- 1 root root 705595 Sep 23 11:23 tmp-1867079391.url -rw-rw-rw- 1 root root 714938 Sep 30 18:02 tmp-1892408965.url -rw-rw-rw- 1 root root 711853 Sep 27 09:44 tmp-1992924892.url -rw-rw-rw- 1 root root 714424 Sep 27 15:06 tmp-2004304119.url -rw-rw-rw- 1 root root 705595 Sep 23 07:34 tmp-2058323394.url -rw-rw-rw- 1 root root 705595 Sep 23 07:44 tmp-2087387426.url -rw-rw-rw- 1 root root 714424 Sep 27 14:44 tmp-2088655953.url -rw-rw-rw- 1 root root 714554 Sep 28 15:39 tmp-2136823997.url -rw-rw-rw- 1 root root 714554 Sep 29 10:50 tmp-2143980462.url -rw-rw-rw- 1 root root 705595 Sep 23 10:56 tmp-238162114.url -rw-rw-rw- 1 root root 714424 Sep 27 15:07 tmp-244130472.url -rw-rw-rw- 1 root root 711853 Sep 26 11:22 tmp-297073847.url -rw-rw-rw- 1 root root 714424 Sep 27 14:37 tmp-307181750.url -rw-rw-rw- 1 root root 714554 Sep 29 13:18 tmp-319612520.url -rw-rw-rw- 1 root root 708172 Sep 24 05:56 tmp-364760669.url -rw-rw-rw- 1 root root 711853 Sep 26 14:23 tmp-381722333.url -rw-rw-rw- 1 root root 705595 Sep 23 07:29 tmp-45712661.url -rw-rw-rw- 1 root root 705595 Sep 23 07:37 tmp-46253092.url -rw-rw-rw- 1 root root 714554 Sep 28 23:17 tmp-489865132.url -rw-rw-rw- 1 root root 714424 Sep 27 14:46 tmp-49543768.url -rw-rw-rw- 1 root root 705595 Sep 23 07:30 tmp-532269974.url -rw-rw-rw- 1 root root 711853 Sep 27 09:48 tmp-54267583.url -rw-rw-rw- 1 root root 714554 Sep 29 06:36 tmp-569650762.url -rw-rw-rw- 1 root root 714554 Sep 29 07:31 tmp-595762458.url -rw-rw-rw- 1 root root 705595 Sep 23 07:31 tmp-626615610.url -rw-rw-rw- 1 root root 714424 Sep 27 14:46 tmp-741555393.url -rw-rw-rw- 1 root root 711853 Sep 26 11:20 tmp-771294825.url -rw-rw-rw- 1 root root 705595 Sep 23 07:36 tmp-793902811.url -rw-rw-rw- 1 root root 714554 Sep 28 10:05 tmp-870656813.url -rw-rw-rw- 1 root root 708172 Sep 24 13:29 tmp-87708617.url -rw-rw-rw- 1 root root 714424 Sep 27 14:43 tmp-926217946.url -rw-rw-rw- 1 root root 714424 Sep 27 14:37 tmp-9295676.url -rw-rw-rw- 1 root root 714938 Sep 30 19:15 tmp-933241343.url -rw-rw-rw- 1 root root 705595 Sep 23 11:22 tmp-970283086.url drwx------ 2 root root 60 Sep 26 09:22 tmux-0/ -rw-rw-rw- 1 root root 263768 Sep 27 09:55 zerosda Couple of questions: 1) Do you have any idea what is generating these tmp files? 2) If I did not rebuild docker.img, would these tmp files get purged from docker.img on their own? /tmp is stored in ram, not inside docker.img All those tmp-xxxx.url files in /tmp are from CA (was a memory leak caused by a typo of mine -> fixed on latest CA update) and can be deleted (or you can just restart the server)

And, even there the vast amount of storage is taken up by the containers. dockerMan presumably also stores its settings there, but everything else is miniscule compared to the size of the containers. We're talking megabytes here, compared to the gigs done by the containers.

Thanks Squid. It that where docker.img is mounted in unraid's filesystem? I would think, but not 100% sure. CA uses it for semi-temporary/reboot persistent storage for things like what's currently displayed, a cache of dockerHub icons (if dockerHub mode is enabled), etc. CA's very temporary files (intermediate downloaded files, various "flags", etc) are stored in /tmp (ram)

At work, but IIRC navigate to /var/lib/docker/unraid

The Community Applications plugin stores some files in it. Total space is ~2 Meg.

I did. It looks like they are related to the Community Applications plugin. Here's a snip from one of the files { "apps": 230, "requests": 72, "last_updated": "25th September 2015 at 17:42:37", "last_updated_timestamp": "1443199357", "applist": [ { "Beta": "False", "Category": "Network:Voip", "Name": "binhex-teamspeak", "Description": "\n TeamSpeak is proprietary voice-over-Internet Protocol (VoIP) software that allows computer users to speak on a chat channel with fellow computer users, much like a telephone conference call. A TeamSpeak user will often wear a headset with an integrated microphone. Users use the TeamSpeak client software to connect to a TeamSpeak server of their choice, from there they can join chat channels and discuss things.[br][br]\n [b][u][span style='color: #E80000;']Configuration[/span][/u][/b][br]\n [b]/config[/b] This is where teamspeak will store it's configuration file, database and logs.[br][br]\n [b][u][span style='color: #E80000;']Notes[/span][/u][/b][br]\n Connect to the server using the TeamSpeak client with the host IP address and port 9987.[br]\n To authenticate use the privilege key shown in the supervisord.log file in the host mapped /config folder.\n ", "Overview": "\n TeamSpeak is proprietary voice-over-Internet Protocol (VoIP) software that allows computer users to speak on a chat channel with fellow computer users, much like a telephone conference call. A TeamSpeak user will often wear a headset with an integrated microphone. Users use the TeamSpeak client software to connect to a TeamSpeak server of their choice, from there they can join chat channels and discuss things.\n ", "Support": "http://lime-technology.com/forum/index.php?topic=38055.0", "Registry": "https://registry.hub.docker.com/u/binhex/arch-teamspeak/", "GitHub": "https://github.com/binhex/arch-teamspeak", "Repository": "binhex/arch-teamspeak", "BindTime": "true", "Privileged": "false", "Networking": { "Mode": "host", "Publish": "\n " }, "Environment": { "Variable": [ { "Name": "", "Value": "" } ] }, "Data": { "Volume": [ { "HostDir": "path to config", "ContainerDir": "/config", "Mode": "rw" CA updated to fix this... My bad -> typo

I did. It looks like they are related to the Community Applications plugin. Here's a snip from one of the files I'll check it out tonight

I lied or maybe I'm just addicted to this In the course of doing something else, CA wound up getting some improvements to further differentiate and organize the lists better. "Super Categories" are now present. This separates Beta Applications and Private Applications (if present) from the main (hopefully stable) applications. Default is to separate them, but you can revert to the old display by disabling the feature in the settings. Fix: Temporary files now moved to ram (were being stored within the docker.img) This should increase rendering speed, especially if docker.img was stored on the array. Fix: More tweaks to the docker conversion engine. -> This is starting to annoy me. Everytime I check, dockerHub makes a slight change to the webpage which results in some containers not converting properly. Sometime I will have to sit down and figure out a new method of handling this. Fix: Sanitize the input on searches - Previously, invalid characters if present would throw up a whack of errors on the screen Updated to 2015.09.20

Works fine for me on folding@home

you can restore defaults in the webgui Not all the settings in the disk.cfg are exposed and restorable in the webgui. Including some that are changed with this script, IIRC. script changes 3 settings. All 3 are exposed in webgui if you want to manually restore them. I think that you manually entered in those values into disk settings. The script shouldn't touch poll_attributes (used by dynamix for how often to update the temperatures, smart values, etc) The script reports values for num_stripes, write_limit, and sync_window. The GUI no longer allows you to modify write_limit (although it should properly set it in the disk.cfg file). That change happened somewhere in the beta series. Your posted picture has dynamix updating the temperatures every 97 minutes

I would look at your flash drive (through the network, not locally)... See if there is a folder called /config/plugins/community.applications If there is, delete it. If there is a file called /config/plugins/community.applications-2015.... then delete it also and try reinstalling everything. It *looks* like the plugin install manager can't create any folders on the flashdrive for whatever reason. I'm curious if you're able to write to it, delete the above folders, etc. The log also showed that this boot was unclean (why it started a parity check when you started it up). This could also happen if the flashdrive wasn't writeable for some reason. I was almost hoping to have seen some bread errors in the logs, because that would have explained everything, but I didn't notice any. After that, you may consider disabling unmenu and its pkg instalations from the "go" file.

You should post your diagnostics. I've never seen that error before, but it sounds like a corrupted flash drive for some reason. Since you're on 6.1.2, a reboot of the server should (hopefully) fix any corruption issues on the flash drive. (Or you've got that bread error again from your other posts)

After a week you'll probably be irritating her and she'll be glad of a new project... Or vice versa. [emoji12]

5 Months from start to finish. At least my wife will be happy for ~ 1 week until I get geared up for something else.

FINISHED Barring any bugs / fixes that might pop up, this plugin is now completed. (Not quite sure whether I'm or ) This last update is removing the code for an unimplemented feature that requires a complete top-down rewrite to properly implement (and would not be of much benefit to the vast majority of users) Its also fixing a bug where "*.xml not found" would appear on a locally attached monitor if private repositories were not present. The only real notable change is in the dockerHub search routines. If a search result is displayed where a recommended (ie: template) application exists, then the system will not allow you to add the raw dockerfile. Instead it will take you back to the template to add from there. Its been a fun ride, (and I'm sure that everyone enjoyed the weekly and at times daily updates) but at this moment I can't think of anything else that can really be added easily without LT integration into unRaid itself. If that comes to fruition, further enhancements may be made. Until that happens, its time for me to switch gears. But, this plugin is still fully supported, and any bugs, etc that happen to appear will still be promptly addressed. A

The plugin has an issue when there is no other plugin installed at 'Tools->System Information'. It is not creating that grouping and therefore does not show up. If another plugin is installed there such as dynamix active streams, it will show up. I'm working on the solution. At work right now, but its probably the same as when plugins had to include the utilities.page file. If you look in the folder for active streams, you'll probably see another .page file called something like system utilizes that is basically empty. Just copy that file into your folder.

I believe that as long as its on the same tab set (ie: Main) you can still switch to the open files tab without the browser locking up

You have to do a search and replace to insert custom html/javascript into the stock unRAID html. I've done it but it is fraught with peril of 1) it not working on a different version and 2) hosing up the page so a user couldn't get to the buttons to shut down unRAID. But as I said in the original thread, that is a perfect place to put it..... plus adding an extra click-through when stopping the array when there are open files that may block it, so you don't get stuck in a loop. You can add another tab to the main tabs by just setting the Menu option in the .page file to be something like "Main:x" where x is the tab number to use (use something like 9 so that you won't be overwriting any other existing tab) Its no different than setting it to be in the tools section (and is exactly how CA winds up on the docker page)

Updated to 2015.09.12 Maintenance Release - Added support for Moderator Comments on specific applications - Added ability to "blacklist" specific applications - Fix: Clearing search term now displays "Choose Category" - Fix: More exceptions added to the docker conversion engine. Complete rewrite of engine into PHP.

Typo in the XML file. Issued a Pull Request on MQTT.xml

The searching for Blank has been fixed for the next release (day or two), with the side effect that you probably won't see that long running script anymore (unless you actually display "All Applications"). But, you might want to consider switching to Chrome as its by far faster a browser than FireFox

Amazingly good or amazingly bad? If the latter I'm open to suggestions [emoji6]