pjneder
-
Posts
113 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by pjneder
-
-
Hmmm...that is interesting. I see exactly what you mean.
I first noticed this the other day in the following way. As I would load my .mkv files, I would see the cache fill up. Occasionally I would bump into a write space problem so I would manually fire Mover (probably something there I need to fix). I got used to watching the Used/Free space of the Cache. It was typically <5GB steady state.
Then, while moving disks around I hit the problem of cloned UUID's which caused a mounting problem, eventually causing the Cache pool to fail to mount. I wiped the still connected disks sitting in unassigned, clearing the UUID and that issue away. Restarted the server, and then I noticed the Cache sitting at 25.8GB Used!
I don't have all of the forensic evidence, but something changed in the reporting [around] when the Cache pool did not mount correctly.
Hence this thread and very informative discussion. Could be, most likely, the docker.img has been 20GB (21.5GB depending on math) this entire time but it was not being shown as used in the Main screen Cache pool....until now.
Guess all is copacetic after all. Further thoughts welcomed.
Best!!!
P.S. Thanks to this I'm likely going to take the red pill and swap my server architecture around...
-
1 minute ago, trurl said:
Your /tmp:/transcode mapping looks correct and your Plex trancoder directory is /transcode, so that should be working. What I don't see is any evidence of this "ballooning".
Maybe ballooning is the wrong word. I was using that to describe the fact that the FS space consumed is listed as 21.5GB w.r.t. to how the Cache FS sees the image, even though the diags say it is only consuming 515MB.
-
2 minutes ago, trurl said:
What evidence do you have that says
It normally doesn't "balloon" in an nonvirtual Unraid, it just runs out of space.
Did you see the screen shots in my previous post? The cache and FS both show the consumption. Trying to figure out how to properly answer the question.
-
20 minutes ago, trurl said:
No experience with that and whether running virtual disks within virtual disks might be an issue. And virtualizing Unraid isn't supported. There is a subforum here where other virtualizing Unraid users support each other.
OK, fair enough. It might not be supported but essentially it works. I started this project long before unRaid became the more beautiful thing it is now with KVM, Docker and more. I host my pfSense, Ubiquity controller and more on there. However, unRaid would now allow me to do all of that so I might switch over to unRaid being the native with everything else hosting as needed.
I expect you know the answers to these questions:
If the /tmp to /transcode (i.e. RAM transcode) was not passing through as expected, AND I had engaged the transcode engine, would that explain the sudden jump in the docker.img size?
Follow up, if I map the /transcode area to an unassigned spinning HDD (i.e. my old 1T), would that A) work B) keep the 1T spinning all of the time?
Thanks!!!
-
32 minutes ago, trurl said:
Did you change something? According to diagnostics your docker image has 20G allocated, and is only using 515M.
What do you have in Plex for transcoder directory? Screenshot please.
Change, yes and no. I have been changing some stuff with my array, by swapping in some larger disks. I cannot remember exactly when the size ballooned up but it was sometime in the past few days. I had been busy re-muxing my media library from .iso files to .mkv files and putting them in a totally new share just for use by Plex. The re-mux occurs on my PC, not unRaid. Then I decided it was a good time to add some storage (even though I wasn't really running low, but I did have some old 1T seagates drives I wanted to pull out).
The no part comes in that I have not touched the Docker or settings except to pull updates as they came available.
I also did some experimenting with playing Plex remotely to my phone & iPad. I can suspect that this would have kicked off transcoding. I had never run Plex streams remotely before, so that would also qualify as a change.
Then I noticed my cache used was much larger. I ran Mover but it stayed quite big. Scratching my head I started looking around and found that docker.img had grown large.
I will point what is likely obvious in the diags. This instance of unRaid is running on ESXi as a VM. 12GB of RAM allocated and access to all 4 vCPUs.
What I do notice in my ESXi dashboard, which is new, is that ESXi thinks VMware tools are not installed, which is new. Not sure how worried I should be about that, but I will have to do some looking. I am pretty sure I had this working in the past.
Thanks!
-
Here you go. I really appreciate the help!
I had already checked the container size, because I was surprised when I saw my cache sitting at 25.8GB used. Went hunting for the cause. Found the large docker.img file, sitting at 21.5GB.
-
1 minute ago, trurl said:
Do you use the DVR capability in Plex?
Nope. Just double checked and nope, still have the "set up plex dvr" button. Had no intentions of that with my current setup.
-
3 minutes ago, trurl said:
As explained at the very first link in the Docker FAQ:
Reading up in here along the way. Was looking so hard for Plex support that I missed the general FAQ with Plex related topics.
-
11 minutes ago, Hoopster said:
Post your Plex docker run command. Maybe the problem will be apparent there.
root@localhost:# /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='PaulsPlexServer' --net='host' -e TZ="America/Los_Angeles" -e HOST_OS="Unraid" -e 'PLEX_CLAIM'='claim-ybLt8gdqUcLu1yTDSosP' -e 'PLEX_UID'='99' -e 'PLEX_GID'='100' -e 'VERSION'='latest' -v '/tmp':'/transcode':'rw' -v '/mnt/user/Plex':'/data':'rw' -v '/mnt/user/Plex':'/data2':'rw' -v '/mnt/user/appdata/PlexMediaServer':'/config':'rw' 'plexinc/pms-docker'
0aa1f2a0f850bdfe280d07964ba0a95f47824b5455c102aacabb6c1bed02ea88
ignore the extra mapping to /data2. Just did that to get the run command to show up.
-
6 minutes ago, Hoopster said:
Post your Plex docker run command. Maybe the problem will be apparent there.
Gosh I hate to seem an idiot because I'm not....but here goes. Where is the run command found? It just isn't obvious to me screen tired eyes at the moment.
-
@Hoopster I found this older thread and as you stated it was helpful even now in validating my Plex docker config.
However, I will resurrect this because I have another question. As I've been getting my Plex (pass enabled) server library loaded, I've noticed an issue. My setup is basically what was mentioned above but even simpler. I map /mnt/user/Plex to /data and then organize my content within the Plex share. I use /tmp for /transcode.
The problem is that my docker.img file has now ballooned to 21.5GB and Plex is the only docker I'm running. So it is all coming from there. I don't even have that many movies shoved into /mnt/user/Plex yet.
I found this thread because some other thread suggested that there was "probably a mount inside the docker that should be outside" so I started hunting. I don't think that is the case but maybe I'm misunderstanding just how much appdata it creates.
Hints and ideas?
Thank you!
-
6 hours ago, pjneder said:
I should have noted, I tried multiple settings. My first path was to just have it zero the drive without the pre/post tests. Agreed it is already known to be good, reliable and nominal SMART stats.
Regardless, the preclear should work, so my question about the warning and behavior still stands. Your support is much appreciated!
Added:
Actually, I see a different but related issue in the array. I decided to start writing some media files and I notice that Disk 2&3 are greenballed yet the parity disk is not showing as spun up. That should not be possible. Something seems to be fishy in the enumeration at this time.
I got this fixed. It was due to repeating UUIDs, as I had not wiped the signature from the disks and they were in unassigned but conflicted with pool disks. All sorted now.
Thanks!
-
Thanks! @johnnie.black!!!
running the wipefs -a on the sde and sdl was the trick to managing that. I will hopefully remember than in the future. I still have 1 more 1T to retire and slot my other 3T into that spot. I will be able to not repeat the mistake.
Curious, if one immediately uses the FORMAT button on the unassigned device, will that fix this as well? i.e. is there any other recommended way to remove the drive from the config without physically doing the disconnect? My thing was that I was keeping it there in case something happen during the migration should I have needed to replace.
Thanks!
-
I think I have hit a similar issue as covered in this thread. I'm reading it but not exactly sure what to do, so I will tread carefully and await some advice.
Attached are my anonymized diags output.
Basics of my sequence:
Plugged in 2x new 4T WD Red drives enumerated as sdi and sdj
Pre-cleared both.
Stopped array, unassigned sde (3T) from Partiy.
Assigned sdi as Parity, started, rebuilt parity
Stopped array, unassigned sdl (1T) and added sdj to array, started, rebuilt sdj
Went to zero out the sde 3T using preclear, started having problems. Did a clean restart then the cache went unmountable and I started seeing these "cannot scan" errors.
Oh yeah, all my dockers are gone now since the Cache pool didn't start. Hoping to recover the pool and not have to start over.
Any pointers would be appreciated!
Thanks!
-
1 hour ago, trurl said:
Why? Seems like it has already had a good workout and that is the only purpose of preclear.
I should have noted, I tried multiple settings. My first path was to just have it zero the drive without the pre/post tests. Agreed it is already known to be good, reliable and nominal SMART stats.
Regardless, the preclear should work, so my question about the warning and behavior still stands. Your support is much appreciated!
Added:
Actually, I see a different but related issue in the array. I decided to start writing some media files and I notice that Disk 2&3 are greenballed yet the parity disk is not showing as spun up. That should not be possible. Something seems to be fishy in the enumeration at this time.
-
Got what is hopefully an easy question. Probably answered but not found while searching the forums.
In the process of upgrading my parity drive to larger and adding a disk, now I have the former 3T parity drive sitting in Unassigned devices. All good and as expected. Array is started and nominal. Therefore I wanted to do a preclear on the 3T parity, currently sitting on sde1. If I start preclear with default options the UI says "starting" but no stats appear and it does not appear to be working the disk. Checking the disk log I see:
May 4 17:10:58 Floater emhttpd: WDC_WD30EFRX-68AX9N0_WD-WMC1T2998971 (sde) 512 5860533168
May 5 05:41:22 Floater emhttpd: WDC_WD30EFRX-68AX9N0_WD-WMC1T2998971 (sde) 512 5860533168
May 5 05:41:51 Floater root: WARNING: adding device /dev/sdi1 gen 225 but found an existing device /dev/sde1 gen 234
May 5 05:41:56 Floater emhttpd: shcmd (10913): /sbin/btrfs device delete /dev/sdi1 gen 224 but found an existing device /dev/sde1 gen 234 /mnt/cache &Not sure what I should do here. Any hints?
Thanks!
-
[sOLVED]!
OK, I figured out what the problem was. I had selected "Other (32b)" as the type of Linux. Hence ESXi would never allow me to assign a VMXNET3. Once I switched that to "Ubuntu (32b)" I got that choice in the drop down and unRAID (with http://lime-technology.com/forum/index.php?topic=11449.0 installed) found the interface just fine.
Now I'm working all sweet like.
P.S. FWIW pfSense is another story. Figured out how to do the necessary custom install of VMWare tools, but traffic shaping is acting all fussy so I've dropped back to E1000 until I can sort that out.
-
Thanks for the response. I follow everything you are saying and I'm pretty familiar with how that works. Attached are a few snips from the VM summary tabs.
I understand how *nix generally enumerates interfaces. In the case of pfSense and unRaid both it will automatically find the interfaces, even if it does renumber them. For me, I must still be missing something simple because the VMXNET3 don't show up at all on unRAID or pfSense.
I have an Ubuntu appliance VM from Bitnami running Wordpress and in that VM it does show up as VMXNET3. On that VM the tools show as "Installed (current)" and not as 3rd party. Same in my Win7 VM running apcupsd.
Therefore the real question is: How have folks gotten VMXNET3 adapters to work on unRAID?
thanks!
-
...for the 82579LM to be recognized in ESXi, you will need to install extra, non-standard drivers.
...for attaching the VMXNET3 device to a VM, in order to show up inside the VM, the vmware tools need to be installed in that VM.
I wonder if I got the *right* non-standard driver. Both of the interfaces work fine, as E1000's, but I haven't gotten VMXNET3 to work on any VM so far...
-
...for the 82579LM to be recognized in ESXi, you will need to install extra, non-standard drivers.
Check.
...for attaching the VMXNET3 device to a VM, in order to show up inside the VM, the vmware tools need to be installed in that VM.
Check.
Still no joy.
-
I am not getting the option to assign a VMXNET3 NIC to my unRAID VM. Furthermore, although not unRAID related directly, if I assign VMXNET3 NICS to my pfSense VM they do not show up.
For reference the NICs on this mobo are both Intel.
82574L and 82579LM
When I look in the vSwitch properties under Network Adapters tab I see that the driver is listed a e1000e. Curious if anyone has thoughts on what I need to fix.
Thanks!
-
BQ,
Thanks for the excellent summary on how to do this!!
While doing my own hunting around I came across another interesting page that provides some great background for those considering whether to buy the M1015.
http://www.servethehome.com/ibm-m1015-part-1-started-lsi-92208i/
On the page about cross-flashing here: http://www.servethehome.com/ibm-serveraid-m1015-part-4/ He mentions leaving off the
-b mptsas2.rom
piece in order to get the card to boot faster at start. Seems kind of handy to me. Not that you will start your ESXi server very often, but it might be interesting to some, especially native unRAID installs.
Cheers.
-
Joe,
Handy advise that...
In my structure, under "Media" is only directories. You have to go another level deeper to see files and thumbs. I'm wondering what could be getting accessed?
Thanks!
-
I'm new to cache_dirs. Don't know why I didn't use it before except that I was lame...
Anyway, running it under the plug-in interface on 5.0-rc12a. This is all on a VM. When I open my "Media" share in Windows, disk #2 always spins up. Media is spread across several disks.
I'm using pure default config on the plug-in interface.
root@Floater:~# free -l total used free shared buffers cached Mem: 2074512 739824 1334688 0 334304 211760 Low: 890824 511788 379036 High: 1183688 228036 955652 -/+ buffers/cache: 193760 1880752 Swap: 0 0 0
Doesn't look like I'm running into memory limits. ESXi also shows way below the 2GB commit limit for the VM.
Where can I look for more debug data?
Thanks,
Paul
Keep certain VM's running without array started
in Feature Requests
Posted
+1 vote on this. Keeping the topic alive. I am a Plus license and would happily pay more for this feature!