TyantA Posted July 22, 2016 Share Posted July 22, 2016 It's time. Unraid 5.0.5 has treated me well, but it's time to venture to 6. I did a guinea pig run with a friend's server and generally speaking everything went well. Then again it was a simple clean start as he didn't have much running. Now it's my turn. I started taking screenshots of configs and such, then ventured to my cache drive, an old Raptor 10k 150GB drive, to see what was on it. My primary concern is trying to see if I can keep my plex watched status, and library metadata because I hate restarting that! Now I'm not so sure I want to... what I found was staggering. My cache/apps/plex/library directory is over 90GB!? Is this... normal? The metadata alone is about 7GB. Crazy! On the one hand, I almost want to start clean to free up space. On the other... I'm on a lowly internet connection; I don't want to download that all again (or re-start tracking what I've watched). Quote Link to comment
StevenD Posted July 22, 2016 Share Posted July 22, 2016 If all you care about is watched status, use the Trakt plugin. I use it to sync watched status between Plex and Emby. Quote Link to comment
flaggart Posted July 22, 2016 Share Posted July 22, 2016 Not sure it is normal unless you have a huge library. For comparison I have: Movies: 2186 Tv: 13099 Plex takes up 17G. I do not have a "library" subdir - are you running an older version of the plugin? Quote Link to comment
Squid Posted July 22, 2016 Share Posted July 22, 2016 If it saves all the thumbnails, rather than generating them on the fly, then its probably close to the mark Quote Link to comment
TyantA Posted July 22, 2016 Author Share Posted July 22, 2016 Definitely an older version. I believe I set this up around the unraid 4.6/7 era. Might have updated it once or so. Thanks for the Trakt suggestion. I'll look into it. Yes, that's the main goal. @flaggart - my library isn't that large, I think I'm about 1k movies, 6k episodes. Even more curiously, "size on disk" is telling me 228GB... I just checked all my user shares and don't see the data anywhere else, so what windows is reporting is impossible since it's a 150GB drive. Anyway, if Trakt can work on 5.05 and 6.x then I think I know what route I'm taking. My overall mission is to get 6 up and running, figure out the best way to install my SSD with VM use in mind, read up on VMs, get Windows 8.1 installed so I can upgrade to W10 prior to the July 29th cut-off! Quote Link to comment
Squid Posted July 22, 2016 Share Posted July 22, 2016 Even more curiously, "size on disk" is telling me 228GB... I just checked all my user shares and don't see the data anywhere else, so what windows is reporting is impossible since it's a 150GB drive. Quite common with Windows, network shares, and hardlinks. Somehow Windows magically thinks that my Plex's appdata is >2tb (I got bored of watching it continually increase the size before I stopped it) on a 240GB cache drive. Quote Link to comment
lostwebb Posted July 22, 2016 Share Posted July 22, 2016 That will be because Plex is generating thumbnails. I had to turn that off a while ago as it filled my cache drive. I found a way on the Plex forum of removing the thumbnails without damaging the database and then disabled the generation. As far as I can tell the only places that it uses them is on the now playing page where you can see what users are watching on the Plex server and the thumbnail when skipping through videos on compatible players. This never bothered me so was happy to have it off. P.S. Nice to see someone with one of those Raptor drives, I bought one back in the day when they were awesome and it died, I put it in a drawer and then like 3 years later I randomly checked the warranty and realised it was 5 YEARS! and got a replacement. They are rather noisy though, I couldn't believe how much quicker Plex ran even for other people that share my server when browsing the library when I swapped to an SSD. Mine was only a 74GB though, never used it in UnRaid though, Its still knocking around in the garage somewhere should get it out and see if it still works. Quote Link to comment
aptalca Posted July 22, 2016 Share Posted July 22, 2016 Not sure it is normal unless you have a huge library. For comparison I have: Movies: 2186 Tv: 13099 Plex takes up 17G. I do not have a "library" subdir - are you running an older version of the plugin? You must have video preview thumbnails turned off. With it on, it takes up 110GB for me and I have slightly less media than you do Quote Link to comment
talkto_menow Posted July 22, 2016 Share Posted July 22, 2016 Do not worry. Watched status is stored in sqlite database. The large size is due is to thumbs and maybe some cache files. Before you proceed you need to save this file \\TOWER\plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library It contains all information you need + reference to artwork. Remember! That for this to work your new installation has to be the same as before that includes libraries you created in Plex. For example I have Volume mapping /mnt --> Host path /mnt that means that for new installation I would have to use the same mappings. If I put /mnt --> /mnt/Media, Plex would fail to play my movies because it would not find it in that location. Now you can start from scratch. Install plex docker, re-create libraries in Plex Server. Plex would create all necessary metadata like posters and fanart. Stop plex docker when you are done. Go to this folder \\TOWER\plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases\ replace file com.plexapp.plugins.library with the one you saved before Important! you have to delete 2 additional files. Plex will create new ones com.plexapp.dlna.db-shm com.plexapp.dlna.db-wal If you do not delete them Plex would not recognize replaced database. All artwork should be fine too. As I mentioned database holds only reference to them, not actual file path. Quote Link to comment
TyantA Posted July 23, 2016 Author Share Posted July 23, 2016 P.S. Nice to see someone with one of those Raptor drives. Yeah, it's been kicking around for ages. Couldn't part with it given the amount you paid for them back in the day. Yes, loud, but perfect for a server. So are video preview / thumb settings within PMS UI or...? Do not worry. Watched status is stored in sqlite database... Excellent, thank you! I had saved all the files the last time I migrated but didn't know what I was looking for. I'm about to embark on this project now. Question: Is there any reason not to go with limetech's PMS docker? I tend to prefer to stick with "official" things unless there's a good reason not to. Quote Link to comment
lostwebb Posted July 23, 2016 Share Posted July 23, 2016 I think I got mine used off eBay. But it still wasn't cheap then. It's in the PMS server settings under library I think. There are a few settings they have recently added regarding thumbnails. Change it to advanced view and check it out! Sent from my iPhone using Tapatalk Quote Link to comment
TyantA Posted July 23, 2016 Author Share Posted July 23, 2016 Will do. Just got 6.1.9 up and running. Running new permissions now. Going for the clean install then will try to bring over the sqlite db as talkto_menow outlined. Next up: to decide which PMS docker to grab (Edit: this thread proved to be useful. Linuxserver it is (as I need plexpass)) Edit: Given the size that the PMS library tends to grow (assuming I don't disable thumbnail features) I'm trying to decide how best to handle drives moving forward. As mentioned, my cache currently is the 150GB Raptor 10k drive. I have a new Crucial M500 240GB SSD I plan to run my VM(s) from. I'm just starting to read about how VMs work on unraid. Would I be better off keeping the SSD out of the array or creating a cache pool with the 150GB Raptor? Also, I happen to have a 3TB drive I haven't yet used. I just read through the file system bit and am wondering if, since the drive is currently free, I should start the transition to XFS. As in, use it to start moving everything over. I know I don't have to, but my only other option will be doing it as drives get replaced (I have no more room for additional drives). Quote Link to comment
TyantA Posted July 23, 2016 Author Share Posted July 23, 2016 Do not worry. Watched status is stored in sqlite database. The large size is due is to thumbs and maybe some cache files. Before you proceed you need to save this file \\TOWER\plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library It contains all information you need + reference to artwork. Remember! That for this to work your new installation has to be the same as before that includes libraries you created in Plex. For example I have Volume mapping /mnt --> Host path /mnt that means that for new installation I would have to use the same mappings. If I put /mnt --> /mnt/Media, Plex would fail to play my movies because it would not find it in that location. Now you can start from scratch. Install plex docker, re-create libraries in Plex Server. Plex would create all necessary metadata like posters and fanart. Stop plex docker when you are done. Go to this folder \\TOWER\plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases\ replace file com.plexapp.plugins.library with the one you saved before Important! you have to delete 2 additional files. Plex will create new ones com.plexapp.dlna.db-shm com.plexapp.dlna.db-wal If you do not delete them Plex would not recognize replaced database. All artwork should be fine too. As I mentioned database holds only reference to them, not actual file path. The one thing I'm not clear on is - while it needs to be installed to the same place (as I understand it) and shares need to be mapped the same, you say I can start form scratch. Am I deleting the original plex folder (aside from the backed up database file?) Or just installing the docker over it in the same directory? I'm guessing the latter makes less sense with the exception of potentially keeping existing artwork, etc. Edit: I originally just pointed docker to the same path without deleting things first. That said, I think I'm safe as it created a new Library directory (as opposed to library that existed before). I presume that case difference may pose a problem? If it does work, I'll be able to delete the massive library folder and move on from there. Quote Link to comment
jumperalex Posted July 23, 2016 Share Posted July 23, 2016 The thumbs nails are used for any GUI that allows you scan fwd/rwd so it can present those thumbnails as a visual queue. I say "any" when in fact that is probably only ROKU but I can't be sure. What I can be sure about is that without them using my ROKU would be painful, so thumbnails are a must for me and are auto generated. If I run out of space I'll just buy more space. I also don't tend to horde media after I've watched it, but some movies are kept for future rewatch unlike tv shows (well The Wire and Firefly will never leave my server until the day I die!) Quote Link to comment
TyantA Posted July 23, 2016 Author Share Posted July 23, 2016 I have a HTPC for my primary playback device and a barely used WDTV in the basement. My folks have a Roku device but I don't have enough bandwidth to stream to them so... I think I've run into an issue. I tried adding my TV Series share as it was before but the docker disappears when I do as it doesn't like the space. So between the library > Library and now TV Series > ? is there any hope for recovering that database? Quote Link to comment
talkto_menow Posted July 23, 2016 Share Posted July 23, 2016 I was not clear. Plex server can be installed in different folder, but mappings need to be the same, because database has file paths for each video file. I created tool for myself that allows to move plex server from one pc to another and corrects file paths. I will try to make it available to everyone. Sent from my iPhone using Tapatalk Quote Link to comment
talkto_menow Posted July 23, 2016 Share Posted July 23, 2016 As you mentioned before, you Library folder is very large. You should actually go through each folder and check the folder file size. This way you would have an idea why it so big. I do not think thumbs are cause of it. Always backup your library files before your proceed. Plex makes a copy of com.plexapp.plugins.library in its original folder (I think it's Plex Pass feature) Of course you can use your existing Plex/Library folder for new installation. However, I did this couple times and I had problems starting Plex server. Turns out you must delete this 2 files before you start Plex Server com.plexapp.dlna.db-shm com.plexapp.dlna.db-wal Quote Link to comment
StevenD Posted July 23, 2016 Share Posted July 23, 2016 My Plex appdata is right at 90GB. I knew that never had thumbnail generation turned on, and I had gone through the procedure to delete any thumbnails that had been created. So, I was quite curious as to why my appdata was so large. I have 78GB of Plex metadata and I discovered that a lot of the backdrops that have been downloaded are 4k resolution and roughly 4-6MB EACH! On Emby, they are 2k resolution and never more than 600KB each. Quote Link to comment
TyantA Posted July 24, 2016 Author Share Posted July 24, 2016 I was not clear. Plex server can be installed in different folder, but mappings need to be the same, because database has file paths for each video file. I created tool for myself that allows to move plex server from one pc to another and corrects file paths. I will try to make it available to everyone. That makes sense. And your tool would be amazing! As it stands, I have no choice but to change the folder TV Series exists in. And my audio. So if there's still a way to keep the 'watched' statuses despite the changes, that would be awesome. If not, well, it's not the first time I've started from scratch. As you mentioned before, you Library folder is very large. You should actually go through each folder and check the folder file size. This way you would have an idea why it so big. I do not think thumbs are cause of it. Always backup your library files before your proceed. Plex makes a copy of com.plexapp.plugins.library in its original folder (I think it's Plex Pass feature) Of course you can use your existing Plex/Library folder for new installation. However, I did this couple times and I had problems starting Plex server. Turns out you must delete this 2 files before you start Plex Server com.plexapp.dlna.db-shm com.plexapp.dlna.db-wal Here's the directory breakdown in \Library\Application Support\Plex Media Server: Cache 1.56GB Media 81.1GB Metadata 6.76GB I have yet to find the source of large media files. I've gone into a dozen or so folder and just see small thumbs and some subtitles. My Plex appdata is right at 90GB. I knew that never had thumbnail generation turned on, and I had gone through the procedure to delete any thumbnails that had been created. So, I was quite curious as to why my appdata was so large. I have 78GB of Plex metadata and I discovered that a lot of the backdrops that have been downloaded are 4k resolution and roughly 4-6MB EACH! On Emby, they are 2k resolution and never more than 600KB each. Interesting. So your larger directory was metadata. I guess Plex is trying to go for the ultimate visual fidelity Not overly useful on a 1080p screen, but futureproof! Quote Link to comment
StevenD Posted July 24, 2016 Share Posted July 24, 2016 All of my TVs are 4k, but I see no reason to have 4k Plex backdrops. I have failed at finding a way to limit backdrop size. Quote Link to comment
aptalca Posted July 24, 2016 Share Posted July 24, 2016 I was not clear. Plex server can be installed in different folder, but mappings need to be the same, because database has file paths for each video file. I created tool for myself that allows to move plex server from one pc to another and corrects file paths. I will try to make it available to everyone. That makes sense. And your tool would be amazing! As it stands, I have no choice but to change the folder TV Series exists in. And my audio. So if there's still a way to keep the 'watched' statuses despite the changes, that would be awesome. If not, well, it's not the first time I've started from scratch. As you mentioned before, you Library folder is very large. You should actually go through each folder and check the folder file size. This way you would have an idea why it so big. I do not think thumbs are cause of it. Always backup your library files before your proceed. Plex makes a copy of com.plexapp.plugins.library in its original folder (I think it's Plex Pass feature) Of course you can use your existing Plex/Library folder for new installation. However, I did this couple times and I had problems starting Plex server. Turns out you must delete this 2 files before you start Plex Server com.plexapp.dlna.db-shm com.plexapp.dlna.db-wal Here's the directory breakdown in \Library\Application Support\Plex Media Server: Cache 1.56GB Media 81.1GB Metadata 6.76GB I have yet to find the source of large media files. I've gone into a dozen or so folder and just see small thumbs and some subtitles. My Plex appdata is right at 90GB. I knew that never had thumbnail generation turned on, and I had gone through the procedure to delete any thumbnails that had been created. So, I was quite curious as to why my appdata was so large. I have 78GB of Plex metadata and I discovered that a lot of the backdrops that have been downloaded are 4k resolution and roughly 4-6MB EACH! On Emby, they are 2k resolution and never more than 600KB each. Interesting. So your larger directory was metadata. I guess Plex is trying to go for the ultimate visual fidelity Not overly useful on a 1080p screen, but futureproof! The "media" folder contains created metadata like screenshots (as opposed to downloaded), including the video preview thumbnails. These are about 20MB each and there should be one for every single media file. With thousands of movies and TV episodes, it really adds up. You can turn it off in the settings. Quote Link to comment
StevenD Posted July 24, 2016 Share Posted July 24, 2016 Well, for grins, I stopped Plex, renamed the Metadata folder and restarted. I only see a few missing posters on the dashboard, so Im going to let it run and see what happens. Quote Link to comment
TyantA Posted July 29, 2016 Author Share Posted July 29, 2016 Well, for grins, I stopped Plex, renamed the Metadata folder and restarted. I only see a few missing posters on the dashboard, so Im going to let it run and see what happens. What happened in the end? Quote Link to comment
StevenD Posted July 29, 2016 Share Posted July 29, 2016 Well, for grins, I stopped Plex, renamed the Metadata folder and restarted. I only see a few missing posters on the dashboard, so Im going to let it run and see what happens. What happened in the end? It took a while for Plex to refresh everything. I was obviously missing a lot of artwork. I also had to manually refresh 100+ TV shows and movies to get the artwork to show up again. At the end of the day, my Metadata dir is back to 52GB. Not much of a savings, and not sure if anything is still missing though. Quote Link to comment
talkto_menow Posted August 1, 2016 Share Posted August 1, 2016 I created a tool that would help you move Plex database between computers. It's available on github https://github.com/mpcdigitize/plexdbfix/raw/master/PlexDbFix.zip also on my blog http://mpcdigitize.com/blog/2016/07/24/plexdbfix-moving-database-between-computers/ Please let me know if you encounter any issues. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.