Jump to content

Josh.5

Members
  • Content Count

    12
  • Joined

  • Last visited

Community Reputation

0 Neutral

About Josh.5

  • Rank
    Member

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Oh right. Cheers for the log. That is very helpful Sent from my ONE E1003 using Tapatalk
  2. These are really good suggestions. I've decided to create a web UI (probably just a simple thing nothing flash). This should solve all of your problems @trekkiedj and give you guys better control over what can be done and what cannot. I have opened issues for the features you have requested: https://github.com/Josh5/docker-libraryoptimise/issues If you think of any others feel free to open them there also.
  3. I've opened some issues on github: https://github.com/Josh5/docker-libraryoptimise/issues Feel free to open any more as you see fit
  4. Are you able to post some details of the container config? Possibly even a screenshot? Sent from my ONE E1003 using Tapatalk
  5. A docker container can accept as many paths as you wish. It will only complain if you attempt to bind them on top of each other. For example, you can bind "/foo:/container/bar" and "/baz:/container/bar/baz" just fine. But you cannot bind "/foo:/container/bar" and "/baz:/container/bar" as the second bind mount will conflict with the first. @SpaceInvaderOne I will look at implementing these features as you suggest this week. I'll let you know when it is done. As for the workers, to be honest that is uncharted territory for me. Like you mentioned, I believe if you have more cores available to encode files, you may find disk read speed will bottleneck the process. My thinking is that most people should see a performance boost in carrying out 2-3 jobs at a time (more if you have a better setup with more cores available). It is all theoretical at the moment. We would need to run tests to know for sure. Could you give me an idea of what you need for config options on the video encoding? At the moment the video stream is simply converted to x265 keeping the same framerate and resolution as the source.
  6. Thanks for the feedback guys. @trekkiedj Regarding the threads, I will take another look. Yesterday I started re-writing the app to support "workers". This will mean you can configure a number of worker threads (1-10) to handle encoding jobs. Both the inotify watcher and library scan will then just add files to a queue to be handled simultaneously by the workers. This will probably take me a few days to complete as I have a few other projects that need attention this week. @trekkiedj @SpaceInvaderOne You both suggest supporting multiplie library folders. Would it be sufficient to just pass through both locations to the containers /library path? Eg. Path: /library/Series:/mnt/user/Series:rw Path: /library/Movies:/mnt/user/Movies:rw @SpaceInvaderOne Currently there is not a way for the container not to scan the library. This is the primary function that it does on start. Are you suggesting that I change the variable "SCHEDULE_FULL_SCAN_MINS" so that if it is "0" or "off" it will not scan the library at all? If so I'll add this also. The logging issue you raise is very difficult, especially seeing as I am now working on adding up to 10 workers at a time. Having multiple processes writing to a single log file will become very confusing, even if it is piped. You will end up with the logs of all of the processes being mixed together in the log file. I think the most elegant solution would be to write a web UI to display more in-depth progress of the conversion processes. I will rais this as a TODO for when I get time. I will also look into a solution for removing old cache files in this coming week. On the subject of cache files, might I also suggest setting this to a tmpfs on the host rather than an ssd. If you have enough spare ram on your device than this should give you a performance boost and reduce wear on the ssd. Thanks again for all the feedback guys. It is much appreciated.
  7. How have did you get on so far? And thoughts? Sent from my ONE E1003 using Tapatalk
  8. @SpaceInvaderOne I did look initially into H265ize. However I chose not to follow that path as it did not let me simply monitor my current library. I considered building on top of H265ize, but he effort required to modify it was about equal to starting from scratch. Like I said, I pointed it to my library and have had it running for over a day now. If you wish to try it out/contribute the source is sitting here: https://github.com/Josh5/docker-libraryoptimise Add a new container pulling from the "josh5/libraryoptimise:latest" dockerhub repo. (I've attached a screenshot of my current config) Name: LibraryOptimise Repository: josh5/libraryoptimise:latest Variable: PUID=99 Variable: PGID=100 Path: /library:/mnt/user/TVShows/Kids/Paw Patrol/:rw Path: /cache:/mnt/user/transcodercache/libraryoptimise/:rw Variable: DEBUGGING=false Variable: SCHEDULE_FULL_SCAN_MINS=60 Other options that I've left as defaults (defaults listed): Variable: SUPPORTED_CONTAINERS=mkv,avi,mov,ts,rmvb,mp4 Variable: VIDEO_CODEC=hevc Variable: AUDIO_CODEC=aac Variable: AUDIO_STEREO_STREAM_BITRATE=128k Variable: OUT_CONTAINER=mkv Variable: REMOVE_SUBTITLE_STREAMS=true For my initial test I did not want it to go ham on my library, so I isolated it to on TV series (TVShows/Kids/Paw Patrol). I would recommend that anyone who wants to try this while we are still "testing" it out also points it to an isolated folder. The /cache folder is where the output is during encoding. This should be pointed to our ssd if you have one. This will run on 2 threads with one running the full library scan and the other watching for new files. This means that you can have up to 2 simultaneous encoding jobs at once. Please let me know if this should be improved on to allow configuring more encoding tasks or if you fell that this is fine. I would appreciate feedback on defaults for the variables. Or if you feel like there should be additional configuration options. I'm not sure it is worth creating a GUI for this, that would be a lot of work for no extra functionality on what it does. I think it would be better to refine the defaults for people so for non-advanced users can just point it at their library and go. Thanks for taking a look guys. Really appreciate it. PS. This is tested on a Ryzen setup (no GPU). The config here is not using any special encoder configuration. HEVC is just using a libx265 encoder. If you do have a setup that could utilise HW acceleration and know how this should be executed with ffmpeg, then let me know and I'll add it.
  9. @SpaceInvaderOne Hey thanks for this video. This helped me a lot! I needed to find a solution for my lack of space. Like you I have a library that is full of un-optimised videos. I think this tutorial is a pretty good idea for new incoming videos. However I think it would be nice to also have a background process that can optimise my current files as well. I also often find that I download a video and want to watch it straight away. Perhaps something went wrong with the episode that was downloaded and I want to mark it as failed and download another copy. The problem I found with your setup here is I need to wait for the conversion after the download before I can watch that video. I believe a better solution is to have a monitor watching your library for new files. When a new file comes in, it will convert it (in a separate folder) and then replace that file once the conversion is complete. Last night I wrote a container which has a monitor that watches your library for file changes. New files that match a list of file extensions are checked to see if they are already 265. If they are not they are converted and then replaced. It also has an optional scheduled event to carry out a full scan of your library. I'm still testing it on my library (it's been running for about 9 hours and is working well so far). Credit to you for pointing me down this road. It's a genius solution over searching for a download in a HEVC format Would you be interested in trying out my container? I should have it finished and pushed today. I would appreciate the feedback. It doesn't have a GUI or anything. Just some variables to specify the quality you want (HEVC) and point it at your library. Josh5
  10. Everything is up and running again. How does one mark this thread as solved?
  11. Thanks for the advice. Can you tell me exactly what parts of it failed? I couldn't get my head around it. To me, it looked like it had just reallocated a sector. Is this not right? At any rate today I went into the shop and grabbed a new WD Red to replace it. It's currently running through a pre-clear as I'm writing this. Hopefully, this is the end of my issues. Is there any hope for an HDD like that which fails? I was thinking of using it for a temp torrent DL cache drive which incurs heavy read writes but affords me no major issue if I suddenly lost all data on the disk one day
  12. Hi there, I'm not 100% sure what I need to do here. Is my parity drive shot? First, can someone please confirm that disk0 is the parity drive? I suspected it was so I ran SMART test on it. (My parity drive is the one: "WDC WD20EARX-00PASB0") As you can see from the diagnostics attached, the drive seems to fail SMART test also. Is there something I can do to fix this? Only just installed that drive a few months ago and ran a full days worth of pre-checks on it with preclear so I assumed it would be all good. thevault-syslog-20170731-1145.zip thevault-diagnostics-20170731-1145.zip