[Support] HaveAGitGat - Tdarr: Audio/Video Library Analytics & Transcode Automation

[daddintech]

Although I utilize Tdarr for transcoding H.264 videos to H.265 videos, I do use it in a somewhat off-label manner.  Rather than Tdarr watching my main videos folder, I move new H.264 movies from a share on a pool drive to a different share on the same drive that Tdarr watches.  This drive originally utilized the XFS file system.  This arrangement worked just fine under this scenario and Tdarr’s folder watch picked up newly copied files and placed them into the transcoding queue.  In preparation for use of this drive for additional functions, I converted this pool device to ZFS.  Now, when I move H.264 videos to the Tdarr watched folder, the folder watch function does not place anything into the transcoding queue.  If I initiate a Scan of the Tdarr watch folder/library, the newly copied video is detected, added to the transcoding queue, and transcoding proceeds without issue.  My question is, is this simply a result of the natural behavior/function of the ZFS file system and how files are copied and/or moved on a drive, or is there something else amiss here?  Due to my low usage of Tdarr, this current behavior is not really a problem as it is only just a few extra mouse clicks.  But, I’m interested as to the cause to satisfy my my own personal curiousness.  
 

Additional information…. I am on the most current versions of Unraid and Tdarr both.  Also, I am new to the ZFS file system and its more detailed behavior/function.

[MrCrispy]

I hope its ok to ask questions here about Tdarr usage in general? My Unraid server is dead (hw issues) so I will be using a cheap sff pc I bought and I'm new to this. My goal is to convert to hevc, having looked at Tdarr, Unmanic and FileFlows it was not clear what to pick but Tdarr seems the oldest and most configurable so I'm picking that. I have a few questions -

 

- my hw will be 2 sff mini pcs, with an Intel Core i5-7600 and a 8500T, they are an HP EliteDesk and Dell Optiplex, both with 8GB ram. I will install a Linux distro (Debian or Ubuntu I think) on both and tdarr via docker.

- from everything I've read Intel QSV is the sweet spot, cpu takes too long and Nvidia sacrifices quality. From Wikipedia it seems both 7th and 8th gen support QSV, correct? 

- is there any reason to use VAAPI and not QSV, I'm assuming QSV is better?

- separate instance vs nodes -  I don't want to pay for the premium version just now, the free version doesn't let you assign libraries to nodes. so e.g. if I have 2 different libraries I want to transcode, is it better to have to instances (one on each pc) with hdd's connected with the media needed, vs having one master node, in which case I'm guessing there will be a lot of network transfer since the media will be randomly assigned? the advantage with a single master seems to be unified stats

- workers vs nodes - does node mean a separate physical machine (vm) and worker a thread on that instance? is there a guidance on how many workers/nodes can be run based on cpu? does cpu even matter or is it all dependent on the iGpu?

and most imp - 

- what plugins are recommended, there are so many. e,g do I use Tdarr_Plugin_JB69_JBHEVCQSV_MinimalFile or Tdarr_Plugin_bsh1_Boosh_FFMPEG_QSV_HEVC or one of the many migz ones, and what tweaks do people make?

- can someone give me a rough idea of the speeds/time I can expect for conversion with above hardware?

- I read in some posts that Unmanic is faster/smaller file size. is this because of a lower crf/preset in their default plugin or is there something else going on? which of the 3 tools do people recommend?

 

I'm asking here because the Unraid forums are some of the most helpful and knowledgeable, thank you in advance.

[J05u]

Guys, anyone had experience to conver library to AV1 using Nvidia gpu? Want to try to convert some of my videos, not sure which plugin I can use

[J05u]

Hello, anyone tried to transoce h264 to av1 with 4000 series?

I am happy with 4060 performance in this area, I was getting 600 fps converting to h265, want to see how much I can get with AV1

Edited August 1, 2023 by J05u

[comet424]

is anyone experiencing in the last update or 2...  when  you are trying to transcode from 264 to 265  or so   it throws all videos in the trascode error/cancelled folder....  it used to work fine couple updates ago    but seems now  any video  the odd one gets in  and works but the rest are thrown out...  the logs dont show any errors really...  anyone know whats going on?  i tried a reboot  of unraid but that didnt help either

[wgstarks]

1 hour ago, comet424 said:

is anyone experiencing in the last update or 2...  when  you are trying to transcode from 264 to 265  or so   it throws all videos in the trascode error/cancelled folder....  it used to work fine couple updates ago    but seems now  any video  the odd one gets in  and works but the rest are thrown out...  the logs dont show any errors really...  anyone know whats going on?  i tried a reboot  of unraid but that didnt help either

Yes. I posted about this a few days ago but didn’t get any response.

https://forums.unraid.net/topic/84070-support-haveagitgat-tdarr-audiovideo-library-analytics-transcode-automation/?do=findComment&comment=1286525

Edited August 3, 2023 by wgstarks

[comet424]

@wgstarks ah ok so i not the only one at least...  i dont get the same logs as you  but i just look at the logs page and all the stuff says INFO  but un clickable...  hopefully there be a fix  soon then

 

 

[wgstarks]

1 hour ago, comet424 said:

@wgstarks ah ok so i not the only one at least...  i dont get the same logs as you  but i just look at the logs page and all the stuff says INFO  but un clickable...  hopefully there be a fix  soon then

 

 

Those are the logs from the failed encode queue.

[comet424]

@wgstarks  my logs are gone at moment  but all i had was something  [INFO]  and then the file name    but thats under Log tab  unless your pulling it from somewhere else..   

 

for now i just skip it and do it later i guess..  was converting my dvds  too but it didnt like it..  also i find the tdarr  doesnt erase  title in the mkv   so when i rip it and when you play it in vlc  i get like  main_vob   as the title  so i gotta use the mktool and force a remove the tags...  dunno if tdarr can do it.

[wgstarks]

I’ve opened an issue on GitHub. You can follow it here.

[wgstarks]

I was able to get the docker working by reverting to 2.00.18.1.

 

Haven’t tested any versions in between so it’s possible newer versions will also work:

[wgstarks]

5 hours ago, wgstarks said:

I’ve opened an issue on GitHub. You can follow it here.

Resolved my issue by updating the transcode plugin. Latest version is working great now.

[J05u]

Any chance to transcode to av1 from h264? Using Nvidia 4000 series

[comet424]

anyone having issues to transcode mp4 to mkv...  it used to work.. but since i had problems  i uninstalled and reinstalled tdarr adn tdarr node...  to fix issues before  now it cant convert mp4   i have to use  mktool  to convert them to mkv  and then  tdarr to convert to 265  

[SavageAUS]

It appears a lot of questions in this thread go unanswered but I will try anyway.

I have the tdarr and tdarr_node containers running just fine, libraries added and I can see my GPU doing the transcodes BUT all 8 of my CPU cores are getting maxed out as well.

According to spaceinvaders video the impact on the CPU should no where be where it is at.

Spoiler

 

Edited September 3, 2023 by SavageAUS

[showstopper]

I have set tdarr up and it seems to be working, apart from renaming the files and moving them from cache.

Can anyone see what I have done wrong here please? Do I need to create a step to rename and move?

Successfull transcode (no mention of a move attempt):

 

Library options:

 

Transcode options:

[wgstarks]

3 hours ago, showstopper said:

Can anyone see what I have done wrong here please? Do I need to create a step to rename and move?

You don’t show your container mappings but I suspect that you don’t have the container path /mnt/media/OUTPUT mapped to a host path. You can post your docker run command for TDARR if you like.

[showstopper]

2 hours ago, wgstarks said:

You don’t show your container mappings but I suspect that you don’t have the container path /mnt/media/OUTPUT mapped to a host path. You can post your docker run command for TDARR if you like.

Thanks for replying:

 

 

I haven't mapped an output, it is the same as the media library:

[showstopper]

OK - I have just clicked on 'Actions' and 'Accept' and the files are being renamed and moved to the Output folder!

Is there a way to make tdarr automatically do this?

 

Edited September 9, 2023 by showstopper

[wgstarks]

2 hours ago, showstopper said:

OK - I have just clicked on 'Actions' and 'Accept' and the files are being renamed and moved to the Output folder!

Is there a way to make tdarr automatically do this?

 

I would suggest posting your question to the TDARR project page.

[almulder]

Anyone else using the /tmp ram folder for transcoding temp location? I am but it just fills up the tmp folder (ram) and eventually locks me out of unraid gui as there is no ram left.

 

I keep getting these errors: anyone know how to fix? (After the / is the file name)

 

its not removing the cachefiles: (From MC)

 

Note: Both Tdarr and Tadarr-Node have the same paths setup

Edited September 21, 2023 by almulder

[almulder]

Ok I got help from the discord and the issue was I had things in my que before I enabled the "allow GPU workers to do CPU tasks" Restarted from scratch and made sure it was checked before adding libraries and started a fresh scan, now issue is gone. Hope this helps others in the future.

 

Note: It was in the staging area that said CPU required and only CPU option then was working, now GPU does it all.

[SicSemper23]

I'm just getting going with TDARR and running into an issue where the system says it's transcoding, it looks like it's transcoding, but it doesn't seem to be actually converting the files. Spaced saved after a few days and 49 complete transcodes is 0 GB.

 

I followed the Spaceinvader One vid so all my setting are mostly default to what he shows there, with a couple of minor tweaks. Mostly that I'm running an internal node in the main TDARR container, in addition to a TDARR node container.

 

My setup is the server on my Unraid machine with a node on a Windows PC. Video card on that machine is a Radeon R7 200. 

 

TDARR recognizes both nodes, so I thought everything was set up properly, but the Windows node is throwing this error, and I'm not sure what's causing the issue:

 

 

[2023-10-02T04:30:08.569] [ERROR] Tdarr_Node - Node connection error to :((Server IP))

[2023-10-02T04:30:08.569] [ERROR] Tdarr_Node - TransportError: xhr poll error

at Polling.onError (/app/Tdarr_Node/node_modules/engine.io-client/build/cjs/transport.js:45:37)

at Request.<anonymous> (/app/Tdarr_Node/node_modules/engine.io-client/build/cjs/transports/polling.js:251:18)

at Emitter.emit (/app/Tdarr_Node/node_modules/@socket.io/component-emitter/index.js:143:20)

at Request.onError (/app/Tdarr_Node/node_modules/engine.io-client/build/cjs/transports/polling.js:352:14)

at Timeout._onTimeout (/app/Tdarr_Node/node_modules/engine.io-client/build/cjs/transports/polling.js:325:30)

at listOnTimeout (node:internal/timers:569:17)

at process.processTimers (node:internal/timers:512:7) {

description: 0,

context: XMLHttpRequest {

UNSENT: 0,

OPENED: 1,

HEADERS_RECEIVED: 2,

LOADING: 3,

DONE: 4,

readyState: 4,

onreadystatechange: [Function (anonymous)],

responseText: 'Error: socket hang up\n' +

' at connResetException (node:internal/errors:720:14)\n' +

' at Socket.socketOnEnd (node:_http_client:525:23)\n' +

' at Socket.emit (node:events:526:35)\n' +

' at endReadableNT (node:internal/streams/readable:1359:12)\n' +

' at process.processTicksAndRejections (node:internal/process/task_queues:82:21)',

responseXML: '',

status: 0,

statusText: Error: socket hang up

at connResetException (node:internal/errors:720:14)

at Socket.socketOnEnd (node:_http_client:525:23)

at Socket.emit (node:events:526:35)

at endReadableNT (node:internal/streams/readable:1359:12)

at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {

code: 'ECONNRESET'

},

open: [Function (anonymous)],

setDisableHeaderCheck: [Function (anonymous)],

setRequestHeader: [Function (anonymous)],

getResponseHeader: [Function (anonymous)],

getAllResponseHeaders: [Function (anonymous)],

getRequestHeader: [Function (anonymous)],

send: [Function (anonymous)],

handleError: [Function (anonymous)],

abort: [Function (anonymous)],

addEventListener: [Function (anonymous)],

removeEventListener: [Function (anonymous)],

dispatchEvent: [Function (anonymous)]

},

type: 'TransportError'

}

 

[PlanetDyna]

On 12/26/2022 at 5:06 PM, ONI said:

Does tdarr work with AV1 encoder? I ordered a Intel Arc A770 16GB gpu just to encode my entire unraid library (170tb) more efficiently and started configuring my tdarr docker pre-emptively. Sadly I don't see an AV1 encoder in the list but on reddit I see people using it with tdarr. So I am curious if there are codecs that the docker does not have access to yet or what I need to do to make it work.

 

Any help is deeply appreciated. 

I would be interested in that as well. Has anyone got it running and can explain the steps?

[matuopm]

On 9/21/2023 at 7:04 PM, almulder said:

Note: Both Tdarr and Tadarr-Node have the same paths setup

Did you add the NVIDIA paths yourself ? Because I don' have these.