Spitko Posted August 24, 2019 Share Posted August 24, 2019 (edited) Ok this is the weirdest thing I've seen, but it's the last quirk preventing me from retiring my synology box, so here we go. I migrated all the data over via a docker image to the new shares. This seemed to work fine, and I can view the files in explorer, add/remove/edit/open them just fine. HOWEVER, certain programs are unable to read files. They can traverse directories, but will either be unable to see any files, or give obtuse errors when trying to display or open them. I've confirmed this behavior with both Gimp and VLC under windows. For Gimp, the file open dialogue errors out when files are present in the path, and for VLC it fails to open the file (but the file open widget works just fine) The shares themselves are pretty straightforward. They're default configured public shares, and on the windows machines I've tried both mapping them to network drives, or going via UNC paths. I've also confirmed this behavior on two machines, with completely different configurations. I was able to repro this in some software I wrote, and the behavior is similar; I can check if a directory exists and it will return expected results. But checking if a file exists will return false regardless, and attempting to stat or open a file will produce "not found" errors. Running as admin doesn't appear to affect the behavior. However, the synology box does not exhibit these problems. Any ideas on what might be causing this? EDIT: Partial solution/workaround here, may require further investigation to prevent this from happening to others Edited August 28, 2019 by Spitko Quote Link to comment
trurl Posted August 24, 2019 Share Posted August 24, 2019 Tools - Docker Safe New Perms Quote Link to comment
Spitko Posted August 24, 2019 Author Share Posted August 24, 2019 (edited) I couldn't find a "docker safe" anything under tools, but there was a "New Permissions" tool which looks like the right thing perhaps? unraidip/Tools/NewPerms Ran it on all disks against one of the shares I've been testing with. No change in behavior. Edit: Also, to follow up on permissions, here's one of the affected files: -rw-rw-rw- 1 nobody users 976202725 Aug 17 00:50 test.mp4 Edited August 25, 2019 by Spitko Quote Link to comment
Spitko Posted August 28, 2019 Author Share Posted August 28, 2019 (edited) Further testing on this issue: 1) Found the "Docker safe new perms" tool via a plugin, however running this didn't yield any different results 2) Tried making a new directory and pointing my script at that; it was able to write files fine. Files written this way can be read just fine. 3) Permissions between the "bad" files and good ones appear identical, including user and group 4) Also tried logging in with a user account. This creates file under the correct user (using the "users" group) but those files can still be read as nobody just fine. I'm very confused now. I'd also like to get this sorted out before my trial runs out if possible (1 day left), so if anyone has any ideas on what to check or try please let me know. Edited August 28, 2019 by Spitko Quote Link to comment
remotevisitor Posted August 28, 2019 Share Posted August 28, 2019 Is there any difference between the permissions/owner/group of the directory which has the problem files and one which does not? Does the directory name (and any parent directory names) have any special characters in them? Quote Link to comment
Spitko Posted August 28, 2019 Author Share Posted August 28, 2019 (edited) Nope, permissions are identical between the files in question. Names as well; I even tried renaming the old folder, making a new one with the same name as the old, and everything worked fine in that one (ie, my test program could now create, read, and write files to the new folder). If I rename the old folder back the problem recurs. I checked permissions ls -n as well, and ensured there weren't just two groups named "users" or something; the permissions are absolutely identical unless there's some additional bit/flag I'm not aware of that doesn't show in ls -ln It's honestly the weirdest dang thing. Also, to be clear: the folder exists in /mnt/user, which is where I copied the files in docker and where I'm checking permissions from. I assumed copying straight to the diskN paths would be a bad idea (also because I copied more than a drive's worth of data). Also as a reminder, the weirdest (by far) part is that I can manage the files from Explorer just fine. I can open them in Mediaplayer Classic without issue, I can rename and delete them, etc. But VLC will reliably give an 'VLC is unable to open the MRL' error. BUT, if I take the exact same file, copy it to a different folder on the same share with Explorer, it works fine! This is all from the same windows machine; and at no point am I getting UACed or asked to do anything additional. New file permissions: -rw-rw-rw- 1 nobody users 605445386 Aug 7 23:41 Test.mp4 Old file permissions: -rw-rw-rw- 1 nobody users 605445386 Aug 7 23:41 Test.mp4 I can also take this new file, rename it, and copy it back to the old folder and it still plays fine. And to throw out it being a VLC-specific quirk, I get similar results in GIMP; so it does seem to be oriented around programs likely using cross-platform toolkits like GTK. Also worth noting that VLC can play the same file from the old NAS (Synology) just fine as well, using the same mechanisms (Mapped network drive over SMB) Edit: I also found another thread from 2018 with the same problem (By searching the exact VLC error, a cryptic "filesystem error: read error: No error"), no solution though. Edited August 28, 2019 by Spitko Quote Link to comment
Spitko Posted August 28, 2019 Author Share Posted August 28, 2019 Update! I think I found the issue; it might be a mixture of a Samba/SMB bug and possibly an Unraid bug, or alternatively a bug in Krusader (as shipped by binhex) I did a bit more digging and statted two (different) files, one that worked and one that didn't. File: Bad.mp4 Size: 120134495 Blocks: 234640 IO Block: 4096 regular file Device: 21h/33d Inode: 4157 Links: 1 Access: (0666/-rw-rw-rw-) Uid: ( 99/ nobody) Gid: ( 100/ users) Access: 1969-12-31 15:59:59.000000000 -0800 Modify: 2019-08-16 15:28:00.169449270 -0700 Change: 2019-08-27 19:07:52.107529449 -0700 Birth: - File: Good.mp4 Size: 182610839 Blocks: 356664 IO Block: 4096 regular file Device: 21h/33d Inode: 3967 Links: 1 Access: (0666/-rw-rw-rw-) Uid: ( 99/ nobody) Gid: ( 100/ users) Access: 2019-08-27 19:21:43.859255029 -0700 Modify: 2019-08-27 19:52:42.613932984 -0700 Change: 2019-08-27 19:52:42.613175754 -0700 Birth: - The only real difference I can see here is that the bad file has an invalid/missing access time. So, as a test, I did touch Bad.mp4 and suddenly it works fine. As a note, opening the file in media player doesn't seem to update the access time; I assume this is a (reasonable) optimization, meaning the only way to unstick the bad files it to write a script to touch them all. Which might be a fine workaround, but before I do that, does anyone want to dig deeper here, or have a slightly less brute-force solution? Quote Link to comment
Frank1940 Posted August 28, 2019 Share Posted August 28, 2019 (edited) The real question here is what caused bad.mp4 not to have a Access date info generated when it was created. Do you know what program created/generated this file? It sounds like a 'bug' and should be reported. There is a sub-forum for Unraid to do just that but you need a bit more information first regarding what has to occur to reliably generate it. There is one solution that might be done that could 'fix' this issue is to have both New Permission scripts do a 'touch' as part of their action. However, I can see where this might cause other issues as it would update all of the dates on every file... EDIT: See here for why the time on the access time is as it is: https://www.a2hosting.com/blog/whats-the-deal-with-12-31-1969/ Edited August 28, 2019 by Frank1940 Quote Link to comment
Spitko Posted August 29, 2019 Author Share Posted August 29, 2019 I'm a programmer, so I'm all too familiar with unix time stamps. The files were created with this docker image, by mounting a remote SMB share and copying the files from the old server to the local one. It looks like creating files locally doesn't reproduce this bug; it's likely specifc to SMB->local transfers. Quote Link to comment
Frank1940 Posted August 29, 2019 Share Posted August 29, 2019 8 hours ago, Spitko said: I'm a programmer, so I'm all too familiar with unix time stamps. Sorry if I might have appeared as 'talking down' but there are so few people here that have even a smattering of Linux knowledge. (I, personally, know and remember just enough to make me really dangerous...) I hope you will consider participating in providing support on the forum. Here is where to file that bug report: https://forums.unraid.net/bug-reports/ Be sure to read what details and information should be included. Quote Link to comment
unraidun Posted December 17, 2021 Share Posted December 17, 2021 On 8/28/2019 at 12:48 AM, Spitko said: So, as a test, I did touch Bad.mp4 and suddenly it works fine. As a note, opening the file in media player doesn't seem to update the access time; I assume this is a (reasonable) optimization, meaning the only way to unstick the bad files it to write a script to touch them all. Which might be a fine workaround, but before I do that, does anyone want to dig deeper here, or have a slightly less brute-force solution? Can confirm this solved the problem. I was also unable to play files in VLC (other media players worked fine). But as soon as I touched a file, VLC suddenly started working. Did you end up writing a script to recursively scan all directories and touch all files? Mind sharing? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.