April 8, 200818 yr (this also goes to the guy that started the thread about a "fat" version of unRAID... see what you get when Tom adds extra packages) So... in latest betas, MC is in. GREAT! BUT... yes I have a problem (I don't even know if it is an MC issue... see? ). Anyway, I tried to copy my whole NTFS disk I mounted, to the (single for now) disk in the array... All seemed to go ok (left it for the night, although it was going very fast), only to find in the morning that I had many data left out of the copy. Quickly I found the reason. Whatever file contains Greek characters didn't get copied, whatever folder contains Greek characters didn't even go in! (even if inside the filenames where pure latin chars) In MC I see those Greek folders and files (on my NTFS disk, plus the ones I manually copied from network) with trash as Greek. I thought it is not a problem since they display ok over the network (just MC messes them). So for all my disks (since I don't know where I use Greek filenames/folders) I will have first to mount for the "mass" transfer and then REadd it on my USB tray for my Windows machine to finish the job over the network (AND I have to escape all the files that are already in - something I can only do in Vista... do you know another way?)... and suffer the speed. Anyway, reading the directory (ls) in telnet, shows me a mixed Greek-English folder with "escape" characters (like that): -\ Indiana\ Jones\ (\316\244\317\201\316\271\316\273\316\277\316\263\316\257\316\261)/ So the questions are: - Is there any other REAL, underlying problem using full Unicode filenames? - (if not) Is there a way to fix MC from doing this? (I tried altering ASCII modes in MC settings, doesn't help)
April 8, 200818 yr Sorry to learn of the issues with MC with unicode characters. MC is not the only way to copy files in Linux/Unix. My favorite was cpio, but it is not part of the unraid distribution. A very old program to Linux/Unix used to copy files is included on unraid. It is named "tar" (it was historically to copy data to the tape drives for backup. "tar" stands for "T"ape "AR"chive) In any case, the syntax to copy one entire directory tree to another is: cd /mnt/user/data/usb tar -cf - * | (cd /mnt/disk1/ ;tar -xvf -) I'm trying this now on my own unraid, to copy from the mounted backup drive to a disk with some spare space. The first command changes directory to the top of the tree you want to copy. The "cd" inside of the parentheses changes directory for that part of the command in the parentheses to the destination folder.
April 8, 200818 yr with rsync, you don't need tar and cpio to copy. rsync -a . /mnt/diskX copies recursively from the present location to /mnt/disk1 if you add -v you can see progress. if you add -P you can watch the progress incrementally. if you add -n you can test first, then remove -n and do the actuall copy. usually I'll do rsync -avPn . /mnt/diskX after I see what is being copied I redo the command with rsync -avP . /mnt/diskX What I like about rsync is you can also use it to access other hosts either by push or pull rsync -avP gatekeeper:/gatekeeper/MultiMedia/Music /mnt/diskX/music or I can backup the /boot directory rsync -avP /boot gollum:/backups/unraid/boot now normal host syntax requires rsh or ssh. but if you configure and use /etc/rsyncd.conf you can use the following format. rsync -avP rsync://gatekeeper/music /mnt/diskX/music or if I were on gollium rsync -avP rsync://unraid/boot /backups/unraid/boot Here's an example of my rsyncd.conf on unraid. root@unraid:/boot/config/rc.local# more /etc/rsyncd.conf uid = root gid = root use chroot = no max connections = 4 pid file = /var/run/rsyncd.pid timeout = 600 [backups] path = /mnt/user/backups comment = Backups read only = FALSE [music] path = /mnt/user/music comment = Music read only = FALSE [pub] path = /mnt/user/pub comment = Public Files read only = FALSE [boot] path = /boot comment = /boot files read only = FALSE and this line needs to be in /etc/inetd.conf root@unraid:/boot/config/rc.local# grep rsync /etc/inetd.conf rsync stream tcp nowait root /usr/sbin/tcpd /usr/bin/rsync --daemon but that's only if you want to do network transfers using the direct rsync protocol.
April 8, 200818 yr with rsync, you don't need tar and cpio to copy. rsync -a . /mnt/diskX copies recursively from the present location to /mnt/disk1 if you add -v you can see progress. if you add -P you can watch the progress incrementally. if you add -n you can test first, then remove -n and do the actuall copy. usually I'll do rsync -avPn . /mnt/diskX after I see what is being copied I redo the command with rsync -avP . /mnt/diskX Thanks... we never had rsync in the early version of unix I grew up with. I like it as it is included on unRaid AND syntax is easier for Linux newbees. Joe L.
April 8, 200818 yr Author thank you both tar I know is in all Linux distributions... what about rsync? do I need to install something? also, do have to have the .conf file in my case?
April 8, 200818 yr Author ...and since we are at it... I suppose we COULD use cp, cpip, tar or rsync having as destination a user share instead of a disk right? (so that our copy "follows" the rules we've set for the destination user share, like spanning on multiple disks)
April 8, 200818 yr Author ah, rsync is there! (the good thing of having a VPN with home ...I telneted directly to my unRAID... the two disks are still on line so I will probably remote test it... I suppose this will also test the command's ability of not recopying whatever is already there right?)
April 8, 200818 yr I suppose this will also test the command's ability of not recopying whatever is already there right?) That's the beauty of rsync, it can be used to only copy what is newer/modified or at least sync time and permissions. You do not need the conf file if you are syncing/copying locally or have rsh/ssh for network connections. If you do not have/use ssh/rsh and want to do a network rsync, then you will need to configure the /etc/rsyncd.conf file and the /etc/inetd.conf line. If you do not want to configure the /etc/inetd.conf line, then you can run rsync in daemon mode. /usr/bin/rsync --daemon Personally, with my usage pattern, I prefer using inetd so that the daemon is not in memory all the time. See links http://www.fredshack.com/docs/rsync.html http://sunsite.dk/info/guides/rsync/rsync-mirroring.html
April 8, 200818 yr Author thanks I will only use it "locally" via telnet (locally can be from 20Km away, as far as my telnet works)...
April 8, 200818 yr Author ok the problem seems to be worse... (I cannot verify because I am in the middle of a large rsync) even if I ls in the ORIGINAL (ntfs) disk, the folders that start with Greek chars seem to be absent!!! so even rsync doesn't "catch" those!!! maybe the problem is with the whole ntfs filesystem driver... I could get luckier with ntfs-3g (will try to check with wiki about it)
April 10, 200818 yr NLS, here exists a package of software to install support for Greek characters on Slackware. It installs the necessary fonts, etc and says it will work for the system console. It can be found at http://nixbit.com/cat//utilities/slackgreek/ It might provide you with a solution to seeing the correct characters in your file listings. I have not tried it, but then you have not yet attached a zip sample of file/folders to experiment with. Odds are, even if it worked, <bad pun> it would all look like Greek to me anyway </bad pun> Joe L. Edit: looking at its shell script, it needs the following programs to be in place /usr/bin/wget /usr/bin/localedef /usr/bin/unzip since they are not in the default install of unRaid, this script will not work until they are in place. Sorry.
April 10, 200818 yr Author ...sob... ok thanks anyway (but REALLY Tom needs to see this) and yes I will attach what you ask later today
April 10, 200818 yr Author and here you are... (ps. probably need to have those on an mounted ntfs disk to replicate my problem)
April 11, 200818 yr Author OK I have newer news and they are not good at all. Why? - The problem certainly is NOT "Greek" oriented. - The problem UNFORTUNATELY is NOT limited to NTFS volumes mounted on unRAID!!! Involved copy OVER WINDOWS! Read on... Today it was the turn of my MP3 disk to move from NTFS to unRAID. I did the semi-painful task of first rsyncing locally on unRAID (mount NTFS etc.) and then re-add the disk on a USB tray, and make Vista (as I said XP/2003 are not "smart enough" to mix/skip existing folders) copy the rest ("not replacing" what is already copied). Lucky for me I did some size checks after copy. Well with all this intro, you realize what I found: NOT all folder and NOT all files were copied! Yes I am talking about the "local Windows" copy (from NTFS ON Windows to unRAID disk share). Looking deeper into the problem (and this becomes more painful than the initial procedure - that I thought already to be painful enough), I had to check ALL folders and pinpoint the problem. Well most of the folders and files that were in Greek, were copied ok, but going DEEPER in the folder structure (!) they were not! For example: MP3/"greek filename" was copied. MP3/"greek folder"/... was copied. MP3/"english folder 1"/"english folder 2"/"non english filename" was NOT. Going deeper manually (going inside english folder 1 or 2) and retrying the copy, made things work! I also had some songs that the filenames were using French and Spanish (Pink Martini stuff for example) and the problem was there too!!! Now that makes the problem MUCH more major than "little Greece's" problem... (bold so that you notice) Internationalization seems to need some attention (if this product wants to sell internationally). I wonder how this wasn't noted before (then again I haven't tried anything before 4.3.4b6).
April 11, 200818 yr Author More info (at least this is for SOME good news). After I made the difficult semi-manual copy I mention above, I needed to move the whole lot to another disk (the disk I used was a "transit" for emptying another disk and installing this in turn to the array). Thankfully, rsyncing that (reiserfs) to the other (reiserfs) seems to work (moving all files, inc. Greek etc.). Of course looking at the console during the process is pretty funny (seeing huge lists of escape codes scrolling, among "normal" english filenames). Like: AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#225\#316\#276\#316\#261\#316\#271\#317\#201\#316\#277\#317\#215\#316\#275\#317\#204\#316\#261\#316\#271.mp3 3133440 100% 3.45MB/s 0:00:00 (xfer#17007, to-check=2232/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#232\#316\#273\#316\#265\#316\#257\#317\#203\#316\#265 \#316\#246\#317\#216\#317\#204\#316\#261 \#316\#232\#316\#273\#316\#265\#316\#257\#317\#203\#316\#265 \#316\#234\#316\#254\#317\#204\#316\#271\#316\#261.mp3 3373184 100% 3.33MB/s 0:00:00 (xfer#17008, to-check=2231/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#234\#316\#261\#317\#201\#316\#263\#316\#261\#317\#201\#316\#257\#317\#204\#316\#265\#317\#202.mp3 4022272 100% 3.53MB/s 0:00:01 (xfer#17009, to-check=2230/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#237\#316\#271 \#316\#247\#317\#211\#317\#201\#316\#271\#317\#203\#316\#274\#316\#255\#316\#275\#316\#277\#316\#271.mp3 3330048 100% 19.02MB/s 0:00:00 (xfer#17010, to-check=2229/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#237\#317\#205\#317\#201\#316\#261\#316\#275\#316\#255.mp3 5681280 100% 15.52MB/s 0:00:00 (xfer#17011, to-check=2228/20682) Now if someone can "cd" to such a folder and do something on such a filename using the console... comes close to the days I could code in assembly (in fact it is even worse, like using a machine language monitor)... Hehehe... Anyway, the problem remains though and I can definitely baptize it "problem" now.
April 11, 200818 yr More info (at least this is for SOME good news). After I made the difficult semi-manual copy I mention above, I needed to move the whole lot to another disk (the disk I used was a "transit" for emptying another disk and installing this in turn to the array). Thankfully, rsyncing that (reiserfs) to the other (reiserfs) seems to work (moving all files, inc. Greek etc.). Of course looking at the console during the process is pretty funny (seeing huge lists of escape codes scrolling, among "normal" english filenames). Like: AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#225\#316\#276\#316\#261\#316\#271\#317\#201\#316\#277\#317\#215\#316\#275\#317\#204\#316\#261\#316\#271.mp3 3133440 100% 3.45MB/s 0:00:00 (xfer#17007, to-check=2232/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#232\#316\#273\#316\#265\#316\#257\#317\#203\#316\#265 \#316\#246\#317\#216\#317\#204\#316\#261 \#316\#232\#316\#273\#316\#265\#316\#257\#317\#203\#316\#265 \#316\#234\#316\#254\#317\#204\#316\#271\#316\#261.mp3 3373184 100% 3.33MB/s 0:00:00 (xfer#17008, to-check=2231/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#234\#316\#261\#317\#201\#316\#263\#316\#261\#317\#201\#316\#257\#317\#204\#316\#265\#317\#202.mp3 4022272 100% 3.53MB/s 0:00:01 (xfer#17009, to-check=2230/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#237\#316\#271 \#316\#247\#317\#211\#317\#201\#316\#271\#317\#203\#316\#274\#316\#255\#316\#275\#316\#277\#316\#271.mp3 3330048 100% 19.02MB/s 0:00:00 (xfer#17010, to-check=2229/20682) AUDIO/OK - Old/\#316\#224\#316\#267\#316\#274\#316\#267\#317\#204\#317\#201\#316\#257\#316\#277\#317\#205, \#316\#206\#316\#275\#317\#204\#316\#266\#316\#265\#316\#273\#316\#261 - \#316\#237\#317\#205\#317\#201\#316\#261\#316\#275\#316\#255.mp3 5681280 100% 15.52MB/s 0:00:00 (xfer#17011, to-check=2228/20682) Now if someone can "cd" to such a folder and do something on such a filename using the console... comes close to the days I could code in assembly (in fact it is even worse, like using a machine language monitor)... Hehehe... Anyway, the problem remains though and I can definitely baptize it "problem" now. Give me a good 1GL... (I grew up hand assembling stuff) Here is how you can deal with those files/folders from the console. The "-i" option to the "ls" command will print the "inode" number of the file or directory. (Think of inodes as the internal name, and the human readable name (or in this case, unreadable name, a pointer to the inode) So cd /mnt/wherever_your_files_reside ls -i The first number in the line is the inode number If the inode number of the file you want to rename is 97315, then the following will rename it to the new_file_or_folder_name you specify. find . -inum 97315 -exec mv {} new_file_or_folder_name \;
April 12, 200818 yr Author ok makes sense - but hope I will never actually need it ...wiki needs some "international users' problems" section?
Archived
This topic is now archived and is closed to further replies.