Jump to content

WeeboTech

Moderators
  • Posts

    9,472
  • Joined

  • Last visited

Posts posted by WeeboTech

  1. Interesting topic, thanks for bringing it up.

     

    That was a very interesting article.  It does contain some merit and food for thought.

    I've yet to encounter the situation.

    I found it interesting on how a date is chosen as a point in time. In reality, it is size vs MTBF and BER on devices.

     

    In some sense it makes me think unRAID should have multiple smaller arrays.

    It should do this by dividing the larger array into smaller chunks protected by individual parity drives. 

    It was interesting in how the author states RAID6 is not much more protection then RAID5.

     

    A comment by a reader makes allot of sense

    the proper procedure for raid maintenance is to 1) always have a cold spare and 2) always replace failed drives with a new drive, and return the old drive for warranty maintenance.

     

    This is how I've handled things in the past.

     

    What I would like to add is, the chance of multiple drive issues is increased greatly when you purchase multiple drives at the same time.

    I've seen this issue over and over again, not just with RAID5. 

    In our web server farm, if too many drives were bought from one vendor at the same time. We would see them fail with a pattern.

    My suggestion, which worked out well, was to choose multiple vendors and to spread the purchase over time.

     

    In some sense, it's interesting to see the warnings of building large arrays with unRAID.

     

    Points to consider in the unRAID community,

     

    Multiple drive purchases at one time... (as explained above)

     

    Using old drives to gain every last ounce of use out of it. This is one fo the great things about unRAID. I can use a hodge podge of old drives. Yet, chances of multiple drive failures are greatly increased when trying to get every last ounce of life out of it.

     

    Periodic parity checks - I'm a  proponent of this procedure also. In fact I think it should be added as a standard unRAID feature allowing scheduling just like the mover script.  There are raid cards that do this automatically.  A new term to me, called "BIT ROT" makes sense here. Sectors go bad, periodic scans and checks will help you become aware of potential issues allowing you to be proactive, not reactive.

     

    SMART monitoring -  I'm a  proponent of this also, again,  I think it should be added as a standard unRAID feature.

     

    Cold or Warm Spare - This is a budget concern, but the faster you replace a disk, the less likely you will see a multiple drive failure. This does not save you from bit rot. LOL.. Periodic parity checks are the only way around that one!

     

     

    Some questions to ponder...

     

    >> With the relatively slow speed of unRAID parity creation or disk recovery.

    Is it really that slow.. You have to consider how many spindles there are involved, then bus speed.

    Parity check is reasonable if you have drives arranged on efficient buses. Parity create is slower because there is a write involved.

     

    >> That combined with the fact that during these periods the disks are thrashing their asses off increasing the chance of failure.

    and in this case, If the array is idle from use and just creating parity, are they really thrashing, or just reading sequentially.

    To me thrash is significant inefficient movement, Yet during a parity check/create, access is efficient as long as nothing else is causing movement in the machine.

     

    >> e.g. "As we now know, a single disk failure means a second disk failure is much more likely."  Do we REALLY know that?

    I agree. I've always questioned this way of thinking. It's not a given as so many people will repeat... Yet the chance of impending failure is greatly increased for drives purchased at the same time and within the same batch of production. I've seen this countless times over the years. One bad batch can wreak havoc.

     

    >> this is one reason I do not support the idea of replacing smaller drives in unRAID by replacing them with larger ones and letting them rebuild.  I prefer to put the new, larger drive in the system, copy the data, then stop the array and remove the old drive.

     

    What is the difference here?

    Do you feel the rebuild is susceptible to bit rot and you will not be able to reconstruct the new drive reliably?

    If the drive has been removed and replaced, you still have the old data on the old drive.

    If before replacement, a parity check is executed, then chances of reading every drive is higher.

    if a drive is swapped with a larger drive, and the build fails, you have the chance of putting the old drive back and potentially accessing the second failed drive virtually.

    Just some thoughts.

    So what does adding a new drive to the system buy you vs swap replacement?

    A new drive requires parity calculation to be execuited at siome point.

    removal of the old drive requries parity calculation. (of course these can be combined into one step)

     

  2. I've got plenty of effort, I'll try that route. I completely forgot about a regular Slackware install. Thanks WeeboTech.

     

    You do not have to do a full slackware install, only the relevant packages.

    Not sure if you need LPR or CUPS.

    In the past I got it working with just lpr.

    You may have to tweak the samba configuration thereafter if you use SMB style printing. (I've never done this).

  3. - Print Server

     

    I know Tom spent a lot of time trying to remove the print server functionality, but I'd still like to have this feature.

     

    I think this has some merit, but with some of todays printers that have embedded ethernet, you can get the same thing without requiring a server.

    My HP printer has an Ethernet port and software which handles everything without the need of a centralized print server.

  4. Yeah, I didn't until last week :D.  Now if I could just find a rail/enclosure system that lets you adapt a 2.5" drive to a 3.5" slot that would work in my trayless box, I'd be set...

     

    You might be better off selling that and putting the money towards a larger drive.

    It may slow down your monthly parity checks. (You do those right?)

  5. I need to change out my power supply, as it's a tri-rail, with 19a on each

     

    if you use the calculator on Corsair's website they will probably say this power supply is enough.

    The website says specifically -

     

    Triple 12V Rails provide independent reliable power to the CPU, video card and other components with a combined rating of 50A (40A on 520W) maximum! Advanced circuitry design that automatically enables power sharing between the triple +12V rails in an event of overload on any single +12V rail.

     

    >> Planned Expansion (on the off-chance that I might need more than 9 drives):

    Not sure you need all those parts. You could go with 5in3s in a good case and not have any outside wiring.

    So if you were to double check your power supply and needs, then consider the 5in3's and a new case, you could fit it all in one place.

     

  6. My Larger Rig. This has been a long time coming..  ;D

    Although I have the mini-itx version, I finally finished building a larger cost effective version

     

    Parts List

     

    Abit AB9 PRO

    2.6ghz Core 2 Duo - Used on eBay - a Celeron 440 2Ghz is quite sufficient.

    8gb OCZ PC6400 RAM - 4G is sufficient for 99% of the people.

    Centurion 590 - Great cheap case with allot of fan placement.

    Seasonic 500W Power Supply (it was laying around idle)

    Extra 120MM fans ~ $15 (box of 4 coolermasters).

     

    Kingston 2GB Secure Digital (SD) Flash Card w/MobileLite Reader Model FCR-ML+SD/2GB - http://www.newegg.com/Product/Product.aspx?Item=N82E16820134548

     

    iStarUSA T-5-SA 1x5.25" Bay Trayless Anti-vibration SATA Mobile Rack

    http://www.newegg.com/Product/Product.aspx?Item=N82E16816215046

     

    Additional items. 

    Gigabute I-RAM PCI Ram disk

    http://www.gigabyte.com.tw/Products/Storage/Products_Overview.aspx?ProductID=2180

     

    This is for a future application to allow me to use the server as an NFS server for other machines.

    IDE to CF adapter + 4GB CF card (to backup the battery backed ramdisk.. just in case).

    Cheap PCIe 4x video card from eBay.. which will end up getting dropped for a 1X.

     

    Masscool PCIe 1x 2 SATA port controller (for the ramdisks).

    http://www.newegg.com/Product/Product.aspx?Item=N82E16815280008

     

    I'll probably upgrade to a 4X PCIe 4x 4 port adapter to expand to 12 or maybe 15 drives in the future..

    For  right now it seems just as I fill the slots, new drives come out that are large enough to replace current drives.

     

     

    Notes:  I really love the trayless sata design.

    They can use MOLEX or SATA power connectors.

    I choose this model in particular because the vents are larger then other models.

    Some of the other models have the vents blocked a little by the drive. This one has slightly more leeway for air flow.

    I also choose this model because they are fanless. I use the large 120MM fans to suck the air from the front.

    This requires taping up open vents on the case to keep the drives cool.

    I choose this design so I did not have a bunch of tiny fans spinning and used the larger fans instead.

    It's working well for me.

     

    These trayless are not perfect though. When you open the key, it does not power down the drive.

    So you probably want to replace drives while the machine is off.

    Also, the drives tend to get sort of stuck on the inside, so you wiggle them a little and they pop out.

    Still for me I like the idea of swapping bad (or upgrading) drives like they are floppies.

    No tray, no screws, Just unlock, open, wiggle, replace, close, lock, boot, click, click, and you are up and running.

    Also, The SATA cable does not lock into the trayless. in theory, the cable could eventually loosen due to drive vibration.

    Only happened once in the last 6 months.

     

     

    I plan to upgrade eventually to the 5in3 trayless modules, but for right now I do not need more then 9 drives.

     

    I have more pictures here, which shows different views and where to tape up the case.

    http://www.cotrone.com/rob/gallery/Atlas

     

     

  7. You could change the home directory of the user in question to a physical drive on your machine.

    Then the files placed there will continue to be there with customizations after each reboot.

    However, you will probably have to reinstall the package upon each reboot or possibly alter the bzroot image for a more permanent change.

  8. do a

    cat /etc/resolv.conf

     

    Chances are your name is not resolving

    You can substituite the IP address for the hostname as in the example below

     

    root@Atlas:/mnt/disk2/bittorrent# ifconfig eth0

    eth0      Link encap:Ethernet  HWaddr 00:50:8D:9D:7B:AA 

              inet addr:192.168.1.179  Bcast:192.168.1.255  Mask:255.255.255.0

              UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1

              RX packets:263331527 errors:0 dropped:0 overruns:0 frame:0

              TX packets:255971791 errors:0 dropped:0 overruns:0 carrier:0

              collisions:0 txqueuelen:1000

              RX bytes:4217610711 (3.9 GiB)  TX bytes:3314289142 (3.0 GiB)

              Interrupt:16

     

    root@Atlas:/mnt/disk2/bittorrent# /usr/bin/rsync rsync://192.168.1.179/

    backups        Backups

    vmware          VMWare Backups

    music          Music

    pub            Public Files

    boot            /boot files

    mnt            /mnt files

    Videos          VIDEOS

    bittorrent      BITTORRENT

  9. Try these

     

    root@Atlas:/mnt/disk2/bittorrent# whereis rsync

    rsync: /usr/bin/rsync /usr/bin/X11/rsync

    root@Atlas:/mnt/disk2/bittorrent# ls -l /usr/bin/rsync

    -rwxr-xr-x 1 root root 276688 Jun 14 23:04 /usr/bin/rsync*

    root@Atlas:/mnt/disk2/bittorrent# /usr/bin/rsync rsync://Tower

    backups        Backups

    vmware          VMWare Backups

    music          Music

    pub            Public Files

    boot            /boot files

    mnt            /mnt files

    Videos          VIDEOS

    bittorrent      BITTORRENT

     

    Also do

     

    echo $PATH

  10. I do not define these paths.

    The way I understand it. I define the paths to the target directory with

    rsyncd.conf file and then deltacopy client fetches it from the server by the

    name whatever you call it ex. "myshare"

    [myshare]
    path = /mnt/disk1/myshare
    comment = Backup 
    read only = FALSE  
    list = yes
    

    but in my case communication with rsync server works but I can not fetch "myshare".

    Then again I might be wrong :)

     

     

    On your Tower server type  rsync://Tower

    Where Tower is equal to your real hostname.

     

    On mine it is as follows

     

    root@Atlas:/mnt/disk2/bittorrent# rsync rsync://Atlas

    backups        Backups

    vmware          VMWare Backups

    music          Music

    pub            Public Files

    boot            /boot files

    mnt            /mnt files

    Videos          VIDEOS

    bittorrent      BITTORRENT

  11. WeeboTech will probably be here soon with the more expert answer, and I haven't actually used rsync (!), but apart from that, wouldn't you want to use the UNC paths here?

    //tower/disk1/Backup/MKV

    Or if you have set up a User Share for Backup, then

    //tower/Backup/MKV

     

    Proper syntax is

     

    rsync://hostname/myshare/filename

     

    Where Hostname is the destination Host (like http)

    Where myshare is the section defined in the /etc/rsyncd.conf file as in [myshare]

    Everything after myshare/ refers the the location appended to the path = /mnt/disk1/myshare

     

    So If I had a section called /mnt

     

    [mnt]

    path = /mnt

     

    Then you could do a

     

    rsync file... rsync://Tower/mnt/disk1/somedirectory/

  12. >> Am I supposed to make any entry to /etc/inetd.conf file ?

     

    The script S20-init.rsyncd does that for you. (or you could do it manually)

     

    >> What is a rsyncd.pid file.

    It's a semaphore file to signal the daemon is running. It will contact the active pid when rsync is running in daemon mode from inetd.

    it will only be there when you have an active rsync running.

     

     

    >> i just opened some file and edited it to look like rsyncd.conf and saved it as rsyncd.conf, was that OK?

    As long as it was saved as rsyncd.conf, that is fine.

  13. >> Do both locations of rsyncd.conf  suppose to be on flash :

     

    no /etc is on the root filesystem

    /boot/custom/etc is on the flash.

     

    if you edit the file in /etc then that's where it will exist.

    if you edit it in /boot/custom/etc. that's where it will exist.

     

    Perhaps better to edit as /boot/custom/etc/rsyncd.conf

    then cp /boot/custom/etc/rsyncd.conf /etc/rsyncd.conf

     

    Where and how are you doing the edit?

    Are you telneting in and editing with vi or doing it via windows?

  14. I've no experience with Deltacopy.

    I know when using rsync it has a URI type interface as in

     

    rsync://hostname/myshare/filename

     

    myshare is the block defined as

     

    [myshare]

    path = /mnt/disk1/myshare

    comment = Backup

    read only = FALSE 

    list = yes

     

  15. >> 1. Do I create file "rsyncd.conf"  similar to yours and place it in /etc/rsyncd.conf

     

    Yes, but you will also need to copy and save this file somewhere on your flash drive.

    We recommend /boot/custom/etc.

    So you should have /etc/rsyncd.conf and /boot/custom/etc/rsyncd.conf

    which should be the same.

     

    2 .Create a file "S20-init.rsyncd" looking like thi

    Yes. Create it as /boot/custom/etc/rc.d/S20-init.rsyncd

     

    #!/bin/bash
    
    if ! grep ^rsync /etc/inetd.conf > /dev/null ; then
    cat <<-EOF >> /etc/inetd.conf
    rsync   stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/bin/rsync --daemon
    EOF
    read PID < /var/run/inetd.pid
    kill -1 ${PID}
    fi
    
    cp /boot/custom/etc/rsyncd.conf /etc/rsyncd.conf
    

     

    Notice the last line which copies the saved version of rsyncd.conf to the /etc directory.

     

    >> 3.What do you mean call it from /boot/config/go script

     

    THere is a script in the /boot/config directory called "go"

    Modify this script with your editor.

    Add the following line at the end of it to call S20-init.rsyncd

     

    /boot/custom/etc/rc.d/S20-init.rsyncd

     

    • Like 1
  16. What are you trying to do with rsync on the unraid system?

     

    Here's an example of my /etc/rsyncd.conf file

    it gets put in place from /boot/custom/etc/rsyncd.conf to /etc/rsyncd.conf

     

    uid             = root
    gid             = root
    use chroot      = no
    max connections = 4
    pid file        = /var/run/rsyncd.pid
    timeout         = 600
    
    [backups]
        path = /mnt/user/backups
        comment = Backups
        read only = FALSE
    
    [vmware]
        path = /mnt/user/backups/vmware
        comment = VMWare Backups
        read only = FALSE
    
    [music]
        path = /mnt/user/music
        comment = Music
        read only = FALSE
    
    [pub]
        path = /mnt/user/pub
        comment = Public Files
        read only = FALSE
    
    [boot]
        path = /boot
        comment = /boot files
        read only = FALSE
    
    [mnt]
        path = /mnt
        comment = /mnt files
        read only = FALSE
    
    [Videos]
        path = /mnt/user/Videos
        comment = VIDEOS
        read only = FALSE
    
    [bittorrent]
        path = /mnt/user/bittorrent
        comment = BITTORRENT
        read only = FALSE
    

     

     

    Here is a script I use to update /etc/inetd.conf so that it runs rsync when an rsync connection comes in.

     

    It goes in /boot/custom/etc/rc.d/S20-init.rsyncd

    The call it from your /boot/config/go script.

     

     

     

    #!/bin/bash
    
    if ! grep ^rsync /etc/inetd.conf > /dev/null ; then 
    cat <<-EOF >> /etc/inetd.conf
    rsync   stream  tcp     nowait  root    /usr/sbin/tcpd  /usr/bin/rsync --daemon 
    EOF
    read PID < /var/run/inetd.pid
    kill -1 ${PID}
    fi
    

×
×
  • Create New...