Ph9214

Members
 View Profile  See their activity

  • Posts

    66

  • Joined

  • Last visited

 Content Type 

Posts posted by Ph9214

    Automatic NUMA Isolation For Threadripper Systems

    in Feature Requests

    Posted

    9 minutes ago, Jerky_san said:

    I hope for something. Although I create machines in the boundaries of NUMA and such. I think the ram is not completely isolating or something. I have large DCP latency spikes that I cannot get rid of. I've worked on it for days now trying different configs. Spinning up different machines. Nothing seems to work. I wonder if its not my board or the 2990wx is just not able to not have these spikes constantly.

    yes, I have the latency spikes too, although it is mostly un-knoticable it is annoying as h*ll that I cant isolate the memory when it is a listed feature of kvm and libvirt from redhat the lead developers!

    Help Setting Up Split Threadripper System

    in General Support

    Posted · Edited by Ph9214

    I am setting up a Threadripper 1900X 8-Core 16-Thread 2-NUMA system that I am trying to split into 2 VMs each using resources form only one NUMA for hopefully less latency than bare metal. However I am having difficulty isolating the memory as my attempts so far have failed. I have disabled membaloon and followed some instructions form redhat and put 

      <numatune>
    <memory mode='strict' nodeset='1'/>
  </numatune>

    in my xml but it has been ignored as reveled by "numastat"ing qemu-kvm. If you have set up a Threadripper system and have managed to isolate VMmms, please enlighten me. :)

     

    main.xml

    Build a new unRAID machine or buy NAS storage?

    in Lounge

    Posted

     

    Erm.. You dont have parity drives with a raid setup.. So I do not get that reference..

     

    The cache drive can be an SSD for the simple reason that a smaller drive is large enough.. There is no benefit in having the cache drive be as large as an array drive..

     

    Are you actually familiar with unraid ?  If not that is not an issue, we all started with it at some point, but it might be wise to be carefull in beiing very certain about some stuff if you're not really.. Just a tip.. (not for me, but for other new users who are looking for solid info)

     

    sorry, I'm pretty inexperienced with unraid but what I probably should have said was don't use a cache drive on a share where you store vdisks that are larger than the cache drive :P and you actually do use parity drives in raid 5 and 6 and they are usually the main limiting factor on writes.

     

    In RAID5 the parity is stripped accross all the disks in the array. So a file you write is written simultanteoulsy to all drives in the array. In effect the filesystem is split over a multitude of disks, where data to recover from a failed disk (parity info) is also written against all drives.

     

    In unraid all disks have their own filesytem and parity is maintained on a seperate disk. (or, if you want to protect against dual drive failure) on two seperate disks. .

     

    Wrt to the comment on vdisks and array and cache drive size: a vdisk you would not store in the array (and use a cache drive for), you would probably have it sit on the cache drive also for reasons of speed (or, if speed is not a thing, you would have it on the array constantly). The VDISK would never move back to the cache drive (that mechanism is only used when ADDING files, not when changing them).

     

    edit: it turns out I was wrong and rushed to an incorrect conclusion and misread the diagram https://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5 striping the parity wouldn't work.

    Let's Talk C-Groups, Passthrough, Vt-d, Motherboards, ACS Override

    in Lounge

    Posted

    Would really like to see some community testing on this thread.

    the asrock z170 extreme7+ and asrock z170 gaming i7 (which are basically identical) have 3 isolated 16x lanes but the 4th (top middle) 16x lane and all the other pcie lanes are grouped with the chipset and sata controllers and the usb ports are also bonded to the chipsets groups, I can post a full copy paste of the system devices page when I get home.

    Access problem with Windows App Access to unRaid SMB Share?

    in General Support

    Posted

    (I  have always had a rule that working with computers is a bit like playing a game where you don't know all of the rules.  That 'They' make the rules and you have to play by their rules!!!)

     

    So-Very-True!!!

     

    yep it's probably the fact that the name is identical to a windows user folder name :P so try naming it Docs or whatever you feel like other than the standard user folders which include:

     

    I would agree that both you have identified the problem.  However, I did fix it (using the "documents" as a share name) by creating a new share (identical config) using Documents2, then verified it worked, then deleted the documents share, then renamed Documents2 to Documents.  And everything is working.

     

    Given I agree with your assessments on the 'default' names, why would it now work?  Seems like as soon as I renamed Documents2 to Documents, the conflict should show-up again...

     

    Thoughts?

     

    it may be that the actual path and not the name is still Documents2 I really have no idea, it could also be that somehow your share config just got messed up somehow

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

    Just curious what the use case is for this.  With the speeds of modern SSDs, and the typical uses of UnRAID, I don't see any advantage to even bothering with a RAM Disk vs. a high-speed cache drive.  Also, your first post says "This guide" => but there's no link to anything -- where is the actual guide?

     

    oops forgot the link :P and you are right, but just in case you had insane amounts of ram this could be useful, I thought it would also be intresting

     

    So did you forget it again  :)

    I assume you meant to include it with this comment, but there's still no link  8) 8)

     

    You just missed it Gary, it's a RAM link, it's not persistent....  ;D

     

     

      ;D ;D ;D

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

    Just curious what the use case is for this.  With the speeds of modern SSDs, and the typical uses of UnRAID, I don't see any advantage to even bothering with a RAM Disk vs. a high-speed cache drive.  Also, your first post says "This guide" => but there's no link to anything -- where is the actual guide?

     

    oops forgot the link :P and you are right, but just in case you had insane amounts of ram this could be useful, I thought it would also be intresting

     

    So did you forget it again  :)

    I assume you meant to include it with this comment, but there's still no link  8) 8)

     

    sorry I just updated the original post and didn't think about putting it in the reply :P

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

     

     

    sorry about that, I meant to say, don't put ANY files on a tempfs that is not backed up frequently (like every minute), I have experienced the horror of my batch file (in the home directory) disappearing after a reboot!

     

    I have a Command Line plugin that backs up /root on system shutdown then restores it on startup. It's useful for bash history, ssh authorized keys, mc and htop settings and any scripts you have there. It also includes shellinabox, which is a web based terminal. And an awesome ascii lime and system info when you log in.

     

     

    what is it?

    http://lime-technology.com/forum/index.php?topic=42683.msg406446.msg#406446

     

    thx I'll try that when I get home :)

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

     

     

    sorry about that, I meant to say, don't put ANY files on a tempfs that is not backed up frequently (like every minute), I have experienced the horror of my batch file (in the home directory) disappearing after a reboot!

     

    I have a Command Line plugin that backs up /root on system shutdown then restores it on startup. It's useful for bash history, ssh authorized keys, mc and htop settings and any scripts you have there. It also includes shellinabox, which is a web based terminal. And an awesome ascii lime and system info when you log in.

     

     

    what is it?

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

    Just curious what the use case is for this.  With the speeds of modern SSDs, and the typical uses of UnRAID, I don't see any advantage to even bothering with a RAM Disk vs. a high-speed cache drive.  Also, your first post says "This guide" => but there's no link to anything -- where is the actual guide?

     

    oops forgot the link :P and you are right, but just in case you had insane amounts of ram this could be useful, I thought it would also be intresting

    Writing to Multiple Disk at Same Time

    in General Support

    Posted

    I don't know how accurate the high water allocation is but you should be able to start multiple simultaneous transfers by dragging them one at a time (in windows)

     

    alternatively, you could split the share you are copping to into multiple disk exclusive shares and sim link them into the main share folder

    eg share names: Main Movies, Movies Disk 1, Movies Disk 2, Movies Disk 3

     

    http://stackoverflow.com/questions/1951742/how-to-symlink-a-file-in-linux

    Persistent RAM Disk (Still Dangerous)

    in General Support

    Posted

    This guide http://www.observium.org/docs/persistent_ramdisk/ (which I will make a modified version of soon) can be used (without some parts) to make a persistent ram disk

    [glow=red,2,300]***MAKE SURE YOU PUT THE BACKUP ON ONE OF YOUR ARRAY DRIVES !!!NOT!!! ANYWHERE ELSE WHICH INCLUDES THE VAR FOLDER MENTIONED IN THE TUTORIAL***[/glow]

    I do NOT know if the mounting/creation config file will persist over reboots but the backup folder will stay if you put it on the array

    Symlinking Non-Array Drive into share folder

    in General Support

    Posted

    Since you'd need to use the Unassigned Devices plugin to mount your "external drive" why not let it export it as a network share too? You can certainly symlink across file systems. It won't have any effect on parity.

     

    ln -s /mnt/disks/external_disk/shared_folder /mnt/user/my_share/symlink_name

     

    would create an absolute symlink.

     

    ln -s ../../disks/external_disk/shared_folder /mnt/user/my_share/symlink_name

     

    would create a relative symlink.

     

     

    thanks, just wanted to be sure, i'm already mounting with the Unassigned Devices plugin, the share drive feature works so thanks for the tip :)