mrpops2ko

Thank you, i don't quite understand how to perfect the settings to accomplish this. These are mine currently but forgive my lack of process understanding, wouldn't this mean that once my cache hits 50% utilisation it would immediately transfer ALL files that exceed 15 days and not cease transferring them until it satisfies the <14 days requirement? because that'll be a show stopper if so. Or does mover just instantly stop the moment I reach below 50%? because I tested this and had some strange observations, I hit the 50% and then now my cache is at 44%, so some files got transferred. I'm assuming it moves files down to the nearest neighbour % interval?

i'm assuming this isn't possible, since the options don't seem to exist but is it possible to configure mover to work based upon atime and set it to move around 10%? so anytime I hit (my cache is an 8tb nvme cache) 7tb used, it'll clear out 1tb of the least accessed content, onto spinning rust can i accomplish this and if so, how?

theres arguments that macvlan is more efficient than ipvlan in terms of cpu utilisation (at least from me googling the subject when I was deciding) but i think its in the realms of 'i can save 0.1% on my grocery bill by running the london marathon' meaning you shouldn't give a shit about it. macvlan comes with the requirements of promiscuous mode a lot of the times (which can increase latency from reading into it) and also to allow forged mac transmits. so pick your poison really, but macvlan seems to me to be the worse of the options compared to ipvlan

you dont need plop anymore, you can boot directly from USB attached directly to esxi. Plop is generally used for devices which has some issues booting directly from USB but those are mostly a thing of the past now it seems. just remember to change it from BIOS to EFI. then just pick it out from your menu

idk the truth to this but on the face of it, it doesn't seem like a massive problem to solve. we already have HW USB GUID based validation for licencing we already have infrastructure set in place to backup / migrate those USB sticks its not a huge stretch to say have an option where you flag a specific nvme ssd or regular ssd as a 'docker / vm' device and that thing is allowed to boot without the array main splash thing operating. it could purely be limited to people who are valid licence key holders and subject to the same 'once yearly' migration plans etc none of that seems too farfetched to me, seems easily workable if enough desire to implement is had. if i was to pull a statistic out my ass, i'd guess around 75%~ of unraid users have some form of ssd (whether nvme or not) and the way the industry / market is going, you can have them for super duper cheap now. cost is barely becoming a barrier anymore. i've seen $20 deals on 250gb nvme ssd's or $25 deals on 500gb ssd's. no decision will ever have the entire userbase support but i've seen patches / bugfixes / development time spent on niche user experience stuff (like drivers for niche hardware) so if the vast majority of users could have this implemented in the way i described, i dont think it'll cause the entire house of cards to come tumbling down. you might get a few butthurt users who dont have ssd's or dont want to have say bi-weekly online auth or something but they'll soon come round when they need the functionality.

i run unraid as a vm / usb passthrough on esxi 8. whilst i support the idea of this feature i dont think its a huge leap to suggest people just use something else. everybody can run esxi for free near enough, the limitations aren't huge - its performance is very similar to baremetal and you get a degree of abstraction so you can further test whether its an unraid based fault affecting you or not. (or you can run proxmox for free too)

because its ZFS which means I have to incur stripping. Most users who are attracted to unraid or similar solutions do so because of the ease of expansion and JBOD nature of things. When you get data at scale (and my scale isn't even that large, only some 80tb or so) the idea of losing it ALL in one singular failure is scary as hell. I love that its all JBOD and independent. Prior to using unraid, I had been using snapraid for about 7 years. I'm still mulling over the idea of making a guide on how to set up snapraid on unraid because its a significantly better solution all round for integrity (it checksums all the files and keeps records of them, and its very easy to see if theres a parity sync mismatch which files are mismatched so you can independently verify if the parity or the filesystem is correct) - its also very easy to plug and play it. The only major downside to it, is that its not a live parity (its snapshot based so you'd generally execute a script which will tell the array to sync and you can't be writing data to it during that time). re: writeback mode I agree with you, it should be disabled and native unraid solution could lock that flag in place. (as well as potentially exposing a bunch of these options like sequential_cutoff as a GUI field / drop down menu Due to the nature of unraid's cache system, writeback mode is largely redundant isn't it? I also agree it makes no sense to bcache a cache pool outside of some super duper niche and probably datacentre level considerations - nothing for us home users

what would you suggest is the best method to accomplish this on unraid as a transparent read cache for the array? pickings seem pretty slim and bcache seems the best solution for the job all things considered as it stands it is as you have said, but i dont think it'd be reinventing the wheel to have a native implementation of bcache into the unraid ecosystem for the array (and maybe even cache devices too but that seems a bit redundant) the offset functionality is how we maintain each individual array disks accessibility, as independent filesystems - is it not? the parity could have an offset too - hell if we moved this from an 8kb to 1mb offset instead... (so we can better support that alignment fix) is it really the end of the world if the first 1mb of our array just isn't backed up by parity? we could have each drive just start writing from 1mb in and in principle its all interchangeable plug and play?

great to hear but can you expand more on this? from what you've said it wouldn't be capable of caching data accessed on the array? if it isn't then can you elaborate on what use case scenarios we could do with this? I remember reading about using the bcache offset to simply mount with or without the bcache wouldn't that be something unraid could do for the array? like on a parity level unraid could just ignore the first 8kb of each disk (if the cache is enabled, this would be the bcache hook in offset) and then we'd all have a transparent, interchangeable, ephemeral cache I recall reading @limetech played about with it in 2015, if we could get the 8kb offset hook in and the parity disk ignore the first 8kb (or assume its values since it'll never really matter) then that should be sufficient for array integration wouldn't it?

disagree with the method of communication but this seems mostly trivial for limetech to implement in practice the real point of origin where a warning would be or should be issued, is on the clicking of the update button (as a secondary 'are you sure') style window it doesn't strike me beyond the realms of computational possibility to parse a list of plugins already installed and spit out the ones which will be invalid its at that point, the user can say 'nah i cant be bothered updating yet' because if its invalidating more plugins than the effort you plan to go to, and you are relatively happy as is, why bother? the problem ultimately was that people felt blindsided - and overall should be a simple problem to fix

sorry if i came off as combative, it wasn't my intent but its been kinda my observation that sometimes discussions are wholly shut down because its not within the extremely narrow scope of an intended persons use case scenario (see for example the whole political ideological warfare landscape that is the linux kernel peeps dislike for FUSE file systems lol - the windows equivalent, using drivepool is significantly better on a lot of fronts but it just cant do hardlinks) it happens to all of us, i get it, some of us couldn't possibly imagine a scenario where its useful and on face value it sounds absolutely daft (not saying you did any of this, just talking about observations, i've also seen a lot of plonkers advocating for stupid stuff too and when drilled down it was stupid overall) the conversation then devolves into an exercise where (rightly or wrongly) you end up having to extremely justify the minutiae of your deployment - writing literal novels in the process... (again im not saying you did any of this, i'm speaking in generalities of online discussions) also yep thats exactly as you described and its the absolute exact desired state in a tiered storage scenario 'basically cache every file' - thats the absolute bottom dollar god tier 10/10, dialled to 11 position that me and probably a bunch more people want to end up in yep it will significantly reduce your SSD lifespan, its going to do some insane level of write amplification - thats also entirely desired SSDs themselves have insane levels of longevity - if we are talking meta commentary on SSD lifespan we are seeing many SSDs outliving their usefulness because of sheer capacity expansion - SSDs basically just dont die now. at least the good ones. They are not only rated for significant amounts of writes, but those stated values are not reflective of REAL WORLD values for example the samsung 840 pro (dont buy this btw for anybody reading, it has a hardware level defective which samsung 'fixed' in firmware... the 'fix' caused massive write amplfication), its got a rated TBW of some 100-200 TB? (i'm working from memory) but in torture tests where people actually seek to KILL the SSD, its survived for many PETABYTES of data. More than a 10-20x increased vs rated on the box. https://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead/ So yeah I think overall GOOD justification can be made for this kind of stuff. The used SSD market on both NVME and regular SSDs are a gold mine imo. The whole idea with the b-cache is that its interchangeable, ephemeral and you shouldn't give a shit about it - i'd adopt a pump and dump mentality with it I envisage for unraid with a plugin / script, once sufficiently advanced it could be as simple as pointing towards an ssd and clicking go. from a value proposition I think this makes TONS of sense. Imagine someone is coming to you and says 'hey {user} for £16 the next 2 PETABYTES of reads are going to enter into a 128gb cache, which is going to be fast as lightning. Would you be interested in such a deal?' I'd bite their hand off and thank them profusely for giving me such an opportunity. Probably ask if I could rake their yard in gratitude.

oh also re: sequential it appears you can turn it off echo 0 > /sys/block/bcache0/bcache/sequential_cutoff

it doesn't make sense to you to speed up the array storage? at this point i'd just be writing again what i've previously written. do you genuinely not see the point? imagine for example you have small amounts of ram spare and listen exclusively to flac songs and have a random playlist. (i'm making this example up so you get the point) each song you listen to is 30 mb, and you have enough ram spare for 10 songs cached. with an ssd read cache as described above, subsequent random replays (if landing on those songs already listened to) would then be serviced by the ssd rather than the array, freeing up that specific array disk for other tasks and potentially reducing i/o wait times (if in this scenario the disk is being thrashed for some reason, copying a large file for example) like you mentioned, many large and cheap flash storage options are available but spinning rust will always be cheaper - having say a 20tb array, with 8gb of ram and a 2tb b-cache that read caches any accessed data from the 20tb sounds like a desirable thing to me. the major reason for, and major reason why the current system is not good, is that its impossible to predict what of that 20tb array is going to be accessed beforehand. again im not sure why im making up stupid justifications for this, but this example outlined doesn't sound far-fetched to me. you are sat at the dinner table with your family and let them know of a movie you recently watched on your family plex server. the family are rapturous as you regale them on what a great movie it is, after eating they scurry to their bedrooms in order to watch it. In the current system (assuming the example above with highly limited ram, its 80% utilised on docker containers, so a mere 1.6gb free) then this poor array hard drive would be thrashed to shit because its now serving 3 or more people the same file. in a b-cache enabled environment, then no problem that ssd will chew through their requests like it was nothing (leaving your array sound and ready to serve up whatever client wishes to access data from it)

an example of a similar windows based tiered storage approach would be primo cache

to create a read cache the idea being that anything read by the array goes into tiered storage so it first gets read and gets put into ram once the ram is exhausted (and upon coming close to leaving ram entirely) it then gets moved to an ssd and eventually it will exit the ssd too the point being its on the block level and you can reduce array utilisation for subsequent reads within the time span of it being outside of ram but inside the SSD containment amount

im guessing this was never looked at or implemented?

they do if you passthrough the entire controller (so esxi cant see it anymore) its in the manage > hardware > pcie devices then its just a case of giving them to the vm

yep its a shame that this bug is completely ignored. not sure why it is. i made a bug post here but it seems its completely ignored. Unraid's implementation of SMB is completely non-functional. its not even worth considering using it, its absolute garbage. there are some other options you can use, like SFTP, or NFS. those are slower (sometimes significantly) but are passable. NFS is probably the best in terms of general directory browsing, it only takes 15 seconds or so to list my 75k file directory using it. for reference, i can instantly (1-2 seconds) view a directory with 500k files in it, using windows server 2019 SMB. heres some random benchmarks i did. =================================== 13,238 Files, 116 Folders / 1.71 GB (1,845,219,328 bytes) transfer FROM pc TO server windows server 2019 1 minute 45 seconds delete @ 700~ files per second windows server 2019 drivepool, many drive pooling solution (FUSE equiv?) 1 minute 56 seconds delete @ 572~ files per second transfer TO pc FROM server windows server 2019 1 minute 22 seconds windows server 2019 drivepool, many drive pooling solution (FUSE equiv?) 1 minute 35 seconds =================================== NFS unraid benchmark 13,238 Files, 116 Folders / 1.71 GB (1,845,219,328 bytes) transfer FROM pc TO unraid server 28 minutes 51 seconds (it started off really fast and progressively got slower, i bet if you did it in batches of 1k it'd remain fast throughout) delete @ 52~ files per second (increases as time goes on to 130~ files per second) transfer FROM unraid server TO pc 2 minutes 8 seconds =================================== SSHFS / SFTP 13,238 Files, 116 Folders / 1.71 GB (1,845,219,328 bytes) using /mnt/cache_nvme (this should bypass SHFS according to some posts and give disk level speed) transfer FROM pc TO unraid server 3 minutes 37 seconds transfer FROM unraid server TO pc 1 minute 45 seconds ===================================

hi jbrodriguez I made this suggestion and i was wondering if there was scope to incorporate this into your plugin instead of natively (although native would be best). How I imagine it would exist within unbalanced, would be via some kind of granular file slider. locating files above or below a certain filesize

Currently the methods that govern how file placement / movement happens in relation to the cache are; I would like to request 2 additions > size - Files greater than the indicated size are placed on the specified cache pool < size - Files lower than the indicated size are placed on the specified cache pool use case scenario: like most of of you, having a variety of files of different sizes, it makes a lot of sense to leverage the super fast access times and availability of flash memory. Keeping all files under 2mb or less on exclusively flash memory will serve to increase our arrays significantly. As the cache is outside the array, duplication could be involved too alongside hardlinking but thats additional complexity and I would be happy if just the basic implementation could be looked at. i'm reasonably certain it isn't possible to do this currently, is that the case?

what I found kind of interesting was that if you access via SFTP its super fast. This video is showing cached results (i think) as the first time round it didn't perform as fast, but it wasn't far off it. all of this was done with my drives spun down too. I don't know what the fuse filesystem they use governs and what it doesnt, but we can clearly see from that SFTP usage that it doesn't control everything poorly...I don't like to go down without a fight, so maybe if we can come up with a solution which can circumvent its utilisation so i recorded more footage to get a feel for what is happening, or at least some stuff we can see I thought maybe SFTP would just be fast whilst using the disks, but no its clearly just as fast when being utilised from mnt/user or mnt/user0 (i believe this is the same as being from the shares) utilisation spikes quite a bit for what it is, considering throughout the entirety of this my drives were and remained spun down. this is basically just reading from ram (or should be) I looked what happened in netdata during that time, since its basically just this its pretty obvious and then I did the same again whilst running shares smb so yeah im not really sure where to go from here.... ok returned to this idea after an hour of brain storming, we know SFTP is gr8 right so lets fk it we do it live with SFTP? https://github.com/winfsp/winfsp https://github.com/winfsp/sshfs-win when i mounted it as a network drive it didn't work at first, but then reading the documentations it mentions you can change the home location by adding .r to it, which let me into averaging roughly 100 files per second or more, NICE no shitty overhead either but im sure we'll run into a bunch of edge cases, i mean its not like this will just work right? well i'll be damned... so what about networking streaming of audio / video? surely that wont be the same? looks great! oh lets return to those small files, uploading them was fine but maybe it'll be super slow when downloading? nope, great @ 100+ files per second, only bad thing was that during the 'discovery' period of transfer, the video i had playing in the background (located on disk1 whereas the super small file tests are located on cache SSD) and that started to stutter, only during the discovery period though. whilst it was transferring the video playback resumed to normal speed. im pretty pleased at how this all panned out, i think this solution is a viable one for windows users - i can go somewhere from here at least, i'll do more testing later on though limetech i wouldn't say no to a free key for all this testing and solution based strategy

haha im in your exact situation (but i migrated away from a windows server 2019 install) i knew going in that SMB would be bad, but i wasn't prepared for just how bad it is I pretty much have the same conclusion as you do (fortunately i haven't paid for a licence yet). i'm now trying desperately to think of how i can keep unraid in the mix but nothing springs to mind its clear the smb shares aren't fit for purpose - so with developer reluctance in fixing this bug, what could we do? part of what would be needed would be something that takes the disk views and merges them into a single / unified directory, i wonder if something like that could be accomplished by some plugin i also tried that folder caching script, it worked (in that i could spin down a disk and still read the directories) but was plagued with the exact same issues.. oh i never tried disk view with it, let me do that now. nope still all the same

hi i am in the process of migrating to (or maybe away from xD) unraid because of this im noticing the same problems you are, but i don't really get it... like im assuming a lot of people here use SMB right? it isn't some unknown, niche and new technology so how has throughout the entirety of unraids existence, no developer or limetech in general cared to fix this? like im of the mind that i MUST be doing something wrong because of how poor the performance is we can watch it in real time, the 100 - 1000x slower performance of unraid (left) vs windows server (right) and then we can see this issue not exist in disk shares, unless of course you dare use the unraid user shares at the same time its so bad that it even 'infects' the fast disk1 view, and once the shares SMB version are stopped it instantly jumps to being performant and in line with every other major implementation used by other providers... im hoping someone will reply with 'damn pops you are just dumb, you click these 5 buttons and magically the performance you expect is granted' but im not seeing it... are you also noticing similar things to me? is everybody experiencing this? how the hell are you all happy with this? o__O

ah very disappointing to learn its not possible to do this natively, i also would have loved to have my own external router handle all the DHCP requests and not do a half-way house of multiple non-aware DHCP servers. I'm still very likely to purchase at the end of the month but this would definitely help towards tipping my over the edge