Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Tiered storage and specialized write cache pool

Featured Replies

Browsing this forum I've seen that a tiered storage (pool 1 -> pool 2 -> ... -> array) with all the relevant mover functions (promote/demote files based on usage etc...) has already been requested multiple times so please consider this part as a "+1" by me.

Other than that the feature that I would love to see implemented is the ability to create a pool solely assigned to boosting write speed while ingesting new data on the server. The way I envision this "Write cache pool" is that, once we assign one or more drives to this pool, it would only be used to receive incoming data without any form of retention so as soon as a file transfer is completed (or the write cache is full) the mover would immediately empty the write cache to the relevant actual storage of pools + array.

Given that this pool shouldn't ever hold any data on it apart for the time needed to receive it and to move it away it should ideally be set up as striped storage across as many drives we assign to this pool to maximize the throughput during the ingest operations to fully leverage 10g+ networks.

To summarize the order of operations would be:

network interface --------> write cache pool (striped and emptied ASAP) --------> pool + array storage (current implementation)

  • 3 weeks later...

zfs specal vdev disks.

unless your refering to unraid main array partiy system...

disk speed to increas this i've seen is a zfs feautre only by asigning fast storage like a ssd / nvme to teh zpool to assit metadata read writes to the pool.

  • Author

No, this pool that I'm suggesting wouldn't be directly tied to any other pool or the main array but the way that it should work would be to act as temporary storage for any file inbound to the unraid server regardless of the destination (array/other pools) and as soon as a file gets fully copied on it the mover would immediately transfer it to it's destination (array/pool). Doing this way and using fast nvme drives you could basically throw at your server "any" amount of data (given you use big enough drives) at the maximum speed the ethernet connection would allow without the need to have a huge ssd pool tied to the array. The special pool that I'm suggesting wouldn't be mutaually exclusive with the current implementation of pools and array but would be active system wise something like this:

Interface (eth0).png

34 minutes ago, TheCheater said:

No, this pool that I'm suggesting wouldn't be directly tied to any other pool or the main array but the way that it should work would be to act as temporary storage for any file inbound to the unraid server regardless of the destination (array/other pools) and as soon as a file gets fully copied on it the mover would immediately transfer it to it's destination (array/pool). Doing this way and using fast nvme drives you could basically throw at your server "any" amount of data (given you use big enough drives) at the maximum speed the ethernet connection would allow without the need to have a huge ssd pool tied to the array. The special pool that I'm suggesting wouldn't be mutaually exclusive with the current implementation of pools and array but would be active system wise something like this:

Interface (eth0).png

i bleieve that is what the original setup was with disk 1 disk party and pool disk cache. Were the cache pool was the temp fast data to then dump to the disk 1 array.

re review the docs for a simlar setup:
https://docs.unraid.net/unraid-os/using-unraid-to/manage-storage/array/overview/

No image preview

Cache pools | Unraid Docs

In Unraid, a %%cache pool|cache-pool%% is a collection of one or more drives, typically SSDs or high-speed HDDs. These drives temporarily store data before it's moved to your main %%array|array%%. Usi


as some have used zdisk, some have used ram disk for temp storage. This is a hard concept to explain and go over. But limitation of disk would get in the way and on power outage becomes the problem of data corruption. what disk has what data where...

to me it sounds like you want a true temp cache disk for all read and writes of data on unraid before it moves data for the primary stroage in the array...

this is kinda curently done with the share settings:
image.png

image.png


If i'm understanding the feature request. so in the purposed setup / feature you want a disk to act as a mirror to also be one with primary storage and mover sends to the array?
as this could be done with some software raid and overhead...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.