[Plugins] iSCSI GUI and ISCSI Target


SimonF

Recommended Posts

I just wanted to say thank you for the plugin. For the first time yesterday I was able to boot from efi using ipxe and then network boot with iscsi as the disk. The plugin is going to make experimenting with different things easy a breeze and always have a bootable os on my home network. 

 

My next step is to see if I can mount a compressed qcow2 image and network boot. 

  • Like 3
Link to comment
  • 2 weeks later...

Hi there. So first of all thanks for the awesome job on the plugin. Much appreciated!

I've just succesfully set up an iscsi target on my unraid server, but i'm left with a few questions.
First some background info: I'm running an ssd cache pool (4 drives) in unraid that i want to use as the iscsi target. This way i can get striping and mirroring out of btrfs, and use a FileIO image to connect to it over iscsi.
I understand connecting directly to a device introduces less overhead, but that way i would have to set up an individual fileio for each drive (right?).

Now i'm just wondering, if i end up growing the cache pool, can i also increase the size of the fileIO image? And possibly change other properties such as write-back true/false?
Thanks in advance for the feedback!

Link to comment
27 minutes ago, 0wn996 said:

Hi there. So first of all thanks for the awesome job on the plugin. Much appreciated!

I've just succesfully set up an iscsi target on my unraid server, but i'm left with a few questions.
First some background info: I'm running an ssd cache pool (4 drives) in unraid that i want to use as the iscsi target. This way i can get striping and mirroring out of btrfs, and use a FileIO image to connect to it over iscsi.
I understand connecting directly to a device introduces less overhead, but that way i would have to set up an individual fileio for each drive (right?).

Now i'm just wondering, if i end up growing the cache pool, can i also increase the size of the fileIO image? And possibly change other properties such as write-back true/false?
Thanks in advance for the feedback!

I think it depends on the usage, if you are gonna do a lot of writing, then you can mount the drive, however you can use a fileIO image on the btrfs-RAID setup. if you are gonna be using 1gigabit connection, I would recommend sticking with FileIO, 2.5gbs and above use the disk, which I think cannot be used in a raid-config.

Hope this helps!

Link to comment
1 minute ago, maxstevens2 said:

I think it depends on the usage, if you are gonna do a lot of writing, then you can mount the drive, however you can use a fileIO image on the btrfs-RAID setup. if you are gonna be using 1gigabit connection, I would recommend sticking with FileIO, 2.5gbs and above use the disk, which I think cannot be used in a raid-config.

Hope this helps!

Thanks for the swift reply, but I don't quite get that, since the speed of multiple striped drives far exceeds that of a single drive.

So my conclusion would be the other way around. On 1gbit (or 2.5 even) a single ssd will easily saturate the line. Running 10gbit, you should see an advantage from the added speed of the striped array.
My situation is currently 1gbit, but i'll soon upgrade to 10gbit.

 

any idea btw if and how the fileio image can be modified after initial creation?

Link to comment
43 minutes ago, 0wn996 said:

I understand connecting directly to a device introduces less overhead, but that way i would have to set up an individual fileio for each drive (right?).

If you what to use the device it cannot be part of a pool. Also you will loose the redundancy. Each device wouldbe passed as a block device to the initiator. You would have to put the redunacy on the client assuming all devices goto one client.

Link to comment
6 hours ago, 0wn996 said:

but I don't quite get that, since the speed of multiple striped drives far exceeds that of a single drive.

I think what he means is that even of you have a striped pool with 30Gbit/s+ you can't get those speeds on a FileIO image because of the overhead.

 

Theoretically what you've wrote is true but practical a FileIO image is usually always slower when you reach certain speeds.

 

ou can test this right now, simply create a small FileIO image with the write cache disabled and you will see that on certain hardware configurations the speed sometimes even don't reaches 1Gbit/s

 

So a single SSD or NVME over iSCSI will be always faster than a FileIO image.

 

Another idea would be that you've build up a software RAID for example in Windows/Linux with the drives from the Target, never tried this but I think this could be possoble.

 

Hope that clears things up.

  • Like 1
Link to comment
On 12/1/2021 at 1:42 PM, ich777 said:

I think you are using a FileIO volume or am I wrong?

This is pretty normal for a FileIO volume when WriteBack is enabled (but please don't disable it because this will give you much worse performance).

 

Usually I always recommend to use a whole block device (hard disk) because such errors are very uncommon to appear on real disks. But I also understand that using a whole disk is not always possible.

 

No, it's not bad usually this is mainly caused because the write buffer runns full and the disk can't keep up writing the data to the disk, also keep in mind that using a image always causes some overhead.


I am planning to do this today, due to my 4TB WD Red Pro performing very good in reads, but writes seems to cut off goard some programs in Windows. Do I need to disconnect the disk from Unraid (via tools -> New config)?

Link to comment
16 hours ago, ich777 said:

I think what he means is that even of you have a striped pool with 30Gbit/s+ you can't get those speeds on a FileIO image because of the overhead.

 

Theoretically what you've wrote is true but practical a FileIO image is usually always slower when you reach certain speeds.

 

ou can test this right now, simply create a small FileIO image with the write cache disabled and you will see that on certain hardware configurations the speed sometimes even don't reaches 1Gbit/s

 

So a single SSD or NVME over iSCSI will be always faster than a FileIO image.

 

Another idea would be that you've build up a software RAID for example in Windows/Linux with the drives from the Target, never tried this but I think this could be possoble.

 

Hope that clears things up.

Loud and clear. Looks like i'll just be passing on block devices in that case and stay away from fileIO images. I don't feel very confident about software-raiding multiple iscsi devices on windows so i'll just stick with individual drives.
Anyways, when speed is really of the essence i'll stick to local nvme storage on the gaming machine. The iSCSI extra storage is so i can keep more of my steam library permanently installed.
And iscsi is certainly a lifesaver here because both Origin and nvidia gamestream don't play nice with games running from regular network shares. With iscsi, not a problem.

Now it's time to go break my btrfs pool and pass on some block storage :) Cheers!

  • Like 1
  • Haha 1
Link to comment
15 hours ago, maxstevens2 said:

Do I need to disconnect the disk from Unraid (via tools -> New config)?

Depends where the disk is located.

If you want to use it as block device with iSCSI you have to unassign it if you want a FileIO image you don't have to do that.

 

Hope that amswers your question.

  • Like 1
Link to comment
On 1/12/2022 at 6:24 AM, ich777 said:

Depends where the disk is located.

If you want to use it as block device with iSCSI you have to unassign it if you want a FileIO image you don't have to do that.

 

Hope that amswers your question.

Unassign, so that will mean creating new config in my case.

Because I am trying to switch over to mounting a complete disk. Thanks for the help @ich777!afbeelding.png.40c65c46acad2e6fc085438b9c7b418a.png
 

Link to comment

So I tried to resize a volume, but it still reports at 2TB disk, any idea how to decrease the size? Size 'before' was 2TB
afbeelding.thumb.png.b6b8aeb22680ded827f44a2f9c6e0b59.png

 afbeelding.thumb.png.00051eb90a140b6c87fd7cafb15f05a7.png

But it still reports it as 2TB drive:

afbeelding.png.d8e3ad4ca657d1c4835a4f67c3b823da.png

afbeelding.thumb.png.d1ec59f8e3d3d76581a5a53e4363518a.png

 

Also; I am not decreasing the size for further usage, I am only decreasing the size to then move the files over to the same disk, but then added as a block. So basically from disk 1 -> Cache -> disk 1 as a block again (manually transfering data)

 

Edited by maxstevens2
Link to comment
14 minutes ago, maxstevens2 said:

So I tried to resize a volume

I think that is not possible for a iSCSI volume, at least not easily.

One thing that can help is that you remove the FileIO volume from iSCSI and assign it new, but it can also happen that the existing one is overwritten, really not to sure if it will work anyways.

Link to comment
1 minute ago, ich777 said:

I think that is not possible for a iSCSI volume, at least not easily.

One thing that can help is that you remove the FileIO volume from iSCSI and assign it new, but it can also happen that the existing one is overwritten, really not to sure if it will work anyways.

I will try re-adding it :) Otherwise It doesn't matter if I break it, already made a copy of the data so will see what will happen lol.

Link to comment
8 minutes ago, ich777 said:

I think that is not possible for a iSCSI volume, at least not easily.

One thing that can help is that you remove the FileIO volume from iSCSI and assign it new, but it can also happen that the existing one is overwritten, really not to sure if it will work anyways.

 

6 minutes ago, maxstevens2 said:

I will try re-adding it :) Otherwise It doesn't matter if I break it, already made a copy of the data so will see what will happen lol.


It actually worked flawlessly!

afbeelding.thumb.png.0bf4efd9b0a8e7fc0914a73e07182d98.png
afbeelding.png.56c96479890b61dfd389b9a81ca7b325.png

 

The image is now resized, but it also found out the actual image size itself! I typed in 450GB but it probably saw the actual size was 483, so it changed. No data overwritten and all good!

The function is based on this:


Do note to disconnect the .img first (remove it from the initiators list) (after shrinking the volume in Windows!!!!!!!!!!!!!!!!!!!


Hope this also helps others in the future.

Edited by maxstevens2
  • Like 2
Link to comment
  • 3 weeks later...

Hey guys, I've been using Unraid for a couple months and set up a few fileio luns but have a question...

 

I notice fileio seems to thin provision the img.  Unraid does not mark the space as used until the fileio is actually filling up.

If anyone is using fileio are you putting them on a separate drive or is everyone using block?

 

Link to comment
19 hours ago, me114 said:

Hey guys, I've been using Unraid for a couple months and set up a few fileio luns but have a question...

 

I notice fileio seems to thin provision the img.  Unraid does not mark the space as used until the fileio is actually filling up.

If anyone is using fileio are you putting them on a separate drive or is everyone using block?

 


I would say to use FileIO when using a disk that contains data, and you don't have any other disk that you can spare for the function of ISCSI.

I would go for block when you are able to have a complete drive available for this function.

My case:
I have both, 1 4TB HDD completely available to Windows, and a 300GB image on my SSD for game storage (for VM and my gaming rig).

 

  • Like 1
Link to comment
22 hours ago, me114 said:

I notice fileio seems to thin provision the img.  Unraid does not mark the space as used until the fileio is actually filling up.

This should not happen, on what disks (format) do you have the images?

 

22 hours ago, me114 said:

or is everyone using block?

I think most people use Block devices because it's way more performant (with way less overhead then the FileIO) and is easier to set up.

Even if you take a disk from your Windows PC (NTFS formatted and of course not the system disk), put it in your server and add it as a Target in the iSCSI plugin, you can connect it to the same PC over iSCSI without any modifications and it will work as if it's in the PC (depending on your network speed). :)

Link to comment
2 hours ago, ich777 said:

This should not happen, on what disks (format) do you have the images?

 

I think most people use Block devices because it's way more performant (with way less overhead then the FileIO) and is easier to set up.

Even if you take a disk from your Windows PC (NTFS formatted and of course not the system disk), put it in your server and add it as a Target in the iSCSI plugin, you can connect it to the same PC over iSCSI without any modifications and it will work as if it's in the PC (depending on your network speed). :)

Ya, it doesn't seem right.  all fileio img's are formatted NTFS.  On a 3TB (disk3) unraid has a 1TB img, 1.9TB img, 537GB img and other shares.  Disk3 says it still has 1.83TB free, it doesn't add up.  When browsing disk3 all img file sizes are correct.

 

I didn't know you could basically mount an existing drive as iscsi, thats cool. 

I have a 4TB drive I could set as block, but it wouldn't be parity protected.  If I did that I might as well just mount it in my sons PC.

Just looking for any reason to use unraid. :)

 

Link to comment
7 hours ago, me114 said:

On a 3TB (disk3) unraid has a 1TB img, 1.9TB img, 537GB img and other shares.

To be honest I've never tried it with such big FileIO images, always with something around 50GB.

Since iSCSI, should at least, create real image files and reserve the space that is not used.

 

Maybe it's dependent on the file system, do you have your images on a BTRFS or XFS filesystem, but I think it's XFS when you are talking about 'disk3'.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.