Defragment


14 posts in this topic Last Reply

Recommended Posts

I mostly use my unRAID box for storing blurays, music and tv shows. I recently decided to encode my blurays instead of storing huge full bit rate mkvs. I replaced a ton of them with encodes and then adjusted my encode settings (CRF19) and replaced again. I am still in the process of replacing many. I wasn't worried about fragmenting when i started using unRAID because most of what goes on it was gonna stay there. However, all of this replacing must be fragmenting my file system.

 

I read this post ..

And I am wondering if i should do it. Instead of running a huge two or three day defrag i like the idea of letting it run for an hour or two a day. Should i? And what would be the best way to automate it (ELI5 please lol) if so?

 

Thanks :)

Link to post

I have to agree with the reply by garycase in that thread:

#1. It's not usually something to worry about.

#2. If you are concerned about it anyway, then copy the data off, reformat the drive, then copy the data back.

 

Or an alternative to #2, copy the data to a freshly formatted array drive, then do New Config with Retain All and swap the drive assignments.

Link to post

I concur that you should not worry about this unless you are seeing actual performance issues, which seems incredibly unlikely. 

 

But to do the copy and reformat, you'd just need one new disk, and follow the same basic process of an RFS to XFS conversion.

Link to post

I am not exactly worried about it hehe. But i have been moving a ton of data on and off this month. And will be for the coming months.

 

One new disc that would fit all my stuff is $300 CAD. I don't see the the harm in letting it run for a couple hours a day... maybe even less. I just don't know how hehe

Edited by superderpbro
Link to post
  • 4 years later...

I have seen a command to do this for xfs drives.

My array is BTRFS so that's not going to work for me.

I have various Containers and VM dumping data whenever they feel like it and i feel that it's unlikely/impossible that no fragmentation has happened.

(it's currently writing from mover and resilio-sync on the same drive and this isn't uncommon on this system)

I've been reading for years about how we don't need to worry and at the same time i'll defrag my synology that also has btrfs drives.

 

Performances is......... um well i don't know, probably fine but then 50mb/s doesn't seem fine for a 12tb WD Gold (this could just be the task though)

 

I don't think an annual defrag is a insane idea.

Yes i have considered that the way UNRAID deals with parity makes a defrag tricky but i still feel that it's unlikely to never be required.

 

 

Edited by mdrodge
Link to post

I appreciate your assistance and your help is normally very well received Lord trurl.

I've been using that on and off for a while via the plugin and it's proved useful at times. (It's really helped me to stuff more data in from 3+ sources at the same time) 

I don't see the connection between it and defrag though.

It funny how when ever anyone brings up this topic they get either an unrelated answer or just plan told they are doing it wrong.

I look forward to asking the same question next year.

Link to post

My point wasn't to suggest you use Turbo mode or not. My point was to give you more realistic expectations about write speed on the Unraid parity array whichever mode you use.

Link to post

Oh i see lol. Yes it's been quite quick today anyway. I fairly sure my drive speed is based more on the tasks I'm running and network congestion.

I'm not saying i have a performance issue.

I just assumed that if i tried hard enough i can get fragmentation to happen.

I've read that we don't need it on xfs but no one said that we don't for btrfs.

 

If i don't need it then fine but i haven't heard that anywhere.

Link to post
9 hours ago, mdrodge said:

I've read that we don't need it on xfs but no one said that we don't for btrfs.

 

If i don't need it then fine but i haven't heard that anywhere.

 

You made me curious so I did a quick Google search search for "btrfs defrag".  The first result explains how to defrag btrfs and then has a section talking about "problems with defragmenting a btrfs filesystem".  Yikes. Sounds like it massively increases disk usage because it breaks Copy-on-Write pointers. Doesn't seem worth it: https://linuxhint.com/defragment-btrfs-filesystem/

Link to post

Nice answer!

My search engine didn't throw that up, very interesting.

Though the place i read said btrfs would be more prone to fragmentation and it had tools available.

https://itectec.com/ubuntu/ubuntu-btrfs-defragmentation-of-files-folders/

 

Worth it is subjective. If fragmentation was bad enough that subjective worth would be different.

There are autodefag options so I'm guessing someone has though of the issues.

I'm not saying i need to do a defag I'm saying i don't want to never have the option. 

 

((( "move all your data to a empty drive" comments will be ignored." (Not aimed at anyone here today but I've read that on here a few times and i know someone will say it)))

 

Link to post

I'm not the most clued up about the complexity of a file system but wouldn't the copy on write not be an issue if the data was re-written as a single chunk (defraged) and any future writes would have fresh links made or am i not understanding something?

And why would accessing a single continues file cause higher disk usage that a bunch of chunks?

 

I'm not using snapshots (i don't even know if that's possible with UNRAID)

But i have been known to re-transcode the same file and have multiple disk write operations happen at the same time.

 

Luckily the most demanding task by far for me is replication to the backup server and that's not as speed critical i guess.

 

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.