Defragment a NTFS partition from LINUX

Xi0N · 2011-08-29 18:52:10

I have one partition containing a windows system (for special purposes) and i was wondering if i can defrag the partition from Arch...
Reasons to do so:
1) NTFS is know to need to be defragmented from time to time
2) From Arch, no file will be open on the NTFS partition, allowing to defrag every file on the pertition

WHat do you people recommend?

Inxsible · 2011-08-29 19:01:06

I wouldn't recommend it.

NTFS requires some files to be in certain sectors -- especially if that partition is used to boot windows. Any defragger in Linux (and I don't even know if there are any that can handle NTFS) might not care about said system files, because as far as Linux is concerned - they are just regular files.

Your Windows might be rendered unbootable.

Army · 2011-08-29 19:01:10

Searching the AUR for "defrag" I found e.g. fidefrag, a filesystem independent defragmenter. I don't exactly know how this works, but you could give it a try and tell us about it.

Xi0N · 2011-08-29 19:03:34

I was testing defrag, but i think i will do a dd of the partition before going on... just in case...

Army · 2011-08-29 19:53:03

That's never wrong But Inxsible has a valid point here! Performing a defrag in Windows, you see these "green" files, which mustn't be moved.

MoonSwan · 2011-08-29 21:19:18

I saw an article recently (Phoronix I think?) from some sort of Microsoft developers and, from what they said, I gathered that doing defrags on these multi-gigabyte hard drives is pointless. Your net gain is minimal over all. They also pointed out that in the era of 200 MB hard drives, defragging made sense because you had much less space to work with.

Up to you of course but that's just what I remember from what little I read. Take this post with a boulder-sized bit of salt.

Xi0N · 2011-08-29 21:25:21

Yes, its actually a good point..... thanks for the opinion!!!

SysGhost · 2013-10-07 15:58:34

Time to wake up this thread. Why? Well, because I do have a good reason to why one would need an NTFS defragment tool under Linux.

I have quite a few external hard drives (Larger than 100 Gigabyte) and I am using NTFS on them to keep them compatible with other operating systems and solutions (such as media players).
Some of these drives do a lot of writing, and they can be crammed to the brim with stuff, leaving only a few megabytes of free space to work with. This is an extreme situation, but I do believe there are people out there in a similar situation as I am.

One bad thing that happens to these "filled to the brim" hard drives when they're written to often and much: The files will get VERY fragmented.
In my extreme case there isn't a single file that hasn't been smeared all over the drive. It's bad... really bad.

I just don't want to reboot my computer into Windows just to defragment a few external hard drives. And I don't want to keep a virtual machine with a complete Windows install just for that puny task.

The best solution would be a neat NTFS defragment tool.
Army's suggestion: aur/fidefrag-bzr does not work very well. It looks like it isn't being developed anymore. Got a huge bunch of python-related errors (And my python-install is working properly).
I also had to fetch it manually and make my own PKGBUILD

Are there any other solutions that does not involve rebooting to Windows or running silly virtual machines?

Last edited by SysGhost (2013-10-07 15:59:17)

karol · 2013-10-07 16:23:31

Why not just reboot to Windows? Either defragmentation of these ntfs partitions is important enough to use Windows or you should consider switching to a setup that uses a different fs.

SysGhost · 2013-10-07 17:55:21

karol wrote:

Why not just reboot to Windows? Either defragmentation of these ntfs partitions is important enough to use Windows or you should consider switching to a setup that uses a different fs.

In my case. Windows will eventually be removed. Soon I won't have the option to reboot into windows at all.
I have considered a different file system. But as I mentioned, some of my devices requires a "compatible file system", ant that "compatible" list is rather short: FAT32, exFAT and NTFS.

FAT16/32 is out of the questions for obvious reasons.
exFAT might work, but have the same problem as NTFS: No defragment tool under Linux that I know of. Support for it is also limited since it kinda new. It isn't designed for spacious hard disks either. It is supposed to be used on smaller media such as USB thumb drives and media cards.
NTFS is the most compatible file system. It works well with my media players. It also work under other systems and of course works well with other Windows machines.
*nx based file systems such as ext2/3/4 is also out of the question because most products and media players are not compatible with these. Same goes with other Windows machines.

I must say the situation is kinda silly.
Even when one have left Windows all together, one still need to use parts of Windows on daily basis product to keep them "fully compatible".
If the manufactureres of media players and other similar products just could include support for commonly used *nix file systems. we wouldn't need NTFS/exFAT at all

Last edited by SysGhost (2013-10-07 17:59:39)

fschiff · 2013-10-07 18:28:31

Are there any ntfs system rescue disks with tools? ( look here http://www.nu2.nu/pebuilder/ )
Otherwise Ultradefrag run from linux?

edit: remembered url

Last edited by fschiff (2013-10-07 18:32:00)

SysGhost · 2013-10-07 19:01:24

fschiff wrote:

Are there any ntfs system rescue disks with tools? ( look here http://www.nu2.nu/pebuilder/ )
Otherwise Ultradefrag run from linux?
edit: remembered url

NTFS rescue tools have nothing to do with defragmentation to do as far as I know. Actually when it comes to rescuing files, defragmentation is the last thing one want to do.

Wine won't and can't give Windows programs direct I/O access, which most windows defragmentation tools need.
...and when it comes to a Linux version of Ultra Defrag ... does that exist yet? I've heard rumours about it, but have never seen or confirmed it.

Last edited by SysGhost (2013-10-07 19:01:49)

ANOKNUSA · 2013-10-08 12:34:09

Your insistence that you absolutely, positively must not use anything but NTFS for compatibility reasons seems overblown. Perhaps you ought to share exactly why that's the case---e.g., why the "media player" neets NTFS on an external hard drive, and why, if that's the case, you're willing to get rid of Windows anyway.. Until you do, no one here will believe you, since right now it seems that you're willing to deliberately put yourself in a bad situation by destroying the one thing that can solve your problem.

anadon · 2013-10-08 12:51:51

+1
I have a situation like this for a lab I help manage in my uni.

karol · 2013-10-08 13:09:43

anadon wrote:

+1
I have a situation like this for a lab I help manage in my uni.

What situation? You have to use NTFS but would rather not use Windows?

Mr.Elendig · 2013-10-08 14:01:23

Protip: store everything on ext4 or something and just have a small volume for sharing data. (or use cifs/similar)

WonderWoofy · 2013-10-08 14:31:57

Mr.Elendig wrote:

Protip: store everything on ext4 or something and just have a small volume for sharing data. (or use cifs/similar)

I am a fan of using samba (cifs) to share data from my Linux fileserver to other computers. This greatly simplifies things when you have a household with multiple operating systems.

SysGhost · 2013-10-08 20:43:21

ANOKNUSA wrote:

Your insistence that you absolutely, positively must not use anything but NTFS for compatibility reasons seems overblown. Perhaps you ought to share exactly why that's the case---e.g., why the "media player" neets NTFS on an external hard drive, and why, if that's the case, you're willing to get rid of Windows anyway.. Until you do, no one here will believe you, since right now it seems that you're willing to deliberately put yourself in a bad situation by destroying the one thing that can solve your problem.

I really don't know why a media player cannot use other than Windows-related filesystems.
But if you can show me a nice and feature-rich media player that can read *nix file systems from UMS devices, and able to play a wide range of media containers and video formats, I'm all over it.
The "only" really good alternative I've found so far is home built HTPCs with XBMC distributions. (Which I do prefer to be honest)

But the case here is that a lot of my friends, who have no computer skills whatsoever, buy these pre-manufactured media players and use Windows computers. They won't change their systems just because I have different needs. These are the friends I share my portable hard drives with. Thus I need "maximum" compatibility.
Most of these pre-manufactured media players only support FAT32 and NTFS... and maybe exFAT if it's a "good one".

I get a feeling that you didn't read my first post completely, or didn't understand my needs. Besides. What part of my situation is not believable?

EDIT:
If I haven't been clear enough, Here's the situation:
I have a bunch of portable drives. They're used between many different systems, some of them I cannot control what OS they run. (Friends and family)
In order to keep the portable drives compatible with a wide range of different systems, and yet use a filesystem good enough for huge drives, the only choice is NTFS.
FAT32 won't do as it is a deprecated fs.
exFAT might just work, but have the same problems as NTFS do: utilities under Linux is lacking. Besidens not a good fs fora terabyte-sized drive.
*nix filesystems only work with *nix systems. That will keep a majority of my friends and family out, as they cannot read these filesystems without ugly hacks and/or poor third party softwares.
NTFS is the only choice left.

If anyone have suggestion on a FS that can be maintained under Linux, work with a wide range of systems including Mac OS X and Windows and work well on multi terabyte -sized block devices... I'm all over it.

Last edited by SysGhost (2013-10-08 21:03:00)

SysGhost · 2013-10-08 21:28:58

Mr.Elendig wrote:

Protip: store everything on ext4 or something and just have a small volume for sharing data. (or use cifs/similar)

That is a good suggestion. But won't work too well in my case.

To be more exact:
A few of my portable drives is a so called "Project disk", in which I and a few friends store quite sizeable projects of various kinds. (audio and video editing amongst other things)
These portable disks wanders between us, and the project folders are slowly growing in size.

One of the drives is now filled up so well, that the NTFS file system cannot prevent fragmentation any longer.
When I analysed the drive, it was in a VERY bad state. The fragmentation is extreme. That does explain the poor performance.

My friends sure could but the disk under their system for a defragmentation, and might be the only solution for now. But It has to happen too. These friends are a bit lazy when it comes to maintenance. They always leave that to me.
I'd love a NTFS defragmentation utility under Linux so I can take care of this myself.

Keeping a Windows installation only for the rare occasion of a defragmentation seems a bit... overzealous in lack of other words.

Last edited by SysGhost (2013-10-08 21:29:41)

clfarron4 · 2013-10-08 21:41:36

WonderWoofy wrote:

I am a fan of using samba (cifs) to share data from my Linux fileserver to other computers. This greatly simplifies things when you have a household with multiple operating systems.

I'm going to second @WonderWoofy's suggestion of using Samba shares here. I know my University uses massive Linux fileservers to as the in-house filestorage solution that can be accessed anywhere on campus with a campus PC or personal machine, be it Windows, Mac or Linux.

So, converting everything across to ext4 or something like it and setting up a Linux SAMBA server is definitely possible (why I am I thinking of a Raspberry Pi right now...).

nomorewindows · 2013-10-08 21:59:06

Are we talking portable devices that have to have a specific filesystem here or can these devices access what's over the network? If it's the latter pick a Linux fs and a filesharing mechanism that they will all take and to heck with the defrag. There's all kinds of solutions here. Run your windows install under a vm and defrag it from there.

life_enjoyer · 2013-10-08 22:18:30

@SysGhost Format your portable drives to ext4. Make a live linux usb stick (or just use your arch install cd/usb stick). Then, boot your family/friend's computer with the live usb stick. Last, transfer your files from your portable hard drives to their computer from within the live linux or arch install environment.

Last edited by life_enjoyer (2013-10-08 22:29:25)

Gusar · 2013-10-08 22:31:18

Paragon provides a full read/write ext4 driver for Windows. It's free, though it requires registration: http://www.paragon-software.com/home/extfs-windows/

If you don't like that, use ext2fsd. It's open source, though the ext4 support isn't complete, so you'd need to use ext3. Which is a bit of a shame, because the ext4 extents feature is very valuable for large files.

Either way, much more convenient than a LiveUSB.

cookies · 2013-10-08 23:13:55

If there's absolutely no way around ntfs you could simply get an additional empty drive. Then whenever one of your ntfs drives get overly defragmented you simply copy over the files to the empty drive and just reformat the defragmented drive afterwards. Now your data is defragmented and you have a "new" empty drive for the next time (you can use an internal drive for temporary storage, too, if you like, but then you'd have to copy the data back to the external one after formatting it).

I think it's a rather stupid solution that's far from elegant, but it should work

SysGhost · 2013-10-10 08:15:22

I've found a solution.
It seems there is a beta-version of Ultradefrag for Linux. It's utilizes NTFS-3g, so there are some risks to this. Never use this tool without having a backup.
It can be downloaded from: http://b.andre.pagesperso-orange.fr/adv … fs-3g.html
It's named "ultradefrag-5.0.0AB.7.zip"
Once unpacked, one should read "README.linux" before doing anything else. There is some important information one need to know BEFORE compiling.

The compilation might still fail, despite all corrections according to the README.linux, and I have found the culprit.
I have written a patch file to remedy one problem: https://dl.dropboxusercontent.com/u/147 … AB.7.patch

Here's the complete procedure:

wget http://b.andre.pagesperso-orange.fr/ultradefrag-5.0.0AB.7.zip
unzip ultradefrag-5.0.0AB.7.zip
cd ultradefrag-5.0.0AB.7
wget https://dl.dropboxusercontent.com/u/14775223/Various/ultradefrag-5.0.0AB.7.patch
patch -p1 ultradefrag-5.0.0AB.7.patch

Update src/Makefile as needed. One might need to check the LIB2 variable, and update the paths. While at it, correct the other variables if needed.
Then compile and copy the binary:

make
cp udefrag /usr/bin/

I will not take any whatsoever responsibility if anyone destroys anything with this.
Everything here is done at your own risk. Keep in mind: Ultradefrag for Linux is only beta, utilizing a prebundled ntfs-3g library.
I have tested a few runs on a few portable USB hard drives, and it seems to be doing a great job on the NTFS filesystems. I had no data loss nor corruption so far.
But: I STRONGLY advises against using this on a Windows partition and its boot partition. It will render Windows unbootable. It's not because of corruption. It's because "Ultradefrag for Linux" cannot update Windows boot records. (Well. One could do it, and then repair the boot records afterwards.)

Last edited by SysGhost (2013-10-10 08:16:44)

Arch Linux

#1 2011-08-29 18:52:10

Defragment a NTFS partition from LINUX

#2 2011-08-29 19:01:06

Re: Defragment a NTFS partition from LINUX

#3 2011-08-29 19:01:10

Re: Defragment a NTFS partition from LINUX

#4 2011-08-29 19:03:34

Re: Defragment a NTFS partition from LINUX

#5 2011-08-29 19:53:03

Re: Defragment a NTFS partition from LINUX

#6 2011-08-29 21:19:18

Re: Defragment a NTFS partition from LINUX

#7 2011-08-29 21:25:21

Re: Defragment a NTFS partition from LINUX

#8 2013-10-07 15:58:34

Re: Defragment a NTFS partition from LINUX

#9 2013-10-07 16:23:31

Re: Defragment a NTFS partition from LINUX

#10 2013-10-07 17:55:21

Re: Defragment a NTFS partition from LINUX

#11 2013-10-07 18:28:31

Re: Defragment a NTFS partition from LINUX

#12 2013-10-07 19:01:24

Re: Defragment a NTFS partition from LINUX

#13 2013-10-08 12:34:09

Re: Defragment a NTFS partition from LINUX

#14 2013-10-08 12:51:51

Re: Defragment a NTFS partition from LINUX

#15 2013-10-08 13:09:43

Re: Defragment a NTFS partition from LINUX

#16 2013-10-08 14:01:23

Re: Defragment a NTFS partition from LINUX

#17 2013-10-08 14:31:57

Re: Defragment a NTFS partition from LINUX

#18 2013-10-08 20:43:21

Re: Defragment a NTFS partition from LINUX

#19 2013-10-08 21:28:58

Re: Defragment a NTFS partition from LINUX

#20 2013-10-08 21:41:36

Re: Defragment a NTFS partition from LINUX

#21 2013-10-08 21:59:06

Re: Defragment a NTFS partition from LINUX

#22 2013-10-08 22:18:30

Re: Defragment a NTFS partition from LINUX

#23 2013-10-08 22:31:18

Re: Defragment a NTFS partition from LINUX

#24 2013-10-08 23:13:55

Re: Defragment a NTFS partition from LINUX

#25 2013-10-10 08:15:22

Re: Defragment a NTFS partition from LINUX

Board footer