h a l f b a k e r yIf you can read this you are not following too closely.
add, search, annotate, link, view, overview, recent, by name, random
news, help, about, links, report a problem
browse anonymously,
or get an account
and write.
register,
|
|
|
Non-Fragging OS
Always keeps files contiguous, to maximize system performance | |
I just spent 3 days dealing with a problem on my most-
frequently-used computer. A message appeared saying
that I was running dangerously low on disk space. It was
true. So I erased a bunch of unnecessary files and
moved
some other things off that drive onto a different one.
But,
a few
hours later, the same message popped up again!
Something more permanent was needed. The Operating
System was willing to compress all the files on the drive,
which would approximately halve the amount of space
used. OK. But before I did that, I decided to "defrag"
the
drive first. Those who know what I'm about to describe
can skip the next three paragraphs.
There is a rather standardized/generic method that
computer Operating Systems use, with respect to
organizing files on a storage device. The total available
storage space is divided into equal-sized segments, and
a
special table is used to record which segments are being
used to hold which files. Large files often need many
storage segments, and this is where "fragmentation"
enters the picture.
The process for saving a file starts by looking for the
first
available storage-segment, as recorded in the special
table. If multiple adjacent segments are available, fine.
But if not, the table is able to indicate that the file that
is
located in this storage-segment continues in another
distant storage-segment. It works well, and efficiently
uses the total storage-space on a drive, which is why
most
computer Operating Systems use that method.
However, just because storage-space is used efficiently,
that doesn't mean it is ACCESSED efficiently. If the
read/write "head" of a disk drive has to go here and
then
there and then 'way over there to access all the
"fragments" of a total file, that is NOT efficient.
"Defrag"
programs exist to re-write the data on the disk, so that
most files end up using contiguous storage-segments. It
can take a while to do this (I just spent more than 2 days
running defrag programs over and over again, because
of limited available space for shuffling
files
about.)
So, what if the Operating System used a modified filing
method? All it really needs is a second special table.
This
table keeps track of the sizes of all the available
contiguous storage-segments (basically the same thing
as the file sizes). If some file is modified
such that it becomes larger than the space into which it
originally fit, the OS does NOT put most of the file here,
and the rest of it over there. It puts ALL of the file over
there, into a space big enough to hold it, and marks the
space where it used to sit as "available contiguous
storage-segments", to be used for the next file that can
fit
in that space.
The short-term result will of course be a lot of "holes" in
the first special table, and so one could argue that this is
not very space-efficient in terms of overall disk usage,
even if it is very efficient in terms of accessing the
files.
However, a modest modification to the preceding could
help reduce the problem, involving some more special
data....
Using the same second special table, it would also record
the "distance" (in
terms of storage-segments" each file is located from a
"hole" (doesn't matter if hole precedes or follows the file;
distances can be negative or positive). Upon moving the
too-big-to-fit file to a place where it can fit, the OS now
does a simple search for another file --the nearer a hole
the better-- that can fit in the just-available space, and
moves it to there.
The net effect of that is that holes tend to be
consolidated. If the Operating System did the preceding
thing just twice for every file that got too big and was
moved 'way over there, In The Long Run the total
number
of holes in the first special table will, statistically,
probably
seldom exceed a certain number (to be determined). If
the OS did the thing described in the previous paragraph
three times instead of twice (moving a third file into the
space opened up when the second file was moved), the
statistical number of holes will go down. Move 4 files,
and more-hole consolidation happens, and again the
total number of holes tends to go down. Possibly, that
first file, moved 'way over there, can be moved "back",
into the space where one of those other files (plus-
adjacent-hole!) used to be.
All we need to decide is what sort of trade-off do we
want,
between minimizing the numbers of holes in the first
special table, and how many files we want to shuffle any
time one file gets too big, when the goal is to keep files
non-fragmented.
[link]
|
| |
1) It's not the OS that determines how files are stored on
the drive. That's the job of the file system. |
|
| |
2) Fragmentation isn't a problem on most modern file
systems. There are techniques used to effectively prevent
it, at least until the
drive becomes almost full. The only reason you have to
defrag regularly on Windows is because NTFS is crap. |
|
| |
3) Fragmentation is irrelevant on an SSD, and such drives
should never be defragmented because it only serves to put
undue wear on the drive. Not everyone is using SSDs of
course, but that's the way things are going—at a minimum
for OS and application storage, which is what benefits most
from defragging anyway. |
|
| |
4) Your proposed method—of consolidating data and filling
holes—is just a fancy way of saying continuously defrag on
the fly. This is what Windows currently does, and it's an
acceptable Band-Aid fix for the problems with it's broken
file system, but it's not that great an idea in general. Holes
are good. You want as many holes as possible, so files can
grow. The strategy then is to scatter files all around the
drive, with the expectation that some will grow and this
leaves them room to do so. Eventually the drive will fill up,
of course, but the solution is to always leave a certain
percentage of free space on the drive to ensure best
performance. |
|
| |
[ytk], regarding your #1, note that "DOS" was an
acronym for "Disk Operating System". The filing
system was built into the operating system. I'm
not
aware that that fact is significantly different for
other operating systems, even though all of them
do
have special "file manager" programs that INVOKE
the
stuff built into the operating system, to actually
move files about... |
|
| |
Regarding it being a Good Thing to keep holes near
files, this is ONLY true for files that are likely to
grow. A huge percentage of files don't do that!
You would be wasting disk space unnecessarily, for
files that never grow. |
|
| |
Perhaps a disk could be "divided" into two
sections, one reserved for files that change size,
and one reserved for files that don't. All new files
are installed in the second region. As files are
accessed, the OS can mark them as fixed or
growing, and eventually move the fixed files to
the other section of the disk, where there are
never any holes.... |
|
| |
And yes, I am proposing a defrag-on-the-fly
system. However, I am not proposing that it be
TOTALLY THOROUGH --because that obviously
means the system will spend lots of time doing
defrag stuff every time any file grows too big for
its existing storage space, which counts against
overall system performance. I'm only proposing
that
the defrag stuff be done just enough to keep the
holes down to a reasonable number. |
|
| |
// I'm not aware that that fact is significantly different for other operating systems// |
|
| |
Nowadays, "other operating systems" basically means the UNIX-ish family, and they're *not* basically extensions of a file system. Typically, each one is able to handle a range of different file systems, (even NTFS, if they're feeling indulgent). So, now you *are* aware of it. |
|
| |
//note that "DOS" was an acronym for "Disk Operating
System". The filing system was built into the operating
system// |
|
| |
The file system was called FAT. The drivers for the FS
were
built into the OS, but there was no reason why you
couldn't
write drivers for another FS for any operating system,
including DOS. Think about network shares, for
example. When
you write to a networked drive, the OS treats it like
any other
file write, but it's the remote system that actually is
responsible for writing the file to disk. The OS and the
FS are
completely different things, and it's not correct to
think of
them as inseparable by any means. |
|
| |
//Regarding it being a Good Thing to keep holes near
files,
this is ONLY true for files that are likely to grow. A
huge
percentage of files don't do that! You would be
wasting disk
space unnecessarily, for files that never grow.// |
|
| |
Please explain how you're “wasting” disk space by
doing this.
You're not losing any additional space over writing
them
contiguously. You have exactly the same number of
bytes
available either way. Nothing is wasted. |
|
| |
In fact, if you insist on writing files contiguously to the
drive,
I'd call that “wasting” space, because you have all this
extra
space on the drive that you're not putting to good use
as
breathing room! It'd be like having a huge warehouse,
but
nevertheless cramming your merchandise into a single
corner
as tightly as possible. |
|
| |
Let me describe a simple algorithm for storing files on
a disk: |
|
| |
1) Find the biggest hole on the drive.
2) Put the file halfway into that hole.
3) If there's no place big enough for the file, split it
into
multiple chunks and restart at step 1 for each chunk.
|
|
| |
Even if most files don't grow, that's not a problem,
because
some files might and these files have room to. With
this
algorithm, you won't have to start fragmenting files
until the
disk is substantially filled up, even if some files do
grow in
size. But this way, the average amount of “growing
room” for
each file decreases as files are added to the disk. This
works
well unless you have a ton of files that are likely to
grow from very
small sizes to very large sizes. Fortunately, most files
don't do
that. |
|
| |
[pertinax], OK, thanks. It doesn't bother me if the
filing system, with its own appropriate special
tables, qualifies as an "application" in a *nix
environment. |
|
| |
[ytk], FAT stands for "file allocation table", and is
exactly the "first special table" mentioned in the
main text. The overall filing system is more than
just the FAT, because code has to be executed
that makes use of the FAT (plus keep it updated).
And that code was part of the Operating System. |
|
| |
Next, you may have misinterpreted something. I'm
quite aware that the last "storage segment"
associated with a file can be partially filled. This
Idea is NOT about trying to fill it completely, using
part of another file! The "holes" I'm talking about
are "completely empty storage segments" (and are
directly recorded in the ordinary FAT), and
nothing else. So, if a region of a disk can be
reserved for files that don't change in size, they
CAN be packed into that region without any of
THOSE kind of holes --and none of those files
would NEED any "breathing room", since they
DON'T change in size! |
|
| |
Finally, you have specifically detailed
fragmentation as a solution to holes in the FAT.
DUH! The main reason for my writing the main
text here was to prevent fragmentation of files! |
|
| |
Yes, I realize that "fragmentation of available
empty space" is the result of what I proposed --
but note that it is FILES that are read by the
computer, not the empty space, so the existence
of holes doesn't affect performance in the way
that fragmentation affects performance. |
|
| |
Anyway, I also proposed a way of minimizing holes
without greatly affecting ordinary performance
(because most of the time, most files are not
growing in size, and the things I proposed only
"kick in" when one file has grown to use up the
last of the space in the last storage-segment it
occupies, and needs to be moved to a different
place on the disk). |
|
| |
Sure, it could be helpful if there was a hole
following the file, so it could grow into that hole
without being moved. But even most files that
grow don't do it every day. Some only grow when
a software update is done, for example. Why
should those files be associated with holes if
updates only get done once a year? |
|
| |
The whole point of not having holes in the FAT is
to be able to say that the disk is being efficiently
used to store stuff. Nobody wants to buy a new
hard disk if 10% of it is still available as scattered
holes, none big enough for a single unfragmented
file. Fragmentation specifically exists to finish
using that space. I do NOT deny its usefulness. |
|
| |
I'm just saying that fragmentation isn't necessary
UNTIL the disk is almost full! And if the quantity
of holes can be kept to a reasonably small number,
very few files will be fragmented when the "disk
full" error happens. |
|
| |
This is the opposite of what you want to do. You want to maximize
holes. Do a bit of research into file system design and you'll see
why this is the case. |
|
| |
The main point, however, is this: You are trying to solve a
problem that no modern file system has, with the exception of
ones used by Microsoft operating systems. There is no such thing
as maintenance defragging on Macs or Linux systems. The
requirement to defrag is 100% because of bad design choices on
the part of Microsoft, combined with their continued use of legacy
technology for reasons of backwards compatibility. |
|
| |
This problem has been solved for *decades*. Your attempt to
design a filesystem that requires no defragging is roughly
equivalent to designing a way of interacting with your computer
that doesn't require you to write programs in assembly. |
|
| |
What [ytk] said, with brass knobs on. |
|
| |
[suggested-for-deletion], not a new idea. |
|
| |
//The requirement to defrag is 100% because of bad
design choices on the part of Microsoft, combined
with their continued use of legacy technology for
reasons of backwards compatibility// Do you know if
this is still true in Windows 8.1? |
|
| |
Windows. Microsoft. Bad design choices. Legacy technology. |
|
| |
What part of this don't you understand ? |
|
| |
You forgot "Ironic names" |
|
| |
If he has to ask the question, best not to overload him with
information; he's clearly only got a little brane. |
|
| |
//Do you know if this is still true in Windows 8.1?// |
|
| |
Yes. Windows 8.1 just automatically defrags your drive
for you once every week or so. |
|
| |
Waste of time totally useless idea. Not needed and never will be, because: |
|
| |
1. Drives are super dirt cheap now. And you can get terabytes of cloud storage for nothing. There is no excuse for running out of space. Go buy another drive, cheapskate. |
|
| |
2. While you're at it, make it a solid state drive, which eliminates the fragmentation problem. |
|
| |
I was disappointed that frag grenades were not involved in some way. |
|
| |
What everybody said, as long as it was about how Unix/Linux is so much better than Microsmurf stuff.... |
|
| |
Same thing happened to me. It was an out of control virus checker gobbling space no matter how much was freed. You are not alone. |
|
| |
//You are trying to solve a problem that no modern file system has, with the exception of ones used by Microsoft operating systems.// |
|
| |
You are trying to solve a problem that every modern file system has already solved, with the exception of ones used by Microsoft operating systems. |
|
| |
[ytk], I misstated what I meant in a couple places.
I was talking about minimizing QUANTITIES of
holes, by consolidating a small number of them
each time some file outgrew its existing location.
Also, I mentioned something about getting a new
hard disk, but neglected to specify that it would
be the OLD disk that still had 10% of its space
available. |
|
| |
[8th of 7] and others, OK, so the TITLE of this Idea
is WKTE. But if they way those existing systems
do that thing is different from the proposal here,
then there is no reason to delete this. |
|
| |
//[ytk], I misstated what I meant in a couple places. I was talking about
minimizing QUANTITIES of holes, by consolidating a small number of
them each time some file outgrew its existing location.// |
|
| |
I got that part. I'm saying that it's exactly the wrong strategy. The ideal
situation is to have a hole after each file. The notion that you're
wasting disk space by doing so is incorrect. The amount of disk space is
fixed. If you're not using the extra empty space on the disk as
“breathing room”, you're wasting it. |
|
| |
Look at it this way. If we're trying to minimize the number of holes on
the drive, the optimum quantity for holes is obviously 1. So let's say we
have a bunch of files arranged contiguously that fill up the first 10% of a
1TB drive, leaving a single gap of 900GB at the end. Now, let's say a file
in the first 5% of the drive grows beyond the capacity assigned to it. In
order to “optimize” the drive according to your scheme, you need to:
1) Read the entire contents of the file
2) Write the modified file to a scratch area at the end of the disk, and
mark the original file location as empty
3) Find a file (or group of files) that fits into the gap, and read its
contents
4) Write those files to the empty hole, and mark the old location of the
files as empty
5) Read the contents of the original file, which is now in the scratch
space
6) Write the contents of that file to the end of the contiguous group of
files, and mark the scratch space empty.
Okay, so far just growing a single file involves a MINIMUM of three read
and three write operations. But wait! Unless the file that was moved
into the gap left by the original file happened to be the last file on the
disk, we now have a gap open where that file previously was. Okay, loop
back to step 3, and repeat until all of the files are optimized. So a write
operation to a single file at the head of the disk could potentially
involve moving *every single file on the disk*, and on average will
require moving roughly *half* of them. The only alternative is to ignore
these small gaps, but then you're back to your original problem of
increasing the likelihood of fragmentation as more files are added to the
disk. Or you could just not write to these small gaps, but then you really
are wasting space.
And what if a file shrinks, or is deleted? Then you have the same
problem. You're left with a tiny gap, and you have to start juggling files
around to fill it. If you delete the first file on the drive, hope you didn't
have anything to do for the next 6 hours or so while we consolidate gaps
for you.
Compare this to the process of growing a single file where files are
spread out evenly on the drive, so every file has a maximum amount of
space to grow:
1) Expand the file.
2) If the file needs to expand beyond the available gap, read the file
that's in the way, and put it in the middle of the next largest gap.
Repeat from step 1 as necessary.
|
|
| |
The minimum number of operations is a single write, and the average
number of operations will depend on the size of the files you're
generally writing, but for the vast majority of use cases will be fairly
low until the drive is almost full. |
|
| |
permanent vs ephemeral (existence), static vs dynamic (size). |
|
| |
Basically you want 4 partitions/drives to roughly separate chaff from wheat. |
|
| |
C: OS, Program files.
D: Ephemerals: swap, spooler, internet cache, downloads, system restore points, logfiles, etc.
E: Personal: personal. You can move "My Documents" here.
F: Media: static files which aren't being changed or deleted. This could include the mp3 collection, Shakespeare's works in .pdf, etc. etc. |
|
| |
M$ won't let you do it properly; there will still be loose dynamic files in C:, but it keeps fragmentation down to almost nothing. |
|
| |
D: doesn't need to be defragmented - all the files there are only going to be read a couple times then deleted anyways. |
|
| |
E: won't be too fragmented; depends on what you do. |
|
| |
F: will never be fragmented so it never needs to be defragmented. |
|
| |
If you have enough RAM you can ditch the swapfile and internet-cache completely and move all the per-session temp files to a ramdisk. |
|
| |
[ytk], the main text here only specified moving 3
or 4 different files around, when it happens that
the first one outgrew its space. I also specified a
special table to record file-sizes and such, so no
need to scan the whole disk every time some file
gets too big; all the needed data is already in that
special table. I was relying on statistics to keep
the total number of holes to a minimum. So, sure,
I knew that some performance would be lost in
moving the 3 or 4 files around. But ONLY when
one file gets too big. Currently, with lots of
fragmented files, you almost always have a
performance hit. I think it is a fair trade, to
accept an occasional performance hit in exchange
for having no performance hit most of the time. |
|
| |
Also, you seem to have missed the point that
some files almost never change their size. If you
download a .PDF for reference purposes, you are
not planning on editing it. Therefore its size
should never change. What does it need a hole in
the filing system for? Lots of files are in that
category, of not needing any extra space (beyond
the typical bit of space in a single storage-
segment, only partly used by the last part of the
file). |
|
| |
So, in the scheme I proposed, files that never
need to grow will STAY contiguous and stay
unassociated with special holes in the filing
system. Files that do need to grow will probably
be associated with holes, one way or another. |
|
| |
So what do you do when you're downloading a file that you've no clue what size it is. |
|
| |
The whole discussion is blinkered by the physical limitations of your
current rotating magnetic media, which is not "direct access" but
"pseudo-direct semi-sequential indexed access with variable access
time". |
|
| |
And as such, is about as useful as a detailed dissertation on improved
methods of flint-knapping*, and therefore of interest only to
experimental archaeologists, survivalists, and owners of muzzle-
loading flintlock firearms. |
|
| |
*Officially, flint-knapping is the world's oldest profession, which has
led to some rather embarrassing misunderstandings. |
|
| |
// you can get terabytes of cloud storage for nothing // |
|
| |
If you live in a place with high speed internet access. Too
often these days I run up against tech that has clearly been
designed by people who assume the entire world has
overlapping 4G wireless bubbles and fiberoptic landlines. |
|
| |
I'm going back to storing the zeros and ones perpendicular to to the surface of whatever the storage medium is, it's the most logical solution. |
|
| |
[8th of 7] I do think that if you look into zoology
far
enough, you can find a number of examples of
males
offering females tasty morsels in exchange for
mating
privileges. No flint-knapping needed. |
|
| |
[Flying Toaster], at least one browser appears to
download a "temporary file" and doesn't save the
file to the official download location until it is fully
downloaded. That makes some sense, in case the
connection breaks during the download. And of
course if the download succeeds, then its full file-
size would be known at that time.... |
|
| |
//[ytk], the main text here only specified moving 3 or 4 different files around, when it
happens that the first one outgrew its space.// |
|
| |
That doesn't make any sense. You can't have it both ways—either you're moving all the
files around to fill all the gaps, or you're leaving the gaps in place, leading eventually
to fragmentation. |
|
| |
//I also specified a special table to record file-sizes and such, so no need to scan the
whole disk every time some file gets too big; all the needed data is already in that
special table.// |
|
| |
This doesn't mean anything. Of course there's a special table that specifies where the
holes are—that's how the file system finds the data it's looking for. You don't need a
special table just to list the holes—you look at the table of contents for where there's
empty space. The notion of a special table for keeping track of where there *isn't*
data is completely redundant. |
|
| |
I never said anything about scanning the whole disk. Scanning the disk isn't an issue.
The problem has nothing to do with finding where the empty space is on the disk, and
everything to do with consolidating the empty space. The only way to do that is by
rewriting entire files. |
|
| |
//So, sure, I knew that some performance would be lost in moving the 3 or 4 files
around. But ONLY when one file gets too big.// |
|
| |
Or when you delete a file. Guess what's left in that space? A gap. And how do you fill
that gap? With another file. And what do you put in the gap left by THAT file? Yet
another file. And so on until the disk is completely optimized. Repeat for the entire
disk any time a gap opens up—i.e., whenever a file grows, shrinks, or is deleted. |
|
| |
//Currently, with lots of fragmented files, you almost always have a performance hit. I
think it is a fair trade, to accept an occasional performance hit in exchange for having
no performance hit most of the time.// |
|
| |
Except you'll have a *huge* performance hit every time you expand a file, or delete a
file, or do anything that makes a gap. Your disk will be thrashing like crazy trying to
move stuff around in an obsessive-compulsive attempt to stuff all of the data
contiguously at the head of the disk. |
|
| |
//Also, you seem to have missed the point that some files almost never change their
size.// |
|
| |
But some do, and the system has no real way of knowing in advance which ones will
and which ones won't. Better to leave as much space as possible for every file, in case
it does grow. Then you only need to start moving data around for files that grow larger
than their allotted space. And as you add files to the disk, the allotted space for each
file decreases. |
|
| |
//If you download a .PDF for reference purposes, you are not planning on editing it.
Therefore its size should never change.// |
|
| |
Unless you delete it, then its size changes to zero. |
|
| |
//What does it need a hole in the filing system for?// |
|
| |
Because if you don't need the space, eventually you'll be able to stick another file in
there. And if you delete the file, you don't need to move anything around to replace it
with a larger file without fragmenting. |
|
| |
//Files that do need to grow will probably be associated with holes, one way or
another.// |
|
| |
You never specified any way to determine which files will need to grow in the future.
How is the system supposed to know that? |
|
| |
Anyway, why is it important to keep free space contiguous? If you have a 1TB disk
that's 10% full, you're not going to suddenly write an 800 GB file to it. So there's no
reason to have your free space contiguous. But you might well write a bunch of smaller
files, and if your 100GB of data is spread around the drive, you'll have no trouble
finding places to put these files. And as the drive fills up, if you have a bunch of gaps
it's easy to move smaller files into the gaps between other files to clear up a
contiguous zone of free space on an as-needed basis. |
|
| |
I'm sorry, but this is just a completely unworkable idea. The only reason it's not MFD
WKTE is because it's so demonstrably flawed that nobody would ever implement this.
Even Microsoft wouldn't do this, and that's saying something. |
|
| |
[V] Regards downloading, if both Temp (where the temp file is located) and the end-location (where you told it you want it to go) directories are on the same disk, the physical location remains in the same place: the filesystem simply jiggers the FAT so it appears in the new directory while disappearing from the temp. |
|
| |
By contrast your system would download the file into the biggest hole then move it to the best fit location after the download was finished. |
|
| |
I wonder if a pattern would appear on-disk after awhile of using your system. |
|
| |
[ytk], what I'm talking about DOES make sense.
When I last looked at the details of a FAT, I
noticed that it didn't care how long a file was.
Each file was associated with a start-location in
the FAT, and the table itself simply recorded the
next "block" where the file continued, with a flag-
bit indicating when the last block was reached. |
|
| |
So, that second special table I mentioned WOULD
BE primarily a table of file-lengths and the amount
of FAT associated with each file (including any
extra unused data-segments, --holes, that is). It
means that when a big hole opens up in the FAT,
like when a file is deleted, it can be pretty easy to
find a file that could fit into the hole, using the
second table. |
|
| |
I specifically indicated that we want to seek a file-
that-fits that happens to already be associated
with a hole. That means when the file is moved,
the available hole becomes bigger than before,
when the original file was deleted. If you can do
this 2 or 3 times, then you will have consolidated
multiple holes, and consequently the total number
of holes in the Fat will have gone down, after the
file was deleted and those few files got moved
around. |
|
| |
If this is done every time a file is deleted, or
moved because of being too big to fit after the
latest edit,
then certainly the FAT will acquire some holes.
But STATISTICALLY, in the long run, holes will be
consolidated about as often as they get created.
The net result is a fairly small and relatively
constant number of holes. Should you ever decide
to do a "full defrag", it shouldn't take a huge
amount of time, certainly not when compared to
what I wrote at the start of the main text here! |
|
| |
Remember the Title of this Idea. It is about
keeping files non-fragmented. It is also about
preventing lots of holes, NOT so much about
ensuring their number is always Zero. I think that
what I've described would work to accomplish
what I set out to accomplish. |
|
| |
You keep talking about things like a "need" for a
vast number of holes, or a "need" to waste vast
amounts of system performance to prevent all
holes, and I've been describing a compromise with
regard to holes, while staying focused on the main
goal of keeping each file contiguous (with respect
to itself) on the disk, to maximize the
performance associated with file-loads and saves. |
|
| |
[Flying Toaster], you may be right about the temp
file, but it may depend on the browser and the
computer. It might be saved in RAM, for example,
until the download is finished. |
|
| |
//[ytk], what I'm talking about DOES make sense.// |
|
| |
No. It makes no sense to keep a redundant table of
where files *aren't*, when you have a perfectly good
table of where files *are*. |
|
| |
You're clearly not a programmer, or you'd realize that
what you're describing is a classic anti-pattern. You
*never* want to have two different sources describing
the same set of data, because then you face the
problem of determining which one is authoritative.
What if the file allocation table gets updated, but the
free space allocation table doesn't for some reason?
Say the power gets cut after updating one table but
not the other. Well, then you're looking at potential
data loss when the empty space table says “go ahead
and write there” but the file allocation table says
“there's a file there”. So if the solution is to verify
each table against the other, then you're faced with
another problem—which table is correct? Okay, for
safety you could say the file allocation table must
always be correct, in which case why bother with the
free space table at all? If every lookup in the free
space table has to be verified against the allocation
table, then the free space table does you no good. |
|
| |
//When I last looked at the details of a FAT, I noticed
that it didn't care how long a file was. Each file was
associated with a start-location in the FAT, and the
table itself simply recorded the next "block" where the
file continued, with a flag- bit indicating when the last
block was reached.// |
|
| |
FAT is an ancient technology. Even Microsoft doesn't
use FAT anymore, except to maintain backwards
compatibility. And it's for this reason that it's futile to
attempt to improve on the FAT filesystem—any
“improvement” would be incompatible. If you want to
eliminate the problem of file fragmentation, you have
to switch to another filesystem that's not compatible
with FAT. |
|
| |
You still haven't explained *why* you want to get rid of
holes. You can't just assert that holes lead to
fragmentation, because they simply don't. The proof
of this is
that NO MODERN FILESYSTEM HAS A PROBLEM WITH
FRAGMENTATION. |
|
| |
//You're clearly not a programmer, or you'd realize that what you're describing is a classic anti-pattern. You *never* want to have two different sources describing the same set of data, because then you face the problem of determining which one is authoritative.// |
|
| |
Um, never say never.
In fact, it's pretty common in various forms. How you cope with potential disagreements varies wildly depending on the case in hand.
"All programming is an exercise in caching."
-Terje Mathisen (apparently) |
|
| |
As a related example, I remember reading a description of a memory allocation system which stored the allocated blocks (as it must), but also stored details about the empty space in the empty blocks themselves. |
|
| |
For what it's worth, I think you're arguing over a difference in terminology. ytk is saying there should be gaps between files. Vernon is saying that each file should be written as a contiguous block.
I think you're both mostly right. |
|
| |
[ytk], if what I've proposed counts as a fix for
something that is obsolete and has been replaced by
things that work differently from this proposal, then
obviously the thing I proposed counts, perfectly, as
Half-Baked! |
|
| |