LowFrag v1.3 (20-Dec-96) - By Eric Sauvageau.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0. History
~~~~~~~~~~
1.0 - First version - limited release.
1.2 - Wouldn't work if no threshold value was specified from the command
line.
- Fixed major enforcer hits :(
1.3 - The AllocMem() patch wasn't properly patched (Wrong LVO). Patching
the right one slowed down the system too much, so I simply
removed it.
- Now promotes allocs larger than the treshold instead of the smaller
(more common) ones. It's a bit faster.
1. Introduction
~~~~~~~~~~~~~~~
"Out of Memory" the program told you. What the heck?! You have 2 megs of
memory, 1 of which is free, and this 300 Kb program tells you
"Out of Memory"? Then you open a Shell, type "Avail" and press Return.
And Avail tells you this:
Type Available In-Use Maximum Largest
chip 1004888 1091240 2096128 207000
fast 0 0 0 0
total 1004888 1091240 2096128 207000
This is called "memory fragmentation". You have a "playground" of
2 Megabytes of RAM, but little programs are playing all over it, leaving
only small areas free between each of them rather than one large free area.
Yes, programs are like kids, they love to scatter everywhere.
Unfortunately, it's pratically impossible to "unfrag" memory on the Amiga.
Best thing you can do is to try an "Avail Flush" in a Shell, hoping that it
will remove a library from memory, allowing two small areas of 200 Kb each
to join together into a 400 Kb large area, the library separating them
being freed from memory.
Ok, so we can't unfrag memory. How about giving a little more organization
to these "kids", telling them to play together rather than scattering all
over the playground? Well, it ain't that simple. Kids never listens.
You know this, you've been there too.
Fortunately, the memory "playground" has a little more order than a
playground invaded by kids. Before getting access to the "playground",
you must ask Mr. Exec if you can go play on it, and where to play. But
you see, Mr. Exec is very busy watching Mrs. Paula sunbathing on the other
side of the park, so he's doing a quick'n'dirty job: he starts looking
from the closest area, spots the first area large enough for the one
asking for space, and directs it there. And Mr. Exec goes back to the
sight of Mrs. Paula, sunbathing in a tiny bikini.
2. But I have an Amiga, no a kid playground!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is how Exec works. When a program needs a memory area for its code
or data, it asks Exec to assign it a free area, and goes there. This is
what the DOS does when you load a program: it asks Exec for areas where it
can put the program. And then, the programs asks Exec for areas where it
can put the various data, libraries, etc... But problem is, when the
program leaves, some of the data can stay in memory:
1) Files created in the Ram Disk.
2) Libraries/devices loaded.
3) Files in the clipboard, usualy located in the Ram Disk.
4) Areas "forgotten" by the program.
5) Small "patches" left in memory.
And there is surely a few other cases.
Or, another situation: Two programs are loaded one after the other. The
first one leaves, but not the second one. Result: an area of the size of
the first program is left, separated from the remaining free memory by the
second program.
We can't really fix this problem. But we can help reducing the number of
situations where it can occur. This is what LowFrag can do.
3. Soooo... What does LowFrag do?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
LowFrag will eliminate one of the cases where memory gets fragmented:
the case where a program is loaded (100-500 Kb) in memory, and exists
leaving some data behind it in memory (clipboard, libraries, etc...).
Or when another program puts small allocations just after the large
program, so when the large program quites, it leaves a rather large
block separated from the main area.
How does LowFrag works?
Usually, the data left in memory are small blocks (small when compared
to the programs, that is). So, we could reduce fragmentation by
separating the large allocations (usually applications) from the smaller
allocations (usually data, patches, etc...). Exec's memory allocation
has a flag which will do just that:
MEMF_REVERSE -> Allocate memory starting from the top of memory.
When this flag is used on a memory request, Exec will start looking from
the TOP of memory for a free area, rather than from the bottom (as usual).
So, we can separate allocations by putting the "MEMF_REVERSE" flag to the
smaller allocations, so they stay out of the way of the larger allocations.
This is what LowFrag does. When an allocation is larger than a threshold
value, the allocation gets "stamped" with a "MEMF_REVERSE" flag, telling
Exec to find it a suitable area by starting from the top of memory. So
when the application leaves, any smaller allocations left in memory won't
clutter the area usually needed by large.
Here's a concrete case that I tested myself where LowFrag reduce
fragmentation:
1) Launch a terminal (Terminus) - Around 400 Kb
2) Cut some text into the clipboard - A few bytes
3) Exit the terminal.
Without LowFrag: The memory is splitted by the clipboard content. But
with LowFrag, the clipboard content is at the top of the memory. So rather
than "cutting" a 400 Kb block from the main area, it will probably cut
around 10-50 Kb (small allocations done by the terminal: serial.device,
various libraries, etc...). We can call this an improvement I guess :)
4. Sounds like fun... I want to use it!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Simple: copy LowFrag in your C: directory, and somewhere in your
startup-sequence (usually just after SetPatch), add this single line:
Run <>NIL: c:LowFrag value
value is the threshold limit: allocations smaller than this value will get
"promoted" with MEMF_REVERSE. If you don't put any value, then the default
threshold of 32768 (32 Kb) will be used. I found this value to work very
well on my own system, so try the patch first with this value, adjusting it
afterward to suit your system.
LowFrag can be aborted at any time by sending it a CTRL-C code, altough
this isn't recommended: if another program patches AllocMem() or AllocVec()
afterward, it's most likely that the system will crash.
5. That's it
~~~~~~~~~~~~
So enjoy. The whole thing is released as public domain. If you have any
ideas on how to improve the thing, the source code is there in the
archive (written in Amiga E, with some inline assembly), so just go ahead.
A few possible ideas:
- Promote allocations which has the flag MEMF_PUBLIC.
- Have the user define a list of tasks which should always get promoted
(altough this could probably slow down the system. Memory allocation
is one of the cornerstones of the OS, so any changes there will reflect
on the whole system's performances.
6. Hints & tips for programmers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can help reducing the memory fragmentation problem.
a) When possible, use the memory pool system available since Exec V39
(it's been in amiga.lib since V33 I think?).
b) For large executables, split it into separate hunks, so they could
be loaded even on a fragmented system.
c) Make sure you don't leave anything behind when you quit your program,
unless when needed. Testing for memory leaks should be part of every
debugging routine (Enforcer won't intercept by itself ALL kind of bugs,
guys.)
d) When doing an allocation meant to STAY in memory, why don't use the
MEMF_REVERSE flag?
e) For large applications, it is often a good idea to put part of the code
into a shared library, and keep the library open only when it is needed.
This way, it can be flushed from memory when the system needs more
memory.
7. The Author
~~~~~~~~~~~~~
I can be reached via Fidonet: Eric Sauvageau>1:242/907.0
Or via Internet:
merlin@thule.icenet.no
dream@step.polymtl.ca (this is a friend's account)
http://thule.icenet.no/~merlin
This whole little thing is released on the Public Domain. Do whatever you
want with it. As for I, I decline any responsability if you encounter
problems while using this patch. It's been in use on my system and
a friend's for many weeks before I released it, without any problem.
I also wrote a few other goodies for the Amiga:
TDPrefs - Preferences editor for the trackdisk.device.
MFormat - Replacement for CBM's "Format" command.
DevsMan - Manager for the Devs: and Storage directories.
NewIcons3 - The ultimate icon enhancer.
XPKatana - GUI/ARexx interface for the XPK packing system.
See ya!
Merlin
------
|
aminet net>