PDA

View Full Version : Like a midget at a urinal, I would have to be on my toes



Alimonster
04-11-2002, 07:06 PM
This thread is about exe size. What's the smallest win32 (not console!) app that you've managed to produce and how did you do it?

I'm interested in l33t 64K intros 8) , so every K counts here. You may be surprised at the exe size I'll present here. It's not the smallest though. I'd have to be hardcore to get smaller.

5.5K for a fully functional win32 app. Yep. I can go even smaller too if I manage to recompile the RTL (which will be one hell of a feat given that I only have the personal edition without source...). I have ideas on that, which will result in a *very very* small app. However, first I'd like to hear your values so far. What size is yours? :oops:

BlueCat
04-11-2002, 08:33 PM
I have very little experience in this sort of thing, but as I'm the only one here at the moment I'll reply!

I had no idea you could make a win32 app so small, how do you go about doing that? I suppose the smallest app I ever made would have been on my ZX81 about 20 years ago but that doesn't count really :P

Alimonster
04-11-2002, 09:50 PM
I'm using Delphi 6 Personal Edition here.

Okay, a step by step guide for doing this...

1) Start a new project
2) Remove the form ("remove file from project")
3) Go the the project menu and select "view source" (side note: does anyone else wonder why this wasn't View->Project Source instead? Just me? Oh well... :? )
4) You're left sitting at the project source itself (the .dpr file). This is the file that usually means "hands off". However, it's the best way to get tiny exes - ignore the VCL and use direct Win32 stuff. Yucky, but the size makes up for it.
5) Save your project somewhere. Compile it. Notice how this minimal app takes up *351K*!
6) We don't want any VCL stuff - which means no "uses forms". Oh well, back to our Pascal roots, I guess. The .dpr file can be laid out like this:

program Project1; // name depends on what you saved it as

begin

end.

I got rid of the "uses forms" bit there, since that's including VCL stuff. The other bits about "application.initialize" and "application.run" are VCL things too. Now we have a skeleton app with absolutely no functionality. Gee, ain't that useful :). On the plus side, it's down to 8K from 351K(!!).

I got rid of the {$R *.RES}, which includes the icon. This is about 1/2 a K of unnecessary stuff.

Now, the main challenge is to turn this into a working win32 program. This takes a bit of savvy, unfortunately, and especially so if you're used to the comfy VCL. I've got a tutorial on this, pimp pimp, over at my site: thisaways (http://www.alistairkeys.co.uk/gdipart4.shtml)

We want to get a Window Class, register it, create a window, write a message loop (PeekMessage/DispatchMessage) and a window proc to handle some messages. Eesh!

I solved this by copying and pasting function prototypes from the windows.pas and messages.pas (I have the VCL from Delphi 5, but its compiler generates *much* larger win32 exes. D6 is a lot leaner in this respect, though you wouldn't expect as much from the increased VCL program size. I've not tried D7 yet).

My sizetest unit has *no uses clause at all*! Thus, I had to copy over the definitions of each required win32 function. These boiled down to simple declarations of dll functions, e.g.

procedure PostQuitMessage(nExitCode: Integer); stdcall; external 'user32.dll';

With care, I managed to get an win32 window up and running... in 8K(!). That's the same value as the empty unit, btw. *Cough cough*. ;)

Now, I'm not done yet. There's a lovely exe packer that everyone must know about if interested in tiny exes: UPX (http://upx.sourceforge.net). I ran this over my exe and the result is currently sitting at 5.5K.

Note, though... if an empty app sits at 8K, and my thing ended up at 8K, then there must be some reason for the extra bloat such that adding more functionality doesn't change the exe size.

The trick is this: even though no extra units were included, Delphi includes some anyway! There are two relevant files, System.pas and SysInit.pas, which form the RTL (run-time library). Ever wondered where the default Input/Output console vars come from? The files are set up in there. Ever wonder where the HInstance parameter is set up? In there.

The RTL contains a bunch of crap that I don't need. Unfortunately, this is where I'm sitting at just now. I don't have the VCL source code for Delphi 6, which means that I can't create 'dummy' units for those two with nothing in them. If I try (compiling at the command line using 'dcc32.exe') then it lies, saying it can't find the .pas files (even if I create dummy ones).

I've tried modifying the compiled .dcu units without luck - if I had the VCL source for D6 then I'd be all set, dammit. Oh well - 5.5K is good enough for now :).

BlueCat
04-11-2002, 09:59 PM
So that's how it's done, cool. 351K seems a lot for a minimal app, can't say I'd noticed til now. My game demo exe is less than 700K and does a lot more than display a blank form. Now if you can get that exe down to less than 1K I'll be impressed :wink:

Alimonster
04-11-2002, 10:03 PM
I could compress it down easily enough - arrange it so that for the bits, all the 0s are consecutive, then all the 1s. Write the count of each and you'll be left with about 16 bytes of data. Of course, I'll leave decompressing that up to you :roll:

I pimped this over at gamedev in a closed thread, but here it is again: the latest tech in 4K intros (http://www.pouet.net/prod.php?which=7990). This would probably be done entirely in assembly. Cool stuff!

Sly
04-11-2002, 11:20 PM
I have a working OpenGL app that currently sits at 15KB non-compressed. It contains drastically cut-down versions of GL.pas, GLU.pas and Windows.pas.

Alimonster
04-11-2002, 11:28 PM
Have you still not got around to doing that 64K intro yet, Sly? :P Yep, I saw that post on the borland.public.delphi.graphics newsgroup ages ago where you said you would do one in OpenGL ^._.^

Btw, important notice about size: avoid "String"s, believe it or not, because including them causes lots of miscellaneous junk to be chucked in. The same goes for other memory functions, IIRC. Including an unused string variable in my test app caused the size to increase from 8K to 13.5K instantly (both uncompressed).

Also, avoid the math unit. Its inclusion can bump up the size a fair bit and the most interesting stuff is in "system" anyway (i.e., sin and cos).

TheLion
05-11-2002, 08:37 AM
How about PChars do they absorb less memory than a string ?

Alimonster
05-11-2002, 09:22 AM
TheLion, I'll confirm (or otherwise) that idea later on. I only have Delphi 5 here at work, where it doesn't matter (the exes produced end up 16K regardless). I'd imagine that PChars, char arrays ("array[0..255] of char") or ShortStrings would be quite good for size, but you'll have to wait for confirmation, unfortunately. ShortStrings get implicitly converted into Strings, you see, so I can't say for sure with those.

Also, I have to investigate alternative to the usual memory managers. I guess you could use HeapAlloc/HeapFree (assuming I remember the names right!) from Win32.

Incidentally, has anyone played around with size + Delphi7 yet?

Marty
05-11-2002, 02:38 PM
i have managed to produce a 9.1kb small executable (compressed by upx) with directxgraphics header. but maybe with the tricks of alimonster i can get it even smaller.

Alimonster
05-11-2002, 10:01 PM
Interesting! Looks like there are certain conditions that cause bloat with strings.

I declared a string globally (var x: string;). This increased the file size from 8K to 8.5K. However, if I initialised it at the same time (var x: string = 'blah') then the filesize jumped to 13.5K! Hmm! The same test with an integer didn't increase the filesize - nor did PChars or ShortStrings.

Another tidbit: declaring the strings as vars inside a procedure didn't increase the file size at all.

I still want to get rid of that bloody RTL though. Some day...

TheLion
05-11-2002, 10:06 PM
Could those problems with Strings be caused by the effect that Delphi doesn't treat Strings like Strings (Char[255]) anymore, but sees them as an AnsiString (16 bytes) now?

Alimonster
05-11-2002, 10:53 PM
Yes and no, I think. They're not shortstrings any more, but rather pointers. They use a copy-on-write dealie with reference counting. This means that for each string, you get a pointer to the data, a reference count and a length (this is hidden from you). This means that you get memory allocation when using them (sometimes).

The problem would seem to be that it uses some memory allocation behind the scenes - maybe. I seem to recall reading that this is the case, and that it introduces a little overhead (maybe in Ray Lischner's Delphi in a Nutshell book?). Something's fishy there, anyway. It's quite interesting that assigning it a value in the global area gives a massive increase in the exe size! I'd imagine that dynamic arrays would also cause bloat. I'll check now...

YEP! Declaring a dynamic array (var x: array of integer) increases the size up to 8.5K (half a K). If I then use SetLength, the size jumps up to 16K! It looks like the memory management is playing silly buggers. If I use a pointer to integer + getmem then there **was the same size increase as with strings** to 13.5K.

I tried declaring a pointer to an integer and using LocalAlloc instead (a win32 function). Guess what? No size increase - 8K still :). I'm not sure what alloc function is best (apparently LocalAlloc and GlobalAlloc are the same, and HeapAlloc seems slightly more involved, in that you have to create it first(?)).

Another interesting result! I've found that declaring a global var as a string array, like this...

var x: array[0..0] of string = ('blah');

...increased the size to 14K (that's one K more than a normal string, even though it amounts to the same thing). Hmm! (Again, an array of 1 integer, initialised, didn't increase the exe size).

It looks like you should watch out for strings, since they're dynamically managed behind the scenes! The same goes for GetMem or New and dynamic arrays.

It's probably an idea to wrap up memory allocation around the Win32 API calls, then.

Anyone else tried stuff like this? What interesting results are still to come...

Michalson
07-11-2002, 05:08 PM
Well Alimonster asked me to reply to this, I guess because I coded a 999 byte graphical demo just after 4E3 (Link (http://www26.brinkster.com/michalso/4e.htm)). This is going to disappoint you guys but it wasn't coded in Delphi (it isn't even a Windows executable) and to be blunt, despite the fact that I love Delphi (except for tiny demos and web stuff I do all my personal programs in it), it just isn't that good for making small executables, nor was it intended to be. If you want to code small stuff you're going to want to use a tight C compiler or do it in pure assembler. Of course if all you want to do is entry a 64K competion (64K is a lot, especially now that you don't have to write your own 3D engine and can just use OpenGL/DirectX) then Delphi isn't too much of a disadvantage. Once you get over the minimum size the code you add (assuming you don't add libraries) is not going to make a large impact on the size of the exe (again, so long as the code doesn't require any large libraries).

...I'll try making a "small" Delphi app tonight. If I get something smaller than what has already been described I'll post it and explain the methods.[/url]

Alimonster
07-11-2002, 06:38 PM
Actually, I knew that thing was done in asm and that's one of the reasons I asked you to post here :). It seems to me that you'd know more about the internal structure of the PE format, since you've got down to the nitty gritty. I'm hoping that you can bring some of that knowledge to bear - such as nice things to include for {$SetPEFlags} and {$SetPEOptFlags} (Delphi 6+) that could have a bearing on some of the things that the compiler doesn't have options for (unfortunately).

Also, I seem to recall from a gamedev thread that you have access to the code for the RTL (i.e., a better Delphi than Personal Edition), though I could be wrong there. (The thread where somebody asked for the source to System.pas, I think, is the one from which I got that impression.) I only have the Delphi 5 version, which is no use for D6 (I simply couldn't get the damned thing to accept the new .dcu, no matter how many brick walls I hit my head against).

I'm under no illusions that it wouldn't be better to use C++ (w/ Rasmus' 1.5K OpenGL init code at Flipcode) or asm (for obvious reasons). However, that's a moot point to me. I'm wondering how small can Delphi go since it will make my life easier. Also, I'm trying to see if I can match the 4K bump mapping in Turbo Pascal (http://www.pouet.net/prod.php?which=3339) just for personal kicks (there's an article about this in one of the Hugis, IIRC). Of course, I'm not there yet... I may experiment with the Free Pascal Compiler later on, depending on how this goes.

Xorcist
07-11-2002, 10:04 PM
I've tried modifying the compiled .dcu units without luck - if I had the VCL source for D6 then I'd be all set, dammit. Oh well - 5.5K is good enough for now.



Would it be illegal for me to give you the VCL source? Cause I have the Professional Edition, and wouldn't mind wrapping up the whole VCL source directoy and sending it over to you to better the cause. Let me know.

P.S. I have both Delphi 5.0 and 6.0 Professional Editions.

TheLion
08-11-2002, 09:08 AM
About those increased size when strings are being used:

Today I was browsing through my compiler settings and found an option called HUGE STRINGS, now I don't have a lot of experience with compiler setting/directives, but did you have this turned on or off during testing???

Alimonster
08-11-2002, 09:41 AM
*Slaps forehead*

That could make a difference there, of course! ShortStrings don't require memory allocation in the same way as big 'uns (AnsiStrings). I'd imagine that would make a difference. Well spotted! ShortStrings are simply 256-char long strings with the size stored in array position [0] of the string (which you can't directly access).

I'll try that later on today!

Mrwb
29-03-2004, 10:13 PM
Hey, stumbled accross this thread earlier today, and thought i'd have a go in size optimizing. I took the code from example 2.1a at sulaco, stripped it, (got rid of "useless" stuff like error messaging ;)) stripped it some more, and then stripped it even more. The exe was still to big. When packed with upx, it was 13-15kb or something. Then, I downloaded a modified system.pas file, and voilA , the exe was down to 12kb unpacked. After lots and lots of "optimizing", I managed to get the code down to 9kb unpacked (9 216bytes to be excact, with three rotating cubes, two wireframed and one solid).
After deleting the RCData package info and overwriting the resource dvclal with an empty file using ResourceHacker, the file lost a couple of bytes. I then downloaded a program called Realign which realigns Windows EXE files and deletes the DOS stub, making the file more compression friendly. I also used FakeCom.com which is an a small program, "which extracts data contained at the end of file and runs it". If you upx' the file after these operations, it will compress the whole file. (otherwise it won't tuch the exe headers)

The result: a 3 609 byte Delphi OpenGL program which displays three rotating cubes; two wireframed and one solid.

Im sure it can be optimized even further. ;)

BTW: I also made a version with a simple noise texture generator (from the thread at the OpenGL forums), which resulted in an 4 462 byte app.

Inspiration gathered from http://scene.migeel.sk/intro4k.html

WILL
30-03-2004, 10:45 AM
Oooww... very cool. :)

Alimonster
01-04-2004, 09:24 AM
I found out the trick to compiling the RTL units - you need to pass the compiler (dcc32) the "-Y" (capitalised Y) flag before it lets you recompile those units. It would be a good plan to do backups first.

Nice work on finding that article, dude. Was it not in a recent Hugi? It looks familiar...

WILL
01-04-2004, 09:53 AM
I think we should have a small oldskool demo contest one of these days soon. :D


And yes Ali, you have to enter! ;)

Alimonster
01-04-2004, 10:21 AM
I think we should have a small oldskool demo contest one of these days soon. :D


And yes Ali, you have to enter! ;)
Sounds like fun! I'm overdosing on UT2003 and UT2004 at the moment (great games) but I'm sure I'll tire of them soon.

Mrwb
01-04-2004, 05:29 PM
Nice work on finding that article, dude. Was it not in a recent Hugi? It looks familiar...

Yeah, it apeared in the latest hugi I belive. I had forgotten all about it though, until I found this thread.. ;)