PDA

View Full Version : Initialization sections not being run in Linux so.



technomage
31-12-2006, 11:10 AM
I was just wondering if anyone had come across this. I've been testing a new engine I'm writing under Linux and have encoundered a few problems.

The core os the system is a core shared object which sets up the shared memory manager. To accomplish sharing the memory manager (and creating the log files etc) I have a bunch of code in the initialization section of various units e.g



initialization

GetMemoryManager(MemManager);

end.



Now I was getting a weird access violation so I put in a whole bunch of Writeln statements in the initialization sections to check they were being called. And guess what, they weren't :!:

This appears to be different behavior from Free Pascal under windows at the engine works fine in windows under Delphi 5 and 2006 and Free Pascal 2.0.2.

Anyone come across this before :?:

I could work around this but my current design relys on these initialization sections being called.

BTW Initialization sections are called in exe's under linux as the Logger class I am using is a modified one from the JEDI-SDL packed which we know works under Windows, Linux and OSX.

technomage
31-12-2006, 11:45 AM
I have just confirmed this problem in a second DLL/SO. I this module I was generating a CRC32 data table in the initialization section, all crc checks were failing, once this call was moved to a point where it would be called it worked OK.

:?

JernejL
31-12-2006, 12:47 PM
did you use sharemem?

technomage
31-12-2006, 12:59 PM
I use my own shared memory manager dll. This has nothing to do with the shared memory system like i mentioned this works fine under free pascal and delphi under windows. It appears that the code in the initialization sections is just not called under linux when loading a .so.

grudzio
31-12-2006, 01:12 PM
This is from Free Pascal Unix FAQ


Dynamic libraries

These operating systems do support shared libraries (also called dynamic link libraries), Free Pascal currently does not emit position independant code (PIC), as required for the creation of shared libraries.

Therefore, even though the linux compiler target permits creating shared libraries, the usage of that shared library may result in undefined behavior, especially if accessing global variables in the library. Creation of shared libraries is not recommended with the current version of the compiler.

Importing code from shared libraries does work as expected though, since it does not require usage of position independant code.

You have to wait for 2.2 version I suppose.

technomage
02-01-2007, 06:59 PM
OK some more testing has confirmed that neither initialization/finalization or standard



begin
// put code here
end.


works in a shared object under linux. initialization areas are run if you compile an application but not shared objects. I realise the shared library is not recommended in the current version, but there must be a way around this.

In the old versions of pascal there were hooks like dllload and dllunload you could link into to detect when dll's were loaded, surely there must be something similar in unix based operating systems.

Anyone got any ideas :?:

WILL
02-01-2007, 08:52 PM
Last time I checked through my flooded inbox (still have to send a note about Mantis :P) there was quite a bit of hum about Linux and libraries and such under the FPC reporting.

I'd go right to the barn on this one. Easiest way to get in touch with the FPC people is to hop on IRC on freenet or leave an interesting post on the FPC message boards. I usually get in contact with Florian or Vincent (fpcfan) through irc best. Almindor is a regular though and he's usually up to par with all things FPC and Laz...

Almindor
02-01-2007, 09:43 PM
Not exactly sure on this particular issue but if it's just missing PIC support then you can get it working with using latest 2.1.1 and using the "-Cg" switch which will enable PIC code generation.

Didn't test this so you might be first :)

technomage
03-01-2007, 12:11 AM
Not exactly sure on this particular issue but if it's just missing PIC support then you can get it working with using latest 2.1.1 and using the "-Cg" switch which will enable PIC code generation.

Didn't test this so you might be first :)

Thanks for the advice :D

OK, I'll get the 2.1.1 source tree down and see if it works. I havne't compiled free pascal from scratch before, so it will be interesting. but at least we'll know if it solves the problem :D

technomage
04-01-2007, 11:51 AM
Well I tried the 2.1.1 branch. I got the source from the daily snapshots.

the -Cg switch generates an error when compiling. Without the switch is compiles OK, but behaves the same way as 2.0.4..

back to the drawing board then...

I'll post this on the Free Pascal community section.

technomage
07-01-2007, 12:38 PM
After much investigation and head scratching, I believe that what I want to do with free pascal on linux is just not possible (yet).

Having a seperate memory manager in a .so works fine when you just want to use it from an application, but as soon as you want to use it in other .so's you start to hit problems. What I am doing is a little more difficult becasue I am trying to export classes from the .so's to not only the main application but also to the other .so's.

For example

I have the libcore.so which is my memory manager and logger.

I then have libkernel.so which contains all the platform specific stuff in my case this exports a class which create an SDL window and handles loading images and all the vector maths.

I then has libglrenderer.so which uses the exported window class to get information on the window and actually handles drawing stuff in opengl.

Then there is the libscene.so which is scene management and uses the libglrenderer and the libkernel.

All of this is tied together by the main application.

but it seems that under linux this just won't work. So...where to go from here :?: Abandon linux and MacOSX support? Switch to Kylix (no longer supported and I don't know if it can do what I want either). completely rework the engine so it does not use classes.

frankly I don't know.

Setharian
07-01-2007, 03:06 PM
other than manually writing assembly (and thus binding it to a specific CPU), there's no way out...well and of course not using any global variables at all....

technomage
07-01-2007, 03:40 PM
I found out the GCC gets round this by exporting

void __attribute__ ((constructor)) my_init(void)
void __attribute__ ((destructor)) my_fini(void)

form the library, these get called before the dlopen and dlclose functions return to the main application. This would be a perfect place to call the shared memory manager.

I don't know if there is any way to support those attributes under free pascal though :?: :?:

dmantione
07-01-2007, 05:34 PM
We need to find out what those attributes do to the assembler file. Perhaps I can modify the assembler then to call the initialization of the library.;

Setharian
07-01-2007, 08:00 PM
I guess GCC under linux puts the code into the .init/.fini sections in an ELF executable and under windows adds either hidden code into DllMain or it invokes it even before DllMain (somewhere near the entry point) and reacts to PROCESS_ATTACH/PROCESS_DETACH reasons passed as one of the arguments.....but it's a pure guess....

technomage
07-01-2007, 08:00 PM
any ideas on how to find out? this is getting a bit out of my experience zone :?

dmantione
07-01-2007, 08:26 PM
objdump is the tool... Please try to dump symbols, headers, sections. I'll try to do some investigations myself as well.

Setharian
08-01-2007, 06:40 AM
nonetheless it won't help you still....there won't be PIC code, no matter how you substitute the mechanizm :?

even the RTL contains global vars and it uses them....and without PIC the application tries to access the absolute addresses (causing an AV)...under Win all is fine but under Unix-based systems all shared libraries can be relocated at any location in the memory, requring it to access global symbols by relative addresses, offsets from the GOT (Global Offset Table) which is passed by the loader I presume.....under Win, libraries have their imagebase, which is the preffered address at which the dll should be loaded....if it collides (the address is already occupied), the benefits of a dll is lost, all dll code is made private to the process (increasing the memory usage), reallocated at a different address and then loader by using the relocations table patches all the global offsets to reflect the new value...this is one of the reasons why certain programs take quite some time to load if they link with too many dlls which have conflicting image bases...under Unix-based systems there is no rellocation table and the loader does not patch addresses, that's why it requires the code to be PIC-safe and work without relying on its position in the memory....

maybe a bit too much information, but I hope it explains you how it works....

dmantione
08-01-2007, 08:23 AM
This is no problem, you can recompile the rtl with PIC. It does work in Linux/i386 without PIC as far as I know, but there is no sharing of memory then.

technomage
08-01-2007, 07:32 PM
you don't have to flag a library as shared, I think it's optional under gcc, infact I seem to remember reading somewhere that the init and fini stuff does not work well when compiled shared, i could be wrong though.

I'll try some tests under GCC and see if I can get a dump of the code I'll post it over on this (http://community.freepascal.org:10000/bboards/message?message_id=242058&forum_id=24083#242174)
thread on the free pascsal community site.

technomage
08-01-2007, 11:08 PM
OK, I compiled the following code into a shared object



void __attribute__ ((constructor)) lib_init(void)
{
printf("lib_init called");
}

void __attribute__ ((destructor)) lib_quit(void)
{
printf("lib_quit called");
}

void printUPPERCASE( inLine )
char inLine[];
{
char Upstring[256];
char *inptr, *outptr;

inptr = inLine;
outptr = Upstring;
while (*inptr != '\0')
*outptr++ = toupper(*inptr++);
*outptr++ = '\0';
printf(Upstring);
return;
}



I used

gcc -c -fpic test.c

then

gcc -shared -lc -o test.so test.o

and ran

objdump -t -x test.so

Here are the results. after doing a bit of reading it seems the ((constructor)) style stuff ends up in the .ctors section and the ((destructor)) ends up in the .dtors



test.so: file format elf32-i386
test.so
architecture: i386, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x000004f0

Program Header:
LOAD off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**12
filesz 0x00000710 memsz 0x00000710 flags r-x
LOAD off 0x00000710 vaddr 0x00001710 paddr 0x00001710 align 2**12
filesz 0x0000010c memsz 0x00000110 flags rw-
DYNAMIC off 0x0000072c vaddr 0x0000172c paddr 0x0000172c align 2**2
filesz 0x000000c0 memsz 0x000000c0 flags rw-
STACK off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
filesz 0x00000000 memsz 0x00000000 flags rw-
PAX_FLAGS off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
filesz 0x00000000 memsz 0x00000000 flags --- 2800

Dynamic Section:
NEEDED libc.so.6
INIT 0x480
FINI 0x6d0
HASH 0xd4
STRTAB 0x314
SYMTAB 0x184
STRSZ 0xaa
SYMENT 0x10
PLTGOT 0x17f8
PLTRELSZ 0x20
PLTREL 0x11
JMPREL 0x458
REL 0x420
RELSZ 0x38
RELENT 0x8
VERNEED 0x3f0
VERNEEDNUM 0x1
VERSYM 0x3be
RELCOUNT 0x2

Version References:
required from libc.so.6:
0x09691f73 0x00 03 GLIBC_2.1.3
0x0d696910 0x00 02 GLIBC_2.0

Sections:
Idx Name Size VMA LMA File off Algn
0 .hash 000000b0 000000d4 000000d4 000000d4 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
1 .dynsym 00000190 00000184 00000184 00000184 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
2 .dynstr 000000aa 00000314 00000314 00000314 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .gnu.version 00000032 000003be 000003be 000003be 2**1
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .gnu.version_r 00000030 000003f0 000003f0 000003f0 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .rel.dyn 00000038 00000420 00000420 00000420 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
6 .rel.plt 00000020 00000458 00000458 00000458 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
7 .init 00000017 00000480 00000480 00000480 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
8 .plt 00000050 00000498 00000498 00000498 2**2
CONTENTS, ALLOC, LOAD, READONLY, CODE
9 .text 000001dc 000004f0 000004f0 000004f0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
10 .fini 0000001b 000006d0 000006d0 000006d0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
11 .rodata 00000020 000006eb 000006eb 000006eb 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
12 .eh_frame 00000004 0000070c 0000070c 0000070c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
13 .ctors 0000000c 00001710 00001710 00000710 2**2
CONTENTS, ALLOC, LOAD, DATA
14 .dtors 0000000c 0000171c 0000171c 0000071c 2**2
CONTENTS, ALLOC, LOAD, DATA
15 .jcr 00000004 00001728 00001728 00000728 2**2
CONTENTS, ALLOC, LOAD, DATA
16 .dynamic 000000c0 0000172c 0000172c 0000072c 2**2
CONTENTS, ALLOC, LOAD, DATA
17 .got 0000000c 000017ec 000017ec 000007ec 2**2
CONTENTS, ALLOC, LOAD, DATA
18 .got.plt 0000001c 000017f8 000017f8 000007f8 2**2
CONTENTS, ALLOC, LOAD, DATA
19 .data 00000008 00001814 00001814 00000814 2**2
CONTENTS, ALLOC, LOAD, DATA
20 .bss 00000004 0000181c 0000181c 0000081c 2**2
ALLOC
21 .comment 00000136 00000000 00000000 0000081c 2**0
CONTENTS, READONLY
22 .debug_aranges 00000068 00000000 00000000 00000958 2**3
CONTENTS, READONLY, DEBUGGING
23 .debug_info 00000164 00000000 00000000 000009c0 2**0
CONTENTS, READONLY, DEBUGGING
24 .debug_abbrev 00000020 00000000 00000000 00000b24 2**0
CONTENTS, READONLY, DEBUGGING
25 .debug_line 00000179 00000000 00000000 00000b44 2**0
CONTENTS, READONLY, DEBUGGING
SYMBOL TABLE:
000000d4 l d .hash 00000000 .hash
00000184 l d .dynsym 00000000 .dynsym
00000314 l d .dynstr 00000000 .dynstr
000003be l d .gnu.version 00000000 .gnu.version
000003f0 l d .gnu.version_r 00000000 .gnu.version_r
00000420 l d .rel.dyn 00000000 .rel.dyn
00000458 l d .rel.plt 00000000 .rel.plt
00000480 l d .init 00000000 .init
00000498 l d .plt 00000000 .plt
000004f0 l d .text 00000000 .text
000006d0 l d .fini 00000000 .fini
000006eb l d .rodata 00000000 .rodata
0000070c l d .eh_frame 00000000 .eh_frame
00001710 l d .ctors 00000000 .ctors
0000171c l d .dtors 00000000 .dtors
00001728 l d .jcr 00000000 .jcr
0000172c l d .dynamic 00000000 .dynamic
000017ec l d .got 00000000 .got
000017f8 l d .got.plt 00000000 .got.plt
00001814 l d .data 00000000 .data
0000181c l d .bss 00000000 .bss
00000000 l d .comment 00000000 .comment
00000000 l d .debug_aranges 00000000 .debug_aranges
00000000 l d .debug_info 00000000 .debug_info
00000000 l d .debug_abbrev 00000000 .debug_abbrev
00000000 l d .debug_line 00000000 .debug_line
00000000 l d *ABS* 00000000 .shstrtab
00000000 l d *ABS* 00000000 .symtab
00000000 l d *ABS* 00000000 .strtab
00000000 l df *ABS* 00000000 initfini.c
00000000 l df *ABS* 00000000 /var/tmp/portage/glibc-2.3.6-r3/work/build-default-i386-pc-linux-gnu-linuxthreads/csu/crti.S
000004f0 l F .text 00000000 call_gmon_start
00000000 l df *ABS* 00000000 crtstuff.c
00001710 l O .ctors 00000000 __CTOR_LIST__
0000171c l O .dtors 00000000 __DTOR_LIST__
00001728 l O .jcr 00000000 __JCR_LIST__
00001818 l O .data 00000000 p.0
0000181c l O .bss 00000001 completed.1
00000518 l F .text 00000000 __do_global_dtors_aux
00000574 l F .text 00000000 frame_dummy
00000000 l df *ABS* 00000000 crtstuff.c
00001718 l O .ctors 00000000 __CTOR_END__
00001724 l O .dtors 00000000 __DTOR_END__
0000070c l O .eh_frame 00000000 __FRAME_END__
00001728 l O .jcr 00000000 __JCR_END__
0000069c l F .text 00000000 __do_global_ctors_aux
00000000 l df *ABS* 00000000 initfini.c
00000000 l df *ABS* 00000000 /var/tmp/portage/glibc-2.3.6-r3/work/build-default-i386-pc-linux-gnu-linuxthreads/csu/crtn.S
00000000 l df *ABS* 00000000 test.c
00001814 l O .data 00000000 .hidden __dso_handle
00000512 l F .text 00000000 .hidden __i686.get_pc_thunk.bx
000017f8 l O *ABS* 00000000 .hidden _GLOBAL_OFFSET_TABLE_
0000172c g O *ABS* 00000000 _DYNAMIC
00000608 g F .text 00000092 printUPPERCASE
00000480 g F .init 00000000 _init
000005de g F .text 0000002a lib_quit
0000181c g *ABS* 00000000 __bss_start
00000000 F *UND* 00000062 toupper@@GLIBC_2.0
00000000 F *UND* 00000036 printf@@GLIBC_2.0
000006d0 g F .fini 00000000 _fini
00000000 w F *UND* 0000008c __cxa_finalize@@GLIBC_2.1.3
0000181c g *ABS* 00000000 _edata
00001820 g *ABS* 00000000 _end
000005b4 g F .text 0000002a lib_init
00000000 w *UND* 00000000 _Jv_RegisterClasses
00000000 w *UND* 00000000 __gmon_start__

technomage
14-01-2007, 09:17 PM
Thanks to the help of the dev team at Free Pascal, the issues regarding initialization sections being run in library's under Linux has been resolved in the 2.1.1 version of Free Pascal.

A big thank you for the help over the last few days to all involved.

The good news is that ,apart form a few issues my InfinitEnigine framework is now running under Windows (Opengl/DirectX) and Linux (Opengl). for more info check out the post of the InfiniteSpace-Online (http://www.infinitespace-online.net) forum here (http://forum.infinitespace-online.net/viewtopic.php?p=30#30).

The engine makes use of a shared memory manager , and exports classes for the host application to use rather than a function API. the code is identical between windows and linux , delphi and free pascal. So good job to all the people at free pascal :D.

savage
15-01-2007, 10:52 AM
Excellent news! Can't wait to see it working on MacOS X :).