I think al that is needed on how to modify free pascal for the gba is written here: http://linux.gbadev.org/gcc-gba/ .

The important part is crt0.s as this defines the gba (cartridge/executable) header. But even more improtant is the lnkscript that is used. As that glues together the crt0.o with your applicton .o file in a special way. Without that linked in to the executable nothing is going to work.

For direct memory access on the gba i believe pointername := pointer(memoryaddress) is possible. So using pure pascal code it should be possible to change graphics mode and write to video memory etc.

Thumb assembly is the least of the concerns to get things working, more important is pointing a pointer to a specific memory address as written above. PS i believe it possible (even on a gba) to switch between thumb and arm assembly...