Loading DX Textures in an imagelist - which is the fastest?

[Firlefanz]

Hello!

I am using the Omega headers and I am also a member there.
This is a problem of Omega, but everybody with knowledge in DirectX could help, so I post it here, maybe someone has a good Idea.

Omega has his own OmegaImagelists (like in DelphiX) which can store images. My problem is I have 19 mb of huge imagelists and it takes a long time to load them all, too long for me.

A solution would be having all images standalone on the harddisc, but I don't like that solution, 600+ images.

The problem is the Init method takes too long, and I am sure there can be an easier and faster solution.

How it works (in my understanding)

1) Loadfromfile loads the imagelist in a stream.
2) Init calls no 3 for every item
3) Item.Init saves the TPicture to a stream, then creates the texture needed from the new stream.

This one is called first to load the imagelist.

procedure TImageListCollection.LoadFromFile(const Filename: string);
var
  Stream: TMemoryStream;
begin
  Stream := TMemoryStream.Create;
  Stream.LoadFromFile(Filename);
  self.LoadFromStream(Stream);
  Stream.Free;
end;

Then the Init is called.

procedure TOmegaImageList.Init;
var
  i: integer;
begin
  if FOmegaScreen = nil then
    exit;
  FRenderer.OmegaScreen := FOmegaScreen;
  FRenderer.Init;
  if FOmegaScreen.Device = nil then
    exit;
  for i := 0 to ImageList.Count - 1 do
    ImageList.Items[i].Init;
end;

This one is calling the init of each ImagelistItem then.

procedure TImageListItem.Init;
var
  FD3DSurfaceDesc: TD3DSurface_DESC;
  Stream: TMemoryStream;
  sRealFilePath: string;
begin
  //Load picture from filepath if present
  FFilePath := Trim(FFilePath);
  if Length(FFilepath) > 0 then
  begin
    //Make relative paths absolute
    if (Pos(':', FFilepath) = 0) and (Pos('\\', FFilepath) = 0) then
    begin
      ChDir(ExtractFilePath(Application.ExeName));
      sRealFilepath := ExpandFileName(FFilepath);
    end
    else
      sRealFilepath := FFilepath;
    if FileExists(sRealFilePath) then
      FPicture.LoadFromFile(sRealFilepath);
  end;

  if (FPicture.Graphic = nil) then
    exit;

  FDevice :=
    TOmegaImageList(TImageListCollection(Collection).GetOwner).FOmegaScreen.Device;
  FRenderer :=
    TOmegaImageList(TImageListCollection(Collection).GetOwner).FRenderer;
  {
  If (FDevice = nil) then
  begin
   MessageBox(0, pChar('Error: no device assigned!'), 'Error', MB_ICONERROR);
   Application.Terminate;
  end;
  If (FRenderer = nil) then
  begin
   MessageBox(0, pChar('Error: no renderer assigned!'), 'Error', MB_ICONERROR);
   Application.Terminate;
  end;
  }
  Stream := TMemoryStream.Create;
  try
    FPicture.Graphic.SaveToStream(Stream);
    if FTileHeight <= 0 then
      FTileHeight &#58;= FPicture.Graphic.Height;
    if FTileWidth <= 0 then
      FTileWidth &#58;= FPicture.Graphic.Width;
    if not assigned&#40;self.FImage&#41; then
      if Self.Transparent = true then
      begin
        if failed&#40;D3DXCreateTextureFromFileInMemoryEx&#40;
          FDevice, Stream.Memory, Stream.Size,
          0, 0, 0, 0, D3DFMT_UNKNOWN, D3DPOOL_MANAGED,
          D3DX_FILTER_NONE, D3DX_FILTER_NONE,
          $FF000000 or Cardinal&#40;TColor&#40;OmegaColor&#40;self.TransparentColor, 0&#41;&#41;&#41;,
          //Self.TransparentColor,
          nil, nil, Self.FImage&#41;&#41; then
        begin
          MessageBox&#40;0, pChar&#40;'Error creating texture ' + Self.FName&#41;, 'Error',
            MB_ICONERROR&#41;;
          Stream.Free;
          Application.Terminate;
        end;
      end
      else
      begin
        if failed&#40;D3DXCreateTextureFromFileInMemoryEx&#40;
          FDevice, Stream.Memory, Stream.Size,
          0, 0, 0, 0, D3DFMT_UNKNOWN, D3DPOOL_MANAGED,
          D3DX_FILTER_NONE, D3DX_FILTER_NONE,
          0, nil, nil, Self.FImage&#41;&#41; then
        begin
          MessageBox&#40;0, pChar&#40;'Error creating texture ' + Self.FName&#41;, 'Error',
            MB_ICONERROR&#41;;
          Stream.Free;
          Application.Terminate;
        end;
      end;
    Self.FImage.GetLevelDesc&#40;0, FD3DSurfaceDesc&#41;;
    FRealWidth &#58;= FD3DSurfaceDesc.Width;
    FRealHeight &#58;= FD3DSurfaceDesc.Height;
    if FRealWidth < 2 then
      FRealWidth &#58;= 2;
    if FRealHeight < 2 then
      FRealHeight &#58;= 2;
    FHeight &#58;= FPicture.Height;
    if FHeight = 0 then
      FHeight &#58;= 1;
    FWidth &#58;= FPicture.Width;
    if FWidth = 0 then
      FWidth &#58;= 1;
    FNumOfRows &#58;= FHeight div FTileHeight;
    FNumOfColumns &#58;= FWidth div FTileWidth;
  finally
    Stream.Free;
    if Length&#40;FFilePath&#41; > 0 then
    begin
      FPicture.Free;
      FPicture &#58;= TPicture.Create;
    end;
  end;
end;

So isn't it possible to make a new methode called 'LoadandInit' that loads all and makes the textures and works somehow faster, maybe without that TPicture and Savetostream etc?

How can I make it faster?

Thanks,
Firle[/code]

[JSoftware]

using a momorystream wouldn't that force the file to be loaded into memory before reading from it? why not use filestream? it doesn't load the file into memory

[Sly]

Just reading through that code, I see many areas for improvement.

Try changing TImageListCollection.LoadFromFile to use a TFileStream instead of a TMemoryStream. This will prevent the initial hit of allocating 19MB of memory and loading the entire file into it.

procedure TImageListCollection.LoadFromFile(const Filename: string);
var
  Stream: TFileStream;
begin
  Stream := TFileStream.Create(Filename, fmOpenRead or fmShareDenyNone;
  try
    LoadFromStream(Stream);
  finally
    Stream.Free;
  end;
end;

Not much can be done about TOmegaImageList.Init.

TImageListItem.Init could be improved in several places.

- FFilePath should have been trimmed before it was saved. Therefore there should be no need to always trim FFilePath every time you enter this method. Move the call to Trim to inside the if statement. But that's only a tiny problem.

- For every item in the list, you are creating a TMemoryStream and saving the image back to it so that D3DX can load that very same picture back in again. Very inefficient. That's where most of your time is being taken up.

If you don't want 600+ files on your harddisk, then this imagelist is not a very efficient way to avoid that. A better solution would be to use a system similar to a PAK file where multiple files are lumped together into one big file. Manually create each instance of TImageListItem and set the Filename property. Then you just create a TMemoryStream, load the file from the PAK file into the stream, pass that to D3DX, free the stream. Still not the most efficient method, but it should be more efficient than the current method.

I could possibly create a proof of concept for this, but I won't be able to do that for a couple of days.

[Anonymous]

Hello Sly,

thanks a lot for your suggestions. The above code does not work, sorry. It says 'UngA¼ltiges Stream Format', translated 'Not working stream format' or something.

I know this is not very efficient, this is why I want to change it but myself I don't know how.

Thanks for your suggestions,

Firle

[Sly]

I'll download the latest Omega tonight and see if I can get something working.

[Firlefanz]

Ooops, that Guest was me.

The delphisanctuary site is still down. You can download the package from my site:

DX8 package two versions in the first comment of me, DX9 package (which I use) is a few more statements under it.

http://www.ericbehme.de/phpBB2/viewtopic.php?t=132

Thanks a lot for your help, I hope you find something, that would be great!

Firle

[JSoftware]

or maybe even a hashtable for the images?

[Firlefanz]

Hello,

the problem is I want to keep the Imagelist-Unit because it has a lot of other methods I still need.

So maybe a new LoadandInit Method would help, and then a method to save exisitng lists to the new format, but the Imagelist itself must work like it did before. Other units like OmegaSprite etc are depending on it, so we still need the format like it is now.

Never tried a hashtable myself

Firle

[Sly]

I don't think a hash-table would help in this case because it is not searching that is taking the time. It is the loading, saving, loading of each texture during the init that is taking all of the time.

[Firlefanz]

I agree. The init method which is calling the Item.Init, that then saves Picture to stream and load texture from stream takes 90% of the time.

Firle