JUI - Another GUI

[phibermon]

Hi peeps, bit of an update for you, I've now started work on the visual side of the window/widget manager JUI. This is a re-design/re-write of my old VFA Window manager.

I've included screen shots of both the old VFA (craft2y.jpg) and the new JUI (CC_EARLY.JPG) so you can see where it currently stands in terms of matching the features of VFA.

VFA was a single threaded, immediate mode GUI with no message sub system (everything was triggered in events filtering down through the windows/widgets). This worked but was quite slow and the design was around 20 years out of date.

JUI is a multi-threaded, soon to be non-immediate, message driven window manager. Every single window and widget runs in it's own thread, able to deploy messages directly to any other component, including it's parent, in a lock free manner.

Because each widget runs in it's own thread as opposed to only a window having a thread in conventional desktop OS's, it allows for a nice trick: E.g. you can write some code that looped to infinity and place it in the onclick event of a button. The button will lock but every other widget and window is still functional.

The only issue with this design is thread overhead. if you have 100 widgets, you have 100 threads. This can become a problem as you move to several hundred widgets due to the memory allocation of each thread within the OS, basically your limited to around 2000 threads under posix systems, around 600 on Windows systems.

Here is an example unit for creating a window, widgets assigning callbacks etc

(Initialization of the Window manager and various other supporting functions is trivial so I won't display here)

unit cc_guimain;

{$mode objfpc}{$H+}
{$i jui.inc}
{$i pin.inc}
{$i cc.inc}

interface

uses
  Classes, sysutils, sdl, u_gl,

  j2d, j2d_poly, j2d_polygroup, j2d_tspline,

  jui_const, jui_log,  jui_screen, jui_textureman, jui_font,
  jui_mouse, jui_cursor,

  jui_global, jui_message, jui_messenger, jui_keyboard,
  jui_lockable, jui_theme, jui_element, jui_window,
  jui_windowman, jui_widget, jui_basetypes, jui_titlebar, jui_button,
  jui_colourwheel, jui_editbox, jui_listview, jui_scrollbar,

  pin_log, pin_lockable, pin_message, pin_messenger, pin_tcp,

  cc_global, cc_gamestate;


type
    TCCGUIMain = class
    private
          WindowMan : TJUIWindowMan;
    public
          Window : TJUIWindow;
          TitleBar : TJUITitleBar;
          StartServerButton : TJUIButton;
          StartClientButton : TJUIButton;
          EditBox : TJUIEditBox;
          ListView : TJUIListView;

          constructor create(AWindowMan : TJUIWindowMan);
          destructor destroy;override;
          procedure StartServerClick(ButtonMessage : TMsgMouseButton);
          procedure StartClientClick(ButtonMessage : TMsgMouseButton);


    end;


implementation


constructor TCCGUIMain.create(AWindowMan : TJUIWindowMan);
var
   TempButton : TJUIButton;
begin
     WindowMan := AWindowMan;
     
     Window := WindowMan.CreateWindow;
     Window.LocalRect := J2DRect(10,10,360,360);
     Window.Caption := 'GUIMain - Testing Widgets';
     TJUIWidget(TitleBar) := Window.CreateWidget(TJUITitleBar);
     TitleBar.LocalRect := J2DRect(0,0,window.width,19);
     Titlebar.Caption := 'GUIMain';

     TJUIWidget(StartServerButton) := Window.CreateWidget(TJUIButton);
     StartServerButton.LocalRect := J2DRect(5,25,80,45);
     StartServerButton.Caption := 'Start Server';
     StartServerButton.OnMouseClick := @StartServerClick;

     TJUIWidget(StartClientButton) := Window.CreateWidget(TJUIButton);
     StartClientButton.LocalRect := J2DRect(95,25,170,45);
     StartClientButton.Caption := 'Start Client';
     StartClientButton.OnMouseClick := @StartClientClick;

     TJUIWidget(EditBox) := Window.CreateWidget(TJUIEditBox);
     EditBox.LocalRect := J2DRect(180,25,290,41);


     TJUIWidget(ListView) := Window.CreateWidget(TJUIListView);
     ListView.LocalRect := J2DRect(5,55,290,305);
     ListView.Anchor.Right := true;
     listview.anchor.bottom := true;

     listview.Columns[0].Caption := 'I love';
     listview.AddColumn('Fish');
     listview.AddColumn('Fingers');


     listview.AddItem('It''s all good');
     listview.Items[ListView.NumItems-1].SubItems[0].caption := 'This Is A SubItem';


end;

destructor TCCGUIMain.destroy;
begin
     inherited;
end;

procedure TCCGUIMain.StartServerClick(ButtonMessage : TMsgMouseButton);
begin
     if assigned(CCGlobal.PINServer) then
     begin
          CCGlobal.PINServer.OutBoundRelay := nil;
          freeandnil(CCGlobal.ServerState);
          freeandnil(CCGlobal.PINServer);

     end else
     begin
          CCGlobal.PINServer := TPINServer.Create('0.0.0.0',6969);
          CCGlobal.ServerState := TCCServerState.create;

          CCGlobal.ServerState.OutBoundRelay := CCGlobal.PINServer;
          CCGlobal.PINServer.OutBoundRelay := CCGlobal.ServerState;

          CCGlobal.PINServer.Listen;
     end;

end;

procedure TCCGUIMain.StartClientClick(ButtonMessage : TMsgMouseButton);
begin
     if assigned(CCGlobal.PINClient) then
     begin
          CCGlobal.PINClient.OutBoundRelay := nil;
          freeandnil(CCGlobal.ClientState);
          freeandnil(CCGlobal.PINClient);
     end else
     begin
          CCGlobal.PINClient := TPINClient.Create('192.168.0.1',6969);
          CCGlobal.ClientState := TCCClientState.create;

          CCGlobal.ClientState.OutBoundRelay := CCGlobal.PINClient;
          CCGlobal.PINClient.OutBoundRelay := CCGlobal.ClientState;

          CCGlobal.PINClient.Connect;
     end;
end;

end.

So yes, I shall be releasing this at some point but I'm looking for another developer to help me with the workload. There's still a number of widgets I need to re-code and I may need help on moving to a faster VBO rendering method (need new font units, is GLViewPort depreciated? EDIT : GLViewPort is *not* depreciated )

The old VFA is usable in much the same manner, but the design stinks. if you really want it, let me know.

and on a side note I've nearly finished PIN and I'm going to wrap up my hardware skinning lib from P3D, in my opinion the most modern skinning implementation in delphi (EDIT:FreePascal/Lazarus for me personally, all the way ) that I've seen so far, hence why I want to give it to the community.

Peace.

[chronozphere]

Impressive. I don't think this has been done before.

Is it easy to use this for different projects, in terms of combining it with an existing engine?
It also reminds me that I have to get into threading someday. That's still on my list.

Looking forward to the release.

[phibermon]

I'm designing it specifically with other coders in mind. At the core of the system is the messenger object and message types, each messenger object is a thread that has a mutex protected object queue and functions for popping and pushing messages. The basic rule is that as messages move around, they are only within one messenger at a time so they don't have to be thread safe (which is good for performance). The thread as it spins around, checks the queue for messages, calls a virtual; abstract; function (domessage) for handling the message which returns if the message was passed on or not. if not, the message is destroyed.

Most objects in the UI are sub-classes of this messenger object, the implemented domessage for each sub-class handles whatever logic you like for any message type you like, in JUI's case the messages are mouse movement, window close requests, keyboard etc

this means that every thread can handle it's local messages without blocking the other threads, and only rarely blocking when there is simultaneous domessage popping and external pushing.

At the moment the main application thread, by means of another thread has a locked-frame rate. it iterates through the tree of window manager, windows, widgets etc locking that element, calling render then unlocking the element.

This is a problem, it needs to lock as other threads may very well manipulate the values used for rendering (position, colour theme etc). So with a high volume of elements, the rendering thread will have to wait on the occasional lock. I've minimized this best I can but unfortuantly very rarely this causes a visually identifiable 'pause' in events (mouse stalls). it's hardly noticable and as I'm running on a slow system (Atom 330), most people wouldn't notice at all. But I'm a perfectionist so I can't leave it like that

I'm going to solve this by improving/rewriting the locking mechanisms in order to decrease/eliminate the potential for this to happen.

[phibermon]

oh I forgot to mention, it's tested on linux + windows as well as PIN and the P3D skinning stuff. I'm using SDL for my OS window creation, mouse/keyboard input but there's no reason you can't drop it into any framework that provides these things. The goal when it's all working is to abstract the rendering functions so I can support API's other than OpenGL. However at the moment there is no 'region invalidation' which means when one thing moves, everything needs to be drawn again. This is fine for hardware accelerated game setups as you're redrawing everything anyway, but a software render wouldn't match the performance as it would amount to a lot CPU time that you'd want to use for actual program code. Anyone with any experience with this?

[WILL]

Wow this is quite impressive. Nice work.

How lightweight can this be made? Also what are your plans for fonts and potential font file formats you'll be supporting. A good menuing and interface system can really make the difference for a game's polish and accessibility.

[farcodev]

multithreaded UI is welcome !!
I can't wait to see it released

[phibermon]

Wow this is quite impressive. Nice work.

How lightweight can this be made? Also what are your plans for fonts and potential font file formats you'll be supporting. A good menuing and interface system can really make the difference for a game's polish and accessibility.

Well at the moment I'm using bitmap fonts (generated using one of the many different tools), there is no texture filtering on the font and all alignment is centered in pixels which means you get pixel perfect, non blurry text. However, as long as font lib can return geometry/bitmaps and provide methods for obtaining character widths with kerning (opentype headers anyone?) Then it's good. Is there any existing font lib for pascal that you'd recommend? I know I can easily create outline fonts using the Win32 API and the X API under posix systems however I'd much prefer a cross platform lib that's supported on the 3 main platforms

in terms of 'weight', there's around 150kb of source that you need for a functioning system but with a good smart-linker it works out quite well.

All widgets live in their own units so obviously only linking the ones you use.

[chronozphere]

This sounds really good. I can imagine that you will probably use this for small UI's, because of the thread overhead you get otherwise. This is no problem because most games use simple menu's with only a few widgets.

The threadbased design sounds very elegant. If I understand you right, your widget class has a render method. Before executing it, the render-thread locks the thread of the widget, calls the method and unlocks it again. Getting this to run optimal is quite a challenge. The main problem with rendering is, that it can't be easily separated using a few threads, because you get in trouble with your API calls.

Did you have a look at fontstudio made by Nitrogen? I always use it for my projects. It can create very nice bitmap fonts and the corresponding character information. This data can be packed into different formats, a bitmap + separate XML or a TGA with character data inside. I really like it.

I'd like to see more of this!

[phibermon]

I've got an idea to put an end to the rendering 'pause'. I'll keep two copies of the data needed to draw each widget, one for the latest data (datacurrent) and one for the previous data (dataold). The second copy will only be used if a lock could not be obtained. eg:
#
<render thread>

try to lock window
if successful then draw window using 'datacurrent'
if failed then draw window using 'dataold'
move to next element

when say, the position of a window is updated, 'datacurrent' is locked, the change is written, 'datacurrent is unlocked' then 'dataold' is locked, the change is written, 'dataold' is unlocked.

'dataold' is locked by the render thread before the render thread tries to lock 'datanew'. So this way, if 'datacurrent' is locked then the render thread can draw 'dataold' without fear of the thread updating 'datacurrent' locking 'dataold' before the render thread does (because if we lock 'old' then find 'current' is locked, we know that the render thread gets to use 'dataold' first.

so I believe that this solves the problem by guaranteeing that while there's minor overhead locking/unlocking mutexes, the render thread will never be left in a position where it has to wait on another thread as it will always have access to lock free data.

(there's all kinds of techniques for handling such threadding issues, I'll do some research, perhaps there's a better solution)

All of this will be totally transparent to the end user. They don't even know about the threads, all they need to know is that callbacks can come from multiple threads, so they must lock any shared data on their end. the TJUILockable class is a good choice for this. (user side locking will not effect the window manager as when callbacks are called, the element that called them is unlocked as it does so. element properties (button captions, size, colour etc) are accessed using thread safe properties)

[phibermon]

JUI is now being modified so that threads will become members of elements. then you can enable/disable them at will for any object in the system. When a thread is disabled, the parent element is responsible for pumping the messages.

As fun as it is to play with this interconnected, multi-threaded madness, a real world usage scenario does not call for such mass multi-threading, at least not in a GUI. When desktops have more then 32 cores, then I'd recommend it as viable to overcome performance limitations but until then, only the window manager and windows will run inside threads, widgets message queues will be pumped from windows.

However it's not been a waste of time, not in the slightest. You can thread enable any element or derrived messenger object with a single property change, at run time. letting you choose when and if it's a good idea to use a thread. Nothing will change for the end user at all, other that any code run in a callback function (say from a button) will either block the parent, or not. Also creating a robust, stable system capable of handling such a complex hierarchy of threads and optimizing that system to work as fast as possible means that with widget threads turned off, the system won't break a sweat

Thanks to advice given to me by Carver413, I'm going to be implementing a scripting engine (not sure what I'll use yet) into JUI. At first it will be used to layout windows, widgets, set properties fonts etc but the eventual idea is to provide an interface to JUI within the scripting engine, so whole apps could be written in scripts.

All assignable callbacks and properties will be exposed to both the scripting language and compiled code meaning that, for example, you could set up a window and all the widgets in a script, then allocate callbacks in code. or the other way around, or exclusive to each option.

I think this along with control over the threads will increase the potential set of coders that will find JUI useful.

And that's me done

EDIT : there's no reason you won't be able to disable all threads entirely in this manner. I'll also wrap this up with some {$ifdefs} in the lock unlock routines, so when disabled in this fashion, you can get rid of locking unlocking overheads. All you have to do then is make sure you call the pump messages of the window manager every few frames

FURTHER EDIT : For scripting I shall probably use REM objects pascal-script so people don't have to learn the scripting language. It says it's free but I don't know if that's the same for commercial use, some of you may want a commercial friendly license in JUI, so if anyone has any info about this please let me know.