Abstracting Graphical APIs

[Anton]

Hello everyone,

Recently I've been thinking on how to achieve such thing as creating abstract graphical api for rendering 3d graphics, which on it's side could be implemented using Direct3D11, Direct3D9 or OpenGL3.3. It is very possible that I don't start doing it at all, but I'm still very curious for different and efficient approaches on achieving this.

I guess since most of you here are currently writing on the community game engine, you are faced to similar challenges.

Here are my few thoughts, and I'm very interested is someone with experience on similar things share their advices.

The first question I face is -what would be the level on which the api works (higher or lower). I.e. should I design the API to allow the user to draw primitives in BeginDraw/EndDraw style. Or should I create abstract class for a Meshes or Models and for Textures and Materials, which will be implemented in different ways for the different graphic providers then have a Context object to render Mesh objects? Should the mesh's per-vertex data (position, normals, etc) be predefined in a list of different variants or the user should be able to implement their descendant of the Mesh class.
- Maybe a good approach is to create my abstract api on the level similar to the fixed-function pipelines' of the legacy DirectX and OpenGL versions (but implemented via shaders)?
- For now I think it is a good idea to create abstract Context class, Mesh, Texture, Shader/Material, Model(group of meshes with materials), Camera class (which should provide functionality to easy manipulate the camera and feed the matrices to the shaders).
- Maybe support some kind of Scene Graph, where the user can add hyerarchy of Models, and light source objects

Another question is - What about shaders?
- One solution would be just not to allow the api user to create their own shaders and implement(hardcode) different rendering scenarios (for example, flat shading, blin-phong, blin-phong with bump mapping, shadow projection, and others) - lets say i have an abstract Context object, (lets say TGALContext), which have a predefined enum of scenarios, and I call TGALContext.SetScenario(csBlinPhongNormalMapping) or something like this, which internally would use particular group of shaders to render in that particular way.
- I have read that some engines (like Ogre3D) have some kind of their material meta language, which can be translated either to GLSL or HLSL depending on the graphic provider used.
- Another thing I've read is that I can make a material description file, (maybe XML), which describes the inputs of the shader and the outputs, followed by implementation in both HLSL and GLSL for Vertex and Fragment shaders (in both langagues using the same data types as inputs). So when the API user writes shaders, they should write such Material Description file containing implementation both in GLSL and HLSL.

Have someone really implemented such kind of (3d) graphic abstraction in their Pascal engines? And what, do you think, is the best way to do it?

P.S: Sorry for the bad organization of the post and my not so good english :)

[imcold]

Seems that https://github.com/bkaradzic/bgfx does something close to what your goal is, so you may find some inspiration there perhaps. As for API: provide low level functions and build the high level stuff on it, so it would be optional to use?

[phibermon]

In terms of API level I'd recommend you abstract high rather than for low level functions. In terms of 3D API's it's really only DirectX + OpenGL to choose from right now but you can't garuntee that all future versions of those API's will match previous versions on a fundamental basic level of operation. The changes we see between OpenGL1/2 and 3+, droppping of immediate mode etc are evidence that if you're abstracting an API to be future proof, you want to go as high as possible.

Also ask yourself if you really want to support DirectX at all with Freepascal/Delphi. We're unlikely to see Xbox support any time soon and OpenGL exists on the windows platform too, so you get a fully capable 3D API on all operating systems you can potentially target with your language.

Designing a structure/Abstracting between different versions of OpenGL is far simpler than between DirectX/OpenGL as there's enough stuff that's pretty much the same across all of GL/GLES to make it so.

if you want to target android/ios and desktops then you'll need some form of abstraction between GL/GLES differences.


--

If you are going DirectX + OpenGL then the question of shaders, either an XML file or just a file naming convention to store both HLSL and GLSL shaders is the best approach. The abstracted shader language of the ogre engine is a really nice thing but it took a long time to create, it's very complex and very hard to produce optimal shaders generically.

Best keep things simple. So your users will have to write a GLSL and a HLSL shader, no big deal. Other than a few keywords/functions and the general structure, you can reuse pretty much all of your shader code. They'd have to learn a meta-language if you created one anyway, save yourself tons of work and let them spend another 30 minutes writing shaders.

[Anton]

I have read some articles of other people trying to achieve similar thing, and it seems the lower the level of the API is, the harder it gets to design it. And for a hobby project, it doesn't worth the effort. On the other hand, a higher level API would be much easier to design, but it will seriously limit the potential scenarios it can handle.

So for now I would avoid doing such kind of abstraction layer and stick to either DirectX or OpenGL for each application, or maybe use SDL.


Seems that https://github.com/bkaradzic/bgfx does something close to what your goal is, so you may find some inspiration there perhaps. As for API: provide low level functions and build the high level stuff on it, so it would be optional to use?This looks very interesting, I'll definately take a look at how it is designed.


Also ask yourself if you really want to support DirectX at all with Freepascal/Delphi. We're unlikely to see Xbox support any time soon and OpenGL exists on the windows platform too, so you get a fully capable 3D API on all operating systems you can potentially target with your language.In this case the only platforms which doesn't support OpenGL seems to be XBox and WindowsRT, but I'm not interested in those anyway.

I'm interested to know are there big performance differences between OpenGL and DirectX on Windows, I have read some articles stating the Direct3D would perform better on Windows platforms?

[SilverWarior]

I'm interested to know are there big performance differences between OpenGL and DirectX on Windows, I have read some articles stating the Direct3D would perform better on Windows platforms?

From my expirience that largely depends on graphical drivers. One think that you definitly want to avoid is using graphical drivers that are installed by windows itself.
Always instal official drivers from your graphic card manufactuer becouse drivers that come with windows often don't provide full functionality and in case of some older graphics card they even limit OpenGL to OpenGL 1 compatiblility even thou the graphic card itself does full support OpenGL 2.
I'm not sure how it is this with newest graphics cards becouse now I always install official drivers right after installing chipsets drivers for motherboard. Now in case of older computers and ATI graphics card it is also recomended to install sound drivers before graphical drivers especiall for graphics card that already support HDMI so that graphics drivers can corectly interface with sound drivers (I had encountered severl dificulties when this didn't happen). But on newer AMD graphics card this is no longer necessary as they have full sound card support integrated right into them and therefore no longer require your computer to have other sound card.

As far as raw performance goes I have rarely seen any significant diferences when comparing benchmarks between DirectX and OpenGL. And whenever DirectX actually did give higher performance running of DirectX benchmark itself utilized CPU much more than the OpenGL benchmark therefore I can conclude that the aditional performance of DirectX test was due to aditionaly utilizing CPU for certain graphical processing. But in games that probably would not give you much performance boost, infact it could result in worse performasnce becouse it would eat up CPU performance that might be necessry for your game logic.

[Vinzvega]

Hi Anton,

You should take a look of Embarcadero engineer work, on FMX : They done exacly that : We can discuss on the good or the bad of the implemention end/or architecture, but their solution is here and available now.

Their goal was simple : When you create a 3d Component, the drawing must :
- Give the same results on OpenGL (OSX) DirectX (Win) and OpenGLES (Mobile)
- Must permit low level vertex access.
- Must take care of Shaders

The API and class tree is, in my point of view, pretty good, even if the VertexData access and Index access is not very weel design : We are more near DirectX than OGL. :/

Cheers,

Vincent

[phibermon]

The design chosen for FMX is totally unsuitable to base a game engine on in my opinion, it's an abstraction too far if you're looking to implement rendering techniques commonly used in game engines - you'll hit wall after wall. If you only use Delphi and you don't care about lighting, shadows, deferred rendering and anything else that you need to write custom shaders for then go right ahead but bare in mind that FMX is designed *only* to provide a hardware accelerated GUI across windows/osx/mobile to allow smooth animations etc.

To the best of my knowledge you have no direct access to the underlying graphics API and no control over what valuable graphics memory FMX uses, no control over multi-sampling and other OS specifics, where in the pipeline FMX renders, how it renders etc

For these reasons an engine co-existing with FMX is a world of compromise - it just wasn't designed with game developers in mind and that's fine because FMX's target audience are app developers who either don't know how or want to save time in creating hardware accelerated animation effects, something FMX does well enough to make it an attractive option for some.

There are plenty of existing 3D engines and of course you can write your own and if you want a funky GUI then it should be one designed to 'plug-in' in a generic manner to *any* rendering paradigm rather than one that tries to *be* the rendering paradigm.

My own JUI does this (designed from the ground up to be the UI of a 2D/3D hardware engine) and you'll find a number of other GUI's in the C++ world that you can use as dynamic libs if you're looking for a GUI to use with your engine.

But from the point of view of an experienced 3D engine developer? Steer clear of FMX! it's a very poor app-centric design that makes little to no concessions for shared use of the hardware.

Ogre3D is your friend. It's arguably the best multi-purpose 3D engine in existence (only being beaten by engines such as Unreal/Tech when it comes to first person, high density BSP style mesh / texture games)

Unity is popular but only because it does all the hard work for you, it doesn't actually do that hard work in an optimal way that can contend with technical engines such as Ogre.

--

But as stated, I'd just forget about DirectX if I were you. OpenGL is just as fast when used correctly and has a more logical design that's industry standard for everybody outside of Microsoft. The PS3 and PS4 both use GL so I'd say its credentials are good.

The only reason DirectX support would ever be considered worth the time and effort to add (to an engine that also targets non Microsoft platforms) is that certain older built-in laptop graphics adapters have crappy OpenGL support. Nothing to do with the API and everything to do with the drivers. If you're only targeting windows and are absolutely sure you never want to operate outside of the Microsoft eco-system then DirectX is a logical choice plus should you also develop in C++/.NET and wish to target the Xbox or Windows mobile platforms then it's not wasted knowledge.

However anything from Nivida/AMD is fine with OpenGL as are Intel chips from the HD3000 onward, the days of poor OpenGL support in drivers are long gone.

For WindowsRT support you can use a lib that converts OpenGL calls to DirectX (https://code.google.com/p/angleproject/) (and you can't yet target WindowsRT with any of the Object Pascal Compilers plus it'll all change again with Windows 10) and you can totally forget about Xbox support for non-microsoft compilers outside of the hacking scene.

Don't waste your time basically, target OpenGL and have fun developing your engine rather than spending 80% of your time hacking away at a working abstraction between two API's for no benefit.

[Vinzvega]

Wow, cool down man ;)

I ask to Anton to see FMX because FMX implements a real abstraction of underlying hardware for graphics. And I said that we can discuss about the bad and the good, but it exists and available. I add this sentence justly to avoid the kind of post you do.

Everyone here know that FMX is not a game engine, and it is even write in the Embarcadero's documentation. The question was not to use FMX as is, for Anton, but see how it is done, and get ideas.
And Yes, you can access to the basic layer, throught FMX.Context.DX11, FMX.Context.GLES,FMX.Context.GLES.Ios and so on.

When Anton said "I want to try to put in my engine a well design abstraction layer" you heard "I want to make a powerfull 3d engine"
--> WTF ??

Frankly, I'm tired about that everytime we said the word "FMX" in Pascal community, it will unchain a curious angry. A bit of frustrating ?


Anton, to teach something, do not "Steer clear of" everything that could open your mind a give you a chance to choose.

cheers,

[phibermon]

I don't work for Embarcadero, my interests are in sharing knowledge and helping people to create what they want to create using the best possible tools available - not just the ones belonging to a single company.

I know I spoke mainly of game engines but I'm advising to ignore FMX specifically on abstraction grounds - it's closed source, the user will learn nothing. The general process of API abstraction can be learned from thousands of different sources, studying a closed source API when there's much better and open source examples to study is pointless. After all this site is called pascal *game* development and the user it talking about 3D API's so of course we're talking in the context of 3D game engines. So yes I heard "I want to make a 3D rendering engine" and not "I want to draw buttons on propitiatory platforms". They are two very different goals and the abstractions required are totally different.

FMX is a poor design even when cited as an example of abstraction so I wonder why you've chosen to mention it at all?

Frankly, I'm tired of Embarcadero employees recommending unsuitable Delphi features as if they were perfect for games. FMX may make use of 3D API's but don't be tempted to confuse FMX as suitable for game development just because it does. My lawnmower uses petrol, it's not suitable for the race track.

Recommending somebody dedicate their passion to your product just to shift units is immoral. It is their future in game development, not your advertising platform. The compiler is suitable for game development, the IDE is suitable for development in general and indeed many of the components could be useful as well (for game development, especially if you want to create database driven servers) but FMX out of everything Delphi has to offer is the one thing that definitely is *not* suitable for game development. Not in direct use or as an example of 3D API abstraction.

Just to be clear : you can use Direct3D and OpenGL in Delphi without FMX. I'm not sure if that's true on the mobile platforms however but there's always Freepascal.

[Anton]

Thank you all for you answers.

If I choose for myself, I'll definately go with OpenGL, but when I work on projects which are not for myself, I'm not the one making the decision, so I thought such kind of abstraction layer would be useful since it accomplishes the requirements and in the same time it will be beneficiant since it will provide a potential to extend to other graphic apis.


When Anton said "I want to try to put in my engine a well design abstraction layer" you heard "I want to make a powerfull 3d engine"
Mostly my aim was for a general purpose 3D rendering abstract api.
But to make it the right way seems very complex. So, I have given up the idea for now. Yet the topic is still interesting for me.

Here are some posts I found interesting:
- http://www.gamedev.net/topic/640055-how-to-abstract-opengl-or-directx-specific-things-from-classes-specific-to-rendering/
- http://gamedev.stackexchange.com/questions/9364/correct-level-of-abstraction-for-a-3d-rendering-component
- http://www.gamedev.net/topic/649416-abstracting-away-directx-opengl-etc/

[phibermon]

It is a fascinating subject, I wish you the best of luck with your endeavours :)

[Vinzvega]

I don't work for Embarcadero, my interests are in sharing knowledge and helping people to create what they want to create using the best possible tools available - not just the ones belonging to a single company.

I know I spoke mainly of game engines but I'm advising to ignore FMX specifically on abstraction grounds - it's closed source, the user will learn nothing. The general process of API abstraction can be learned from thousands of different sources, studying a closed source API when there's much better and open source examples to study is pointless. After all this site is called pascal *game* development and the user it talking about 3D API's so of course we're talking in the context of 3D game engines. So yes I heard "I want to make a 3D rendering engine" and not "I want to draw buttons on propitiatory platforms". They are two very different goals and the abstractions required are totally different.

FMX is a poor design even when cited as an example of abstraction so I wonder why you've chosen to mention it at all?

Frankly, I'm tired of Embarcadero employees recommending unsuitable Delphi features as if they were perfect for games. FMX may make use of 3D API's but don't be tempted to confuse FMX as suitable for game development just because it does. My lawnmower uses petrol, it's not suitable for the race track.

Recommending somebody dedicate their passion to your product just to shift units is immoral. It is their future in game development, not your advertising platform. The compiler is suitable for game development, the IDE is suitable for development in general and indeed many of the components could be useful as well (for game development, especially if you want to create database driven servers) but FMX out of everything Delphi has to offer is the one thing that definitely is *not* suitable for game development. Not in direct use or as an example of 3D API abstraction.

Just to be clear : you can use Direct3D and OpenGL in Delphi without FMX. I'm not sure if that's true on the mobile platforms however but there's always Freepascal.

Hi,

I read your response at 11pm 30/01 and decided to drop, because, at this moment, it really hurt me.

this morning, at work, I just see your response again, and I see that you edit it after my read, and you wrote it in a much more diplomatic way.

--> You make an effort, and I decide too make an effort too, and I'm answering mainly because you ask think on me that is not right : you have the right to defend you opinion, and it should be suffisant.


To be clear, no, I'm *not* an EMB employees. Sorry Sherlock.


I'm working in financial stuff in Geneva, mainly in calculation in c language, in hpc/gpu tech, and, sure, a bit of delphi for our GUI.

So, I'm not working in fpc or lazarus, because we do not need it, simply, even if we have linux in server side, so we need rad for our gui. So we got delphi. So, Yes, I like delphi for years and I really believe that I can freely speek about delphi here. Even if it is a commercial product.

I apologize really that I found FMX cool about abstraction side of the system and graphics : This match for me the K.I.S.S. mode, it is working, easy to aprehend. So I speak about : that will never happen again here, I promise.

FMX have bad design and must be burn by the Inquisition, Okay, I noted.



I make new design very rarely, and, I confess, despite all my years in coding, I have always diffulty to tag bad or good design in code that I worked on : If the code is efficient in matter of support and modification it will survive. Simple. And I can say to you that quality is not really correlated to its design.

I understand too that the really main problem for you is the fact it is not game oriented 3d engine and it is close sources, and private held.
I forget this 2 importants facts, and see only the abstract side : You right, I Apologize.

--> Certainly, if FMX was open source, you'll find it technically much better ?! LOL (p.s : it's a joke, do not call Torquemada again and ignore this sentence !!)


No. The main problem for me, and it's really worring me : Why have you need to pass your message in a such condesendant way ?
Consdescance confirmed after, when, aparently, you are nominating yourself as a keeper of immorality, and keeping "young" people away of bad choice.

--> You know, when somebody is able to read, he's able to make is own choice without you.

You are showing the path, and that is your honour, But you didn't build it.


As you said, we are here because we all love Pascal and game, and that is suffisant : Useless to try to transform it in a holly mission.

Hope we can exhange later in more positive way.

[SilverWarior]

I must agree with phibermon about the fact that FMX is not best example of abstracting Graphical API.
The main reason for this is the fact that in FMX both graphical and logical part are tightly coupled together while most grahpical engines try to keep graphical and logical part uncoupled.
Keeping logical and graphical parts uncoupled means that logical part won't be slowing down the graphical part or vice versa. Not to mention that it is much easier to implement multithreading support when you have graphical part decoupled from logical part.

Do you want example to see how logical part can affect graphical part in FMX?
Simply create a FMX form with let us say 200 panels positioned on it. Compile the project and then go and rapidly move the mouse over the form and you will see significant increase in CPU utilization. On my laptop this is enough to lower FPS below 15 and having 400 panels on the form I can easily lower the FPS to single digits. And don't think that my laptop is weak. It is still powerfull enough to run farCry3 with athleast 30 FPS on medium graphical settings provided that I disable the AntiAliasing and MultiSampling.


Anywhay when I first tried FireMonkey I was verry exited but got quickly disapointed with its performance.
And what disapointed me even more is that even with that fraction of FMX code that Embarcadero made avalable (mosty source code for certain components) you can't help yourself much if you decide to go and try creating some custom components.
So in the end I ended up making my own UI library which is on hold for about a year now. I don't like talking about it to much becouse I feel ashamed that I still haven't gotten it to a usable state even thou I started working on it about two years ago if not even earlier.

[Vinzvega]

I must agree with phibermon about the fact that FMX is not best example of abstracting Graphical API.
The main reason for this is the fact that in FMX both graphical and logical part are tightly coupled together while most grahpical engines try to keep graphical and logical part uncoupled.
Keeping logical and graphical parts uncoupled means that logical part won't be slowing down the graphical part or vice versa. Not to mention that it is much easier to implement multithreading support when you have graphical part decoupled from logical part.

Do you want example to see how logical part can affect graphical part in FMX?
Simply create a FMX form with let us say 200 panels positioned on it. Compile the project and then go and rapidly move the mouse over the form and you will see significant increase in CPU utilization. On my laptop this is enough to lower FPS below 15 and having 400 panels on the form I can easily lower the FPS to single digits. And don't think that my laptop is weak. It is still powerfull enough to run farCry3 with athleast 30 FPS on medium graphical settings provided that I disable the AntiAliasing and MultiSampling.


Anywhay when I first tried FireMonkey I was verry exited but got quickly disapointed with its performance.
And what disapointed me even more is that even with that fraction of FMX code that Embarcadero made avalable (mosty source code for certain components) you can't help yourself much if you decide to go and try creating some custom components.
So in the end I ended up making my own UI library which is on hold for about a year now. I don't like talking about it to much becouse I feel ashamed that I still haven't gotten it to a usable state even thou I started working on it about two years ago if not even earlier.



Hi SilverWarrior,

Yes, I'm aware of quick performance deprecation you can have with this product : Even in 2D. There are no acceleration structure, so, FMX performance on "stress" is very bad.

In fact, I spoke about low level part (FMX.Context and FMX.Context.[Platform]) but, even if we consider only this part of the product, certainly many issues could be pointed : More or less, the easyness of technology access seduce me.

But as I said juste before, and As I recognized, talking about this subject here was a mistake.
The subject could be closed.


Scuse me cause I'm not understand : When you speak about your UI library, you build it upon which framework ? OpenGL ?

Kind regards,

[SilverWarior]

When you speak about your UI library, you build it upon which framework ? OpenGL ?

The basc design of my UI library is not to directly interface with graphical API's but instead with already built graphical engines. This is done through special abstract layer which serves as interface between my UI library and the graphical engine being used. This would hopefully alo it to be used with most graphical engines out there wheter they are pascal based or not.
So in a way it is more like an extension for existing graphical engines. The reason why I chose this approach is that at the time when I was starting to work on it there were several good pascal based graphical engines available like Asphyre Sphinx 3 or ZenGL.

So basically I'm focusing on high level development. I will leave midle level development (special abstract layer) to the developers or users of specific graphical engines since they have best knowledge of them. It would probably be posible to even develop the middle layer so that it directly interfaces with graphical API's for certain things.

Anywhay I'm afraid I'm still far from finishing it. It turned out to be much more difficult than I imagined, and my original design has already been replaced a few times :D

[phibermon]

I apologize if I caused offence and yes I was not very diplomatic much to my shame. I'm sorry.

Embarcadero and FMX is very much a hot topic for me - Like many Pascal developers in England I was taught Object Pascal and Delphi at School, College and University - Borland were making a big push to get the language recognized throughout the industry and we were all assured through our national institutions backed by the commercial might of Borland that Object Pascal had a future and it *did*, we all dedicated ourselves and our initial careers to Delphi. And all looked good until Embarcadero took over. Their lack of vision and effort followed by a series of poor quality control decisions that has allowed things like FMX to persist despite *repeated* failures and widespread criticism, things like IOS + Android support delayed for many years and Linux support *totally ignored*? these things basically say to the tens of thousands of people across the world that were counting on Embarcadero : We don't care about your future.

Delphi was the only thing keeping mine and many other peoples career prospects open. Just a few years ago Delphi jobs were abundant! and they still would be if Delphi had moved with the times.

Instead of a mighty cross platform system that works on all major platforms with a first rate library to match that of any C++ suite? We get a product stuck on Windows with poor cross platform support that was years and years late (and still woefully incomplete) and FMX : far from being the future of Delphi represents all that is wrong with the decisions made over the past few years. It's slow, badly layed out when compared to the *standard* VCL and enforces a design that makes game development next to impossible on IOS and Android (in pipeline as well GPU resources locked) you know Embarcadero, games? the number one source of revenue on Android and IOS? Thanks for that, really helpful.

So yeah, not a big fan of FMX on either technical or even general grounds. We should of had enterprise class database development on Linux, seamless RAD across all platforms, Android/IOS support *the moment* those systems came out. We should of had a professional, world class compiler and tool-chain. Instead we get animated buttons on IOS. Great. So for me the push for FMX is why Delphi is *no longer* the worlds greatest database client development tool and why mine and many thousands of people's careers have been ruined.

I wouldn't mind having support for other platforms years too late, I wouldn't even mind the total lack of Linux support (you know Embarcadero? the world standard in database servers? Delphi was the world standard for database client development? are you literally not seeing this?) But for FMX to be so *badly* designed as to be unsuitable for exploiting the main revenue streams on the platforms it was designed for? it's nothing short of insulting given the dedication that I and many people have shown in the past.

I hope FMX turns into some amazing fully capable system and we see support for spinning, hardware accelerated buttons on all major platforms and I wish all of its users happiness and joy.

But I won't be recommending FMX for games development and indeed I wouldn't recommend any other proprietary tech that ships with Delphi. It's too much of a gamble on a company that doesn't care about your future. Use open-source / standard libraries and API's even with Delphi - that way your hard work and effort will survive any poor commercial decisions.

I do want to see a brighter future for the language and for those looking to develop games in Pascal I do want to be able to recommend and advise what I objectively see as the best options.

I apologize again for my tone, it is unjustified unless you actually have an affiliation with Embarcadero - I just don't want anybody else's careers ruined on the promises of a company that can't deliver and the thought of an Embarcadero employee pushing FMX as suitable for games development in light of this made me angry. In such a context it is nothing short of greedy lying at the cost of people's future. Given that I can't see how anybody but an Embarcadero employee would recommend FMX? I assumed you were.

I'm sorry. I will now go and write a tetris clone with FMX as way of penance ;)

[phibermon]

So in the end I ended up making my own UI library which is on hold for about a year now. I don't like talking about it to much becouse I feel ashamed that I still haven't gotten it to a usable state even thou I started working on it about two years ago if not even earlier.

looking forward to seeing it! I wish you'd release it :D I've not released mine specifically so I don't step on your toes :P

[SilverWarior]

looking forward to seeing it! I wish you'd release it :D I've not released mine specifically so I don't step on your toes :P

Wait what? Did I read this right? Are you realy waiting with the release of yours as to give me a batter chance of sucsess?

[phibermon]

Wait what? Did I read this right? Are you realy waiting with the release of yours as to give me a batter chance of sucsess?

More because you showed such passion for creating a multi-purpous UI and had so many ideas that I didn't want to detract from your work. There's no decent existing UI like you've got planned available yet and if I were to release mine you might lose some potential users simply because they'd integrated mine before you'd had chance to finish even though yours may very well be better for them.

My passion is for my game engine :D I'll hopefully get some appreciation for my engine one day and I can live with downplaying my UI work so that you may get the maximum respect for something you're passionate about too :)

[SilverWarior]

Wow! I don't know with what did I earned such respect from you but I'm honored you think that way about me. I just hope I won't disapoint you in the end.

Anywhay you don't need to wait for me with publishing of you UI library. The main reason is that I have no idea when it will be finished. It may take a while especially if development will be as slow as it was in the last year.
Anywhay if I achieve desired quality of it I don't think it would matter much wheter it is published first or not. Besides I may even get some good idea from looking at yours. ;)

But the main two reasons why you shouldn't wait for me is:
1. Now that I know you are waiting for me it makes me feel presured which I don't like. Why? When I'm presured I tend to rush things. And when I rush things I tend to make many silly mistakes becouse of which the development takes even longer than it would othervise. This is one of the reasons why I didn't decided to join you in development of your UI library when you invited me to.
2. Based on feedback from many indy game developers that I'm in contact with (mostly giving suggestion on improving their games) I can tell you that most of them are quite frustrated by the fact that even good game engines usually come without decent UI library. Most of them only support verry basic UI library which usually isn't good enough for making decent good loking game UI. So many of them actually had to ither exted those basic UI libraries that come with the game engines they use or make their own. And this is another reason why I'm trying to develop my UI library in such way that it could be used with as many graphical engines as posible.
You say that your pasion is your game engine. Well including your UI library in your game engine would inprove its value greatly and therefore also its sucsess.

So don't wait for me and take care about your sucsess first. Especially since in the end I'm thinking about opening the source for my UI. But I won't do this now becouse I'm afraid that doing so I might be caried away from my initial idea too much.
And for those wondering yes I do intent to shift the focus of my UI library to fully support PGDCE first. But I first need to do another partial rewrite to support another feature that wasn't planned in the beggining but I have seen it requested by some pepole on the internet and my overal design alows pretty easy implementation of it.But I do need to implement it first before I write a design document of what exact capability is needed from the grapgical or game engine so that my UI library can be used with it.

[Vinzvega]

Hi Phibermon,

Thank you for your gentle response. I understand much better your reaction now.

Sorry on my side to not understand that more quickly.

I understand now too at what level you suffer about all the 5 last years of delphi actuality. I lived that with more distance, because not directly stuck with this market : I confess that when EMB bought the FMX technology from a component supplier and out DXE2 with IOS support throught FPC I honestly believe that was a proof of opening, and was very excited about. I was wrong. Sincerly, I'm just a little more optimist (naive ?) than you :)

You now, I grow up too with turbo pascal and Delhi1/2/3 and it was a fabulous period : I was angry when people compare it to VisualBasic, in this time already, Delphi began to be not well understood : Native compiler, advanced VCL : tremendous.

Today, Delphi change (not the VCL, despite the fact it is alway supported, good point !) RTL change, low level change and this new GUI library. Then, come the first problem with compiler deprecation and FMX "yoyo" quality, and persistant bugs. I excused all of that for 2 reasons : FMX was young technology and EMB was (is) alone in the market to get a chance to put Pascal again in front scene. All the other compiler, despite their top technical quality, is far behind. It is sad, but it is.

So, I shared near of all your technical analyze of EMB act about delphi : It is unfortunally true. But when you enumerate all the problem in one time, it is so hard to hear :/
In fact, the fact that EMB target only money (It is a Fund, behind EMB, and this is terrible for a firm, nothing count unless money behind the corporate action holder decision) is a kind of justification of the sad way taken by delphi today.

But I want to be more optimist : Perhaps, from those bad seeds, could grow again a beautifull and strong tree ? I hope. And for me, community is very important for that, but I see here and elsewhere (torry, Delphi.de) that community is less and less present. That is entirely the consequence of EMB act. You right again.
The worst, is that after all this heavy delphi break, EMB try with a new brand (appmethod) to get new people : I think this a loss of energy : Hope it will have good consequence on Delphi EcoSystem, but I doubt.

Hope, as you do, all will be better later. After the war, there are rebirth. Normally.. :)

If, as a redemption, you make a Tetris in FMX, I'll install FPC (command line) and start make OpenGL stuffs with it right this afternoon ! :)
(Ok, it is not really a punishment, but what to do else ?) :)

By !

[phibermon]

Oh don't feel pressured or anything, I've got so much work to do on the engine and now with PCDCE as well that I doubt I'll be releasing anything for at least a year :) And in terms of respect? You care about the community as much as anybody could, you do your best to help people, you're polite, courteous and objective when it comes to ideas. If PGD had a dozen of you it'd be one of the best programming forums let alone the best Pascal forum. You have my fullest respect :)

Yes I intend to release the UI with the engine, it's as capable as any tool-kit GTK etc and has full window management like any OS/Window manager. While the Game engine is openGL only the framework it sits in has a built in 2D engine that has abstracted renderers for Win32/Graphics32, OpenGL, Quartz and a VESA optimized renderer (I was hoping to get it running on CoffeeOS but it'll be a while before that's possible). JUI (the UI) relies on a few parts of the framework and is concidered an add-on like the game engine is. The framework on its own is basically a pascal equivalent to SDL. The UI can render in single-buffered region invalidating mode, direct render, double-buffered with renderbuffers per window (or control), double-buffered direct and a hybrid double-buffered invalidated mode that's optimized for real-time rendering with a number of windows. It times how long it takes to render any region, window control etc and selects the quickest render-path from various buffer/direct schemes for subsequent renders.

The UI also has an 'App' class container to allow it to operate on an OS like level. You can use taskbars/lists for running apps, you can define app dependencies and triggers (for example in my engine tool (JinkEd) if you delete, for example, a camera entity with a camera view app open, the view app will be destroyed as the camera entity is destroyed) the apps can be entirely scripted, directly coded and run in separate threads (and render in separate threads with certain paradigms although on fast GL hardware this render-path is rarely chosen by the manager and is more an optimization for software rendering). It's possible to run any window or indeed any control in it's own thread. Not that you would, but you can. Because scripts are isolated and can run any block in a separate thread it's possible for just a button to crash and the rest of your app to carry on working. (This is also the case with hard-coded elements although writing in this multi-threaded crash aware way is not for the faint hearted)

As well as all the usual menus, drop downs, listviews, treeviews, buttons, radio-groups, sliders, labels, edit boxes etc I've also written some specialized widgets for use in my engine and music studio such as a fairly advanced time-line control for piano rolls (also used for editing the timing/lifespan of mutators in my particle systems) and more uncommon add-ons like widgets for docking and stacking apps into toolbars. It's all unicode and it supports drag and drop internally (trees to lists etc etc) as well as file path drops from windows/linux and osx.

It has anchors for all elements, splitter controls etc. So any layout/resizing scheme you've ever seen in an app can be achieved. Naturally all item and header classes will scale according to the font used, border properties etc You can use different fonts, colours and themes/skins for every single control or window.

It supports custom rendering of any element so you can have skinned interfaces, it includes both square and rounded geometric base classes you can customize and yes it can do all of the silly firemonkey like animation stuff, it can spin and scale any window/button smoothly. Transparency fully supported, 3D rotations, explosions blah blah blah. I really didn't want to add such superfluous things but it's there should anybody really want to use it.

It's primarily designed for OpenGL. It has a pure vertex rendering mode and a customizable shader that allows you to render a very complex UI with thousands of controls without using any GPU memory beyond the loaded shader (you can easily plug it into one of your existing shaders if you have something suitable to save that last little bit of memory) or if memory isn't at a premium you can allow it to create buffers in graphics memory for increased rendering performance. You get to balance performance against memory use as much as you like. I did write a full VBO mode with a past UI that drew everything (including text) but I found that even on really old hardware with thousands of buttons, there was only a marginal performance gain over just using a VBO packed with a few primitives that you reuse to draw everything. Optimized rendering is therefore handled with the buffering/region invalidating raster methods.

The region invalidation works with a special type of widget I have called a Viewport which is used to render 2D/3D custom stuff (so a game for instance). If you decide to handle the callbacks you can have a realtime 3D window that will only re-draw visible portions by way of sending the viewport/camera projection data to the render-pass for the given visible regions

Because it's primarily designed for real-time rendering it also has frame-limiters etc you can use for viewports and you can have Pseudo double-buffering in a single-buffer context so if the user decides to make use of it, one window can be maxed out at 5fps (or locked while a complex scene is rendered) while another window can be churning along at 60fps.

It has primarily been coded on demand while creating a real-time 3D game engine tool that is similar to Blender/Lightwave in capabilities so if you're looking to create a front end to a ray-tracer or want to make the next blender, it's very well suited to the task.

Oh and it can render on a 3D surface inside the game engine itself, either directly with vertex/texture data or purely buffered on a texture. So you can use it to create computer interfaces on doors AKA Doom/Alien Isolation (I suppose the animation effects would be more suitable in such a context) You can also create a GL context and have it aligned with a software rendered window allowing you to encapsulate any GL engine which wasn't designed for or overrides scissors/viewports.

The one thing it doesn't do which I wanted is a client mode where each JUI window is created as a window in the OS, hiding the window frame and using the JUI controls instead. It's possible but I've not written it (yet) you can however hide your window frame and link resizing etc with callbacks to a single JUI window. Dialogs would be better as seperate windows (to be more 'compatible' with Oses like windows) so I may add it eventually.

All in all, I wish I'd never started on the UI. I'd of finished the game engine by now ;)

[phibermon]

Hi Phibermon, Thank you for your gentle response. I understand much better your reaction now.!

Thank you for being gracious and accepting my apology :)

I would dearly love to see Delphi at the top of the tree again. A product that I can say to anybody "You want to make games? 3D applications? you want to get to market faster with a more stable code-base than your competitors? use Delphi". I do understand why EMB have made the push with FMX, I totally get it - I really do. I just think they're totally wrong in doing so. I'm sure that mobile platforms were at the top of the wish list for users - I'm also sure that what they wished for was comparable support to C++, access to the standard API's. EMB ticked the right box, they used the wrong pen.

Delphi is great, it should be right up there on the front page of Slashdot, not laughed about in some random comments at the bottom of a post about alternative languages. At the very minimum, the commercial, expensive Delphi should be far better and more capable than a free alternative. They're getting paid to make it, it should be the best.

Sadly that's not true, Freepascal (and in many IDE respects, Lazarus) supports far more platforms, has a more complete (and standard) API for all of the mobile platforms - it's just a better compiler.

I'm happy that I've got a good cross platform Object Pascal compiler. I can do *everything* (sans target some specific propitiatory platforms like the Xbox) in Object Pascal that I can do with C++. There's nothing but my skill and time stopping me. But that's thanks to Freepascal. It *should* be thanks to Delphi.

Delphi (XE7) at absolute minimum price of £1526 to target all supported platforms, I can only hope to match my C++ rivals on Windows. That's it. I sure as hell can't compete on IOS and Android without access to the standard API and FMX taking up resources I need and enforcing a design I can't work with.

I think that's rather sad :\ I would love to see Delphi at the top of the tree, I've loved it, I've been dedicated to it. I was *proud* to be a Delphi developer.

Delphi is arguably a better IDE than Lazarus and does have some fantastic RAD database components - but at the price of a small car to get the functionality everybody else gets as standard, it's not enough. Especially without Linux support (And I'll be kind and forget the rest)

They should cater for us too, the lone games developers, the small teams. Games are the biggest source of revenue on mobile platforms, it's a fact. And I and everybody else here says Object Pascal is a fully capable language for game development, we know it is, we're doing it right now. In Freepascal - again, it should be Delphi. Would it really kill them to give us a cross platform Delphi at a decent price? dump all of the components! give us a cross platform compiler and the Delphi IDE, surely EMB can do that at a price we can afford? Their price list is proof that it's the components that make up a vast amount of the cost. Starter + Mobile pack would be £713! I'd *almost* stretch to that (with Linux support) just to be able to use the Delphi IDE. But I've got to pay £800 more for a bunch of components I don't want just to be able to target Android and IOS?

--

Off topic.

I think we should all agree not to discuss religion, politics or EMB pricing :D

[SilverWarior]

And in terms of respect? You care about the community as much as anybody could, you do your best to help people, you're polite, courteous and objective when it comes to ideas. If PGD had a dozen of you it'd be one of the best programming forums let alone the best Pascal forum. You have my fullest respect :)

Yes I do care about comunity greatly. Why? As a self taught Delphi programmer I know how hard it can be when you don't know where to turn to get help. When I started learning to program in Delphi its popularity was already in slow decline (that was short before Borland gave up on Delphi). And the only comunity that I was present at the time was Delphi-si comunity which was mostly populated by expirienced busniess programmer who had verry litle intention to try and understand what wild ideas I was trying to do and then help me out.
And boy did I had some wild ideas then (making shell replacment for windows, creating my own drivers for a custom controller - steering wheel that I made, DJ mixing software, etc). When I think about theese ideas now they don't seem so crazy becouse my programming knowledge is probably good enoigh to finish athleast some of them but then my programming knowledge was still far to weak to do anything like that.
So yes becouse I know how hard it can be at start I try to help whenever I can even to total newbies.


Oh and it can render on a 3D surface inside the game engine itself, either directly with vertex/texture data or purely buffered on a texture. So you can use it to create computer interfaces on doors AKA Doom/Alien Isolation (I suppose the animation effects would be more suitable in such a context)

This is also one of the planned feature of my UI.


All in all, I wish I'd never started on the UI. I'd of finished the game engine by now ;)

Don't say that! While you are probably not aware of it I'm willing to bet that doing both at the same time actually lead your into making both UI and game engine better as you would do it othervise. Why? Becouse by doing it so you ran to various quirks and difficulties that are specific either for UI or for graphical engine and adjust the designs of both JUI and your game engine to either avoid them or solve them.
Why I think like this? When I started working on my UI library I thought that all I will have to do is write a few methods which will fire several combinations od draw procedures and that is it. But soon I realized that for making of a UI library much more is required like:
1. Input handling: Without ability to handle user input your UI is not Ui but jst a graphical interface
2. Rendering loops: In order fto render you UI in proper way you need to learn about rendering loops
3. Sceene managment: While this might seem unimportant at first look it is quite important especially if you don't want to draw your whole UI right from scratch every rendering cycle. Reusing of already rendered or should I say composed scenes that hasn't changed from previous rendering cycle is the main way of optimizing your rendering cycles.
4. Resource managment: Again at first this might even seem as not neede at all but you quickly realize that it isn't so. When I'm saying resource here I don't only have graphical resources in mind (texutres) but also any other data that you require for your UI like controll positions, controll states, etc.

Anywhay we are getting a bit off topic here. I just hope that we haven't scared Anton with to much information here :-[

@Anton
How ever you decide I'm sure you can count on the help from more knowlegable pepole here on PGD when you will need it.
Unfortuantely I can't count myself into that group but if you will need any help in working with classes I belive I could be usefull as I belive I have pretty god knowlege about them. Not so much about interfaces but that is mostly becouse I myself don't see the need to use them. So far I have always managed to solve everything with classes alone.

[phibermon]

Yes I do care about comunity greatly. Why? As a self taught Delphi programmer I know how hard it can be when you don't know where to turn to get help. [...] So yes because I know how hard it can be at start I try to help whenever I can even to total newbies.

I try to as well, it's good to have others working towards the same goals :)




This is also one of the planned feature of my UI.

it's cool isn't it? :) I have a tech demo for my engine where there's a computer on a desk and the computer is 'running' a cut down version of the music sequencer I'm writing for the engine. I wrote a 3D front end for mame many years ago that was a room filled with arcade cabinets and each arcade machine would be running a seperate MAME emulated arcade game. At the time I lacked the 3D modeling experience and ability to work with C++ code to make it anything other than a proof of concept. But the whole idea of a system within a system in that way is very cool :D



Don't say that! While you are probably not aware of it I'm willing to bet that doing both at the same time actually lead your into making both UI and game engine better as you would do it othervise. Why? Becouse by doing it so you ran to various quirks and difficulties that are specific either for UI or for graphical engine and adjust the designs of both JUI and your game engine to either avoid them or solve them.

This is true, I can't say I've created the best UI but I can definitely say that my UI is suitable for use with a game engine!


1. Input handling [...] 2. Rendering loops [...] 3. Sceene managment [...] 4. Resource managment

Yes :) All have been key considerations and the design of the both the UI and game engine would be different if it wasn't for the existence of the other. The game engine developed in tandem with the UI and scene editing tool has resulted in a design that can be what you need it to be with the bare minimum of effort on part of the user. There's nothing in the engine that can't be controlled with UI elements, it's forced me to implement a consistent and simple interface.

Same with the UI, the way input is handled between the two, the way it renders, specifics that don't come into play with standard UI development (managing the depth buffer, minimizing texture memory use, self optimizing render paths etc) - Many things are the way they are because it's best for using the 3D hardware along with a 3D engine.

Yep and off topic, my apologies, I do tend to ramble ;)

[Anton]

Anywhay we are getting a bit off topic here. I just hope that we haven't scared Anton with to much information here :-[

@Anton
How ever you decide I'm sure you can count on the help from more knowlegable pepole here on PGD when you will need it.
Unfortuantely I can't count myself into that group but if you will need any help in working with classes I belive I could be usefull as I belive I have pretty god knowlege about them. Not so much about interfaces but that is mostly becouse I myself don't see the need to use them. So far I have always managed to solve everything with classes alone.Oh, on the contrary, I find everything very interesting. I didn't post anything since I solved my problem by not making such a layer. :) Thanks for your help. I'm now writing a small library for rendering in D3D11. About interfaces, for the last year or two I began to love working with interfaces.

As for FMX I can't really comment, since I have no experience with it. The only thing I know is from other people that it is heavy in terms of performance.



My passion is for my game engine :D I'll hopefully get some appreciation for my engine one day and I can live with downplaying my UI work so that you may get the maximum respect for something you're passionate about too :)
This is very interesting. Can you tell more? What features do you plan/support in your engine /do you have a demo video?


You now, I grow up too with turbo pascal and Delhi1/2/3 and it was a fabulous period : I was angry when people compare it to VisualBasic

it should be right up there on the front page of Slashdot, not laughed about in some random comments at the bottom of a post about alternative languages.Yeah, this is sad. Very often I meet people with novice skills on programming (in lets say C/C++) to tell me that it Delphi/Pascal is not a serious language (more like something which someone invented because he had much free time) but maybe this is more a mainstream unconscious view, rather than objective. Most of the times when I mention "Delphi" often people ask me "Does anyone use this anymore?" BTW, I have seen forums with people fighting C against C++, so what's left for C++ against Delphi. :D

[SilverWarior]

it's cool isn't it? :) I have a tech demo for my engine where there's a computer on a desk and the computer is 'running' a cut down version of the music sequencer I'm writing for the engine.

I don't have anything as fancy as you. Not yet anywhay. But when I manage to finish my UI ... Let just say that I already have a few dozens ideas about how I could use this :D


Yep and off topic, my apologies, I do tend to ramble ;)

I sometimes also have huge urge to share my thoughts ;) so I'm guilty aswell :D

[SilverWarior]

Yeah, this is sad. Very often I meet people with novice skills on programming (in lets say C/C++) to tell me that it Delphi/Pascal is not a serious language (more like something which someone invented because he had much free time) but maybe this is more a mainstream unconscious view, rather than objective. Most of the times when I mention "Delphi" often people ask me "Does anyone use this anymore?" BTW, I have seen forums with people fighting C against C++, so what's left for C++ against Delphi. :D

When I talk with other programmers about Delphi/Objet Pascal their response is usually something like this:
1. Programmer that have never done any programming in Delphi/Object Pascal: "What Delphi? You can't do any serious application with it. You need much more than simply placing some components on the form."
And the main reason for their response to be like this is that Emb arcadero always advertise how Delphi comes with bunch of usefull components, Live Bindings, etc but rarely advertize any language properies. Only recently you could see a few of new language features being advertized like Generics, Advanced Referencing, etc.
2. Programmer that has athleast tried doing some programming in Delphi/Object Pascal: "That isn't a bad language but I don't think it is capable enough for some serious applications. There arent many usefull examples or libraries for it available."
3. Programmers that have done quite some prgramming in Delphi/Object Pascal but have switched to other programming languages: "Pretty good language, lacking some of the newest features. Has rather poor and dwindling comunity and its development IDE is verry expensive. So it is better that you switch to some mainstream language soon if you want to be able to get any programing job in the future".
What these guys fail to realize that they are the main reason why the comunity keeps getting smaller and smaller. But stayung wold mean that they wouldn't have acces to the newest windows features like those that come out in .NET programming language.

So one day I have asked one of them who migrated from Delphi/Object Pascal to Visual Studio/.NET: "Why have you decided to migrate to .NET?"
And his answer was: For the software I'm developing I requires access to NFC hardware."
So my next question was: "Isn't there a library for Delphi available that would support this."
He says: "No ther isnt."
So I ask him: "Have you considered making such library yourself?"
He looks at me and goes: "Are you crazy? Why should I spent several weaks or perhaps even a month designing and implementing such library if one already exists for .NET?"
So my next question was: "Could you share how much time have you spent porting your application from Delphi to .NET?"
His asnwer that he harldy said it was: "I have spent four months proting the code to .NET"
And my response was: "So you spent four months porting your whole application to .NET instead of spending one month implementing the missing library. It haredly seems woth it"
And then my final question was: "How many new bugs have crawled in during the porting of your program?"
I have never got the answer to the last question. I hues that is becuse I hit the nerve of that programmer with it. He definitly knew that the choice of porting the whole application to .NET just becouse of one library was a poor one but he wasn't prepared to admit it.

Anywhay when someone asks me as to which programming language they should use I usually say something like this:
"Chose the one whose sintax structure suits you more. It is definitly better to program in programming language that feels natural to you and lack some features becouse in the end you can probably implement them yourself rather than programming in a programming language that offers you all of the features but you are having problems with its sytax structure becouse it doesen't seems logical to you.
It is like talking in foregin language. You can easily learn meaing of certain words but if the languages syntax isn't logical to you you will have realy hard time creating a meaningfull sentace out of those words."

[Anton]

We are getting far away from the topic by the way :)

Yeah, I agree that everyone should use that language/tools which fits them better and makes them comfortable.
I see a tendency, anyway, that people would reject to learning something which looks simple, but prefer complex-looking thing (with more symbols, slashes, all kinds of braces, etcetra) for the aim that they look smarter in world's eyes (which I doubt is a smart choice by itself). :D