pLua Beta available

[jdarling]

Since I've had enough questions over the past few weeks and I've managed to get most of pLua complete, I'm putting up a download link for anyone interested. Included in the download is the start of the documentation, ALL of the support files, some demos (still to be expanded), and the start of a wrapper generator.

Some things are different in pLua then in the last distro I put out for everyone. First, it isn't built on top of the Lua wrappers put out by LuaEdit (kept the lua.pas file and updated but thats the only one). Second, LuaUtils no longer exists (so if you need it keep your old version). Third, new ways to manage Objects and Records built in (demos also included).

As I said, this is a beta, and things will be changing. What I know will be changing is; The documentation is being updated, the Wrapper Generator will be completed (only scripting left), More Demos, and of course any bug fixes :).

I strongly encourage anyone using my old units to start using these new units as the FIX MANY BUGS! Sorry, but a lot had to change, but no features were lost.

In fact, a great new feature that many asked for is the pLuaObject wrapper stuff. This allows you to wrap up existing object types without having to create a management object. You can also register objects into multiple lua instances and use them from withing multiple lua instances.

Changes to come later on will include updates to the TLuaThread object.

Anyways, enough bla bla, here is the link: http://www.eonclash.com/LUA/pLua.zip

On a final note: I'm looking for help with this code. If you have any interest in working on Documentation (I'm using FPDoc currently), Demos, or bug fixes send me a message. Once all of the existing stuff is wrapped up I plan on starting to port the actual Lua 5.1 source to pascal :) so the fast this gets done the faster I start on that.

[Robert Kosek]

I'm looking to try this in a day or two, things have been a bit busy lately, as a configuration language. What I'll be doing is having in separate scripts a single variable that is really a large structure that varies based on the actual name of the variable returned with the script.

My primary question is: how can I find a specifically named variable, or list of variables, in a file and then grab the table structure into a record? I am guessing that I'll be processing each script, freeing the parser after getting the data, and then moving to the next script in that directory. Is there a better way?

[jdarling]

Well, yes and no :). There is a method within pLua called plua_tovariant. If you give it a variable name that represents a table it will then create a variant array that you can traverse but won't get the names for each element. To do that take a look at plua_tovariantarray (I think it was called, I don't have the source in front of me right now :(). Pass in a string list and get back the index of the named items.

Another way is pure Lua, as an example lets say I need to look and see if the variable FullScreen is defined and if it is read it as a boolean. The Pascal code would look as follows:lua_pushstring(L, 'FullScreen');//Place the variable name on the stack.
lua_gettable(L, LUA_GLOBALSINDEX); //find it in the global space
if lua_isnil(L, -1) then
// Code here to handle that it doesn't exist
else
begin
// We found it, now check the type
if lua_isbool(L, -1) then
LocalFullScreen := lua_toboolean(L, -1)
else
// Handle the fact that its the wrong type (throw an error or whatever)
;
end;

Finally, if I haven't given you enough options yet :) and your using the TLua object you can read the values directly as variants:MyLua.Value['FullScreen']

As for the loading/unloading aspect, if you want to clear the Lua namespace (meaning that you don't want to carry values from one script to the next at all) then you will need to call Close (I'm going to assume your using TLua) before calling loadfile. If you don't care if values are carried over then simply call loadfile and then execute.

A nice new feature (that I just finished today and haven't uploaded yet but will first thing in the morning) is that Class and Record wrappers that are surfaced using the pluaObject and pluaRecord units are automatically registered into ALL TLua instances :).

Hope that helps, and look for more updates soon.

[Robert Kosek]

A nice new feature (that I just finished today and haven't uploaded yet but will first thing in the morning) is that Class and Record wrappers that are surfaced using the pluaObject and pluaRecord units are automatically registered into ALL TLua instances :).Awesome. Is there a way I can, in my code, tell a variable "you are now this type" and read it as a specific record? I ask because type is often taken for granted in Lua. ;)

This would make things very handy indeed for configuration and game "templates," etc.

[jdarling]

Well, considering that Lua isn't a typed language there really isn't a way to type its variables :). What you can do is create a record type and register it to your Lua environment. Then register a global version of that record type and have your script reference that.

My explanation is a bit weak, but I do plan on building some type of demo around this before the end of the weekend. Soon as I get it completed I'll drop a note and post a link to the updated download.

[Robert Kosek]

Well, considering that Lua isn't a typed language there really isn't a way to type its variables :). What you can do is create a record type and register it to your Lua environment. Then register a global version of that record type and have your script reference that.Okay. So what I would do is register the format of the record with Lua, and then register a global variable as that type. I get that much. ;) I wouldn't bother with the Close function then as I don't want to reset the whole thing, but I could push a constant defined default back into the register, correct? I think that would be the best way.

The only problem is if you have many things to configure based off the name of the variable, I assume there isn't an "isVariableChanged" function. So to tell what variable was changed, is it best to compare the record against the blank default?

[jdarling]

That would be one way of doing it. You could also use a customized writer and set a flag when the value of the field is updated. Since you will have to build out a writer anyways, this would probably be the best way. Of course, you could also go a step further and create a virtual record (a record structure in Lua that doesn't really exist in Lua). As I said I'm working on a Demo to show all of the above, but can't guarntee its completion except that I plan on completing it before the end of the weekend :)

[jdarling]

What do you know, maricles do happen even when its not Christmas. Anyways I manged to get the demo built out (minimally but I think its enough for you to get the idea) and get the updates into the Wrapper Generator (still A LOT more work to go) as well as a few other minor performance tweaks (no API's changed).

Same download link as before:http://www.eonclash.com/LUA/pLua.zip

Take a look at Demos/ConfigApp for a sample of using virtual records and the Demos/pLuaRecords for a sample of wrapping up a normal record. Though the pLuaRecords demo needs to be updated to use the new handlers instead of the way its doing it now (either way will work).

[Robert Kosek]

Well, there's no timetable on this obviously. But once I start to test these things and try them out I'll write things like examples to help others learn too.

A virtual record would work well I think, but what about records in records? Obviously Lua handles tables in tables just fine, but it would be so helpful if I could have a virtual record in a virtual record. ;)

I intend to be having records like this, for example, split into seperate files:
-- Earth-class planets...
planet_type = {
id = "green",
name = "Verdant",
name_patterns = {"VVCCC","CVCCV","CVVCV"},
-- Production:
exports = {
{"agricultural", 2.0}, -- production per annum
{"lumber", 1.5},
{"light metals", 1.5},
{"medium metals", 1.0},
{"terraformables", 0.5}
},
-- Consumption:
imports = {
{"fissables", 0.66}, -- imports per annum
{"heavy metals", 0.75},
{"electronics", 1.00}
},
bombard = {
threshold = 500, -- damage points a world can take...
transform = "desert", -- turns into this when "dead"
recovery = 50 -- points recovered per annum
},
terraform = nil,
description = [[
A verdant world covered with forests, plains, and a high amount of plant life. There is nothing it can be terraformed into, though nuclear bombardment has a tendency to kill these worlds. These worlds are rarer and very valuable. Valued exports are agricultural, lumber, and things used to terraform other worlds.
]]
}

-- The desert world...
planet_type = {
id = "desert",
name = "Desert",
name_patterns = {"CVCVCV"}, -- whatever. :P
exports = {
{"glasswares", 1.75},
{"silicates", 1.50},
{"electronics", 1.25}
},
imports = {
{"agricultural", 0.75},
{"light metals", 0.25},
{"medium metals", 0.33},
{"heavy metals", 0.40},
{"fissables", 0.50},
},
bombard = {
threshold = 300, -- desert worlds are fragile
transform = "glass", -- and turn into glass-worlds,
recovery = 0 -- but never recover from damage!
},
terraform = {
threshold = 1200, -- costly terraforming
transform = "temperate", -- turns to a half-desert world
cost_mod = 1.0
},
description = [[
A desert world requires maintenance for survival, but produces components valuable to society: electronics. Other minor exports do leave the world, but these do not recover the cost of maintenence so well as electronics do. While costly to maintain, these worlds can be terraformed, without extravagant cost, into more valuable temperate worlds.
]]
}

[Luuk van Venrooij]

Nice going to check this out later. My engine still needs scripting:)

[Robert Kosek]

Completely missed your previous post, Jeremy. Sorry.

I was hoping I could assign the properties as a table. I would vastly prefer to work with a table, in fact, because it is more orderly. Though, I could use the object types with a constructor that registers the object with a "template factory".

How would I define configuration like this?
-- Config.Caption = 'Caption set from config.lua'
-- Config.Color = HexToInt('00FF00')
Config = {
Caption = 'Caption set inside a table!',
Color = HexToInt('FF0000')
}

I'll start reading the documentation as I find the time, so I'll post if I find it myself.

EDIT: kind of like a "plua_PopRecordFromTable" variant, though I'll have to research table functionality better. ;)

[jdarling]

This is one of those "Due to the nature of the beat things" where Lua has its own way of managing variables within itself. In this case it would require overriding the global __index and __newindex handlers as well as setting up a metatable for the globals index table.

This is something that I'm working on, but don't have complete yet. In fact its still very much in the design phase due to the complexity it can introduce.

What about a simple setter method? As an example:function lua_SetConfig(L : PLua_State) : integer; cdecl;
begin
result := 0;
lua_pushliteral(L, 'Caption');
lua_gettable(L, -2);
if not lua_isnil(L, -1) then
frmMain.Caption := plua_tostring(L, -1);
lua_pop(L, 1);
lua_pushliteral(L, 'Color');
lua_gettable(L, -2);
if not lua_isnil(L, -1) then
frmMain.Color := lua_tointeger(L, -1);
lua_pop(L, 1);
end;

Then your Lua code would change to:
aConfig = {
Caption = 'Caption set inside a table!',
Color = HexToInt('FF0000')
}

SetConfig(aConfig)

If that doesn't help, then I'd have to say patience is the key at this point :). Of course you could write the handler in Lua itself and require it if you would prefer.

[Robert Kosek]

Well, I guess I could write it like that and use a for loop to iterate through the various records. Then it'd be something like "PushPlanet", which just sounds funny. :lol: *tink!* I do think that would be the best option though. Then I could just separate everything out between scripts based on type to keep it orderly.

How would I handle sub-tables like in my example though? What if there were no terraforming elements like I put in one example? I'd guess if the element wasn't nil you could then treat it like a table and load it, but that's complex. :?

BTW, what is the difference between:
function lua_isnil(L : Plua_State; n : Integer) : Boolean;
//...
function lua_isnone(L : Plua_State; n : Integer) : Boolean;
function lua_isnoneornil(L : Plua_State; n : Integer) : Boolean;

[jdarling]

Hmm... I honestly don't know what the difference is between the methods, I'll take a look and see if I can figure it out though :).

As for the sub tables and records or objects take a look at the PushExisting methods and RegisterExisting. RegisterExist registers a member as a global variable while PushExisting allows you to place an existing one on the stack. Similar to the virtual record I showed earlier, you can use a virtual sub-element (member, variable whatever) and simply register it back to your main one. Similar to the following (just the registration code is shown, I think you can get the idea on how to actually build out the types):function GetMySub(RecordPointer : pointer; l : Plua_State; paramidxstart, paramcount : integer) : Integer;
begin
// Get the value of the Color and put it on the stack
// Note: If you are using virtual records then set MyExistingPointer to nil
plua_pushexisting(l, MyExistingPointer, RecordTypesList['TMySub'], false);
result := 1;
end;

procedure InitMyStuff;
var
ri : PLuaRecordInfo;
begin
ri := RecordTypesList.Add('TConfig');
// add stuff here
// Here is the property handler that will return the TMySub
plua_AddRecordProperty(ri^, 'MySub', @GetMySub, nil); // nil makes it read only!
ri := RecordTypesList.Add('TMySub');
// more here
end.


I also noticed that when I built the distro I forgot to include the latest version of pluaRecord (my bad). Updated the distro and now it should have the methods described above :). Same link as before I'll just keep updating that DL until I get to a stable V1.

I'll also see if I can't get the time to create a version of the config demo that shows how to do this. Well, at least one way to do it :)

[Robert Kosek]

Okay, I will have a look at it this afternoon.

And I was referring to subtables through the "Push" method you gave such as:
function lua_SetConfig(L : PLua_State) : integer; cdecl;
begin
result := 0;
lua_pushliteral(L, 'Caption');
lua_gettable(L, -2);
if not lua_isnil(L, -1) then
frmMain.Caption := plua_tostring(L, -1);
lua_pop(L, 1);
lua_pushliteral(L, 'Color');
lua_gettable(L, -2);
if not lua_isnil(L, -1) then
frmMain.Color := lua_tointeger(L, -1);
lua_pop(L, 1);
end;How would I go about, intelligently, handling potentially recursive tables here like the example table I gave? I'm just worrying about going too deep and shooting myself in the foot with arrays and tables right now.

My example was:
planet_type = {
id = "green",
name = "Verdant",
name_patterns = {"VVCCC","CVCCV","CVVCV"},
-- Production:
exports = {
{"agricultural", 2.0}, -- production per annum
{"lumber", 1.5},
{"light metals", 1.5},
{"medium metals", 1.0},
{"terraformables", 0.5}
},
-- Consumption:
imports = {
{"fissables", 0.66}, -- imports per annum
{"heavy metals", 0.75},
{"electronics", 1.00}
},
bombard = {
threshold = 500, -- damage points a world can take...
transform = "desert", -- turns into this when "dead"
recovery = 50 -- points recovered per annum
},
terraform = nil,
description = [[
A verdant world covered with forests, plains, and a high amount of plant life. There is nothing it can be terraformed into, though nuclear bombardment has a tendency to kill these worlds. These worlds are rarer and very valuable. Valued exports are agricultural, lumber, and things used to terraform other worlds.
]]
}If you look up the terraform table in the entry could be nil or a table in the given format; as can bombard, export and import really. But, Import and export is an array of sub-tables as a sub-table. So that just seems messy to me. How should I go about making that clean in Delphi and Lua both?

[jdarling]

New version of pLua has been uploaded with MANY bug fixes and a few new features shown off in the demos. New demo in Demos/pLuaObjects2 that shows how to register an existing object with the new pLuaObject model. Again, I'd strongly suggest that you update your pLua install if your using it as the new fixes could affect some projects.

Download URL: http://www.eonclash.com/LUA/pLua.zip

[Robert Kosek]

Hey Jeremy, how is pLua going? I know it's been a month, and this is probably necromancy, but I don't particularly care.

I've decided, now that I'm getting back into game programming after a complete reformat, that I'll start trying this configuration bit again. However this time what I thought of is that instead of each object type having its own record type, it would hinder a singleton game object parent class, I could make things using an associative array. I would have to make an object with an array of key/value pairs, but that's easy. But this way I can actually cycle through each piece of a Lua table with lua_next (http://www.lua.org/manual/5.1/manual.html#lua_next) and then use each key/value pair to store the templates. Then any objects become extensible by a new class only requiring a few extra table values. :)

What do you think?

[jdarling]

pLua is still going, but progress is slow due to some side projects (my house being the biggest time consumer) and my work all wanting lots of attention. In the little time I have had I've been looking at how to override the global __index method without causing too many problems in other places. I've also been looking into the custom setters you were talking about. The easiest thing to do would be to create a custom object (TMyConfigType or whatever) and register it into the Lua environment. Then in your scripts create an instance of it and assign the values. In your custom type you could have the setter for the fields perform the lua_next easily enough (just remember to check lua_isnil to know when you hit the end of the table).

Hope this helps, but if not I'll see if I can't find a few minutes to put together an example of how exactly to do this.

[Robert Kosek]

Along that line, do indexers work when I register an object? For instance I have a class to do all this associative array work that I mentioned, it's far easier than I thought years ago, and it uses an indexer to let it act like an array. The data is stored inside a single linked list. There isn't that much to it, but they wouldn't be setting properties so much as they would be making a virtual table.

The basis is:
TAssociativeArray = class
private
first, last: TAssocArrayItem;
function GetItem(S: String): Variant;
procedure SetItem(S: String; V: Variant);
public
property Values[Index: String]: Variant read GetItem write SetItem; default;
destructor Destroy; override;
end;

I try the wrapper generator and the output it blank because it doesn't like "string", "variants", or my "TAssocArrayItem" record. I only have the above class checked for conversion by the way.

[jdarling]

Robert, try downloading the latest (just uploaded a few minutes ago) version of pLua from http://www.eonclash.com/LUA/pLua.zip and extracting it. There is a few minor updates to uLuaObject.pas to support generic readers and writers (UnhandledReader and UnhandledWriter in TLuaClassInfo) and a new Demo (Demos\AssociativeArray) showing exactly what your asking for. I took your basic type header, minimized it, and completed it as a demo application. Basic idea is run the app, and type something in the box (dog, cat, and bird are all pre-populated by the script.lua file).

Hope that answers your questions, and hopefully here soon I'll be back under active development with pLua and not just ripping something out every time someone asks a question :). I'd love to get the documentation done for pLua as I think many would find it useful.

PS: The wrapper generator isn't complete yet, and thats the main reason you didn't get anything out of it :)

[Robert Kosek]

That works like a charm, thanks. :)

How would a register an "assignment" protocol to the object in addition to the indexer? Or, if the indexer has to go, then just the assignment protocol. So I can assign a table and then walk it to get the values. If you could help point me in the right direction there it would be much appreciated.

Oh, just a quick question. Is the UnhandledReader/UnhandledWriter support for array-like access only, or can it be used for assignment in a "ar = {}" command?

Edit:

Actually, now that I think about it more, I'm going to end up with a "recursive" associative array that will end up providing my templates. So maybe registration of the type is unnecessary. Instead I could, like you advised awhile back, use a surrogate function and pass the table plus an identifier string to a function.

So since I'm still unclear as to how, how would I declare/register a function to accept two parameters (first a string, then a table) and then in the function begin to walk the table into an associative array?

EG:
MyStuff = {
"var1" = "whatever",
"var2" = 42.42424242
}
AddTemplate('TMyStuff:2001', MyStuff);

I kinda understand how to use the function lua_next to walk the table, but not how to get the table's ID and then pass through the walk-loop. You don't need to write it all, I just need to understand Lua's stack concept--which is kinda painful.

[jdarling]

In order to handle the ar={} problem you would have to override the global indexer's. This can cause some real problems within the Lua environment, though many people do it, and I haven't played with the idea enough to actually suggest a good way of doing it. Remember the indexer is at the parent (meaning the ar.__index and ar.__newindex will handle ar.* and ar calls).

As for registering a method that will accept a string and a table. There are two ways to do this in pLua; First is to use the normal Lua way of handling the passed in params.function lua_AddTemplate(L : PLua_State) : Integer; cdecl;
var
fieldName,
TemplateName : AnsiString;
begin
result := 0;
// First check to make sure that ONLY 2 params got passed in
// it would be a good idea to check to see if we have the proper
// types coming in, but I'm not doing that :)
if lua_gettop(L) <> 2 then
exit;
// Now get the TemplateName
TemplateName := pLua_tostring(L, 1); // 1st param
lua_pushnil(L);
while (lua_next(L, 2) <> 0) do
begin
fieldName := plua_tostring(l,-2);// get the key
// the value is at -1, you can do something with it here :)
end;
lua_pop(L, 1); // remove our working table and the last nil pointer
end;

The second way would be to use pLua's built in variant handlers and read the passed in table as a variant array. You will need a temporary string list to hold the key names:function lua_AddTemplate(L : PLua_State) : Integer; cdecl;
var
lua_values : Variant;
lua_keys : TStringList;
TemplateName : AnsiString;
begin
result := 0;
if lua_gettop(L) <> 2 then
exit;
lua_keys := TStringList.Create;
try
TemplateName := plua_tostring(L, 1);
lua_values := plua_TableToVariantArray(L, 2, lua_keys);
// Now you can itterate lua_values as an array and lua_keys contains
// the keys for each table element.
finally
lua_keys.Free;
end;
end;

As for registering your method, its the same in both cases. Either use your TLua instance's RegisterMethod or use pLua's built in RegisterMethod call to register it to the global environment.

I did all of the above freehand without and IDE so there may be a few typo's, but it should get the idea across.

[Robert Kosek]

Yeah, I get the idea.

After looking at pLua's implementation of TableToVariantArray and ToVariant, I see a potential problem: collision. If I used a series of subtables to describe parts of data that my application read, not even the keys would help because the keys would be reused. Now, if we concatenated the sub-tables key/name into the keys beneath it (including any of its sub-tables) we could completely avoid collision. :)

Just an idea, but one I think could become important.

I'll tinker around with trying to figure out how to handle the child tables, but no promises that I get anything put together. At least nothing stable/fast anyway. :D But I'll give it a whirl when I get some free time.

EDIT:

Yesterday evening I came up with a way to make a multi-depth associative array, based off the prior class, by including a child object reference. I haven't tried it yet, but as a rule it shouldn't be too hard. What I'll do is include a pointer stack for the last accessed items and then be able to restore a prior point. This way for parent/child searching all it takes is a few pushes and pops and you still have a reference to the base parent. ;) I think I can base the function for table conversion off this same logic, though access would require:
myarray&#91;&#91;'grandparent','parent','child'&#93;&#93;;As an array of string would be the indexer and not a normal string. (I'm quite sure index functions can't be varargs. :lol:)

[tuxscreen]

Hi Jeremy,

nice tool. As I'd like to use it in my delphi projects I tried it and had to fix
it in some places to get it to compile.

I would dive in and send you a patched version, but I think this might be already too late, because you might have already worked on it, so a diff against my version would not help.

I had to patch the following items:
PtrInt is not defined in delphi, FPC helpfile tells me that this type is deprecated and we should use PtrUInt or something completely different?
(BTW. I've still not dived deep enough into the LUA stuff to test this PtrInt routines :( )

Some files contain {$MODE..} compiler switches w/o the surrounding ifdefs for FPC.

event-handler assignments dont use the @-operator in delphi, I dont know fpc very well, but I there must be a way for both worlds.

I fixed also many warnings and hints the compiler throwed at me, but these where mostly unused variable stuff and if clauses which are always true...

I would like to participate in this "little" project and test against the latest delphi compilers, if you like.

Greetings
Marc