Steam wrapper, exploring options

[klausvdl]

I noticed the considerable contributions of Relfos (here (https://www.pascalgamedevelopment.com/showthread.php?32389-Steamworks-pascal-headers)) and thecocce (Lazarus forum (https://forum.lazarus.freepascal.org/index.php?topic=26968.0)) to the theme. Their approach requires to build a custom DLL (with Visual Studio) against the current Steamworks SDK, if I'm right. So every time the Steamworks SDK gets updated, someone has to update the custom DLL too, I guess. Perhaps you can stay with a certain SDK version over time. But if you don't have yet any SDK you have to download the current version and rely on an appropriate custom DLL.

Anyway the Steamworks API, although being a C++ construct, provides at least some simple C-functions for init and shut-down (steam_api.h). So in Pascal, after loading the steam_api.dll (win32), you can access those functions without any custom DLL:



unit steam;

{$H+}{$mode objfpc}

interface

uses
dynlibs, ctypes, sysutils;

const
steamlib = 'steam_api.dll';

function steam_init: longint;

implementation

var
steamlib_handle: tlibhandle = nilhandle;

var
steamapi_init: function(): boolean; cdecl = nil;
steamapi_shutdown: procedure(); cdecl = nil;

procedure steam_exit;
begin
steamapi_shutdown();
unloadlibrary(steamlib_handle);
end;

function steam_init: longint;
begin
steamlib_handle := loadlibrary(steamlib);

if steamlib_handle=nilhandle then exit(0); // no steam DLL, run without steam

addexitproc(@steam_exit);

pointer(steamapi_init) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_Init'));
pointer(steamapi_shutdown) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_Shutdown'));

if not steamapi_init() then exit(-1); // steam client not running, program should halt

result := 1; // ok
end;

end.




program steam_game;

uses
..., steam;

...

begin

...

if steam_init=-1 then begin

// show message: steam client not running

halt;
end;

...

end.


I'm using this right now in my own game project. The Steam overlay is showing up and you even can access it. Further you can ensure that the Steam client is running. But all the other common Steam features are not available this way.

Then I stumbled upon the release notes regarding Steamworks SDK v1.32 (February 2015). One note reads as follows:

Added an auto-generated "flat" C-style API for common Steamworks features (steam_api_flat.h).
At this point the exploration begins.

I want to continue in an upcoming post. But feel free already to comment, to correct me or to give some advice regarding the possibilities of using that "flat" C-style API.

Thanks!

[JernejL]

We could start an auto-updating and compiling project, maybe with help of chet ( https://github.com/neslib/Chet ) to build pascal linking units.

I've done something similar - but incomplete, to build newton dlls for use with pascal.

[klausvdl]

Hey, glad about your interest!
Well, this exploration is meant to be a more pragmatic approach. At least for me it's a matter of available time.
I'm not sure at all if it's possible to make use of that steam_api_flat.h. Have to see what can be done ...

[JernejL]

Give it a try with chet, it's worth a try, the file looks rather simple:

https://github.com/smefcc/smefs-Indigo-Remastered/blob/master/INDIGO/steam_sdk/steam_api_flat.h

[klausvdl]

Yeah, I was sparing with hints. Let me be more specific about what I have in mind with my pragmatic approach: I want to use just a few Steam features in my game (Steam achievements for example) without translating the whole API. I don't want to install a C environment (Visual Studio e.g.) for this limited purpose. And I want to avoid an additional custom DLL (which requires maintenance).

Thanks for linking the steam_api_flat.h! Nice transition ;)

So, these are the first 10 lines (of nearly 700) of function declarations in steam_api_flat.h:



S_API HSteamPipe SteamAPI_ISteamClient_CreateSteamPipe(intptr_t instancePtr);
S_API bool SteamAPI_ISteamClient_BReleaseSteamPipe(intptr_t instancePtr, HSteamPipe hSteamPipe);
S_API HSteamUser SteamAPI_ISteamClient_ConnectToGlobalUser(intptr_t instancePtr, HSteamPipe hSteamPipe);
S_API HSteamUser SteamAPI_ISteamClient_CreateLocalUser(intptr_t instancePtr, HSteamPipe * phSteamPipe, EAccountType eAccountType);
S_API void SteamAPI_ISteamClient_ReleaseUser(intptr_t instancePtr, HSteamPipe hSteamPipe, HSteamUser hUser);
S_API class ISteamUser * SteamAPI_ISteamClient_GetISteamUser(intptr_t instancePtr, HSteamUser hSteamUser, HSteamPipe hSteamPipe, const char * pchVersion);
S_API class ISteamGameServer * SteamAPI_ISteamClient_GetISteamGameServer(intptr_t instancePtr, HSteamUser hSteamUser, HSteamPipe hSteamPipe, const char * pchVersion);
S_API void SteamAPI_ISteamClient_SetLocalIPBinding(intptr_t instancePtr, uint32 unIP, uint16 usPort);
S_API class ISteamFriends * SteamAPI_ISteamClient_GetISteamFriends(intptr_t instancePtr, HSteamUser hSteamUser, HSteamPipe hSteamPipe, const char * pchVersion);
S_API class ISteamUtils * SteamAPI_ISteamClient_GetISteamUtils(intptr_t instancePtr, HSteamPipe hSteamPipe, const char * pchVersion);
...


"Flat" C-style means to me that some C++ classes reveal their public methods.
The first function is SteamAPI_ISteamClient_CreateSteamPipe. I take this as: within the SteamAPI there is a Steam class ISteamClient which has a method CreateSteamPipe().
In C++ one would call it like this: ISteamClient->CreateSteamPipe(). Or in Object Pascal: ISteamClient.CreateSteamPipe().
The return value HSteamPipe seems to be a handle. That's longint to me.
That instancePtr is puzzling for now. Just pointer.

According to my code above I would declare:



var
steamapi_isteamclient_createsteampipe: function(instanceptr: pointer): longint; cdecl = nil;
steampipe_handle: longint;


And in the above function steam_init I would add:



pointer(steamapi_isteamclient_createsteampipe) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_ISteamClient_CreateSteamPipe'));


Now if you call it like that ...



steampipe_handle := steamapi_isteamclient_createsteampipe(nil);


... you get Access Violation, which comes as no surprise.
Let me reveal for now, that Nil has to be replaced by a meaningful value, namely a pointer to ISteamClient, which is the Interface of the Steam Client.

To be continued.

[Chebmaster]

That instancePtr is puzzling for now. Just pointer.
Consider this:


type
TMyClass = class
function MyMethod(): boolean;
end;

But inside MyMethod implementation, you have access to automatically added variables Result and Self. Result is what received the function result while Self is the class instance from which this method was called:


var
MyInstance: TMyClass;
b: boolean;
...
b:= MyInstance.MyMethod;
..but where does Self come from?
It's not magic, it's not rocket science. Any class method is in fact a regular procedure/function with one hidden "Self" parameter in its parameter list. Both Pascal and C++, could be boiled to


function MyClass_MyMethod(Self: TMyClass): boolean;
Edit: virtual methods are the same it's the process of *calling* them that is more tricky.

In this form you can *export* it from, say, your DLL, C-style. I'm sure *lots* of windows API functions that require you first get a handle to object then pass that handle to various other functions, may very well be constructor and class methods in disguise. Because THandle is ptruint, a pointer-sized unsigned integer.
Things like TMyClass are typed pointers as well.

[Chebmaster]

P.S. A working sample from my GUI code that exploits this:
(inspited by web programming in JavaScript I use for living)


TEventSubscriptionMethodType = (
esmt_simple, //a method with no parameters
esmt_string //a method receiving single UnicodeString
);

TEventSubscription = record
Receiver: TChepersyObject ;
Method: AnsiString;
MethodType: TEventSubscriptionMethodType;
DefaultString: UnicodeString;
end;

function TControl.PassEvent(var ES: TEventSubscription; w: UnicodeString): boolean;
type tmethod_with_str_par = procedure(slf: TObject; par: UnicodeString);
var
method: procedure(slf: TObject);
rclass: CChepersyObject;
//method_with_str_par: procedure(slf: TObject; par: ansistring) //#$%@#. It compiles, works, but code completion is shoot >:( absolute method; //har har, two variables accupying the same address Ain't I clever?
//a hack: calling orbitrary class method by its string name.
//watch the parameter list! (shall either be none or one ansistring,
//with ClickReceiverParam either nil or a non-empty string accordingly!)
begin
if not Assigned(ES.Receiver) then begin
//...irrelevant to this topic
end
else begin
if w = '' then w:= ES.DefaultString;
Result:= Yes;
pointer(method):= ES.Receiver.MethodAddress(ES.Method);
if not Assigned(pointer(method)) then begin
AddLog(RuEn(
'Класс %0: не удалось передать событие, подписчик (%1) не имеет метода "%2" '#13' параметр: "%3"',
'Class %0: failed to pass event, the subscriber (%1) doesn''t have a method "%2" '#13' parameter "%3"'),
[self.ClassName, ES.Receiver.Classname, ES.Method, w]);
Exit(No);
end;
try
if ES.MethodType = esmt_simple then method(ES.Receiver)
else begin
tmethod_with_str_par(method)(ES.Receiver, w);
end;
except
Die(RuEn(
'Крах класса %0 при вызове подписчика события, %1.%2(%3)',
'Class %0 crashed at calling the event subscriber %1.%2(%3)'),
[self.ClassName, ES.Receiver.Classname, ES.Method, w])
end;
end;
end;

[klausvdl]

Thanks, Chebmaster! This adds some background knowledge and fits well what I found out in the meantime. That's reassuring.:)

Ok, all the following was a matter of tedious research and a bit trial-and-error. I knew almost nothing of it when I started this thread, and I wasn't confident:

1. Don't call SteamAPI_ISteamClient_CreateSteamPipe (see my last post). This is already called on the API's initialization. You get that handle from SteamAPI_GetHSteamPipe, which is declared in steam_api.h.

2. You have to connect to the Steam client's interface with SteamInternal_CreateInterface, which is declared in steam_api_internal.h. From this function you get a pointer to that interface, and you have to pass this interface pointer to every function whose name begins with "SteamAPI_ISteamClient_".
By the way, that's what the parameter instancePtr is intended for (see Chebmaster's post above).

3. From the Steam client you get pointers to all the other interfaces. In order to get the pointer to ISteamUtils for example you have to call SteamAPI_ISteamClient_GetISteamUtils.

4. Then you can use the member functions of each interface. And again you have to pass the appropriate interface pointer through that parameter instancePtr.

5. Additional requiremement: If you want to get a pointer to any of the above interfaces (see 2. and 3.), you have to specify the interface's version by passing a string.
The current version string is defined in the header file for each interface.
In isteamclient.h we find this one:

#define STEAMCLIENT_INTERFACE_VERSION "SteamClient017"
And in isteamutils.h:

#define STEAMUTILS_INTERFACE_VERSION "SteamUtils009"

Putting it all together:



unit steam;

{$H+}{$mode objfpc}

interface

uses
dynlibs, ctypes, sysutils;

const
steamlib = 'steam_api.dll';

function steam_init: longint;

implementation

var
steamlib_handle: tlibhandle = nilhandle;
steampipe_handle: longint;
pisteamclient: pointer = nil;
pisteamutils: pointer = nil;

var
steamapi_init: function(): boolean; cdecl = nil;
steamapi_shutdown: procedure(); cdecl = nil;
steamapi_gethsteampipe: function(): longint; cdecl = nil;
steaminternal_createinterface: function(chversion: pchar): pointer; cdecl = nil;
steamapi_isteamclient_getisteamutils: function(instanceptr: pointer; hsteampipe: longint; chversion: pchar): pointer; cdecl = nil;
steamapi_isteamutils_getappid: function(instanceptr: pointer): longword; cdecl = nil;

procedure steam_exit;
begin
steamapi_shutdown();
unloadlibrary(steamlib_handle);
end;

function steam_init: longint;
var
chversion: string;
appid: longword;
begin
steamlib_handle := loadlibrary(steamlib);

if steamlib_handle=nilhandle then exit(0); // no steam DLL, run without steam

addexitproc(@steam_exit);

pointer(steamapi_init) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_Init'));
pointer(steamapi_shutdown) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_Shutdown'));
pointer(steamapi_gethsteampipe) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_GetHSteamPipe'));
pointer(steaminternal_createinterface) := getprocedureaddress(steamlib_handle, pchar('SteamInternal_CreateInterface'));
pointer(steamapi_isteamclient_getisteamutils) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_ISteamClient_GetISteamUtils'));
pointer(steamapi_isteamutils_getappid) := getprocedureaddress(steamlib_handle, pchar('SteamAPI_ISteamUtils_GetAppID'));

if not steamapi_init() then exit(-1); // steam client not running, program should halt

steampipe_handle := steamapi_gethsteampipe();

chversion :='SteamClient017';
pisteamclient := steaminternal_createinterface(pchar(chversion));

chversion :='SteamUtils009';
pisteamutils := steamapi_isteamclient_getisteamutils(pisteamclient , steampipe_handle, pchar(chversion));

appid := steamapi_isteamutils_getappid(pisteamutils);

// output to logfile: appid

result := 1; // ok
end;

end.

Ok, this really works!

But it's not enough to make use of common Steam features. Most functions have to get data from the Steam server, or they send data to it. This is done asynchronously, and therefor you have to work with callback functions or callback results. There is a C++ template thingy that I have to puzzle out, and of course there are more internal functions involved. Would be to easy otherwise.

To be coninued.

[klausvdl]

Ok, first of all: it is possible to make use of Steam's callback mechanism.

The tricky thing is that you have to pass a C++ class as an argument. So you have to mimic this class by the means of Pascal. This requires defining a record for example and arranging it's internals. It needs a tabel of virtual functions and some parameters. Also you have to mimic the calling convention thiscall which requires to read out the ECX register by an inline asm instruction. Fortunately that C++ class is not too big.

Moreover I found an old discussion on Steam dev forums where two guys worked out that stuff, one of them Relfos. ;)

It appears in Relfos' unit SteamCallback.pas (https://github.com/Relfos/steamworks_wrappers/blob/master/headers/SteamCallback.pas) (Object Pascal) and although his whole project is based on Steamworks.NET (CSteamworks.DLL) that callback handling part doesn't really need that, unless I am mistaken.

As for myself I don't use classes in my implementation. By now I'm using the callback mechanism only for one thing: every time the player activates/deactivates the Steam overlay (Shift+Tab) I pause/unpause the game. Some questions remain regarding the handling of calling conventions on CPU64 and/or Linux etc.
Now I should be able to use common Steam features (achievements, leaderboard, save files to cloud etc.) We'll see about that.

If anyone wants to see plain Pascal code then I can provide another listing. Otherwise I'm pointing to Relfos' unit. :)

[Chebmaster]

Maybe this is worth worth feature request for adding thiscall convention to future fpc?

P.S. and

Asm
mov myself, ECX;
End;

should be


{$ifndef cpu64}
Asm
mov myself, ECX;
End['ecx'];
{$endif}

[klausvdl]

I guess it's not clearly readable (nested), but Relfos did it just the other way round:


{$IFDEF CPU64}
...
{$ELSE}
...
Asm
mov myself, ECX;
End;
...
{$ENDIF}


Hm, adding that calling convention thiscall would probably make little sense without additional mechanisms for accessing C++ classes. Would be nice if one could declare a whole pascal class as "cppdecl" or whatever, so that somehow the linker would be able to access the C++ classes symbols/properties within a DLL ... ???

[klausvdl]

I have been asked for an additional listing covering Steam achievements.
So this is my current unit "steam" which deals with achievements and stats (integer stats only).

Please note: This only works with the compiler target Win32 for now!

https://docs.google.com/document/d/10O1GtdrLxHQxYbDQ9KyG5DcGLofBRnRAWFbxprcDcpA/edit?usp=sharing
(https://docs.google.com/document/d/10O1GtdrLxHQxYbDQ9KyG5DcGLofBRnRAWFbxprcDcpA/edit?usp=sharing)
The following is a rough game framework in order to show how to use the unit.



program mygame;
uses
...
steam;

procedure update_event;
begin
steam_update;
...
end;

procedure run_level;
begin
...
repeat
update_event;
...
if steam_overlay then begin
if not gamepaused then start_break;
end
else begin
if gamepaused then finish_break;
end;
...
until quit;
...
end;

procedure mainloop;
begin
...
repeat
update_event;
steam_debuguserstats;
...
run_level;
...
until quit;

if not mission_survived then begin
incandstore_istat(1); // stat "NumLosses"
i := get_istat(1);
case i of
// died 1x
1: setandstore_achievement(0); // Lazarus
// died 9x
9: setandstore_achievement(1); // cat
// died 99x
99: setandstore_achievement(2); // Coyote
end;
end;
...
end;

begin
...
if steam_init({your APP ID})=-1 then halt;
...
mainloop
end.

[JernejL]

Not bad update, that's something at least.

May i propose that a steam -> c wrapper library could be automaticly generated with including pascal headers in something like a scripting language like php? i had a lot of luck with this for newton dynamics.