Dllama - Local LLM Inference

[drezgames]

logo.png

A simple and easy to use library for doing local LLM inference directly from your favorite programming language.

Download
tinyBigGAMES/Dllama: Local LLM inference Library (github.com)

Simple example

uses
  System.SysUtils,
  Dllama;


begin
  // init
  if not Dllama_Init('config.json', nil) then
    Exit;
  try
    // add message
    Dllama_AddMessage(ROLE_SYSTEM, 'You are a helpful AI assistant');
    Dllama_AddMessage(ROLE_USER, 'What is AI?');
    
    // do inference
    if Dllama_Inference('phi3', 1024, nil, nil, nil) then
    begin
      // success
    end
  else
    begin
      // error
    end;
  finally
    Dllama_Quit();
  end;
end.

Media

[SilverWarior]

I just tried this project of yours and was not impressed. While it seemed that some text was being written by the AI the output was incoherent mess of various words. I don't have enough knowledge on this topic to even begin to guess what might be wrong.

[Jonax]

I'm afraid I didn't even try the project. So much to do, so little time. Besides I'm no github user. So can't really comment at all other than the general point that it's good to see some pascal activity. Keep up the good work.

[drezgames]

I just tried this project of yours and was not impressed. While it seemed that some text was being written by the AI the output was incoherent mess of various words. I don't have enough knowledge on this topic to even begin to guess what might be wrong.

Hmm, which Test did you run? What type of output did you get? Which model did you use? Did you change the template format in config.json? If it's not correct for example, you will not get correct output. Are you running in GPU or CPU mode, etc.

The fact that local LLM inference was not even possible 2-3 years ago on consumer hardware is impressive in itself! Amazing how fast AI is advancing.

If you can give some info about the problem, i'm sure we can get to the bottom of it. I want everyone to be able enjoy AI.

[drezgames]

I just tried this project of yours and was not impressed. While it seemed that some text was being written by the AI the output was incoherent mess of various words. I don't have enough knowledge on this topic to even begin to guess what might be wrong.

Thx!