Robert Kosek
26-05-2007, 02:34 PM
I'm going to do something I don't normally do, but I'll give full source along with this. It's just a test bed, but part of a bigger project.
One of my annoyances is that data is either arbitrarily compiled, thus never changing except when you recompile, or or stored in an external format. Sometimes these external formats are just too heavy and bloated for a given purpose. Thus I was working on a makeshift parser and tokenizer which relies heavily upon the use of variants. If these wacky bugs continue I swear I'll switch to FPC!
Anyway. I made a very simple console application that takes an input string and tokenizes it into an array of strings. From there the first is a command followed by data. So to permit things like mixing integers and floats I started using variants. There are only 4 functions:
Sum: totals all following numbers regardless of format or type. Works like a charm.
Sub: Subtracts number 2 from number 1; also works like a charm.
Power: Calculates x1 to the x2 power; dies like Dracula with a stake in his chest.
Exit: <_<
What I have determined through the use of breakpoints is that in power the two numbers are never assigned, and that the output is never written. Almost as if that part of the code just isn't called at all; but is should be.
Anyone have any clues? I'm using Turbo Delphi with the hotfix rollup with the following code. I'm about to go mad! I have tried 3-4 different ways of assignment but I can't get variants in this section to function, or breakpoints either (anymore). Oddest of all, when I first added the functionality it WORKED -- now it doesn't. I will assuredly need variants once I start the true function call implementations!
Remember, I am showing this for troubleshooting purposes; don't use it without asking me first. Here's the code:
program tokenizer;
{$APPTYPE CONSOLE}
uses
SysUtils, Math, Variants;
type TDynStrArray = array of string;
function tokenize(line: string): TDynStrArray;
var temp: string;
l,i: cardinal;
inStr,wasStr: boolean;
begin
l := 0;
temp := line;
instr := false;
while temp <> '' do begin
if l > length(result)-1 then
SetLength(result, l+8);
wasStr := false;
for i := 1 to Length(temp) do
case temp[i] of
'`': begin inStr := not inStr; wasStr := true; end;
' ',#0: if not inStr then Break;
end;
if not wasStr then
result[l] := copy(temp,1,i-1)
else
result[l] := copy(temp,2,i-3);
Delete(temp,1,i);
inc(l);
end;
setlength(result,l);
end;
var inputtokens: TDynStrArray;
input: string;
i: integer;
exit: boolean = false;
v1,v2: variant;
begin
writeln('Enter tokens seperated by spaces, strings delimited by `''s.');
repeat
v1 := 0;
v2 := 0;
input := '';
setlength(inputtokens,0);
write('> '); readln(input);
writeln;
if input <> '' then begin
inputtokens := tokenize(input);
for i := 0 to Length(inputtokens) - 1 do
writeln('Token'#9'#',i,': "',inputtokens[i],'"');
writeln;
if inputtokens[0] = 'sum' then begin
v1 := Variant(inputtokens[1]) + 0;
for i := 2 to length(inputtokens) - 1 do begin
v2 := inputtokens[i];
v1 := v1 + v2;
end;
writeln('Sum:'#9,v1);
end else if inputtokens[0] = 'sub' then
if length(inputtokens) >= 3 then
writeln('Sub:'#9,Variant(inputtokens[1]) - Variant(inputtokens[2]))
else if inputtokens[0] = 'pwr' then
if length(inputtokens) >= 3 then begin
v1 := 0.0 + Math.Power(StrToFloatDef(inputtokens[1],0),StrToFl oatDef(inputtokens[2],0.0));
writeln('Power:'#9,v1);
end else writeln('Error, PWR given too few parameters.')
else
writeln('Unknown command "',inputtokens[0],'"');
exit := pos('exit',input) > 0;
end;
writeln;
until (exit);
end.
(Someone needs to look into posting... I had to disable HTML to post this at all; it died between the list and the code, but the code wasn't wrapped correctly.)
One of my annoyances is that data is either arbitrarily compiled, thus never changing except when you recompile, or or stored in an external format. Sometimes these external formats are just too heavy and bloated for a given purpose. Thus I was working on a makeshift parser and tokenizer which relies heavily upon the use of variants. If these wacky bugs continue I swear I'll switch to FPC!
Anyway. I made a very simple console application that takes an input string and tokenizes it into an array of strings. From there the first is a command followed by data. So to permit things like mixing integers and floats I started using variants. There are only 4 functions:
Sum: totals all following numbers regardless of format or type. Works like a charm.
Sub: Subtracts number 2 from number 1; also works like a charm.
Power: Calculates x1 to the x2 power; dies like Dracula with a stake in his chest.
Exit: <_<
What I have determined through the use of breakpoints is that in power the two numbers are never assigned, and that the output is never written. Almost as if that part of the code just isn't called at all; but is should be.
Anyone have any clues? I'm using Turbo Delphi with the hotfix rollup with the following code. I'm about to go mad! I have tried 3-4 different ways of assignment but I can't get variants in this section to function, or breakpoints either (anymore). Oddest of all, when I first added the functionality it WORKED -- now it doesn't. I will assuredly need variants once I start the true function call implementations!
Remember, I am showing this for troubleshooting purposes; don't use it without asking me first. Here's the code:
program tokenizer;
{$APPTYPE CONSOLE}
uses
SysUtils, Math, Variants;
type TDynStrArray = array of string;
function tokenize(line: string): TDynStrArray;
var temp: string;
l,i: cardinal;
inStr,wasStr: boolean;
begin
l := 0;
temp := line;
instr := false;
while temp <> '' do begin
if l > length(result)-1 then
SetLength(result, l+8);
wasStr := false;
for i := 1 to Length(temp) do
case temp[i] of
'`': begin inStr := not inStr; wasStr := true; end;
' ',#0: if not inStr then Break;
end;
if not wasStr then
result[l] := copy(temp,1,i-1)
else
result[l] := copy(temp,2,i-3);
Delete(temp,1,i);
inc(l);
end;
setlength(result,l);
end;
var inputtokens: TDynStrArray;
input: string;
i: integer;
exit: boolean = false;
v1,v2: variant;
begin
writeln('Enter tokens seperated by spaces, strings delimited by `''s.');
repeat
v1 := 0;
v2 := 0;
input := '';
setlength(inputtokens,0);
write('> '); readln(input);
writeln;
if input <> '' then begin
inputtokens := tokenize(input);
for i := 0 to Length(inputtokens) - 1 do
writeln('Token'#9'#',i,': "',inputtokens[i],'"');
writeln;
if inputtokens[0] = 'sum' then begin
v1 := Variant(inputtokens[1]) + 0;
for i := 2 to length(inputtokens) - 1 do begin
v2 := inputtokens[i];
v1 := v1 + v2;
end;
writeln('Sum:'#9,v1);
end else if inputtokens[0] = 'sub' then
if length(inputtokens) >= 3 then
writeln('Sub:'#9,Variant(inputtokens[1]) - Variant(inputtokens[2]))
else if inputtokens[0] = 'pwr' then
if length(inputtokens) >= 3 then begin
v1 := 0.0 + Math.Power(StrToFloatDef(inputtokens[1],0),StrToFl oatDef(inputtokens[2],0.0));
writeln('Power:'#9,v1);
end else writeln('Error, PWR given too few parameters.')
else
writeln('Unknown command "',inputtokens[0],'"');
exit := pos('exit',input) > 0;
end;
writeln;
until (exit);
end.
(Someone needs to look into posting... I had to disable HTML to post this at all; it died between the list and the code, but the code wasn't wrapped correctly.)