Native pascal code for templates

[darnocian]

The template engine is currently based off a parser and interpreter. This is pretty useful for many purposes especially where dynamic interpretation requirements exist. However, templates can take time to evaluate by the interpreter depending on the logic, even if it is in the order of ms, it is still suboptimal.

Once an app and templates are mature/ ready for release, some circumstances will allow for templates to be fixed. The benefit of the design, is that most of the template language constructs map onto Delphi language constructs. We can introduce a pre-processor code generation step that emits code that can be registered with the TTemplateRegistry, allowing for native compiled speed when it comes to output being generated. Even then, there can still be a fallback to interpretation if required.

This may have some implications:

• <% require ('TType') %> becomes a requirement in a template so we can identify the type of data available so we can use native types. Otherwise, we need to revert to using variants which is less desirable.
• max run time may become an optional feature. if you have tested your script, it could be disabled. the main implication is in loops actually in case there is some bad logic.
• dev's CI or the IDE would need to call a command line app (sempare.template.nativegenerator) to generate the include file/unit.
• might need to enforce a few other requirements regarding type management. e.g. ternary will need to enforce types returned from true/false evaluation to be of the same type.

comment below if interested. This may be available only to commercial supporters.

[darnocian]

I've done a small proof of concept so far. There may be some guidelines to how to write templates that can be native code. Further, my approach allows for native code to co-exist with interpreted code.

e.g.
unit: Model.pas contains:

unit Model;
interface
type
  TIndex = record
    StartIdx : integer;
    EndIdx : integer;
  end;
implementation
end.

Code generator on a template:

<% require 'Model.TIndex' %>
<% for i := StartIdx to EndIdx %>
   <% i %>
<% end %>

creates code the code:

unit MyTemplates;
interface
procedure InitTemplates;
implementation
uses
    Model,
    Sempare.Template,
    Sempare.Template.CodeGen;
procedure InitTemplates;
begin
    Template.Resolver.SetTemplate('for_in_1', TNativeTemplate.Create('for_in_1',
        procedure (const _AContext: INativeContext)
        begin
            var _LValue := _AContext.Value.AsType<Model.TIndex>();
            _AContext.Write(#13#10);
            for var i := _LValue.StartIdx to _LValue.EndIdx do begin
                _AContext.Write(#13#10'   ');
                _AContext.Write(_AContext.Encode(i));
                _AContext.Write(#13#10);
            end;
        end));
end;
{$IFNDEF DISABLE_SEMPARE_AUTO_INIT}
initialization
    InitTemplates;
{$ENDIF}
end.

All that is required is to include the unit in the project. The unit injects the native template into the template registry, so transitioning from having interpreted code to having native code is 'right click - add unit' once the code generation step is done. Ok - setting up code generation currently requires a little bit of work, but again something I hope can be automated.

The nice thing in the above, the code generator uses the information from the <% requires 'Model.TIndex' %> to identify the TIndex record and then binds natively onto the parameters, without needing RTTI. Unit references to types would be automatically included. There is still a little RTTI in terms of the TValue, but it is just a container for the data payload.

Benchmark results:

Benchmark: Native template (Runs: 100000)
Done in 312ms (or 0.00312ms per run) 
Benchmark: Interpreted template (Runs: 100000)
Done in 4348ms (or 0.04348ms per run)

That is almost 14 times faster! We sort of expect that, but nice to see the improvement of a simple for loop using native code vs interpreted code using RTTI.

[darnocian]

Been experimenting with LLVM. Possibly, we just use the LLVM JIT.