For the needs of a project, I wrote a little template library some time ago. However, I never really liked it and didn't really finish it. On the good side, I made it flexible with customizable markers, so I was able to use {{ }} or something else as markup. On the bad side, I was using regular expressions and simple search to carve out things in markers to be expanded. Another annoyance is that Emacs already has so many templating and text processing libraries built-in, so I dislike the idea of using yet another one. I am already using org-capture for other purposes, and org-capture has templates. Could I possibly use those templates instead of an extra dependency?
It turns out there are some annoyances in terms of how they expand templates. For example, org-capture will automatically untabify the template. In a code generator, this is of course not desirable. Imagine generating a Makefile or a Python source file where tabs are structural elements which can't be removed. Org-capture also adds a newline to the generated text, forcing me to trim the final result, which causes extra memory allocations. In general, org-capture-fill-template, the function where the relevant work is done, does a bit more than what I actually need. Most important of all, it is somewhat awkward to use embedded Lisp due to how they process %() placeholder. Also, while it can be used in other contexts, the function is tightly coupled with org-mode too.
Instead, while I was experimenting, it come to my mind I could use structural movement rather than regular expressions to extract the content to be expanded. While regular expressions can be made to work in this context, they seem to lead to more processing than needed, and they seem to be hard to get correct in all contexts.
For some sort of content-aware parsing, Emacs has syntax tables, as they call them. By modifying syntax, as described in the manual, we can tell Emacs which characters starts and ends a synctatic element. Since placeholders, or markers, start and end with a character, this seems like it could work in this case. It also turnes out, it is relatively easy way to add this:
(defvar template-fill-syntax (make-syntax-table))
(aset template-fill-syntax ?< '(4 . 62))
(aset template-fill-syntax ?\( '(4 . 41))
(aset template-fill-syntax ?\[ '(4 . 93))
(aset template-fill-syntax ?{ '(4 . 125))That is about what we need to tell Emacs we are using <>, (), [] and {} as delimiters, for my purpose here. By using those characters as list delimiters, we can tell Emacs to treat the content between them in the same way as it treat Lisp lists, so we can move back and forth in a text buffer in a structured way. For those not familiar with Lisp and structural movement a lá Paredit, one can think of tree-sitter perhaps.
In this case, I need just to move forward in a template until we find matching closing character, with forward-list. Emacs will take care of counting opening and closing characters along the way:
(defmacro with-template-string (&rest body)
(with-syntax-table template-fill-syntax
`(progn
(delete-char -1) ;$
(let* ((beg (point))
(open-char (char-after))
(end (progn (forward-list) (point)))
(close-char (char-before))
(it (buffer-substring-no-properties (1+ beg) (1- end))))
(delete-region beg end)
(condition-case error
(princ ,@body (current-buffer))
(error
(insert (format "$!%ccould not insert %s: %s%c"
open-char close-char it error))))))))The little macro above is written as anaphoric macro. It makes available the content between the end markers as a string to code which can than read it as lisp code, or process it as plain text, or do whatever it wants with it. The full code for my template expander which is heavily inspired by org-capture templates turnes to be relatively simple after I had that macro:
(defun template-fill (template &rest properties)
"Fill a TEMPLATE and return the filled template as a string.
TEMPLATE is a string in which following placeholders are allowed:
$[filename] - insert content of FILENAME
$(expression) - evaluate lisp EXPRESSION and insert the result
$<timeformat> - insert time formatted according to TIMEFORMAT
${ENV} - insert value of environment variable ENV
PROPERTIES is a list with properties that can be passed to the template
parser. Following properties are recognized:
:timestamp - use insted of current-time in TIMEFORMAT template.
:target-file - save expanded template into a file, otherwise return as a string"
(cl-macrolet
((with-template-string (&rest body)
(with-syntax-table template-fill-syntax
`(progn
(delete-char -1) ;$
(let* ((beg (point))
(open-char (char-after))
(end (progn (forward-list) (point)))
(close-char (char-before))
(it (buffer-substring-no-properties (1+ beg) (1- end))))
(delete-region beg end)
(condition-case error
(princ ,@body (current-buffer))
(error
(insert (format "$!%ccould not insert %s: %s%c"
open-char close-char it error)))))))))
(cl-labels
((escaped ()
(when (and (> (- (point) (point-min)) 1)
(= (char-before (1- (point))) ?\\))
(forward-char -1)
(delete-char -1)
(forward-char)
t)))
(with-temp-buffer
(setq buffer-file-name nil)
(insert template)
(goto-char (point-min))
(while (search-forward "$" nil t)
(pcase (char-after)
(?\[ (unless (escaped)
(with-template-string
(insert-file-contents-literally (expand-file-name it)))))
(?\( (unless (escaped)
(with-template-string (eval (read it)))))
(?< (unless (escaped)
(with-template-string
(format-time-string
it (or (plist-get properties :timestamp) (current-time))))))
(?{ (unless (escaped)
(with-template-string (or (getenv it) ""))))))
(let ((file (plist-get properties :target-file)))
(if file
(write-region nil nil file 0)
(buffer-substring-no-properties (point-min) (point-max))))))))Usage is realtively straightforward:
(template-fill "${HOME}") => "/home/arthur"
(template-fill "$(user-full-name)") => "Arthur Miller"
(template-fill "$((+ 2 3))") => "5"
(template-fill "$('(+ 2 3))") => "(+ 2 3)"
(template-fill "\\$((+ 2 3))") => "$((+ 2 3))"
(template-fill "$<%Y-%m-%d %H:%M>") => "2026-01-02 07:47"The most interesting thing here is the embedded lisp expansion with $() placeholder. Any Lisp expression will be evaled and the result inserted in the place of the $() placeholder. The placeholder, or the code markers, is not part of the expression itself. In other words, in $(expression), the expression is a symbolic expression, or a Lisp expression, and the same rules for evaluation are as in repl with M-: for example (it uses eval under the hood).
Finally, the most useful to me is $[some-file] placeholder, which lets me insert content of some-file. If some-file contains templates, they will get expanded as well, which is what I need to write code and text generators.
As a little note about the implementation: I really didn't want to have each placeholder handled explicitly in the pcase above. I discovered after some testing that it was quite difficult to write a function that will work well in all contexts, especially since I want to manipulate Lisp objects in the $() context, and text objects in the $<> placeholder. I was testing with both greedy and non-greedy regexes, and found that both have problems in some contexts.
As a first version of this, I actually wrote a small paren-counting function, so I could count opening and closing delimiters I care about. However, doing it correctly turned out to not be trivial at all, since even such a function has to be somewhat aware of the context, at least if we want to allow say something like lisp. For example, consider %() where we have an odd number of parentheses:
(template-fill "$((print \"Hard one: ')\"))") => "Hard one: ')"It is not impossible, but not so easy either, since we have to be aware of the context, for example, the quoted symbol ').
I don't think the syntax table approach is fool-proof either, but thus far, Emacs seems to handle it well, and we get it basically for free, so let see how well it goes. I would be happy to hear if someone has a better approach to this.
This is also my first post on this blog. I have just set it up, so there might be bugs. I really have no idea how one set up comments in Hugo, if it is even possible on Codeberg serving static web pages. That means, you will unfortunately have to catch me on Reddit, or send me an email if you have a better suggestion for solving this.