What Comes to Mind

An Infix Backtick Operator for C++

Steve Downey — Mon, 29 Jun 2026 12:00:00 GMT

I am working on a proposal to let any callable be written between two backticks as an infix binary operator: x `f` y means exactly f(x, y). It is borrowed from Haskell, it desugars to an ordinary call, and I have it working in both Clang and GCC. This post is the announcement and the design tour.

1. The pitch in one line

C++ lets you spell a handful of binary operations infix — a + b, a < b, a | b — and lets you overload them. Everything else is a call: min(a, b), dot(u, v), gcd(m, n), approx_equal(x, y). The proposal is to erase that distinction at the call site. Put a callable between backticks and it becomes a binary operator:

a `plus` b              // plus(a, b)
a `std::min` b          // std::min(a, b)
m `at` k                // at(m, k)
u `dot` v               // dot(u, v)
x `approx_equal` y      // approx_equal(x, y)

The text between the backticks is not a name the language knows about. It is an arbitrary call-eligible expression, and the whole thing is defined to be the call you would have written anyway. There is no new overloadable operator` , no operator table to register, no fixity declarations. x `f` y is f(x, y), and that is the entire semantic.

Haskell programmers will recognise this immediately: `div`, `mod`, `elem` are exactly this construction. The backtick is available in C++ because today it has no meaning outside string, character, and raw-string literals, all of which are lexed before the punctuator stage and are therefore untouched.

2. It desugars to a call, and that is the whole point

The single most important design decision is that the operator lowers to a call expression in the front end, before overload resolution runs. It is not a macro, not a syntactic rewrite with its own rules, not a new AST category with its own type rules. It builds the same call node the compiler would build for f(x, y).

That one choice means the feature inherits, for free and correctly:

overload resolution and ADL,
templates and SFINAE,
constexpr evaluation,
conversions, value categories, and lifetime,
code generation.

None of it is re-implemented. If f(x, y) compiles and does a thing, then x `f` y compiles and does the same thing. If f(x, y) is ambiguous or ill-formed, so is the backtick form, with the same diagnostic. The feature has essentially no semantics of its own to get wrong, which is exactly what you want from sugar.

A few consequences fall straight out of "it is just the call":

x `T` y                 // == T(x, y) : functional-style construction; CTAD applies
a `std::pair` b         // == std::pair(a, b)

A type name is callable, so a type in the slot constructs. Evaluation order is the call's evaluation order: operand order is unspecified, and since C++17 the callee — here, the thing between the backticks — is sequenced before both operands, even though it is written between them. Nothing new to learn; if you know how a call behaves, you know how this behaves.

3. How it parses

Two questions decide how an infix operator feels: how tightly it binds, and how a chain groups.

Precedence. Backtick is the highest-precedence binary operator — tighter than *, looser than the unary/prefix operators. Both operands are cast-expressions, so prefix operators attach symmetrically:

-a `f` -b               // f(-a, -b)     symmetric, like every other binary op
a * b `f` c             // a * f(b, c)   binds tighter than *

I considered making it bind tighter than unary minus so that -x `f` y would mean -f(x, y). That was rejected: it would make backtick the only operator in the language where a leading - floats out of its operand, so -a `f` -b would mean -f(a, -b) — asymmetric and hard to teach. Matching every other binary operator won.

Associativity. Left, like reading order:

a `f` b `g` c           // g(f(a, b), c)

The slot. The thing between the backticks is parsed as an assignment-expression — anything you could write as the callee of a call, excluding a top-level comma. Qualified names, member-access expressions, even a lambda, are all fine.

4. The one genuinely tricky part

The open and close delimiter are the same token. A naive parser, having read x `f, would see the closing backtick and think a second backtick operator was starting. C++ already has exactly this problem with > inside template argument lists — vector<vector<int>> — and solves it with a parser flag that says "right now, > is not an operator." I do the same thing: a BacktickIsOperator flag in Clang (modelled on GreaterThanIsOperator), a backtick_is_operator_p in GCC (modelled on greater_than_is_operator_p), false while parsing the slot and restored inside nested parentheses.

A side effect of the same-token delimiter is that there is no such thing as a bare nested backtick. The first interior backtick always closes the slot, so what looks like "nesting" is token-identical to an ordinary left-associative chain — and a chain is exactly what it parses as. To actually nest, you parenthesise:

x `f `g` h` y           // a chain:  h(f(x, g), y)
x `(f `g` h)` y         // nested:   (g(f, h))(x, y)

This is the same answer-changes-on-regrouping situation as a - b - c versus a - (b - c), which no compiler diagnoses because subtraction is left-associative and the parentheses just override the grouping. Backtick is no different. The language defends against honest mistakes, not against someone who builds a type that is simultaneously callable, value-convertible, and asymmetric and then omits the parentheses on purpose.

5. A second use for the same token: escaping keywords

Because the proposal claims the backtick as a punctuator, it is worth asking what else the token could do, and there is one obvious companion use: escaping a keyword so it can name an entity. This is the escape hatch that lets a future keyword avoid breaking existing code that already used that word as an identifier — Swift's `class`, Kotlin's backtick identifiers, F#'s double-backtick names, Rust's r# raw identifiers all do a version of it.

void `new`();           // declares a function named "new"
`new`(a, b);            // calls it
obj.`delete`();         // member named "delete"

The two uses never collide because they live in mutually exclusive grammatical positions. In operand, primary, or declarator position the grammar wants a name, so a backtick opens a keyword-escape. In post-operand position the grammar wants a binary operator, so a backtick is the infix operator. This is the same position-based disambiguation C++ already does for *, &, and <. The escape yields an ordinary identifier, so lookup, mangling, linkage, and ABI are unchanged — void `new`(); links as a function named new.

The two uses share one lexical token and one committee, so I am proposing them together, in one paper, rather than letting two independent designs drift into contradiction.

6. What it is good for

The shallow answer is readability for binary operations that read better infix: lo `clamp` hi, a `gcd` b, p `implies` q, lhs `approx_equal` rhs, set_a `intersect` set_b. Named operations stop being visually second-class to the dozen built-in symbols.

The deeper answer is that the slot is an arbitrary callable and the operator chains, and those two facts reach further than they first appear. A one-line helper turns backtick into a left-to-right threading operator:

inline constexpr auto pipe =
    [](auto&& x, auto&& f) -> decltype(auto)
    { return std::invoke(std::forward<decltype(f)>(f),
                         std::forward<decltype(x)>(x)); };

x `pipe` f `pipe` g `pipe` h        // h(g(f(x))) — data-flow order

The decisive case is that this drives the existing range adaptor closures unchanged. views::filter(pred) and views::transform(fn) are already unary callables — c | a is defined as a(c) — so pipe feeds them directly, same result and same laziness as the | pipe, with no bespoke operator| overloads involved:

r `pipe` views::filter(pred) `pipe` views::transform(fn)

And it reopens something the language otherwise reserves to a fixed set of built-ins. You cannot get short-circuit evaluation back by overloading && or || — an overloaded one evaluates both sides. But a helper whose right operand is a callable controls whether and when that side runs:

// short-circuiting logical implication:  p ==> q  ≡  !p || q
inline constexpr auto implies =
    [](bool p, auto&& q) -> bool { return !p || q(); };

p `implies` [&]{ return expensive(); }   // q() runs only when p holds

The monadic / fallible chain is just the zero-ceremony special case of this, where the stages are already functions:

parse(s) `mbind` validate `mbind` store;   // stops at the first error

None of these helpers are part of the proposal — the paper is language-only, and each helper is a few lines of ordinary user code. They are here as motivation, to show the operator's reach. Field experience can later decide which, if any, earn a place in a library.

7. It is not the pipeline operator, and does not want to be

The obvious neighbour is Barry Revzin's |> (P2011, the "pizza" operator). Both bottom out in a call and their two-argument cases coincide — a `plus` b, a |> plus(b), and plus(a, b) are the same call. But they are different shapes and they compose rather than compete:

Backtick is symmetric binary infix: the callee sits between two operands, the result is an overload-resolved call, and the right-hand side is a single value. Beyond two operands it cannot thread.
|> is a non-overloadable syntactic rewrite: x |> f(a, b, c) prepends x to an arbitrary-arity call. It binds low, threads through stages, and is the right tool for chains.

So backtick supplies infix detail inside a pipeline stage, |> threads the value between stages:

r |> filter([](auto e){ return e `mod` 2 `eq` 0; }) |> sum()
//                              \____ eq(mod(e, 2), 0) ____/

Backtick deliberately leaves the |> spelling unspelled so the two can coexist in one program.

8. Implementation status

This is not a paper design with a hand-wave at implementability. The operator and the keyword-escape are both prototyped, gated behind an opt-in -fbacktick flag, in two independent compilers. Both forks are public, on a backtick branch:

Clang (steve-downey/llvm-project @ backtick) — lexer token, precedence level, parser hook, Sema desugar to a CallExpr, driver flag, a transparent AST wrapper so -ast-print round-trips the surface syntax, and clang-format support for both uses. Full ADL. Tests live under clang/test/ (the backtick-* files in Parser/, Lexer/, and SemaCXX/).
GCC (steve-downey/gcc @ backtick) — libcpp token, parser precedence and slot handling, desugar via finish_call_expr, the same flag, and full ADL on the slot. Tests live under gcc/testsuite/g++.dg/backtick/.

Two independent implementations that agree is the strongest evidence of implementability I can bring to EWG/CWG. Every pipeline pattern above was compiled and run against the built -fbacktick Clang, so the §"what it is good for" claims are implementation experience, not assertion.

9. What I am proposing, and what comes next

A pure core-language feature, targeting C++29: the infix backtick operator plus the keyword-escape, in one paper, with no standard-library additions. The desugar-to-a-call MVP is the whole language change; the AST wrapper and a Compiler Explorer deployment strengthen the story but do not block it.

If you have a binary operation that has always read worse as f(x, y) than it would as x `f` y — a domain predicate, a metric, a combinator — that is the use case, and I would like to hear about it. The two compiler forks linked above are public if you want to build the branch and try it; comments, objections, and motivating examples are all welcome — reach me at sdowney@gmail.com — before this goes in front of the committee.

Refreshing a Stale Git Subtree

Steve Downey — Mon, 29 Jun 2026 03:37:51 GMT

I write my WG21 papers with MPark/WG21, a Pandoc-based framework I vendor into the paper repo as a git subtree. The framework had a major overhaul–the build system split apart, and Pandoc jumped from 2.18 to 3.9–and my copy was 98 commits behind. Worse, the subtree had drifted in two directions at once: real local patches for a TLS-intercepting corporate network, and a pile of pointless autoformatter churn from my own pre-commit hooks. This is how I dragged it back to a verbatim copy of upstream, moved to the new flat.mk include, and pushed every local change back out of the subtree so the next update is a one-liner.

1. What the overhaul changed

A few things upstream matter to anyone who integrates MPark/WG21.

The Make machinery was split. The old monolithic Makefile–the one you used via include wg21/Makefile–became three fragments: wg21.mk is the engine you never include directly, flat.mk is for all-papers-in-one-directory, and paper.mk is for one-directory-per-paper. The old include wg21/Makefile still works through a backward-compat shim, but it nags you with a deprecation warning on every build and points you at flat.mk.

Pandoc went from 2.18 to 3.9.0.2. That brings MathJax to MathML, stricter raw-HTML handling, and changed citeproc behavior. Rendering can move under you, so this is the part to actually look at rather than assume.

And a few behaviors shifted. Bare make now builds HTML, LaTeX, and PDF; it used to build only PDF. make clean now removes only the output directory, and wiping the downloaded toolchain is make distclean. The generated stable-reference file was renamed srefs.md to srefs.defs.

2. Where I started, and why it was a mess

The layout was an ordinary flat one.

transcode/
`-- papers/
    |-- Makefile           # one line: include wg21/Makefile
    |-- transcode-view.md  # the actual paper
    |-- TEST.md
    `-- wg21/              # the framework, as a git subtree

The trouble was all hiding inside papers/wg21/. A subtree vendors the framework's files directly into your tree, which is the whole point–and also the whole problem. Anything that edits files in your repo edits the framework too, and you won't necessarily notice.

So two things had happened. My pre-commit config didn't exclude the subtree, so ruff and friends had quietly reformatted wg21.py (489 lines of pure churn), refs.py, and others. None of it changed behavior. All of it was a future merge conflict. On top of that I had genuine local patches for a corporate MITM-TLS network: a CA-bundle export, verify=False in refs.py where it fetches wg21.link, a switch from pip to uv, and a truststore dependency.

One pile I wanted to throw away. The other I needed to keep. Neither belonged in the subtree, which is the lesson I'll keep repeating: the vendored copy should be a verbatim mirror of upstream and nothing else.

3. Branch, and a baseline to diff against

git switch -c chore/update-wg21-subtree

# Build the current (Pandoc 2.18) output, to diff against later.
cd papers && make transcode-view.html
cp generated/transcode-view.html /tmp/baseline.html

Crossing a Pandoc major version, I want to see what moved, not trust that nothing did. The baseline costs a couple of minutes and pays for itself.

4. Refreshing the subtree

git subtree pull refuses to run with a dirty tree, so stash anything unrelated first, then pull from upstream master, squashed, the same way the subtree was originally adopted.

git stash push -m park-unrelated -- some/unrelated/file
git subtree pull --prefix=papers/wg21 \
    https://github.com/mpark/wg21.git master --squash \
    -m "Update wg21 subtree to mpark/master (Pandoc 3.9, Make overhaul)"

Because my subtree had diverged, this conflicted, exactly as it should. Six files collided on content, and two were modify/delete conflicts where upstream had deleted files I'd touched. My goal was simple, so the resolution was simple: the subtree should end up identical to upstream master. Take theirs, everywhere.

# Take upstream ("theirs") for the content conflicts, then stage them.
git checkout --theirs -- papers/wg21/Makefile \
    papers/wg21/data/filters/citetitle.py \
    papers/wg21/data/filters/wg21.py \
    papers/wg21/data/templates/wg21.latex.patch \
    papers/wg21/deps/install-pandoc.sh \
    papers/wg21/deps/requirements.txt
git add papers/wg21/...   # the files above

# Upstream deleted these; accept the deletion.
git rm -f papers/wg21/generated/TEST.html papers/wg21/tools/TEST-side-by-side.py

4.1. The merge conflict you don't get

Here is the part that a casual "resolve and commit" walks straight past. A three-way merge only raises a conflict when both sides changed the same file. For a file I had churned but upstream hadn't touched since the squash base, Git auto-merges by silently keeping my version. No conflict. No prompt. The reformatted file just survives.

The conflict it doesn't raise is the one that bites.

So don't trust the conflict list. Diff the whole subtree against upstream and make sure it's byte-for-byte.

# "mode sha path" for the staged subtree, prefix stripped:
git ls-files -s papers/wg21 \
  | awk '{p=$4; sub(/^papers\/wg21\//,"",p); print $1, $2, p}' | sort > /tmp/staged.txt

# The same listing from a checkout of upstream master:
git -C /path/to/mpark-wg21-checkout ls-tree -r master \
  | awk '{print $1, $3, $4}' | sort > /tmp/upstream.txt

diff /tmp/staged.txt /tmp/upstream.txt && echo "IDENTICAL TO UPSTREAM"

This caught two files--refs.py and toc-depth.py–still carrying their reformatted selves. I overwrote them with the upstream copies, re-ran the diff until it was empty, and only then committed the merge.

cp /path/to/mpark-wg21-checkout/data/refs.py      papers/wg21/data/refs.py
cp /path/to/mpark-wg21-checkout/data/toc-depth.py papers/wg21/data/toc-depth.py
git add papers/wg21/data/refs.py papers/wg21/data/toc-depth.py
# diff is empty -> commit the in-progress subtree merge
git commit --no-verify

The squashed merge keeps the subtree metadata intact, so the next refresh is just another git subtree pull --squash.

5. One line to `flat.mk`

The migration proper is a single line in my Makefile, not the subtree's.

include wg21/flat.mk

While I was in there I folded the environment handling into the same file, because papers/Makefile is mine to edit and the subtree is not.

6. Keeping the subtree pristine

This is the whole philosophy, so I'll say it plainly: never patch a file inside the subtree. Every line you change there is a line you re-fight on every pull. Both kinds of drift get pushed back out.

6.1. Environment patches go in my Makefile

The framework fetches wg21.link with Python requests and builds a venv with pip. On a TLS-intercepting network both have to trust the intercepting root CA. Instead of editing refs.py to pass verify=False, or swapping uv into the install script, I point everything at the system CA bundle from papers/Makefile.

# papers/Makefile
# Point Python / requests / pip at the system CA bundle so the wg21.link fetch
# and the venv install validate certs, without touching the subtree.
export SSL_CERT_FILE      := /etc/ssl/certs/ca-certificates.crt
export REQUESTS_CA_BUNDLE := /etc/ssl/certs/ca-certificates.crt
export PIP_CERT           := /etc/ssl/certs/ca-certificates.crt

include wg21/flat.mk

The reward is that the stock upstream refs.py now fetches wg21.link with verification passing–no verify=False, no InsecureRequestWarning–and the venv builds with plain pip. The uv and truststore patches turned out to be unnecessary once the certificates simply validated. uv is genuinely nice for speed; I'll propose a clean PIP override upstream rather than fork the install script to get it back.

6.2. Churn gets fenced out of pre-commit

To stop my hooks reformatting the subtree forever, I excluded it.

# .pre-commit-config.yaml
# papers/wg21/ is mpark/wg21 vendored as a git subtree; keep it pristine so
# `git subtree pull` stays conflict-free.
exclude: 'template/|copier/|infra/|port/|papers/wg21/'

7. Verifying, because Pandoc moved a major version

I built every format and compared against the baseline.

cd papers
make transcode-view.html
make transcode-view.latex transcode-view.pdf

The things I actually looked at: comparison tables (::: cmptable) still render; math is MathML rather than MathJax; the  line stays an invisible HTML comment instead of leaking as visible text; the title block and citations look right.

The build also surfaced three problems that were already there in the 2.18 baseline, so not the upgrade's fault, but worth knowing. Seven citations defined in my local .bib files don't resolve, because the framework's bibliography configuration wins over the document's own bibliography: key. A duplicate email: key in the paper's YAML means Pandoc 3.x keeps only the last one. And emoji and superscript glyphs aren't in the default LaTeX font, so they quietly vanish from the PDF. Bugs for another day, but I'd rather know about them.

8. Living with it

The flat layout is pleasant once it's in place. From papers/:

make transcode-view.html    # -> generated/transcode-view.html
make transcode-view.pdf     # -> generated/transcode-view.pdf
make transcode-view.latex   # the intermediate LaTeX

make                        # ALL papers, ALL formats (html + latex + pdf)
make html                   # all papers, HTML only
make clean                  # remove generated/ (NOT the toolchain)
make distclean              # also remove the downloaded Pandoc + venv

Two things to remember. Bare make now builds three formats, not one, so name the target if CI only wants HTML. And the downloaded Pandoc and the venv live under papers/wg21/deps/ and are git-ignored; the first build re-fetches them, and make distclean gets the space back.

9. Refreshing, next time

Because the subtree is pristine and fenced off from the hooks, the next update is the one-liner the tool always promised.

git subtree pull --prefix=papers/wg21 \
    https://github.com/mpark/wg21.git master --squash

If it ever conflicts again, the recipe is the same: resolve toward upstream, then run the ls-files versus ls-tree diff to prove the subtree is byte-identical before committing.

10. What I'd tell myself

Subtree versus submodule is a real choice, not a coin flip. A subtree vendors files into your tree, which means your repo-wide tooling will happily edit the vendored code. Decide that on purpose and fence it off.

Patches belong downstream. Anything you can say in your own Makefile or config–an environment variable, an include, an exclude–should live there, not inside the vendored copy.

And the merge that silently keeps your side is the dangerous one, not the noisy conflict you can see. After any subtree refresh, diff against upstream. Trust the diff, not the conflict list.

Moving Forward With Legacy Encodings

Steve Downey — Sun, 24 May 2026 17:11:21 GMT

1. Abstract

Reverse-parsing legacy multibyte text encodings — such as Shift_JIS, Big5, or GB18030 — using only local context is an unsolvable problem. Unlike UTF-8, which guarantees $O(1)$ self-synchronization, legacy encodings have heavily overlapping lead and trail byte ranges. Consequently, even if you begin at a known, valid character boundary, computing the byte-width of the preceding character requires an $O(N)$ backward scan to the beginning of the string to resolve the parity of the sequence.

The WHATWG decoding algorithms provide no mitigation, as their forward-looking state machines reset completely at every boundary. Robust reverse iteration through these encodings cannot be solved algorithmically in situ; it requires maintaining an external cache of boundary offsets established during a forward pass.

What follows is the story of attempting to find a way out, and why the math forces us to fail.

2. The Story of Failing to Find a Way Out

2.1. Act I: The Overlapping Abyss

If you are dropped into the middle of an arbitrary point in a string using a WHATWG legacy multibyte encoding, you cannot find the beginning of a character just by reading backward.

This happens because legacy encodings lack self-synchronization. Let $L$ be the set of valid lead bytes and $T$ be the set of valid trail bytes. In UTF-8, these sets are strictly disjoint ($L \cap T = \emptyset$). In legacy encodings, the sets intersect aggressively. In Shift_JIS, the byte 0x81 is valid as both a lead byte and a trail byte.

2.1.1. The Specific Example

Imagine you are handed a data buffer and dropped at index 3:

Buffer: [0x81, 0x81, 0x81, 0x81, 0x81]
Index:    0     1     2     3     4

If you look locally at index 3 (which is 0x81), is it the start of a character or the second half of one? You cannot know without scanning backward to the true beginning of the string:

If the string actually started at index 1: The parsing goes [0x81 0x81] (Char 1) and [0x81 0x81] (Char 2). Your byte at index 3 is a lead byte.
If the string actually started at index 0: The parsing goes [0x81 0x81] (Char 1), [0x81 0x81] (Char 2), leaving index 4 orphaned. Your byte at index 3 is a trail byte.

Every single byte relies entirely on the even/odd parity established by the byte preceding it. If the string is malformed or sliced, the ambiguity is even worse, as dropped trail bytes leave no algorithmic breadcrumbs going backward.

2.2. Act II: The Illusion of the Anchor

Suppose your position is not random. You just parsed forward from the beginning of the string, so you know for an absolute fact you are sitting exactly at a valid character boundary.

You still cannot reliably step backward. Knowing the end of the previous character does not tell you its length.

2.2.1. The Specific Example

Imagine looking backward from your known, forward-parsed cursor and seeing this sequence in Shift_JIS:

... 0x81 0x81 0x81 0x40 | <CURSOR>

We know 0x40 is ASCII @ (a valid 1-byte character) AND a valid trail byte. We know 0x81 is a lead byte AND a trail byte. Where does the character before your cursor begin?

Scenario A (Odd number of preceding ~0x81~s): The sequence from the start of the string was [0x81 0x81] (U+3001 Ideographic Comma) followed by [0x81 0x40] (U+3000 Ideographic Space). $\rightarrow$ Result: The character before your cursor is 2 bytes long, and you just parsed a Japanese space.
Scenario B (Even number of preceding ~0x81~s): The sequence from the start of the string was [0x81 0x81] (U+3001), [0x81 0x81] (U+3001), followed by a standalone [0x40] (U+0040 '@'). $\rightarrow$ Result: The character before your cursor is 1 byte long, and you just parsed an English '@' symbol.

Even though you are sitting at a known boundary, you are forced into the exact same backward dependency chain to resolve the parity of the preceding bytes just to know if you should step back 1 byte or 2 bytes.

2.3. Act III: State Machine Amnesia

Finally, you might think to capture the exact WHATWG parser state at your cursor position, rather than just the pointer.

This is the final dead end. Having the parser state provides absolutely zero historical context because the WHATWG state machine suffers from "amnesia" at character boundaries. Decoders operate as forward-looking Mealy machines. When you reach a valid character boundary, the decoder finishes emitting the Unicode scalar value and immediately resets its internal state to the Default state (waiting for a new ASCII character or a new multibyte lead byte).

2.3.1. The Specific Example

Let's track the exact parser state for the end of the two scenarios from Act II to see why it fails to help us read backwards.

Trace A (The preceding character was '@'):
1. Parser reads 0x81 $\rightarrow$ State changes to: Shift_JIS lead
2. Parser reads 0x81 (Emits U+3001) $\rightarrow$ State resets to: Default
3. Parser reads 0x40 (Emits U+0040) $\rightarrow$ State resets to: Default
Trace B (The preceding character was Ideographic Space):
1. Parser reads 0x81 $\rightarrow$ State changes to: Shift_JIS lead
2. Parser reads 0x40 (Emits U+3000) $\rightarrow$ State resets to: Default

In both traces, you are left at the exact same byte position, looking at the exact same bytes behind you, and your parser state is completely identical: Default. The parser state only tells you what to do with the next byte you encounter.

3. Formal Definitions

3.1. Self-Synchronizing Code

A code is self-synchronizing if, given an arbitrary point in a bit stream, the decoder can determine the boundaries of the next encoded sequence within a bounded number of bits ($O(1)$ scan). UTF-8 achieves this via distinct prefix bit-patterns (e.g., 110xxxxx for a 2-byte lead, 10xxxxxx for a trail).

3.2. Decoder State Amnesia

The property of WHATWG decoders where the internal state matrix transitions to the origin state upon the emission of any complete scalar value. Because the state maps purely to pending bytes, no history of completed byte widths is preserved.

4. Tables and Proofs (For the Unbelieving)

If you still suspect there might be a clever bitwise trick to differentiate a lead byte from a trail byte in Shift_JIS, observe the exact byte ranges defined by the standard.

The overlap is not a minor edge case; it is systemic. Every single valid lead byte is also a valid trail byte.

Byte Type	Allowed Hexadecimal Ranges
Lead	`[0x81, 0x9F]` $\cup$ `[0xE0, 0xFC]`
Trail	`[0x40, 0x7E]` $\cup$ `[0x80, 0xFC]`

If we compute the intersection ($Lead \cap Trail$), we find that the entirety of the lead byte range is subsumed by the trail byte range.

Condition	Hexadecimal Range	Consequence for Reverse Parsing
$B \in L \land B \notin T$	$\emptyset$	No byte unambiguously marks the start of a multibyte character.
$B \in T \land B \notin L$	`[0x40, 0x7E]` $\cup$ `0x80`	Can be an ASCII character or a trail byte.
$B \in (L \cap T)$	`[0x81, 0x9F]` $\cup$ `[0xE0, 0xFC]`	Utter ambiguity. Requires $O(N)$ lookbehind.

5. References

WHATWG Encoding Standard, Section 4 (Decoders): Explicitly defines the forward-looking architecture of legacy decoders and the mandatory Shift_JIS lead to Default state transitions. Link
WHATWG Encoding Standard, Section 13 (Shift_JIS): Outlines the index pointer maps showing the overlapping lead and trail ranges. Link
The Unicode Standard, Version 15.0, Section 2.5 (Encoding Forms): Discusses the architectural necessity of non-overlapping (self-synchronizing) code unit boundaries, contrasting UTF-8 with legacy designs. LinkWrite your post here.

Tailwind, Modus Themes, and the Blog Theming Workflow

Steve Downey — Sun, 10 May 2026 22:00:00 GMT

I replaced the Foundation 6 theme on this blog with Tailwind CSS. The immediate motivation was a CSS conflict—Foundation's global code and kbd rules bled into org-mode source blocks—but the deeper reason is that Tailwind has the community and documentation that Foundation no longer does.

This post documents the workflow: how the theme is structured, how syntax highlighting CSS connects Emacs to the browser, and what the current configuration choices are.

The Theme Stack

The blog uses three Nikola themes stacked via the parent chain:

sdowney-tailwind → nikola-tailwind-blog → nikola-tailwind-base → base

nikola-tailwind-base provides the Tailwind Play CDN loading, a responsive navbar, a centered content column wrapped in Tailwind's prose class, and CSS for org-mode source blocks (centering, language labels, copy buttons). It has no opinions about fonts or accent colors.

nikola-tailwind-blog adds an indigo accent, post/index/tag templates, and the visual structure of a blog.

sdowney-tailwind is just my personal layer: Google Fonts (Source Sans 3, Open Sans, Source Code Pro), Font Awesome, and the modus-vivendi syntax highlighting CSS.

Each tier is a standard Nikola theme directory under themes/. Nikola finds templates by walking the chain, so I only override what I actually change.

How Tailwind Is Loaded

The Tailwind Play CDN is a <script> tag that processes utility classes at runtime:

<script src="https://cdn.tailwindcss.com?plugins=typography"></script>

The Typography plugin provides the prose class, which styles all content elements (headings, paragraphs, lists, blockquotes, code, tables) inside the main content area. For a blog where content comes from org-mode export, this is exactly right—I don't control the HTML org produces, and prose handles it gracefully.

The Tailwind config is inline in a <script> block, defined in my personal base_helper.tmpl:

var heading = ['"Open Sans"', 'Roboto', 'Arial', 'sans-serif'];
tailwind.config = {
    theme: {
        extend: {
            fontFamily: {
                sans: ['"Source Sans 3"'].concat(heading),
                heading: heading,
                mono: ['"Source Code Pro"', 'Inconsolata', 'ui-monospace', 'Courier', 'monospace'],
            },
            typography: {
                DEFAULT: {
                    css: {
                        'h1, h2, h3, h4, h5, h6': {
                            fontFamily: heading.join(', '),
                        },
                    },
                },
            },
        },
    },
};

Font stacks are defined once. The heading array is reused in three places.

Source Block Rendering

Org-mode's HTML exporter produces this structure:

<div class="org-src-container">
  <pre class="src src-cpp">
    <code><span class="org-keyword">template</span> ...</code>
  </pre>
</div>

The src-cpp class carries the language. JavaScript in base.tmpl reads it, maps cpp to "C++", and injects a label in the top-left corner and a copy button in the top-right. Any language works—unknown ones display their raw name.

The CSS positions the container with position: relative, margin: auto, width: fit-content, min-width: 60%, and gives the pre a dark background (--code-bg) with padding. The padding-top: 2.25rem makes room for the label and button.

Syntax Highlighting: The Modus Connection

Org-mode's HTML exporter uses htmlize to apply face colors as CSS classes. When org-html-htmlize-output-type is 'css, it produces <span class"org-keyword">=, <span class"org-type">=, etc. The corresponding CSS must exist in an external stylesheet.

Protesilaos Stavrou's modus-themes provide a structured palette: an alist of (semantic-name . hex-color) pairs. Face definitions reference these palette names. font-lock-keyword-face, for example, maps to magenta-alt-other in modus-vivendi.

secretaire-css.el bridges Emacs and the browser:

Load the target theme (e.g., modus-vivendi-tinted)
Require deferred major modes so their faces exist
Read the palette via (modus-themes-current-palette)
For each face, resolve attributes and reverse-map hex → palette name
Emit CSS rules using var(--palette-name) references
Include only referenced palette entries in the :root block

The result:

:root {
    --bg-main: #0d0e1c;
    --fg-main: #ffffff;
    --magenta-alt-other: #b6a0ff;
    --green: #44bc44;
    /* ... only used entries ... */
}

/* font-lock-keyword-face: magenta-alt-other */
.org-keyword {
    color: var(--magenta-alt-other);
    font-weight: bold;
}

The tailwind-base.css file in the base tier references these variables:

:root {
    --code-bg: var(--bg-main, #1e1e2e);
    --code-fg: var(--fg-main, #cdd6f4);
}

Change the modus variant, regenerate, and code block backgrounds follow.

Current Configuration

What's using the Play CDN

The Tailwind Play CDN generates CSS client-side. There is no node_modules, no package.json, no build step. Page load includes a brief delay while Tailwind processes the utility classes. For a personal blog with moderate traffic, this is acceptable.

If it becomes a problem, the switch to a pre-built CSS file is straightforward: install the Tailwind CLI, run npx tailwindcss -o tailwind.css, replace the <script> with a <link>.

Typography plugin via prose

The prose prose-lg prose-indigo classes on the content <div> provide:

Reasonable line lengths and spacing
Styled headings, links, lists, blockquotes, tables
Code blocks with monospace font
prose-indigo for accent-colored links

Anything outside prose (navbar, footer, tag pills) uses not-prose or lives outside the prose wrapper.

Fonts

Body: Source Sans 3 (with Open Sans fallback)
Headings: Open Sans
Code: Source Code Pro

All loaded via Google Fonts. Tailwind's font-sans and font-mono utilities reference these through the config.

Accent color: indigo

Tailwind's indigo palette provides the accent: bg-indigo-700 for the header banner, text-indigo-600 for links, bg-indigo-100 text-indigo-700 for tag pills. Changing to another color is a mechanical search-replace across the blog tier templates.

Icons: Font Awesome 7.0.1

Loaded via CDN in the personal tier. Navigation links in conf.py include Font Awesome <i> tags directly in the link text.

What I Might Tweak Later

Dark mode

The syntax highlighting is already dark (modus-vivendi). The page chrome is light. A proper dark mode would add dark: variants to the templates and switch the modus CSS to operandi for the light code blocks. Tailwind makes this straightforward with darkMode: 'media' in the config.

Production Tailwind

If page load times bother me, switching to a pre-built CSS file removes the Play CDN runtime cost. The config is already defined inline; moving it to tailwind.config.js is mechanical.

Palette-aware CSS generation

secretaire-css currently falls back to htmlize for the actual face scanning when the modus palette API isn't available. The full palette-aware path (emitting var() references instead of flat hex) works with modus-themes 3.x and 4.x. See docs/secretaire-css-design.org for what the ideal output looks like.

Fonts

I'm eyeing Atkinson Hyperlegible Next for body text. Changing fonts is a two-place edit: the Google Fonts <link> tag and the Tailwind config in base_helper.tmpl.

Surround With UUID

Steve Downey — Thu, 09 Apr 2026 21:41:26 GMT

The question of why C++ is standardized through ISO comes up fairly often. This is what I came up with as an explanation the last time I tried to answer that.

It's a radically oversimplified too long elevator pitch.

Why Standard Organizations

Steve Downey — Sat, 07 Mar 2026 15:21:36 GMT

The question of why C++ is standardized through ISO comes up fairly often. This is what I came up with as an explanation the last time I tried to answer that.

It's a radically oversimplified too long elevator pitch.

"Why Standards Organizations" is actually a really good question, as is the related question of if C and C++ should move from ISO.

Standards organizations exist because standards are important and governments know they aren't particularly competent to create them. But standards can also have legal impact, like the fire code that says building materials must meet some criteria to be legal for people to live in. Screws have to match the tools and the holes they are meant to go in.

The people who have the expertise to define all this are the people who make things. That's actually a problem. Letting BigScrewCo say that BigScrew #7 is the standard means they have a huge advantage. BigScrewCo and BigDriverCo could conspire to cut everyone else out of the market.

So you want an independent organization, with rules to prevent that.

That org gave Microsoft, IBM, and Bell Labs the legal cover to meet with each other and conspire about their competing products. Remember that those companies, in particular, had already agreed to remedies for their antitrust violations. So meeting under the auspices of ANSI made all the lawyers, if not happy, at least less worried.

Open Source did not exist. GNU and the FSF were brand new. Collaborating in public wasn't a thing, either, because we didn't have The Web, even if we did have early Internet.

There are a lot of standards organizations now [insert xkcd:Standards]. Nor does it seem particularly necessary from a legal or technical perspective to host something like a language or ecosystem standard within one. However, existing orgs give some level of credibility.

That's somewhat important, because it short circuits a lot of questions about how good, or what process, etc, for the quality of a standard.

It also helps on the other end, getting participation. It's much easier to explain to my manager and my company that I'm doing work on an ISO, or ECMA, Apache, or Python Software Foundation project and get all the approvals to engage and spend work hours on it. Similarly for GCC or LLVM or a bunch of other established open source projects.

New projects, or my own projects, are possible, but more work. Harder to get approvals, or to spend working hours or actual cash on travel for. And that is similar for everyone everywhere, even to some extend for a sole proprietorship consultancy.

So, yes, while we could set up something entirely new, that makes other things more complicated, and it is more work. And a new organization has to be able to answer the question of why should we listen to René and Steve, even if René is one of the leading experts on the C++ ecosystem and tools, and Steve is old and has grey hair, but works for a really big company.

LLVM and GCC are definitely interested in getting answers to this, especially for the parts that no one ought to care about as long as there is one answer, but neither is really the right group. It also turns out that even walled garden ecosystems end up caring about C, C++, and Fortran, because they want access to the code written in those languages, but integration is a terrible mess.

We joke that the most common build system is not CMake, it's setup.py.

So, a moderate sized post, with a not terribly satisfying engineering answer, because like most hard problems, it's not a technical problem it is a people problem.

C++ stays at ISO because ISO has rights to the standard, and getting those rights to move it elsewhere is a hard problem, and would make everyone involved have to do some work just to continue to participate. Doing all that would have to be a massively clear win, and so no one is trying to take on the impossible task.

The Sender Sub-Language

Steve Downey — Sat, 28 Feb 2026 23:40:21 GMT

The paper The Sender Sub-Language by Vinnie Falco <vinnie.falco@gmail.com> and Mungo Gill <mungo.gill@me.com> makes extensive use of my work at https://github.com/steve-downey/sender-examples. The code is also the basis for my talk at C++Now 2023, Using the C++ Sender/Receiver Framework: Implement Control Flow for Async Processing. They present the code accurately and fairly, and I am very happy they found it useful in describing and understanding the capabilities of Senders in the framework. There is no higher praise than someone finding your work useful to build upon.

Nonetheless, we come to different overall conclusions about the Sender/Receiver framework.

There are several papers in the February 2026 pre-Croydon mailing which are, taken together, asking to reconsider the sender / receiver framework, generally and especially in the context of networking. As this is the meeting in which C++26 will be finalized for publication, this is the last chance for any objections to anything being included. It would definitely have been better to have these objections and any new information earlier, but, since changing anything after the standard is shipped is far more difficult, even sustained opposition with new information is important to consider.

This post, however, is not my consideration of the new papers.

I simply wish to put the work I did in 2022 and 2023 in context.

1. The Papers

The papers which I am looking at

Number	Title	Author	Subgroup
P2583R0	Symmetric Transfer and Sender Composition	Mungo Gill, Vinnie Falco	LEWG Library Evolution
P4003R0	Coroutines for I/O	Vinnie Falco, Mungo Gill, Steve Gerbino	LEWGI SG18: LEWG Incubator,LEWG Library Evolution
P4007R0	Senders and Coroutines	Vinnie Falco, Mungo Gill	LEWG Library Evolution
P4014R0	The Sender Sub-Language	Vinnie Falco, Mungo Gill	LEWG Library Evolution
P4029R0	The SG14 Priority List for C++29/32	Michael Wong	SG14 Low Latency

2. Abstracts and Suggested Polls

These quotes are drawn from their respective papers to provide context.

2.1. P2583R0: Symmetric Transfer and Sender Composition

2.1.1. Abstract

C++20 provides symmetric transfer (P0913R1) - a mechanism where await_suspend returns a coroutine_handle<> and the compiler resumes the designated coroutine as a tail call. Coroutine chains execute in constant stack space. std::execution (P2300R10) composes asynchronous operations through sender algorithms. These algorithms create receivers that are structs, not coroutines. No coroutine_handle<> exists at any intermediate point in a sender pipeline. When a coroutine co_await s a sender that completes synchronously, the stack grows by one frame per completion. P3552R3’s std::execution::task inherits this property. The sender model’s zero-allocation composition property and symmetric transfer’s constant-stack property cannot both be satisfied. One requires structs. The other requires coroutines. This paper describes the mechanism, provides implementation experience, and documents the tradeoff.

2.2. P4003R0: Coroutines for I/O

2.2.1. Abstract

C++20 coroutines have five properties that, taken together, make them uniquely suited to asynchronous I/O: type erasure through coroutine_handle<> , customization through promise_type , stackless independently resumable frames, symmetric transfer through await_suspend , and compiler-managed state that persists across suspension points. Each was designed for generality. Their conjunction yields something no single property suggests: the optimal basis for byte-oriented I/O.

We used C++20 coroutines directly for I/O - timers, sockets, DNS, TLS, HTTP - and observed what the language already provides. The protocol that emerged is the IoAwaitable: a system for associating a coroutine with an executor, stop token, and frame allocator, and propagating this context forward through a coroutine chain to the operating system API boundary where asynchronous operations are performed.

2.2.2. Suggested Straw Polls

SG4 polled at Kona (November 2023) on P2762R2 “Sender/Receiver Interface For Networking”

“Networking should support only a sender/receiver model for asynchronous operations; the Networking TS’s executor model should be removed”

SF F N A SA

5 5 1 0 1

Consensus.

The approach described in this paper - a coroutine-native I/O model using C++20 language features - was not among the alternatives considered.

Poll 1. A coroutine-native I/O model is a distinct approach from both the Networking TS executor model and the sender/receiver model.

Poll 2. New research into coroutine-native I/O, not available at the time of the Kona poll, warrants consideration.

2.3. P4007R0: Senders and Coroutines

2.3.1. Abstract

std::execution serves its domain well. Different asynchronous domains have different costs, and a single model cannot minimize all of them simultaneously. This paper identifies four structural gaps where the sender model meets coroutines: three at the boundary - error reporting, error returns, and frame allocator propagation - and one inside the composition mechanism - the symmetric transfer gap documented in P2583R0. Each gap is the cost of a property the sender model requires for compile-time analysis (P2300R10, P4014R0 ). They are not design defects, they are tradeoffs. Mandating that standard networking be built on the sender model would force coroutine I/O users to pay these costs. A coroutine-native I/O research report (P4003R0) made the gaps visible by showing that partial results, error returns, cancellation, and frame allocator propagation emerge naturally when I/O is designed for coroutines. The findings hold regardless of that report’s specific design. This paper recommends: ship std::execution for C++26, defer task (P3552R3, “Add a Coroutine Task Type”) to C++29, and explore coroutine-native I/O designs alongside sender-based designs.

2.3.2. Suggested Straw Polls

“I/O completions that carry both an error code and a byte count present a design challenge for the three-channel completion model.”
“The coroutine integration in std::execution has open design questions that would benefit from further iteration.”
“WG21 should explore coroutine-native I/O designs alongside sender-based designs.”

2.4. P4014R0: The Sender Sub-Language

2.4.1. Abstract

C++26 introduces a rich sub-language for asynchronous programming through std::execution (P2300R10) . Sender pipelines replace C++’s control flow, variable binding, error handling, and iteration with library-level equivalents rooted in continuation-passing style and monadic composition. This paper is a guide to the Sender SubLanguage: its theoretical foundations, the programming model it provides, and the engineering trade-offs it makes. The trade-offs serve specific domains well. The question is whether other domains deserve the same freedom to choose the model that serves them.

2.4.2. Suggested Straw Polls

“ std::execution serves coroutine-driven async I/O less ideally than heterogeneous compute.”
“Coroutine-driven async I/O should have the same freedom to optimize for its domain as heterogeneous compute did

2.5. P4029R0: The SG14 Priority List for C++29/32

This paper is a forward looking priority list for SG14 Low Latency. It has overlapping concerns about the sender / receiver model with the other papers.

2.5.1. Executors/coroutine backward flow + lazy execution vs direct forward flow model for CPU-bound I/O and Networking.

Decouple Networking from std::execution SG14 advise that Networking (SG4) should not be built on top of P2300. The allocation patterns required by P2300 are incompatible with low-latency networking requirements.
Standardize "Direct Style" I/O Prioritize P4003 (or similar Direct Style concepts) as the C++29 Networking model. It offers the performance of Asio/Beast with the ergonomics of Coroutines, maintaining the "Zero-Overhead" principle.

3. The Sender Model

As described in my earlier papers and talks, and accurately summarized by P4014R0: The Sender Sub-Language, senders are a framework for working with continuations and for supporting Continuation Passing Style, or CPS. The CPS formalism is a full model of computation, Turing complete, and equivalent in expressiveness to Single Static Assignment, the current, common, model that compilers use for lowering programming languages. CPS used to be far more common, and is still in use in some language implementations today. The reasons for choosing one over the other is entirely out of scope for this discussion.

The Sender model is also closely tied to the Cont monad. For background see Control.Monad.Cont, Haskell/Continuation passing style, and The Mother of all Monads. The monad abstracts away the chaining and connection of the results to the eventual receiver. Senders augment the model by adding the three channels used to communicate between the processes, allowing for propagation of values, errors, and cancellation requests throughout the graph.

The Sender / Receiver framework is also a model of Structured Concurrency, Structured Concurrency, Martin Sustrik, Notes on structured concurrency, or: Go statement considered harmful – Nathaniel J. Smith, a system for ensuring that concurrent operations have clear entry and exit points and will all complete before exiting. It does so in part by also using delimited continuations, ones that have fixed, rather than unbounded, end points, Continuations and Delimited Control – Oleg Kiselyov

In evaluating a library, particularly one that will be frozen in the C++ standard, where good libraries go to die, I want to know if what is being offered is complete, and if it is not, if what is offered has everything necessary and sufficient to base further work on outside the library. We made this mistake, for example, with the initial Ranges work where there was no mechanism for a user to write a pipe-able algorithm with what was specified. This was quickly rectified, fortunately. With format, the ability to statically check the format string was back-filled, and was not ABI compatible, causing vendors much distress in shipping the new component.

It was with this in mind that I was looking at what was well specified and available in the original P2300 papers. In general I prefer systems that have firm theory behind them over ones that merely, pragmatically, appear to work. As an industry we were all burned badly by the pthreads library, which appeared to work, but we now know can not by itself. The pthread model is also one that we have demonstrated, repeatedly, that human beings can not reason about. On the other hand, object orientation still has no good theory, but has been demonstrably successful. The abstractions it provides allow humans to reason at a much higher level about the complex systems we are producing.

The work in 2022 and 2023 was with the goal in mind of showing that the end user facing parts of the model, senders and the basic adapters in the proposal, were necessary and sufficient as a basis for further work. That the primitives would not need to be scrapped or changed in ways that would break code once written. This is not the same as saying there need be no other adapters or algorithms provided, or that the sender primitives are the most efficient implementations. The question was how far can the model be pushed without having to work with operational state directly.

The base sender operations, just, then, and let_value, allow all structured programming constructs to be created, including structured concurrency. The model is complete and well founded.

The proposal, as adopted, also provides for user implementation and inter-operation of types that have operational states, senders and sender adapters themselves. Analogously to how the standard allows for user written containers to participate in the containers/iterators/algorithms triad that comprised the original STL proposal. Users may not write new containers or iterators often, but there is no particular magic involved, and writing one should be well within the capabilities of most C++ programmers.

A sender is somewhat more difficult, as it is new so there is less documentation, but it is not intrinsically hard. In addition, the APIs tend to encourage correct and safe use. Senders describe the shape of work and should be generally reusable. Programmers will work with senders and sender adaptors several orders of magnitude more than they will write a sender or sender adapter from scratch.

My question was not, what is missing that can be added. That is self evidently lots of things. We are still adding range algorithms and new ranges after years of work and more are in the pipeline for C++29 already. My question was, can real applications be created with nothing more than what is easily available in senders, and can an application programmer who does not wish to create a sender today, still deliver a sender graph of async work.

That was, and is, clearly, to me, yes.

Exported: 2026-06-29 08:50:27

Building vcpkg dependencies with project toolchain

Steve Downey — Sat, 27 Dec 2025 17:03:15 GMT

Making sure vcpkg delivers packages built with your toolchain is not hard, but much of the advice on the internet is flat wrong. You need to specify your toolchain both in your project and in the the vcpkg triplet. There's an airgap between your project and the dependency in vcpkg install. The CMake settings can't just flow through.

1. Toolchain is more than just the compiler

Inside a link context everything needs to use the same definitions for everything or your program breaks. If you are lucky it will fail to link. If you are unlucky it will link and explode at runtime. Or you might have the worst kind of undefined behavior–working the way you expect it to. For now.

The One Definition Rule tries to say that all of the definitions in a program must mean the same thing. Using different flags for different translation units is a simple way of giving different meanings to definitions. This can range from preprocessor defines in the flags, to conditional compilation based on the compiler, or standard version, to subtle changes in meaning based on the standard version or other flags, such as the literal encoding to render characters and string constants into.

The toolchain should encompass all of the ABI affecting flags, and most flags affect ABI, or at least codegen, in some way. Differing codegen, while not as dangerous as an ABI break, can still be an unpleasant surprise.

2. Using a toolchain with `vcpkg` in-project

There are several techniques for using vcpkg for dependency management and specifying a toolchain to use for compilation of the artifacts in a project. For an open source project, the problem is generally how to do so without locking everyone into vcpkg. The current theory for CMake is to provide dependency provider hooks via CMAKE_PROJECT_TOP_LEVEL_INCLUDES, which are a list of files cmake will process before anything else, and in particular before the project statement. The Beman Project has a primitive dependency provider, use-fetch-content.cmake, that converts find_package into FetchContent and downloads and builds from Git URLs. This is an alternative to vendoring dependencies in-tree directly using git submodules or git subtrees.

As an aside, git submodules are like vice-grips–the wrong tool for every job.

For vcpkg, you can integrate using the same mechanism, setting CMAKE_PROJECT_TOP_LEVEL_INCLUDES to $(VCPKG_ROOT)/scripts/buildsystems/vcpkg.cmake. This is entirely in place of setting the CMAKE_TOOLCHAIN_FILE to vcpkg.cmake, including vcpkg.cmake in a local toolchain file, or using VCPKG_CHAINLOAD_TOOLCHAIN_FILE to instruct vcpkg.cmake to load a toolchain after processing the triple file.

The initial configure will look something like:

cmake -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(VCPKG_ROOT)/scripts/buildsystems/vcpkg.cmake \
      -DCMAKE_TOOLCHAIN_FILE=./etc/gcc-16-toolchain.cmake \
      -B .build -S .

With the gcc-16 toolchain file looking something like:

include_guard(GLOBAL)

set(CMAKE_C_COMPILER gcc-16)
set(CMAKE_CXX_COMPILER g++-16)
set(GCOV_EXECUTABLE "gcov-16" CACHE STRING "GCOV executable" FORCE)

set(CMAKE_CXX_STANDARD 26)

set(CMAKE_CXX_FLAGS
    "-Wall -Wextra -std=gnu++26 -Wno-maybe-uninitialized"
    CACHE STRING
    "CXX_FLAGS"
    FORCE)

set(CMAKE_CXX_FLAGS_DEBUG
    "-O0 -fno-inline -g3"
    CACHE STRING
    "C++ DEBUG Flags"
    FORCE
)
set(CMAKE_CXX_FLAGS_RELEASE
    "-Ofast -g0 -DNDEBUG"
    CACHE STRING
    "C++ Release Flags"
    FORCE
)
set(CMAKE_CXX_FLAGS_RELWITHDEBINFO
    "-O3 -g -DNDEBUG"
    CACHE STRING
    "C++ RelWithDebInfo Flags"
    FORCE
)
set(CMAKE_CXX_FLAGS_TSAN
    "-O3 -g -fsanitize=thread"
    CACHE STRING
    "C++ TSAN Flags"
    FORCE
)
set(CMAKE_CXX_FLAGS_ASAN
    "-O3 -g -fsanitize=address,undefined,leak"
    CACHE STRING
    "C++ ASAN Flags"
    FORCE
)

set(CMAKE_CXX_FLAGS_GCOV
    "-O0 -fno-default-inline -fno-inline -g --coverage -fprofile-abs-path"
    CACHE STRING
    "C++ GCOV Flags"
    FORCE
)

set(CMAKE_LINKER_FLAGS_GCOV "--coverage" CACHE STRING "Linker GCOV Flags" FORCE)

get_filename_component(RPATH "~/.local/lib64" ABSOLUTE)

set(CMAKE_EXE_LINKER_FLAGS
    "-Wl,-rpath,${RPATH}"
    CACHE STRING
    "CMAKE_EXE_LINKER_FLAGS"
    FORCE
)

This provides CMake build types DEBUG, RELEASE, RELWITHDEBINFO, TSAN, ASAN, and GCOV, overriding the built-in default flags. It exports the coverage processor to use, picked up by my project, and adds rpath to the linked executable to where I have the gcc-16 libstdc++.so.6 installed. GCC recently bumped the GLIBCXX version number to 35, so it is no longer compatible with the system libstdc++.so.6.

Note that it doesn't have any vcpkg variables or includes.

Building with the cmake command line flags above will end up calling vcpkg for any packages mentioned in the vcpkg.json file and providing them to be found by find_package calls. It's not quite how a cmake dependency provider works, but functions very much the same. However, vcpkg will build dependencies the way the system triple defines things, which will be with the system compiler and default flags, and may be entirely incompatible with your project.

3. Building Dependencies with your Toolchain.

The mechanism vcpkg provides for injecting a toolchain into a dependency is a custom triple that provides a VCPKG_CHAINLOAD_TOOLCHAIN_FILE. Because of the intermediate invocation of vcpkg, setting VCPKG_CHAINLOAD_TOOLCHAIN_FILE in your project does not affect how vcpkg builds or provides anything.

We can, however, smuggle the variable we need through the environment.

The custom triple I'm using, x64-linux-custom.cmake, looks like:

set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)

set(VCPKG_CMAKE_SYSTEM_NAME Linux)

set(VCPKG_CHAINLOAD_TOOLCHAIN_FILE "$ENV{PROJECT_VCPKG_TOOLCHAIN}")

And I make sure to export PROJECT_VCPKG_TOOLCHAIN into the environment as part of my build. I use a Makefile to drive my project workflow, so it's straightforward to set and export. Other workflow mechanisms are an exercise for the reader.

The cmake invocation picks up flags and environment:

export PROJECT_VCPKG_TOOLCHAIN=$(realpath ./etc/gcc-16-toolchain.cmake)
cmake -DCMAKE_TOOLCHAIN_FILE=$(PROJECT_VCPKG_TOOLCHAIN) \
      -DVCPKG_OVERLAY_TRIPLETS=$(realpath ./cmake) \
      -DVCPKG_TARGET_TRIPLET=x64-linux-custom \
      -B .build -S .

With this, vcpkg will use the toolchain I specify to build dependencies. It also treats the VCPKG_CHAINLOAD_TOOLCHAIN_FILE as ABI significant for its build caching. This means that if you include other files into your toolchain, the contents of those files do not count. A small issue if you are using layered toolchain files, sharing some flags between them such as for gcc-15 vs gcc-16 in e.g. a gcc-flags.cmake file. But I also know switching between many compilers is a very minority use case.

4. Someone is Wrong on the Internet

This took far longer to work out than it should because there is much advice that is flat out wrong.

In particular from AI generated summaries and articles.

Setting VCPKG_CHAINLOAD_TOOLCHAIN_FILE in your project or on the cmake command line cannot affect how vcpkg builds and retrieves dependencies.

You must set a custom triple file that sets VCPKG_CHAINLOAD_TOOLCHAIN_FILE in order to use that CMake toolchain with vcpkg.

Substitution is Sometimes a Failure

Steve Downey — Fri, 28 Nov 2025 17:06:58 GMT

Why are optional::transform and optional::and_then not constrained by invocable?

1. Optional "Monadic" Interface

Monadic operations for std::optional

transform is the c++ spelling for map or fmap
and_then is monadic bind for optional
or_else is dual to and_then

or_else also has:

Constraints: F models invocable and T models {move,copy}_constructible.

transform and and_then do not.

They don't work if you don't give them invocables, and rely on invoke_result_t to compute the result. So why not constrain them?

2. SFINAE is shallow

The problem is if another template has to be instantiated in order to evaluate the template of interest, and that template has an error, you get an error, not a substitution failure. And, it turns out, lambdas are a common source of the problem and a common case in real use.

Consider the code:

void f(int&);

auto l = [](auto& y) {
    f(y);
    return 42;
};

The two important parts are the auto& parameter and the implicit deduced return type.

We can rewrite it, to make things possibly more obvious:

struct Func {
    template <typename T>
    auto operator()(T& t) {
        f(t);
        return 42;
    }
};

and to slightly spoil things, the problem code is effectively:

static_assert(std::invocable<Func, int const&>);

which produces:

<source>: In instantiation of 'auto Func::operator()(T&) [with T = const int]':
type_traits:2565:26:   required by substitution of 'template<class _Fn, class ... _Args> static std::__result_of_success<decltype (declval<_Fn>()((declval<_Args>)()...)), std::__invoke_other> std::__result_of_other_impl::_S_test(int) [with _Fn = Func; _Args = {const int&}]'
type_traits:2576:55:   required from 'struct std::__result_of_impl<false, false, Func, const int&>'
type_traits:3038:12:   recursively required by substitution of 'template<class _Result, class _Ret> struct std::__is_invocable_impl<_Result, _Ret, true, std::__void_t<typename _CTp::type> > [with _Result = std::__invoke_result<Func, const int&>; _Ret = void]'
type_traits:3038:12:   required from 'struct std::is_invocable<Func, const int&>'
type_traits:3286:71:   required from 'constexpr const bool std::is_invocable_v<Func, const int&>'
concepts:336:25:   required from here
<source>:12:10: error: binding reference of type 'int&' to 'const int' discards qualifiers
   12 |         f(t);
      |         ~^~~
<source>:2:12: note:   initializing argument 1 of 'void f(int&)'
    2 |     void f(int&);
      |            ^~~~
Compiler returned: 1

Compiler Explorer

as the compiler is unhappy about trying to call the function f with a const int&.

If the lambda or Func is changed to have a non-deduced return type, the instantiation errors from the check to invocable go away, although you still get an error calling either with a const int.

So why do we run into this with transform if we were to constrain it with invocable ?

The compiler needs to figure out the overload set in order to resolve which one to use from the set. There are four of them two for the l- and r- value category and two for the const overloads.

template<class F> constexpr auto transform(F&& f) &;
template<class F> constexpr auto transform(F&& f) const &;
template<class F> constexpr auto transform(F&& f) &&;
template<class F> constexpr auto transform(F&& f) const &&;

with differing computations of the resulting optional being returned.

using U = invoke_result_t<F, decltype(std::move(*val))>;
//or
using U = remove_cv_t<invoke_result_t<F, decltype(*val)>>;

So, in order to work out what the templated transform's signature really is, it has to compute what the invocable returns, and since the invocable has deduced return type, it needs to instantiate it, and instantiating with const int causes an error.

This is unfortunate.

If we constrain transform we get the same errors as above. See here, with just enough of an optional to compile.

3. Constraints, what are they good for

Not absolutely nothing.

Constraints in the library:

Constraints: the conditions for the function's participation in overload resolution ([over.match]).

[Note 1: Failure to meet such a condition results in the function's silent non-viability. — end note]

[Example 1: An implementation can express such a condition via a constraint-expression ([temp.constr.decl]). — end example]

[structure.specifications] 3.1

Constraints are for making an overload not exist if the constraint isn't met. It's not a way of signaling an error. Those are Mandates:

Mandates: the conditions that, if not met, render the program ill-formed.

[Example 2: An implementation can express such a condition via the constant-expression in a static_assert-declaration ([dcl.pre]). If the diagnostic is to be emitted only after the function has been selected by overload resolution, an implementation can express such a condition via a constraint-expression ([temp.constr.decl]) and also define the function as deleted. — end example]

[structure.specifications] 3.2

Asking to run and_then on a non-invocable probably ought not to say there is no such function, but instead tell you it can't be invoked. I'm now not convinced that or_else should be constrained this way. It's not significantly better for o.or_else(5) to fail to resolve, mentioning invocable, than produce an error that invoke_result_t doesn't work, or that f can't be invoked. The kind of error is a minor detail.

Constraints that let you control the choice of alternatives are wonderful, and requires clauses are normal programmer accessible, unlike SFINAE, or even enable_if. But without an overload set to constrain, there possibly should not be a constraint.

4. Can we do better?

There are some notes in P0798 that suggest that Deducing This might help, P0847.

The idea would to be to NOT have all the value category overloads that need to be checked, but to just have a single one that deduces what this is and provide it as a template parameter for further use. The contained parameter could be forwarded using forward_like<Self>.

P0847 has discussion about how deducing this might be applied to optional. There's also discussion of deducing this and the SFINAE-unfriendly auto at C++23’s Deducing this: what it is, why it is, how to use it.

With the tools we have today, it looks possible, but still slightly messy. I managed to get my implementation of optional to compile and pass its own tests with.

template <class F, class Self>
    requires(
        std::invocable<F,
                       decltype(std::forward_like<Self>(std::declval<T>()))>)
constexpr auto transform(this Self&& self, F&& f)
    -> optional<std::invoke_result_t<
        F,
        decltype(std::forward_like<Self>(std::declval<T>()))>> {
    using U = std::invoke_result_t<F,
                                   decltype(std::forward_like<Self>(
                                       std::declval<T>()))>;
    static_assert(!std::is_array_v<U>);
    static_assert(!std::is_same_v<U, in_place_t>);
    static_assert(!std::is_same_v<U, nullopt_t>);
    static_assert(std::is_object_v<U> || std::is_reference_v<U>);
    if (self.has_value()) {
        return optional<U>{detail::from_function,
                           std::forward<F>(f),
                           std::forward_like<Self>(self.value_)};
    }
    return optional<U>;
}

If there were a std::forward_like_t, it might be possible to reduce some of the noise in computing the value category used for the T. I also have not thought extensively about if the requires clause is truly needed in light of the invoke_result_ that can now be used in the trailing return type.

Nikola Blog Infrastructure

Steve Downey — Mon, 23 Dec 2024 18:15:27 GMT

I've migrated from WordPress to a static blog generator–Nikola. This is how it works.

1. Nikola

Nikola is a static site generator written in python. It's mostly boring, which is a very good thing for software meant for use to be. Its home is https://getnikola.org and the source is on github at https://github.com/getnikola/nikola. It's many years old at this point, but not dead.

The interesting part for me is that someone had already written a plugin that compiled org mode files, my preferred markup language, to the html that Nikola wants to publish. It wasn't quite to my tastes, but the changes were all on the org export side in elisp, and the python invocation of emacs to do the translation didn't need particular changes. That update is at https://github.com/steve-downey/wctm/tree/src/plugins/orgmode.

I've also set up staging at a github pages repo rendered at https://steve-downey.github.io/ since it's a static site. The real version is at https://sdowney.org.

Conversion was straightforward using the tools that Nikola provided and following the directions for importing a WordPress xml backup.

2. Orgmode plugin

Pulling a plugin into my working site was easy

nikola plugin -i orgmode

copied the code from https://github.com/getnikola/plugins/tree/master/v8/orgmode and I just had to update the main configuration file, conf.py, to map org files to the orgmode compiler.

COMPILERS = {
    "rest": ['.txt', '.rst'],
    "markdown": ['.md', '.mdown', '.markdown'],
    "html": ['.html', '.htm'],
    "orgmode": ['.org'],
}

I then updated the source code for the elisp html exporter using code from https://github.com/steve-downey/wg21org/blob/main/emacs.d/init.el which installs org from upstream rather than using the built-in orgmode and configures my preferences. I may end up using more of ox-wg21html.el as a framework for the exported HTML, only updating to use Bootstrap for css classes instead of my own or the org builtin.

3. LaTeX support via KaTeX

I've configure Nikola to support LaTeX via KaTeX rather than MathJax. It is supposed to be a bit faster. This means inline math like $x^2$ or $1 < 2$ or with breaks like: \[ \int_0^\infty e^{-x^2} dx = {{\sqrt{\pi}} \over {2}} \] Or in running text where I want to mention \[ \frac{-b \pm \sqrt{b^2 - 4 a c}}{2a} \] as the quadratic equation. It does mean I don't have access to latex blocks like

\begin{equation}
  n_{i+1} = \frac{n_{i} (d-i) (e-1)}{(i+1)}
\end{equation}

Maybe I won't miss it.

Condition	Hexadecimal Range	Consequence for Reverse Parsing
\(B \in L \land B \notin T\)	\(\emptyset\)	No byte unambiguously marks the start of a multibyte character.
\(B \in T \land B \notin L\)	`[0x40, 0x7E]` \(\cup\) `0x80`	Can be an ASCII character or a trail byte.
\(B \in (L \cap T)\)	`[0x81, 0x9F]` \(\cup\) `[0xE0, 0xFC]`	Utter ambiguity. Requires \(O(N)\) lookbehind.

What Comes to Mind

An Infix Backtick Operator for C++

1. The pitch in one line

2. It desugars to a call, and that is the whole point

3. How it parses

4. The one genuinely tricky part

5. A second use for the same token: escaping keywords

6. What it is good for

7. It is not the pipeline operator, and does not want to be

8. Implementation status

9. What I am proposing, and what comes next

Refreshing a Stale Git Subtree

1. What the overhaul changed

2. Where I started, and why it was a mess

3. Branch, and a baseline to diff against

4. Refreshing the subtree

4.1. The merge conflict you don't get

5. One line to flat.mk

6. Keeping the subtree pristine

6.1. Environment patches go in my Makefile

6.2. Churn gets fenced out of pre-commit

7. Verifying, because Pandoc moved a major version

8. Living with it

9. Refreshing, next time

10. What I'd tell myself

Moving Forward With Legacy Encodings

1. Abstract

2. The Story of Failing to Find a Way Out

2.1. Act I: The Overlapping Abyss

2.1.1. The Specific Example

2.2. Act II: The Illusion of the Anchor

2.2.1. The Specific Example

2.3. Act III: State Machine Amnesia

2.3.1. The Specific Example

3. Formal Definitions

3.1. Self-Synchronizing Code

3.2. Decoder State Amnesia

4. Tables and Proofs (For the Unbelieving)

5. References

Tailwind, Modus Themes, and the Blog Theming Workflow

The Theme Stack

How Tailwind Is Loaded

Source Block Rendering

Syntax Highlighting: The Modus Connection

Current Configuration

What's using the Play CDN

Typography plugin via prose

Fonts

Accent color: indigo

Icons: Font Awesome 7.0.1

What I Might Tweak Later

Dark mode

Production Tailwind

Palette-aware CSS generation

Fonts

Surround With UUID

Why Standard Organizations

The Sender Sub-Language

1. The Papers

2. Abstracts and Suggested Polls

2.1. P2583R0: Symmetric Transfer and Sender Composition

2.1.1. Abstract

2.2. P4003R0: Coroutines for I/O

2.2.1. Abstract

2.2.2. Suggested Straw Polls

2.3. P4007R0: Senders and Coroutines

2.3.1. Abstract

2.3.2. Suggested Straw Polls

2.4. P4014R0: The Sender Sub-Language

2.4.1. Abstract

2.4.2. Suggested Straw Polls

2.5. P4029R0: The SG14 Priority List for C++29/32

2.5.1. Executors/coroutine backward flow + lazy execution vs direct forward flow model for CPU-bound I/O and Networking.

3. The Sender Model

Building vcpkg dependencies with project toolchain

1. Toolchain is more than just the compiler

2. Using a toolchain with vcpkg in-project

3. Building Dependencies with your Toolchain.

4. Someone is Wrong on the Internet

Substitution is Sometimes a Failure

5. One line to `flat.mk`

2. Using a toolchain with `vcpkg` in-project