Design decisions
Every decision that shapes the language or the compiler, with the reason. Numbered so that code comments and STATUS.md can point at one.
Every decision that shapes the language or the compiler, with the reason.
Numbered so that code comments and STATUS.md can point at one. A decision is
revisited only when something new is known, not because it was inconvenient.
The project
D1. Wolang is its own project, not an engine module. The language is for general use; a game engine is one host among several, and its adapter arrives in wave 15. Keeping the core engine-free is what stops engine concepts from leaking into the type system.
D2. The compiler stays C++20 — self-hosting is not a goal. Rewriting the compiler in Wolang needs the finished language first and doubles the work for years. It remains an option after phase D, not a commitment.
D3. The standard library is written in Wolang above a thin non-Wolang
floor: a C++ platform layer for syscalls, and compiler intrinsics for what the
language cannot express (memcpy, atomics, SIMD, sqrt). core becomes
possible in wave 7, when the interpreter can run it.
D4. The interpreter is the reference semantics, not a stepping stone. The JIT, the AOT tier and the WASM backend are mechanical lowerings of the same verified IR, and differential fuzzing across all four is what keeps them honest.
D5. arm64 before x64 for the dev-tier code generator, because that is the development machine.
D6. Chromium C++ style everywhere, no exceptions and no RTTI. Commits are Conventional Commits, one line, no body, no trailers.
D7. docs/ is in English, STATUS.md is in Russian. The docs are public
artifacts of the language; STATUS.md is the working log. Replaced by D220.
Memory
D8. Hybrid memory, optional collector. Values with mutable value
semantics; references are second class, so they may be parameters and locals
but not fields — memory safety without a borrow checker. Owned<T> for unique
ownership, Shared for shared, arena { } for frame arenas, and class
with a garbage collector as a separate optional component. A program that
declares no class links no collector, which is what makes WASM and embedded
targets practical.
Types
D9. Integer literals have arbitrary precision until used. A literal takes
the type its context expects, defaulting to int. 0xFFFFFFFFFFFF needs no
suffix, comptime arithmetic is exact, and a literal that does not fit its
destination is an error rather than a silent wrap.
D10. [N]T and []T are language constructs; Array<T> is a library
type. Fixed arrays and slices allocate nothing and core needs them, so
they belong to the language. A growable array needs an allocator, so it
belongs to std (wave 13).
D11. T? is Option<T>. One concept, one implementation, flow typing
that works the same for values and references. The compiler niche-optimizes
it, so Actor? is a nullable pointer at runtime.
D12. Optionals need no Some/None vocabulary. The language already
spells presence and absence: T? in a type, x? in a pattern, null for
absence, ?? for a default. Importing two more words for a concept already
expressible is a cost with no benefit.
D13. Error sets are inferred, and T! is postfix. fun Load() -> Level!
collects the errors the body can produce, so ? never needs a conversion
between error types. Postfix matches T?, so a type reads left to right and
the two combine as Pawn?!. An error is a tag without data; a payload means
writing Result<T, E> explicitly.
D14. Enum variants are members of their type, with a short form in
match. Shape.Circle(r) everywhere, and bare Circle(r) inside a match
whose scrutinee type is known. No name collisions between enums, and no type
name repeated on every arm.
D15. No fixed type. @Deterministic code puts floats in a strict mode:
no FMA contraction, no fast-math, fixed operation order. IEEE-754 is itself
deterministic — the nondeterminism comes from the compiler, not the hardware.
A library Fixed<N> can arrive in wave 13 if measurements call for it.
D15a. There are no implicit numeric conversions, not even widening.
const n: s64 = narrow is an error; narrow as s64 is how it is written.
Arbitrary-precision literals (D9) remove the common case that would
otherwise make this tiresome, and a conversion that is always spelled is a
conversion that can always be found. This retires the planned W0114
"implicit narrowing" lint: there is no implicit narrowing to warn about.
D15b. An owned Text is accepted where a borrowed text is wanted. An
implicit borrow, and safe for the reason references are second class (D8): a
text cannot be stored in a field, so the view can never outlive the string
it points into. Without this, printing an interpolated string — the most
common operation in any language — would not type check, and requiring a
sigil for it would buy nothing a borrow checker is not already providing.
D15c. Arithmetic is not pure, because overflow traps. Integer overflow
and division by zero are traps, and a trap is a defined outcome the program
can observe. Deleting an unused a + b would therefore turn a program that
faults into one that does not, so dead-code elimination may not remove it
without a proof that it cannot trap. That proof is range analysis and
belongs with the optimizer proper. The alternative — wrapping silently, so
that arithmetic becomes pure — buys optimizations by turning an arithmetic
bug into a wrong answer somewhere else.
Syntax
D16. Control flow is expression-shaped. if, match, loop, while,
for, foreach, unsafe, arena and blocks are all expressions, so
const x = if c { a } else { b } needs no special case.
D17. An operator at the start of a line begins a new statement. A line
starting with - is a unary minus, never a continuation, so two statements
can never silently fold into one. . and ?. are exempt, because
leading-dot method chains are normal style.
D18. No turbofish. f<T>(x) is resolved by scanning ahead for balanced
angle brackets followed by (. The cost is that a function type cannot be an
explicit generic argument in expression position; both affected forms are rare
enough to be worth the simpler rule.
D19. A dot in an expression is always member access. World.Spawn becomes
a path during name resolution, which is what languages without a ::
separator do.
D20. as is the cast operator, binding tighter than any binary operator.
D20a. fun, need and offer replace fn, import and pub. fun is
the whole word rather than an abbreviation. need and offer are a matched
pair that says what a module does: it needs names from elsewhere and
offers its own. No other language uses this pair, which is the point — the
words describe the relationship rather than borrowing a convention. fn,
import and pub are reserved, so old habits get a message.
D21. extend T and extend T as Ability replace impl. The target comes
first either way, so the declaration reads as a sentence. impl Ability for Type is a known stumbling block: its word order matches neither English nor
intuition. impl is reserved, so the Rust spelling gets a message rather than
a confusing parse failure.
D22. Storage layout is a property of the storage, not of the type.
soa []Particle rather than @layout(soa) struct Particle. This is a
capability difference, not a spelling one: the same type can be one value in
one place and a struct-of-arrays in another, and the choice can be flipped
where it is measured.
D22a. A derived dimension is declared, not written inline.
dimension Velocity = Length / Time names the product once; there is no
Length/Time syntax in general type position. A dimension name then is a
type — the type of quantities carrying it — so speed: Velocity is how a
velocity is declared. Confining the arithmetic to one declaration keeps type
syntax free of operators and makes the concept nameable.
D23. Declaration-introducing words are contextual keywords — state,
unit, machine, derived, effect, dimension, enter, exit,
always, with, without, soa, aos. Gameplay code names variables
state and unit constantly; stealing those spellings would be a daily cost
for a one-time grammar convenience.
D24. A unit suffix on a number is just a symbol. Units are user declared, so the lexer records the spelling and sema resolves it.
D25. Block comments nest. The only reason to reach for /* */ is to
comment out a region, and a region often already contains a comment.
D26. Identifiers are Unicode XID and must already be in NFC. Letters from any script, but no emoji, symbols or invisible characters — a zero-width space in a name would hide one name inside another. Non-NFC names are rejected rather than normalized, which is what guarantees one name has one spelling. A combining mark that cannot compose is accepted, so Devanagari, Thai and Vietnamese names work.
Modules and the build
D27. A module is a directory, not a file. Its name is the package name plus the path below the source root. Adding a file never requires editing a module declaration, and the layout matches how a game splits into plugins.
D28. The manifest is declarative TOML. External tools — CI, a registry, a
dependency bot — must be able to read a package's name and version without a
Wolang interpreter. A build script (build.wo) can arrive beside the
manifest in wave 7+, exactly as build.rs sits beside Cargo.toml.
D29. Package discovery is not a query. The query engine can only track
facts it is told about, and the filesystem is not one of them. A workspace is
loaded explicitly, and loading one bumps the Workspace input.
Incremental compilation
D30. Query dependencies are discovered, not declared. Every Get() a
computation makes is recorded, so a computation and its dependency list can
never drift apart.
D31. A cached value may never carry a source position. Positions belong to the revision they were taken in; anything that reports a diagnostic resolves its spans against the current syntax tree.
D32. The item tree holds no positions and no syntax handles. Both shift when a body is edited, and an item tree that changes after a body-only edit would defeat its own purpose.
D33. The cache stores item trees only, and is never trusted. Tokens and trees are large and are needed only when the file is about to be read anyway. A stale, corrupt, truncated or foreign cache is discarded.
D34. A file outside a package resolves no imports. Checking a single file is a legitimate thing to do and must not invent errors.
Effects
D35. Effects and capabilities are one mechanism, inferred inside a module
and explicit on offer signatures. One mechanism covers determinism,
allocation-free hot paths, mod sandboxing, comptime purity and server/client
separation — five unrelated attribute checks collapse into one.
D35a. Effect inference is one query per module, not one per declaration. A per-declaration query turns mutual recursion into a query cycle, and the engine's cycle detection would then give an answer that depends on which declaration was asked about first. Order-dependent results in an incremental compiler are not a performance problem, they are a correctness one. The fixpoint therefore runs over a whole module at once, and reporting is a third query so that type checking — which collects the rows — does not depend on the module-wide answer it feeds.
D35b. An offer function with effects must state them. Inference stops
at the module boundary, because inferring across it would need
ModuleEffects(A) to ask ModuleEffects(B) and back again. Stating the row
on the signature is what the boundary buys, and it is where a reader looks
anyway.
D35c. An empty effect row is a claim, not ignorance. comptime needs to
say "nothing observable"; "not stated" is the absence of an entry. Conflating
the two would make every unannotated function look pure.
D35d. Ability conformance is declared, not structural. extend T as Ability
is the declaration. A type that happens to have the right methods has not
promised anything, and a bound is a promise.
Runtime policies
D36. Hot reload restarts an incompatible coroutine. Predictable and
explainable in one sentence, which matters more for editor iteration than
preserving state. @HotReload(FinishOld) or @HotReload(Cancel) overrides it
per function.
D37. A fault in a shipping build isolates the culprit. The coroutine or system that faulted stops, the rest of the game keeps running, and a report goes to telemetry. One broken enemy beats a drop to the menu. A project switches the policy in its manifest; a server usually should.
D38. Mods ship as .wo source. Readable, reviewable, patchable, and
learnable from — which is how strong modding communities actually work.
Compiled by the dev tier on the player's machine, or by the IR interpreter
where a JIT is not allowed.
D39. The dev tier gets a real register allocator (linear scan) from the start. Longer to build than accumulator stencils, but better code from day one rather than a rewrite once the benchmarks disappoint.
D40. An HTTP server is a first-party library, and it starts at wave 10 as the driver for the async design — not at the end as a showcase. A server is the harshest test a concurrency model gets: thousands of tasks, each mostly waiting, with cancellation and deadlines everywhere. Writing a minimal HTTP/1.1 server while structured concurrency is being designed is what keeps the design honest; writing it afterwards only proves the design was already fixed.
It lives in its own package (wolang-http), consumed through a path
dependency, which also exercises the package graph. The transport layer —
kqueue, epoll, io_uring — belongs in std (wave 13), not in the HTTP
library: a server is one user of readiness notification, not its owner.
Three language features make it unusually good rather than merely possible:
a arena per request is arena-per-request, which is how the fastest
servers already work but spelled as a language construct; effects and
capabilities give per-handler sandboxing checked at compile time, so a route
declared uses net cannot touch the filesystem and a plugin route can be
granted capabilities = "net" and nothing else; inferred error sets mean
fun Handle(r: Request) -> Response! without a hand-written error enum.
TLS is bound, not written — BoringSSL or rustls through the C ABI (wave 9). Nobody should write TLS, and pretending otherwise would be the one decision here that could not be walked back.
Code generation
D41. as converts between numbers and does nothing else, and a narrowing
conversion traps. D33 removed implicit widening, so as is the only
numeric conversion there is — not a reinterpretation of bits, which is what
unsafe is for. Converting anything that is not a number is E0124. A
conversion that cannot hold the value traps rather than keeping the low
bits, because overflow traps everywhere else in this language and a cast
that quietly changed a number would be the one place it does not.
D42. A generic function is never lowered; one specialization is emitted per
instantiation. A generic declaration is a recipe, and lowering it would put
a T in the IR that every back end would then need an answer for — the
verifier rightly rejects a call passing an s32 where a T is declared. The
specialization is named after its arguments, so two call sites that inferred
the same arguments share one copy. Cross-file instantiation needs
whole-program lowering and waits for the linking story.
D43. A trap in compiled code is a check before the operation, not a signal handler. This was wave 8's open question. A check costs an instruction and an untaken branch on every arithmetic operation; a handler is free on the hot path but is tied to the platform, cannot work for WASM, and makes every arithmetic instruction a potential non-local exit. The check is portable and predictable, and the plan is to remove it where range analysis proves it unnecessary — the same analysis that would let dead arithmetic be deleted (D15c). Paying for a check that an analysis can later remove is a better place to be than depending on a mechanism one target cannot have.
D44. The register allocator hands out callee-saved registers only. Nine integer registers and eight floating point ones instead of twenty-odd, and in exchange a call needs no spilling around it: every value the allocator placed survives a call because the callee is obliged to restore it. The prologue saves what the function used. For a dev tier that compiles fast and has to be right, removing the caller-save problem is worth more than the extra registers.
D45. An aggregate lives in the frame and is passed by pointer. A struct, a tuple, an array and an optional never get a register in this tier: the value of such an SSA name is an address. One rule then covers every aggregate however it was produced, and the return convention coincides with the platform's indirect return. Scattering fields across registers is an optimizing-tier decision.
D46. A trap is reported by a non-local exit. Generated code calls the host
through the one pointer it keeps, and the host records the trap and leaves
through a landing pad established when compiled code was entered. A generated
frame holds machine values and nothing that needs unwinding, so leaving
through one costs nothing — and the alternative, a check after every call,
would be paid on the path that does not fault. The context has to live
outside automatic storage: an object changed between setjmp and longjmp
has an indeterminate value afterwards, and the optimizer is entitled to fold
the trap kind back to the zero it was initialized with. It did.
D47. The trap vocabulary belongs to the IR, not to the interpreter. A division by zero is a property of the operation. Both tiers have to name it identically or the differential check between them would be comparing two vocabularies.
D48. A trap message says only what both tiers know. The interpreter could print the offending index; compiled code cannot put a runtime value into a string it built while compiling. Keeping the dynamic value out of the message is what makes the two tiers' outcomes comparable in full — and the value is in the frame, which is where a debugger looks for it.
D49. Every compiled function builds a frame record, even a leaf. Two instructions in the prologue, and in exchange a stack walk is two loads per frame and needs no unwind table at all. A trap deep in compiled code can name its callers, which is what makes a fault in generated code debuggable before there is a debugger.
D50. The dev tier compiles what it can and says what it did not. A
function holding a string, calling something uncompilable, or wanting more
argument registers than exist is left to the interpreter, and wolang asm
prints the reason. The interpreter is the reference semantics and is always
there, so declining is cheap — and a tier that guessed would be the one thing
nobody could debug.
D51. wolang run stays on the interpreter until the FFI arrives —
settled in wave 9: the FFI arrived, std.io.Print is written in Wolang,
and run uses the compiled tier when it can. --interpret forces the other
one. The reasoning is kept because it is why the dev tier was exercised
through wolang test first. Any
interesting main prints something, and printing goes through an intrinsic
the dev tier cannot call yet. The compiled tier is exercised where it is
honest to: wolang test runs every @Test in both tiers and compares, and
wolang bench times it. Running main through compiled code would also
quietly drop the step limit, which is a property of the run and not
something a tier gets to change.
The boundary
D58. A declared uses unsafe absorbs the obligation rather than
re-exporting it. unsafe is not a capability like io or net — those
say what the code needs from the outside world, and a caller has to know.
unsafe says someone checked, and the declaration that states it is the one
taking responsibility. Without this there is no such thing as a safe
abstraction over an unsafe primitive: std.io.Print would make every
program that prints an unsafe one, and a package with unsafe = "deny"
could not print at all.
D59. A program runs once. Wave 7 ran every program twice — once on the
unoptimized IR and once on the optimized one — and compared, which was worth
it while the only thing a program could do was fill a sink the driver owned.
Once a program can call write, running it twice writes twice. A side
effect is not something to repeat for the compiler's benefit, so the
comparison moved to wolang test: every @Test runs in both tiers, and a
test is the one place a program is known to terminate.
Names
D52. const and var, not let and var. Two visually distinct words
for two different things, rather than one word and a modifier. Mutable
locals are most of the locals gameplay code has, so var x must not cost
more to write than the immutable form -- and const is free to mean "never
changes" because compile-time evaluation is spelled comptime.
D53. task fun and wait, not async and await. Shorter, and wait
says what the machine does rather than naming a protocol. The suspension
point stays visible at the call site, which is the one thing the alternative
-- making suspension an effect in the uses row and dropping wait
entirely -- would have given up. That alternative is the more interesting
design and may come back when the effect system has the concurrency wave
behind it; until then a reader can see where a function yields.
D54. Owned<T> and Shared<T>, and arena for a scope. They say in
English what the relationship is: one owner, several owners, a region freed
all at once. arena is also the word the gameplay world already uses for
exactly this allocator.
D55. s32, not i32. Both halves of the width table are integers and
only one of them is signed, so s carries the information i does not.
Gameplay SDKs have spelled it this way for years. int and float remain
the aliases for code that does not care about a width, and f32/f64 keep
their spelling because there is no second letter to disagree about.
D56. text and Text, not str and String. Lowercase for the view,
PascalCase for the type that owns its bytes -- the same rule every declared
type in the language already follows. A view costs nothing and an owned
string allocates, and the capital letter is where that difference is
visible.
D57. []T and [N]T, not [T] and [T; N]. The brackets say what the
shape is before the element does, so a type reads left to right: a slice of
T, an array of N of them. It is also two characters shorter in the array
case, and the semicolon inside a type bracket was the one piece of
punctuation nobody could guess the meaning of.
The words these replaced -- let, async, await, region -- are
reserved rather than free, so writing one gets a diagnostic instead of a
confusing parse failure.
D60. Methods come before everything that wants them. A fun inside a
struct, a class or an extend is a member function with self as an
implicit first parameter, and a bare name in its body is a member of the
enclosing type -- which section 4.4 of the spec already promised. Until
this wave none of it was lowered, and worse, none of it was reported: a
method call produced an undefined value and the checker said nothing. So
methods are built first, and the holes get diagnostics, before signals,
machines or anything else that would be written on top of them.
D61. A class is a handle to a heap object, and a destructor is what frees
it. Wave 9 deferred Owned/Shared because both need "a type that frees
something when it goes out of scope" and defer had to exist first. It
does. So instantiation allocates, a binding that holds a handle drops it at
scope exit unless it was moved, and Owned/Shared are then two types
rather than two mechanisms.
D62. A signal has both a compile-time and a run-time side. Connections
that are known statically expand into direct calls at the Emit, which
costs nothing and covers the engine's own wiring. Run-time Connect over a
list of function values covers mods and anything dynamic, and waits for
function values to be a thing in the language. One word, two
implementations, and the fast one is the default rather than an
optimization.
D63. A derived value caches and is invalidated by a write. The spec
says it "recomputes when an input changes rather than on every read", which
is a claim about when, not a figure of speech: the compiler works out
which fields the expression reads and a write to any of them marks the
cache dirty. A read of a clean cache is a load. The alternative -- a method
without parentheses -- would be simpler and would make the spec's sentence
false, so the sentence stays and the analysis gets written.
D64. A keyword is a name where only a name can appear. After a ., and
where a declaration gives its own name, a keyword is read as a name:
o.tag, need std.task, var state: int. There is nothing for it to be
ambiguous with in those positions, and the alternative is a language that
makes its users rename their data -- tag, state, task and component
are words gameplay code wants. A keyword in an expression is still a
keyword, so a field named one is written self.tag rather than tag.
Making every declaration keyword contextual everywhere would remove that
last wrinkle and is a bigger change to the grammar than it looks.
D65. One address model, three tiers. A field of a place, an element of
a buffer and a method's receiver are all "somewhere inside something", and
each tier answers that its own way: the compiled tiers have real addresses,
and the interpreter -- whose values carry their shape rather than living in
memory -- has a place: a frame, a slot and the steps to walk from it. An
element step adds to the one before it, because a pointer to element three
plus two points at element five; a field step goes inwards. A place names
its frame, so a slice of a local array can be passed to a callee, and that
is safe for the same reason E0121 exists.
What this replaced was worse than slow: assigning to a field compiled to
nothing, assigning to an element compiled to nothing, and for over a
range lowered to an undefined value. Three silent wrong answers, found by
writing the programs a buffer needs.
D66. The length of a sequence is an operand, not a property of a type.
An array knows how long it is and a slice does not, so index.addr takes
the length as its third operand and the bounds check reads that. One check
covers an array, a slice and a borrowed string, and no tier has to know
which it is looking at. .len is an int for the same reason it exists:
it is read in arithmetic and compared with literals far more often than it
is handed to an allocator.
D67. One string in every tier, built by the runtime. A string is a pointer and a length -- interpreted, optimized and compiled alike -- and what builds one is a call into the runtime that both tiers make. Before this the interpreter held a C++ string and the compiled tier held a pair, so a program that formatted anything could only run in one tier, and an interpolated string dragged its whole function into the interpreter.
Equality is a call for the same reason: two pointers are not an answer to
"are these the same bytes", and comparing the pairs -- which is what
structural equality does -- said that two copies of "GET" were different
strings. Both tiers now call wolang_text_equal, and the interpreter calls
it directly rather than through a thunk, because the reference tier has to
work on a machine with no backend at all.
D68. arena { } is the allocator, not a promise about one. Everything
allocated inside the block -- objects, strings -- comes from one arena, and
the whole arena is released at the end, including when the way out is a
return or a break. One free for the lot, however much was built: a
request handler allocates as it likes and pays once, and twenty thousand
blocks allocating a million objects between them peak at three megabytes.
This is the answer to leaking for now, and a better answer than per-value
destructors for the workload that matters first. Owned<T> and a collector
are still ahead; what keeps a value from outliving its arena is E0121,
which has been enforced since wave 9.
D69. A server is a test the language cannot argue with. D40 put a
minimal HTTP server in this wave to drive the concurrency design, and it
did -- but what it mostly drove was everything else. Writing it turned up
ten holes, and every one was a wrong answer rather than a missing feature:
a buffer that a callee filled and nobody could see, a module-level constant
that lowered to an undefined value, a frame the dev tier declined so the
whole request path ran interpreted, a sub sp, sp, x9 that silently
allocated nothing.
None of those would have been found by writing another test for a feature that exists. The lesson is kept as a decision because it is a rule about how to build the rest: build the thing that uses the language end to end, and keep it in the repository where it runs on every change.
D70. The stack pointer is not a register like the others. In the
shifted-register form of add and sub on arm64, register 31 is the
zero register where the stack pointer would be -- so sub sp, sp, x9
allocates nothing and add x9, sp, x10 reads zero. The assembler now
refuses that encoding instead of emitting it, because the alternative is
an afternoon spent in a debugger on generated code. A large frame is
adjusted with two shifted immediates, which reach sixteen megabytes
between them and name the stack pointer legally.
D71. A machine is a type, and its methods are what the table means.
machine Door { state Closed { on Interact -> Opening } ... } declares a
type whose value is which state it is in. From the table the compiler
generates one method per event, one predicate per state, and the enter
and exit blocks as methods of their own -- so door.Interact() and
door.IsOpen() resolve with no special case above the member table, and a
machine from another module works for the same reason a class does.
A machine is a handle, like a class: an event changes which state it is
in, and a method that changes its object has to be given the object rather
than a copy. always is resolved when the table is lowered rather than at
run time -- entering a state that always leads on enters both, and every
state on the way runs its enter. A cycle of always stops at the state
it has already been through.
What is not here: on Interact after 3s, which needs a clock the language
does not have yet, and a wait inside enter, which would make a
transition suspend. Both are in the syntax and neither is lowered; the
spec marks the section a draft and this is the part that is still draft.
D72. A synthesized member needs an identity of its own. A machine is
one declaration and many functions, so (file, item) does not tell them
apart -- and a call that cannot tell two methods apart calls whichever came
first, which is what happened. Members and IR functions carry a slot
beside the item now, and the linker matches on all three.
D73. soa []T is a layout, not a type. The same record, the same
fields and the same code reading them; what changes is where the storage
puts them. A pass that touches one field of a wide record reads every
byte of every cache line it pulls in instead of one field's worth of each,
and a game spends most of its frame in exactly that kind of pass.
Each field gets an array of its own. Their sizes are summed in declaration order, which makes the start of a field's array the sizes before it times the number of elements -- and that is what keeps an address cheap: one multiply, not one per field. The count each array really holds is rounded up to sixteen, which is what makes the multiplier a compile-time constant and leaves every array sixteen-byte aligned for a vector load later.
What this does not do is make a function work on both layouts: the layout
is in the type, so fun Sum(bodies: soa []Body) takes the one it was
written for. Polymorphism over layout needs a generic parameter that
ranges over storage, which is not here yet.
D74. A call takes the type arguments it writes. Inference reads the
arguments, so a signature that mentions its parameter only in its result
-- SoaNew<Particle>(count), whose only argument is a number -- has
nothing to be inferred from. The explicit arguments are matched against
the parameters the signature mentions, in the order they appear, and
inference fills in the rest.
Found by what it broke rather than by thinking: before this, the call's
type stayed soa []T and every field offset was computed against a type
parameter, so every field of every element landed on top of the others and
a sum of four elements came out as the last value written four times.
D75. Arithmetic the compiler writes for itself is not checked. An
address is a multiply and two adds, and a trap in one of them would name a
line the program never wrote. The instruction carries is_unchecked, and
a tier emits it without the overflow check and the reference tier wraps
rather than traps.
The second reason is the one that mattered more: an instruction that cannot trap can be moved out of a loop. Hoisting a checked multiply out of a loop that never runs would trap where the program would not have, so the flag is what makes the arithmetic below hoistable at all.
D76. An element address is expanded into the arithmetic it is.
index.addr and soa.addr exist so that lowering needs no layout. Once
the layout is known, keeping them opaque costs more than it saves: the
base of a field's array and the length it is checked against are the same
every time round the loop, and nothing can hoist what it cannot see
inside. A pass with the layout in hand expands soa.addr into unchecked
arithmetic, grouped as (base + array) + index * size so that the half
about the field hoists out and the half about the element stays.
What stays behind is index.check, which is the part that is about this
element. It produces no value, so the pass that removes dead code deleted
it until it was marked as doing something -- a bounds check that
disappears is the kind of bug a benchmark reports as an improvement.
index.addr is left alone: the reference tier reads an array inside a
slot as a path rather than an address, so arithmetic is not something it
could stand in for.
D77. Two passes for the loops this language is for. Common subexpressions, one copy per block: reading a field, adding to it and writing it back asks for the same address twice. And loop invariants, hoisted into the one block that runs before the loop: the base of a slice's storage, its length, the start of a field's array.
Together they took a million-entity integration from 107 ms to 29, which is more than every other pass has been worth put together -- the passes that were there folded constants and promoted slots, and neither does anything for a loop over a buffer.
A loop is found by a jump backwards in reverse postorder and dominance, not by the jump alone. A resumed task is entered by a jump table that lands in the middle of its loops, so "the only way in from outside" is not the same question as "always runs before" -- which the task tests said in the form of a value used where it was not defined.
D78. A generic type is instantiated where it is built. Box<int>(7)
binds the declaration's parameters from what the call wrote, or from the
arguments when it wrote nothing, and the result is the instance rather
than the declaration. A field of it then reads as int, a layout can be
taken of it, and two instantiations are two layouts.
Members live on the declaration, so an instance's member table is the
declaration's with its parameters bound. That is built once, in the
resolution that answers MembersOf, rather than at each of the dozen
readers -- a layout, a lowering, an interpreter, a checker -- because each
one getting it right separately is a dozen chances to get it wrong. The
one that was wrong was the layout: an instance was not a named type to it,
so every field of a Box<int> sat at offset zero and reading one gave
another.
A method of a generic type is a recipe, like a generic function: one copy
per instantiation, keyed by the mangled name Box<s32>.Get. The
substitution comes from the receiver rather than from inference, which
makes it the simpler half -- by the time a method is called, what its
type stands for is already decided. And whether a method writes through
its receiver follows the declaration: an instance of a class is a handle
exactly when the class is.
D79. A generic is copied where it is used, with the names of where it
was written. The module a generic lives in cannot know what its
importers will ask for, so the copy is made in the module that asks. What
makes that work is that the copy borrows the defining file's view for as
long as it takes to lower: its syntax, its member tables, the types its
checker recorded. Store<T>'s body resolves SoaNew in the file that
wrote it, not in the one that called it.
Lowering therefore has a current file rather than one file, and everything it looks a name up in is keyed by that file. The bug that said so: a table of this module's declarations was searched by syntax id while another file's syntax was in force, and a constructor call came out as a call to an unrelated constant that happened to sit at the same index.
A specialization has no source identity. It is reached by the name it was
mangled with -- Store<Position>.Add -- because two instantiations of one
generic would otherwise claim the same (file, declaration) pair, and a
linker that resolves by that pair binds a call to whichever copy it finds
first. Which is a call into another type's layout.
D80. The reference tier reads and writes aggregates through pointers. A class holding a slice has a sixteen-byte field, so writing it means writing a pointer and a length at the offsets the layout gives. The interpreter had no way to do that at all: it switched on the value and every aggregate fell through to a trap. It now walks the layout, which is the same answer the compiled tiers get from the same store -- and a slice is spelled out rather than looked up, because neither of its fields is a member anyone declared.
What is still not possible is putting a local array into a heap object in that tier: a local lives in a frame slot, which the interpreter holds as a value rather than at an address, so there is no pointer to write. The storage an ECS keeps is allocated, so this has not come up outside a test that was written wrong.
D81. A load is reused until something writes. Within a block, reading the same address twice with nothing stored or called in between gives the same answer, so the second read is dropped. A loop that reads a field of its object for every use pays for it once instead.
What this does not do is move a load out of a loop. That needs to know that nothing the loop writes lands where the load reads, which is alias analysis -- and without it the base of a store's row and the length it is checked against are re-read a million times. It is the largest single thing between this and the layout difference showing up as more than a few percent, and it waits for the wave that brings the analysis.
D82. A system declares its access, and the schedule follows. mut T
writes, T reads, with T and without T filter -- which is reading
whether an entity has the component. From the declarations alone the
compiler lays the systems out in stages: everything in a stage may run at
once, because nothing in it writes what another touches. Two that would
share a stage and conflict are rejected rather than ordered arbitrarily
(E0127), because the arbitrary order is the bug a scheduler like this
exists to prevent.
The access lives in the syntax, not in the type: Query<mut Velocity, with Falling> resolves to one query type over the components it names, and
the modifiers are read off the declaration by the pass that schedules. One
exception travels in the type -- whether the component the loop binds is
written -- because foreach v in q has to know whether v may be assigned
to, and that is a checking question rather than a scheduling one.
Why in the compiler rather than in a library: the schedule is a property of every declaration at once, and nothing a library can see spans them. The graph is a module-level query, so two systems in different files are scheduled against each other and an edit to one rebuilds only what depends on the answer.
What is not here: a Query value built from a world, and therefore running
a system at all. The storage is a library (game.ecs), the schedule is the
compiler's, and the join between them -- which store a query reads, and how
a stage is handed to a thread -- is the next piece. Running the stages in
parallel needs threads in the runtime, which do not exist yet.
D83. The programs are generated, and the tiers are compared. Every hand-written codegen test runs one program through all three tiers and compares; the fuzzer writes the programs. A seed names one program for ever, so a disagreement can be looked at again -- which is the whole reason the generator uses its own xorshift rather than anything from the standard library, whose engines are free to differ between implementations.
It earned its keep in the first two hundred programs, with two miscompiles in passes that had been there for three waves:
- Constant folding was unsigned.
-2 / -10folded to one instead of zero, and every signed comparison on two constants was wrong the same way. Overflow was folded into its wrapped answer rather than left for the trap it is at run time. - An element address was treated as pure, so loop-invariant hoisting
moved one out of an
ifinside a loop and into the loop's preheader. The program returned nine before and trapped after: the bounds check is part of what the instruction does, and what can trap cannot be moved to a path that would not have reached it.
What the generator does not do is interesting too. It writes loops whose counter it keeps out of the body, because a body that may assign to the counter is a body that may never end -- and a compiled tier has no step limit to stop it with. A program the reference tier could not finish is skipped rather than compared: it says nothing about the other two.
Traps are compared rather than avoided. Division by zero, overflow and an index past the end are defined outcomes, so a tier that wraps where another traps is exactly the bug this is looking for.
D84. A derived object is a base object at the same address. A class's
fields are the fields it inherits, root first, and then its own. So a
handle to a Cube is a handle to a Square is a handle to a Shape, and
an upcast is nothing at run time -- the IR says so with a cast because
every type in it is exact, and the cast emits no instruction.
Everything that asks what a class has now asks along the chain: the
constructor takes the inherited fields first, a method reads a field its
base declared without qualification, and a field access through a handle
reaches an inherited member. Each of those was a separate place that only
knew about a type's own members, and each one was a separate bug report
waiting -- Cube has no member named tag for a field Square declared.
D85. The number in the object is the class's declaration. A virtual
call reads a token out of the object and uses it to find the body. The
token is (file << 32) | item -- which declaration this class is -- and
not an index into anything a module owns, because an object created in one
module is dispatched in another and the two have to agree.
Slots are numbered from the root: the root's virtual methods in
declaration order, then each derived class's new ones, with an override
taking the slot of what it overrides. That makes one number mean one
method everywhere the class is visible, including a module that only sees
the declaration.
The tiers differ only in how they get from the token to the body. The reference tier looks through the class tables of the modules it holds. The compiled tier asks the runtime, which the compiler told where each method ended up once the code was placed -- a C call per virtual call, which is the price of the number meaning the same thing in both tiers. Inlining that lookup is the obvious next step, and it is why the registry is a flat array rather than a hash map.
A class gets the header only when something in its chain is virtual, so a
class with no virtual methods costs exactly what it did before.
D86. An ability value is a pair: what it points at, and which class that is. Two words, like a slice, and for the same reason -- neither half is a member anyone declared, and a call on it needs both. The method is named by the ability's declaration, so the body is found without the call site knowing anything about what implements it.
A class hands over its handle, because that is already an address. A value type is copied onto the heap: an ability value is the one thing in this language that outlives the expression that made it -- a list of shapes is the point of having it -- and the reference tier's locals have no address that survives a call, so a borrow of a local could not work in every tier. Nothing frees the copy yet; that waits for destructors, and it is the next thing this needs.
A method of a value type takes self as a value, and an ability call has
only an address, so the table gets a one-line function that loads the
value and calls the method. Every vtable over value types needs that
thunk; a class needs none.
The same wolang_class_lookup answers both this and a virtual call,
because both ask the same question: this object, this method declaration,
which body? That is what made the selector worth having instead of a slot
number -- a class's own virtuals and every ability it implements share one
table without agreeing about an ordering.
D87. deinit is what a type does when a value of it goes away, and that
makes the type move rather than copy. The two halves are one feature: a
type that frees something cannot be in two places, so handing it to
something else is a move and the name it was in no longer holds it. A
type without a deinit is unaffected -- it copies, exactly as before --
which is what keeps the rule opt-in rather than a tax on every class.
A scope exit runs the destructor and then frees the object, through the
same machinery defer uses, which is why defer was built first (D61).
Whether it runs is a flag: set where the binding is initialized,
cleared where it is moved, tested on the way out. The flag rather than the
analysis, because lowering would otherwise need to know what the checker
worked out about every path -- and the flag folds away wherever the passes
can see both ends.
What counts as a move is the question the checker already answers for
String: a use is a borrow when it is read through -- b.size,
b.Method(), assigning to b.field -- and a move when the value itself is
handed over. Lowering follows the same rule, so the two cannot drift.
A parameter that was handed ownership destroys it on the way out. That is
what makes a move a transfer rather than a leak, and it is why a function
taking an owning type is written as taking the type: a borrow is written
&T.
D88. A module-level var has storage, and a module-level declaration
says its type. A const becomes a function the inliner folds, which is
right for something immutable; a var can be written, and a function that
recomputes the initial value is not storage. Before this, counter = counter + 1 compiled to nothing at all and nothing said so.
The storage belongs to the runtime, not to either tier, for the reason the class tables do: a program may be half interpreted and half compiled, and a counter written by one half has to be the counter read by the other. The token is the declaration, so both tiers ask for the same storage without anything handing out addresses. A global starts as its type's zero and takes the source's value in the file's init function, which runs before anything else -- and the storage is reset at the start of every run, so that running a program twice is running it twice rather than continuing it.
The type has to be written (E0130). Not a style rule: what a name means
to another file is worked out from signatures alone, and inferring this one
from its initializer would make a signature depend on a body -- which is
the thing the incremental design is built not to do.
D89. An enum is a tag and room for the largest payload. A variant that
carries nothing is the tag alone, which is what an enum was before this; a
variant that carries values puts them beside the tag, at one offset shared
by every variant. That is what lets a match read them from one place
once it knows which variant it is looking at.
The payload's type is the instruction's rather than the declaration's.
Nothing else would do: which variant it is is exactly what a match is
for, so the field table cannot name a type for the payload and the
instructions that build and read it carry the variant instead.
A variant that carries values is typed as a function from them to the
enum, which is how AiState.Chase(target, speed) ends up being an
ordinary call at the use site -- and how the checker knows what a pattern
binding holds.
D90. A T! is which it is, and the value when it is a value. The tag
is zero for a value and otherwise says which error, named by the error
tag's own symbol -- so the construction and the test agree without either
side knowing about the other, including across files.
An error tag is that number as a value. return Corrupt coerces it into
the pair; match compares against it. The alternative -- an undefined
value standing for the error -- is what the first version did, and an
undefined value is a trap in two of the three tiers.
? is an early return: if the tag is not zero, the same tag goes back as
this function's failure, with no value beside it. That is the whole point
of the operator -- the alternative is a match at every call -- and it is
why a signature's error set is the union of what its body propagates.
D91. The collections are written in Wolang, and a map takes its hashing
as values. List<T> and Map<K, V> are classes in std, over
ArrayNew and Free, with a deinit that gives the storage back. That
is the return on generics across modules and destructors: before either,
a growable container could not be written in the language at all.
A map compares and hashes through fun(K) -> int and fun(K, K) -> bool
rather than through an ability. An ability would need a Self type to say
"equal to another one of me", which the language does not have yet -- and
the shape this leads to is honest rather than apologetic: MapNew(IntHash, IntEquals) says what it does, and a map keyed by something unusual needs
no declaration anywhere.
Two things the library ran into first, and both are the language being right rather than in the way. A hash cannot be a 32-bit multiply that throws the high bits away, because throwing them away is the overflow trap -- so it is done in sixty-four bits and masked back. And a function value in a field had no way into memory in the reference tier until now: it is a module and an index there, so what goes into the field is the token a call turns back into the value.
D92. The platform's numbers stay in the platform layer. open is a C
call the language can make, but its flags are not the same number on two
systems -- O_CREAT is 64 on Linux and 512 on Darwin. A library written
in Wolang cannot say both, and the version that guessed opened a file with
no write permission on one of them, which is how the first std.fs test
failed. So the flags live in C, behind wolang_open_read and
wolang_open_write, and everything above them is Wolang.
The clocks are two functions rather than one for the same reason the language has two words for mutability: they answer different questions. A monotonic clock never goes backwards and is what "how long did that take" wants; a wall clock does go backwards when the system is corrected and is what "when was that" wants. One function would make the wrong one the default half the time.
D93. xs[a..b] is a view, and a slice of a slice is arithmetic. The
pointer is where element a is and the length is b - a, so no copy
happens and a view of part of a buffer costs two words. Both ends may be
the length itself -- an empty tail is a useful thing to ask for -- so the
bound each is checked against is one past it.
D94. A slot is not an address. &x on a whole local needs the address
of the frame slot, which the reference tier does not have: its places are
paths, not numbers. So there is an instruction for it -- the reference
tier answers with the slot, a tier with real frames answers with the
address -- and it is emitted only where an address is wanted.
The first version emitted it for every read and write of a local, which made every slot's address escape and stopped slot promotion dead: a loop counter went back to making a round trip through the frame, at a third of the speed. The lesson is the one the register allocator taught in wave 8: a change that makes something expressible has to be confined to where it is needed, because the optimizer's questions are about what cannot happen.
D95. A server asks which connections are ready. The blocking loop held
one client at a time: read waits, and everything else waits with it. The
polled loop asks the kernel which of the descriptors it is holding has
something to say, so one thread serves as many connections as the system
will give descriptors for -- and an idle connection costs one entry in a
kernel table rather than a thread.
Measured on this machine, against the same handler: 500 concurrent clients, 20,000 requests, 43,455 requests a second and no errors. The blocking loop cannot be measured the same way at all -- with 500 clients it serves them one after another.
The mechanism is the platform's and the loop is Wolang. kqueue here and
epoll there differ in structure size and field order, which is not
something a library written in this language can spell, so poll.h is
three calls and http.poll is a class over them. The same division as
std.fs (D92), for the same reason.
One request per ready connection per turn, rather than draining each: a client that keeps asking would otherwise hold the loop for as long as it likes. The listener is drained, because it reports readable once however many connections arrived.
D96. A type says how to walk itself with three names. for x in thing
asks thing.Iterate() for something with Next and Value, and that is
the whole protocol. An ability would be the obvious home for it, but an ability
that said "what I walk over" needs an associated type the language does
not have -- and three names need no new feature at all.
The loop is built in lowering rather than desugared in the parser, so a
diagnostic points at the loop the program wrote. E0131 is what a for
over something with none of those names reports, and it says which three
to add.
Walking borrows. for key in map asks the map how to be walked and reads
what comes back; it does not take what the name holds, so the map is
still there afterwards -- which the move analysis had to be told, because
an owning value handed to anything is otherwise a move (D87).
D97. Threads, a lock and one atomic, and the reference tier runs a
spawned body on the spot. The smallest set that makes "do these at the
same time" expressible: Spawn, Join, a mutex by handle, and an atomic
add. Everything above it -- a pool, a channel, a work queue -- is Wolang
over those, which is where it belongs.
A thread in the reference tier would be a second interpreter: its values
are not addresses and its frames are its own. So Spawn there runs the
body immediately, on the calling thread, and hands back a handle that is
already finished. A program whose answer does not depend on interleaving
gives the same answer in both tiers -- which is the only kind of program
the differential check can compare -- and one that does depend on it has
a bug worth finding rather than hiding.
A function value crosses the C boundary as what it is, an address. That
is what every threading API in C takes for a callback, and the rule that
kept it out (E0802) was about layouts this compiler decides
unilaterally, which a function pointer is not.
D98. A third tier: the same IR, through LLVM, into a binary. The
interpreter is the reference and the dev tier is what a developer waits
for; neither is what a program ships as. wolang build turns the
optimized IR into an object file and links it with the runtime archive,
and the result needs no compiler to run.
Through LLVM's C API rather than its C++ one, because this project builds
without RTTI. The translation is deliberately literal: one LLVM
instruction per IR instruction, a block parameter as a phi node -- the
form the IR was given so that this would be one line -- and every
aggregate as the address of storage made once in the entry block. Every
value is an integer or a float of its own width, and a pointer is an
i64: the optimizer has already turned element addresses into ordinary
arithmetic, so a value that is added to has to be a number, and it
becomes a pointer at the one place that reads through it.
Three things the dev tier does for free had to be said out loud here. A frame is zeroed, because a program that writes two bytes of a socket address and hands all sixteen to C needs the other fourteen to be zero. A trap reports and exits, because a binary has nobody to report an outcome to. And the class tables are registered at startup by a function the backend writes, because nothing is there to register them as it compiles.
What the backend cannot translate is named rather than guessed at: the driver prints every function left out, since a binary missing a body is incomplete and which body is what makes that worth fixing.
D99. A block gives nothing back by itself. A function that declares a
result returns it with return, or is written with => expr. A block
whose last statement happens to be an expression is not a return.
This was a hole rather than a decision: such a function compiled, and
returned whatever the register held. E0132 is the diagnostic, and
loop { } with no break in it is not an error because it never reaches
the end at all.
D100. A world the runtime holds, and three intrinsics to reach it.
component, Query and system with a schedule have existed since wave
11 with nowhere to put the data: a system was checked, laid out in a
stage, and could not run. The storage is one dense array per component
token -- the same (file << 32) | item number a class carries -- with an
entity index beside it, held by the runtime.
In the runtime rather than in Wolang because a world is a map from a
type to an array, and the language cannot spell a map whose keys are
types. A token can, so the library asks for the array it means. Three
things the compiler had to answer for that to be writable in Wolang:
TokenOf<T>() (the number, carried in a Token<T> so that the result
type says which), SizeOf(value) (how many bytes a value takes, which
only a tier knows -- hence type.size), and ViewFromRaw<T> (a view of
memory as values, the way BytesFromRaw is one of bytes).
Three real bugs fell out of writing it, each silent before:
E0133 -- a type parameter that appears nowhere in a signature can never
be said by a call, and such a call lowered to a body that was not there;
a generic body calling another generic of its own file looked for it in
the calling file and found a declaration instead; and return value
where T? was declared emitted no wrap at all, because the checker
records the type the context wanted and lowering believed it. The last
one is why Coerce now asks the value what it is rather than asking the
expression.
D101. A system is a function of the world, and a stage is a set of them
run at once. system S(q: Query<mut Position, Velocity>) compiles to
void S(uint64_t world, float delta): the query is not a parameter at run
time but a declaration of access, so what the body gets is a walk over
the entities of that world which have what it named. RunSystems(world, delta) is the schedule the compiler worked out -- stage by stage, with
the runtime starting a thread per system in a stage, which is what the
stage means (D82).
mut T in a query resolves to a mutable reference to the component. That
is what makes the loop's binding assignable -- a system that writes says
so once, in its signature -- and what the loop binds is where the
component is, not a copy of it.
What foreach costs was worth three rounds. A call per entity to ask
where the component is, with a lock and two lookups inside it, ran 100k
entities in 6.5 ms per frame. The walk goes along the first component's
column by index rather than along the smallest; the filter's columns are
resolved once per query instead of per entity; and the addresses are
handed over as one array, so the loop itself is arithmetic. Same answer,
1.2 ms. What is left is one lookup per entity per extra component the
filter names, and an array of pointers built per query -- both of which
archetype storage would remove, and neither of which is worth the
complexity before something needs it.
D102. Owned<T> and Shared<T> are what destructors were for. A
class value is a handle to a heap object and nothing frees one: there is
no collector, so the language has to let a program say when something goes
away. Owned<T> is sole ownership -- moving it hands the value over, and
its destructor frees it. Shared<T> is a count beside the value, with
Clone to take another handle and the last one out freeing it. Both are
ordinary Wolang in std.own, over deinit, moves and the three
intrinsics of D100.
No implicit copy for either, and that falls out rather than being decided: a type with a destructor is moved rather than copied (D87), so taking another handle is something the program says.
Two compiler bugs came out of writing them. A type could not construct one
of itself -- Holder<T>(value) inside class Holder<T> bound T to the
T in scope, and a check that read "bound to itself" as "nothing bound
it" rejected it. And a runtime function declared both by a library
(extern "C" fun wolang_object_free(at: *mut u8)) and by lowering (where
a handle is a word) kept one declaration and two opinions about its
parameter; a call now says what the surviving declaration says.
D103. A comptime call with constant arguments is a constant. One
with no arguments was already evaluated inside the compiler, to check that
it could be; one with arguments needed the call site, which is this. The
fold happens on the IR, before the passes: before them so that the
reference tier sees the same program, and before inlining in particular,
which would otherwise take the call apart and leave nothing to evaluate.
Fib(20) + 1 in a body is one const.int 6766 in every tier.
It runs again after the passes, for the calls whose arguments only became
constant once propagation had been through. A trap is reported the way it
is for one with no arguments (E0901), because a comptime call that
divides by zero is the program telling the compiler something is wrong.
D104. trait is spelled ability. The concept is unchanged -- a set
of requirements a type can satisfy, with static dispatch through a bound
and dynamic dispatch through a value (D86) -- and the word is the one a
gameplay programmer already uses for "what this thing can do".
ability Drawable { ... } and extend Circle as Drawable { ... } read as
one sentence, which trait never quite did.
The old spelling stays reserved so that a file written before the rename
says trait is now spelled ability rather than failing to parse
somewhere further along.
D105. The bounds check a loop already made is removed. while i < bodies.len { bodies[i].x } compared the same two values twice: once
because the program said so, and once because reading an element must not
be allowed to read anything else. The second is removed where the first
is the same comparison against the same length, on the edge where it was
true -- the one shape that needs no proof about arithmetic.
Two things had to come first. A field read of something built right here
is forwarded (extract 1 of a soa.new is the count it was given), which
is what makes the length of SoaNew<Body>(kCount) the constant kCount
again; and facts are inherited only along an edge into a block with one
way in, which is what makes one pass over the blocks sound -- a loop
header has two ways in, so it starts knowing nothing, and its body is
told what the branch said.
The 1M-entity integration went from 43.3 ms to 25.1 ms, and the one-field
pass from 44.7 to 22.3. What is left in the loop is a multiply, four adds
and the loads -- and the AoS variant now wins the integration benchmark
(21.1 ms against 25.1), which is the honest answer for a pass that
touches four fields of six: soa is for the ones that touch one or two,
where it is ahead by a quarter.
D106. A closure is a function written where it is used, and it takes
nothing with it. |n| n + 1 becomes a function of the module with the
body the expression wrote, and its value is that function's address --
which is what a fun(...) value already is in every tier. Nothing is
allocated, nothing is boxed, and a closure crosses the C boundary like
any other function pointer.
The parameter types come from the type the closure is being given to:
Apply(|n| n + 1, 10) where Apply takes a fun(int) -> int needs no
annotation, and one written with no such context says so (E0134).
Capturing is a diagnostic rather than a feature (E0135), and the message
names the local and what to do instead. The reason is the same one behind
the rest of the language: a captured local has to live somewhere after the
frame goes away, and that somewhere is either an allocation the program
did not ask for or a lifetime nobody checked. A type with the state and a
method on it says both, and costs nothing that is not written down.
What a capturing closure would cost is worth writing down, because it is the next decision here and not a small one: a function value would become a pair -- code and environment -- which changes its layout, the indirect call in three back ends, what crosses to C, and who frees the environment. Said plainly rather than left as "later".
D107. WrappingAdd, WrappingSub and WrappingMul. Everything else
in this language traps on overflow, which is right for a program counting
damage and wrong for one hashing a key or stepping a generator: those want
the low bits of a wrapped result. The three have no body -- what they
stand for is the instruction the compiler already writes for its own
address arithmetic -- and they are what made std.random and a decent
hash writable in Wolang rather than in C.
Two things fell out of writing std.sort over them. A closure passed to a
generic now gets its parameter types from the parameter with the type
variables filled in by the arguments before it -- Sort(items, |a, b| a < b) binds T from items and the closure from T. And a generic that
calls itself records that its own parameter stands for itself: inference
has nothing to do there, recording it as "no arguments" left the call with
no instantiation to find, and a recursive Quicksort lowered to an
undef.
D108. An import used only by an enum's payload counts as used. Array(items: List<Json>)
is the only mention of List in a module, and the unused-import warning
did not see it: variant payload types are resolved where the enum is
checked, and that walk did not record the names it used.
D109. A method on a reference is the method of what it refers to.
Without it an owning value could not be lent at all: handing a Builder
to a helper moves it (D87), so the way to let the helper write into one is
&mut Builder -- and a reference with no members made that unusable. A
reference to a class holds the handle, so the handle is loaded out of it;
a reference to a value type already is the address a method takes. &x
borrows rather than moves, and a reference is a pointer in the IR, which
is what keeps a &mut Builder parameter and the &mut builder passed to
it the same type.
D110. + joins two strings. Interpolation covers most of it --
"{a}{b}" is the same call -- but a program building a string from pieces
it has in hand reads better with +, and every user tries it first. A
string plus a number is still an error, with a message that says which
two things + takes.
D111. Five places where lowering believed the checker's recorded type.
The checker records the type the context wanted at an expression. That
is what makes Accepts(T?, T) work, and lowering read it as "what this
expression is" in five places -- each of which produced a value of the
wrong shape with no diagnostic anywhere:
- a call whose result is used where a
T?is expected was typedT?while the function returned aT; - a field read in the same position was typed
T?, so an enum's bytes were read as an optional's; return valuewhereT?was declared emitted no wrap at all (D100).
The rule now: an instruction's type is what produces it -- the callee's
result, the member's type, the value's own type -- and the conversion to
what the context wanted is a Coerce like every other.
Three more bugs came out of the same work, each found by writing
std.json rather than by a test expecting them: the drop flags of one
function were still in scope while the next was lowered, so a flag's
instruction id meant something else entirely; local.addr in the
reference tier answered with a frame-relative slot, which meant a
different slot in the callee it was passed to; and storing or loading a
tagged enum through a pointer used an aggregate's field offsets, where an
enum's payload is at the offsets that variant gave it.
D112. A request is views, and a router is a list of patterns. What a server needs beyond a method and a path -- headers, a query string, a body, path parameters, a status and content type -- is now there, and all of it costs no allocation: a request's headers are a fixed run of name/value views into the connection's buffer, and so are a response's.
Fixed rather than grown, and that is the decision. A List owns its
storage, so a request holding one would be a value that has to be
destroyed -- while a request is built by a parser, handed to a router,
rebuilt with its parameters, given to a handler and dropped at the end of
a turn. Every one of those steps would be a question about who frees it,
and two of them got the answer wrong before this. A fixed run has no
answer to give: twenty-four headers and eight parameters, and a request
with more keeps the first ones.
The router matches segment by segment -- :name takes a segment, *
takes the rest -- in the order the routes were added. No tree and no
regular expressions: an API has tens of routes, and the walk is what makes
:id cost a pair in an array. A path that matches with the wrong method
is 405 rather than 404, which is the difference between "no such
thing" and "not like that".
A write through a mutable reference is the reference's business rather
than the binding's: fun Matches(.., into: &mut Request) says the thing
referred to may be written, and the name holding the reference is not
what is being assigned.
D113. Freeing storage an arena handed out is not a free. A
destructor that runs inside an arena block frees what it owns -- and
inside a block, what it owns came from the arena, which reclaims it when
the block ends. free on that is a pointer the allocator never gave
anyone, and it aborted the process on the first request a server served
with a string builder in it.
So wolang_object_free asks the arena first. Not a flag on the pointer or
a header before it: the arena knows its chunks, the check is a walk over a
handful of ranges, and it happens only on a free. The arena stack is also
per thread now, which it had to be the moment a stage of systems ran in
threads (D97).
D114. An optional expectation is not an arithmetic one. -total where
an int? is wanted is a negation of an int; passing the optional down
made the operand an optional and then reported - as needing a number.
The expectation is stripped of its optional before it reaches an
arithmetic operand -- which is the checking-side twin of D111, where the
same confusion made lowering emit the operation at the wrong type.
D115. Arguments are given to the runtime, and an address can be a
number. A program's arguments come from whatever started it -- main's
argv in a binary, whatever followed -- when the driver ran it -- so
the entry point hands them over and everything else asks the runtime.
std.env is that, plus the environment: Lookup answers with an optional
because a variable set to nothing and one that is not there mean different
things.
p as u64 is now allowed in unsafe code. Comparing an address with
zero, hashing one, handing one to C as a word: all three were
unspellable, and the promise that a number is an address is exactly what
unsafe records.
D116. wolang doc writes what a package offers, from the source. A
signature is taken from the program's own spelling rather than rebuilt
from the syntax tree: a default value, a where clause and the order of
the effects all say something, and a renderer that rebuilt them would be
a second grammar to keep in step. Markdown because it reads as text and
renders where the code lives.
Only what another module can name: a private helper is an implementation
detail by definition, and docs/api/*.md is what the four libraries
offer today -- regenerated with one command rather than written by hand.
D117. The editor's end is a protocol over the queries that already
exist. wolang lsp speaks the Language Server Protocol on stdin and
answers with what the compiler already knows: the diagnostics check
produces, the types the checker recorded, the declaration the resolution
points at, the text the formatter writes. Nothing in it is a second front
end, which is the only way an editor and a build can agree about a file.
Whole-document sync rather than incremental edits: the compiler holds the
text, and an incremental edit would be a second copy of it to keep in
step. Its own small JSON reader rather than std.json -- that one is
Wolang, and this is the compiler talking over a pipe before any Wolang has
run.
What it answers today: diagnostics on open and on change, hover with the type and the doc comment, go to definition for anything the file resolution knows, the file's symbols, formatting, and completion from file scope. A completion that knew about locals needs a scope per position, which the checker does not record yet -- said here rather than left as a surprise.
D118. The IR carries where every instruction came from. A side table
parallel to the instructions, not a field: an Inst is a fixed size on
purpose and a span is three words. Every pass that makes an instruction
keeps the table in step, and the inliner copies the callee's span --
an inlined body is still the callee's code, and a trap in it should name
the callee's line.
Three things come out of it. A trap says
app.wo:4:10 (@main:%55): index is outside a length of 4 in both tiers
instead of naming an instruction number. wolang build emits DWARF, so a
debugger stops on a line of Wolang, steps statement by statement, and
names Wolang frames in a backtrace -- which needed dsymutil as well,
because on this platform the line table stays in the object file and the
binary holds only a map into it. And a profiler can attribute a sample to
a function, which is the next thing this makes possible.
--no-debug leaves it all out, which is a fine thing for a release build
to ask for.
D119. A method may have type parameters of its own. fun Map<T>(f: fun(int) -> T) -> T on a plain class Box is a recipe like any other
generic: one copy per set of arguments, named with them so that
Box.Map<int> and Box.Map<f32> are two functions. It type-checked
before this and lowered to a call with no body behind it, which is the
worst kind of gap -- silent.
Two things in the checker had to give way. Half an answer is enough for a
closure: fun(int) -> T says what the closure takes while its result is
what is being inferred, so the hint is passed down and the result is
taken from the body. And an expectation that still mentions a type
parameter is a pattern rather than a type -- adapting against it
reported the very answer that binds the parameter as a mismatch.
D120. A tag-only enum round-trips through memory as one field. The
interpreter holds an enum with no payload as a one-field aggregate --
which is what match reads with extract 0 -- and memory holds it as a
number. A global one was written as nothing and read back as a bare
number, so match on it trapped in the interpreter while both compiled
tiers answered: the same producer-and-consumer disagreement as D111, one
tier further down. Found by std.log, whose level is a global enum.
D121. The libraries a real program reaches for. std.fmt (columns,
bases, decimals, sizes, durations), std.path (textual, no filesystem),
std.utf8 (code points, kept out of std.text because most programs
never need them), std.hash (FNV-1a, with the warning that it is not
keyed), std.encoding (base64, hex, URL), std.net (a socket floor of
its own, so a program can speak something other than HTTP), std.process
(exit, pid, a shell command whose string is the caller's
responsibility), std.log (levels, a global threshold, standard error)
and std.io's other half (stdin by line, stderr).
std.time grew a calendar: civil-from-days both ways, ISO 8601, and the
date format HTTP wants. No time zones -- they are a database that changes
four times a year, and a program that needs one should read that database
rather than have this guess.
Writing them found two miscompilations. An inlined WrappingAdd lost its
"do not trap" flag, because the inliner copied an instruction field by
field and that field was not in the list -- so a hash overflowed where it
is supposed to wrap.
D122. A request is read whole before a handler sees it. The old loop
parsed whatever one read returned, which works until a body crosses a
packet boundary -- that is, until someone posts something. Every
connection now has a buffer of its own: bytes accumulate until
Content-Length says the request is all there, the handler runs, and
what arrived after it stays for the next turn, which is what makes
pipelining work. Limits come with it -- a head of 16 KiB and a request of
one megabyte by default, both answered with 413 rather than by growing
-- and so do the two things the loop has to know before there is a
Request to ask: the length, and whether the client said Connection: close. HEAD is a GET whose body is not written, and the length still
says what it would have been.
D123. A method is not reachable by its bare name. A method lives in
the module's function table under its owner's name, and it was also
findable by its bare name -- so a plain call to a function called the
same thing went to the method, with no receiver. The verifier caught it
and nothing else did: Bytes(value) in a file with a class that has a
Bytes method called the method.
D124. An offset too large for either addressing form is computed. A frame with a four-kilobyte buffer in it puts the locals after it out of reach of both the scaled and the unscaled form, and the assembler insisted one of them fit. It now folds the offset into x17 -- the register the emitter keeps out of the allocator's pool for exactly this -- through the immediate add and subtract, which are the only forms that may name the stack pointer.
D125. A layout is a question about a type, not about a file. The file
being run may never have named the List<int> another file of the
package built, and a member table lives in a file's resolution: the type
came back with no fields at all, a field address fell back to offset
zero, and one field was written over another. A layout store now sees
every checked file's tables, and a field it cannot find is a trap rather
than a guess.
D126. Mounting a router is prefixing its routes. A route is a pattern
and a function, so app.Mount("/api/v1", &api) copies them with the
prefix in front -- and the match stays one walk over segments instead of
one walk per level. OPTIONS and 405 both answer with the methods a
path has routes for.
D127. The IR has one string type. Owned or borrowed, a string is a
pointer and a length; who frees it is tracked beside the value, in the
ownership flags, and not in the type. Keeping two made a branch that
yielded an interpolated string into a variable the checker typed as owned
a type mismatch -- which the verifier reported and which stopped the
whole module from compiling. Erasing string to str also widened what
the dev tier accepts: a function returning an owned string is now
compiled rather than left to the interpreter.
D128. Wolang embeds behind a C ABI. include/wolang/embed/wolang.h
is one header with no C++ in it: make a vm, load a package or some
source, call a function by name, and bind your own functions so the
script can call back. The compiled tier runs what it can and the
interpreter runs the rest -- the same arrangement wolang run has, which
is why a host gets machine-code speed without asking for it.
Three decisions worth stating. A host's functions are resolved through a
registry the runtime consults before the loader, so a program that
binds host_log gets the host's and not one that happens to be in the
process. A string crosses as a pointer and a length, which is what both
tiers hold one as, and a text the program answered is copied out --
because the program's own copy may live in an arena that is about to be
given back. And what a host may call is what the package offered: a
helper a script did not export is no more part of what it offers a host
than of what it offers another module.
This is not a sandbox, and the header says so: a script can declare
extern "C" and reach anything the process can, the same way the
standard library does.
D129. A package says what it links against. [link] libraries = "SDL2, m" in a manifest, with paths and frameworks beside it. wolang build passes them to the linker; wolang run has no link step, so it
opens them in the process instead -- which is what makes the same
extern "C" declaration work in all three tiers. A package binding a C
library is the package that knows which library, so this travels with it
rather than with the program.
D130. wolang bind reads a C header. The system's own preprocessor
expands it first, because a header is C and C has a preprocessor; what
this reads is declarations. Out comes a file of extern "C" fun,
constants from enum members, and aliases for simple typedefs -- a
starting point a person edits, not a binding pretending to be complete.
The honest parts are the limits: a struct by value, a variadic and a type
with no mapping are reported rather than dropped, a pointer to an
opaque struct is an address, and a callback parameter is an address too.
175 declarations out of the system's math.h compile as they are.
D131. Three machines, because there are three answers. The compiler has an interpreter, an ARM64 tier and an x86-64 tier, and they have to agree. A laptop can check two of them, which is the whole reason there is a CI file: Linux x86-64 and macOS ARM64, debug and release, with the sanitizers on one of them. Every job runs the unit tests, each library's own tests, the differential fuzzer with the run number as its seed -- so the programs differ run to run -- the examples, and the formatter against itself. A second job links the embedding library into a C program and runs a script through it, because an embedding API that is not called from C is a guess.
D132. A package starts with one command. wolang new game writes a
manifest, a source file with a main and a test next to it, a README and
a .gitignore -- four files a person reads rather than twenty they skip.
--lib writes a library instead. Finding the standard library is the
only hard part: WOLANG_STD if it is set, a tree the new package sits
inside, <prefix>/lib/std next to an installed compiler, and the tree
this compiler was built from -- in that order.
wolang test . now runs the tests of what was asked for rather than of
everything it depends on, which is what makes a new package's output one
line instead of seventy-four. --all asks for the rest. And cmake --install lays out exactly what new looks for.
D133. An interpolated string is given back. A program that built
strings in a loop grew without bound: "value {n} of many" allocates,
and nothing freed it. Three things were missing, and all three are the
same mistake in different places.
The binding owns it. An owned string is a type of its own (Text), so
the scope exit frees one the same way it destroys a class -- through the
flag that says whether the name still holds it. Which needed the
unerased type, because erasing a string to its two words is exactly
what loses the difference between owning the bytes and viewing someone
else's.
The pieces go back as they are consumed. A concatenation copies both
sides into a new allocation, so "a {b} c {d}" frees each intermediate
immediately: one live string instead of one per piece.
And a borrow is not a move. The checker records which uses are implicit
borrows -- an owned string handed where a text is wanted -- and the
merge from per-declaration results into the file's result was dropping
that list. So every string passed to a function looked like a move,
nobody owned it, and nobody freed it. Twenty million interpolations now
run in a flat twenty-six megabytes; before, the same program reached a
gigabyte.
D134. The date header is written once a second. HTTP dates have one-second resolution, so a server answering tens of thousands of requests a second can write it once and hand out the same string. Worth the two globals: it is a calendar calculation and seven allocations per response otherwise. The server does 39 700 requests a second in the JIT and 42 900 as an ahead-of-time binary, in 2 MB of resident memory, and 28 500 with a log line per request -- which is what a line to standard error costs.
D135. A port, one listener per thread. ServeMany(app, port, most, threads) runs the same application from several threads, each with its
own listener on the same port through SO_REUSEPORT: the kernel spreads
connections across the queues, so no two threads wake for the same one.
Sharing a single listener instead makes every thread wake for every
connection, which is the thundering herd. Zero threads means one per
core, and a system without SO_REUSEPORT falls back to one listener --
which still works with one thread.
What a program keeps in a module-level var is now shared across those
threads, and whether that is safe is the program's business: a counter
wants AtomicAdd and a table wants a Mutex. Said out loud rather than
left to be discovered.
D136. What ab found. Three bugs, in a morning, from pointing a real
benchmarking client at the server.
A 1.0 client left hanging: we answered Connection: keep-alive to
everyone, and an HTTP/1.0 client that did not ask for it waits for the
connection to close to know the response ended. The version is now part
of the request, and it decides what happens when the client says nothing.
A global holding arena memory: the cached date string was built
inside a request's arena block, so the block reclaimed it and the
global pointed at memory the next request wrote over -- which ab saw as
responses of differing length. The date lives in a global buffer now,
and the lesson generalizes: a global that holds a string built inside an
arena block is a dangling pointer the moment the block ends.
A global with no initializer was no global at all: var date_bytes: [40]u8 never got storage, because the declaration path checked for an
initializer before checking for mutability -- so reading it called a
function with no body. A var with no initializer is storage that starts
as its type's zero, which is what the runtime's storage already is.
With all three fixed: 53 900 requests a second on one thread and 58 900
on eight, with keep-alive and no failures -- and ab itself is the
bottleneck at that point.
D137. The embedding API is something you ship. A host wants more than
"load and call": a function looked up once and called every frame, the
script's module-level vars as settings it owns, the program's output in
its own console, a step limit for code it did not write, and a reload
when a file changes. All of that is in the C header now, and wolang.hpp
is the same API with the ceremony removed -- header only, over the C ABI
rather than over the compiler's own headers, so a host built with another
compiler still links.
Two things had to change underneath. Module-level vars were reset
before every run, which is right for a test and a differential check
(D88) and wrong for a host: it sets a global and then calls a function,
and the reset between the two wiped what it set. Runs now say whether to
keep them. And a host's output is captured by registering write in the
runtime's symbol table, which is consulted before the loader -- so a
program that prints reaches the host without knowing it.
Shipping it is the other half: cmake --install lays out headers, the
static and shared libraries, the standard library's sources, a CMake
package (find_package(Wolang CONFIG) → Wolang::embed) and a
relocatable pkg-config file. A release builds that layout per platform
and an embedding-only archive beside it. CI links a host project through
find_package, because an embedding API nobody links from outside the
tree is a guess.
D138. A closure takes a copy of what it reads. |n| n * factor now
works: the names a closure's body reads from the function around it are
copied into an environment, and the closure's value is the pair every
function value now is -- where the code is, and what it closed over. A
plain function closed over nothing, so a call through a value asks which
of the two shapes it has: one compare and a branch, in the IR, so every
tier gets the answer from the one place that knows it.
The environment lives as long as the call that made it, and is given back
when that scope ends. Which is the whole of the restriction: a closure
that captures may be passed down -- a comparator, a callback, a loop
body, which is what closures are mostly for -- but returning one, storing
one in a module-level var or putting one in a field is an error
(E0136), because what it captured would be gone. State that outlives a
call is what a class with a method is for; an environment with a lifetime
of its own is an allocation someone has to own, and a class is where this
language says who.
Three things had to give way underneath. A function value became two words, which changed its layout, how it is read out of memory, and what crosses into C -- a C function takes the code word, because it has no idea what an environment is, and the same goes for a signal's handler list. The checker records which names each closure captured, in the order it read them, which is the environment's layout. And the merge from per-declaration results into a file's was dropping that list, the same way it dropped the borrows (D133) -- two lists added to a merge that should have had them from the start.
std.list gained what closures are for: Any, All, Count, Find,
Each, Fold, MapInPlace and Keep, which is filtering without
allocating. And List.Items() hands out the part of its storage that is
in use, through a std.mem.Prefix that is the same pair-making
instruction ViewFromRaw is.
D139. wolang run --profile says where the time went. A sampling
profiler: a timer fires a thousand times a second, the handler writes
down the program counter, and afterwards each sample is attributed to the
function whose compiled code it landed in. Sampling rather than counting,
because counting changes what it measures -- and because a profile of the
machine-code tier is the one worth having. Samples that land outside the
program's own code -- the runtime, the allocator, the kernel -- are one
line rather than hidden, because "most of the time is in the allocator"
is an answer.
It names a generic instantiation and a closure the way the rest of the
compiler does, so @Partition<s32> and @closure$2 are what a profile
of Sort(numbers, |a, b| a < b) shows.
D140. wolang repl is a compiler in a loop. Each line becomes a
file of one session package: a declaration joins the session, and
anything else becomes a function that shows what the line came to. One vm
for the whole session, so a var keeps what was written into it and a
fun stays declared; a line that does not compile is dropped by
rebuilding the session from the lines that did. <?> at the end of what
a line printed means its value was nothing -- the line was a statement --
so what it printed itself is what gets shown.
It is built on the embedding API rather than on anything new, which is how three real bugs in that API came out:
A host's source could not say need. A file added by a host belonged
to no package, so need std.io resolved to nothing. Host sources now
belong to one package of their own -- made once, not per call -- whose
dependencies are every package the host loaded.
A module-level var of another file was read as a constant: lowering
only knew its own file's globals, so a cross-file read gave back the
value the variable started with and threw every write away. It is the
same instruction with the same token now, built from the declaration the
binding points at.
And a name is not unique across a program. wolang_vm_call found a
function by name and the compiled tier then looked its name up again --
so a session that declared Peek reached std.json's private Peek
instead, with the wrong number of arguments. Lookups prefer the most
recently loaded definition with a matching arity, and the call is made by
the declaration a function came from rather than by its name.
The interpreter's own Print now reaches a host too: it collects what a
program printed rather than writing it, and a host that asked for the
output gets that as well.
D141. A dependency may be fetched, and what was fetched is written
down. engine = { git = "...", tag = "v1.2.0", dir = "lib/engine" } in
a manifest, wolang fetch to get it, and wolang.lock recording the
repository, the revision that was asked for and the commit it resolved
to -- which is what makes a second machine build the same code.
Three decisions. Fetching is git through the git on the path: a
package manager that cannot be told "use the git you already have" is one
that fights the machine it runs on. Only fetch reaches the network --
check on a missing dependency is an error with a fix in it (E0061),
because a compiler that downloads something because a file was saved is
not one anyone can trust. And a revision is required: a dependency
without one changes under you, and the lock file would have nothing to
record.
Where a checkout lands depends only on the repository and the revision, so two packages wanting the same dependency share one copy, and two revisions of the same repository are two directories.
D142. --record and --replay make a run happen again. Everything
the machine answers -- the clock, standard input, the process id -- is
written down on the way out and handed back on the way in, in order. So
"it only goes wrong on the server" becomes a file: record there, replay
on a laptop, and the program sees the same clock, the same input and the
same random numbers, because the seed came from the clock.
The journal is text, with the bytes in hexadecimal: one that cannot be read in an editor is one nobody looks at when the replay disagrees. The shims take over through the runtime's symbol registry (D137), so a program reaches them without knowing they are there -- and a run with no journal touches none of it.
What it does not do is make threads deterministic: they interleave as the machine decides, and a program whose outcome depends on that is not fixed by this. Said out loud rather than discovered.
D144. A client, because serving is half of HTTP. Fetch(url),
PostJson(url, body), and Open/Ask for a program making several
calls on one connection -- over std.net, which is the same socket floor
the server stands on. A URL is taken apart into a host, a port and a
target; the answer is read until Content-Length says it is whole, which
is a loop because a read stops wherever it likes.
Two things are said out loud rather than pretended. There is no TLS:
https answers NeedsTls, because a TLS implementation is not something
this language should contain and a package binding one through [link]
is where it belongs. And a reply's bytes are one allocation that the
reply's parts are views into -- so they outlive the call, and a program
making many calls puts them in an arena.
Writing it found a checker bug that had been hiding behind error
propagation: Ok(value) in a match on a T! never got a type, because
Ok is not a declared member of anything. An error type is accepted
everywhere, so nothing complained until a reference to such a binding
turned up -- &<error> against &Thing, a long way from the cause.
D145. Output is buffered when a program says so. std.io.Out
collects and writes in blocks: fifty thousand lines through Print is
fifty thousand system calls and half the time in the kernel; through
Out it is one write per sixty-four kilobytes, and the same program
spends no system time at all.
Explicit rather than automatic, because the two callers want opposite
things: a program whose output is a log wants every line out now, and
one writing a report wants it fast. Flush is where a program says
which, and a writer that goes out of scope flushes itself -- one that
dropped the last line would be a writer nobody could trust.
D146. A deque and a heap, because a list is not either of them.
std.deque is a ring over one allocation: pushing and popping at either
end is arithmetic, which is what a work list and a breadth-first walk
need and what taking from the front of a List cannot be. Growing it
lays it out straight again, so the arithmetic stays the same afterwards.
std.heap is a binary heap whose order comes in as a function value, the
way Sort's does: HeapNew<int>(|a, b| a < b) is a smallest-first
queue. Sorting a whole list to take one element off it is the thing it
exists to avoid.
Both Pop the same way: the caller checks Len first. An optional of
T would be a second shape to lay out for every T, and a queue with
nothing in it has nothing to give.
D147. A parallel loop is a closure and a range. ParallelFor(count, threads, |from, to| ...) splits [0, count) into chunks and runs one
per thread, each with a range to itself -- so the body is an ordinary
loop and the work is divided with no locking. Zero threads means one per
core, the calling thread takes the last chunk (so a one-thread loop
spawns nothing), and the remainder is spread one element per chunk rather
than left to the last. Two hundred thousand elements of real work: 851 ms
on one thread, 142 ms on eight.
It is declared with no body, like TokenOf and RunSystems: what the
runtime needs is the two halves of the body's function value -- the
code and what it closed over (D138) -- and taking a function value apart
is something only lowering can do. The language has no way to do it, and
should not.
The reference tier runs it in one go, with the whole range: the interpreter is what every test is compared against, and the sequential answer is the one that has to match. Which also says what a body may not do -- write to what it captured, because a capture is a copy and here there is one per thread. A result comes back through storage the body was given, by index.
D148. Names, through the system's resolver. std.net.Connect took a
dotted address, which means a client that could not reach anything a
person would type. Resolve asks the system -- getaddrinfo, so
/etc/hosts, the resolver's configuration and mDNS all answer the way
they do for every other program on the machine. A language with its own
DNS would answer differently from the machine it runs on.
An address is 64 bits wide here, because 128.0.0.0 already does not fit
a signed 32-bit one -- which is what made resolving a real host overflow.
D149. The rest of a filesystem. Exists, KindOf, SizeOf,
ChangedAt, Remove, Rename, MakeDirectory and List: a tool needs
all of them, and a program should not have to declare stat itself. A
listing comes back as names separated by line breaks rather than as a
list -- the caller splits it with std.text.Pieces, which is what it
would do with a list anyway, and this way the whole listing is one call
and one allocation.
D150. TLS is bound, not implemented. lib/tls is OpenSSL through
extern "C", with [link] libraries = "ssl, crypto" in its manifest: a
language that shipped its own cryptography would be shipping its own
CVEs. The certificate is verified against the system's roots and the
name is checked against the certificate -- TLS that checks only the first
is TLS any server can answer.
Two things came out of writing it. SSL_set_tlsext_host_name is a
macro in OpenSSL, so a binding reaches what it expands to --
SSL_ctrl with control code 55 -- which is exactly why wolang bind
produces a starting point rather than a binding (D130). And a bare
library name goes through the loader's own search, which on macOS finds a
shim for libcrypto and warns that it is being loaded unsafely: a
manifest that says where the library is means it, so the named paths are
tried first.
D151. The client does what a client has to do. A built Call --
method, headers, body, query, basic or bearer authentication, timeout,
redirect limit, certificate verification -- and Perform runs it:
https over the system's TLS, redirects followed (a 303 becomes a
GET with no body, which is what the status means), chunked answers
put back together, and a timeout so that a server which never answers
does not become a program that never returns.
The plain and the TLS read loops are the same loop twice rather than one loop over an ability: a socket and a stream are two types with the same three methods, and an ability with a method per direction is more machinery than the duplication costs.
D152. Nothing goes into a block nothing reaches. An if whose every
arm returns or breaks has a join block nothing jumps to -- and lowering
put the enclosing structure's jump in it, which the verifier rightly
called a block unreachable from the entry. The builder can now be
poisoned: after such a join, everything appended is dropped until the
next block begins. An empty unreachable block is something both the
verifier and the passes already ignore.
D153. Another program, with pipes. std.process.Run hands a string
to the shell and answers an exit code: enough for a build step, wrong for
a tool. Start and StartWith are fork and exec with three pipes --
the output, the error stream and the code, and the input written to it --
and no shell, because a command built out of a string is where the oldest
hole in computing lives. The command is split on blanks with quoted runs
kept whole, which is how a path with a space in it survives without a
shell being involved.
Two lowering bugs came out of writing the binding. What goes into a place
is of the place's type: var p: *u8 = &xs[0] makes a pointer that may
be written where the place holds one that may not, and a slot whose
stores disagree with its own type is one the passes promote into a block
parameter that disagrees too. Both the declaration and the assignment now
coerce, which is the same rule as D127 one level down.
D154. Arguments past the registers. Eight integer registers carry arguments on arm64 and six on x86-64. Past those, the dev tier indexed its table of argument registers without checking -- so a call with more arguments read whatever followed the table in memory and passed nonsense, in silence, while the tier's own refusal list claimed such a function was skipped. Two bugs that hid each other: the refusal made the garbage rare, and the garbage made the refusal look prudent.
Both now do what the ABI says. A frame reserves an outgoing area at the stack pointer, sized by the hungriest call in the function, and arguments past the registers are stored there; a parameter past the registers is read from above the frame, where the caller left it. The refusal is gone, because there is nothing left to refuse.
The interpreter's way out of the language had the same bound and reported
it as "symbol was not found", which sends a person looking in the wrong
place. It builds one thunk per number of stack arguments -- so the copies
are a fixed sequence of immediate-offset stores rather than arithmetic on
the stack pointer -- and says kTooManyArguments when there are more
than it will build for.
This is what glTexImage2D needs (nine arguments), and what every
windowing and graphics binding needs after it.
D155. A window of our own. platform/ is this project's backend for
each system -- Cocoa, Win32, X11 -- behind one C API: a window, what a
person does to it, a buffer of pixels, a GL context and a queue of sound.
Not a binding to someone else's windowing library: that would be a
dependency to ship, a second layer to debug, and a thing to install
before a program that draws a rectangle can run.
It is its own shared library rather than part of the runtime, so a
program that draws nothing links nothing -- a compiler that pulled in
Cocoa to print a number would be paying for a window it never opens. The
gfx package reaches it through [link], the same mechanism tls uses
for OpenSSL (D150), and a relative path in a manifest is now relative to
the package rather than to wherever the command was run.
Two ways to put something on the screen, one window. Present hands over
pixels -- 0xAARRGGBB, rows top to bottom -- which is what a tool, a
plotter or a software renderer wants, and what the drawing in gfx fills
in. A GL context is the other: StartGl, draw with OpenGL, Swap. The
GPU is reached by calling it, not by an abstraction over it.
Sound is a queue rather than a callback. A callback would be the system's audio thread calling into a tier that may still be compiling; a program that makes sound has a frame loop to push from, and a ring buffer with one writer and one reader needs no lock.
What a backend nobody has run is worth is nothing, which is why
examples/window exists: it opens a window, draws a moving wave at a
hundred and fifty frames a second, reads the keyboard, plays a tone and
says so in its title.
D156. ptrnull. The language could turn an address into a number and
not the other way around, so a binding had no way to write NULL -- half
of C's functions take one and the other half give one back. lib/tls
worked around it by freeing a one-byte buffer and keeping the address: a
pointer that is neither null nor valid, as a sentinel.
ptrnull is the address no object has. It takes whatever pointer type
the context wants, like null does for optionals, and *mut u8 when
there is none -- the pointer every other converts from without breaking a
promise. A number as an address (0x1000 as *mut u8) is allowed too, in
unsafe, for a binding that was handed one.
D157. The machines, checked. Linux on arm64 joined the matrix: same architecture as the laptop, different system, and the one combination that had never been built. A language that says it runs somewhere and never builds there is making a claim about a machine nobody has.
D158. Sockets through the runtime. std.net and http each bound
the system's own socket, send and close. That works on every system
that copied Berkeley's interface, and Windows is the one that did not
quite: a socket there is a handle rather than a descriptor, closing one
is closesocket, the library has to be started before the first call,
and send takes an int where everything else takes a size_t.
A binding cannot be written twice in a language with no conditional
compilation, so the difference lives in C now -- wolang_socket_open,
connect, listen, accept, send, receive, close, and the three
options a server sets. Both Wolang packages are the same everywhere, the
sockaddr_in nobody could name is built where the headers are, and the
server still answers fifty-three thousand requests a second, which is
what says the layer costs nothing.
D159. Windows. The rest of what a system is: VirtualAlloc for the
pages a tier compiles into, WSAPoll for readiness, CreateProcess and
three pipes for another program, LoadLibrary and GetProcAddress for a
binding, _open in binary mode so a file round trips, and the standard
library for threads, locks and the filesystem -- pthread_create and
CreateThread differ in every detail and in none that matters here, and
a second implementation of a lock is a second place for a race to hide.
Two things that look like details and are not. Writable and executable
memory at once is what every exploit of the last thirty years wanted, so
pages are written, then made executable, on every system. And a command
is split into words by the same code everywhere and joined back together
with this system's quoting for CreateProcess -- so that what a caller
wrote means the same thing on both.
CI builds and runs it, because a port nobody runs is a port that does not work.
D160. Code for another machine. --target <triple> on wolang build, and every target this LLVM has rather than this machine's only:
a compiler that can only emit code for the machine it runs on cannot
build the Windows binary from the laptop. The driver grew from
seventy-six megabytes to a hundred and sixty-seven, which is what every
target costs and what every other toolchain pays.
What comes out is an object file, and the message says so: linking it
needs that machine's runtime and that machine's linker, neither of which
this build has. A triple that is this machine links the usual way, so
--target is not a different command for the same thing. Verified by
compiling the same program for Windows, Linux on two architectures,
WebAssembly and RISC-V, and reading what the object files say they are.
D161. Print in a file compiled on its own. A single file links no
standard library, so Print there is a declaration with no body: the
interpreter answered it with an intrinsic, the compiled tiers could not
answer it at all, and a binary built from such a file trapped the moment
it printed. One runtime symbol, pointed at by the one declaration that
has no definition, and all three tiers do the same thing.
D162. One document, four formats. JSON, TOML, INI and YAML are four
spellings of the same six things: a nothing, a truth, a number, some
text, a list and a table. std.doc is that value, and every format
reads into it -- so a program that reads configuration does not care
which file it came from, and Get(value, "server.ports.0") is learned
once. Written as four parsers over one tree rather than four trees.
D163. A nested generic is not two arguments. List<List<T>> ends on
one token that closes two lists, and the comma after it belongs to
whatever the type is in. The argument loop ate that comma as a
separator, so fun f(a: List<List<T>>, b: int) parsed as a list with two
arguments and a function with one parameter. The close is checked before
the comma now.
D164. An owned string freed where it was made. A string built by
interpolation is owned and freed at the end of the scope that made it;
one stored in a structure that outlives that scope is a dangling
pointer. std.csv built column names that way and read them back as
rubbish. A builder's Release is what hands a string to something that
will outlive the call -- which is the wart D127 left, written down here
because it bit.
D165. Tags, where a document has no room for them. YAML's standard
tags decide how a scalar is read -- !!str 3 is text and !!int "7" is
seven -- and an unknown tag leaves the value alone. A document value here
has no room to record which tag it carried, and a reader that refused a
file over a tag it did not know would be a reader nobody could use.
D166. A matcher that answers. std.regex is a backtracking matcher
over a compiled program: literals, classes, anchors, groups,
alternation, the repetitions and their reluctant forms, and two flags.
No look-around and no back-references, because each turns a matcher into
a different machine.
Backtracking has exponential cases -- (a+)+b on a line of as is the
famous one -- so the walk has a budget. Past it the answer is no rather
than never, which is the honest thing for a library to do to a program
that did not choose its own pattern.
D167. Indexing through a reference. A field and a method already
reach through one; an index did not, so a &mut [40]int parameter was
an array the callee could not read. Through a reference now, in the
checker and in lowering -- the place is rooted at the address the
reference holds rather than at where the reference itself lives.
D168. Arrays in memory, in the reference tier. Two bugs, both of them
the same shape: an array has elements at a stride and a struct has fields
at offsets, and the interpreter used the second rule for both. A [4]int
field written to the heap was skipped entirely -- zero in the reference
tier, nine in machine code -- and rows[1][2] on a [2][3]int added the
indices instead of stepping into the row, which walked off the end.
Found by writing a regular expression engine, which keeps a fixed array of group positions in a struct and copies it through a list. Which is the rule this project keeps relearning: a real program finds what a test for a feature does not.
D169. A type in brackets is still a type. ArrayNew<[4]int>(1) was
read as a comparison with an array literal, because the heuristic that
tells a generic call from a < did not allow brackets between the angle
brackets. Nor a function type, so Keep<fun(int) -> bool>(f) was the
same. Both are allowed now.
D170. Randomness that is not a sequence, and hashes that are ours.
std.random is a generator: fast, repeatable, and exactly wrong for a
token. std.crypto takes its bytes from the system -- getentropy, or
the cryptographic provider on Windows -- because a language that rolled
its own would be rolling its own mistakes.
SHA-256 and SHA-512 are written in Wolang rather than bound: two hundred
lines of arithmetic with published vectors, no keys to mishandle, and a
language that cannot hash a salt without a C library has a hole in it.
Anything with a cipher or a key exchange in it is still lib/tls, which
binds the system's own (D150).
The arithmetic wraps where SHA says it wraps -- written as a wider add and a mask, because every operator in this language traps on overflow and here the wrapping is the algorithm.
D171. A byte is one byte wide. The interpreter wrote an integer
through a pointer using the width of the value it happened to be
carrying, not the width of the type being written. A u8 went out as
eight bytes and took the seven after it with it -- which is how a digest
in a heap buffer came back with its first half belonging to whatever was
written next to it. Found by the sanitizer, in one line, after an hour of
reasoning about allocators.
D172. Strings have an order. a < b on two strings compared the
pairs -- a pointer and a length -- so the answer was about where the
strings happened to live. Now it is a call into the runtime that compares
bytes, which for UTF-8 is also the order of the code points. The three
tiers agreed on the nonsense before and agree on the answer now.
D173. Both sides of an operator are one type. 48 + byte built an
add whose operands were a literal's type and a byte's. Nothing complained
until two such returns met at an inlined function's continuation, where
one was wider than the other.
Three fixes, smallest first: the side that is not a bare literal decides the type, both operands are cast to it, and the inliner casts a returned value that does not match the call's type. The second is the one that mattered -- an operator whose operands disagree has no meaning in this IR, and the verifier said so as soon as anything looked.
D174. The logo is a script. assets/make_art.py draws the mark, the
wordmark and the mascot, and writes both the SVG a page embeds and the
PNGs an extension ships. One source for both, so they cannot drift; a
scanline rasteriser and a PNG writer in a hundred lines, so the build
needs no drawing library. The mark is a wolf's head whose ears are the
outer peaks of a W, which is the two things the name is.
D175. A .vsix without npm. A Visual Studio Code package is a zip
with two manifests in it, so tools/package/make_vsix.py is a zip with
two manifests in it. What it packages is an extension written in plain
JavaScript with no dependencies -- a framed JSON-RPC client and the
translation between what LSP says and what the editor's API wants, which
is two hundred lines.
What that buys: the extension builds and installs on a machine with no
npm, no network and no login. What it costs: vscode-languageclient
would have done the two hundred lines. The trade is worth it for a
language whose whole point is that the compiler answers every question --
an extension that cannot be built is one nobody can try.
D176. Zones come from the system. A time zone is not arithmetic: it
is a database of what each government decided, revised several times a
year, and every machine already has a copy that everything else on it
agrees with. ZoneOffset, ZoneName, CivilIn and EpochIn ask that
copy. EpochIn asks twice -- the offset depends on the moment and the
moment is what is being worked out -- which is what gets the hour after a
clock change right.
ParseIso reads the one format worth building in, because it is the one
every machine writes.
D177. Deflate, written here. std.compress is a real compressor --
a hash chain of three-byte matches, fixed Huffman codes -- and a
decompressor that reads stored, fixed and dynamic blocks. Bound to zlib
it would have been shorter; written here it is arithmetic over bytes
with a published format, which is the same argument as the hashes
(D170).
The test that matters is not the round trip: it is that the system's
gzip -t says the output is valid and gzip -dc prints the input back,
and that what the system's gzip wrote reads here.
std.mem.Suffix came out of it: Prefix had no other half, so a reader
walking a buffer had to copy what was left. Both are intrinsics, because
only the tiers know how far along an element is.
D178. []u8! is a slice that may fail. The element of a slice is
parsed without suffixes now, so ! and ? belong to the whole type --
which is what a function signature means by it every time. A slice of
optionals is [](T?), and a type in brackets is that type: (T) is T
and (T,) is a tuple of one.
D179. The rest of a filesystem, and a child's world. Where the
program is, where the person's things are, where something temporary
belongs, what a path really is with . and .. worked out, copying,
appending, walking a tree, and taking one away. And for a child process:
somewhere to run and something to add to its environment -- added rather
than replacing, because a child with no PATH cannot find anything.
D180. A file kept open. std.fs reads and writes whole files, which
is the right shape for a configuration file and the wrong one for a
store: a store appends records and reads them back at the offset it
remembers. So the runtime has the handful of calls that needs -- open,
read at, append, size, sync, truncate, close -- and fsync is in there
because it is the whole difference between a store and a cache.
D181. A store is a log and an index. lib/db appends every write --
key, value, a checksum -- and keeps an index in memory saying where the
newest record for each key is. Opening reads the log once; a record whose
checksum does not match, or that the file ends in the middle of, is where
the log stops, and the file is truncated there. Which is exactly what a
machine that lost power during an append leaves behind.
Not a B-tree: a tree is the right answer when the index does not fit in memory, and it is a month of work to get page splitting, the free list and crash recovery right. A log is a day's work and survives a crash by construction. The index is sorted, so ordered walks and ranges work -- the one thing a log-structured store usually gives up.
Verified the way the claim deserves: fifty keys written, a torn record appended by hand, and the store reopened with all fifty intact and the file trimmed back.
D182. A value returned from a fallible. return 7 from a -> int!
returned a bare seven where a pair was expected: the checker records the
literal as int! -- it adapted it -- and lowering built a const.int
typed int! and then saw nothing left to wrap. The interpreter read
the seven as a struct and found no field; the compiled tiers read the
match as unreachable.
A literal is built at the element type now, and the wrap is the Coerce
that was always there. The lesson is the one this project keeps
relearning: what a value is comes from the instruction that produced
it, and what the context wanted is a separate question (D111, D173).
D183. The manual's programs compile. A manual full of programs that
do not build teaches the wrong thing and nobody finds out.
tools/package/check_docs.py pulls every fenced wolang block that
claims to be a program -- one with a fun main in it -- writes it into a
package and runs wolang check over it. CI does this on every commit.
It found nine broken examples the first time it ran, in a file written an hour earlier, by someone who had the library open in another window.
D184. This project is built with clang. Not a preference: the
compiler uses clang's overflow builtins for the checked arithmetic every
Wolang operator compiles to, __int128 for the wide multiply a constant
divisor becomes, and statement expressions in the trap machinery. The
configure step says so and stops, rather than letting a build fail two
hundred errors later.
On Windows that means clang-cl -- clang's compiler, Microsoft's flags and
headers -- which is what the LLVM installer provides and what the
windows preset selects.
Trying it with Microsoft's own compiler turned up eight portability bugs
that were real under clang-cl too, and they are fixed: setenv,
clock_gettime, popen, _longjmp and sys/time.h behind one name
each; std::thread and <mmreg.h> missing from the Windows backend; a
cursor handle that needed a wide pointer; and file_clock::to_sys, which
one standard library has and the other does not -- so the difference
between the two epochs is measured once at run time rather than named.
D185. Installing is one line. install.sh and install.ps1 work out
which build the machine wants, download it, check it against the
published SHA256SUMS, unpack it under ~/.wolang or
%LOCALAPPDATA%\Wolang, and put that on the PATH. Nothing is written
outside the prefix and nothing asks for a password -- a language that
needs sudo to try is a language fewer people try.
POSIX shell rather than bash, because this runs on a machine where
nothing is installed yet. --from-source is there for a platform with no
build published, and it insists on clang like everything else (D184).
The release now carries every platform -- Linux on two architectures, macOS on two, Windows -- plus the checksums the installers read and the editor extension.
D186. The whole compiler as one file. make_amalgamation.py inlines
every header and every source into one .cc that builds with a plain
clang++ -std=c++20 -c, beside the wolang.h a host includes. For a
host that wants to embed the language and not a build system, that is the
whole delivery: two files.
Three things had to be true for it to work. An .inc is an X-macro table
and is included several times on purpose, so it is inlined every time
rather than once -- the first version produced an empty opcode table and
a static assertion that fired a long way from the cause. A header's
include guard has to come off, or every include inside it looks
conditional. And half a dozen file-local names were too generic to share
one translation unit: four Entry structs, two States, two Checkers,
five kNames. They now say which file they are in, which is better
anyway.
The emitters needed more than a rename: both kept their tables in an
anonymous namespace beside each other, so they now sit inside
arm64/x64 instead. Two anonymous namespaces in different enclosing
namespaces are different namespaces.
CI builds the amalgamation and runs a host against it on every commit.
D187. One name, one file. On Windows a shared library brings an
import library with it, and ours was called wolang_embed.lib -- the
same name the static library has, which is two build rules writing one
file and a build that stops. The import library has its own name now.
D188. A real installer on Windows. A .msi, built by WiX from
tools/package/wolang.wxs: it installs under Program Files, adds its
bin to the system PATH as a component -- so removing the product
removes the entry -- registers itself in Add/Remove Programs with an
icon, and upgrades in place rather than sitting beside the old version.
The payload is whatever cmake --install lays out, harvested at build
time, so a file added to the project does not need a line in an
installer as well.
The icon is the same mark as everything else, written as an .ico by
the script that draws the rest (D174): a header and a PNG per size,
which is what the format has allowed since Vista.
The PowerShell script stays for people who want a copy in their home directory and no administrator.
D189. Every file says what it needs. include/wolang/base/unicode.h
used std::string and included <string> nowhere: libc++ drags it in
through something else, Microsoft's library does not, and the build fell
over on Windows with an error about a type that has been there all along.
There were eighty-one of those.
tools/package/check_includes.py asks the question on every commit --
which standard names a file uses, and whether it includes the header each
one comes from. It is a heuristic over the names this project actually
uses, not a model of C++; what it buys is that the next one is found by
CI rather than by somebody on another operating system.
D190. A type comes with the value. FindAll hands back a
List<Match>, and the caller never mentions List. Calling .Len() on
it produced an error type -- silently, with no diagnostic -- and the
program trapped at run time with nothing explaining why.
The cause: member tables were merged into a file's resolution from the
modules it imports directly, and List belongs to std.list, which the
caller never imported. So the caller had a type it knew the name of and
no members.
The merge walks the import graph transitively now. That also means a file is invalidated when a transitively imported file changes -- which is correct, because its types' members did change. The cost of the old behaviour was a class of silent miscompilation in every program that passes a container across a package boundary, which is every program.
Found by writing the cookbook: the example that searched a file with a regular expression did not work, and the manual is where a library's shape gets used the way a stranger would use it (D183).
D191. The first fix. Fix and FixEdit had been in the diagnostic
since the beginning and nothing produced one. W0101 -- a name nothing
uses -- now carries the change that silences it: an insertion of _ at
the name, which is exactly what the message already told a person to do.
Two ways to apply it. The language server answers
textDocument/codeAction with the fixes of whatever diagnostic the
cursor is on, translated and nothing else -- the compiler decided the
change, the server does not have a second opinion. And wolang fix
applies them without an editor, over a whole tree, printing what it would
do unless --write is given.
Edits are applied from the end of the file backwards, so that an earlier offset is still where it was.
D192. A constant divisor is a shift. The dev tier emitted a hardware
division whatever the divisor was. For a constant power of two -- / 8,
% 16, / -4 -- it is now a shift, with the bias that makes a signed
shift round towards zero the way division does, and the zero check goes
away because a constant divisor is not zero.
Worth doing in this tier rather than leaving it to LLVM, because this is
the tier run and test use, and % 10-shaped code is in every number
printer.
The general case -- a multiply by a magic constant for a divisor that is
not a power of two -- is still a division here. It needs a "high half of
the product" opcode in the IR, and the value of adding one is smaller
than it looks: a non-power-of-two constant divisor in a hot loop is rare,
and wolang build goes through LLVM, which does it.
Both back ends have the same arithmetic in their own instructions, and the reference tier still divides -- so every case is compared against a division by the differential fuzzer and by the test that walks eighty values through six divisors.
D193. A class is counted, not leaked. class allocated and never
gave anything back unless the class had a deinit. Now every object
carries a count of how many places hold it, and the last one to let go
runs the destructor and frees it.
The count lives in the word in front of the object rather than inside it: no layout changes, so a field is where it always was and the three tiers go on agreeing about offsets without being told anything. The compiler only ever sees the address it was given.
Where the counting happens: new answers with one, a store into a place
that outlives the expression takes a reference, a value on its way out of
a function keeps one, and a binding releases what it holds when its scope
ends. A value that arrived already counted -- from new or from a call --
is handed on rather than counted again, which is what stopped the first
attempt from ever reaching zero.
What is not counted: an argument. A function borrows what it is handed, because the caller is holding it for the duration of the call, and counting there would be two atomic-shaped operations per call for nothing. A callee that keeps what it was given does so with a store, and the store is what takes the reference.
Two things still leak, and both are written down rather than hidden: a cycle, which is what reference counting cannot see; and a handle overwritten in a field rather than dropped, because releasing the old value would mean reading the place back first. Leaking there is the right way round -- the other mistake frees something that is still in use.
Owns and Releases are deliberately different questions. Owns is
"does handing this on give it away", which is what makes a World passed
to the functions that work on it a borrow rather than a move (D87).
Releases is "does this binding hold a reference". Conflating them made
every class parameter a move and quietly freed a world halfway through
the function that was using it.
D194. The trampoline takes stack arguments too. The body has read arguments past the registers off the stack since D154. The trampoline a host call enters had not been told: it wrote past the end of its register table, which the sanitizer called a global buffer overflow and a release build would have called a wrong answer. The x86-64 side had an honest refusal in front of it instead -- "takes more arguments than registers", which left every such function to the interpreter.
D195. A byte operation needs REX. Without the prefix, registers four
to seven in an x86-64 byte operation name ah, ch, dh and bh --
the high byte of another register. The assembler forced it for a store
and for setcc and not for the dozen other instructions that have a byte
form. Forced in the one helper now, because a dozen call sites is a dozen
chances to forget.
D196. Scratch zero is the accumulator. Three places in the x86-64 emitter asked for scratch zero and then wrote the accumulator, which is the same register: the address of an array element became twice the base of the array, and wrapping a value in an optional stored the tag to the address the tag had just overwritten. A program that put either in a place the allocator had to spill crashed, and the crash was in generated code with no source line on it.
Found by running the x86-64 build under Rosetta on an Apple Silicon
machine and then dumping what it emitted: wolang asm --raw prints the
bytes, and a disassembler says the rest. That is now the way this tier is
checked without a second machine -- the whole suite ran, and 94 tests in
it crashed.
D197. What the x86-64 tier had quietly stopped doing. Two more from the same session: a virtual call read the token out of the object a second time, as if the token were the handle, and called the lookup with whatever the first field happened to hold -- lowering has read it already, and the other back end passes it straight through. And a call with more arguments than registers refused the whole function, from before the outgoing area worked.
D198. A narrow answer is only as wide as its type. A function that
answers a bool or an s32 may leave anything it likes in the rest of
the register, and both back ends took the whole sixty-four bits. So
wolang_world_alive answered false and the program saw true, because the
high bits were someone else's arithmetic. The result is now narrowed to
its type where it is taken, which is one instruction and the difference
between right and nearly right.
D199. A temporary may not be where the operand is. The shift-instead-
of-divide sequence (D192) was written twice, and the x86-64 copy used the
low scratch registers for its own temporaries -- which is where the
operands of the instruction that called it live. So % 64 read a value
that the sequence itself had just written over, and std.utf8 encoded a
code point into bytes that did not fit a byte.
The lesson is about scratch registers rather than about division: a helper that is handed an operand register must copy it out before it writes any scratch it has not been told is free.
D200. The swap slots on x86-64 were a word wide. Values that trade places around a loop go through a slot each, and a slice needs a slot wider than a word -- in eight bytes it keeps its address and loses its length. The arm64 emitter was told this (D107); this one was not, and wrote a sixteen-byte slice into an eight-byte slot, over the slot beside it. A loop carrying a slice and a counter came back round with the counter in the slice's length.
Five of the standard library's tests failed on that one bug -- base32,
SHA-256 over long input, signing, a socket talking to itself, and the
parallel for. All five are loops that carry a slice.
The rule this leaves behind: a bug fixed in one emitter is a bug to look for in the other, and the comment that explains the fix should be read as a question about the other machine.
D201. Windows x86-64 has its own calling convention. The x86-64
tier spoke System V everywhere, so on Windows every compiled function
read its arguments out of rdi and rsi while the host had put them in
rcx and rdx: Add(2, 3) answered 0, and every test that reached
machine code failed or crashed. The port was never run -- CI lived in a
GitHub workflow and the repository was on GitLab. It has since moved to
GitHub.
One convention per machine, not one inside and another at the edges: the
runtime calls compiled code directly (a parallel for, a system of the
ECS), so every body has to be callable from C. On Windows that means four
argument positions shared by integers and floats, thirty-two bytes of
shadow space under every call, rsi and rdi saved by the callee (the
tier uses them as scratch, so every body saves them), floating-point
scratch only in xmm0-xmm5 (the rest belong to the caller), and a
probe of every page when a frame is bigger than one. Where an argument
goes is answered by one cursor shared by the prologue, the call, the
entry stub and the frame layout, so they cannot disagree again. The
interpreter's thunk into C got the same treatment.
D202. A trap on Windows leaves without unwinding. longjmp there is
an unwind: it walks every frame between the trap and the landing pad
through the exception tables, and generated code has none, so a trap was
STATUS_BAD_FUNCTION_TABLE instead of a trap. Nothing between the two
holds a destructor (D59), so the jump buffer's frame is cleared after
setjmp and the jump becomes a plain register restore -- the same thing
_longjmp is elsewhere.
D203. The standard library asks the runtime, not POSIX. std.io,
std.fs, std.process and std.time declared write, read, close,
getpid, usleep and clock_gettime themselves, which is a library that
works on two systems out of three. They call wolang_fd_*,
wolang_process_id, wolang_sleep_micros and wolang_clock now, and
each system answers in the runtime. It also fixed a Linux bug nobody had
seen: clock 6 is CLOCK_MONOTONIC on Darwin and CLOCK_MONOTONIC_COARSE
on Linux, so Now() there ticked in milliseconds. std.env.System()
says which system it is, and the tests that start programs ask it rather
than assuming sh, cat and /tmp.
D204. Time zones on Windows come from the system's ICU. The C library
there understands EST5EDT and nothing with a slash in it, so
Europe/Kyiv was UTC. Windows has shipped ICU as icu.dll since 2019;
it is loaded when asked for, and the C library is the fallback when it is
missing. Still the system's database rather than one of ours (D176).
D205. What a CI that never ran was hiding. fmt --check lib failed
on every system, because a directory that is not a package was read as a
file; forty-odd files had drifted from the formatter since. wolang.h,
the header a C host includes, included <cstdint>, which C does not
have. The documentation check wrote a Windows path into TOML, where its
backslashes are escapes. The driver looked for libwolang_platform.so
on Windows. Each is fixed, and the Windows job now runs everything the
Linux one does except the ahead-of-time tier, whose linking is not wired
there yet.
D206. A counted class lets go of what it holds. D193 counted the
handle and stopped there, and a counter that measures live objects
(wolang_objects_live) showed how far that was from enough: an object
that died kept every object its fields pointed at, a field written twice
kept the first value, a var assigned twice kept the first object, a
Leaf? was not counted at all, and handing a local to any function
leaked it -- the use cleared the local's flag as if it were a move, and a
class with no deinit is not one.
The rules now, in every tier, because all of it is lowering:
- A class with a
deinitmoves (D87); one without is counted, and handing it on is another holder. Only what moves has a flag a use can clear. - Every class has a
$dropfunction: the destructor, a release of every field that holds a class or aT?of one, the storage. A function and not inline code, becauseclass Node { var next: Node? }would expand forever. - A store into an object's field, a module-level
varor a local slot takes a reference to what goes in and lets go of what was there -- after the store, so thata.x = a.xdoes not free what it is keeping. - A constructor and a binding take a reference to what they are given; a call borrows, so a reference made only to be passed is released when the call returns, and one made by an expression statement is released on the spot.
T?of a class is counted through its tag.
Cycles still leak. weak is what breaks them, and it is next.
D207. weak var breaks a cycle. A parent that holds its child and a
child that holds its parent is the shape of every scene graph, and
counting alone never frees it. A field marked weak holds a C? of a
class without keeping it alive: reading it asks the runtime whether the
object is still there and answers a counted reference or null.
The second word in front of every object -- sixteen bytes were reserved
for one count (D193) -- is the weak count, plus one for all the strong
references together, which is the arrangement Rust's Arc uses. The
object goes when the strong count reaches zero; its storage goes when
the weak count does, so a weak field never points at freed memory. The
upgrade is a compare-and-swap that will not raise a count from zero.
weak is contextual, like derived, and only allowed on a C? field
of a class (E0137): a weak local would be null by the next line, and
a weak field of a value would be copied without being counted.
Found on the way: p?.x on a class read the field out of the handle as
if the handle were the object, which is a wrong answer in machine code
and a trap in the reference tier. It reads through the address now.
D208. Ahead of time on Windows, through LLVM-C.dll. LLVM's Windows
installer ships the DLL that is all of LLVM's C API and nothing to
compile against it. tools/package/fetch_llvm.py takes the headers, the
import library and that DLL out of the official release archive into
build/llvm-sdk, CMake finds them there, and the DLL is copied beside
the programs that load it. One DLL rather than LLVM's static libraries:
no C runtime to agree with, every target included, and the C API is
the only part this project uses (D98).
wolang build links with clang-cl and the same C runtime the runtime
archive was built against; LinkCommand is the one place that says how,
for the driver, the fuzzer and the tests alike.
Pinned to LLVM 22.1.8, not the 23 the compiler itself is built with.
The differential fuzzer found it in the first sixty programs: LLVM 23's
interprocedural constant propagation folds x != 6 % x to false when
x comes from a call and is known not to be zero. The IR before the
pass is correct and LLVM 18 and 22 agree with the interpreter; 23.1.3
and 23.1.4 do not. WOLANG_AOT_IR=<file> writes the module before and
after the optimizer, and WOLANG_LLVM_ARGS hands LLVM its own flags --
-opt-bisect-limit is what named the pass.
D209. A class inside a value is counted too, and classes do not move.
A struct, an enum and a T? or T! of either that holds a class is a
holder of it: a copy retains every class it carries, its end releases
them -- Pair$drop for a struct, a switch on the tag for an enum -- and
Free on a slice releases its elements first, because storage starts as
zeroes and zeroes are an empty reference of every counted type.
That only adds up if nothing moves a class. D87 moved a type with a
deinit; for a class that made handing a List out of an enum's payload
a move out of something borrowed, and the enum and the caller both freed
it. A class is counted now whether or not it has a deinit, and the
deinit runs when the last reference goes. Moves are left to values
with a deinit and to owned strings.
What it took to agree:
- A call's result is fresh -- it owns what it holds -- and so are
new, a struct or variant built here, and the answer of??, which takes the value of a fresh optional and retains that of a borrowed one. Fresh is per function: an instruction id means nothing in another one. - The subject of a
matchthat is fresh is held until the scope ends, so every payload bound from it stays valid in every arm. - A generic method lowered for
List<Field>asks aboutField, notT, and asks the file that declaredField, not the one that declaredList. - A
matcharm's answer is coerced to the type of the wholematch;LeafwhereLeaf?was expected was a verifier error nobody had hit.
WOLANG_TRACE_RC=1 prints every allocation, retain, release and free,
which is how the last of these were found. What still leaks: a text
inside a document value is a view of bytes nobody owns, which is a
question about std.doc, not about counting.
D210. The documentation lives on mowoengine.com. Beside the engine's
guides, in the same engine-docs repository the site pins:
tools/package/make_site_docs.py turns docs/ and the API reference
wolang doc writes into MDX under Wolang/Site/, and the site renders
them at /docs/wolang/ and /docs/wolang-api/. The guides stay Markdown
here, next to the code, and every example in them is still compiled on
each commit; the script only escapes what MDX would read as an
expression and turns links between files into links between pages.
.github/workflows/docs.yml publishes on every push to alpha, the way
the engine's docs workflow does.
wolang doc writes one package to one file. The CI step that asked it
for three directories and a folder of per-module files had never run.
D211. TLS lives in the runtime, and on Windows it is Schannel.
lib/tls declared OpenSSL's functions itself and linked ssl and
crypto, which is a package that works where OpenSSL is installed and
nowhere else -- not on Windows, and not in a binary built ahead of time
on a machine that has no OpenSSL headers. Four runtime calls replace it:
wolang_tls_open, _read, _write and _close, over a runtime socket.
Windows answers them with Schannel through SSPI, the TLS every Windows
program already trusts, with the system's roots and its own name check.
Elsewhere OpenSSL is loaded with dlopen the first time a stream opens,
so a program that never speaks TLS needs no OpenSSL at all, and the
package needs no [link] table. Still the system's TLS rather than ours
(D150); only where it is reached from moved.
Checked on both systems against a real server and against expired, self-signed and wrong-name certificates, which all refuse.
D212. An ability can leave a type to whoever implements it. type Item in an ability, type Item = int in extend T as Ability, and
C.Item in generic code. What it bought straight away: for over a C: Iterable, which is a function that walks any collection -- std.iter is
Iterator and Iterable, and List, Deque, Map, Split and Chars
implement them. D96's three names still work for a type that says nothing
more.
C.Walker.Item is a type of its own kind, a projection: an owner and a
name. Inside the generic body it stays one, so walker.Value() + 1 is an
error when nothing said Item is a number. Once the owner is concrete --
after a call infers C, and in every copy monomorphization makes -- it
is whatever the implementation answered, read off the member table of the
file that declared the type. Bare Item inside the ability is a
projection of the ability itself, which a use rewrites onto the type it
is used on.
extend<T> Box<T> did not work at all before this: its members were kept
under Box<T> in the extension's own T, which no Box<int> ever
equals, and lowering named its methods Box<<error>>. They belong to
Box now, rewritten into the declaration's parameters, and a method of
one is a generic recipe like a method declared inside the class.
type Count with nothing after it is E0138 outside an ability.
D213. Objects cross the embedding boundary, and the host counts its
own. An engine's actor is made and freed by the engine; a script
holding one has to say so without owning it. extern class Actor { fun Hurt(amount: s32) } is a handle like any class, with three differences:
retaining and releasing one calls the host through
wolang_host_retain/_release instead of the header Wolang's own objects
carry, a weak var of one asks the host's alive when it is read rather
than keeping a weak count, and a method is the host's Actor_Hurt, taking
the object first. The runtime finds the callbacks by a tag both sides
compute -- the FNV-1a hash of the class's name -- so the compiled tiers
pass a constant and need no table of their own. Construction and fields
are E0139: the data lives on the host's side, where the engine's own
code reads it.
The other direction is kWolangObject, a Wolang object the host holds a
reference to. Releasing the last one has to run the class's $drop,
which used to exist only when something in the program released one, so
every non-generic class a file declares gets one now. Methods and
$drops are found by name rather than by declaration, because a $drop
has none. Two counting gaps showed up on the way: held = null did not
release what was there, because the literal had no counted type, and an
operand made only to be compared -- tracker.seen != null upgrades a weak
field into a reference of its own -- was never let go.
D214. A vm takes one call at a time, from any thread. Every entry point that touches a vm holds its lock, a recursive one, so a host function that calls back into the vm it was called from does not deadlock itself. The alternative was to leave it to the host, which is what "not thread-safe" said before, and every engine would have written the same mutex. Parallelism is separate vms, which were already independent: the compiled tier's context belongs to its vm, the counts on objects are atomic, and the host's class table has its own lock.
That last part was not true when it was first written: module-level
vars lived in one table for the whole process, keyed by file and
declaration, so two vms holding the same program shared them. They live
in a global space now, one for the process and one per vm, and a call
into a vm enters its space for the thread making it. Threads a script
starts enter the space of the thread that started them.
D215. A reload keeps the program's state. wolang_vm_reload used to
throw everything away, which is a restart and not a reload. It builds the
new program in a new global space, then carries each module-level var
across by name when the type has the same shape -- the name, and for a
declared type every stored field by name and shape, all the way down. A
value is moved, not copied: the old program's globals are never read or
released again. A class that gained or lost fields keeps the fields that
are still there, written into the object the new initializer made. A
class with virtual methods is never carried, because its descriptor
belongs to the code that made it. Whatever starts again is listed in
wolang_vm_notes.
The old program is retired rather than freed -- its database, its code
and its globals stay until the vm is destroyed. A carried value can point
into it: a text literal is bytes in the old program, and freeing it
would leave the new program holding a dangling string. Function handles
name their function instead of pointing at it, so they survive.
D216. The debugger is the interpreter. docs/debugging.md used to
say a Wolang debug adapter would be a worse lldb. That was true for a
binary, and no help at all for a script inside an engine, which has no
binary of its own and runs in the JIT. Teaching three code generators to
stop at a line -- deoptimization, patchable breakpoints, register maps
for every local -- is the expensive way. The interpreter already runs
every program the compiled tiers do, and the differential fuzzer keeps
it running them the same way, so a program being debugged runs there.
Lowering records every local the source named: where its value is (the
slot of a var, the value of a const or a parameter) and the range of
instructions it is in scope for. The debugger runs the IR as lowering
left it, before any pass, so that table is exact. The interpreter asks
the debugger whenever it reaches a new place in the source, and the
session decides by line, by depth and by what the editor last asked for
-- a call that returns lands in the middle of its caller's line, which
is not a new line, so a breakpoint there does not fire twice.
wolang dap speaks the protocol over standard input and output for
launch. An embedding host calls wolang_vm_debug_listen for attach,
and while an editor is attached every call into that vm runs on the
interpreter; when it goes, the compiled tier is back. The session sits
behind a transport it does not know about, which is how one class serves
a pipe, a socket and a unit test.
D217. A capability is stated where the door is, and a mod's grant is the
host's. capabilities = "" looked like a sandbox and was not one.
unsafe is absorbed by a declaration that states it (the rule that makes
a safe wrapper possible), and std.fs.Open stated uses unsafe and
nothing else -- so a package granted nothing could open files through it,
and the compiler said nothing. Reviewing two hundred signatures by hand
would have fixed it until the next one.
The capability is stated on the extern declaration instead: the
primitive that touches the world says which part, wolang_fd_read uses io, wolang_socket_connect uses net, wolang_clock uses time,
wolang_entropy uses rand. unsafe is still absorbed and these are
not, so they propagate through every wrapper, and the compiler found the
two hundred signatures that had to say so (E0701, E0704). A new
std function cannot forget, because it would not compile.
The other half is who writes the grant. A mod's manifest is the mod's,
so wolang_vm_load_mod applies the host's grant to the package before it
is checked -- narrowed by the manifest when it lists capabilities, never
widened -- with unsafe denied and the [link] table dropped. And a
sandboxed package cannot declare extern (E0705): calling one would
need unsafe, but an extern class method has no body and no effects of
its own, so extern class wolang { fun system(...) } bound
wolang_system without either.
D218. Engine bindings are generated from the engine's reflection.
MowoEngine's header tool already writes everything MCLASS, MFUNCTION
and MPROPERTY declare as JSON, for the editor and the Visual Script
palette. wolang bind --reflection reads the same files and writes a
Wolang package and the C++ glue that answers it, so the bindings are as
complete as the reflection and nobody keeps two lists in step.
The C boundary takes numbers, bool and pointers, and the engine's API is
Vec3s and strings. A method that needs either gets a raw form -- out: *mut Vec3, name: *u8, nameLength: u64 -- and a method in an extend
block that turns script values into it. That needed one rule changed: a
method of an extern class used to be the host's whatever it was, and
now it is the host's only when it has no body. Properties are reached by
the offset the engine's own reflection records, asked for once per
property, because an MPROPERTY is usually private and the glue is not
a friend. A std::string answer is left out until the boundary has a way
to hand bytes back that the script then owns.
Checked against the engine as it is: 165 classes, 734 functions and 256 properties, a generated package that checks clean, and glue that compiles against the engine's headers with the engine's own flags.
need mowo.{World} for a uses World clause was reported as unused,
because only expressions counted as uses. An effect a uses clause names
counts now.
D219. A frame's calls are words, and a shipped script is a library.
The first measurement of a host ticking a thousand actors through a
script said 61× slower than C++, and the script was not the reason: each
call looked the function up by name, boxed its arguments into a vector,
found the compiled body by searching every function, and decoded the
answer through a layout. A WolangFunction now prepares itself once per
program -- the compiled entry and how each argument goes into a word --
and a call is the words on the stack and a jump into the stub, the same
way the scheduler resumes a task. A reload makes the preparation stale by
giving the program a new generation. That took the dev tier to 11×.
A shipped game does not run the dev tier. wolang build --library actors
compiles a package into an object a host links: no main, an
actors_start that registers classes and starts the module's vars, and
every offered function under actors_<Name>, an alias so the module's
own calls are untouched. Through that, the same Tick is 1.7× the C++ --
which calls virtual methods directly where the script goes through C
functions that call them.
D220. STATUS.md is in English. D7 kept the working log in Russian
because only its author read it. The repository is open now, and
STATUS.md is where a contributor learns what is done and what is
open -- README.md and CONTRIBUTING.md both send people there. A
working log nobody else can read is a closed door in an open project.
D221. A bodiless primitive does not absorb unsafe. A declared
uses unsafe absorbs the obligation because the author of a body takes
responsibility for it. A primitive with no body (Free, ArrayNew,
ViewFromRaw in std.mem) has nobody vouching, so its unsafe passes to
the caller, exactly as an extern call does. Without this a sandboxed mod
could reach raw memory through std.mem with no uses unsafe of its own.
D222. Naming a function as a value costs its effects. Binding
const w = WriteAll and calling w through a local escaped the effect
check, because only direct calls were charged. Any reference to a function
item that is not the callee of a call now charges its full row to the
enclosing function, as if called, so capabilities and @Deterministic or
comptime contexts see it. Registering a layer is therefore charged
where it is named, which is conservative.
A method is charged the same way, called or named (b.Out,
Type.Out); methods were not charged at all before.
D223. The HTTP server answers only requests it can frame. A request is
parsed from the bytes that arrived and no more -- the pool buffer is
reused, so what lies past received is another client's. Content-Length
must be digits (nine or fewer are read, more are "too large" and get 413),
a second one must agree, and Transfer-Encoding is refused with 501 (with
a Content-Length too, 400): the server does not implement chunked
requests, and ignoring the header lets a proxy and this server disagree on
where a request ends. Idle connections are closed by deadline, not held
forever.
D224. Every tier answers integer edge cases the way the interpreter
does. Constant folding leaves to run time anything that would trap there:
64-bit overflow, the most negative value over minus one, and negating it.
Folding them gave the wrapped answer in one tier and a trap in another,
and MIN / -1 executed inside the compiler faulted the host. Unsigned
integers interpolate through their own runtime call, because widening a
u64 past 2^63 to s64 changed its value. Float % is C's fmod in
every tier, called through the runtime, so there is one answer to give.
D225. Every depth is bounded where it is made. Sema, lowering, the
printer and the formatter walk the tree recursively, and a 3000-term
1+1+... or ----1 overflowed the native stack in each of them. The
parser's paren limit never saw it: left chains are built by loops. The
parser now tracks the height of the expression it is building -- one per
link of a chain, per prefix operator, per postfix, per else if -- and
reports E0041 past 1024 instead of producing the node, so every later
walker sees a bounded tree and none needs its own guard.
The same rule for types: a generic that calls itself with a bigger type
(F<T> calling F<[T]>) is a new copy each time, and lowering never
finished. An instantiation whose argument nests past 64 levels is E0140.
E0117 now covers what the member table could not see: an enum's payloads
and a generic's own instance (W<W<T>> inside W<T> is the declaration
again), and the layout's IsSupported is depth guarded for the case where
sema misses one.
An unsuffixed integer literal too wide for int is E0109 where the
default would be taken (D9: an error, not a silent wrap), and <literal> as s64 reads the literal at the cast's type. The interpreter also measures the
native stack, not only Limits::depth: a debug build's frames are large
enough that a runaway comptime recursion crashed the compiler before 1024
calls, and it now stops with the same depth trap.
D226. Tiers agree on integer-cast traps, trap text and float text. A
cast between s64 and u64 (or a signed value to u64) traps when the
value is negative as a signed number, in the JIT and AOT as in the
interpreter. Trap messages are one table in effect: the interpreter says
"index is outside the length" where the length is a runtime value, because
compiled code cannot print it, and the codegen test harness now compares
trap text, not only the kind. Floats print as the shortest digits that read
back as the same number (a float/f32 gets its own shortest f32 digits),
and every NaN prints nan whatever its sign bit, from one function used by
the runtime and the interpreter.
D227. A reader of someone else's bytes bounds what they can make it
do. A document is input, and input decides how deep the parser recurses
and how much it allocates. JSON stops at 128 levels of nesting, YAML and
TOML at 64, a regex at 32 nested groups, 1000 repeats and a program of
20000 steps; InflateLimited and GunzipLimited stop at the output size
the caller names. Each limit answers the module's own error (Malformed,
BadPattern) rather than trapping, because a trap in a server is the
whole server. The depths are low because the interpreter recurses on the
host's stack. YAML aliases share one handle, so a billion-laughs document
reads in linear time and needs no cap.
D228. Reference roots and arena escapes are found by a conservative
walk. E0121 no longer recognises only a bare &name: the root of a
reference is the local reached through fields and indices (stopping at a
slice, reference or pointer, whose referent lives elsewhere), a local
[N]T used where a slice is expected (a return, or an assignment to a
slice), and, for a call whose result is a reference or slice, the deepest
local among the roots of its arguments -- the result is assumed derived
from every one. Receivers are not walked. Values held in other locals are
not tracked. E0142 (arena escape) uses the same spirit: a value built
inside arena { } -- a class construction, or a call returning one, or a
name bound to such a value inside the block -- may not be stored in a
variable declared outside the block, a global, or a field of an outer
object, nor leave through return, break or the block's value, because
the arena is reset at the block's end and the object is uncounted (forged
pointers otherwise, even in sandboxed code). Using it inside the block is
fine.
D229. A deinit cannot keep self, and a move out of a match is a move.
A deinit that stored self somewhere that outlives it raised the count
of an object that was about to be freed: the storage went and the reference
stayed, a segfault in machine code and stale reads in the interpreter. The
rule is Swift's: when a destructor returns and the count is not zero the
object escaped, and that is a trap in every tier (deinit let self escape).
The check is wolang_object_escaped, called from the generated $drop
right after the deinit, so all tiers share it. Assigning self to a
field or global directly inside a deinit is E0145 before it gets that
far; a call that stores it is only caught by the trap.
Moves are tracked across match: a payload bound by an arm is the
scrutinee's own storage, so moving it out (handing it on by value) moves
the scrutinee, and matching it again is E0122. A function written as
=> expr left without the destroy a return runs, so a by-value
parameter that owns something (a deinit value) was never dropped; it
runs the deferred drops now.
Closed by D233: a captured deinit value, a payload no arm moves, and a
parameter moved twice.
D232. What crosses a thread. Spawn absorbed unsafe and ran any
fun(), so two threads pushing to one module-level List raced and
segfaulted, and a capturing closure handed to it had no environment to
run with. The rule: a function marked @Thread runs what it is given on
another OS thread (Spawn, ParallelFor; every std API that does so
carries the mark). The function or closure passed there must not read or
write a module-level var, directly or through anything it calls
(E0147), unless the caller declares uses unsafe, which takes
responsibility for the sharing, as it does for extern (D221). Locks and
atomics are reached through such a var handle, so code using them
states uses unsafe with a comment. A closure that captures is E0148
for plain @Thread (Spawn passes no environment), and no uses unsafe
lifts that; @Thread(captures) (ParallelFor) allows it because the
runtime copies the environment per thread (D147).
The check is a flag per body (names a mutable global) plus every call it makes, closed over the module in the same pass shape as the effect fixpoint (database.cc, effect diagnostics), not a new effect: an effect would have to be declared on every function in between. Limits: calls into other modules and method calls are not followed, and a function value held in a variable is not judged; the interpreter runs a spawned body inline (D97) so it never sees the race. Separately, the JIT driver now gives the whole tier up to the interpreter if an emit fails after the probe passed, instead of aborting the compiler.
D244. UDP in std.net. Game networking wants datagrams, so the
runtime gained wolang_udp_open/bind/send_to/receive_from/broadcast
beside the TCP calls (sockets.cc, POSIX and Winsock in one file, D158),
and std.net gained UdpSocket, BindUdp, BindUdpOn and Datagram.
Every primitive is uses net on its extern (D217). Closing, blocking
and the bound port reuse the TCP calls, and a UDP fd goes to the same
poller as a TCP one. Receive never writes past the caller's buffer: a
longer datagram is cut and Datagram.truncated is set (MSG_TRUNC via
recvmsg on POSIX, WSAEMSGSIZE on Windows). A non-blocking socket with
nothing to read answers null (runtime -2). Sends of zero bytes or more
than 65507 are refused (-1). The Windows path is written to mirror the
POSIX one but is not built or run here.
D243. What a machine and a task do not do yet is refused, not ignored.
on X after 3s parsed and then fired at once (D71 said so; the compiler
did not). It is now E0159, reported at the transition. The syntax stays
reserved for a machine that is given time by a generated Tick(dt). A
wait in enter was already refused by E0301, since a block is not a
task. Section 8 promised that a task is cancelled with the scope that
spawned it, and nothing cancels a task. A spawned task runs to its end.
Stopping one early is a flag it reads. The promise is struck from the spec.
D242. A dependency can say which versions it accepts.
std = { path = "..", version = "^1.2" }, or with a repository. The grammar
is Cargo's: 1.2 means ^1.2; ~, =, >=, >, <=, <, a comma for
"and", *. There is no resolution: the one copy found must satisfy every
requirement on it, or the build says which package asked and what it found
(E0053). A bare string, std = "1.2", is a version, which is what every
other manifest means by it. It is an error until there is a registry,
rather than a path as before. The inline-table reader no longer splits on a
comma inside quotes. Pre-release ordering and ranges across a registry are
deferred. The lock file records the version found for each fetched package.
D245. lib/image: PNG, QOI and BMP in Wolang. Pictures are read
and written by a package of their own, written here for the reason deflate
was (D177): published byte layouts over arithmetic, and the first code a
hostile file reaches. It depends on std only; gfx does not know it, and
Image.Get answers 0xAARRGGBB, the canvas format, so the bridge is a
loop. An Image owns RGBA8 bytes. Every size in a file is checked before
an allocation: at most 16384 on a side and 64 Mi pixels (TooLarge), the
PNG inflater is held to the size the header implies with InflateLimited
(D227), chunk CRCs, the zlib header and Adler-32 are verified, and bad or
truncated input is Malformed (or Unsupported for real features not
done: compressed or palette BMP, unknown critical PNG chunks), never a
trap. int is 32 bits, so 32-bit file fields are read with a clamp to
2^31-1 and checksums as u32. PNG decodes every colour type, depth 1 to 16
(16 is cut to 8) and Adam7; the encoder writes RGBA8 with the per-row
minimum-sum filter and the std deflate. QOI is complete; BMP reads 24 and
32 bit and writes 32 bit. PNG vectors in the tests come from an independent
encoder (Python zlib) cross-read with Pillow.
D233. A value that carries a deinit value owns it, and what D229 left open is closed.
A struct or enum with a field or a variant payload whose type has a
deinit is an owner itself: it moves instead of being copied (so a
second use is E0122), and its end runs a $drop that drops the carried
value -- the field, or the payload of the active variant -- after its own
deinit. Before, enum Wrap { Has(r: Res) } leaked r when no arm moved
it, and Outer(Res(1)) leaked as well. Limits, the same in the checker
(IsOwning) and in lowering (OwnsByDeinit) because a mismatch is a
double drop: named types only (not generic instances, not classes, which
are counted, D209), and not the owned string.
Matching a name reads it; an arm that hands its payload on is what moves
the name (its flag is cleared through the payload). Taking an owning field
out of the value that owns it (Take(o.r)) would drop it twice, so it is
E0149; read through the field (o.r.id) instead, or move all of o.
A closure borrows what it captures: the environment is a copy that cannot
outlive the frame (D138), the function around it still drops the value,
and a closure body that hands a captured deinit value on would drop it
twice, so that is E0149 too. Closure bodies are walked by the flow
check for this (their params shadow captures); nothing they do changes
the flow after the closure.
A by-value parameter is a flow local, so moving it twice is E0122.
std.text.Bytes returned a writable []u8 over text that may be a
literal: Bytes("Hello")[0] = 74 made every "Hello" in the program
"Jello", and in AOT a literal is in read-only memory. The language has no
read-only slice, and a copy would leak because nothing frees it, so the
least disruptive sound choice is to make the caller answer for it: an
@Exposes attribute on a function means its uses unsafe is not
absorbed by the body (the usual rule of D221), so calling Bytes is
unsafe in the caller. The library already declared uses unsafe where
it called Bytes. A real read-only slice type is the proper fix and is
left for later.
D246. module.Name reaches what a module offers. need std.json
used to bind the module's own name as a nameless type, so json.Parse(x)
typed as an error with no diagnostic and lowered to undef; a bare need
also dumped the module's whole surface in unqualified, where the first of
two Parses silently won. Now every need, bare or with a list, makes its
last path segment a qualifier: json.Parse, json.Value, json.Corrupt
(function, type, constant, error tag; json.Value.Num(1) is the type then
its variant). The resolver keeps these as FileResolution::qualified,
bindings spelled json.Parse that point at the real declaration, so
lowering and linking (by file and item) tell two Parses apart. A local,
a self member or a file-scope name called like the module wins; an
unknown or non-offer name is E0144. Unqualified access is unchanged,
but when two different modules offer one name and the file uses it bare the
use is E0143 (first still wins, so the rest checks). Not covered yet:
qualified names in patterns, qualified assignment to another module's
var, and ambiguity of bare type names in annotations. There is no
need x as y; the qualifier is the last segment.
D241. The embedding ABI says its version and keeps its shape.
wolang.h gains WOLANG_ABI_VERSION (major<<16 | minor) and
wolang_abi_version(), so a host checks at load what it was compiled
against. A WolangValue is a kind, a reserved word and one union of the
scalar or handle, beside a text and its length: 32 bytes where it was 56.
The field names stay, so host source compiles unchanged. kind is a
uint32_t, because an enum's size is the C compiler's choice. Kinds are only
ever added, and a host must accept one it does not know. A struct that will
grow starts with its size: WolangVmOptions for wolang_vm_create_with
(which answers null for a size it does not know; wolang_vm_create stays).
Calls answer a WolangStatus (kWolangOk, kWolangError, kWolangTrap)
instead of "non zero", with the same 0 and 1. An object value keeps its
type's id in the reserved word, since the union holds the object. A C11
translation unit asserts the layout in the test build. MowoEngine's
generated glue calls through C functions and words (D218, D219), not values,
so it does not change.
D248. A release is one tag push, and the version is written once. The
compiler's version lives in project(VERSION) in CMakeLists.txt; the
driver, the language server and the embedding API take it from there at
build time (WOLANG_VERSION_STRING), and tools/package/bump_version.py
sets the only two other copies (the VS Code and Zed manifests), which
check_version.py keeps equal in wl docs and CI. The embedding ABI
version, when it exists, is separate. release.yml first checks the tag
is v<that version> and that CHANGELOG.md has its section; builds every
archive with the pinned LLVM 22 so wolang build works from a release;
publishes with the changelog section as the notes; then installs the
published release with install.sh and install.ps1 on three systems.
The installers refuse a download with no checksum line unless told
--no-verify, fall back to the /releases/latest redirect when the API is
rate limited, and read the archive's top directory instead of assuming
it. WOLANG_DOWNLOAD_BASE points them at a local server, for
tools/package/test_install.sh only.
D247. The platform layer grows a cursor, window control, pads and
dialogs. What a game or a tool needs next to a window (D155) goes in
platform/ and lib/gfx, one C function each and one backend per system,
with none.cc answering no. Cursor: visibility, seven shapes and a lock
for mouse-look; a locked pointer reports movement in scroll_x/scroll_y
of kMouseMoved (X11 grabs and warps to the centre rather than loading
XInput2, Win32 clips and reads raw input, Cocoa disassociates the cursor).
Window: fullscreen (Cocoa uses the system Space, since a hand-made
borderless window cannot become key), size, minimum size, position, file
drops (kDropped, XDND text/uri-list on X11, WM_DROPFILES, the
dragging destination) and kScaleChanged (Xft.dpi on X11, which is the
only scale X offers).
Gamepads are four slots in one Xbox-shaped layout, no deadzone (the
program's call; gfx.Deadzone is the helper), up and left negative like
a screen. Linux reads evdev directly, non-blocking, rescanning once a
second, with no libudev; Windows loads XInput at run time; macOS uses
GameController.framework, the smaller of it and IOKit HID because it
already knows the pads. Connect and disconnect come through Window.Poll
behind the window's own events. The evdev and URI-list decoding is
header-only and system-free, so it is unit-tested from byte buffers.
Dialogs block. Windows and macOS use their own; Linux runs zenity or
kdialog with fork and exec (never a shell) and otherwise answers
unavailable, which DialogsAvailable() reports first. Not done: XInput2
raw motion on X11, pad rumble, multiple-file open, file-type filters.
D231. The fuzzer generates the language, not a calculator. It made
int and float arithmetic, so every bug the audit found in folding,
casts, labelled loops, narrow types and counting was out of its reach.
It now generates every integer width with boundary values, casts,
shifts, bitwise operators, labelled break/continue, match on ints
and enums with payloads, optionals, T!, interpolated text, classes
with aliasing and a live-object count before and after, and compares
trap text as well as trap kind. Operations that can trap are rare (6%)
so most programs run to a value. A unit test checks that every generated
program type checks, so a generator bug is not mistaken for a compiler
bug, and wolang fuzz <file> replays one program for minimising.
D249. An integer constant of an aggregate type is that type's zero.
An error of a T! is built with an integer 0 as its payload, because a
failed T! has no value and nothing may read one without matching first.
When T is an aggregate -- a slice, a text, a struct -- the back ends
gave that constant an area but emitted it as an integer, through a spill
slot it was never given. kNowhere * 8 wraps to just below the spill area,
which is the last saved register, so returning an error overwrote a
register the caller was keeping: on arm64 a loop's tag, on x86-64 the
pointer an optimised host kept its result in. The constant's area is now
zeroed with the frame's slots and never written, AOT makes it LLVM's null
of the type, and asking for the spill offset of a value with none is a
check failure instead of a write into the frame.
D250. The compiler will be self-hosted in stages, after 1.2.0. D2 said the
compiler stays C++20 and self-hosting is not a goal. That held while the
language was moving; 1.2.0 lands the last breaking decisions (D235-D240), and
a compiler is the largest program anyone will write in Wolang, so writing it
there finds what the language and std lack faster than any benchmark. The
C++ compiler is not replaced: it stays as the bootstrap, as Rust's did, and
the runtime, the platform layer, the interpreter, the JIT emitters and the
LSP stay C++ longest. Work goes stage by stage as in
docs/plans/self-hosting.md, each stage checked by
giving the new and the old part the same input and requiring the same output.
Nothing is self-hosted inside 1.2.0 except the groundwork: std that a
compiler needs, and the IR reader.
D238. int is s64. A length, an index and a hash are int, and int was 32
bits, so a collection, a file read into memory or a hash table stopped at 2^31. Making only len
wider would break every var i = 0 ... i < xs.len, because there are no mixed-width operations (D33).
Making int wider breaks almost nothing written in the language. A word-sized isize would differ
between wasm32 and x86-64, and one answer on every target is the rule this project keeps. The
boundaries keep their widths: C's int is s32 in extern and in bindings, and a stored format
writes an explicit width. float stays f32: that is the GPU's width, and a different question.
A tag is u32 wherever it is read; int names a width the program chose, never a layout's.
D239. An ability can name the type that implements it, and an operator is an
ability. Self in an ability is the implementing type: a projection like
Item (D212), so the machinery that rewrites Item onto its owner rewrites
Self, and every monomorphised copy is concrete. A projection named Self
whose owner is not an ability is its owner. Inside extend T or a type's own
body Self is T. Anywhere else it is E0156. An ability that takes or
returns Self cannot be a value type (E0157): a pair of data and table
cannot say what other is. An ability marked @Operator lends its one method
to an operator: Plus +, Minus -, Times *, Divide /, Negate -x, Equals == !=, Compare < <= > >=; a malformed one is E0160. std.ops offers Add, Sub, Mul, Div, Neg, Eq, Ord and Hash. a + b on a Vec3 is the call
a.Plus(b), decided in the checker, so lowering sees an ordinary call and the
tiers have nothing new to agree on. The ability is required, not just the
method's name: relaxing that later breaks nothing, and tightening it would.
Numbers meet every operator ability, bool and char meet Equals and
Compare, and text meets Equals, so a bound like <T: Add> takes int.
D235. An error can carry data, and a set is written or open. error Malformed(line: int, column: int) is an error with a payload. A function states
which errors it gives back, -> Value!(Malformed, TooDeep), or writes T! and
means any. Inference was promised and never built: every T! accepted every
error, so a match that missed one compiled and trapped. A written set is
checked: returning or ?-passing an error outside it is E0151. A match over
it is exhaustive (E0201). An open T! needs a catch-all arm. An offered
function states its set (E0150), for the reason it states its effects: a caller
in another module cannot see the body. A payload is boxed in a counted object,
one optional slot per payload error the set names, in tag order, so a set with no
payload errors keeps today's two-field layout. An open T! carries no payload
(E0152), because its layout cannot depend on a body, and a payload error is
built only where a T! is wanted: alone it has nowhere to keep its data. Errors
are told apart by declaration, not by name: a tag is FNV-1a of module.Name,
never zero, the same in every tier, every build and every reload. This replaces
D13's "inferred" and "no data", and D90's "named by the symbol". The value of a
payload error before it is put into a T! is the tag and the slot, and it is not
counted on its own.
D236. unsafe fun is a promise the caller makes; uses unsafe is one the
author made. A body that states uses unsafe vouches for itself and absorbs
it (D35, D221). Some bodies cannot: a function whose safety depends on the
pointer or length it is given is only as safe as its caller. unsafe fun says
so. Its body may do unsafe things without stating it, and every call or naming
of it (D222) charges unsafe to the caller, as extern does, whatever its body
or its uses clause says. A bodiless primitive is written unsafe fun: uses unsafe there said one thing and meant the other (E0153). An unsafe fun
cannot override or implement a method that is not one (E0154), because a call
through the base would skip the promise; the other way round is allowed. unsafe
before fun, task, virtual, override or extern is the modifier; unsafe { is still the block. std.mem, TextFromRaw, ParallelFor, AtomicAdd,
socket.Read and gfx.TextFromC are unsafe fun.
D237. What allocates is worked out, and only @Hot asks. alloc was an
effect nobody inferred, so @Hot accepted anything. Allocation is now a
property of a body, found the way D232 finds a thread's globals. Constructing a
class, ArrayNew and SoaNew, building owned text (interpolation, +), a call
to a bodiless function that returns owned text, and an error that carries data
(D235) set it. A call or a naming (D222) passes it on, across modules and
through methods, because List.Push is in another module and is a method. A
@Hot function that reaches it is E0155, with the chain of calls. It is not
a row effect: stating it on every function that formats a message would cost
everyone, for one attribute's benefit. The work is one program-wide query that
is asked only when some declaration is @Hot, so a program without one pays
nothing. Not followed: extern calls, a function value that the hot body did
not name, virtual dispatch past the base method, ability-bound calls inside a
generic body. uses alloc and the alloc capability are gone; a name in uses
that is no effect is E0158 (so alloc is one), and alloc in capabilities
is E0053.
D240. A closure that outlives its call owns what it captured. A function
value stays two words: the code and an environment. A function-typed parameter
is borrowed: the environment may be in the caller's frame, so storing it,
returning it or giving it to a keep parameter is E0161. keep fun(...) on a
parameter says the callee may keep it. A function type anywhere else (a field, a
global, a result, a local annotation, a type argument) is kept already, and
keep there is E0046. A function name, and a literal that captures nothing,
are kept; a literal that captures is kept only where its expected type is, and
then its environment is a counted heap object (Tuple{drop address, captures}).
The last release runs the drop, which lets go of each capture and frees the
object. A value that flows into a kept place is retained, one that leaves is
released, so a kept function value is counted like a class handle. A closure
that does not escape is what it was: a frame environment, no header, no count.
What an escaping closure takes is copied (plain values), retained (counted
values), copied again (owned Text) or moved (a value with a deinit; the name
around it is then E0122, and the drop runs the deinit, which D229 left
undone). A reference, a pointer, a slice or a borrowed function cannot be taken
along (E0162). E0136 is retired. The C ABI is unchanged: a C callback takes
the code word, and a capturing closure to an extern parameter stays refused.
Not covered: retaining around a call through a field or global (the gap of
D193), a closure that stores itself in self (a cycle), and a kept closure
built inside arena {} that outlives the block. A borrowed value reaching a
kept slot through a generic T is E0161 too: the type argument is kept.
D261. The embedding package carries no LLVM. wolang_core linked LLVM's
component libraries PUBLIC and Wolang::core is exported, so an installed
WolangTargets.cmake and a host that linked it named libraries the host does
not have. Ahead-of-time compilation belongs to the driver: codegen/aot/aot.cc
is its own target, wolang_aot, which holds the LLVM definitions, include
directories and libraries. The driver and the tests link it; it is never
installed or exported, so nothing a host reads mentions LLVM. The amalgamation
already left aot.cc out. tools/package/test_install.sh fails when an
installed target file or wolang-embed.pc contains "llvm".
D262. The reflection bindings cover statics, text, structs and object fields,
and ask the host before touching an object. The generator of D218 left out a
third of an engine: its static functions, anything that answered a
std::string, a struct, a TObjectPtr, and a rotator; it built a Quat with
w first where the engine takes x, y, z, w; and its glue only tested self
for null, so an object the engine had destroyed was called anyway. Now a quaternion
is x, y, z, w and FRotator is Rotator{pitch, yaw, roll}. TObjectPtr<X> is
an object like X*, read through .Get(). A std::string answer is kept in a
thread-local string in the glue and handed over as data and length, and the
wrapper makes an owned copy of it at once, because the next call overwrites the
buffer; a text property is the same pair of accessors. A reflected struct whose
fields are all numbers is answered through an out-pointer the way a vector is
(a struct is never taken as a parameter: it has no constructor to build it
from). A static function becomes a free offer fun, over an extern "C" fun
named Class_Name; the name is the engine's, or the class name without its
one-letter prefix in front of it when two classes have the same one. Every
call and every property access tests MowoWolangAlive first, unless the driver
is given --no-alive-check. The reflection JSON is assumed to give a struct as
"kind": "struct" with properties shaped like a class's.
D252. A thread body is followed into every module and every method. D232 walked
only the calls inside the thread body's own module and only the functions of
that module, so a body that called lib.Bump, or a method such as
counter.Hit(), reached a module-level var without a word. The walk now
runs over the whole program, the same graph @Hot asks about (D237), and a
method is its own node. A function that states uses unsafe, or is an
unsafe fun, stops the walk: its author vouched for what two threads sharing
it do, as the caller does (D232). A function handed to a @Thread parameter
may be written Name or module.Name (D246). Not covered: a function held in
a variable and then handed over, which names nothing the checker can follow, is
not judged. Refusing it is a later step: it needs to tell a parameter that is
itself @Thread from a local.
D259. A host names a function by module, loads a lone file with its need, and sees Print. Four gaps in the embedding API, found by the engine integration. (1) wolang_vm_function_name was one flat namespace, so two packages that both offer Tick could not both be called: a function may now be named module.Name (sample.mover.Tick) in wolang_vm_call and wolang_vm_has, wolang_vm_function_module says which module offers function i (ABI 1.1, D241), and a bare name that two different modules offer is refused with a note saying to qualify it. Files of one module (the host package a session's lines share, D140) do not conflict. (2) A single .wo given to wolang_vm_load went through LoadFile with no package and could not need anything; it now joins the host package the way source text does, and the standard library is loaded first from WOLANG_STD or a lib/std above the file, the way the driver finds it. (3) wolang_vm_destroy released nothing: a module-level var that held a host object, or a C? of one, never got its release. It now lets go of every global of every loaded file, and LetGo covers an optional. (4) std.io.Print reaches wolang_fd_write, not libc write, so the writer a host set never saw it; the runtime gains wolang_set_fd_write_hook, which wolang_vm_set_writer and _set_reporter install for fds 1 and 2, in the interpreter, the JIT and AOT alike (a null hook costs one atomic load). The hook is process-wide like the writer itself; per-VM writers belong to the host-space decision. Also here: KindOf answers kWolangObject for a script class and kWolangHandle for an extern class (an optional of either too), so wolang_function_parameter is no longer kWolangVoid for them, and a program object passed as an argument is boxed instead of becoming void (the "undefined value" of the engine's item 17). A host handle bound to a T? parameter is absent when null, and a foreign function that answers an optional handle maps 0 to absent, so if const fires. target?.Method() type-checks and lowers as the method's result made optional.
D255. A reload that fails, or would change a class the host holds, changes nothing. wolang_vm_reload rebuilt the vm in place before it knew the new program compiled, so a typo in an editor left every call failing until the next good save, and an object the host held could be read with a layout it no longer had. Now the new program is loaded into a staged vm first, with its initializers and the compiled tier off. If it does not compile, the vm keeps the program that was running, wolang_vm_errors carries the new program's errors, and the reload answers an error with wolang_vm_ok still 1. The vm also tracks the objects the host holds: those a call handed to it and those it took a reference to, until it lets go. If the new program has no class of that name, or its stored fields (name and shape, descriptor classes included) differ, the reload is refused and the notes name the class. Migrating a held object in place is not built: the old address is still held by the host and possibly by other objects, so moving it needs a walk that rewrites every reference, which cannot be written correctly without running it. That is the follow-up; the refusal is the safe floor. The bindings, host classes and writers of the vm, its answers and its held objects carry across the rebuild, and wolang_vm_object_get finds a class by its name rather than by an index that a reload changes. Cost: a reload compiles the program twice.
D257. A vm's bindings, host classes and writers belong to the vm. wolang_vm_bind, wolang_vm_register_class and the writers ignored their vm argument, so two vms could not bind one symbol to different functions, and one engine world's output reached another's log. They now live in the vm's global space, which a turn already enters (D214) and which threads a program starts inherit. A lookup asks that space first and the process after, so a host that never made a second vm sees no change. wolang_vm_set_writer and _set_reporter install a hook in the space that wolang_fd_write and the write symbol consult, replacing the process-wide hook of D259. A reload copies the bindings and classes into the new space.
D258. A host can make and fill a script object and read the program's classes. The engine scanned .wo sources to find classes and could not make one. wolang_vm_object_new allocates a class's object with every field zero and wolang_vm_object_set writes a numeric, boolean or character field; a class with virtual methods or a field zero cannot stand for (text, a class, an array) is refused, because filling it from the host would need the program's allocator and drop rules. wolang_vm_class_count/_name, _field_count/_name/_kind and _attribute_count/_name read the host's own files' classes, their stored fields in order and the attributes written on a class. The ABI minor is 2 (D241); nothing existing changes.
D263. math is a package of float vectors, matrices and quaternions whose operators are abilities. Vec2, Vec3, Vec4, Mat3, Mat4 and Quat are structs of float (f32, as in std.math: a gameplay number is a position, not a bank balance). + - * / unary- and == on them are the std.ops abilities of D239, so a + b is the call a.Plus(b) and a generic <T: Add> takes a Vec3. Between vectors * and / are per component; scaling by a number is Scaled(k) and Divided(k), because an operator ability takes Self and one type cannot also take a float. Mat3, Mat4 and Quat meet Mul (a matrix product, a Hamilton product, the right side applied first) and Eq. == is exact; NearTo(other, tolerance) is the comparison a test wants. Square roots and trigonometry come from std.math, so the functions that use them state uses unsafe as theirs do. Normalized() of a zero vector is zero, not a trap. Quat.Slerp negates one side to take the short way and falls back to a normalised lerp when the two are nearly equal. The package is math, the module of the same name in std stays std.math; a program writes need math.{Vec3} and need std.math.{Sqrt}. No general matrix inverse and no projection matrix yet.
D256. A script class does not derive from an extern class. A subclass of an extern class compiled, but its instances were script objects with no host object behind them, so a call that reached the host dereferenced a pointer the host never made, and an upcast to the extern type handed the host that same pointer (the engine's item 19). A class whose base is an extern class is now E0164, with the hint to hold the host object in a field. An extern class may derive from another extern class: both are host types and the host decides what the base means. A host-backed subclass, where the host makes the derived object, needs a decision of its own and is not part of 1.2.0.
D253. A module-level weak var holds its object weakly. weak var was parsed on any declaration and checked only inside a class, so at module level it quietly behaved as a strong reference: an assignment retained, a read never asked whether the object was still there, and after the host set alive = false a != null check was still true (the engine's item 2). A module-level weak var is now allowed for a C? of a class or of an extern class, and works as a weak field does (D207): a write takes the weak count and lets go of the old one, a read upgrades to a counted reference or null, and the file's initializer lets go of the strong reference its initial value arrived with. Any other weak on a module-level let or on something that is not a C? is E0137, and a weak local does not parse (E0040): it would be null by the next line. Binding::is_weak carries it to the files that use the name, and the cache format moved to version 6. A VM that goes away or reloads lets go of a weak global's weak count, not a strong one, and a weak host handle costs the host nothing.
D254. A host class may give its objects a token, so a weak reference is not fooled by address reuse. alive(object) is asked when a weak reference is read, and a host that frees an object and builds another at the same address answers yes for the new one (the engine's item 9). A class may now be registered with a token callback, uint64_t token(void* object, void* user), through the size-prefixed WolangHostClass of wolang_vm_register_class_ex (D241; ABI 1.3). A weak slot that points at a host object of such a class holds a small cell made by wolang_host_weak_make: the object and the token it had when the reference was stored. A read calls wolang_host_weak_upgrade, which asks alive first and only then token, so the token callback is never asked of an object the host has already freed, and answers null unless both agree; the reference it hands back is counted like any other. A class with no token keeps the slot as the plain pointer, so nothing changes for it, and nothing costs more than the existing alive call. The class has to be registered before the program is loaded: the cell is what the slot holds, and a registration made afterwards would read a pointer as a cell. Two things differ from the first sketch of this decision. The slot stays one word wide, because a cell behind the word leaves every layout, struct copy and tier as it was; the cost is one small allocation per weak store of a host object of a class with a token. And the token is a number the host chooses, so an index and a serial both fit. A weak var or field is released with wolang_host_weak_free, including when a VM is destroyed or reloaded.
D264. The IR has a typed text form that a reader takes back. The stable dump (PrintModule) leaves out what a reader would need: the type of nearly every value, the string pool, the classes, the callee's index, an alloca's slot count, the flags of a function, and it spells some types the same way (*T? is both a pointer to an optional and an optional pointer). Changing it would change every golden test, so it stays. PrintModuleTyped (wolang ir --typed) is a second form that says all of it, one instruction per line, and ir::ReadModule rebuilds an ir::Module from it (stage 3 of D250 compares IR dumps of the C++ and Wolang lowerings, and needs this). Types are written in a grammar of their own (opt(T), ptrmut(T), slice(T), array(N, T), tuple(A, B), fn(A) -> R, s64, text); a kind the reader cannot build from text (a declared struct, class or enum, an instance, a fallible) is written in backticks as Describe spells it, and the caller of ReadModule gives the types that may appear (ReadOptions::named). Callees are written @"name"#index: the index is what the reader uses and the name is a check for the human. Instruction ids are kept as written, so print, read and print again is the identity; an instruction that produces no value has no id in the text and gets a fresh one at the end. Source spans, debug locals and the ClassInfo method order beyond what is printed are not carried: the form is for comparing IR, not for resuming a compile. The result of ReadModule is meant to pass ir::Verify; the tests check that.
D260. A machine is given its time by Tick(dt). D243 reserved
on X after 3s for a machine with a clock. A machine with an after now has
a hidden member holding the seconds spent in the current state, and a generated
Tick(dt: float) -> bool: it adds dt to that time and takes the first
transition of the current state whose delay has run out, through the exit,
state and enter steps an event takes, then says whether one fired. Every change
of state, by an event or by Tick, sets the time back to zero. The clock is the
caller's: pausing is not calling Tick and slow motion is a smaller dt, so a
host needs no hook for the game clock (engine item 23). The delay is a number
of seconds; a quantity is read as its number whatever its dimension, and
anything else is E0101 (E0159 is retired). A machine with an event called Tick keeps it and
gets no clock. Self is bound in a machine's enter and exit blocks. This
amends D243.
D251. A slice is read-only unless it says mut. []T is a view whose
elements cannot be written and []mut T one whose elements can, as *T and
*mut T, &T and &mut T already are. []mut T fits where []T is wanted
and not the other way round; an array fits either, since it is the storage. A
slice cut out of another by a range keeps the mutability of what it was cut
from. Writing an element (xs[i] = v, xs[i] += v, xs[i].x = v) or taking
&mut xs[i] through a []T is E0165; a write that goes through a handle the
slice holds (cells[i].n = 1 for a class) changes the object, not the
slice, and is allowed. .data of a []mut T is a *mut T, of a []T a *T.
The syntax is []mut T (and soa []mut T): mut follows the brackets.
This replaces the stopgap of D233, which made the caller of std.text.Bytes
promise never to write through its view: Bytes now answers []u8 and needs
no uses unsafe from its caller. @Exposes remains for a library author who
needs it, and nothing in std uses it. What allocates or views raw storage
(ArrayNew, SoaNew, BytesFromRaw, ViewFromRaw) answers []mut T;
Prefix and Suffix take and answer []T and PrefixMut is the writable
one, which List.Items uses.
Mutability is a checker property. The IR erases it, as it erases a dimension,
so the layout, the three tiers and the verifier are unchanged; there is no
verifier rule for it, because every producer of a slice would have to keep the
bit exactly right in the IR for the rule to mean anything. This breaks source
that writes through a []T: the fix is the mut the diagnostic points at.
Not covered: a method that changes a value-type element in place
(xs[i].Move() on a struct), and an array bound with const handed to a
[]mut T.
D276. font reads TrueType and rasterizes it in Wolang, and every part but file lookup is computation. The package parses a font from bytes (FontLoad): the table directory, head, maxp, hhea, hmtx, loca, glyf (simple and composite glyphs, with scale and 2 by 2 transforms), cmap formats 4 and 12, and a horizontal kern format 0 subtable. It is written here and not bound to FreeType for the reason image and deflate are (D177, D245): the reader is the first code a hostile file reaches, so every offset, count and table is checked against the bytes that are there, a glyph is capped in points (16384) and components (64, depth 6), and the way out of a bad file is Malformed, Unsupported or TooLarge, never a trap. OpenType with CFF outlines (OTTO) is Unsupported, and GPOS kerning and hinting are not read: the first needs a charstring interpreter and the second is not needed at the sizes a game draws. Rasterize fills the flattened outline with exact signed-area coverage per pixel (a scanline accumulation, so edges are anti-aliased without sampling and a pixel-aligned box is exactly 255), at a size in pixels per em. LayoutText places code points by advance and kerning, breaks at newlines and at spaces when a width is given, and breaks inside a word that is longer than the line. Atlas packs glyph bitmaps on shelves into one coverage bitmap and caches a slot per glyph and size, so a renderer uploads it once. SampleFont is a built-in 5 by 7 font in one case, generated from rectangles by tools/gen/make_font_fixtures.py, so text can be drawn with no file and the tests have real bytes. The part that needs the machine, SystemFontFile and FontLoadFile, answers null or Missing when there is no such file, and Available() says whether a common system font was found. Class values are passed by value, not as &Font: a handle is already a reference.
D277. ui is immediate-mode, its output is a draw list, and the draw list is plain data. The program describes the interface on every frame and each widget call answers what happened to it (if ui.Button("Play") { ... }), so there is no widget tree to keep in step with the program's state. A frame is Begin, widgets, End; a widget's identity is the order it was called in, which is enough for the handful of kinds the package has (label, button, checkbox, slider, text field, list box, panel) and costs the program nothing to remember. What the frame produces is a list of DrawCommands, a filled rectangle or a run of text with a position and an 0xAARRGGBB colour, and nothing else: layout, hit testing and the state machine of press, hold and release are therefore computation, tested with no window, and any renderer (the software one in the package, a gpu batch later) draws the same list. Render(image, ui) is the software renderer; it blends with the alpha each colour carries, and a frame is checked against a stored picture by its checksum. Input is a struct the program fills, by hand or from a gfx event as it is. A click is a release over the widget the press began on, and active is forgotten on the frame the button is up, so a lost release cannot wedge the interface. The labels of a frame are copied into one Builder the context owns and each command holds an offset and a length into it, not an owned Text: a class with a Text field corrupts the heap when its last handle goes (found while writing this package, in the reference interpreter and the compiled tier alike; it is a gap in the compiler, not in ui), and a builder per frame is the shape that needs nothing freed per label anyway.
D272. Ahead-of-time output links without a C compiler. On Linux
wolang build used to call c++, so a container with no compiler built
nothing. Now LinkCommand picks between two linkers. The system one (c++)
stays, and is used whenever it is on PATH and the owner has not asked for the
other. The bundled one is ld.lld from a sysroot that holds musl's start-up
files and libc.a, a libc++ built for musl, and the runtime archive compiled
against them (tools/package/make_sysroot.sh, built in Alpine and released as
wolang-sysroot-linux-<arch>.tar.gz). The sysroot is WOLANG_SYSROOT, or
lib/wolang/sysroot beside the runtime archive of an installed compiler. It is
chosen when there is no c++ or cc on PATH, or when WOLANG_LINKER=bundled;
WOLANG_LINKER=system forces the old way. The output is a static executable,
so it runs on any Linux of the same architecture. The same source and flags give
the same bytes: the object is a pure function of the program (a test emits one
twice from separate databases and compares), and the bundled link line passes
--build-id=none. macOS and Windows keep the platform linker, which is always
there. Also: the AOT link line carries the sanitizers the runtime archive was
built with, and the release C runtime when they are on, because clang-cl
refuses /MDd with -fsanitize=address. Not done: the sysroot has been
written but not yet built in CI, and nothing pins the hashes of what it
downloads; its MANIFEST records them so that the first release can.
D273. The cost of a script call is published. An engine budgets scripts per
actor, so it needs calls per millisecond and allocation per call, per tier, on a
build that is not a debug build. benchmarks/actors ticks a crowd of actors in
the interpreter, the JIT, ahead-of-time code and C++ behind virtual calls, and
prints a table of microseconds a frame and a second table at 1000 actors a
frame: calls per millisecond, and objects allocated per call. The second
number needs a counter that does not forget: wolang_objects_allocated counts
every object ever allocated, arena or not, beside wolang_objects_live; the
script reads it through an extern "C" declaration and the AOT build through
the exported Allocated. docs/benchmarks.md holds the measured table with the
machine described, and is part of the site. wl bench [preset] runs it, and
the windows-release preset exists so that a Windows number is not a debug
number. The table is measured again when the runtime or a back end changes
enough to move it.
D279. audio mixes in Wolang and plays through gfx; compressed audio is read as far as its headers. A Clip is interleaved f32 samples with a rate and a channel count. WavDecode reads 8, 16, 24 and 32-bit PCM and 32-bit float (float bits are turned into a number by arithmetic, because Wolang has no bit reinterpretation), WavEncode writes 8, 16 and 24-bit PCM, Resample converts rate by linear interpolation and Remix changes mono and stereo. A Mixer sums voices (a mono clip panned with equal power, a stereo clip by balance), applies voice, bus and master gain, clamps to -1..1 and drops finished voices; it touches no device, so its output buffer is what a test compares. Output pairs a mixer with gfx.OpenAudio and Pump queues what the device has room for; with no device it answers 0. Ogg Vorbis is read to its headers (OggVorbisInfo: channels, rate, nominal bitrate, length from the last granule position, comments), with every page's CRC checked against an independently computed vector. The audio packets are not decoded: a decoder written without one real stream to check it against is a decoder nobody can trust, the machine that wrote this had no encoder and no sample file, and vendoring a C decoder would break the rule of no new dependencies. Decoding waits for a reference stream; until then Clip is the type a decoder will fill. Errors are Malformed, Unsupported and TooLarge, never a trap on a hostile file (D245).
D280. gpu is a Backend ability with a null backend and an OpenGL 3.3 core backend, and the GL calls live in the platform library. A Device holds a Backend; buffers, layouts, programs, textures, targets, state and draws are its methods, and the handles are small structs the caller deletes. A second backend (Metal, Vulkan) is another extend T as Backend; callers do not change. The null backend draws nothing and records every call as a line of text, hands out ids, counts misuse (an unknown id, a draw with no program, a uniform with none in use, a texture with no size) and compiles a shader as far as checking for main, so the package and the programs on it are tested on every runner with no display. wolang_gl_proc returns an address and Wolang cannot call one it was handed, so platform/gpu.cc looks the 3.3 entry points up once and exposes them as wolang_gpu_* functions over plain numbers (header include/wolang/platform/gpu.h); vertex data goes up as floats and indices as 32-bit integers through the array binding, because the element binding belongs to a layout and core profile refuses it with none bound. ProgramNew returns Program!(CompileFailed) and the error carries the compiler's log (D235). The OpenGL backend is covered by tests only as far as answering null with no window; a context exists on few CI runners, so a drawing program (examples/app) is run by hand before a release.
D281. app is a fixed-timestep loop around gfx, gpu and audio, and it runs without a window. A Game has Update(dt) at a fixed step, Draw(alpha) once a frame with the fraction of a step left over, and OnEvent(event). The Clock clamps a long frame to max_frame, owes at most eight steps a frame (a stall does not become a spiral), and with a step of zero runs one variable step. AppNew opens the window, uses the OpenGL backend when there is a context and the null one when there is not, opens sound with a mixer that works with or without a device, and Run is the interactive loop; RunFrames runs the same update, draw and sound sequence from a time the caller gives it, which is how a server, a replay or a test runs a game. A Config is overlaid from an ini file (ConfigParse), every number clamped and every missing or unreadable key left at its default; ConfigDirectory is %APPDATA%, Library/Application Support or $XDG_CONFIG_HOME, with ~/.config as the fallback.
D283. The Wolang lexer is checked against the C++ one, token for token. tools/selfhost/syntax (package wosyntax) lexes Wolang in Wolang. Its token kinds are generated from token_kind.inc by tools/selfhost/gen_kinds.py, so the two cannot drift, and wolang tokens --stable prints Kind offset length flags per token for the C++ side. tools/selfhost/diff_tokens.py runs both over every tracked .wo file and every wolang block in the guides and fails on the first token that differs; the error flag is compared too, so a malformed number or an unterminated string is flagged in both or in neither. The payloads the parser needs (the value of a number, the decoded parts of a string with the spans of its interpolations, the code point of a character) ride on the tokens, as in C++. One thing is approximated: identifier letters outside ASCII are decided by a short list of ranges that are not letters (punctuation, arrows and symbols, private use, emoji), not by the Unicode tables, and NFC is not checked. That is a gap to close before stage 4; the corpus check passes with it.
D284. The standard library gets the speed a compiler needs before it gets a compiler. A benchmark package (tools/selfhost/bench) times the maps, Builder, ReadText and allocation; what it found is fixed in std. std.mem.Copy is one call (wolang_memory_copy) where Builder.Append and Builder.Reserve looped a byte at a time. fs.ReadAll sizes its buffer to the file when the path is a regular file and smaller than the limit, and ReadText views what it read instead of copying it. Map.Slot masks, because the table is always a power of two. fs.Exists, IsFile and IsDirectory read the tag with match: the dev tier does not compare two enum values, and a function it refuses takes every caller into the interpreter. The figures are in docs/plans/self-hosting.md; the mask did not show above the noise and is kept because it is correct and costs nothing.
D285. std.arena is a bump allocator a program holds across calls. ArenaNew(chunk_size) answers an Arena; Take(count) hands out a run of bytes aligned to eight, Reset hands the same storage out again, and the drop gives every chunk back. A run never moves, so a view of it is good until Reset or the arena's end; a run bigger than a chunk gets a chunk of its own. It deals in bytes and not in values of a type: a typed view is ViewFromRaw, and a text built in an arena is still a text nobody frees (D67). The language's arena { } block stays what it was, the arena of one block.
D286. The package fetch and lock is the first tool written in Wolang. tools/selfhost/fetch reads the manifests with std.toml, walks the dependency graph as the workspace loader does, clones with the git on the path and writes wolang.lock. It needs no parser, which is why it comes first. tools/selfhost/diff_fetch.py checks it against wolang fetch: two copies of an application, one fetched by each tool from local repositories, must have byte-identical lock files. The check found that the C++ fetch never worked on Windows (single quotes and 2>/dev/null in a command cmd.exe runs); it quotes and discards errors per platform now. The Wolang tool runs git through std.process.Start, with no shell, so a path or a revision cannot be read as a command. It leaves out the version-requirement check and the unknown-key report, which stay with the build.
D287. Two tier disagreements a parser found. The reference interpreter trapped on or (and and, xor) of two bools, which an or-pattern lowers to, while the compiled tiers took it; it computes them now. And == or != on an enum whose variants carry nothing was an eq of two aggregates, which the dev tier refuses, taking every caller of the function into the interpreter; lowering compares the tags instead, as match reads them, and an enum with data is still compared whole. Both have a test in CodegenTest that runs the three tiers, and the second has one in LowerTest on the IR.
D288. The Wolang parser reads valid programs and is checked against the C++ one. tools/selfhost/syntax parses to a tree shaped like ast.h and prints it as ast_printer.cc does. diff_ast.py compares the two dumps on every tracked .wo file and every wolang block of the guides; files the C++ parser rejects are counted and left out. It leaves out recovery, diagnostics and the depth limit, which are stage 2 work of their own, and it fails with the token it stopped at rather than guess. Writing it found that Class() ignores field initializers and leaves class-typed fields null (docs/plans/self-hosting.md, gap 7), which is a question for the language and not decided here.
D289. A class owns the bytes of its Text fields. A Text field is stored as the two words a borrowed text is, the pointer and the length, so an object that has one is allocated with room for both. The layout read a class member by its declared type, Text, which it did not know, and gave it no bytes: the two-word store at construction ran past the allocation and corrupted the heap when the handle went. The rule is the one a local follows (D133): the object takes the bytes it is built with or assigned, writing a new Text into the field gives back the old, and the last release of the object gives back what it holds, in Class$drop. A text field is a view and is never freed. Fields of a struct are not changed: lowering erases them before they are laid out, and a struct's owned text is a separate question.
D271. Generated bindings say which ABI and which struct layouts they were made
for. The glue wolang bind --reflection writes begins MowoWolang_Bind_*
with wolang_vm_expect_abi(vm, WOLANG_ABI_VERSION) and, for each value struct
it passes by pointer (a vector, a rotator, a reflected struct), one
wolang_vm_expect_struct(vm, name, layout, count). The layout is the host C++
compiler's own answer, {sizeof, alignof, offsetof...}, and not a hash of it:
a mismatch can then say what each side expected, and a hash could not. The
library accepts the bindings when the ABI major is equal and their minor is no
greater than its own; it compares a struct's numbers with the layout the
compiler gives the struct of that name in the program, when the program loads
or at once if it is loaded. Any difference fails the load, with a line naming
the struct and both layouts, and wolang_vm_ok is zero. The expectations
survive a reload. A change to how the compiler lays a struct out is an ABI
change and bumps the major version. No syntax; ABI 1.4 adds the two calls.
This is the compatibility rule the engine asked for (engine items 13 and 16).
D265. A host gives a call a budget of ticks, and a runaway script stops.
wolang_vm_set_budget(vm, ticks) (ABI 1.4) limits every call. A tick is taken
at each entry into a script function and each jump back to an earlier block,
which is where a program that does not end spends its time; straight-line code
costs nothing. A call that uses its budget up stops with kWolangTrap, the
trap is the step limit kind with the function and place, and the vm is
untouched: the next call starts with a full budget. Zero, the default, is no
limit. It is counted in both tiers. The interpreter keeps a counter; the
compiled tier keeps one in the context it already carries, taking a tick with a
subtract and a branch on the sign, in code compiled with budgeted, and
nothing at all in code compiled without it. Setting a budget on a vm that is
loaded builds the compiled tier again, once, when the budget turns on or off.
Where the two tiers draw the line between "a jump back" differs by a block or
two (the interpreter by block number, the compiler by where the block was
placed), so a budget is a guard against a loop that never ends and not a
meter: do not expect the same count from both. A host function a script calls,
which calls a script function again, starts that inner call with the full
budget. arm64 code does not count yet; a vm with a budget runs the interpreter
there. The older wolang_vm_set_step_limit counts interpreter instructions,
keeps calls off the compiled tier and is left as it is. A call that traps does
not run the destructors of what it owned, as with any trap.
D270. A mod is sandboxed by what it is granted, and is never granted
unsafe. The mechanism is the one D217 built: wolang_vm_load_mod takes the
capabilities the host grants, spelled as a uses clause, and a package that
reaches a function needing anything else does not load (E0703); it may not
declare extern (E0705). Nothing is checked while the mod runs, because the
checking is static and total over the effects. The grant is an allow-list, so
what the host names is all the mod gets; a deny-list would mean a new effect
added later is granted by default. net and io are separate effects, so a
mod granted io to read a file cannot open a socket. A process cannot be
started with io alone either: std.process.Run, like every function that
reaches the network, is uses ..., unsafe, and unsafe is what a mod never
gets. The answer to the question D270 left open is that unsafe is never
allowed to a mod: unsafe can reach any memory, so no list of capabilities can
contain it. A host that writes unsafe in the grant is not obeyed; the name is
dropped and the mod is held to the rest. What is not split is io: reading a
file, writing one and printing are all io, and finer grants would need the
effect split in std, which is not decided. Time is bounded by D265 and memory
is not bounded at all.
D269. A runtime error says where it happened and how the script got there;
the engine's debugger speaks DAP. A trap that reaches a host through
wolang_vm_call now leaves in wolang_vm_errors the first line it always had,
wolang: division by zero at @Down:%12, then the place in the source, --> file:line:column, and one in Function (file:line:column) line for each
activation of the script's call stack, innermost first, up to 64. The
interpreter knows the line each frame was at, because it records the call it
made; the compiled tier gives the trapping function and the functions that
called it, with no line. The same text goes to the reporter the host installed
with wolang_vm_set_reporter, so an engine that shows scripts' errors in its
message log gets them without asking for them. No syntax and no new call.
Breakpoints, stepping, locals and the call stack are what DAP already carries
(D216): wolang_vm_debug_listen opens it over TCP and the editor, or the
engine's own debugger front end speaking DAP, drives the vm. A set of callbacks
that would let the host drive the vm without a socket is not built; the socket
is local, and an engine that wants the callbacks can put a DAP client in
process.
D266. A field marked @Save is written by name and read back, whole object graphs at a time. A save game needs the state of a program written down and read back by a program that may have changed since, and hot reload needs the same thing for the objects a host holds (D255). @Save on a stored field of a class or a struct marks it. std.save.Save(object) writes the marked fields of the object and of every object they reach; Load<T>(bytes) rebuilds them and answers Corrupt for bytes that are not a save of the class. Each field is written under its name and a code for its type, so a reader built from a changed program keeps the fields it still has: one the bytes have and the class lacks, or holds under another type, is dropped; one the class has and the bytes lack stays zero. Each object is written once however many fields refer to it, so a shared reference stays shared and a cycle comes back a cycle. A field that is not marked is not saved. A struct is a value, so a marked field of a struct type is written whole, unless the struct marks fields of its own and then only those. What can be written is a closed set: numbers, bool, char, text, enums that carry nothing, structs, classes without a virtual method (an object that has no descriptor cannot say which class it is), and an optional of those. Anything else on a marked field is E0176, and @Save anywhere but a stored field of a class or a struct is E0175, because a hole in a save file is found the day a player loses a game. An object is written as its declared class: a field of a base class that holds a derived object writes the base's fields. The compiler describes the saved fields as a text schema and the runtime walks memory by it, so the interpreter, the JIT and the AOT tier read and write the same bytes (little-endian). The schema leaves each offset and size for the tier to fill in, because lowering has no layout: type.size gained a field operand, so type.size T @n is where field n of T sits and type.size T @size is how big a class's object is, answered by each tier from its own layout. ShapeOf<T>(), HandleOf and FromHandle<T> are the three calls the compiler answers itself. Decoding checks the whole of the bytes before it allocates anything, so a save that ends early allocates nothing. The runtime entry points wolang_save_encode and wolang_save_decode take the schema and give the table of offsets back to the allocator.
D266, continued: a reload carries the objects the host holds, and says where they went. D255 refused a reload that changed the class of an object the host held, because moving one in place means rewriting every reference to it. The graph save of D266 does that without an in-place move. When the host has registered wolang_vm_set_remap (ABI 1.4), a reload writes down the held objects and everything they reach through stored fields, under the old program's layout, before the old program is replaced, and reads them back under the new program's once it is loaded. Both layouts are described by a schema with every offset spelled out from LayoutStore, so the interpreter, the JIT and the AOT tier lay objects out the way this one reads them. Fields are matched by name and by shape: one that is still there keeps its value, a new one is zero, a removed one is dropped, and one whose type changed starts from zero. Because the whole reachable graph is moved at once, a reference shared between two objects stays shared, and a module-level var that pointed at a moved object is given the new one before the globals are carried, so the host and the program still share one object. The vm moves the host's references to the new objects and calls the registered function once per held object with the old and the new, after the globals are carried; the old objects stay with the retired program (D215). A class with a virtual method can be held by the host and is given its descriptor again, but cannot be reached from another object, because an object cannot say which class it is. A class the new program lacks, a weak field, an array, a closure, a host object or an enum that carries values still refuses the reload and the line names the field. With no function registered nothing changes from D255. This replaces the stopgap in D255's last paragraph.
D267. A routine is a task, and the caller's clock moves it. The engine wanted code that waits for two seconds, for an event, for the next frame, tied to the game's clock, so that gameplay is not written as state machines by hand (engine item 23). The plan proposed new words for it, routine fun and wait. But a task fun already is a function that suspends, wait is where it does, and the compiler already splits every task into a state machine that the interpreter, the JIT and the AOT tier run alike (ir/tasks.h). A second spelling of the same thing would have made the language bigger and taught nothing, so there is no routine: the owner-settled name of the plan is dropped for the reason that its meaning already has a name. What was missing was a clock the caller owns. std.tasks.Tick(dt) hands the scheduler dt seconds and gives every task that was ready one step; one that is not finished comes back on the next tick, and one spawned during a tick starts on the next. Pausing is not ticking and slow motion is a smaller dt, so a host needs no hook for its clock, as with machine (D260). std.tasks.Wait(seconds) is a task with no body, a second suspension primitive beside Yield: the compiler writes its machine, which on its first step turns the seconds into the moment to finish at (the scheduler's clock plus the seconds, kept in the frame) and then finishes once the clock has passed it, or at once for a wait of nothing. Because it has no body it can be waited on from any module (E0305 is unchanged for tasks with one). wait Next is wait Yield(). An event is a flag the task looks at, while !event.raised { wait Yield() }, which keeps the waiting in the module that knows what the event means; std.tasks.Event is the flag. The clock is a f64 that starts at zero and a wait counts from the clock of the tick it begins in, so a one-second wait begun in the first of ticks of half a second finishes in the third. The scheduler belongs to the space of the vm that is running (D214, D257), so two vms have their own tasks and their own clock and cannot resume each other's. wolang_vm_tick(vm, dt, &steps) and wolang_vm_tasks_pending (ABI 1.5) are the host's side: it starts a task by calling a script function that spawns one, and ticks. The tasks of a vm must be declared in one of the host's files, because a task is known to the scheduler by a number that means something only to the file that numbered it. wolang test now runs a test with the scheduler installed, in both tiers, so a test may spawn and tick.
D268. Replication is declared in the script and reported to the host; Wolang sends nothing. The first game is networked and server-authoritative, and a script has to be able to say which of its state the engine should replicate and which of its calls cross the wire, in the engine's own words. @Replicated on a stored field of a script class, and @Server, @Client and @Multicast on its methods, say it. They take arguments for the conditions the engine has: @Replicated(owner) for a field only the owning connection receives, and owner, reliable or unreliable on an RPC. Wolang does no networking, so the attributes change no code: the checker validates them and the reflection API reports them (wolang_vm_class_field_replication, wolang_vm_class_method_*, ABI 1.5), and the host maps them to its own policy. The checker refuses what a host could not send. A replicated field and every parameter of an RPC must be a type that is saved by value, which is the set of D266 except that an object reference means nothing on the other side (E0179); a struct is sent whole and must hold only such fields. An RPC returns nothing (E0179), because the call goes one way and the answer is another RPC. The attributes belong to a member of a script class, where there is an object to replicate: on a struct, on a free function, on a method for @Replicated or a field for an RPC, two on one member, or an argument they do not take is E0178. A host object (extern class) is the engine's own, and its replication is the engine's, so the attributes are refused there too. The binding generator therefore writes none of them. The methods the reflection lists are the class's own and then those of the classes it derives from.
D275. Loose ends of 1.2.0: the text a call or + built goes back, an unknown error in a pattern is named, a function held in a variable is not followed onto a thread, an annotated T! is a T!, an annotated ability is an ability, and a const array is not a writable slice. (1) An interpolated string frees the pieces it consumed (D133), but only knew a number, a bool or a float had been built just now; the owned Text that a call returned, a + joined or a nested interpolation made was left, so "x{Make(n)}y" and Make(n) + "z" leaked one string each. An expression that builds an owned text is now known in one place (BuildsOwnedText): a call returning Text, a +, or an interpolation with something in it. ?? is not one, since it hands back a text it found, and neither is a literal with nothing to interpolate. The concatenation, the + operator and the right side of += give those back. The runtime counts the strings its heap holds, wolang_texts_live, as it counts objects, so a test can prove it: arena strings are not counted, because the block frees them. Not fixed here: assigning to an owned Text variable does not give back the value it held, and a ?? that picks its right operand leaves that operand to nobody; both are gaps in the ownership of whole values, not of an interpolation. (2) A match on a T! with an arm whose name is not an error in scope used to fall through to E0201, which said an arm was missing when the truth was a name was. It is E0166 now, at the pattern, and the arm's bindings are checked as errors so nothing else is reported for it. A plain binding such as other => ... is still a catch-all. (3) D252 left a function held in a variable unjudged. @Thread is on the function that starts the thread, so every function value handed to it is the body of a thread: a name or a closure is followed, and anything else that is a local holding a function is E0163, unless the caller states uses unsafe or is itself @Thread, in which case it is passing on a value its own callers hand it and are judged for. A call result or a field of function type is still not judged. (4) A const e: int!(P) = P(1, "x") took the type of what it was given, the error P, a tuple of a tag and a box that nothing releases, so the box leaked whether e was returned on one path or dropped on all of them. An annotation that says T! now makes the binding a T!, as one that says T? makes an optional, and the box is released with the rest of the value. (5) D251 left two cases uncovered. A method of a value type receives a copy and cannot assign to self (E0125), so xs[i].Move() on a []P never changed the element and needs no check. An array bound with const handed to a []mut T parameter is E0105, as assigning to one of its elements is: the binding promised nobody would write it. (6) var backend: Backend = NullNew() was a Null, the concrete type of what it started with, so assigning any other implementer of Backend was E0101. An annotation that names an ability, or a class the value upcasts to, now gives the binding the annotated type, and the value is wrapped into the ability value when it is stored.
D290. The formatter keeps the parentheses of a cast that ends the left side of <. After as float the parser reads a < as the start of generic arguments, so stop as float < y1 { does not parse. The formatter dropped the parentheses of (stop as float) < y1 because a cast binds tighter than a comparison, and wrote code the parser rejects. An operand that ends in a cast (a as float, a + b as float, -a as float) is now wrapped in parentheses when it is the left operand of <; <=, << and every other operator are unaffected, since they are different tokens. The ambiguity itself stays in the parser: a cast followed by < is how a generic type is written. The formatter also never breaks a line inside the {} of a string, whatever the column limit: a call wrapped there ends the string and the program stops parsing.
D291. A name the host gives finds the program's type, not a dependency's helper. wolang_vm_expect_struct(vm, "Vec2", ...) and the class lookups behind object_new, object_get and the reload's held-object relinking took the first type of that name anywhere in the database, so a private struct Vec2 in std.ops was measured instead of the host's own and every package load failed on a size mismatch. A name is now resolved the way a reader would: a qualified name (game.math.Vec2) is that module's type; a bare name is first a type in the files the host loaded itself, then a type a dependency offers, and never a dependency's private type. expect_struct accepts several remaining matches only when they share one layout, and an object_new or class lookup that still finds two is an error telling the caller to qualify the name. Amends D271 (layout check) and D258 (host-made objects).
D292. The C API saves and loads an object graph, with the bytes std.save writes (continues D266). D266 gave a script Save and Load, and the 1.3.0 notes said the C API did the same, but it only carried objects across a reload. wolang_vm_save(vm, object, &bytes, &length) writes the fields marked @Save of a held object and of every object they reach; the bytes belong to the vm and stay valid until the next save, a reload or wolang_vm_destroy. wolang_vm_load_save(vm, "Class", bytes, length, &out) rebuilds the graph and hands the host the root with one reference, the class named the way every host lookup names one (D291). Bytes that are not a save of the class give kWolangError, and nothing was allocated, because the decoder checks all of the bytes first. A class with a virtual method, or one with a marked field a save cannot write, is refused with a line in wolang_vm_errors, the same cases as E0176 and D266. The schema text is not written a second time: the compiler's writer moved out of lower.cc into ir/save_schema.cc and takes the lookups it needs, so lowering still leaves @k places for the tier and the embedding API answers the same places from LayoutStore and passes the table to SaveEncode and SaveDecode. The text, and so the bytes, are identical to a script's, which is what lets a save written by either side load on the other. The vm keeps the schema of each class for as long as the program lasts; a reload builds the vm again, so the schemas and the last buffer go with it. ABI 1.5 -> 1.6; wolang.hpp has no object calls and gained nothing.
D293. A host can load a package it holds as data, under the name its scripts need (amends D140 and D259). Source handed over with wolang_vm_load_source joined one package called host, and the standard library was found only through WOLANG_STD or a lib/std above a file. A host that embeds its bindings (mowo) and std as data could not give a script the module names mowo or std. wolang_vm_load_package_source(vm, name, manifest_toml, paths, sources, count) (ABI 1.6 -> 1.7) loads a package whose files are in memory. It lives in the made-up directory memory:<name>, its manifest is registered as an auxiliary file there, and a file src/ui/button.wo is in the module <name>.ui, as on disk (LoadPackageFromMemory in compiler/workspace). The manifest is trusted like an on-disk package's, unsafe and [link] included, because the host wrote it; only wolang_vm_load_mod sandboxes (D217). Its dependencies are found by name among the packages loaded before it, a path in one is a placeholder (the grammar still wants one), and a version is checked against the copy loaded (D242), so a host loads std first. A null manifest means the package depends on everything loaded, as host does, and gets the standard library the way a lone source does (D259), unless it is called std; "gets the standard library" is decided by a line that begins with need (a text search of the source, not the parsed imports), and finding std still depends on WOLANG_STD and the working directory. Refused with E0060: the name host, a name already loaded, a manifest naming another package, a dependency not loaded, a path outside the source directory or not a .wo file, a path with an empty, . or .. segment, a path given twice. A manifest that does not parse and a version that does not fit (E0053) are refused before the package is registered, so a corrected retry under the same name loads. A reload replays a memory package from the data the vm kept, through one LoadAgain that replaces the two copies of the path-or-source choice in wolang_vm_reload. Not done: replacing one memory package without a reload, and a package fetched from the host through a callback.