diff options
| -rw-r--r-- | docs/implementation/codfns.html | 8 | ||||
| -rw-r--r-- | implementation/codfns.md | 8 |
2 files changed, 8 insertions, 8 deletions
diff --git a/docs/implementation/codfns.html b/docs/implementation/codfns.html index dbfbafa5..22c86ebf 100644 --- a/docs/implementation/codfns.html +++ b/docs/implementation/codfns.html @@ -10,15 +10,15 @@ <h2 id="compilation-strategy">Compilation strategy</h2> <p>Co-dfns development has primarily been focused on the core compiler, and not parsing, code generation, or the runtime. The associated Ph.D. thesis and famous 17 lines figure refer to this section, which transforms the abstract syntax tree (AST) of a program to a lower-level form, and resolves lexical scoping by linking variables to their definitions. While all of Co-dfns is written in APL, other sections aren't necessarily designed to be data-parallel and don't have the same performance guarantees. For example, the parser appears to be written with some sort of parser combinators. In contrast, BQN is entirely written in a data-parallel style. It does not maintain the same clean separation between compiler sections: <a href="../spec/token.html">token formation</a> and literal evaluation is separated into its own function, but parsing, AST manipulation, and code generation overlap.</p> <p>The core Co-dfns compiler is based on manipulating the syntax tree, which is mostly stored as parent and sibling vectors—that is, lists of indices of other nodes in the tree. BQN is less methodical, but generally treats the source tokens as a simple list. This list is reordered in various ways, allowing operations that behave like tree traversals to be performed with scans under the right ordering. This strategy allows BQN to be much stricter in the kinds of operations it uses. Co-dfns regularly uses <code><span class='Value'>⍣</span><span class='Function'>≡</span></code> (repeat until convergence) for iteration and creates nested arrays with <code><span class='Value'>⌸</span></code> (Key, like <a href="../doc/group.html">Group</a>), but BQN has only a single instance of iteration to resolve quotes and comments, plus one complex but parallelizable scan for numeric literal processing, and only uses Group to extract identifiers and strings. This means that most primitives, if we count simple reductions and scans as single primitives, are executed a fixed number of times during execution, making complexity analysis even easier than in Co-dfns.</p> -<p>A goal for BQN was to not only write the compiler in BQN but to use BQN for the runtime as much as possible, while Co-dfns uses a conventional runtime written in C with ArrayFire. This goal has largely been achieved, so that the current BQN runtime uses a very small number of basic array operations currently provided by Javascript. However, more basic operations may be added in the future for performance reasons.</p> +<p>A goal for BQN was to not only write the compiler in BQN but to use BQN for the runtime as much as possible, while Co-dfns uses a conventional runtime written in C with ArrayFire. This goal has largely been achieved, so that the current BQN runtime uses a very small number of basic array operations currently provided by Javascript. However, basic operations may be added in the future for performance reasons.</p> <h2 id="code-style">Code style</h2> <p>Initially, the BQN compiler was written with a similar style to Co-dfns, primarily because I wanted to quickly have a working BQN implementation and reach parity with the Co-dfns compiler (not the runtime!). This style is now shifting to support several goals that Co-dfns doesn't have:</p> <ul> -<li>Better error reporting</li> +<li>Error reporting</li> <li>Teaching</li> <li>Multiple backends (bytecode, Wasm, machine code)</li> <li>Optimization</li> <li>Testing and debugging the compiler</li> </ul> -<p>So far the most notable difference is the addition of per-line comments. Aaron advocates the almost complete separation of code from comments (thesis) in addition to his very terse style as a general programming methodology. I think these choices are good for rapid development but not for maintainance and explaining the code to other developers. So far I have commented the tokenizer fully, but I consider the rest of the compiler too unstable and messy for these kinds of comments. The comments are meant as a supplement to long-form material on implementation, which I plan to write.</p> -<p>Co-dfns does not check for compilation errors. BQN should have complete error checking, and good error messages; the current error checking covers many cases but is still incomplete. I have found that such checking imposes a low cost on compilation speed (perhaps 25%), but a much greater cost on the compiler codebase size and legibility, and have yet to find a way to make these legibility costs manageable. The good news is that user-friendly error reporting has almost no performance cost for working programs, because error reporting code only needs to be run when compilation fails, and the only extra information that needs be stored to support it is source code locations of tokens. Maybe the array model can even give better error diagnosis than sequential compilers in some cases by examining the input as a whole to find the most likely cause of the mistake.</p> +<p>So far the most notable difference is the addition of per-line comments. Aaron advocates the almost complete separation of code from comments (thesis) in addition to his very terse style as a general programming methodology. I find that this practice makes it hard to connect the documentation to the code and is very slow in providing a summary or reminder of functionality that a comment might. However, I do plan to write long-form material providing the necessary context and explainations required to understand the compiler.</p> +<p>Co-dfns doesn't check for compilation errors, while BQN has complete error checking and good error messages, and includes source positions in compiler errors as well as in the compiled code for use in runtime errors. I have found that such checking imposes a low cost on compilation speed (perhaps 25%), but a greater cost on the compiler codebase size and legibility. Maintaining source locations is also a small expense relative to other compilation. Aside from that maintainance, more user-friendly error reporting has almost no performance cost for working programs, because error reporting code only needs to be run when compilation fails. This leaves room for potentially very sophisticated error analysis to attempt to track down the root cause of a compilation error, but I haven't yet done any work along these lines.</p> diff --git a/implementation/codfns.md b/implementation/codfns.md index 518c3535..659f655c 100644 --- a/implementation/codfns.md +++ b/implementation/codfns.md @@ -12,17 +12,17 @@ Co-dfns development has primarily been focused on the core compiler, and not par The core Co-dfns compiler is based on manipulating the syntax tree, which is mostly stored as parent and sibling vectors—that is, lists of indices of other nodes in the tree. BQN is less methodical, but generally treats the source tokens as a simple list. This list is reordered in various ways, allowing operations that behave like tree traversals to be performed with scans under the right ordering. This strategy allows BQN to be much stricter in the kinds of operations it uses. Co-dfns regularly uses `⍣≡` (repeat until convergence) for iteration and creates nested arrays with `⌸` (Key, like [Group](../doc/group.md)), but BQN has only a single instance of iteration to resolve quotes and comments, plus one complex but parallelizable scan for numeric literal processing, and only uses Group to extract identifiers and strings. This means that most primitives, if we count simple reductions and scans as single primitives, are executed a fixed number of times during execution, making complexity analysis even easier than in Co-dfns. -A goal for BQN was to not only write the compiler in BQN but to use BQN for the runtime as much as possible, while Co-dfns uses a conventional runtime written in C with ArrayFire. This goal has largely been achieved, so that the current BQN runtime uses a very small number of basic array operations currently provided by Javascript. However, more basic operations may be added in the future for performance reasons. +A goal for BQN was to not only write the compiler in BQN but to use BQN for the runtime as much as possible, while Co-dfns uses a conventional runtime written in C with ArrayFire. This goal has largely been achieved, so that the current BQN runtime uses a very small number of basic array operations currently provided by Javascript. However, basic operations may be added in the future for performance reasons. ## Code style Initially, the BQN compiler was written with a similar style to Co-dfns, primarily because I wanted to quickly have a working BQN implementation and reach parity with the Co-dfns compiler (not the runtime!). This style is now shifting to support several goals that Co-dfns doesn't have: -- Better error reporting +- Error reporting - Teaching - Multiple backends (bytecode, Wasm, machine code) - Optimization - Testing and debugging the compiler -So far the most notable difference is the addition of per-line comments. Aaron advocates the almost complete separation of code from comments (thesis) in addition to his very terse style as a general programming methodology. I think these choices are good for rapid development but not for maintainance and explaining the code to other developers. So far I have commented the tokenizer fully, but I consider the rest of the compiler too unstable and messy for these kinds of comments. The comments are meant as a supplement to long-form material on implementation, which I plan to write. +So far the most notable difference is the addition of per-line comments. Aaron advocates the almost complete separation of code from comments (thesis) in addition to his very terse style as a general programming methodology. I find that this practice makes it hard to connect the documentation to the code and is very slow in providing a summary or reminder of functionality that a comment might. However, I do plan to write long-form material providing the necessary context and explainations required to understand the compiler. -Co-dfns does not check for compilation errors. BQN should have complete error checking, and good error messages; the current error checking covers many cases but is still incomplete. I have found that such checking imposes a low cost on compilation speed (perhaps 25%), but a much greater cost on the compiler codebase size and legibility, and have yet to find a way to make these legibility costs manageable. The good news is that user-friendly error reporting has almost no performance cost for working programs, because error reporting code only needs to be run when compilation fails, and the only extra information that needs be stored to support it is source code locations of tokens. Maybe the array model can even give better error diagnosis than sequential compilers in some cases by examining the input as a whole to find the most likely cause of the mistake. +Co-dfns doesn't check for compilation errors, while BQN has complete error checking and good error messages, and includes source positions in compiler errors as well as in the compiled code for use in runtime errors. I have found that such checking imposes a low cost on compilation speed (perhaps 25%), but a greater cost on the compiler codebase size and legibility. Maintaining source locations is also a small expense relative to other compilation. Aside from that maintainance, more user-friendly error reporting has almost no performance cost for working programs, because error reporting code only needs to be run when compilation fails. This leaves room for potentially very sophisticated error analysis to attempt to track down the root cause of a compilation error, but I haven't yet done any work along these lines. |