From 831b556f20d1e6427d561c0fcb4bb0c6d592314a Mon Sep 17 00:00:00 2001
From: Marshall Lochbaum <mwlochbaum@gmail.com>
Date: Mon, 15 Nov 2021 14:16:25 -0500
Subject: Update and edit suggested VM testing sequence

---
 docs/implementation/vm.html | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

(limited to 'docs/implementation')

diff --git a/docs/implementation/vm.html b/docs/implementation/vm.html
index b6875235..0934ff7d 100644
--- a/docs/implementation/vm.html
+++ b/docs/implementation/vm.html
@@ -728,13 +728,14 @@
 <p>The compiler takes the runtime as <code><span class='Value'>𝕨</span></code> and source code as <code><span class='Value'>𝕩</span></code>. To evaluate BQN source code, convert it into a BQN string (rank-1 array of characters), pass this string and runtime to the compiler, and evaluate the result as bytecode. Results can be formatted with the formatter for use in a REPL, or used from the implementation language.</p>
 <p>Two formatter arguments <code><span class='Function'>Glyph</span></code> and <code><span class='Function'>FmtNum</span></code> are not part of the runtime. <code><span class='Function'>Glyph</span></code> assumes <code><span class='Value'>𝕩</span></code> is a primitive and returns the character (not string) that represents it, and <code><span class='Function'>FmtNum</span></code> assumes <code><span class='Value'>𝕩</span></code> is a number and returns a string representing it.</p>
 <h3 id="testing"><a class="header" href="#testing">Testing</a></h3>
-<p>I recommend roughly the following sequence of tests to get everything working smoothly. It can be very difficult to figure out where in a VM things went wrong, so it's important to work methodically and make sure each component is all right before moving to the next.</p>
+<p>I recommend roughly the following sequence of tests to get everything working smoothly. It can be very difficult to figure out where in a VM things went wrong, so it's important to work methodically and make sure each component is all right before moving to the next. In order to run test cases before the compiler runs, I strongly recommend building an automated system to compile the test to bytecode using an existing BQN implementation, and run it with the VM being developed.</p>
 <p>Because the compiler works almost entirely with lists of numbers, a correct fill implementation is not needed to run the compiler. Instead, you can define <code><span class='Function'>Fill</span></code> as <code><span class='Number'>0</span><span class='Modifier2'>⊘</span><span class='Function'>⊢</span></code> and <code><span class='Modifier2'>_fillBy_</span></code> as <code><span class='Brace'>{</span><span class='Function'>𝔽</span><span class='Brace'>}</span></code> to always use a fill element of 0.</p>
 <ul>
-<li>Test core runtime functions directly by calling them within the implementation language.</li>
 <li>Test the virtual machine with the output of <code><span class='Value'>src</span><span class='Function'>/</span><span class='Value'>cjs.bqn</span></code> on the primitive-less test expressions in <a href="https://github.com/mlochbaum/BQN/blob/master/implementation/../test/cases/bytecode.bqn">test/cases/bytecode.bqn</a>.</li>
-<li>Now test the self-hosted compiler by running it directly on small expressions.</li>
-<li>For a larger test, use <a href="https://github.com/mlochbaum/BQN/blob/master/implementation/../test/cases/prim.bqn">test/cases/prim.bqn</a>. The result should be an empty list <code><span class='Bracket'>⟨⟩</span></code> indicating no failed tests.</li>
+<li>There isn't currently a test suite for provided functions (although <a href="https://github.com/mlochbaum/BQN/blob/master/implementation/../test/cases/simple.bqn">test/cases/simple.bqn</a> has some suitable tests for arithmetic): your options are to write tests based on knowledge of these functions and primitive tests, or try to load the runtime and work backwards from any failures. The r1 runtime runs code to initialize some primitive lookup tables so failures are likely.</li>
+<li>Once the runtime is loaded, begin working through the tests in <a href="https://github.com/mlochbaum/BQN/blob/master/implementation/../test/cases/prim.bqn">test/cases/prim.bqn</a> with the full runtime but no self-hosted compiler.</li>
+<li>After primitive tests pass, try to load the compiler, and run it on a short expression. If it runs, you have a complete (not necessarily correct) system, and remaining tests can be run end-to-end!</li>
 <li>Now, if you haven't already, add a call to <code><span class='Function'>SetPrims</span></code>. Test for inferred properties: identity, under, and undo.</li>
 <li>If all tests pass you can probably compile the compiler.</li>
+<li>Headers and namespace support aren't required to support the runtime or compiler, but they can be tested as you add them with the header and namespace tests.</li>
 </ul>
-- 
cgit v1.2.3