aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMarshall Lochbaum <mwlochbaum@gmail.com>2021-01-30 16:40:40 -0500
committerMarshall Lochbaum <mwlochbaum@gmail.com>2021-01-30 16:40:40 -0500
commitafadc45f29d34de5b4bce0b1ebe751436939a77b (patch)
treeb031c85c5bf40c79d45ab41ed76d0af37a869a97
parent4fbf17eb5e9dcb3aac891832eabf63c715df4feb (diff)
Prevent names like _99 in the spec
-rw-r--r--docs/spec/token.html2
-rw-r--r--spec/token.md2
2 files changed, 2 insertions, 2 deletions
diff --git a/docs/spec/token.html b/docs/spec/token.html
index b75d088f..0374e0b8 100644
--- a/docs/spec/token.html
+++ b/docs/spec/token.html
@@ -9,7 +9,7 @@
<p>BQN source code should be considered as a series of unicode code points, which we refer to as &quot;characters&quot;. The separator between lines in a file is considered to be a single character, newline, even though some operating systems such as Windows typically represent it with a two-character CRLF sequence. Implementers should note that not all languages treat unicode code points as atomic, as exposing the UTF-8 or UTF-16 representation instead is common. For a language such as JavaScript that uses UTF-16, the double-struck characters <code><span class='Value'>𝕨</span><span class='Function'>𝕎</span><span class='Value'>𝕩</span><span class='Function'>𝕏</span><span class='Value'>𝕗</span><span class='Function'>𝔽</span><span class='Value'>𝕘</span><span class='Function'>𝔾</span></code> are represented as two 16-bit surrogate characters, but BQN treats them as a single unit.</p>
<p>A BQN <em>character literal</em> consists of a single character between single quotes, such as <code><span class='String'>'a'</span></code>, and a <em>string literal</em> consists of any number of characters between double quotes, such as <code><span class='String'>&quot;&quot;</span></code> or <code><span class='String'>&quot;abc&quot;</span></code>. Character and string literals take precedence with comments over other tokenization rules, so that <code><span class='Comment'>#</span></code> between quotes does not start a comment and whitespace between quotes is not removed, but a quote within a comment does not start a character literal. Almost any character can be included directly in a character or string literal without escaping. The only exception is the double quote character <code><span class='String'>&quot;</span></code>, which must be written twice to include it in a string, as otherwise it would end the string instead. Character literals require no escaping at all, as the length is fixed. In particular, literals for the double and single quote characters are written <code><span class='String'>'''</span></code> and <code><span class='String'>'&quot;'</span></code>, while length-1 strings containing these characters are <code><span class='String'>&quot;'&quot;</span></code> and <code><span class='String'>&quot;&quot;&quot;&quot;</span></code>.</p>
<p>A comment consists of the hash character <code><span class='Comment'>#</span></code> and any following text until (not including) the next newline character. The initial <code><span class='Comment'>#</span></code> must not be part of a string literal started earlier. Comments are ignored entirely and do not form tokens.</p>
-<p>Identifiers and numeric literals share the same token formation rule. These tokens are formed from the <em>numeric characters</em> <code><span class='Number'>¯∞π.0123456789</span></code> and <em>alphabetic characters</em> <code><span class='Modifier'>_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ</span></code> and the oddball <code><span class='Value'>𝕣</span></code>. Any sequence of these characters adjacent to each other forms a single token, which is a <em>numeric literal</em> if it begins with a numeric character and an <em>identifier</em> if it begins with an alphabetic character. Numeric literals are also subject to <a href="literal.html">numeric literal rules</a>, which specify which numeric literals are valid and which numbers they represent. If the token contains <code><span class='Value'>𝕣</span></code> it must be either <code><span class='Value'>𝕣</span></code>, <code><span class='Modifier'>_𝕣</span></code>, or <code><span class='Modifier2'>_𝕣_</span></code> and is considered a special name (see below). As the value taken by this identifier can only be a modifier, the uppercase character <code><span class='Value'>ℝ</span></code> is not allowed.</p>
+<p>Identifiers and numeric literals share the same token formation rule. These tokens are formed from the <em>numeric characters</em> <code><span class='Number'>¯∞π.0123456789</span></code> and <em>alphabetic characters</em> <code><span class='Modifier'>_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ</span></code> and the oddball <code><span class='Value'>𝕣</span></code>. Any sequence of these characters adjacent to each other forms a single token, which is a <em>numeric literal</em> if it begins with a numeric character and an <em>identifier</em> if it begins with an alphabetic character. If a token begins with an underscore then its first non-underscore character must be alphabetic: for example, <code><span class='Modifier'>_99</span></code> is not a valid token. Numeric literals are also subject to <a href="literal.html">numeric literal rules</a>, which specify which numeric literals are valid and which numbers they represent. If the token contains <code><span class='Value'>𝕣</span></code> it must be either <code><span class='Value'>𝕣</span></code>, <code><span class='Modifier'>_𝕣</span></code>, or <code><span class='Modifier2'>_𝕣_</span></code> and is considered a special name (see below). As the value taken by this identifier can only be a modifier, the uppercase character <code><span class='Value'>ℝ</span></code> is not allowed.</p>
<p>Following this step, the whitespace characters space and tab are ignored, and do not form tokens. Only these whitespace characters, and the newline character, which does form a token, are allowed.</p>
<p>Otherwise, a single character forms a token. Only the specified set of characters can be used; others result in an error. The classes of characters are given below.</p>
<table>
diff --git a/spec/token.md b/spec/token.md
index 93dd1dfa..91608c47 100644
--- a/spec/token.md
+++ b/spec/token.md
@@ -10,7 +10,7 @@ A BQN *character literal* consists of a single character between single quotes,
A comment consists of the hash character `#` and any following text until (not including) the next newline character. The initial `#` must not be part of a string literal started earlier. Comments are ignored entirely and do not form tokens.
-Identifiers and numeric literals share the same token formation rule. These tokens are formed from the *numeric characters* `¯∞π.0123456789` and *alphabetic characters* `_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ` and the oddball `𝕣`. Any sequence of these characters adjacent to each other forms a single token, which is a *numeric literal* if it begins with a numeric character and an *identifier* if it begins with an alphabetic character. Numeric literals are also subject to [numeric literal rules](literal.md), which specify which numeric literals are valid and which numbers they represent. If the token contains `𝕣` it must be either `𝕣`, `_𝕣`, or `_𝕣_` and is considered a special name (see below). As the value taken by this identifier can only be a modifier, the uppercase character `ℝ` is not allowed.
+Identifiers and numeric literals share the same token formation rule. These tokens are formed from the *numeric characters* `¯∞π.0123456789` and *alphabetic characters* `_abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ` and the oddball `𝕣`. Any sequence of these characters adjacent to each other forms a single token, which is a *numeric literal* if it begins with a numeric character and an *identifier* if it begins with an alphabetic character. If a token begins with an underscore then its first non-underscore character must be alphabetic: for example, `_99` is not a valid token. Numeric literals are also subject to [numeric literal rules](literal.md), which specify which numeric literals are valid and which numbers they represent. If the token contains `𝕣` it must be either `𝕣`, `_𝕣`, or `_𝕣_` and is considered a special name (see below). As the value taken by this identifier can only be a modifier, the uppercase character `ℝ` is not allowed.
Following this step, the whitespace characters space and tab are ignored, and do not form tokens. Only these whitespace characters, and the newline character, which does form a token, are allowed.