Skip to content
Viv

2. Lexical elements

This chapter provides a technical overview of lexical structure in the Viv DSL, with a focus on how a source file is converted into a sequence of tokens.

Source encoding

Section titled “Source encoding”

A Viv source file is treated as UTF-8 text.

Comments

Section titled “Comments”
EBNF 
comment = "//" { any character except newline } (newline | EOF)

A comment is introduced with // and continues to the end of the line (or to the end of the file). As such, all of these are valid comments:

// A comment can take up its own line
    // Indentation doesn't matter
action greet:  // Trailing comments work too
    ...
// Final line can be a comment without a trailing newline

Comments cannot be placed inside strings and template strings, however:

"// this is not a comment"

Comments are ignored, in the sense that they act like whitespace.

Block comments are not supported.

Whitespace

Section titled “Whitespace”

Whitespace between tokens, including tabs and newlines, is ignored. This includes spaces (U+0020), horizontal tabs (U+0009), carriage returns (U+000D), and newlines (U+000A). In this respect, authors are free to format their Viv code according to taste.

Note that whitespace characters appearing inside strings and template strings are part of those tokens themselves, and thus are not skipped.

Tokens

Section titled “Tokens”

A Viv source file is divided into tokens, the smallest units of meaning recognized by the grammar. Tokens are separated by whitespace and comments, which are otherwise ignored (as explained above).

There are seven classes of tokens—identifiers, sigils, operators, punctuation, keywords, constants, and literals—each of which is described below.

Identifiers

Section titled “Identifiers”
EBNF 
identifier       = (* must not match reserved_keyword or reserved_internal_keyword *)
                   letter { letter | digit | "_" } { "-" ( letter | digit | "_" ) { letter | digit | "_" } } .
letter           = "A" … "Z" | "a" … "z" | "_" .
digit            = "0" … "9" .
reserved_keyword          = "elif" | "else" | "end" | "if" | "include" | "loop" .
reserved_internal_keyword = "__" identifier .

An identifier names an author-defined item, such as a construct, role, or variable.

Identifiers are case-sensitive and alphanumeric, with hyphens and underscores being permitted. They MUST begin with a letter or underscore, and a hyphen MUST be followed by at least one letter, digit, or underscore. For example, crush, _friend2, and plot-revenge are all valid identifiers, while 2friend, -revenge, and bad- are not.

An identifier MUST NOT be one of the following reserved words, which are required for syntactic disambiguation:

elif    else    end    if    include    loop

Note that reserved words are also case-sensitive (e.g., Include is not reserved).

Additionally, an identifier MUST NOT be prefixed with __, which marks a set of reserved fields that are used internally by the Viv runtime.

Sigils

Section titled “Sigils”

The Viv sigils are single-character tokens that prefix names to indicate their type or scope. Additionally, there is a related single-character suffix that we call a decorator.

These markers are described at length in a dedicated chapter, but the following table provides a summary:

SymbolNamePurpose
@Entity sigilMarks the entity type.
&Symbol sigilMarks the symbol type.
$Scratch-scope sigilMarks the scratch variable scope.
_Local-scope sigilMarks the local variable scope.
>Plan-phase sigilMarks a plan phase.
*Group-role decoratorMarks a group role.

Operators

Section titled “Operators”

The following operators, each detailed elsewhere, are treated as tokens:

CategoryOperators
Arithmetic+, -, *, /
Relational==, !=, <, <=, >, >=, in, knows, caused, triggered, preceded
Assignment=, +=, -=, *=, /=, append, remove
Logical!, &&, ||
Reference., ->, [, ], ?
Custom function call~
Probabilistic casting%, ~
Inscriptioninscribe, inspect

Note that certain symbols correspond to multiple operators, either because they function as distinct operators in different contexts (e.g., *, ~) or because they happen to occur in multi-symbol operators (e.g., - vis-à-vis -= and ->).

Punctuation

Section titled “Punctuation”

The following punctuation tokens structure the syntax: :, ;, ,, (, ), {, }, [, ], <, >.

Note that the square brackets here are distinct from the reference operators [ and ], because they appear in different contexts with distinct functions. Likewise, the angle brackets < and > serve as punctuation in sugared bindings, distinct from the relational operators of the same form.

Keywords

Section titled “Keywords”

As a DSL, Viv features a variety of special keywords to help authors specify concerns that are pertinent in the domain. These keywords can be broken into several categories, which are given below. Certain keywords appear in multiple categories, because they serve different purposes in different contexts (e.g., from).

Note that only a subset of these keywords are reserved words prohibited for identifiers, as explained above.

Unit headers

Section titled “Unit headers”

These keywords structure the headers that introduce the top-level constructs that make up a source file:

action    action-selector    from       include         pattern
plan      plan-selector      query      reserved        template
trope     with

Action fields

Section titled “Action fields”

Additional keywords mark and structure the fields within an action definition:

associations   conditions   default    effects      embargoes   for
gloss          importance   join       reactions    report      roles
saliences      scratch      tags

Role fields

Section titled “Role fields”

These keywords structure the fields within a role definition:

as    from    is    n    renames    spawn

Role labels

Section titled “Role labels”

The role labels are keywords:

action      anywhere    bystander    character    initiator
item        location    partner      precast      recipient
spawn       symbol

Reaction fields

Section titled “Reaction fields”

The fields of a reaction are keywords:

abandon    location    priority    queue    repeat
time       urgent      with

Temporal constraints

Section titled “Temporal constraints”

A number of keywords allow authors to place temporal constraints on constructs like reactions and queries:

after      and        before     between    from


one        two        three      four       five
six        seven      eight      nine       ten
eleven     twelve


minute     minutes    hour       hours      day        days
week       weeks      month      months     year       years


am         pm

Embargo fields

Section titled “Embargo fields”

The fields of an embargo are keywords:

location    roles    time

Set predicates

Section titled “Set predicates”

Keywords are used in set predicates:

all    any    exactly    none

Query fields

Section titled “Query fields”

Keywords are used in query definitions:

action         active         ancestors      associations    bystanders
conditions     descendants    importance     initiator       location
partners       present        recipients     roles           salience
tags           time

Plan instructions

Section titled “Plan instructions”

Keywords are used in plan instructions:

all         any         close       end
timeout     untracked   until      wait

Selector fields

Section titled “Selector fields”

Keywords are used in selector definitions:

conditions    randomly    roles
selector      target

The following two-word keyword sequences also appear in selectors:

in order      with weights

Bindings

Section titled “Bindings”

Keywords are used in precast bindings:

none    partial    with

Flow and control

Section titled “Flow and control”

Keywords are used to specify local variables, conditionals, and loops:

as    else    elif    end    if    loop

Domain-specific expressions

Section titled “Domain-specific expressions”

Keywords support domain-specific expressions:

fit     fits    over    search    sift

Constants

Section titled “Constants”

In Viv, a constant is a fixed, named value that denotes a predefined option for a given field.

Temporal constants

Section titled “Temporal constants”

Constants used to anchor temporal constraints:

action    ago    hearing    now

Embargo constants

Section titled “Embargo constants”

Constants used to parameterize embargoes :

anywhere    forever    here

Plan constants

Section titled “Plan constants”

Constants used as plan instructions:

advance    fail    succeed

Search constants

Section titled “Search constants”

Constants used to parameterize search domains:

chronicle    inherit

Literals

Section titled “Literals”
EBNF 
literal = enum | string | number | boolean | null .

What follows are the literal forms that Viv supports.

Booleans

Section titled “Booleans”

The boolean literals are true and false.

Null

Section titled “Null”

The null literal is null.

Numbers

Section titled “Numbers”
EBNF 
number  = [ sign ] digits [ "." digits ] .
sign    = "+" | "-" .
digits  = digit { digit } .
digit   = "0" … "9" .

In terms of number literals, Viv supports both decimal integers (0, 77, -31, +88) and decimal fractions (3.14, -0.5, +99.9).

Numbers MUST have at least one digit before the decimal point in fractions—e.g., 0.5 is valid, while .5 is not. A single leading + or - is allowed.

Only base-10 literals are supported; there are no hexadecimal, octal, or binary forms. Exponent notation and digit separators are not supported.

Strings

Section titled “Strings”
EBNF 
string_literal = '"' { any character except '"' or newline } '"'
               | "'" { any character except "'" or newline } "'" .

Viv string literals may be single-quoted ('Hello!') or double-quoted ("Goodbye..."). A string ends at the matching quote, and escaping is not supported.

Template strings

Section titled “Template strings”
EBNF 
template_string = '"' ( template_gap | template_char )+ '"'
                | "'" ( template_gap | template_char )+ "'" .
template_gap    = "{" expression "}" | reference .
template_char   = (* any character except '"', '{', '}', and sigil characters *) .

Viv template strings interpolate expressions that are enclosed in curly brackets:

"{@bully.name} insults {@target.name} by calling them a {~getRandomInsult()}."

Template strings can also interpolate references without use of brackets:

"@bully insults @targets* at @this.location->address."

Note that the preceding example is just syntactic sugar for this:

"{@bully} insults {@targets*} at {@this.location->address}."

A string is parsed as a template string (rather than a plain string literal) when it contains a template gap: an expression contained in curly braces, or else a bare reference beginning with a sigil. For more information on how template strings will be rendered by a runtime, see the section on rendering template strings.

Lists

Section titled “Lists”
EBNF 
list = "[" [ expression { "," expression } ] "]" .

A Viv list literal sequences zero or more ordered expressions, the evaluations of which do not have to match in type.

Examples:

[]
[&thing]
[77, "lol", @foo, ~getSomething()]

Objects

Section titled “Objects”
EBNF 
object         = "{" [ key_value_pair { "," key_value_pair } ] "}" .
key_value_pair = key ":" expression .
key            = string | bare_key .
bare_key       = letter { letter | digit | "_" | "-" } .

Viv object literals are key–value pairs in a JavaScript-like notation, where keys may either be enclosed in quotes or bare. A value is any expression.

Examples:

{}
{"foo": 77}
{ a: ~getSomething(), "b": @foo.name, zzz: "sleep" }

Enums

Section titled “Enums”
EBNF 
enum = [ "+" | "-" ] "#" identifier .

A Viv enum literal is an identifier preceded by #, optionally with a leading + or - sign to mark the sign of a numeric enum. Enum literals can be used anywhere literal values are allowed.

Examples:

#BORING
-#SMALL
+#BIG

Tokenization example

Section titled “Tokenization example”

Consider the following source file:

// Say hey!
action greet:
    gloss: "Greets a friend"
    roles:
        @greeter:  // Here's a trailing comment
            as: initiator
        @friend:
            as: recipient

In Viv, this file would be tokenized as follows:

action
greet
:
gloss
:
"Greets a friend"
roles
:
@
greeter
:
as
:
initiator
@
friend
:
as
:
recipient