Morel language reference

This document describes the grammar of Morel (constants, identifiers, expressions, patterns, types, declarations), and then lists its built-in operators, types, functions. Properties affect the execution strategy and the behavior of the shell. The shell is Morel's interactive read-eval-print loop.

Query expressions (from, exists, forall) are described in more detail in the query reference.

Grammar

This reference is based on Andreas Rossberg's grammar for Standard ML. While the files have a different notation, they are similar enough to the two languages.

Differences between Morel and SML

Morel aims to be compatible with Standard ML. It extends Standard ML in areas related to relational operations. Some of the features in Standard ML are missing, just because they take effort to build. Contributions are welcome!

In Morel but not Standard ML:

• Queries (expressions starting with exists, forall or from) with compute, distinct, except, group, intersect, into, join, order, require, skip, take, through, union, unorder, where, yield, yieldAll steps and in and of keywords
• elem, implies, notelem binary operators
• current, elements, ordinal nilary operators
• typeof type operator
• type_string expression operator
• lab = is optional in exprow
• record.lab as an alternative to #lab record; for tuples, tuple.1, tuple.2 etc. as an alternative to #1 tuple, #2 tuple
• postfix method-call syntax exp.f () and exp.f arg, where f is a function whose first parameter is named self
• identifiers and type names may be quoted (for example, `an identifier`)
• with functional update for record values (from OCaml)
• overloaded functions may be declared using over and inst
• attributes ([@attr] / [@@attr] / [@@@attr]) based on OCaml
• (*) line comments (syntax as SML/NJ and MLton)
• (** ... *) doc comments (from OCaml)

In Standard ML but not in Morel:

• word constant
• longid identifier
• references (ref and operators ! and :=)
• exceptions: handle and user-defined exception declarations (raise and built-in exceptions are supported)
• while loop
• data type replication (type)
• withtype in datatype declaration
• abstract type (abstype)
• structures (structure)
• signature refinement (where type)
• signature sharing constraints
• local declarations (local)
• operator declarations (nonfix, infix, infixr)
• open
• before and o operators

Constants

con → int                       integer
    | float                     floating point
    | char                      character
    | string                    string
int → [~]num                    decimal
    | [~]0xhex                  hexadecimal
float → [~]num.num              floating point
    | [~]num[.num]e[~]num
                                scientific
char → #"ascii"                 character
string → "ascii*"               string
num → digit digit*              number
hex → (digit | letter) (digit | letter)*
                                hexadecimal number (letters
                                may only be in the range A-F)
ascii → ...                     single non-" ASCII character
                                or \-headed escape sequence

Identifiers

id →  letter (letter | digit | ''' | _)*
                                alphanumeric
    | symbol symbol*            symbolic (not allowed for type
                                variables or module language
                                identifiers)
symbol → !
    | %
    | &
    | $
    | #
    | +
    | -
    | /
    | :
    | <
    | =
    | >
    | ?
    | @
    | \
    | ~
    | `
    | ^
    | '|'
    | '*'
var → '''(letter | digit | ''' | _)*
                                unconstrained
      ''''(letter | digit | ''' | _⟩*
                                equality
lab → id                        identifier
      num                       number (may not start with 0)

Expressions

exp → con                       constant
    | [ op ] id                 value or constructor identifier
    | exp1 exp2                 application
    | exp1 id exp2              infix application
    | '(' exp ')'               parentheses
    | '(' exp1 , ... , expn ')' tuple (n ≠ 1)
    | { [ exprow ] }            record
    | #lab                      record selector
    | '[' exp1 , ... , expn ']' list (n ≥ 0)
    | '(' exp1 ; ... ; expn ')' sequence (n ≥ 2)
    | let dec in exp1 ; ... ; expn end
                                local declaration (n ≥ 1)
    | exp . lab ()              postfix call (no argument)
    | exp1 . lab exp2            postfix call (with argument)
    | exp : type                type annotation
    | exp1 andalso exp2         conjunction
    | exp1 orelse exp2          disjunction
    | if exp1 then exp2 else exp3
                                conditional
    | case exp of match         case analysis
    | raise exp                 exception raising
    | type_string exp           type of exp as a string (exp is not evaluated)
    | fn match                  function
    | current                   current element (only valid in a query step)
    | elements                  elements of current group (only valid in compute)
    | ordinal                   element ordinal (only valid in a query step)
    | exp1 over exp2            aggregate (only valid in compute)
    | from [ scan1 , ... , scans ] step1 ... stept [ terminalStep ]
                                relational expression (s ≥ 0, t ≥ 0)
    | exists [ scan1 , ... , scans ] step1 ... stept
                                existential quantification (s ≥ 0, t ≥ 0)
    | forall [ scan1 , ... , scans ] step1 ... stept require exp
                                universal quantification (s ≥ 0, t ≥ 0)
    | exp expAttr1 ... expAttrn
                                attributed expression (n ≥ 1)
exprow → [ exp with ] exprowItem [, exprowItem ]*
                                expression row
exprowItem → [ lab = ] exp
match → matchItem [ '|' matchItem ]*
                                match
matchItem → pat => exp
scan → pat in exp [ on exp ]    iteration
    | pat = exp                 single iteration
    | val                       unbounded variable
step → distinct                 distinct step
    | except [ distinct ] exp1 , ... , expe
                                except step (e ≥ 1)
    | group exp1 [ compute exp2 ]
                                group step
    | intersect [ distinct ] exp1 , ... , expi
                                intersect step (i ≥ 1)
    | [ left | right | full ] join scan1 , ... , scans
                                join step (s ≥ 1)
    | order exp                 order step
    | skip exp                  skip step
    | take exp                  take step
    | through pat in exp        through step
    | union [ distinct ] exp1 , ... , expu
                                union step (u ≥ 1)
    | where exp                 filter step
    | yield exp                 yield step
    | yieldAll exp              yieldAll step
terminalStep → into exp         into step
    | compute exp               compute step
groupKey → [ id = ] exp
agg → [ id = ] exp [ of exp ]

Patterns

pat → con                       constant
    | _                         wildcard
    | [ op ] id                 variable
    | [ op ] id [ pat ]         construction
    | pat1 id pat2              infix construction
    | '(' pat ')'               parentheses
    | '(' pat1 , ... , patn ')' tuple (n ≠ 1)
    | { [ patrow ] }            record
    | '[' pat1 , ... , patn ']' list (n ≥ 0)
    | pat : type                type annotation
    | id as pat                 layered
patrow → '...'                  wildcard
    | lab = pat [, patrow]      pattern
    | id [, patrow]             label as variable

Types

typ → var                       variable
    | [ typ ] id                constructor
    | '(' typ [, typ ]* ')' id  constructor
    | '(' typ ')'               parentheses
    | typ1 -> typ2              function
    | typ1 '*' ... '*' typn     tuple (n ≥ 2)
    | { [ typrow ] }            record
    | typeof exp                expression type
    | typ expAttr1 ... expAttrn
                                attributed type (n ≥ 1)
typrow → lab : typ [, typrow]   type row

Declarations

dec → vals valbind              value
    | fun funbind               function
    | type typbind              type
    | datatype datbind          data type
    | signature sigbind         signature
    | over id                   overloaded name
    | dec declAttr1 ... declAttrn
                                attributed declaration (n ≥ 1)
    | floatingAttr              floating attribute
    | empty
    | dec1 [;] dec2             sequence
valbind → pat = exp [ and valbind ]*
                                destructuring
    | rec valbind               recursive
    | inst valbind              overload instance
funbind → funmatch [ and funmatch ]*
                                clausal function
funmatch → funmatchItem [ '|' funmatchItem ]*
funmatchItem → [ op ] id pat1 ... patn [ : type ] = exp
                                nonfix (n ≥ 1)
    | pat1 id pat2 [ : type ] = exp
                                infix
    | '(' pat1 id pat2 ')' pat'1 ... pat'n [ : type ] = exp
                                infix (n ≥ 0)
typbind → [ vars ] id = typ [ and typbind ]*
                                abbreviation
datbind → datbindItem [ and datbindItem ]*
                                data type
datbindItem → [ vars ] id = conbind
conbind → conbindItem [ '|' conbindItem ]*
                                data constructor
conbindItem → id [ of typ ]
vals → val
    | '(' val [, val]* ')'
vars → var
    | '(' var [, var]* ')'

Attributes

expAttr → '[@' attrName [ payload ] ']'
                                expression / type attribute
declAttr → '[@@' attrName [ payload ] ']'
                                declaration attribute
floatingAttr → '[@@@' attrName [ payload ] ']'
                                floating attribute
attrName → id [ . id ]*
                                dotted attribute name
payload → exp                   expression payload
    | : type                    type payload

Modules

sigbind → id = sig spec end [ and sigbind ]*
                                signature
spec → val valdesc              value
    | type typdesc              abstract type
    | type typbind              type abbreviation
    | datatype datdesc          data type
    | exception exndesc         exception
    | empty
    | spec1 [;] spec2           sequence
valdesc → id : typ [ and valdesc ]*
                                value specification
typdesc → [ vars ] id [ and typdesc ]*
                                type specification
datdesc → datdescItem [ and datdescItem ]*
                                datatype specification
datdescItem → [ vars ] id = conbind
exndesc → id [ of typ ] [ and exndesc ]*
                                exception specification

A signature defines an interface that specifies types, values, datatypes, and exceptions without providing implementations. Signatures are used to document module interfaces and, in future versions of Morel, will be used to constrain structure implementations.

Signature declarations appear at the top level (see grammar in Declarations).

Specifications

A signature body contains specifications that describe the interface:

Value specifications declare the type of a value without defining it:

val empty : 'a stack
val push : 'a * 'a stack -> 'a stack

Type specifications can be abstract (no definition) or concrete (type alias):

type 'a stack              (* abstract type *)
type point = real * real   (* concrete type alias *)
type ('k, 'v) map          (* abstract with multiple params *)

Datatype specifications describe algebraic datatypes:

datatype 'a tree = Leaf | Node of 'a * 'a tree * 'a tree

Exception specifications declare exceptions:

exception Empty                  (* exception without payload *)
exception QueueError of string   (* exception with payload *)

Examples

A simple signature with abstract type and value specifications:

signature STACK =
sig
  type 'a stack
  exception Empty
  val empty : 'a stack
  val isEmpty : 'a stack -> bool
  val push : 'a * 'a stack -> 'a stack
  val pop : 'a stack -> 'a stack
  val top : 'a stack -> 'a
end

Multiple signatures declared together using and:

signature EQ =
sig
  type t
  val eq : t * t -> bool
end
and ORD =
sig
  type t
  val lt : t * t -> bool
  val le : t * t -> bool
end

Current Limitations

The current implementation supports parsing and pretty-printing signatures but does not yet support:

• Structure declarations that implement signatures
• Signature refinement (where type)
• Signature sharing constraints
• Signature inclusion (include)

These features may be added in future versions.

Notation

This grammar uses the following notation:

Syntax	Meaning
symbol	Grammar symbol (e.g. con)
keyword	Morel keyword (e.g. if) and symbol (e.g. ~, "(")
[ term ]	Option: term may occur 0 or 1 times
[ term1 | term2 ]	Alternative: term1 may occur, or term2 may occur, or neither
term*	Repetition: term may occur 0 or more times
's'	Quotation: Symbols used in the grammar — ( ) [ ] | * ... — are quoted when they appear in Morel language

Built-in operators

Operator	Precedence	Meaning
*	infix 7	Multiplication
/	infix 7	Division
div	infix 7	Integer division
mod	infix 7	Modulo
+	infix 6	Plus
-	infix 6	Minus
^	infix 6	String concatenate
~	prefix 6	Negate
::	infixr 5	List cons
@	infixr 5	List append
<=	infix 4	Less than or equal
<	infix 4	Less than
>=	infix 4	Greater than or equal
>	infix 4	Greater than
=	infix 4	Equal
<>	infix 4	Not equal
elem	infix 4	Member of list
notelem	infix 4	Not member of list
:=	infix 3	Assign
o	infix 3	Compose
andalso	infix 2	Logical and
orelse	infix 1	Logical or
implies	infix 0	Logical implication

abs is a built-in function (not an operator, because it uses function syntax rather than prefix or infix syntax). It is overloaded: its type is int -> int when applied to an int argument, and real -> real when applied to a real argument. It is equivalent to Int.abs and Real.abs respectively.

Built-in types

Primitive: bool, char, int, real, string, unit

Datatype:

• datatype 'a descending = DESC of 'a (in structure Relational)
• datatype ('l, 'r) either = INL of 'l | INR of 'r (in structure Either)
• datatype 'a list = nil | :: of 'a * 'a list (in structure List)
• datatype 'a option = NONE | SOME of 'a (in structure Option)
• datatype 'a order = LESS | EQUAL | GREATER (in structure General)

Eqtype:

• eqtype 'a bag = 'a bag (in structure Bag)
• eqtype 'a vector = 'a vector (in structure Vector)

Exception:

• Bind (in structure General)
• Chr (in structure General)
• Div (in structure General)
• Domain (in structure General)
• Empty (in structure List)
• Error (in structure Interact)
• Option (in structure Option)
• Overflow (in structure Option)
• Size (in structure General)
• Subscript (in structure General)
• Unordered (in structure IEEEReal)

Structures

Structure	Description
Bag	Unordered collection of elements with duplicates. bag, nil, null, fromList, toList, length, @, hd, tl, getItem, take, drop, concat, app, map, mapPartial, find, filter, partition, fold, exists, all, tabulate, nth, only
Bool	Boolean values and operations. bool, not, toString, fromString, andalso, orelse, implies, =, <>, <, >
Char	Character values and operations. char, string, minChar, maxChar, maxOrd, ord, chr, succ, pred, compare, <, <=, >, >=, =, <>, contains, notContains, isAscii, toLower, toUpper, isAlpha, isAlphaNum, isCntrl, isDigit, isGraph, isHexDigit, isOctDigit, isLower, isPrint, isSpace, isPunct, isUpper, toString, fromString, fromInt, toCString, fromCString, scan
Datalog	Datalog query interface. execute, translate, validate
Date	Calendar date and time values. date, month, weekday, Date, compare, day, fmt, fromString, fromTimeLocal, fromTimeUniv, hour, isDst, localOffset, minute, second, toString, toTime, weekDay, year, yearDay
Either	Values that are one of two types. either, isLeft, isRight, asLeft, asRight, map, mapLeft, mapRight, app, appLeft, appRight, fold, proj, partition
Fn	Higher-order function combinators. id, const, apply, o, curry, uncurry, flip, repeat, equal, notEqual
General	Basic types, exceptions, and utility functions. unit, exn, order, Bind, Match, Chr, Div, Domain, Fail, Overflow, Size, Span, Subscript, exnName, exnMessage, o, before, ignore, !
IEEEReal
Int	Fixed-precision integer operations. int, toLarge, fromLarge, toInt, fromInt, precision, minInt, maxInt, +, -, *, div, mod, quot, rem, compare, <, <=, >, >=, ~, abs, min, max, sign, sameSign, fmt, toString, fromString
IntInf	Arbitrary-precision integer operations.
Interact	Interactive session utilities. use, useSilently
List	Polymorphic singly-linked lists. list, Empty, null, length, @, hd, tl, last, getItem, nth, only, take, drop, rev, concat, revAppend, app, map, mapPartial, find, filter, partition, foldl, foldr, exists, all, tabulate, collate, except, intersect, mapi
ListPair	Operations on pairs of lists. UnequalLengths, zip, unzip, map, app, all, exists, foldr, foldl, allEq, zipEq, mapEq, appEq, foldrEq, foldlEq
Math	Mathematical functions for real numbers. real, pi, e, sqrt, sin, cos, tan, asin, acos, atan, atan2, exp, pow, ln, log10, sinh, cosh, tanh
Option	Optional values. option, Option, getOpt, isSome, valOf, filter, join, app, map, mapPartial, compose, composePartial
PP	Wadler-Leijen pretty-printer. doc, empty, line, lineBreak, softLine, softBreak, hardLine, text, beside, nest, group, align, hang, indent, hsep, vsep, sep, hcat, vcat, cat, fillSep, fillCat, punctuate, encloseSep, parens, braces, brackets, render
Range	Operations on ranges of ordered values. continuous_set, discrete_set, range, contains, toBag, toList, continuousSetOf, discreteSetOf, flatten, ranges, complement
Real	Floating-point number operations. real, radix, precision, maxFinite, minPos, minNormalPos, posInf, negInf, +, -, *, /, rem, ~, abs, min, max, sign, signBit, sameSign, copySign, compare, <, <=, >, >=, =, <>, unordered, isFinite, isNan, isNormal, toManExp, fromManExp, split, realMod, checkFloat, realFloor, realCeil, realTrunc, realRound, floor, ceil, trunc, round, fromInt, fmt, toString, fromString
Relational	Relational algebra operations for Morel queries. descending, compare, count, empty, iterate, max, min, nonEmpty, only, sum
String	String operations. string, char, maxSize, size, sub, extract, substring, ^, concat, concatWith, str, implode, explode, map, translate, tokens, fields, isPrefix, isSubstring, isSuffix, compare, collate, <, <=, >, >=, =, <>
StringCvt	String conversion utilities and types. radix, realfmt, padLeft, padRight
Sys	System interface utilities. clearEnv, colorSchemes, deduceColorScheme, env, file, parseTree, plan, planEx, set, show, showAll, unset
Time	Time values and operations. time, Time, zeroTime, fromReal, toReal, toSeconds, toMilliseconds, toMicroseconds, toNanoseconds, fromSeconds, fromMilliseconds, fromMicroseconds, fromNanoseconds, +, -, compare, <, <=, >, >=, now, fmt, toString, fromString
Variant	Dynamically-typed variant values. variant, parse, print
Vector	Immutable fixed-length arrays. vector, maxLen, fromList, tabulate, length, sub, update, concat, appi, app, mapi, map, foldli, foldri, foldl, foldr, findi, find, exists, all, collate
Word	Unsigned-integer (word) operations. word, wordSize, toLarge, toLargeX, toLargeWord, toLargeWordX, fromLarge, fromLargeWord, toLargeInt, toLargeIntX, fromLargeInt, toInt, toIntX, fromInt, andb, orb, xorb, notb, <<, >>, ~>>, +, -, *, div, mod, compare, <, <=, >, >=, ~, min, max, fmt, toString, fromString

Properties

Each property is set using the function Sys.set (name, value), displayed using Sys.show name, and unset using Sys.unset name. Sys.showAll () shows all properties and their values.

Name	Type	Default	Description
banner	string	Morel version ...	Startup banner message displayed when launching the Morel shell.
colorScheme	string	null	Color scheme for syntax highlighting in the shell: a built-in scheme ('dark', 'light' or 'none'), or a user-defined scheme. If unset, the scheme is deduced from the environment.
directory	file		Path of the directory that the 'file' variable maps to in this connection.
excludeStructures	string	^Test$	Regular expression that controls which built-in structures are excluded from the environment.
hybrid	bool	false	Whether to try to create a hybrid execution plan that uses Apache Calcite relational algebra.
inlinePassCount	int	5	Maximum number of inlining passes.
lineWidth	int	79	When printing, the length at which lines are wrapped.
matchCoverageEnabled	bool	true	Whether to check whether patterns are exhaustive and/or redundant.
matchStrict	bool	false	Whether the script-test harness compares output verbatim, rather than modulo whitespace and bag-element order.
now	string	null	Overrides the current time. Value is an ISO-8601 string (e.g. '2024-01-01T00:00:00Z'). If not set, the system clock is used.
optionalInt	int	null	For testing.
output	enum	classic	How values should be formatted. "classic" (the default) prints values in a compact nested format; "tabular" prints values in a table if their type is a list of records.
printDepth	int	5	When printing, the depth of nesting of recursive data structure at which ellipsis begins.
printLength	int	12	When printing, the length of lists at which ellipsis begins.
productName	string	morel-java	Name of the Morel product.
productVersion	string	0.8.0	Current version of Morel.
relationalize	bool	false	Whether to convert to relational algebra.
scriptDirectory	file		Path of the directory where the 'use' command looks for scripts. When running a script, it is generally set to the directory that contains the script.
stringDepth	int	70	When printing, the length of strings at which ellipsis begins.
stringFold	int	null	In tabular mode, the column width at which long strings are folded across multiple lines. If not set, folding is disabled. Legal values are 1 or greater.
terminalBackground	string	null	The terminal's background color, of the form 'rgb:RRRR/GGGG/BBBB'. Set by the shell at startup; used to deduce the color scheme when 'colorScheme' is unset.
timeZone	string	null	Overrides the local timezone. Value is a timezone ID (e.g. 'UTC' or 'America/New_York'). If not set, the JVM default timezone is used.

Apache Calcite adapter

When the hybrid property is set, Morel tries to translate each query into Apache Calcite relational algebra, falling back to its own evaluator for any part it cannot translate. Limitations include:

• min and max cannot be pushed down for word values or composite values (tuples, records, lists, and datatypes such as option).

The shell

Morel includes an interactive shell (a read-eval-print loop, or REPL). Launch it by running morel with no arguments:

$ ./morel
morel-java version 0.8.0 (java version "25", JLine terminal, xterm-256color)
- "Hello, world!";
val it = "Hello, world!" : string
-

At the - prompt, type an expression or declaration followed by a semicolon, and Morel evaluates it and prints the result. A statement may span several lines; while it is incomplete, the prompt changes to =. Type Ctrl-D (or the exit command) to quit.

Command history

The shell remembers the commands you type, so that you can recall and edit earlier commands using the up and down arrow keys. History is saved between sessions in the file ~/.morel/history, and is reloaded when the shell starts. If the ~/.morel directory does not exist, the shell creates it.

Each line in the history file consists of a Unix timestamp, a colon, and the text of a command. If the file cannot be created — for example, if the home directory is not writable — the shell prints a warning and continues without saving history for that session.

Color schemes

In an interactive terminal, the shell highlights the code you type, coloring each token according to its category: keyword, symbol, numeric, constant, string, comment, and type variable. (Plain identifiers are left in the terminal's default color.)

The color scheme is chosen by the colorScheme property, whose value is one of:

• dark — colors tuned for a dark terminal background;
• light — colors tuned for a light terminal background;
• none — no highlighting;
• the name of a user-defined scheme.

If the property is unset (the default), the scheme is deduced from the environment by Sys.deduceColorScheme: none if the terminal does not support color (if standard output is not a terminal, NO_COLOR is set, or TERM is dumb), otherwise light or dark according to the terminal's background. The shell determines the background by asking the terminal for its background color (the OSC 11 escape sequence); outside the shell, or if the terminal does not answer, it falls back to the COLORFGBG environment variable, and otherwise to dark.

Choose a scheme at startup with the --color-scheme flag:

$ ./morel --color-scheme=light

or at any time from within the shell:

- Sys.set ("colorScheme", "none");

A future release will let you define your own schemes as properties files in ~/.morel/color-schemes/.