Beamtalk REPL Protocol Test Suite

REPL TCP-protocol tests that validate the complete compilation and execution pipeline by evaluating Beamtalk expressions through the REPL JSON protocol.

History: Previously called the "E2E test suite" and located at tests/e2e/. The directory was renamed to tests/repl-protocol/ in BT-2085 because these tests exercise one specific surface (the REPL TCP protocol), not "end-to-end across surfaces". Cross-surface parity tests live under tests/parity/.

Feature Coverage Matrix

Feature	Status	Test File	Notes
Literals
Integer literals	✅	literals.bt	42, 0, 1000000
String literals	✅	literals.bt	'hello', ''
Boolean literals	✅	booleans.bt	true, false
Binary Operations
Addition	✅	arithmetic.bt	3 + 4
Subtraction	✅	arithmetic.bt	10 - 3
Multiplication	✅	arithmetic.bt	5 * 6
Division	✅	arithmetic.bt	20 / 4 → float
Math precedence	✅	arithmetic.bt	2 + 3 * 4 → 14
Unary Messages
Block value	✅	unary_messages.bt	[42] value
Integer negated	✅	unary_messages.bt	5 negated → -5
Integer abs	✅	unary_messages.bt	-5 abs → 5
Integer isZero	✅	unary_messages.bt	0 isZero → true
Integer isEven	✅	unary_messages.bt	4 isEven → true
Integer isOdd	✅	unary_messages.bt	5 isOdd → true
String length	✅	unary_messages.bt	'hello' length → 5
String isEmpty	✅	unary_messages.bt	'' isEmpty → true
Keyword Messages
value:	✅	keyword_messages.bt	`[:x
value:value:	✅	keyword_messages.bt	`[:x :y
value:value:value:	✅	keyword_messages.bt	Three-arg blocks
Blocks
Zero-arg blocks	✅	blocks.bt	[42] value
One-arg blocks	✅	blocks.bt	`[:x
Two-arg blocks	✅	blocks.bt	`[:x :y
Nested blocks	✅	blocks.bt	`[[:x
Return statements	✅	blocks.bt	`[:x
Variable Persistence
Simple assignment	✅	variable_persistence.bt	x := 42 persists
Variable reference	✅	variable_persistence.bt	x reads back
Multiple variables	✅	variable_persistence.bt	x + y works
Variable reassignment	✅	variable_persistence.bt	x := 100 updates
Control Flow
Block evaluation	✅	control_flow.bt	[5 + 3] value
Block with variables	✅	control_flow.bt	Uses REPL bindings
whileTrue:	🔧	blocks.bt	Non-mutating loop ([false] whileTrue: [42]) works; assignments inside blocks don't persist (BT-90)
whileFalse:	🔧	—	Implemented but assignments inside blocks don't persist (BT-90)
timesRepeat:	✅	actor_local_mutations.bt	Tested in stdlib
to:do:	✅	nested_to_do.bt	Tested in stdlib
Boolean Operations
ifTrue:ifFalse:	✅	booleans.bt	true ifTrue: [1] ifFalse: [2] → 1
ifTrue:	✅	booleans.bt	true ifTrue: [42] → 42
ifFalse:	✅	booleans.bt	false ifFalse: [42] → 42
and:	✅	booleans.bt	true and: [false] → false
or:	✅	booleans.bt	false or: [true] → true
not	✅	booleans.bt	true not → false
Cascades
Cascade syntax	🔧	cascades.bt	Use // @load + stateful tests¹
Actors
spawn	🔧	actors.bt	Use // @load to load class definitions²
Async messages	🔧	actors.bt	Use // @load + stateful tests
await	🔧	actors.bt	Use // @load + stateful tests
Error Handling
Division by zero	✅	errors.bt	1 / 0 → badarith
Semantic Analysis
Stored closure field error	✅	semantic_diagnostics.bt	@load-error on stored closure
Sealed class error	✅	semantic_diagnostics.bt	@load-error on sealed subclass
Reflection validator	✅	semantic_diagnostics.bt	respondsTo: with non-symbol
Undefined variable	✅	semantic_scope.bt	Compile-time error
Variable scoping	✅	semantic_scope.bt	Block params, closures
Self in methods	✅	semantic_scope.bt	Actor methods use self

Legend:

• ✅ = Fully tested and working
• 🔄 = Implemented but needs refinement (returns future in REPL)
• 🔧 = Implemented, REPL-protocol test infrastructure ready (needs real tests)
• 📋 = Documented, implementation in progress
• — = No separate test file (documented elsewhere)

Footnotes:

1. Cascades send async actor messages. Use // @load tests/repl-protocol/fixtures/counter.bt to load an actor class, then test cascades with stateful expressions.
2. Actor classes must be defined in files and loaded with // @load. See tests/repl-protocol/fixtures/ for example actors.

Directory Structure

tests/repl-protocol/
├── README.md              # This file
├── cases/                 # Test case files
│   ├── actors.btscript          # Actor documentation (syntax examples)
│   ├── arithmetic.btscript      # Arithmetic operations (+, -, *, /)
│   ├── blocks.btscript          # Block/closure tests
│   ├── booleans.btscript        # Boolean literals (true, false)
│   ├── cascades.btscript        # Cascade documentation (syntax examples)
│   ├── control_flow.btscript    # Control flow (block evaluation, variables)
│   ├── errors.btscript          # Error handling tests
│   ├── semantic_diagnostics.btscript # Semantic analysis error tests
│   ├── semantic_scope.btscript  # Variable scope and resolution tests
│   ├── keyword_messages.btscript # Keyword message sends
│   ├── literals.btscript        # Integer and string literals
│   ├── unary_messages.btscript  # Unary message sends
│   └── variable_persistence.btscript # Variable assignment and persistence
└── fixtures/              # Actor/class definitions for stateful tests
    └── counter.bt         # Simple counter actor class

Running Tests

# Run REPL protocol tests only
just test-repl-protocol

# Or directly via cargo (these tests are #[ignore]d, require --ignored flag)
cargo test --test repl_protocol -- --ignored

# Run with verbose output
cargo test --test repl_protocol -- --ignored --nocapture

Prerequisites:

• Erlang/OTP must be installed (for the REPL runtime)
• The project must be built (just build)

Test File Format

Test files use the .bt extension and contain Beamtalk expressions with expected results.

Basic Format

Each test case consists of an expression followed by a // => comment with the expected result:

3 + 4
// => 7

[:x | x + 1] value: 5
// => 6

Comments

Regular comments (not starting with // =>) are ignored and can be used for documentation:

// Test basic arithmetic
3 + 4
// => 7

// Test with larger numbers
1000 + 2000
// => 3000

Error Testing

To test that an expression produces an error, use ERROR: prefix in the expected result:

undefined_variable
// => ERROR: Undefined variable

The test passes if the error message contains the specified text.

See tests/repl-protocol/cases/errors.btscript for examples of error test cases.

Compilation Error Testing (@load-error)

To test that a file fails to compile with a specific error, use the @load-error directive:

// @load-error tests/repl-protocol/fixtures/bad_class.bt => cannot assign to field

The directive attempts to load the file and expects compilation to fail. The test passes if the error message contains the specified substring. If the load succeeds, the test fails.

See tests/repl-protocol/cases/semantic_diagnostics.bt for examples.

Multi-line Expressions

Currently, each expression must be on a single line within the test file format. This is a test parser limitation, not a language limitation - Beamtalk itself supports multi-line expressions. The test format reads one line at a time looking for // => markers. Multi-line test format support may be added in the future.

Stateful Tests (Actors and Cascades)

Test files support stateful multi-expression tests where variables persist between expressions within the same file. This is essential for testing actors and cascades.

Variable persistence within a file:

// Assign a variable
x := 42
// => 42

// Use it in the next expression - state persists!
x + 10
// => 52

Loading actor classes:

To test actors and cascades, first load a file containing class definitions using the // @load directive:

// @load tests/repl-protocol/fixtures/counter.bt

// Now Counter class is available
counter := Counter spawn
// => <pid>

// Send messages (state persists)
counter increment
// => 1

// Cascades - send multiple messages to same receiver
counter increment; increment; getValue
// => 3

Fixture files:

• tests/repl-protocol/fixtures/ - Actor class definitions for REPL-protocol tests
• tests/repl-protocol/fixtures/counter.bt - Simple counter actor example

Note: Bindings are cleared at the start of each test file, but persist across all expressions within that file. Loaded modules also persist for the file's duration.

Writing New Tests

1. Create a new .btscript file in tests/repl-protocol/cases/
2. Add expressions with // => expected results
3. Run just test-repl-protocol to verify

Guidelines

• One concept per file: Group related tests (e.g., all arithmetic in arithmetic.bt)
• Start simple: Begin with basic cases before edge cases
• Test both success and failure: Include error cases where appropriate
• Add comments: Document what each test is checking
• Keep expressions simple: The test format doesn't support multi-line expressions

Test Harness

The test harness (crates/beamtalk-cli/tests/repl_protocol.rs) handles:

1. REPL startup: Starts a REPL workspace automatically (ephemeral port)
2. REPL connection: Uses TCP socket to communicate with REPL
3. Test execution: Parses .bt files and evaluates each expression
4. Result comparison: Compares actual results with expected values

Protocol

The harness uses the same JSON protocol as the REPL CLI:

// Request
{"op": "eval", "id": "1", "code": "3 + 4"}

// Success response
{"id": "1", "value": "7"}

Troubleshooting

Debugging Output

To see BEAM output during tests:

cargo test --test repl_protocol -- --ignored --nocapture

Tests timeout

The default timeout is 30 seconds per expression. If tests are timing out:

1. Check if the expression is causing an infinite loop
2. Verify the Erlang runtime is functioning correctly
3. Check for resource exhaustion