greywall/docs/testing.md

Testing

We maintain a test suite covering unit tests, integration tests, and smoke tests.

Quick Start

# Run all tests
make test

# Run unit tests
go test ./...

# Run integration tests
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...

# Run smoke tests (end-to-end)
./scripts/smoke_test.sh

Test Types

Unit Tests

To verify individual functions and logic in isolation.

Run:

go test ./internal/...

Integration Tests

Integration tests verify that the sandbox actually restricts/allows operations as expected. They spawn real processes under the sandbox and check outcomes.

Files:

• internal/sandbox/integration_test.go - Cross-platform tests
• internal/sandbox/integration_linux_test.go - Linux-specific (Landlock, seccomp, bwrap)
• internal/sandbox/integration_macos_test.go - macOS-specific (Seatbelt)

What they test:

• Filesystem restrictions (read/write blocking)
• Network blocking and proxy integration
• Command blocking
• Developer tool compatibility (Python, Node, Git)
• Security scenarios (symlink escape, path traversal)
• Platform-specific features (seccomp syscall filtering, Seatbelt profiles)

Run:

# All integration tests (platform-appropriate tests run automatically)
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...

# Linux-specific only
go test -v -run 'TestLinux' ./internal/sandbox/...

# macOS-specific only
go test -v -run 'TestMacOS' ./internal/sandbox/...

# With verbose output
go test -v -count=1 ./internal/sandbox/...

Smoke Tests

Smoke tests verify the compiled fence binary works end-to-end. Unlike integration tests (which test internal Go APIs), smoke tests exercise the CLI interface.

File: scripts/smoke_test.sh

What they test:

• CLI flags (--version, -c, -s)
• Filesystem restrictions via settings file
• Command blocking via settings file
• Network blocking
• Environment variable injection (FENCE_SANDBOX, HTTP_PROXY)
• Tool compatibility (python3, node, git, rg) - ensure that frequently used tools don't break in sandbox

Run:

# Build and test
./scripts/smoke_test.sh

# Test specific binary
./scripts/smoke_test.sh ./path/to/fence

# Enable network tests (requires internet)
FENCE_TEST_NETWORK=1 ./scripts/smoke_test.sh

Platform-Specific Behavior

Linux

Linux tests verify:

• Landlock - Filesystem access control
• seccomp - Syscall filtering (blocks dangerous syscalls)
• bwrap - User namespace isolation
• Network namespaces - Network isolation via proxy

Requirements:

• Linux kernel 5.13+ (for Landlock)
• bwrap (bubblewrap) installed
• User namespace support enabled

macOS

macOS tests verify:

• Seatbelt (sandbox-exec) - Built-in sandboxing
• Network proxy - All network traffic routed through proxy

Requirements:

• macOS 10.15+ (Catalina or later)
• No special setup needed (Seatbelt is built-in)

Writing Tests

Integration Test Helpers

The integration_test.go file provides helpers for writing sandbox tests:

// Skip helpers
skipIfAlreadySandboxed(t)              // Skip if running inside Fence
skipIfCommandNotFound(t, "python3")    // Skip if command missing

// Run a command under the sandbox
result := runUnderSandbox(t, cfg, "touch /etc/test", workspace)

// Assertions
assertBlocked(t, result)      // Command should have failed
assertAllowed(t, result)      // Command should have succeeded
assertContains(t, result.Stdout, "expected")

// File assertions
assertFileExists(t, "/path/to/file")
assertFileNotExists(t, "/path/to/file")

// Config helpers
cfg := testConfig()                          // Basic deny-all config
cfg := testConfigWithWorkspace(workspace)    // Allow writes to workspace
cfg := testConfigWithNetwork("example.com")  // Allow domain

Example Test

func TestLinux_CustomFeature(t *testing.T) {
    skipIfAlreadySandboxed(t)
    
    workspace := createTempWorkspace(t)
    cfg := testConfigWithWorkspace(workspace)
    
    // Test that writes outside workspace are blocked
    result := runUnderSandbox(t, cfg, "touch /tmp/outside.txt", workspace)
    assertBlocked(t, result)
    assertFileNotExists(t, "/tmp/outside.txt")
    
    // Test that writes inside workspace work
    insideFile := filepath.Join(workspace, "inside.txt")
    result = runUnderSandbox(t, cfg, "touch "+insideFile, workspace)
    assertAllowed(t, result)
    assertFileExists(t, insideFile)
}

CI

A GitHub Actions workflow runs build, lint, and platform-specific tests.

Tests are designed to pass in CI environments (all dependencies installed) and local development machines (either Linux or MacOS).

If tests fail in CI, it indicates a real problem with the sandbox (not an environment limitation). The tests should fail loudly if:

• bwrap can't create user namespaces
• Landlock is not available
• Seatbelt fails to apply profiles
• Network isolation isn't working

Test Coverage

Check test coverage with:

go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out  # View in browser
go tool cover -func=coverage.out  # Summary

Debugging Test Failures

View sandbox logs

# Run with verbose Go test output
go test -v -run TestSpecificTest ./internal/sandbox/...

Run command manually

# Replicate what the test does
./fence -c "the-command-that-failed"

# With a settings file
./fence -s /path/to/settings.json -c "command"

Check platform capabilities

# Linux: Check kernel features
cat /proc/sys/kernel/unprivileged_userns_clone  # Should be 1
uname -r  # Kernel version (need 5.13+ for Landlock)

# macOS: Check sandbox-exec
sandbox-exec -p '(version 1)(allow default)' /bin/echo "sandbox works"

Test Naming Conventions

• Test<Platform>_<Feature> - Platform-specific tests (e.g., TestLinux_LandlockBlocksWrite)
• TestIntegration_<Feature> - Cross-platform tests (e.g., TestIntegration_PythonWorks)
• Test<Function> - Unit tests (e.g., TestShouldBlockCommand)