greywall/docs/testing.md

Testing

We maintain a test suite covering unit tests, integration tests, and smoke tests.

Quick Start

# Run all tests
make test

# Run unit tests
go test ./...

# Run integration tests
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...

# Run smoke tests (end-to-end)
./scripts/smoke_test.sh

Test Types

Unit Tests

To verify individual functions and logic in isolation.

Run:

go test ./internal/...

Integration Tests

Integration tests verify that the sandbox actually restricts/allows operations as expected. They spawn real processes under the sandbox and check outcomes.

Files:

• internal/sandbox/integration_test.go - Cross-platform tests
• internal/sandbox/integration_linux_test.go - Linux-specific (Landlock, seccomp, bwrap)
• internal/sandbox/integration_macos_test.go - macOS-specific (Seatbelt)

What they test:

• Filesystem restrictions (read/write blocking)
• Network blocking and proxy integration
• Command blocking
• Developer tool compatibility (Python, Node, Git)
• Security scenarios (symlink escape, path traversal)
• Platform-specific features (seccomp syscall filtering, Seatbelt profiles)

Run:

# All integration tests (platform-appropriate tests run automatically)
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...

# Linux-specific only
go test -v -run 'TestLinux' ./internal/sandbox/...

# macOS-specific only
go test -v -run 'TestMacOS' ./internal/sandbox/...

# With verbose output
go test -v -count=1 ./internal/sandbox/...

Sandboxed Build Environments (Nix, etc.)

If you're packaging greywall for a distribution (e.g., Nix, Homebrew, Debian), note that some integration tests will be skipped when running go test during the build.

Greywall's Landlock integration on Linux uses a wrapper approach: the greywall binary re-executes itself with --landlock-apply inside the sandbox. Test binaries (e.g., sandbox.test) don't have this handler, so Landlock-specific tests automatically skip when not running as the greywall CLI.

Tests that skip include those calling skipIfLandlockNotUsable():

• TestLinux_LandlockBlocksWriteOutsideWorkspace
• TestLinux_LandlockProtectsGitHooks
• TestLinux_LandlockProtectsGitConfig
• TestLinux_LandlockProtectsBashrc
• TestLinux_LandlockAllowsTmpGreywall
• TestLinux_PathTraversalBlocked
• TestLinux_SeccompBlocksDangerousSyscalls

Test Type	What it tests	Landlock coverage
go test (integration)	Go APIs, bwrap isolation, command blocking	Skipped (test binary can't use --landlock-apply)
smoke_test.sh	Actual greywall CLI end-to-end	✅ Full coverage

For full test coverage including Landlock, run the smoke tests against the built binary (see "Smoke Tests" section below).

Nested sandboxing limitations:

• macOS: Nested Seatbelt sandboxing is not supported. If the build environment already uses sandbox-exec (like Nix's Darwin sandbox), greywall's tests cannot create another sandbox. The kernel returns forbidden-sandbox-reinit. This is a macOS limitation.
• Linux: Tests should work in most build sandboxes, but Landlock tests will skip as explained above. Runtime functionality is unaffected.

Smoke Tests

Smoke tests verify the compiled greywall binary works end-to-end. Unlike integration tests (which test internal Go APIs), smoke tests exercise the CLI interface.

File: scripts/smoke_test.sh

What they test:

• CLI flags (--version, -c, -s)
• Filesystem restrictions via settings file
• Command blocking via settings file
• Network blocking
• Environment variable injection (GREYWALL_SANDBOX, HTTP_PROXY)
• Tool compatibility (python3, node, git, rg) - ensure that frequently used tools don't break in sandbox

Run:

# Build and test
./scripts/smoke_test.sh

# Test specific binary
./scripts/smoke_test.sh ./path/to/greywall

# Enable network tests (requires internet)
GREYWALL_TEST_NETWORK=1 ./scripts/smoke_test.sh

Platform-Specific Behavior

Linux

Linux tests verify:

• Landlock - Filesystem access control
• seccomp - Syscall filtering (blocks dangerous syscalls)
• bwrap - User namespace isolation
• Network namespaces - Network isolation via proxy

Requirements:

• Linux kernel 5.13+ (for Landlock)
• bwrap (bubblewrap) installed
• User namespace support enabled

macOS

macOS tests verify:

• Seatbelt (sandbox-exec) - Built-in sandboxing
• Network proxy - All network traffic routed through proxy

Requirements:

• macOS 10.15+ (Catalina or later)
• No special setup needed (Seatbelt is built-in)

Writing Tests

Integration Test Helpers

The integration_test.go file provides helpers for writing sandbox tests:

// Skip helpers
skipIfAlreadySandboxed(t)              // Skip if running inside Greywall
skipIfCommandNotFound(t, "python3")    // Skip if command missing

// Run a command under the sandbox
result := runUnderSandbox(t, cfg, "touch /etc/test", workspace)

// Assertions
assertBlocked(t, result)      // Command should have failed
assertAllowed(t, result)      // Command should have succeeded
assertContains(t, result.Stdout, "expected")

// File assertions
assertFileExists(t, "/path/to/file")
assertFileNotExists(t, "/path/to/file")

// Config helpers
cfg := testConfig()                          // Basic deny-all config
cfg := testConfigWithWorkspace(workspace)    // Allow writes to workspace
cfg := testConfigWithNetwork("example.com")  // Allow domain

Example Test

func TestLinux_CustomFeature(t *testing.T) {
    skipIfAlreadySandboxed(t)
    
    workspace := createTempWorkspace(t)
    cfg := testConfigWithWorkspace(workspace)
    
    // Test that writes outside workspace are blocked
    result := runUnderSandbox(t, cfg, "touch /tmp/outside.txt", workspace)
    assertBlocked(t, result)
    assertFileNotExists(t, "/tmp/outside.txt")
    
    // Test that writes inside workspace work
    insideFile := filepath.Join(workspace, "inside.txt")
    result = runUnderSandbox(t, cfg, "touch "+insideFile, workspace)
    assertAllowed(t, result)
    assertFileExists(t, insideFile)
}

CI

A GitHub Actions workflow runs build, lint, and platform-specific tests.

Tests are designed to pass in CI environments (all dependencies installed) and local development machines (either Linux or MacOS).

If tests fail in CI, it indicates a real problem with the sandbox (not an environment limitation). The tests should fail loudly if:

• bwrap can't create user namespaces
• Landlock is not available
• Seatbelt fails to apply profiles
• Network isolation isn't working

Test Coverage

Check test coverage with:

go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out  # View in browser
go tool cover -func=coverage.out  # Summary

Debugging Test Failures

View sandbox logs

# Run with verbose Go test output
go test -v -run TestSpecificTest ./internal/sandbox/...

Run command manually

# Replicate what the test does
./greywall -c "the-command-that-failed"

# With a settings file
./greywall -s /path/to/settings.json -c "command"

Check platform capabilities

# Linux: Check kernel features
cat /proc/sys/kernel/unprivileged_userns_clone  # Should be 1
uname -r  # Kernel version (need 5.13+ for Landlock)

# macOS: Check sandbox-exec
sandbox-exec -p '(version 1)(allow default)' /bin/echo "sandbox works"

Test Naming Conventions

• Test<Platform>_<Feature> - Platform-specific tests (e.g., TestLinux_LandlockBlocksWrite)
• TestIntegration_<Feature> - Cross-platform tests (e.g., TestIntegration_PythonWorks)
• Test<Function> - Unit tests (e.g., TestShouldBlockCommand)