test: add integration and smoke tests (#4)

This commit is contained in:
JY Tan
2025-12-26 14:56:20 -08:00
committed by GitHub
parent f86d9a2c82
commit 6fdd1af057
14 changed files with 2171 additions and 18 deletions

View File

@@ -22,6 +22,7 @@ Fence is a sandboxing tool that restricts network and filesystem access for arbi
- [Architecture](../ARCHITECTURE.md) - How fence works under the hood
- [Security model](security-model.md) - Threat model, guarantees, and limitations
- [Linux security features](linux-security-features.md) - Landlock, seccomp, eBPF details and fallback behavior
- [Testing](testing.md) - How to run tests and write new ones
## Examples

View File

@@ -70,6 +70,17 @@ This provides **defense-in-depth**: both bwrap mounts AND Landlock kernel restri
> [!NOTE]
> The eBPF monitor uses PID-range filtering (`pid >= SANDBOX_PID`) to exclude pre-existing system processes. This significantly reduces noise but isn't perfect—processes spawned after the sandbox starts may still appear.
### When network namespace is not available (containerized environments)
- **Impact**: `--unshare-net` is skipped; network is not fully isolated
- **Cause**: Running in Docker, GitHub Actions, or other environments without `CAP_NET_ADMIN`
- **Fallback**: Proxy-based filtering still works; filesystem/PID/seccomp isolation still active
- **Check**: Run `fence --linux-features` and look for "Network namespace (--unshare-net): false"
- **Workaround**: Run with `sudo`, or in Docker use `--cap-add=NET_ADMIN`
> [!NOTE]
> This is the most common "reduced isolation" scenario. Fence automatically detects this at startup and adapts. See the troubleshooting guide for more details.
### When bwrap is not available
- **Impact**: Cannot run fence on Linux

View File

@@ -32,6 +32,17 @@ sudo dnf install bubblewrap socat
sudo pacman -S bubblewrap socat
```
### Do I need sudo to run fence?
No, for most Linux systems. Fence works without root privileges because:
- Package-manager-installed `bubblewrap` is typically already setuid
- Fence detects available capabilities and adapts automatically
If some features aren't available (like network namespaces in Docker/CI), fence falls back gracefully - you'll still get filesystem isolation, command blocking, and proxy-based network filtering.
Run `fence --linux-features` to see what's available in your environment.
## Verify Installation
```bash

233
docs/testing.md Normal file
View File

@@ -0,0 +1,233 @@
# Testing
We maintain a test suite covering unit tests, integration tests, and smoke tests.
## Quick Start
```bash
# Run all tests
make test
# Run unit tests
go test ./...
# Run integration tests
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...
# Run smoke tests (end-to-end)
./scripts/smoke_test.sh
```
## Test Types
### Unit Tests
To verify individual functions and logic in isolation.
**Run:**
```bash
go test ./internal/...
```
### Integration Tests
Integration tests verify that the sandbox actually restricts/allows operations as expected. They spawn real processes under the sandbox and check outcomes.
**Files:**
- [`internal/sandbox/integration_test.go`](/internal/sandbox/integration_test.go) - Cross-platform tests
- [`internal/sandbox/integration_linux_test.go`](/internal/sandbox/integration_linux_test.go) - Linux-specific (Landlock, seccomp, bwrap)
- [`internal/sandbox/integration_macos_test.go`](/internal/sandbox/integration_macos_test.go) - macOS-specific (Seatbelt)
**What they test:**
- Filesystem restrictions (read/write blocking)
- Network blocking and proxy integration
- Command blocking
- Developer tool compatibility (Python, Node, Git)
- Security scenarios (symlink escape, path traversal)
- Platform-specific features (seccomp syscall filtering, Seatbelt profiles)
**Run:**
```bash
# All integration tests (platform-appropriate tests run automatically)
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...
# Linux-specific only
go test -v -run 'TestLinux' ./internal/sandbox/...
# macOS-specific only
go test -v -run 'TestMacOS' ./internal/sandbox/...
# With verbose output
go test -v -count=1 ./internal/sandbox/...
```
### Smoke Tests
Smoke tests verify the compiled `fence` binary works end-to-end. Unlike integration tests (which test internal Go APIs), smoke tests exercise the CLI interface.
**File:** [`scripts/smoke_test.sh`](/scripts/smoke_test.sh)
**What they test:**
- CLI flags (--version, -c, -s)
- Filesystem restrictions via settings file
- Command blocking via settings file
- Network blocking
- Environment variable injection (FENCE_SANDBOX, HTTP_PROXY)
- Tool compatibility (python3, node, git, rg) - ensure that frequently used tools don't break in sandbox
**Run:**
```bash
# Build and test
./scripts/smoke_test.sh
# Test specific binary
./scripts/smoke_test.sh ./path/to/fence
# Enable network tests (requires internet)
FENCE_TEST_NETWORK=1 ./scripts/smoke_test.sh
```
## Platform-Specific Behavior
### Linux
Linux tests verify:
- Landlock - Filesystem access control
- seccomp - Syscall filtering (blocks dangerous syscalls)
- bwrap - User namespace isolation
- Network namespaces - Network isolation via proxy
Requirements:
- Linux kernel 5.13+ (for Landlock)
- `bwrap` (bubblewrap) installed
- User namespace support enabled
### macOS
macOS tests verify:
- Seatbelt (sandbox-exec) - Built-in sandboxing
- Network proxy - All network traffic routed through proxy
Requirements:
- macOS 10.15+ (Catalina or later)
- No special setup needed (Seatbelt is built-in)
## Writing Tests
### Integration Test Helpers
The `integration_test.go` file provides helpers for writing sandbox tests:
```go
// Skip helpers
skipIfAlreadySandboxed(t) // Skip if running inside Fence
skipIfCommandNotFound(t, "python3") // Skip if command missing
// Run a command under the sandbox
result := runUnderSandbox(t, cfg, "touch /etc/test", workspace)
// Assertions
assertBlocked(t, result) // Command should have failed
assertAllowed(t, result) // Command should have succeeded
assertContains(t, result.Stdout, "expected")
// File assertions
assertFileExists(t, "/path/to/file")
assertFileNotExists(t, "/path/to/file")
// Config helpers
cfg := testConfig() // Basic deny-all config
cfg := testConfigWithWorkspace(workspace) // Allow writes to workspace
cfg := testConfigWithNetwork("example.com") // Allow domain
```
### Example Test
```go
func TestLinux_CustomFeature(t *testing.T) {
skipIfAlreadySandboxed(t)
workspace := createTempWorkspace(t)
cfg := testConfigWithWorkspace(workspace)
// Test that writes outside workspace are blocked
result := runUnderSandbox(t, cfg, "touch /tmp/outside.txt", workspace)
assertBlocked(t, result)
assertFileNotExists(t, "/tmp/outside.txt")
// Test that writes inside workspace work
insideFile := filepath.Join(workspace, "inside.txt")
result = runUnderSandbox(t, cfg, "touch "+insideFile, workspace)
assertAllowed(t, result)
assertFileExists(t, insideFile)
}
```
## CI
A [GitHub Actions workflow](/.github/workflows/main.yml) runs build, lint, and platform-specific tests.
Tests are designed to pass in CI environments (all dependencies installed) and local development machines (either Linux or MacOS).
If tests fail in CI, it indicates a real problem with the sandbox (not an environment limitation). The tests should fail loudly if:
- bwrap can't create user namespaces
- Landlock is not available
- Seatbelt fails to apply profiles
- Network isolation isn't working
## Test Coverage
Check test coverage with:
```bash
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out # View in browser
go tool cover -func=coverage.out # Summary
```
## Debugging Test Failures
### View sandbox logs
```bash
# Run with verbose Go test output
go test -v -run TestSpecificTest ./internal/sandbox/...
```
### Run command manually
```bash
# Replicate what the test does
./fence -c "the-command-that-failed"
# With a settings file
./fence -s /path/to/settings.json -c "command"
```
### Check platform capabilities
```bash
# Linux: Check kernel features
cat /proc/sys/kernel/unprivileged_userns_clone # Should be 1
uname -r # Kernel version (need 5.13+ for Landlock)
# macOS: Check sandbox-exec
sandbox-exec -p '(version 1)(allow default)' /bin/echo "sandbox works"
```
## Test Naming Conventions
- `Test<Platform>_<Feature>` - Platform-specific tests (e.g., `TestLinux_LandlockBlocksWrite`)
- `TestIntegration_<Feature>` - Cross-platform tests (e.g., `TestIntegration_PythonWorks`)
- `Test<Function>` - Unit tests (e.g., `TestShouldBlockCommand`)

View File

@@ -1,5 +1,82 @@
# Troubleshooting
## "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted" (Linux)
This error occurs when fence tries to create a network namespace but the environment lacks the `CAP_NET_ADMIN` capability. This is common in:
- **Docker containers** (unless run with `--privileged` or `--cap-add=NET_ADMIN`)
- **GitHub Actions** and other CI runners
- **Ubuntu 24.04+** with restrictive AppArmor policies
- **Kubernetes pods** without elevated security contexts
**What happens now:**
Fence automatically detects this limitation and falls back to running **without network namespace isolation**. The sandbox still provides:
- Filesystem restrictions (read-only root, allowWrite paths)
- PID namespace isolation
- Seccomp syscall filtering
- Landlock (if available)
**What's reduced:**
- Network isolation via namespace is skipped
- The proxy-based domain filtering still works (via `HTTP_PROXY`/`HTTPS_PROXY`)
- But programs that bypass proxy env vars won't be network-isolated
**To check if your environment supports network namespaces:**
```bash
fence --linux-features
```
Look for "Network namespace (--unshare-net): true/false"
**Solutions if you need full network isolation:**
1. **Run with elevated privileges:**
```bash
sudo fence <command>
```
2. **In Docker, add capability:**
```bash
docker run --cap-add=NET_ADMIN ...
```
3. **In GitHub Actions**, this typically isn't possible without self-hosted runners with elevated permissions.
4. **On Ubuntu 24.04+**, you may need to modify AppArmor profiles (see [Ubuntu bug 2069526](https://bugs.launchpad.net/bugs/2069526)).
## "bwrap: setting up uid map: Permission denied" (Linux)
This error occurs when bwrap cannot create user namespaces. This typically happens when:
- The `uidmap` package is not installed
- `/etc/subuid` and `/etc/subgid` are not configured for your user
- bwrap is not setuid
**Quick fix (if you have root access):**
```bash
# Install uidmap
sudo apt install uidmap # Debian/Ubuntu
# Make bwrap setuid
sudo chmod u+s $(which bwrap)
```
**Or configure subuid/subgid for your user:**
```bash
echo "$(whoami):100000:65536" | sudo tee -a /etc/subuid
echo "$(whoami):100000:65536" | sudo tee -a /etc/subgid
```
On most systems with package-manager-installed bwrap, this error shouldn't occur. If it does, your system may have non-standard security policies.
## "curl: (56) CONNECT tunnel failed, response 403"
This usually means: