test: add integration and smoke tests (#4)
This commit is contained in:
@@ -22,6 +22,7 @@ Fence is a sandboxing tool that restricts network and filesystem access for arbi
|
||||
- [Architecture](../ARCHITECTURE.md) - How fence works under the hood
|
||||
- [Security model](security-model.md) - Threat model, guarantees, and limitations
|
||||
- [Linux security features](linux-security-features.md) - Landlock, seccomp, eBPF details and fallback behavior
|
||||
- [Testing](testing.md) - How to run tests and write new ones
|
||||
|
||||
## Examples
|
||||
|
||||
|
||||
@@ -70,6 +70,17 @@ This provides **defense-in-depth**: both bwrap mounts AND Landlock kernel restri
|
||||
> [!NOTE]
|
||||
> The eBPF monitor uses PID-range filtering (`pid >= SANDBOX_PID`) to exclude pre-existing system processes. This significantly reduces noise but isn't perfect—processes spawned after the sandbox starts may still appear.
|
||||
|
||||
### When network namespace is not available (containerized environments)
|
||||
|
||||
- **Impact**: `--unshare-net` is skipped; network is not fully isolated
|
||||
- **Cause**: Running in Docker, GitHub Actions, or other environments without `CAP_NET_ADMIN`
|
||||
- **Fallback**: Proxy-based filtering still works; filesystem/PID/seccomp isolation still active
|
||||
- **Check**: Run `fence --linux-features` and look for "Network namespace (--unshare-net): false"
|
||||
- **Workaround**: Run with `sudo`, or in Docker use `--cap-add=NET_ADMIN`
|
||||
|
||||
> [!NOTE]
|
||||
> This is the most common "reduced isolation" scenario. Fence automatically detects this at startup and adapts. See the troubleshooting guide for more details.
|
||||
|
||||
### When bwrap is not available
|
||||
|
||||
- **Impact**: Cannot run fence on Linux
|
||||
|
||||
@@ -32,6 +32,17 @@ sudo dnf install bubblewrap socat
|
||||
sudo pacman -S bubblewrap socat
|
||||
```
|
||||
|
||||
### Do I need sudo to run fence?
|
||||
|
||||
No, for most Linux systems. Fence works without root privileges because:
|
||||
|
||||
- Package-manager-installed `bubblewrap` is typically already setuid
|
||||
- Fence detects available capabilities and adapts automatically
|
||||
|
||||
If some features aren't available (like network namespaces in Docker/CI), fence falls back gracefully - you'll still get filesystem isolation, command blocking, and proxy-based network filtering.
|
||||
|
||||
Run `fence --linux-features` to see what's available in your environment.
|
||||
|
||||
## Verify Installation
|
||||
|
||||
```bash
|
||||
|
||||
233
docs/testing.md
Normal file
233
docs/testing.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# Testing
|
||||
|
||||
We maintain a test suite covering unit tests, integration tests, and smoke tests.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
make test
|
||||
|
||||
# Run unit tests
|
||||
go test ./...
|
||||
|
||||
# Run integration tests
|
||||
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...
|
||||
|
||||
# Run smoke tests (end-to-end)
|
||||
./scripts/smoke_test.sh
|
||||
```
|
||||
|
||||
## Test Types
|
||||
|
||||
### Unit Tests
|
||||
|
||||
To verify individual functions and logic in isolation.
|
||||
|
||||
**Run:**
|
||||
|
||||
```bash
|
||||
go test ./internal/...
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
Integration tests verify that the sandbox actually restricts/allows operations as expected. They spawn real processes under the sandbox and check outcomes.
|
||||
|
||||
**Files:**
|
||||
|
||||
- [`internal/sandbox/integration_test.go`](/internal/sandbox/integration_test.go) - Cross-platform tests
|
||||
- [`internal/sandbox/integration_linux_test.go`](/internal/sandbox/integration_linux_test.go) - Linux-specific (Landlock, seccomp, bwrap)
|
||||
- [`internal/sandbox/integration_macos_test.go`](/internal/sandbox/integration_macos_test.go) - macOS-specific (Seatbelt)
|
||||
|
||||
**What they test:**
|
||||
|
||||
- Filesystem restrictions (read/write blocking)
|
||||
- Network blocking and proxy integration
|
||||
- Command blocking
|
||||
- Developer tool compatibility (Python, Node, Git)
|
||||
- Security scenarios (symlink escape, path traversal)
|
||||
- Platform-specific features (seccomp syscall filtering, Seatbelt profiles)
|
||||
|
||||
**Run:**
|
||||
|
||||
```bash
|
||||
# All integration tests (platform-appropriate tests run automatically)
|
||||
go test -v -run 'TestIntegration|TestLinux|TestMacOS' ./internal/sandbox/...
|
||||
|
||||
# Linux-specific only
|
||||
go test -v -run 'TestLinux' ./internal/sandbox/...
|
||||
|
||||
# macOS-specific only
|
||||
go test -v -run 'TestMacOS' ./internal/sandbox/...
|
||||
|
||||
# With verbose output
|
||||
go test -v -count=1 ./internal/sandbox/...
|
||||
```
|
||||
|
||||
### Smoke Tests
|
||||
|
||||
Smoke tests verify the compiled `fence` binary works end-to-end. Unlike integration tests (which test internal Go APIs), smoke tests exercise the CLI interface.
|
||||
|
||||
**File:** [`scripts/smoke_test.sh`](/scripts/smoke_test.sh)
|
||||
|
||||
**What they test:**
|
||||
|
||||
- CLI flags (--version, -c, -s)
|
||||
- Filesystem restrictions via settings file
|
||||
- Command blocking via settings file
|
||||
- Network blocking
|
||||
- Environment variable injection (FENCE_SANDBOX, HTTP_PROXY)
|
||||
- Tool compatibility (python3, node, git, rg) - ensure that frequently used tools don't break in sandbox
|
||||
|
||||
**Run:**
|
||||
|
||||
```bash
|
||||
# Build and test
|
||||
./scripts/smoke_test.sh
|
||||
|
||||
# Test specific binary
|
||||
./scripts/smoke_test.sh ./path/to/fence
|
||||
|
||||
# Enable network tests (requires internet)
|
||||
FENCE_TEST_NETWORK=1 ./scripts/smoke_test.sh
|
||||
```
|
||||
|
||||
## Platform-Specific Behavior
|
||||
|
||||
### Linux
|
||||
|
||||
Linux tests verify:
|
||||
|
||||
- Landlock - Filesystem access control
|
||||
- seccomp - Syscall filtering (blocks dangerous syscalls)
|
||||
- bwrap - User namespace isolation
|
||||
- Network namespaces - Network isolation via proxy
|
||||
|
||||
Requirements:
|
||||
|
||||
- Linux kernel 5.13+ (for Landlock)
|
||||
- `bwrap` (bubblewrap) installed
|
||||
- User namespace support enabled
|
||||
|
||||
### macOS
|
||||
|
||||
macOS tests verify:
|
||||
|
||||
- Seatbelt (sandbox-exec) - Built-in sandboxing
|
||||
- Network proxy - All network traffic routed through proxy
|
||||
|
||||
Requirements:
|
||||
|
||||
- macOS 10.15+ (Catalina or later)
|
||||
- No special setup needed (Seatbelt is built-in)
|
||||
|
||||
## Writing Tests
|
||||
|
||||
### Integration Test Helpers
|
||||
|
||||
The `integration_test.go` file provides helpers for writing sandbox tests:
|
||||
|
||||
```go
|
||||
// Skip helpers
|
||||
skipIfAlreadySandboxed(t) // Skip if running inside Fence
|
||||
skipIfCommandNotFound(t, "python3") // Skip if command missing
|
||||
|
||||
// Run a command under the sandbox
|
||||
result := runUnderSandbox(t, cfg, "touch /etc/test", workspace)
|
||||
|
||||
// Assertions
|
||||
assertBlocked(t, result) // Command should have failed
|
||||
assertAllowed(t, result) // Command should have succeeded
|
||||
assertContains(t, result.Stdout, "expected")
|
||||
|
||||
// File assertions
|
||||
assertFileExists(t, "/path/to/file")
|
||||
assertFileNotExists(t, "/path/to/file")
|
||||
|
||||
// Config helpers
|
||||
cfg := testConfig() // Basic deny-all config
|
||||
cfg := testConfigWithWorkspace(workspace) // Allow writes to workspace
|
||||
cfg := testConfigWithNetwork("example.com") // Allow domain
|
||||
```
|
||||
|
||||
### Example Test
|
||||
|
||||
```go
|
||||
func TestLinux_CustomFeature(t *testing.T) {
|
||||
skipIfAlreadySandboxed(t)
|
||||
|
||||
workspace := createTempWorkspace(t)
|
||||
cfg := testConfigWithWorkspace(workspace)
|
||||
|
||||
// Test that writes outside workspace are blocked
|
||||
result := runUnderSandbox(t, cfg, "touch /tmp/outside.txt", workspace)
|
||||
assertBlocked(t, result)
|
||||
assertFileNotExists(t, "/tmp/outside.txt")
|
||||
|
||||
// Test that writes inside workspace work
|
||||
insideFile := filepath.Join(workspace, "inside.txt")
|
||||
result = runUnderSandbox(t, cfg, "touch "+insideFile, workspace)
|
||||
assertAllowed(t, result)
|
||||
assertFileExists(t, insideFile)
|
||||
}
|
||||
```
|
||||
|
||||
## CI
|
||||
|
||||
A [GitHub Actions workflow](/.github/workflows/main.yml) runs build, lint, and platform-specific tests.
|
||||
|
||||
Tests are designed to pass in CI environments (all dependencies installed) and local development machines (either Linux or MacOS).
|
||||
|
||||
If tests fail in CI, it indicates a real problem with the sandbox (not an environment limitation). The tests should fail loudly if:
|
||||
|
||||
- bwrap can't create user namespaces
|
||||
- Landlock is not available
|
||||
- Seatbelt fails to apply profiles
|
||||
- Network isolation isn't working
|
||||
|
||||
## Test Coverage
|
||||
|
||||
Check test coverage with:
|
||||
|
||||
```bash
|
||||
go test -coverprofile=coverage.out ./...
|
||||
go tool cover -html=coverage.out # View in browser
|
||||
go tool cover -func=coverage.out # Summary
|
||||
```
|
||||
|
||||
## Debugging Test Failures
|
||||
|
||||
### View sandbox logs
|
||||
|
||||
```bash
|
||||
# Run with verbose Go test output
|
||||
go test -v -run TestSpecificTest ./internal/sandbox/...
|
||||
```
|
||||
|
||||
### Run command manually
|
||||
|
||||
```bash
|
||||
# Replicate what the test does
|
||||
./fence -c "the-command-that-failed"
|
||||
|
||||
# With a settings file
|
||||
./fence -s /path/to/settings.json -c "command"
|
||||
```
|
||||
|
||||
### Check platform capabilities
|
||||
|
||||
```bash
|
||||
# Linux: Check kernel features
|
||||
cat /proc/sys/kernel/unprivileged_userns_clone # Should be 1
|
||||
uname -r # Kernel version (need 5.13+ for Landlock)
|
||||
|
||||
# macOS: Check sandbox-exec
|
||||
sandbox-exec -p '(version 1)(allow default)' /bin/echo "sandbox works"
|
||||
```
|
||||
|
||||
## Test Naming Conventions
|
||||
|
||||
- `Test<Platform>_<Feature>` - Platform-specific tests (e.g., `TestLinux_LandlockBlocksWrite`)
|
||||
- `TestIntegration_<Feature>` - Cross-platform tests (e.g., `TestIntegration_PythonWorks`)
|
||||
- `Test<Function>` - Unit tests (e.g., `TestShouldBlockCommand`)
|
||||
@@ -1,5 +1,82 @@
|
||||
# Troubleshooting
|
||||
|
||||
## "bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted" (Linux)
|
||||
|
||||
This error occurs when fence tries to create a network namespace but the environment lacks the `CAP_NET_ADMIN` capability. This is common in:
|
||||
|
||||
- **Docker containers** (unless run with `--privileged` or `--cap-add=NET_ADMIN`)
|
||||
- **GitHub Actions** and other CI runners
|
||||
- **Ubuntu 24.04+** with restrictive AppArmor policies
|
||||
- **Kubernetes pods** without elevated security contexts
|
||||
|
||||
**What happens now:**
|
||||
|
||||
Fence automatically detects this limitation and falls back to running **without network namespace isolation**. The sandbox still provides:
|
||||
|
||||
- Filesystem restrictions (read-only root, allowWrite paths)
|
||||
- PID namespace isolation
|
||||
- Seccomp syscall filtering
|
||||
- Landlock (if available)
|
||||
|
||||
**What's reduced:**
|
||||
|
||||
- Network isolation via namespace is skipped
|
||||
- The proxy-based domain filtering still works (via `HTTP_PROXY`/`HTTPS_PROXY`)
|
||||
- But programs that bypass proxy env vars won't be network-isolated
|
||||
|
||||
**To check if your environment supports network namespaces:**
|
||||
|
||||
```bash
|
||||
fence --linux-features
|
||||
```
|
||||
|
||||
Look for "Network namespace (--unshare-net): true/false"
|
||||
|
||||
**Solutions if you need full network isolation:**
|
||||
|
||||
1. **Run with elevated privileges:**
|
||||
|
||||
```bash
|
||||
sudo fence <command>
|
||||
```
|
||||
|
||||
2. **In Docker, add capability:**
|
||||
|
||||
```bash
|
||||
docker run --cap-add=NET_ADMIN ...
|
||||
```
|
||||
|
||||
3. **In GitHub Actions**, this typically isn't possible without self-hosted runners with elevated permissions.
|
||||
|
||||
4. **On Ubuntu 24.04+**, you may need to modify AppArmor profiles (see [Ubuntu bug 2069526](https://bugs.launchpad.net/bugs/2069526)).
|
||||
|
||||
## "bwrap: setting up uid map: Permission denied" (Linux)
|
||||
|
||||
This error occurs when bwrap cannot create user namespaces. This typically happens when:
|
||||
|
||||
- The `uidmap` package is not installed
|
||||
- `/etc/subuid` and `/etc/subgid` are not configured for your user
|
||||
- bwrap is not setuid
|
||||
|
||||
**Quick fix (if you have root access):**
|
||||
|
||||
```bash
|
||||
# Install uidmap
|
||||
sudo apt install uidmap # Debian/Ubuntu
|
||||
|
||||
# Make bwrap setuid
|
||||
sudo chmod u+s $(which bwrap)
|
||||
```
|
||||
|
||||
**Or configure subuid/subgid for your user:**
|
||||
|
||||
```bash
|
||||
echo "$(whoami):100000:65536" | sudo tee -a /etc/subuid
|
||||
echo "$(whoami):100000:65536" | sudo tee -a /etc/subgid
|
||||
```
|
||||
|
||||
On most systems with package-manager-installed bwrap, this error shouldn't occur. If it does, your system may have non-standard security policies.
|
||||
|
||||
## "curl: (56) CONNECT tunnel failed, response 403"
|
||||
|
||||
This usually means:
|
||||
|
||||
Reference in New Issue
Block a user