mirror of
https://github.com/Monadical-SAS/cubbi.git
synced 2025-12-20 04:09:06 +00:00
* feat: migrate container configuration from env vars to YAML config files
- Replace environment variable-based configuration with structured YAML config files
- Add Pydantic models for type-safe configuration management in cubbi_init.py
- Update container.py to generate /cubbi/config.yaml and mount into containers
- Simplify goose plugin to extract provider from default model format
- Remove complex environment variable handling in favor of direct config access
- Maintain backward compatibility while enabling cleaner plugin architecture
* feat: optimize goose plugin to only pass required API key for selected model
- Update goose plugin to set only the API key for the provider of the selected model
- Add selective API key configuration for anthropic, openai, google, and openrouter
- Update README.md with comprehensive automated testing documentation
- Add litellm/gpt-oss:120b to test.sh model matrix (now 5 images × 4 models = 20 tests)
- Include single prompt command syntax for each tool in the documentation
* feat: add comprehensive integration tests with pytest parametrization
- Create tests/test_integration.py with parametrized tests for 5 images × 4 models (20 combinations)
- Add pytest configuration to exclude integration tests by default
- Add integration marker for selective test running
- Include help command tests and image availability tests
- Document test usage in tests/README_integration.md
Integration tests cover:
- goose, aider, claudecode, opencode, crush images
- anthropic/claude-sonnet-4-20250514, openai/gpt-4o, openrouter/openai/gpt-4o, litellm/gpt-oss:120b models
- Proper command syntax for each tool
- Success validation with exit codes and completion markers
Usage:
- pytest (regular tests only)
- pytest -m integration (integration tests only)
- pytest -m integration -k "goose" (specific image)
* feat: update OpenCode plugin with perfect multi-provider configuration
- Add global STANDARD_PROVIDERS constant for maintainability
- Support custom providers (with baseURL) vs standard providers
- Custom providers: include npm package, name, baseURL, apiKey, models
- Standard providers: include only apiKey and empty models
- Use direct API key values from cubbi config instead of env vars
- Only add default model to the provider that matches the default model
- Use @ai-sdk/openai-compatible for OpenAI-compatible providers
- Preserve model names without transformation
- All providers get required empty models{} section per OpenCode spec
This ensures OpenCode can properly recognize and use both native
providers (anthropic, openai, google, openrouter) and custom
providers (litellm, etc.) with correct configuration format.
* refactor: model is now a combination of provider/model
* feat: add separate integration test for Claude Code without model config
Claude Code is Anthropic-specific and doesn't require model selection like other tools.
Created dedicated test that verifies basic functionality without model preselection.
* feat: update Claude Code and Crush plugins to use new config system
- Claude Code plugin now uses cubbi_config.providers to get Anthropic API key
- Crush plugin updated to use cubbi_config.providers for provider configuration
- Both plugins maintain backwards compatibility with environment variables
- Consistent plugin structure across all cubbi images
* feat: add environments_to_forward support for images
- Add environments_to_forward field to ImageConfig and Image models
- Update container creation logic to forward specified environment variables from host
- Add environments_to_forward to claudecode cubbi_image.yaml to ensure Anthropic API key is always available
- Claude Code now gets required environment variables regardless of model selection
- This ensures Claude Code works properly even when other models are specified
Fixes the issue where Claude Code couldn't access Anthropic API key when using different model configurations.
* refactor: remove unused environment field from cubbi_image.yaml files
The 'environment' field was loaded but never processed at runtime.
Only 'environments_to_forward' is actually used to pass environment
variables from host to container.
Cleaned up configuration files by removing:
- 72 lines from aider/cubbi_image.yaml
- 42 lines from claudecode/cubbi_image.yaml
- 28 lines from crush/cubbi_image.yaml
- 16 lines from goose/cubbi_image.yaml
- Empty environment: [] from opencode/cubbi_image.yaml
This makes the configuration files cleaner and only contains
fields that are actually used by the system.
* feat: implement environment variable forwarding for aider
Updates aider to automatically receive all relevant environment variables
from the host, similar to how opencode works.
Changes:
- Added environments_to_forward field to aider/cubbi_image.yaml with
comprehensive list of API keys, configuration, and proxy variables
- Updated aider_plugin.py to use cubbi_config system for provider/model setup
- Environment variables now forwarded automatically during container creation
- Maintains backward compatibility with legacy environment variables
Environment variables forwarded:
- API Keys: OPENAI_API_KEY, ANTHROPIC_API_KEY, DEEPSEEK_API_KEY, etc.
- Configuration: AIDER_MODEL, GIT_* variables, HTTP_PROXY, etc.
- Timezone: TZ for proper log timestamps
Tested: All aider tests pass, environment variables confirmed forwarded.
* refactor: remove unused volumes and init fields from cubbi_image.yaml files
Both 'volumes' and 'init' fields were loaded but never processed at runtime.
These were incomplete implementations that didn't affect container behavior.
Removed from all 5 images:
- volumes: List with mountPath: /app (incomplete, missing host paths)
- init: pre_command and command fields (unused during container creation)
The cubbi_image.yaml files now only contain fields that are actually used:
- Basic metadata (name, description, version, maintainer, image)
- persistent_configs (working functionality)
- environments_to_forward (working functionality where present)
This makes the configuration files cleaner and eliminates confusion
about what functionality is actually implemented.
* refactor: remove unused ImageInit and VolumeMount models
These models were only referenced in the Image model definition but
never used at runtime since we removed all init: and volumes: fields
from cubbi_image.yaml files.
Removed:
- VolumeMount class (mountPath, description fields)
- ImageInit class (pre_command, command fields)
- init: Optional[ImageInit] field from Image model
- volumes: List[VolumeMount] field from Image model
The Image model now only contains fields that are actually used:
- Basic metadata (name, description, version, maintainer, image)
- environment (loaded but unused - kept for future cleanup)
- persistent_configs (working functionality)
- environments_to_forward (working functionality)
This makes the data model cleaner and eliminates dead code.
* feat: add interactive configuration command
Adds `cubbi configure` command for interactive setup of LLM providers
and models through a user-friendly questionnaire interface.
New features:
- Interactive provider configuration (OpenAI, Anthropic, OpenRouter, etc.)
- API key management with environment variable references
- Model selection with provider/model format validation
- Default settings configuration (image, ports, volumes, etc.)
- Added questionary dependency for interactive prompts
Changes:
- Added cubbi/configure.py with full interactive configuration logic
- Added configure command to cubbi/cli.py
- Updated uv.lock with questionary and prompt-toolkit dependencies
Usage: `cubbi configure`
* refactor: update integration tests for current functionality
Updates integration tests to reflect current cubbi functionality:
test_integration.py:
- Simplified image list (removed crush temporarily)
- Updated model list with current supported models
- Removed outdated help command tests that were timing out
- Simplified claudecode test to basic functionality test
- Updated command templates for current tool versions
test_integration_docker.py:
- Cleaned up container management tests
- Fixed formatting and improved readability
- Updated assertion formatting for better error messages
These changes align the tests with the current state of the codebase
and remove tests that were causing timeouts or failures.
* fix: fix temporary file chmod
373 lines
13 KiB
Python
373 lines
13 KiB
Python
"""
|
|
Integration tests for Docker interactions in Cubbi Container.
|
|
These tests require Docker to be running.
|
|
"""
|
|
|
|
import subprocess
|
|
import time
|
|
import uuid
|
|
import docker
|
|
|
|
|
|
# Import the requires_docker decorator from conftest
|
|
from conftest import requires_docker
|
|
|
|
|
|
def execute_command_in_container(container_id, command):
|
|
"""Execute a command in a Docker container and return the output."""
|
|
result = subprocess.run(
|
|
["docker", "exec", container_id, "bash", "-c", command],
|
|
capture_output=True,
|
|
text=True,
|
|
)
|
|
return result.stdout.strip()
|
|
|
|
|
|
def wait_for_container_init(container_id, timeout=5.0, poll_interval=0.1):
|
|
start_time = time.time()
|
|
|
|
while time.time() - start_time < timeout:
|
|
try:
|
|
# Check if /cubbi/init.status contains INIT_COMPLETE=true
|
|
result = execute_command_in_container(
|
|
container_id,
|
|
"grep -q 'INIT_COMPLETE=true' /cubbi/init.status 2>/dev/null && echo 'COMPLETE' || echo 'PENDING'",
|
|
)
|
|
|
|
if result == "COMPLETE":
|
|
return True
|
|
|
|
except subprocess.CalledProcessError:
|
|
# File might not exist yet or container not ready, continue polling
|
|
pass
|
|
|
|
time.sleep(poll_interval)
|
|
|
|
# Timeout reached
|
|
return False
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_session_create_with_volumes(
|
|
isolate_cubbi_config, test_file_content
|
|
):
|
|
"""Test creating a session with a volume mount."""
|
|
test_file, test_content = test_file_content
|
|
session = None
|
|
|
|
try:
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create a session with a volume mount
|
|
session = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-volume-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False, # Don't mount current directory
|
|
volumes={str(test_file): {"bind": "/test/volume_test.txt", "mode": "ro"}},
|
|
)
|
|
|
|
assert session is not None
|
|
assert session.status == "running"
|
|
|
|
# Wait for container initialization to complete
|
|
init_success = wait_for_container_init(session.container_id)
|
|
assert init_success, "Container initialization timed out"
|
|
|
|
# Verify the file exists in the container and has correct content
|
|
container_content = execute_command_in_container(
|
|
session.container_id, "cat /test/volume_test.txt"
|
|
)
|
|
|
|
assert container_content == test_content
|
|
|
|
finally:
|
|
# Clean up the container (use kill for faster test cleanup)
|
|
if session and session.container_id:
|
|
container_manager.close_session(session.id, kill=True)
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_session_create_with_networks(
|
|
isolate_cubbi_config, docker_test_network
|
|
):
|
|
"""Test creating a session connected to a custom network."""
|
|
session = None
|
|
|
|
try:
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create a session with the test network
|
|
session = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-network-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False, # Don't mount current directory
|
|
networks=[docker_test_network],
|
|
)
|
|
|
|
assert session is not None
|
|
assert session.status == "running"
|
|
|
|
# Wait for container initialization to complete
|
|
init_success = wait_for_container_init(session.container_id)
|
|
assert init_success, "Container initialization timed out"
|
|
|
|
# Verify the container is connected to the test network
|
|
# Use inspect to check network connections
|
|
import docker
|
|
|
|
client = docker.from_env()
|
|
container = client.containers.get(session.container_id)
|
|
container_networks = container.attrs["NetworkSettings"]["Networks"]
|
|
|
|
# Container should be connected to both the default cubbi-network and our test network
|
|
assert docker_test_network in container_networks
|
|
|
|
# Verify network interface exists in container
|
|
network_interfaces = execute_command_in_container(
|
|
session.container_id, "ip link show | grep -v 'lo' | wc -l"
|
|
)
|
|
|
|
# Should have at least 2 interfaces (eth0 for cubbi-network, eth1 for test network)
|
|
assert int(network_interfaces) >= 2
|
|
|
|
finally:
|
|
# Clean up the container (use kill for faster test cleanup)
|
|
if session and session.container_id:
|
|
container_manager.close_session(session.id, kill=True)
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_session_create_with_ports(isolate_cubbi_config):
|
|
"""Test creating a session with port forwarding."""
|
|
session = None
|
|
|
|
try:
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create a session with port forwarding
|
|
session = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-ports-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False, # Don't mount current directory
|
|
ports=[8080, 9000], # Forward these ports
|
|
)
|
|
|
|
assert session is not None
|
|
assert session.status == "running"
|
|
|
|
# Verify ports are mapped
|
|
assert isinstance(session.ports, dict)
|
|
assert 8080 in session.ports
|
|
assert 9000 in session.ports
|
|
|
|
# Verify port mappings are valid (host ports should be assigned)
|
|
assert isinstance(session.ports[8080], int)
|
|
assert isinstance(session.ports[9000], int)
|
|
assert session.ports[8080] > 0
|
|
assert session.ports[9000] > 0
|
|
|
|
# Wait for container initialization to complete
|
|
init_success = wait_for_container_init(session.container_id)
|
|
assert init_success, "Container initialization timed out"
|
|
|
|
# Verify Docker port mappings using Docker client
|
|
import docker
|
|
|
|
client = docker.from_env()
|
|
container = client.containers.get(session.container_id)
|
|
container_ports = container.attrs["NetworkSettings"]["Ports"]
|
|
|
|
# Verify both ports are exposed
|
|
assert "8080/tcp" in container_ports
|
|
assert "9000/tcp" in container_ports
|
|
|
|
# Verify host port bindings exist
|
|
assert container_ports["8080/tcp"] is not None
|
|
assert container_ports["9000/tcp"] is not None
|
|
assert len(container_ports["8080/tcp"]) > 0
|
|
assert len(container_ports["9000/tcp"]) > 0
|
|
|
|
# Verify host ports match session.ports
|
|
host_port_8080 = int(container_ports["8080/tcp"][0]["HostPort"])
|
|
host_port_9000 = int(container_ports["9000/tcp"][0]["HostPort"])
|
|
assert session.ports[8080] == host_port_8080
|
|
assert session.ports[9000] == host_port_9000
|
|
|
|
finally:
|
|
# Clean up the container (use kill for faster test cleanup)
|
|
if session and session.container_id:
|
|
container_manager.close_session(session.id, kill=True)
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_session_create_no_ports(isolate_cubbi_config):
|
|
"""Test creating a session without port forwarding."""
|
|
session = None
|
|
|
|
try:
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create a session without ports
|
|
session = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-no-ports-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False, # Don't mount current directory
|
|
ports=[], # No ports
|
|
)
|
|
|
|
assert session is not None
|
|
assert session.status == "running"
|
|
|
|
# Verify no ports are mapped
|
|
assert isinstance(session.ports, dict)
|
|
assert len(session.ports) == 0
|
|
|
|
# Wait for container initialization to complete
|
|
init_success = wait_for_container_init(session.container_id)
|
|
assert init_success, "Container initialization timed out"
|
|
|
|
# Verify Docker has no port mappings
|
|
import docker
|
|
|
|
client = docker.from_env()
|
|
container = client.containers.get(session.container_id)
|
|
container_ports = container.attrs["NetworkSettings"]["Ports"]
|
|
|
|
# Should have no port mappings (empty dict or None values)
|
|
for port_spec, bindings in container_ports.items():
|
|
assert bindings is None or len(bindings) == 0
|
|
|
|
finally:
|
|
# Clean up the container (use kill for faster test cleanup)
|
|
if session and session.container_id:
|
|
container_manager.close_session(session.id, kill=True)
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_session_create_with_single_port(isolate_cubbi_config):
|
|
"""Test creating a session with a single port forward."""
|
|
session = None
|
|
|
|
try:
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create a session with single port
|
|
session = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-single-port-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False, # Don't mount current directory
|
|
ports=[3000], # Single port
|
|
)
|
|
|
|
assert session is not None
|
|
assert session.status == "running"
|
|
|
|
# Verify single port is mapped
|
|
assert isinstance(session.ports, dict)
|
|
assert len(session.ports) == 1
|
|
assert 3000 in session.ports
|
|
assert isinstance(session.ports[3000], int)
|
|
assert session.ports[3000] > 0
|
|
|
|
# Wait for container initialization to complete
|
|
init_success = wait_for_container_init(session.container_id)
|
|
assert init_success, "Container initialization timed out"
|
|
|
|
client = docker.from_env()
|
|
container = client.containers.get(session.container_id)
|
|
container_ports = container.attrs["NetworkSettings"]["Ports"]
|
|
|
|
# Should have exactly one port mapping
|
|
port_mappings = {
|
|
k: v for k, v in container_ports.items() if v is not None and len(v) > 0
|
|
}
|
|
assert len(port_mappings) == 1
|
|
assert "3000/tcp" in port_mappings
|
|
|
|
finally:
|
|
# Clean up the container (use kill for faster test cleanup)
|
|
if session and session.container_id:
|
|
container_manager.close_session(session.id, kill=True)
|
|
|
|
|
|
@requires_docker
|
|
def test_integration_kill_vs_stop_speed(isolate_cubbi_config):
|
|
"""Test that kill is faster than stop for container termination."""
|
|
import time
|
|
|
|
# Get the isolated container manager
|
|
container_manager = isolate_cubbi_config["container_manager"]
|
|
|
|
# Create two identical sessions for comparison
|
|
session_stop = None
|
|
session_kill = None
|
|
|
|
try:
|
|
# Create first session (will be stopped gracefully)
|
|
session_stop = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-stop-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False,
|
|
ports=[],
|
|
)
|
|
|
|
# Create second session (will be killed)
|
|
session_kill = container_manager.create_session(
|
|
image_name="goose",
|
|
session_name=f"cubbi-test-kill-{uuid.uuid4().hex[:8]}",
|
|
mount_local=False,
|
|
ports=[],
|
|
)
|
|
|
|
assert session_stop is not None
|
|
assert session_kill is not None
|
|
|
|
# Wait for both containers to initialize
|
|
init_success_stop = wait_for_container_init(session_stop.container_id)
|
|
init_success_kill = wait_for_container_init(session_kill.container_id)
|
|
assert init_success_stop, "Stop test container initialization timed out"
|
|
assert init_success_kill, "Kill test container initialization timed out"
|
|
|
|
# Time graceful stop
|
|
start_time = time.time()
|
|
container_manager.close_session(session_stop.id, kill=False)
|
|
stop_time = time.time() - start_time
|
|
session_stop = None # Mark as cleaned up
|
|
|
|
# Time kill
|
|
start_time = time.time()
|
|
container_manager.close_session(session_kill.id, kill=True)
|
|
kill_time = time.time() - start_time
|
|
session_kill = None # Mark as cleaned up
|
|
|
|
# Kill should be faster than stop (usually by several seconds)
|
|
# We use a generous threshold since system performance can vary
|
|
assert (
|
|
kill_time < stop_time
|
|
), f"Kill ({kill_time:.2f}s) should be faster than stop ({stop_time:.2f}s)"
|
|
|
|
# Verify both methods successfully closed the containers
|
|
# (containers should no longer be in the session list)
|
|
remaining_sessions = container_manager.list_sessions()
|
|
session_ids = [s.id for s in remaining_sessions]
|
|
assert session_stop.id if session_stop else "stop-session" not in session_ids
|
|
assert session_kill.id if session_kill else "kill-session" not in session_ids
|
|
|
|
finally:
|
|
# Clean up any remaining containers
|
|
if session_stop and session_stop.container_id:
|
|
try:
|
|
container_manager.close_session(session_stop.id, kill=True)
|
|
except Exception:
|
|
pass
|
|
if session_kill and session_kill.container_id:
|
|
try:
|
|
container_manager.close_session(session_kill.id, kill=True)
|
|
except Exception:
|
|
pass
|