cubbi/SPECIFICATIONS.md

MC - Monadical AI Container Tool

Overview

MC (Monadical Container) is a command-line tool for managing ephemeral containers that run AI tools and development environments. It works with both local Docker and a dedicated remote web service that manages containers in a Docker-in-Docker (DinD) environment.

Technology Stack

MC Service

• Web Framework: FastAPI for high-performance, async API endpoints
• Package Management: uv (Astral) for dependency management
• Database: SQLite for development, PostgreSQL for production
• Container Management: Docker SDK for Python
• Authentication: OAuth 2.0 integration with Authentik

MC CLI

• Language: Python
• Package Management: uv for dependency management
• Distribution: Standalone binary via PyInstaller or similar
• Configuration: YAML for configuration files

System Architecture

Components

1. CLI Tool (mc): The command-line interface users interact with
2. MC Service: A web service that handles remote container execution
3. Container Drivers: Predefined container templates for various AI tools

Architecture Diagram

┌─────────────┐           ┌─────────────────────────┐
│             │           │                         │
│  MC CLI     │◄─────────►│  Local Docker Daemon    │
│  (mc)       │           │                         │
│             │           └─────────────────────────┘
└──────┬──────┘
       │
       │ REST API
       │
┌──────▼──────┐           ┌─────────────────────────┐
│             │           │                         │
│ MC Service  │◄─────────►│  Docker-in-Docker       │
│ (Web API)   │           │                         │
│             │           └─────────────────────────┘
└─────────────┘
       │
       ├──────────────┬───────────────┐
       │              │               │
┌──────▼──────┐ ┌─────▼─────┐  ┌──────▼──────┐
│             │ │           │  │             │
│  Fluentd    │ │ Langfuse  │  │ Other       │
│  Logging    │ │ Logging   │  │ Services    │
│             │ │           │  │             │
└─────────────┘ └───────────┘  └─────────────┘

Core Concepts

• Session: An active container instance with a specific driver
• Driver: A predefined container template with specific AI tools installed
• Remote: A configured MC service instance

User Configuration

MC supports user-specific configuration via a YAML file located at ~/.config/mc/config.yaml. This provides a way to set default values, store service credentials, and customize behavior without modifying code.

Configuration File Structure

# ~/.config/mc/config.yaml
defaults:
  driver: "goose"  # Default driver to use
  connect: true    # Automatically connect after creating session
  mount_local: true  # Mount local directory by default

services:
  # Service credentials with simplified naming
  # These are mapped to environment variables in containers
  langfuse:
    url: ""  # Will be set by the user
    public_key: "pk-lf-..."
    secret_key: "sk-lf-..."

  openai:
    api_key: "sk-..."

  anthropic:
    api_key: "sk-ant-..."

  openrouter:
    api_key: "sk-or-..."

docker:
  network: "mc-network"  # Docker network to use
  socket: "/var/run/docker.sock"  # Docker socket path

remote:
  default: "production"  # Default remote to use
  endpoints:
    production:
      url: "https://mc.monadical.com"
      auth_method: "oauth"
    staging:
      url: "https://mc-staging.monadical.com"
      auth_method: "oauth"

ui:
  colors: true  # Enable/disable colors in terminal output
  verbose: false  # Enable/disable verbose output
  table_format: "grid"  # Table format for session listings

Environment Variable Mapping

The simplified configuration names are mapped to environment variables:

Config Path	Environment Variable
services.langfuse.url	LANGFUSE_URL
services.langfuse.public_key	LANGFUSE_INIT_PROJECT_PUBLIC_KEY
services.langfuse.secret_key	LANGFUSE_INIT_PROJECT_SECRET_KEY
services.openai.api_key	OPENAI_API_KEY
services.anthropic.api_key	ANTHROPIC_API_KEY
services.openrouter.api_key	OPENROUTER_API_KEY

Environment Variable Precedence

1. Command-line arguments (-e KEY=VALUE) take highest precedence
2. User config file takes second precedence
3. System defaults take lowest precedence

Security Considerations

• Configuration file permissions are set to 600 (user read/write only)
• Sensitive values can be referenced from environment variables: ${ENV_VAR}
• API keys and secrets are never logged or displayed in verbose output

CLI Configuration Commands

# View entire configuration
mc config list

# Get specific configuration value
mc config get defaults.driver

# Set configuration value (using simplified naming)
mc config set langfuse.url "https://cloud.langfuse.com"
mc config set openai.api_key "sk-..."

# Reset configuration to defaults
mc config reset

CLI Tool Commands

Basic Commands

# Create a new session locally (shorthand)
mc

# List active sessions on local system
mc session list

# Create a new session locally
mc session create [OPTIONS]

# Create a session with a specific driver
mc session create --driver goose

# Create a session with a specific project repository
mc session create --driver goose --project github.com/hello/private

# Create a session with a project (shorthand)
mc git@github.com:hello/private

# Close a specific session
mc session close <id>

# Connect to an existing session
mc session connect <id>

# Stop the current session (from inside the container)
mc stop

Remote Management

# Add a remote MC service
mc remote add <name> <url>

# List configured remote services
mc remote list

# Remove a remote service
mc remote remove <name>

# Authenticate with a remote service
mc -r <remote_name> auth

# Create a session on a remote service
mc -r <remote_name> [session create]

# List sessions on a remote service
mc -r <remote_name> session list

Environment Variables

# Set environment variables for a session
mc session create -e VAR1=value1 -e VAR2=value2

# Set environment variables for a remote session
mc -r <remote_name> session create -e VAR1=value1

Logging

# Stream logs from a session
mc session logs <id>

# Stream logs with follow option
mc session logs <id> -f

MC Service Specification

Overview

The MC Service is a web service that manages ephemeral containers in a Docker-in-Docker environment. It provides a REST API for container lifecycle management, authentication, and real-time log streaming.

API Endpoints

Authentication

POST /auth/login        - Initiate Authentik authentication flow
POST /auth/callback     - Handle Authentik OAuth callback
POST /auth/refresh      - Refresh an existing token
POST /auth/logout       - Invalidate current token

Authentik Integration

The MC Service integrates with Authentik at https://authentik.monadical.io using OAuth 2.0:

1. Application Registration:

   • MC Service is registered as an OAuth application in Authentik
   • Configured with redirect URI to /auth/callback
   • Assigned appropriate scopes for user identification

2. Authentication Flow:

   • User initiates authentication via CLI
   • MC CLI opens browser to Authentik authorization URL
   • User logs in through Authentik's interface
   • Authentik redirects to callback URL with authorization code
   • MC Service exchanges code for access and refresh tokens
   • CLI receives and securely stores tokens

3. Token Management:

   • Access tokens used for API authorization
   • Refresh tokens used to obtain new access tokens
   • Tokens are encrypted at rest in CLI configuration

Sessions

GET /sessions - List all sessions
POST /sessions - Create a new session
GET /sessions/{id} - Get session details
DELETE /sessions/{id} - Terminate a session
POST /sessions/{id}/connect - Establish connection to session
GET /sessions/{id}/logs - Stream session logs

Drivers

GET /drivers - List available drivers
GET /drivers/{name} - Get driver details

Projects

GET /projects - List all projects
POST /projects - Add a new project
GET /projects/{id} - Get project details
PUT /projects/{id} - Update project details
DELETE /projects/{id} - Remove a project

Service Configuration

# mc-service.yaml
server:
  port: 3000
  host: 0.0.0.0

docker:
  socket: /var/run/docker.sock
  network: mc-network

auth:
  provider: authentik
  url: https://authentik.monadical.io
  clientId: mc-service

logging:
  providers:
    - type: fluentd
      url: http://fluentd.example.com:24224
    - type: langfuse
      url: https://cloud.langfuse.com
      public_key: ${LANGFUSE_INIT_PROJECT_PUBLIC_KEY}
      secret_key: ${LANGFUSE_INIT_PROJECT_SECRET_KEY}

drivers:
  - name: goose
    image: monadical/mc-goose:latest
  - name: aider
    image: monadical/mc-aider:latest
  - name: claude-code
    image: monadical/mc-claude-code:latest

projects:
  storage:
    type: encrypted
    key: ${PROJECT_ENCRYPTION_KEY}
  default_ssh_scan:
    - github.com
    - gitlab.com
    - bitbucket.org

Docker-in-Docker Implementation

The MC Service runs in a container with access to the host's Docker socket, allowing it to create and manage sibling containers. This approach provides:

1. Isolation between containers
2. Simple lifecycle management
3. Resource constraints for security

Connection Handling

For remote connections to containers, the service provides two methods:

1. WebSocket Terminal: Browser-based terminal access
2. SSH Server: Each container runs an SSH server for CLI access

Logging Implementation

The MC Service implements log collection and forwarding:

1. Container logs are captured using Docker's logging drivers
2. Logs are forwarded to configured providers (Fluentd, Langfuse)
3. Real-time log streaming is available via WebSockets

Project Management

Persistent Project Configuration

MC provides persistent storage for project-specific configurations that need to survive container restarts. This is implemented through a dedicated volume mount and symlink system:

1. Configuration Storage:

   • Each project has a dedicated configuration directory on the host at ~/.mc/projects/<project-hash>/config
   • For projects specified by URL, the hash is derived from the repository URL
   • For local projects, the hash is derived from the absolute path of the local directory
   • This directory is mounted into the container at /mc-config

2. Driver Configuration:

   • Each driver can specify configuration files/directories that should persist across sessions
   • These are defined in the driver's mc-driver.yaml file in the persistent_configs section
   • Example for Goose driver:

     persistent_configs:
       - source: "/app/.goose"         # Path in container
         target: "/mc-config/goose"    # Path in persistent storage
         type: "directory"             # directory or file
         description: "Goose memory and configuration"
     

3. Automatic Symlinking:

   • During container initialization, the system:
     • Creates all target directories in the persistent storage
     • Creates symlinks from the source paths to the target paths
   • This makes the persistence transparent to the application

4. Environment Variables:

   • Container has access to configuration location via environment variables:

     MC_CONFIG_DIR=/mc-config
     MC_DRIVER_CONFIG_DIR=/mc-config/<driver-name>
     

This ensures that important configurations like Goose's memory store, authentication tokens, and other state information persist between container sessions while maintaining isolation between different projects.

Adding Projects

Users can add projects with associated credentials:

# Add a project with SSH key
mc project add github.com/hello/private --ssh-key ~/.ssh/id_ed25519

# Add a project with token authentication
mc project add github.com/hello/private --token ghp_123456789

# List all projects
mc project list

# Remove a project
mc project remove github.com/hello/private

Project Configuration

Projects are stored in the MC service and referenced by their repository URL. The configuration includes:

# Project configuration
id: github.com/hello/private
url: git@github.com:hello/private.git
type: git
auth:
  type: ssh
  key: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ...encrypted key data...
    -----END OPENSSH PRIVATE KEY-----
  public_key: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI...

Driver Implementation

Driver Structure

Each driver is a Docker image with a standardized structure:

/
├── entrypoint.sh      # Container initialization
├── mc-init.sh         # Standardized initialization script
├── mc-driver.yaml     # Driver metadata and configuration
├── tool/              # AI tool installation
└── ssh/               # SSH server configuration

Standardized Initialization Script

All drivers include a standardized mc-init.sh script that handles common initialization tasks:

#!/bin/bash

# Project initialization
if [ -n "$MC_PROJECT_URL" ]; then
    echo "Initializing project: $MC_PROJECT_URL"

    # Set up SSH key if provided
    if [ -n "$MC_GIT_SSH_KEY" ]; then
        mkdir -p ~/.ssh
        echo "$MC_GIT_SSH_KEY" > ~/.ssh/id_ed25519
        chmod 600 ~/.ssh/id_ed25519
        ssh-keyscan github.com >> ~/.ssh/known_hosts 2>/dev/null
    fi

    # Set up token if provided
    if [ -n "$MC_GIT_TOKEN" ]; then
        git config --global credential.helper store
        echo "https://$MC_GIT_TOKEN:x-oauth-basic@github.com" > ~/.git-credentials
    fi

    # Clone repository
    git clone $MC_PROJECT_URL /app
    cd /app

    # Run project-specific initialization if present
    if [ -f "/app/.mc/init.sh" ]; then
        bash /app/.mc/init.sh
    fi
fi

# Driver-specific initialization continues...

Driver Configuration (mc-driver.yaml)

name: goose
description: Goose with MCP servers
version: 1.0.0
maintainer: team@monadical.com

init:
  pre_command: /mc-init.sh
  command: /entrypoint.sh

environment:
  - name: MCP_HOST
    description: MCP server host
    required: true
    default: http://localhost:8000

  - name: GOOSE_ID
    description: Goose instance ID
    required: false

  # Project environment variables
  - name: MC_PROJECT_URL
    description: Project repository URL
    required: false

  - name: MC_PROJECT_TYPE
    description: Project repository type (git, svn, etc.)
    required: false
    default: git

  - name: MC_GIT_SSH_KEY
    description: SSH key for Git authentication
    required: false
    sensitive: true

  - name: MC_GIT_TOKEN
    description: Token for Git authentication
    required: false
    sensitive: true

ports:
  - 8000   # Main application
  - 22     # SSH server

volumes:
  - mountPath: /app
    description: Application directory

persistent_configs:
  - source: "/app/.goose"
    target: "/mc-config/goose"
    type: "directory"
    description: "Goose memory and configuration"

Example Built-in Drivers

1. goose: Goose with MCP servers
2. aider: Aider coding assistant
3. claude-code: Claude Code environment
4. custom: Custom Dockerfile support

Security Considerations

1. Container Isolation: Each session runs in an isolated container
2. Authentication: Integration with Authentik for secure authentication
3. Resource Limits: Configurable CPU, memory, and storage limits
4. Network Isolation: Internal Docker network for container-to-container communication
5. Encrypted Connections: TLS for API connections and SSH for terminal access

Deployment

MC Service Deployment

# docker-compose.yml for MC Service
version: '3.8'

services:
  mc-service:
    image: monadical/mc-service:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./config:/app/config
    ports:
      - "3000:3000"
    environment:
      - AUTH_URL=https://authentik.monadical.io
      - LANGFUSE_API_KEY=your_api_key
    networks:
      - mc-network

networks:
  mc-network:
    driver: bridge

Project Repository Integration Workflow

Adding a Project Repository

1. User adds project repository with authentication:

   mc project add github.com/hello/private --ssh-key ~/.ssh/id_ed25519
   

2. MC CLI reads the SSH key, encrypts it, and sends to MC Service

3. MC Service stores the project configuration securely

Using a Project in a Session

1. User creates a session with a project:

   mc -r monadical git@github.com:hello/private
   

2. MC Service:

   • Identifies the project from the URL
   • Retrieves project authentication details
   • Sets up environment variables:

     MC_PROJECT_URL=git@github.com:hello/private
     MC_PROJECT_TYPE=git
     MC_GIT_SSH_KEY=<contents of the SSH key>
     

   • Creates container with these environment variables

3. Container initialization:

   • The standardized mc-init.sh script detects the project environment variables
   • Sets up SSH key or token authentication
   • Clones the repository to /app
   • Runs any project-specific initialization scripts

4. User can immediately begin working with the repository

Implementation Roadmap

1. Phase 1: Local CLI tool with Docker integration
2. Phase 2: MC Service REST API with basic container management
3. Phase 3: Authentication and secure connections
4. Phase 4: Project management functionality
5. Phase 5: Driver implementation (Goose, Aider, Claude Code)
6. Phase 6: Logging integration with Fluentd and Langfuse
7. Phase 7: CLI remote connectivity improvements
8. Phase 8: Additional drivers and extensibility features