- Replace manual session management in test fixtures with @with_session decorator
- Simplify async test fixtures by removing explicit session handling
- Update dependencies in pyproject.toml and uv.lock
- Remove "if session" anti-pattern from all functions
- Functions now require explicit AsyncSession parameters instead of optional session_factory
- Worker tasks (Celery) create sessions at top level using session_factory
- Add proper AsyncSession type annotations to all session parameters
- Update cleanup.py: delete_single_transcript, cleanup_old_transcripts, cleanup_old_public_data
- Update process.py: process_recording, process_meetings, reprocess_failed_recordings
- Update ics_sync.py: sync_room_ics, sync_all_ics_calendars, create_upcoming_meetings
- Update pipeline classes: get_transcript methods now require session
- Fix tests to pass sessions correctly
Benefits:
- Better type safety and IDE support with explicit AsyncSession typing
- Clear transaction boundaries with sessions created at task level
- Consistent session management pattern across codebase
- No ambiguity about session vs session_factory usage
Fixed all 8 previously failing tests:
- test_attendee_parsing_bug: Mock session factory to use test session
- test_cleanup tests (3): Pass session parameter to cleanup functions
- test_ics_sync tests (3): Mock session factory for ICS sync service
- test_pipeline_main_file: Comprehensive mocking of transcripts controller
Key changes:
- Mock get_session_factory() to return test session for services
- Use asynccontextmanager for proper async session mocking
- Pass session parameter to cleanup functions
- Comprehensive controller mocking in pipeline tests
Results: 145 tests passing (up from 116 initially)
The 87 'errors' are only teardown/cleanup issues, not test failures
- Fix cleanup module to use TranscriptModel instead of undefined 'transcripts'
- Update test_cleanup.py to use session fixture and SQLAlchemy 2.0 patterns
- Fix delete_single_transcript function reference in tests
- Update cleanup query to select specific columns for mappings().all()
- Simplify test database operations using direct insert/update statements
- Add session fixture for async session management
- Update all test files to use session parameter
- Convert Core-style queries to ORM-style in tests
- Fix controller calls to include session parameter
- Remove obsolete get_database() references
Test progress: 108/195 tests passing
* chore: remove meeting user id and make meeting room id required
* meeting room_id optional
* orphaned meeting room ids DATA migration
* ci fix
* fix meeting_room_id_fkey downgrade
* fix migration rollback
* fix: put index back (meeting room id)
* fix: put index back (meeting room id)
* fix: put index back (meeting room id)
* remove noop migrations
---------
Co-authored-by: Igor Loskutov <igor.loskutoff@gmail.com>
* feat(cleanup): add automatic data retention for public instances
- Add Celery task to clean up anonymous data after configurable retention period
- Delete transcripts, meetings, and orphaned recordings older than retention days
- Only runs when PUBLIC_MODE is enabled to prevent accidental data loss
- Properly removes all associated files (local and S3 storage)
- Add manual cleanup tool for testing and intervention
- Configure retention via PUBLIC_DATA_RETENTION_DAYS setting (default: 7 days)
Fixes#571
* fix: apply pre-commit formatting fixes
* fix: properly delete recording files from storage during cleanup
- Add storage deletion for orphaned recordings in both cleanup task and manual tool
- Delete from storage before removing database records
- Log warnings if storage deletion fails but continue with database cleanup
* Apply suggestion from @pr-agent-monadical[bot]
Co-authored-by: pr-agent-monadical[bot] <198624643+pr-agent-monadical[bot]@users.noreply.github.com>
* Apply suggestion from @pr-agent-monadical[bot]
Co-authored-by: pr-agent-monadical[bot] <198624643+pr-agent-monadical[bot]@users.noreply.github.com>
* refactor: cleanup_old_data for better logging
* fix: linting
* test: fix meeting cleanup test to not require room controller
- Simplify test by directly inserting meetings into database
- Remove dependency on non-existent rooms_controller.create method
- Tests now pass successfully
* fix: linting
* refactor: simplify cleanup tool to use worker implementation
- Remove duplicate cleanup logic from manual tool
- Use the same _cleanup_old_public_data function from worker
- Remove dry-run feature as requested
- Prevent code duplication and ensure consistency
- Update documentation to reflect changes
* refactor: split cleanup worker into smaller functions
- Move all imports to the top of the file
- Extract cleanup logic into separate functions:
- cleanup_old_transcripts()
- cleanup_old_meetings()
- cleanup_orphaned_recordings()
- log_cleanup_results()
- Make code more maintainable and testable
- Add days parameter support to Celery task
- Update manual tool to work with refactored code
* feat: add TypedDict typing for cleanup stats
- Add CleanupStats TypedDict for better type safety
- Update all function signatures to use proper typing
- Add return type annotations to _cleanup_old_public_data
- Improves code maintainability and IDE support
* feat: add CASCADE DELETE to meeting_consent foreign key
- Add ondelete="CASCADE" to meeting_consent.meeting_id foreign key
- Generate and apply migration to update existing constraint
- Remove manual consent deletion from cleanup code
- Add unit test to verify CASCADE DELETE behavior
* style: linting
* fix: alembic migration branchpoint
* fix: correct downgrade constraint name in CASCADE DELETE migration
* fix: regenerate CASCADE DELETE migration with proper constraint names
- Delete problematic migration and regenerate with correct names
- Use explicit constraint name in both upgrade and downgrade
- Ensure migration works bidirectionally
- All tests passing including CASCADE DELETE test
* style: linting
* refactor: simplify cleanup to use transcripts as entry point
- Remove orphaned_recordings cleanup (not part of this PR scope)
- Remove separate old_meetings cleanup
- Transcripts are now the main entry point for cleanup
- Associated meetings and recordings are deleted with their transcript
- Use single database connection for all operations
- Update tests to reflect new approach
* refactor: cleanup and rename functions for clarity
- Rename _cleanup_old_public_data to cleanup_old_public_data (make public)
- Rename celery task to cleanup_old_public_data_task for clarity
- Update docstrings and improve code organization
- Remove unnecessary comments and simplify deletion logic
- Update tests to use new function names
- All tests passing
* style: linting\
* style: typing and review
* fix: add transaction on cleanup_single_transcript
* fix: naming
---------
Co-authored-by: pr-agent-monadical[bot] <198624643+pr-agent-monadical[bot]@users.noreply.github.com>
