SKILL.md
Conversation Data Management Skill
Use this skill when working with conversation logs, training data, and AI memory management.
Workflow Steps
- Data Source Analysis
- Identify conversation log formats and structures - Parse different export formats (JSON, text, database dumps) - Extract user/assistant message pairs
- Data Cleaning and Validation
- Remove corrupted or incomplete conversations - Validate message content and metadata - Ensure chronological ordering
- Database Schema Design
- Design tables for user/assistant message storage - Include metadata (timestamps, session IDs, user IDs) - Plan indexing for efficient retrieval
- Import Implementation
- Create parsing scripts for different data formats - Handle batch imports with progress tracking - Implement error handling for malformed data
- Memory Integration
- Connect conversation history to AI context - Implement session-based retrieval - Test conversation continuity
Key Principles
- Data Integrity: Preserve original conversation context and metadata
- Scalability: Design for large conversation datasets
- Privacy: Handle user data appropriately (anonymized when needed)
- Performance: Optimize queries for real-time conversation retrieval
Common Patterns
- Session Grouping: Organize conversations by session for continuity
- User Association: Link conversations to users when available
- Metadata Preservation: Keep timestamps, sources, and context
- Batch Processing: Handle large imports efficiently
Data Formats Handled
- Training Logs: Structured text with Provider/Question/Answer format
- JSON Exports: Standard conversation JSON structures
- Database Dumps: SQL exports with relationship preservation
- API Logs: Raw request/response pairs
Quality Assurance
- Completeness: Ensure all conversations imported successfully
- Accuracy: Verify message content and metadata integrity
- Performance: Test retrieval speed with large datasets
- Integration: Validate AI can access and use conversation memory</content>
<parameter name="filePath">/home/koo/github/Luxuryshoppingwebsite/.github/skills/conversation-data-management/SKILL.md