MQL5-Google-Onedrive/REPO_SYNC_SETUP.md

# Repository Sync Setup Guide

This guide explains how to set up and use the automatic repository synchronization between A6-9V/MQL5-Google-Onedrive and L6-N9 with REST API key management.

## Overview

The repository sync system provides:
- **Automatic synchronization** between A6-9V and L6-N9 repositories
- **REST API integration** for secure authentication and notifications
- **Bidirectional sync** capabilities (push, pull, or both)
- **Scheduled auto-sync** every 6 hours
- **Manual trigger** via GitHub Actions

## Prerequisites

1. **GitHub Personal Access Token (PAT)**
   - Required for accessing the L6-N9 repository
   - Needs `repo` scope for full repository access

2. **REST API Endpoint (Optional)**
   - For receiving sync notifications and status updates
   - Secured with API key authentication

## Setup Instructions

### 1. Generate GitHub Personal Access Token

1. Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)
2. Click "Generate new token (classic)"
3. Give it a descriptive name (e.g., "Repo Sync Token")
4. Select scopes:
   - ✅ `repo` (Full control of private repositories)
5. Click "Generate token"
6. **Copy the token immediately** (you won't be able to see it again)

### 2. Configure GitHub Secrets

Add the following secrets to your repository (Settings → Secrets and variables → Actions):

#### Required Secrets:
- **`REPO_SYNC_TOKEN`**: Your GitHub Personal Access Token
  - Used for authenticating with the L6-N9 repository

#### Optional Secrets:
- **`L6_N9_REPO`**: Target repository name (default: `A6-9V/L6-N9`)
- **`L6_N9_SYNC_BRANCH`**: Target branch name (default: `main`)
- **`REST_API_KEY`**: Your REST API authentication key
- **`REST_API_URL`**: Your REST API endpoint URL (e.g., `https://api.example.com/sync`)

### 3. Using the GitHub CLI (Optional)

You can set secrets using the GitHub CLI:

```bash
# Set the sync token
gh secret set REPO_SYNC_TOKEN --body "ghp_your_token_here"

# Set optional configuration
gh secret set L6_N9_REPO --body "A6-9V/L6-N9"
gh secret set L6_N9_SYNC_BRANCH --body "main"

# Set REST API credentials (if using)
gh secret set REST_API_KEY --body "your_api_key_here"
gh secret set REST_API_URL --body "https://api.example.com/sync"
```

### 4. Local Environment Setup

For local testing, create a `.env` file:

```bash
# Copy the example file
cp .env.example .env

# Edit with your values
nano .env
```

Add your credentials:

```env
# Repository Sync Configuration
REPO_SYNC_TOKEN=ghp_your_token_here
L6_N9_REPO=A6-9V/L6-N9
L6_N9_SYNC_BRANCH=main

# REST API Configuration
REST_API_KEY=your_api_key_here
REST_API_URL=https://api.example.com/sync
REST_API_SYNC_ENABLED=true
```

## Usage

### Automatic Sync

The sync runs automatically:
- **On push to main/master branch**: Immediately syncs changes
- **Scheduled**: Every 6 hours (0:00, 6:00, 12:00, 18:00 UTC)

### Manual Sync via GitHub Actions

1. Go to your repository on GitHub
2. Click **Actions** tab
3. Select **Repository Sync with L6-N9** workflow
4. Click **Run workflow**
5. Choose sync direction:
   - **push**: Copy from A6-9V to L6-N9 (default)
   - **pull**: Copy from L6-N9 to A6-9V
   - **bidirectional**: Sync both ways
6. Click **Run workflow**

### Manual Sync via Command Line

```bash
# Check configuration
python3 scripts/repo_sync_manager.py --check-config

# Push changes to L6-N9 (default)
python3 scripts/repo_sync_manager.py

# Push changes explicitly
python3 scripts/repo_sync_manager.py --direction push

# Pull changes from L6-N9
python3 scripts/repo_sync_manager.py --direction pull

# Bidirectional sync
python3 scripts/repo_sync_manager.py --direction bidirectional
```

## What Gets Synced

The following directories and files are synchronized:

### Directories:
- `mt5/MQL5/` - MT5 indicators and expert advisors
- `scripts/` - Automation scripts
- `config/` - Configuration files
- `docs/` - Documentation

### Files:
- `README.md` - Project documentation
- `requirements.txt` - Python dependencies
- `.env.example` - Environment variable template

## REST API Integration

### API Endpoint Requirements

Your REST API endpoint should:
- Accept POST requests
- Handle JSON payloads
- Support Bearer token authentication

### Request Format

```json
{
  "event": "repo_sync",
  "source": "A6-9V/MQL5-Google-Onedrive",
  "target": "A6-9V/L6-N9",
  "direction": "push",
  "timestamp": "2024-01-01T12:00:00Z",
  "branch": "main",
  "commit": "abc123...",
  "phase": "start|complete|error",
  "status": "success|failed"
}
```

### Response Codes

- `200`, `201`, `202`: Success
- `401`: Unauthorized (check API key)
- `500`: Server error

## Troubleshooting

### Sync Fails with Authentication Error

**Problem**: "Authentication failed" or "403 Forbidden"

**Solution**:
1. Verify `REPO_SYNC_TOKEN` is set correctly
2. Check token has `repo` scope
3. Ensure token hasn't expired
4. Regenerate token if necessary

### REST API Notification Fails

**Problem**: API notification returns error

**Solution**:
1. Verify `REST_API_KEY` is correct
2. Check `REST_API_URL` is accessible
3. Review API endpoint logs
4. Note: API failures are non-critical and won't stop the sync

### No Changes Synced

**Problem**: Workflow completes but no changes appear in L6-N9

**Solution**:
1. Check if there were actually changes to sync
2. Verify the sync direction is correct
3. Check L6-N9 repository permissions
4. Review workflow logs for specific errors

### Target Repository Not Found

**Problem**: "Repository not found" error

**Solution**:
1. Verify `L6_N9_REPO` value is correct (format: `owner/repo`)
2. Ensure token has access to the target repository
3. Check if L6-N9 repository exists and is accessible

## Security Best Practices

1. **Never commit tokens or API keys** to the repository
2. **Use GitHub Secrets** for all sensitive values
3. **Rotate tokens regularly** (every 60-90 days)
4. **Use fine-grained permissions** when possible
5. **Monitor sync activity** through workflow logs
6. **Restrict API endpoints** to known IP ranges if possible

## CI/CD Integration

The sync workflow integrates with your CI/CD pipeline:

1. **Pre-sync validation**: Checks configuration before syncing
2. **Post-sync notification**: Sends status to REST API
3. **Failure handling**: Non-critical failures don't block the workflow
4. **Summary report**: Generates detailed summary in workflow logs

## Monitoring

### Check Sync Status

View recent syncs:
1. Go to **Actions** tab
2. Filter by **Repository Sync with L6-N9** workflow
3. Review recent runs and their status

### Workflow Logs

Each sync generates a detailed log including:
- Configuration check results
- Files and directories synced
- API notification results
- Commit information
- Success/failure status

## Advanced Configuration

### Custom Sync Paths

Edit `.github/workflows/repo-sync.yml` to customize what gets synced:

```yaml
# Add or remove directories
cp -r custom-dir target-repo/custom-dir/ || true
```

### Sync Filters

Exclude specific files or directories:

```yaml
# In the sync step
rsync -av --exclude='*.log' --exclude='temp/' source/ target/
```

### Multiple Target Repositories

Create separate workflows for different targets:
1. Copy `repo-sync.yml` to `repo-sync-target2.yml`
2. Update `TARGET_REPO` environment variable
3. Add corresponding secrets

## Support

For issues or questions:
- Review workflow logs in GitHub Actions
- Check [GitHub Actions documentation](https://docs.github.com/en/actions)
- Review [REST API integration guide](https://docs.github.com/en/rest)
- Contact repository maintainers

## Related Documentation

- [GitHub Secrets Setup Guide](GITHUB_SECRETS_SETUP.md)
- [CI/CD Workflow Guide](docs/CD_WORKFLOW_GUIDE.md)
- [OneDrive Sync Guide](README.md#github-automation)
- [Environment Variables Guide](.env.example)