MQL5-Google-Onedrive/REPO_SYNC_SETUP.md

Repository Sync Setup Guide

This guide explains how to set up and use the automatic repository synchronization between A6-9V/MQL5-Google-Onedrive and L6-N9 with REST API key management.

Overview

The repository sync system provides:

• Automatic synchronization between A6-9V and L6-N9 repositories
• REST API integration for secure authentication and notifications
• Bidirectional sync capabilities (push, pull, or both)
• Scheduled auto-sync every 6 hours
• Manual trigger via GitHub Actions

Prerequisites

1. GitHub Personal Access Token (PAT)

   • Required for accessing the L6-N9 repository
   • Needs repo scope for full repository access

2. REST API Endpoint (Optional)

   • For receiving sync notifications and status updates
   • Secured with API key authentication

Setup Instructions

1. Generate GitHub Personal Access Token

1. Go to GitHub Settings → Developer settings → Personal access tokens → Tokens (classic)
2. Click "Generate new token (classic)"
3. Give it a descriptive name (e.g., "Repo Sync Token")
4. Select scopes:
   • ✅ repo (Full control of private repositories)
5. Click "Generate token"
6. Copy the token immediately (you won't be able to see it again)

2. Configure GitHub Secrets

Add the following secrets to your repository (Settings → Secrets and variables → Actions):

Required Secrets:

• REPO_SYNC_TOKEN: Your GitHub Personal Access Token
  • Used for authenticating with the L6-N9 repository

Optional Secrets:

• L6_N9_REPO: Target repository name (default: A6-9V/L6-N9)
• L6_N9_SYNC_BRANCH: Target branch name (default: main)
• REST_API_KEY: Your REST API authentication key
• REST_API_URL: Your REST API endpoint URL (e.g., https://api.example.com/sync)

3. Using the GitHub CLI (Optional)

You can set secrets using the GitHub CLI:

# Set the sync token
gh secret set REPO_SYNC_TOKEN --body "ghp_your_token_here"

# Set optional configuration
gh secret set L6_N9_REPO --body "A6-9V/L6-N9"
gh secret set L6_N9_SYNC_BRANCH --body "main"

# Set REST API credentials (if using)
gh secret set REST_API_KEY --body "your_api_key_here"
gh secret set REST_API_URL --body "https://api.example.com/sync"

4. Local Environment Setup

For local testing, create a .env file:

# Copy the example file
cp .env.example .env

# Edit with your values
nano .env

Add your credentials:

# Repository Sync Configuration
REPO_SYNC_TOKEN=ghp_your_token_here
L6_N9_REPO=A6-9V/L6-N9
L6_N9_SYNC_BRANCH=main

# REST API Configuration
REST_API_KEY=your_api_key_here
REST_API_URL=https://api.example.com/sync
REST_API_SYNC_ENABLED=true

Usage

Automatic Sync

The sync runs automatically:

• On push to main/master branch: Immediately syncs changes
• Scheduled: Every 6 hours (0:00, 6:00, 12:00, 18:00 UTC)

Manual Sync via GitHub Actions

1. Go to your repository on GitHub
2. Click Actions tab
3. Select Repository Sync with L6-N9 workflow
4. Click Run workflow
5. Choose sync direction:
   • push: Copy from A6-9V to L6-N9 (default)
   • pull: Copy from L6-N9 to A6-9V
   • bidirectional: Sync both ways
6. Click Run workflow

Manual Sync via Command Line

# Check configuration
python3 scripts/repo_sync_manager.py --check-config

# Push changes to L6-N9 (default)
python3 scripts/repo_sync_manager.py

# Push changes explicitly
python3 scripts/repo_sync_manager.py --direction push

# Pull changes from L6-N9
python3 scripts/repo_sync_manager.py --direction pull

# Bidirectional sync
python3 scripts/repo_sync_manager.py --direction bidirectional

What Gets Synced

The following directories and files are synchronized:

Directories:

• mt5/MQL5/ - MT5 indicators and expert advisors
• scripts/ - Automation scripts
• config/ - Configuration files
• docs/ - Documentation

Files:

• README.md - Project documentation
• requirements.txt - Python dependencies
• .env.example - Environment variable template

REST API Integration

API Endpoint Requirements

Your REST API endpoint should:

• Accept POST requests
• Handle JSON payloads
• Support Bearer token authentication

Request Format

{
  "event": "repo_sync",
  "source": "A6-9V/MQL5-Google-Onedrive",
  "target": "A6-9V/L6-N9",
  "direction": "push",
  "timestamp": "2024-01-01T12:00:00Z",
  "branch": "main",
  "commit": "abc123...",
  "phase": "start|complete|error",
  "status": "success|failed"
}

Response Codes

• 200, 201, 202: Success
• 401: Unauthorized (check API key)
• 500: Server error

Troubleshooting

Sync Fails with Authentication Error

Problem: "Authentication failed" or "403 Forbidden"

Solution:

1. Verify REPO_SYNC_TOKEN is set correctly
2. Check token has repo scope
3. Ensure token hasn't expired
4. Regenerate token if necessary

REST API Notification Fails

Problem: API notification returns error

Solution:

1. Verify REST_API_KEY is correct
2. Check REST_API_URL is accessible
3. Review API endpoint logs
4. Note: API failures are non-critical and won't stop the sync

No Changes Synced

Problem: Workflow completes but no changes appear in L6-N9

Solution:

1. Check if there were actually changes to sync
2. Verify the sync direction is correct
3. Check L6-N9 repository permissions
4. Review workflow logs for specific errors

Target Repository Not Found

Problem: "Repository not found" error

Solution:

1. Verify L6_N9_REPO value is correct (format: owner/repo)
2. Ensure token has access to the target repository
3. Check if L6-N9 repository exists and is accessible

Security Best Practices

1. Never commit tokens or API keys to the repository
2. Use GitHub Secrets for all sensitive values
3. Rotate tokens regularly (every 60-90 days)
4. Use fine-grained permissions when possible
5. Monitor sync activity through workflow logs
6. Restrict API endpoints to known IP ranges if possible

CI/CD Integration

The sync workflow integrates with your CI/CD pipeline:

1. Pre-sync validation: Checks configuration before syncing
2. Post-sync notification: Sends status to REST API
3. Failure handling: Non-critical failures don't block the workflow
4. Summary report: Generates detailed summary in workflow logs

Monitoring

Check Sync Status

View recent syncs:

1. Go to Actions tab
2. Filter by Repository Sync with L6-N9 workflow
3. Review recent runs and their status

Workflow Logs

Each sync generates a detailed log including:

• Configuration check results
• Files and directories synced
• API notification results
• Commit information
• Success/failure status

Advanced Configuration

Custom Sync Paths

Edit .github/workflows/repo-sync.yml to customize what gets synced:

# Add or remove directories
cp -r custom-dir target-repo/custom-dir/ || true

Sync Filters

Exclude specific files or directories:

# In the sync step
rsync -av --exclude='*.log' --exclude='temp/' source/ target/

Multiple Target Repositories

Create separate workflows for different targets:

1. Copy repo-sync.yml to repo-sync-target2.yml
2. Update TARGET_REPO environment variable
3. Add corresponding secrets

Support

For issues or questions:

• Review workflow logs in GitHub Actions
• Check GitHub Actions documentation
• Review REST API integration guide
• Contact repository maintainers

Related Documentation

• GitHub Secrets Setup Guide
• CI/CD Workflow Guide
• OneDrive Sync Guide
• Environment Variables Guide