Replaced `os.path.exists()` + `os.path.getmtime()` with a single `os.stat()` call in `scripts/web_dashboard.py` to reduce syscalls by 50% for cache checks. Also pre-calculated static file paths at module level to avoid redundant `abspath` and `join` calls on every request.
Impact:
- Reduces filesystem operations per request.
- Improves code cleanliness by centralizing path constants.
- Verified with existing tests and manual curl check.
- Added `scripts/market_research.py` to analyze market data using Gemini.
- Added `scripts/upgrade_repo.py` to suggest code upgrades based on research.
- Added `scripts/schedule_research.py` to orchestrate periodic research.
- Updated `config/startup_config.json` to include the scheduler.
- Added `schedule` and `google-generativeai` to `requirements.txt`.
- Added `--monitor` flag to `scripts/startup.sh`.
- Fixed circular nameref warning in `scripts/startup.sh`.
- Updated `startup.sh` to pass the monitor flag to `startup_orchestrator.py`.
- Verified system startup and web dashboard accessibility.
- Created `mt5/MQL5/Include/ZoloBridge.mqh` for shared, secure (HTTPS) bridge logic with proper JSON sanitization.
- Created `ExpertMACD_Enhanced.mq5`, `ExpertMAMA_Enhanced.mq5`, and `ExpertMAPSAR_Enhanced.mq5` using standard library and ZOLO bridge.
- Refactored `EXNESS_GenX_Trader.mq5` and `SMC_TrendBreakout_MTF_EA.mq5` to use `ZoloBridge.mqh` and updated default URL to `https://genx-fx.com/api/signal`.
- Updated `scripts/deploy_docker_hub.sh` and `scripts/update_vps.sh` to support `DOCKER_USERNAME` and `DOCKER_PASSWORD` environment variables.
- Added `scripts/fix_profiles.py` to copy defaults to empty profile directories.
- Added `SECRETS_TEMPLATE.md` for credential management guidance.
- Updated EA link properties to Forge URL.
Optimize `scripts/merge_best_prs.py` by using `concurrent.futures.ThreadPoolExecutor` to close duplicate PRs in parallel. This significantly reduces execution time by masking network latency for independent API calls.
Benchmarks show a reduction from ~1.31s to ~0.25s for closing 13 PRs with simulated 0.1s latency.
💡 What: Separated the /health endpoint from the main dashboard rendering logic. It now returns a lightweight JSON response.
🎯 Why: The previous implementation rendered the full Markdown dashboard for every health check, consuming unnecessary CPU and I/O resources during frequent polling.
📊 Impact: Reduces health check processing time from file reading + markdown parsing (~milliseconds) to a simple JSON return (~microseconds).
🔬 Measurement: Verified with new test script scripts/test_web_dashboard.py and updated render.yaml/app.yaml to use the new endpoint.
- Create `app.yaml` for Google App Engine Flexible environment.
- Update `scripts/deploy_cloud.py` to support `gcp` argument with specific instructions for project `infra-outrider-snqdt`.
- Update `mt5/MQL5/Experts/SMC_TrendBreakout_MTF_EA.mq5` default `GeminiModel` to `gemini-1.5-pro` and update comments.
- Consolidate file validation logic into a single loop
- Check file size before reading content to prevent loading large files into memory
- Add error handling for file reading
- Reduce I/O operations and loop iterations
This improves the efficiency of the CI validation script, especially for repositories with many files or when large files are accidentally introduced.
- Implemented a Flask-based web dashboard in `scripts/web_dashboard.py` to serve project status and documentation.
- Integrated the web dashboard into `config/startup_config.json` as a managed component.
- Updated `requirements.txt` with `flask` and `markdown` dependencies.
- Added `/health` endpoint for cloud platform health checks.
- Verified system integrity with existing validation and testing scripts.
Refactored `scripts/test_automation.py` to run independent integration tests in parallel using `concurrent.futures.ProcessPoolExecutor`.
* 💡 **What:** Used `ProcessPoolExecutor` to run tests concurrently.
* 🎯 **Why:** To improve test suite performance.
* 📊 **Measured Improvement:** ~30% faster (3.3s -> 2.25s).
* **Details:** Used processes instead of threads to ensure `contextlib.redirect_stdout` works correctly for each test without interference.
- Fix cron entry to use absolute paths instead of variables
- Add parents=True to mkdir for better error handling
- Change PowerShell scheduled task to use Limited privileges for security
- All tests still passing
Co-authored-by: Mouy-leng <199350297+Mouy-leng@users.noreply.github.com>
- AUTOMATION_FEATURES.md with comprehensive feature overview
- Integration test suite for verifying all scripts
- Logs directory README for documentation
- All tests passing successfully
Co-authored-by: Mouy-leng <199350297+Mouy-leng@users.noreply.github.com>
- Python orchestrator with JSON config, logging, and monitoring
- Windows batch script for simple automation
- PowerShell script with scheduled task creation
- Linux/WSL shell script with systemd/cron support
- Detailed documentation and quick start guides
- Example custom script template
- Configuration file with MT5 paths
- Updated .gitignore for logs and temp files
Co-authored-by: Mouy-leng <199350297+Mouy-leng@users.noreply.github.com>
