Article-22608-News-Calendar-SQLite-Persistence-Layer/README.md

Article-22608-News-Calendar-SQLite-Persistence-Layer

This repository is an article-derived reference project based on the original MQL5 article. It does not claim to reproduce the full original source code unless files are explicitly attached.

## Overview

This project documents the MQL5 article implementation that replaces a compile-time CSV news source with a shared SQLite database layer for a news dashboard Expert Advisor.

The described design persists:

- economic calendar event history
- triggered trade event IDs for restart-safe deduplication
- a shared data store accessible from both live charts and the Strategy Tester

The article also adds a canvas-based download progress bar and a live-mode historical download workflow.

## Original Article

- **Article ID:** 22608
- **Author:** Allan Munene Mutiiria
- **Publication date:** 2026.05.26
- **Category:** MQL5 Articles / Algorithmic trading / Economic calendar dashboard
- **URL:** https://www.mql5.com/en/articles/22608

## Repository Purpose

This repository serves as a reference/reconstruction project for the article’s architecture and code excerpts around persistent calendar storage in MQL5.

Its purpose is to capture the database-centered redesign introduced in this article:

- replacing embedded CSV resources with SQLite
- storing the database in the common terminal folder
- sharing one persistent store between live mode and tester mode
- restoring triggered trade IDs after restart
- downloading historical calendar data on demand without recompilation

## Key Concepts

- SQLite integration through the MQL5 Database API
- common-folder database sharing via `DATABASE_OPEN_COMMON`
- schema creation for `events` and `triggered` tables
- upsert-based event persistence using `INSERT OR REPLACE`
- transaction-based bulk writes for calendar history imports
- time-window event loading for tester execution
- trade deduplication persistence using stored triggered event IDs
- pruning old rows to limit database growth
- canvas-based progress bar visualization during downloads
- unified refresh logic across live and tester workflows

## Algorithm / Architecture Summary

The article describes a two-table SQLite design:

1. **`events` table**
- stores calendar value records and display/trading metadata
- uses `value_id` as the primary key
- indexes `event_time` for fast time-range queries

2. **`triggered` table**
- stores previously traded `event_id` values
- preserves deduplication state across restarts

### Runtime flow

- **Initialization**
- open or create the shared database
- create required schema if missing
- branch by runtime mode:
- **tester mode:** load events for the configured backtest window from SQLite
- **live mode:** prune old rows, restore recent triggered IDs, optionally download historical data, then load current live events

- **Live download path**
- query calendar history from the MQL5 calendar API
- enrich values with event/country metadata
- convert them into the EA’s `NewsEvent` structure
- upsert rows in a single database transaction
- display progress using a `CCanvas` overlay

- **Trade deduplication**
- append fired event IDs to an in-memory array
- persist them into the `triggered` table
- reload the last 24 hours of triggered IDs on restart

- **Tick handling**
- use a single refresh path
- redraw only when the event set changes
- keep trade checks active on every tick

## Mentioned or Attached Files

### Explicitly attached files

- `MQL5 News Calendar EA PART 12.mq5` — main Expert Advisor entry point
- `News Core.mqh` — shared data definitions, theme/layout data, helper structures
- `News Database.mqh` — SQLite lifecycle, schema creation, upsert/load/prune/triggered-state functions, download progress support
- `News Interact.mqh` — interaction handling and lifecycle wiring
- `News Logic.mqh` — live calendar loading, filtering, trading logic, DB-backed deduplication
- `News Render.mqh` — dashboard rendering

### Files mentioned in text

- no additional files were clearly mentioned beyond the attached module set

## Statistics

- **Language:** MQL5
- **Primary storage model:** SQLite database in Common/Files
- **Tables described:** 2 (`events`, `triggered`)
- **Index described:** 1 (`idx_events_time`)
- **Article code scope:** database layer, lifecycle integration, UI progress rendering, tester/live data flow

## Tags

`MQL5`, `MetaTrader 5`, `SQLite`, `Database API`, `Economic Calendar`, `News Trading`, `Strategy Tester`, `Persistence`, `Canvas UI`, `Expert Advisor`

## Difficulty

**Intermediate to Advanced**

Requires familiarity with:

- MQL5 module organization
- calendar API usage
- database statements and transactions
- canvas rendering
- EA lifecycle and tester/live mode branching

## Limitations

- The full original repository content is not independently verified here beyond the article text and listed attachments.
- This README is based on the article narrative and code excerpts; some implementation details may depend on prior parts of the series.
- If the actual attached source files are not present in the processed repository input, this project should be treated as a reference reconstruction rather than a complete build-ready codebase.
- Installation, compilation, and execution steps are not provided here because the article excerpt does not fully specify the complete environment and dependency setup.

## Reference

Original article: [https://www.mql5.com/en/articles/22608](https://www.mql5.com/en/articles/22608)