MQLArticles/Utils/File

← Utils

Utils/File

Folder-level file operations. Everything lives in namespace TSN.

Main features

• Clean a folder: recursively delete the files of a directory that match the include filter and not the exclude one (CFolderOpsClean). A sub-folder whose path matches the include filter is cleaned entirely.
• Encrypt / decrypt a folder: every matching file in a directory (recursive), using Fernet (CFolderOpsEncrypt).
• Include / exclude filters: shared by all of them (CFolderOpsBase): set them from a string like "*.tmp|*.log", or load/save them as JSON (Load, Save).

Basic usage

Clean a folder

TSN::CFolderOpsClean clean;
clean.SetArrIncluyed("*.tmp|*.log"); // files to delete (patterns, '|' separated)
clean.CleanFolder("MQLArticles\\Temp\\", FILE_COMMON); // directory (with trailing \), common flag

Encrypt and decrypt

uchar key[CRYPTOBYLEO_FERNET_KEYBYTES];
// ... fill the key ...

TSN::CFernet fernet;
fernet.Key(key);

TSN::CFolderOpsEncrypt folder_ops;
folder_ops.Fernet(&fernet);

// encrypt: (dir, common flag, delete original, extension of the encrypted file)
folder_ops.SetArrIncluyed("*.txt|*.csv");
folder_ops.SetInitialPass(CRYPTOBYLEO_FERNET_UTCTIME);   // timestamp stored in every token
folder_ops.EncryptFolder<TSN::CURandomPRNG<Xoshiro256>>("MQLArticles\\Data\\", true, true, ".enc");

// decrypt: (dir, common flag, delete encrypted file, ttl, extension of the decrypted file)
folder_ops.SetArrIncluyed("*.txt.enc|*.csv.enc");
folder_ops.SetInitialPass(CRYPTOBYLEO_FERNET_UTCTIME);   // "now" for the ttl check
folder_ops.DecryptFolder("MQLArticles\\Data\\", true, true, INT_MAX, "-");

SetInitialPass(now) stores now in the object: it is the timestamp written into the tokens when encrypting and the current time used for the TTL check when decrypting. ttl is in seconds (INT_MAX disables the check). Both operations use the Fernet rules for ext_encrypt / ext_decrypt, see its README.

Warning: delete_prev_file = true with an empty extension deletes the result. Fernet saves the output first and then deletes the source, so if both have the same name (ext_encrypt / ext_decrypt empty, or ext_decrypt without . and -) the freshly written file is deleted. Use a non-empty extension, or delete_prev_file = false.

CFernet and the PRNG come from external TSN libraries (not part of Utils). Full runnable scripts in FolderOps/Test/.

How filtering works

For every entry found while walking the tree:

• Files: if the file matches the exclude filter it is skipped. Otherwise it is processed only if it matches the include filter.
• Folders: if a folder matches the include filter, everything under it is processed, ignoring both filters, until the walk leaves that folder. Otherwise the walk just descends into it.
• Empty include filter = nothing is processed. There is no "match all" default; add a pattern that matches everything.

All three operations (CleanFolder, EncryptFolder, DecryptFolder) behave the same way.

Filter patterns

Multiple patterns are separated by | (or any separator you pass to SetArrIncluyed / SetArrExcluyed).

Pattern syntax is custom, not standard glob. See the StrMatch module for the supported patterns.

What is matched against the patterns:

• The full path relative to the root (MQL5\Files or Common\Files), not just the file name. Use \ as separator.
• Folder paths end with \.

Paths and flags

• init_dir is relative to MQL5\Files (or Common\Files with the common flag) and must end with \.
• CleanFolder takes the flag as an int (FILE_COMMON or 0), EncryptFolder / DecryptFolder and Load take a bool, and Save takes extra int file flags.

Config files (JSON)

Save / Load use two arrays, incluyed and excluyed:

{ "incluyed": ["..."], "excluyed": ["..."] }

A missing field is treated as an empty list (a warning is recorded). Remember that an empty include list means nothing is processed.

Error handling

• A failure on one file never aborts the run; the remaining files are still processed.
• Problems are recorded as warnings by all three classes. They are reset by SetInitialPass() and Load().
• A warning only stores its kind, not the file that caused it. At most 16 are kept per run (FILEFOLDEROPS_MAX_WARNS); the following ones are dropped silently. Define MQLARTICLES_FOLDEROPS_ENABLE_LOGS to see each failure.
• An empty or missing folder is FILE_FOLDEROPS_WARN_NOT_FILE_FOLDER, which is normal, not a failure.

Reading the results

folder_ops.SetInitialPass(CRYPTOBYLEO_FERNET_UTCTIME);   // CFolderOpsClean::SetInitialPass() takes no argument
folder_ops.DecryptFolder("MQLArticles\\Data\\", true, true, INT_MAX, "-");

for(int i = 0; i < folder_ops.TotalWarns(); i++)
  Print(EnumToString(folder_ops.Warn(i)));

Method	Returns
TotalWarns()	number of warnings recorded in the current run
Warn(i)	the i-th warning (ENUM_FILE_FOLDEROPS_WARN)
LastErr()	last hard error (ENUM_FILE_FOLDEOPS_ERR), set by Load / Save

Notes and preconditions

• Call SetInitialPass() before each top-level operation to reset internal state. For CFolderOpsEncrypt it takes the now described above and it is required: without it now is 0, so tokens get timestamp 0 and the TTL check never rejects anything.
• Fernet(...) must be set before EncryptFolder / DecryptFolder. A NULL encryptor is a precondition violation (a critical runtime error in MQL5).
• When encrypting with delete_prev_file = false and an extension set, the output is written in the same folder being walked. Make sure the include filter does not match the encrypted extension, or already-encrypted files may be processed again.
• The operations are not transactional: if the run is interrupted or some files fail, the folder is left partially processed.