CryptoByLeo/Src/Fernet/README.md

Fernet

Authenticated symmetric encryption in the style of Fernet: AES-128-CBC with PKCS#7 padding, authenticated with HMAC-SHA256 (encrypt-then-MAC). Pure MQL5, no DLLs required. Everything lives in namespace TSN, class CFernet.

Besides raw buffers, CFernet can encrypt and decrypt whole files, with a configurable output file name and a time-to-live (TTL) check on decryption.

Token format

EncryptRaw / EncryptFile produce a binary token (not base64):

Offset	Size	Field
0	1	Version, always 0x80 (CRYPTOBYLEO_FERNET_VER)
1	8	Timestamp, big endian: the datetime you pass when encrypting
9	16	IV, random per encryption
25	16·n	AES-128-CBC ciphertext of the data + PKCS#7 padding
end − 32	32	HMAC-SHA256 of every previous byte

• The ciphertext is the data rounded up to the next multiple of 16. Padding is always present, so data that is already a multiple of 16 grows by a full block.
• Smallest valid token: 73 bytes (1 + 8 + 16 + 16 + 32, i.e. empty data).
• Decryption checks, in this order: version byte, TTL, HMAC (constant-time comparison), CBC decryption, PKCS#7 padding. Nothing is decrypted before the HMAC matches.

Keys

The key is 32 bytes (CRYPTOBYLEO_FERNET_KEYBYTES) and is split in two:

Bytes	Use
0 – 15	HMAC-SHA256 key (CRYPTOBYLEO_FERNET_HMAC_KEY_BYTES)
16 – 31	AES-128 key (CRYPTOBYLEO_FERNET_AES_KEY_BYTES)

Constants (Def.mqh):

Macro	Value
CRYPTOBYLEO_FERNET_VER	0x80
CRYPTOBYLEO_FERNET_KEYBYTES	32
CRYPTOBYLEO_FERNET_HMAC_KEY_BYTES / CRYPTOBYLEO_FERNET_AES_KEY_BYTES	16 / 16
CRYPTOBYLEO_FERNET_UTCTIME	TimeGMT()

There are two ways to set the key:

1. Manual: Key(const uchar& key[]). Needs at least 32 bytes (only the first 32 are used). Returns false and sets CRYPTOBYLEO_FERNAT_ERR_KEY_INVALID_SIZE otherwise.
2. Automatic: Key<TCryptoRandomFunc>() generates the 32 bytes itself with TCryptoRandomFunc::RandomCryptoBytes. CFernet has no getter for the key, so a generated key only lives inside that instance: use it for data you will also decrypt with the same instance (sessions, temporary files). To persist data, generate and store the key yourself and use the manual mode.

Random source

Every function that needs randomness (Key<T>(), EncryptRaw<T>, EncryptFile<T>) takes the source as a template parameter. Any class with this static function works:

static void RandomCryptoBytes(uchar& out[], int oult, int o_s = 0);

The library ships two, see ../URandom: CURandom (Windows bcrypt.dll, needs DLLs enabled) and CURandomPRNG<TBackend> (pure MQL5, no DLLs, weaker entropy). A class that returns a constant is also valid and gives reproducible tokens, which is what Test.mq5 does.

Time and TTL

The datetime you pass to EncryptRaw / EncryptFile is stored as is in the token, and ExtractTimestap returns it. The "current time" is not read internally: every decrypt function receives it as the now parameter, and the token is rejected as expired when:

now - timestamp >= ttl      (ttl in seconds)

The Fernet spec uses GMT, so use the macro CRYPTOBYLEO_FERNET_UTCTIME (it expands to TimeGMT()) both as the time when encrypting and as now when decrypting. Mixing clocks (for example TimeLocal() to encrypt and GMT to decrypt) shifts the age by your time offset. Because now is a parameter, you can also pass a fixed value to make tests reproducible.

A ttl of INT_MAX effectively disables the check, and 0 rejects every token that is not dated in the future. A timestamp in the future (now earlier than timestamp) is never considered expired.

ExtractTimestap only reads the field. It does not verify the HMAC, so the value is not trustworthy until a decryption succeeds.

API

Functions

Function	Success	Failure
bool Key(const uchar& key[])	true	false
void Key<TRand>()
int EncryptRaw<TRand>(const uchar& in[], uchar& token[], datetime time)	exact token size	no failure path
int DecryptRaw(const uchar& in[], int t, uchar& msg[], datetime now, int ttl)	message size (can be 0)	-1
bool EncryptFile<TRand>(file_name, common_flag, delete_prev_file, time, ext_encrypt = "")	true	false
bool DecryptFile(file_name, common_flag, delete_prev_file, datetime now, int ttl, string ext_decrypt = "")	true	false
bool DecryptFile(file_name, common_flag, delete_prev_file, uchar& res_data[], datetime now, int ttl)	true	false
bool DecryptFile(file_name, common_flag, delete_prev_file, string& res_data, datetime now, int ttl, uint cp = CP_ACP)	true	false
datetime ExtractTimestap(const uchar& token[])	the timestamp	0
ENUM_CRYPTOBYLEO_FERNAT_ERR LastErr()	last error

On failure, LastErr() tells why, see Errors.

Raw functions

• EncryptRaw: in and token must be different arrays. token is only grown (never shrunk) to fit, so it can end up larger than the returned size: use the returned value as the token length. The data is not modified.
• DecryptRaw: t is the number of valid bytes in in (the token length). now and ttl drive the expiration check, see Time and TTL. The message is written from index 0 of msg, which is only grown, so msg can be larger than the message: use the returned value as its length. On failure (-1) the content of msg is not valid.
• ExtractTimestap: reads bytes 1 to 8 after checking the version byte. It does not check the array length.

File functions

common_flag is true for the Common\Files folder and false for the terminal's MQL5\Files. delete_prev_file deletes the source file after the result has been saved.

Function	Order of operations
EncryptFile	read the whole file → encrypt → save file_name + ext_encrypt → delete the source if delete_prev_file
DecryptFile to a file	load the whole file → DecryptRaw → save to the name given by ext_decrypt → delete the source if delete_prev_file
DecryptFile to res_data	load the whole file → DecryptRaw → delete the source if delete_prev_file. Nothing is written.

• The two res_data overloads always trim the result to its real size. The string one converts the bytes with the code page cp.
• If any step before the save fails, nothing is written and nothing is deleted.
• If the save worked but the delete did not, the function still returns true (see Errors).

Warning: delete_prev_file = true with an output that has the same name as the source deletes the result. The output is saved first and the source is deleted afterwards, so if both are the same file the freshly written file is deleted and the data is lost. This happens in EncryptFile when ext_encrypt is "", and in DecryptFile to a file when ext_decrypt ends up as the same name (see the tables below). Use delete_prev_file = false in those cases.

Usage

Raw buffers

#include "Src\\Fernet\\Main.mqh"
#include "Src\\URandom\\PRNG.mqh"

TSN::CFernet fernet;

uchar key[CRYPTOBYLEO_FERNET_KEYBYTES];   // 32 bytes
TSN::CURandomPRNG<Xoshiro256>::RandomCryptoBytes(key, CRYPTOBYLEO_FERNET_KEYBYTES);
if(!fernet.Key(key))
   Print("Key error: ", EnumToString(fernet.LastErr()));

uchar plain[], token[], msg[];
StringToCharArray("Hola mundo", plain, 0, StringLen("Hola mundo"), CP_UTF8);

const int token_size = fernet.EncryptRaw<TSN::CURandomPRNG<Xoshiro256> >(plain, token, CRYPTOBYLEO_FERNET_UTCTIME);
ArrayResize(token, token_size);           // optional: trim to the real size

const int msg_size = fernet.DecryptRaw(token, token_size, msg, CRYPTOBYLEO_FERNET_UTCTIME, 3600);   // valid for 1 hour
if(msg_size == -1)
   Print("Decrypt error: ", EnumToString(fernet.LastErr()));
else
  {
   ArrayResize(msg, msg_size);
   Print(CharArrayToString(msg, 0, WHOLE_ARRAY, CP_UTF8));
  }

Automatic key

TSN::CFernet fernet;
fernet.Key<TSN::CURandomPRNG<Xoshiro256> >();   // random key, only valid for this instance

Files

// data.txt -> data.txt.enc (the original is deleted)
fernet.EncryptFile<TSN::CURandomPRNG<Xoshiro256> >("data.txt", true, true, CRYPTOBYLEO_FERNET_UTCTIME, ".enc");

// data.txt.enc -> data.txt (the encrypted file is deleted)
fernet.DecryptFile("data.txt.enc", true, true, CRYPTOBYLEO_FERNET_UTCTIME, 3600, "-");

// decrypt without writing anything, keep the encrypted file
string text;
fernet.DecryptFile("data.txt.enc", true, false, text, CRYPTOBYLEO_FERNET_UTCTIME, 3600, CP_UTF8);

Encrypting or decrypting a whole folder is built on top of this class (CFolderOpsEncrypt in the MQLArticles repository, Utils/File).

File names

Examples use the input file data.txt for EncryptFile and data.txt.enc for DecryptFile.

ext_encrypt (EncryptFile)

The output is file_name + ext_encrypt, whatever the string is.

ext_encrypt	Output file
"" (default)	data.txt: the same file, overwritten with the token
".enc"	data.txt.enc
"enc"	data.txtenc: appended as is, so include the dot yourself

With "" and delete_prev_file = true the token is deleted right after being saved, see the warning above.

ext_decrypt (DecryptFile to a file)

The rules are checked in this order:

1. The string contains a . and its last character is -: that last - is dropped, the last extension of the file is replaced by the rest.
2. The string contains a .: it is appended to the file name.
3. The string has no . but contains a -: the last extension is removed.
4. Anything else (including ""): the same file name, so the file is overwritten with the plaintext.

ext_decrypt	Output for data.txt.enc
"" (default)	data.txt.enc (overwritten with the plaintext)
"-"	data.txt
".bak"	data.txt.enc.bak
".bin-"	data.txt.bin
"txt"	data.txt.enc (no dot and no dash: rule 4)

Rule 4 outputs the same name as the source, so with delete_prev_file = true the plaintext is deleted, see the warning above.

If the file name has no extension at all, rules 1 and 3 build an empty name and the call fails with CRYPTOBYLEO_FERNAT_ERR_DEC_FILE_SAVE_ERR.

Errors

Always check the return value first, then ask LastErr(): it starts as CRYPTOBYLEO_FERNAT_ERR_NOT and is not reset after a successful call, so after a success it can still hold an old error.

Value	Set by	Meaning
ERR_NOT	constructor	No error yet (initial value).
ERR_KEY_INVALID_SIZE	Key	Fewer than 32 bytes were given. Returns false.
ERR_OPEN_FILE	EncryptFile	The source file could not be opened. Returns false.
ERR_SAVE_CRYPTHF_ILE	EncryptFile	The token could not be written. Returns false.
ERR_DEL_FILE_IN_CRYPT	EncryptFile	The token was written but the source file could not be deleted. Returns true.
ERR_DEC_INVALID_VER	DecryptRaw, ExtractTimestap	First byte is not 0x80.
ERR_DEC_EXPIRED	DecryptRaw	now - timestamp >= ttl.
ERR_DEC_HMAC_CORRUPTED	DecryptRaw	HMAC mismatch: wrong key or modified data.
ERR_DEC_INVALID_PADDING	DecryptRaw	The HMAC matched but the padding is not valid.
ERR_DEC_FILE_LOAD_ERR	DecryptFile	The file could not be read. Returns false.
ERR_DEC_FILE_SAVE_ERR	DecryptFile to a file	The output file could not be written. Returns false.
ERR_DEC_FILE_DEL_ERR	DecryptFile	The result was handled but the encrypted file could not be deleted. Returns true.

The table omits the common prefix: the full names are CRYPTOBYLEO_FERNAT_ERR_* (see Def.mqh). The DecryptRaw errors also surface through every DecryptFile overload, which returns false.

Notes and limits

• Size: MQL5 arrays are limited to INT_MAX elements and CFernet loads the whole file in memory, so the file plus the 73-byte overhead (and the working copies) must fit. In practice stay under roughly 1.5 GB.
• No length validation in DecryptRaw: it assumes in is a token produced by this class and that t is its real size (at least 73 bytes). Do not feed it arbitrary or truncated data.
• No check that a key is set: CFernet does not track whether Key was called, so call it before encrypting or decrypting.
• Failed deletes do not fail the call: when saving worked but deleting the previous file did not, the function still returns true and only LastErr() reports it.
• Not compatible with other Fernet implementations: the token is raw binary, not the base64url text of the official spec, so tokens cannot be exchanged with Python's cryptography package without an extra encoding step.
• The string overload of DecryptFile keeps a static buffer: the decrypted bytes stay in it after the call, and it is shared by all CFernet instances. Use the uchar[] overload if you need to control where the plaintext lives.
• One operation at a time per instance: CFernet keeps the AES state and the IV inside the object, so each call must finish before the next one starts on the same instance.
• Reproducible tests: with a constant random source (see Test.mq5), a fixed key and a fixed timestamp the token is always the same.