# libGringotts manual

As we said, libGringotts provides a simple way to process your data, or save them to some file, in an higly configurable and completely secure way, using strong encryption and hashing algorithms to ensure no one will ever be able to read them (well... as long as they don't know the password... ;-). It includes some other useful features, like generation of encrypted temporary files, secure memory management, and functions for the evaluation of password quality.

Here is a complete documentation on libGringotts' interface and internals. First, you'll be presented with the various non-standard data structures and codes you'll use; after these, the functions will be introduced. Maybe you'll want to step right to the examples, which are a sort of tutorial; finally, the very file format will be described.

### Table of contents

- Typedefs & enumerations

- Enumerations
- Encapsulations ("objects")

- Error codes
- Functions

- General purpose functions
- libGringotts context-related functions
- libGringotts keyholder-related functions
- Memory-to-memory encryption/decryption functions
- File encryption/decryption functions
- Encrypted Temporary File-related functions
- Miscellaneous utility functions

- An example is better than 103 words
- In depth: the file formats

- libGringotts file format, version 3
- libGringotts file format, version 2
- libGringotts file format, version 1

### Typedefs & enumerations

In the various functions, you'll use the following types. Some are enumerations, that can take fixed values (also described below), some other just encapsulations, that keep a bunch of data about the context, or other info.

#### Enumerations

The default values are shown in bold text.

grg_crypt_algo | this parameter describes the algorithm used for encryption.
| GRG_RIJNDAEL_128 | The AES winner, Rijndael.
| GRG_AES
| GRG_SERPENT | The AES finalist Serpent
| GRG_TWOFISH | The AES finalist Twofish
| GRG_CAST_256 | The AES contestant CAST 256
| GRG_SAFERPLUS | The AES contestant Safer+
| GRG_LOKI97 | The AES contestant Loki 97
| GRG_3DES | Triple DES algorithm
| GRG_RIJNDAEL_256 | Rijndael, with 256-bit block size
grg_hash_algo | the algorithm used to hash the password to obtain the encryption key
| GRG_SHA1 | SHA-1 160-bit hash algorithm
| GRG_RIPEMD_160 | RIPEMD-160 160-bit hash algorithm
grg_comp_algo | the algorithm used to compress data before saving it
| GRG_ZLIB | ZLib deflate algorithm
| GRG_BZIP | BZip2 algorithm (more efficient, slower)
grg_comp_ratio | the compression ratio
| GRG_LVL_NONE | No compression stage at all
| GRG_LVL_FAST | Compression, level 3
| GRG_LVL_GOOD | Compression, level 6
| GRG_LVL_BEST | Compression, level 9
grg_security_lvl | fine-tuning of some details regarding security
| GRG_SEC_NORMAL | Normal behavior, enough for all purposes.
| GRG_SEC_PARANOIA | Paranoid settings. For now, it uses `/dev/random` instead of `/dev/urandom`, slowing things a good deal.

#### Encapsulations ("objects")

GRG_CTX | An object that represent the "context" of the usage of the libraries. It should be created at the beginning of the application, as it contains all the parameters that regulate the behaviour of the encryption process. Anyway, most of these parameters can be changed at runtime, i.e. to specify different algorithms according to the user's choices.
GRG_KEY | An object that encapsulates the encryption key (a "keyholder"). It's created starting from a password, and must be re-created anytime the password changes. The password itself isn't stored in memory in plaintext, this way, as it's immediately hashed.

GRG_TMPFILE

This objects represents an encrypted temporary file, and must be used to indentify a particular instance of it in the various operations. Encrypted temporary files haven't a name in the filesystem, but the reference is only held by the program; they are encrypted with a random key, so lurkers can't retrieve data from raw readings of the filesystem. Anyway, they aren't compressed, for sake of speed. An enc. temp file can be created, and written once; then you have read-only access on the data.

### Error codes

When the value returned by a encryption/decryption function is negative, it represents an error. The following values may be used to compare the code with, and find its cause.

GRG_OK | All went fine! :-)
GRG_WRITE_COMP_ERR | Can't compress data. Probably a fault or misinstallation of the compression libraries used.
GRG_WRITE_ENC_INIT_ERR | Can't initialize libMCrypt for compression. Probably a wrong installation
GRG_WRITE_FILE_ERR | Can't open the file for writing. Check permissions...
GRG_TMP_NOT_WRITEABLE | A temporary file has been written once, and can't be written anymore. You can only read it, or close it and create another one.
GRG_READ_FILE_ERR | Can't open the file for reading. Check permissions, or file existence.
GRG_READ_MAGIC_ERR | The file doesn't start with the magic number ("header") provided. Probably wrong file type.
GRG_READ_CRC_ERR | The CRC check failed. File corrupted...
GRG_READ_PWD_ERR | The supplied password was wrong
GRG_READ_ENC_INIT_ERR | Can't initialize libMCrypt for decompression. Probably a wrong installation.
GRG_READ_UNSUPPORTED_VERSION | The file format version of the file you're attempting to read is not supported.
GRG_READ_COMP_ERR | Can't decompress data. Probably a fault or misinstallation of the compression libraries used.
GRG_TMP_NOT_YET_WRITTEN | A temporary file can't be read, because it hasn't been written yet.

The following can be returned by the fuction `grg_file_shred()`:

GRG_SHRED_CANT_OPEN_FILE | The function doesn't manage to open the file to wipe.
GRG_SHRED_YET_LINKED | The file you're trying to wipe have more than one hard link. Overwriting it with random data would destroy also the other file.
GRG_SHRED_CANT_MMAP | The system can't memory map the file. It's probably a system problem; see `man 2 mmap` for hints.

The following are generic errors, that can be returned by all functions that return an int:

GRG_MEM_ALLOCATION_ERR | The system can't allocate memory. Probably memory is full, or there are serious system errors.
GRG_ARGUMENT_ERR | An argument supplied to the function isn't valid; probably, you're supplying a NULL pointer when not allowed.

### Functions

So, here we are. At the very core :-)

#### General purpose functions

`unsigned char *grg_get_version (void);`

> This returns the version of the library, as a string. For example, "1.0.0" or "5.2.22". It's allocated, so you must `free()` it after use.

`unsigned int grg_get_int_version (void);`

> The version, again, but as a number; this way, you can make comparisons between versions. For example, "1.0.0"=10000 and "5.2.22"=50222, so it's TRUE that 50222 > 10000.

#### libGringotts context (GRG_CTX) related functions

`GRG_CTX grg_context_initialize (const unsigned char *header, const grg_crypt_algo crypt_algo, const grg_hash_algo hash_algo, const grg_comp_algo comp_algo, const grg_comp_ratio comp_lvl, const grg_security_lvl sec_lvl);`

> Initializes a new libGringotts session, given its parameters. The header is a 3-letter string that specifies which will be the first three bytes of your files, to use as identifier. The other parameters are the algorithms and settings you want to use, as described in the "Enumerations" section, above. If the function returns NULL, an error occoured.

`GRG_CTX grg_context_initialize_defaults (const unsigned char *header);`

> Initializes a new libGringotts session, with some default values. The header is the same as the previous function, and like that one, if the return value is NULL, it indicates an error.

`void grg_context_free (GRG_CTX gctx);`

> Frees the resources encapsulated in a context gctx. It MUST be always called, upon a session termination.

`grg_crypt_algo grg_ctx_get_crypt_algo (const GRG_CTX gctx);
grg_hash_algo grg_ctx_get_hash_algo (const GRG_CTX gctx);
grg_comp_algo grg_ctx_get_comp_algo (const GRG_CTX gctx);
grg_comp_ratio grg_ctx_get_comp_ratio (const GRG_CTX gctx);
grg_security_lvl grg_ctx_get_security_lvl (const GRG_CTX gctx);`

> Gets the various settings encapsulated in a context.

`void grg_ctx_set_crypt_algo (GRG_CTX gctx, const grg_crypt_algo crypt_algo);
void grg_ctx_set_hash_algo (GRG_CTX gctx, const grg_hash_algo hash_algo);
void grg_ctx_set_comp_algo (GRG_CTX gctx, const grg_comp_algo comp_algo);
void grg_ctx_set_comp_ratio (GRG_CTX gctx, const grg_comp_ratio comp_ratio);
void grg_ctx_set_security_lvl (GRG_CTX gctx, const grg_security_lvl sec_level);`

> These functions changes the settings for a given context, to adapt its future behaviour to the programmer's needings.

`unsigned int grg_get_key_size_static (const grg_crypt_algo crypt_algo);
unsigned int grg_get_key_size (const GRG_CTX gctx);
unsigned int grg_get_block_size_static (const grg_crypt_algo crypt_algo);
unsigned int grg_get_block_size (const GRG_CTX gctx);`

> Intuitively enough, they obtain informations on the size of block and key of an ancryption algorithm; they can be obtained by the algorithm id (the _static version of each) or by the one encapsulated in a specific context.

#### libGringotts keyholder (GRG_KEY) related functions

`GRG_KEY grg_key_gen (const unsigned char *pwd, const int pwd_len);`

> Generates a new keyholder, from a password pwd (that can be any byte sequence), of length pwd_len. If pwd_len is -1, the length is autodetected, and the password must be NULL-terminated. It returns NULL on errors.
> Note: the password is not stored in plaintext, and the memory occupied by the keyholder is fixed and does not depend on the password size.

`GRG_KEY grg_key_clone (const GRG_KEY src);`

> Clones a keyholder src into a new one.

`int grg_key_compare (const GRG_KEY k1, const GRG_KEY k2);`

> Compares two keyholders, k1 and k2. If they are equals, it returns 0; else, 1.

`void grg_key_free (const GRG_CTX gctx, GRG_KEY key);`

> Releases the resources associated with the keyholder key, using the context gctx. It securely wipes it, so no one can recover its data later on.

#### Memory-to-memory encryption/decryption functions

`int grg_validate_mem (const GRG_CTX gctx, const void *mem, const long memDim);`

> Validates a memory segment containing libGringotts' data. In other words, it simply says if it's valid or not, and if not, why. Returns one of the error codes described above. The memory address is located at mem, of length memDim.

`int grg_update_gctx_from_mem (GRG_CTX gctx, const void *mem, const long memDim);`

> Detects the algorithms used to produce a libGringotts-encoded memory segment (at mem, of length memDim), and updates the context gctx to contain these data.

`int grg_encrypt_mem (const GRG_CTX gctx, const GRG_KEY keystruct, void **mem, long *memDim, const long origDim);`

> Encodes data to a memory region. The encryption is performed with the password contained in keystruct and the settings specified in gctx. The encoded data produced will be written to a newly-allocated (to free after use) memory region, pointed by *mem, and its length will be written to *memDim. origData and origDim are, respectively, the data to be encoded (which can be any byte sequence, not just strings) and their length. As ever, if origDim = -1 the length is autodetected, and origData must be NULL-terminated. The return value is a libGringotts error code, to compare in the table above.

`int grg_decrypt_mem (const GRG_CTX gctx, const GRG_KEY keystruct, const void *mem, const long memDim, unsigned char **origData, long *origDim);`

> Reads data from a libGringotts-encoded data sequence, located at mem (of size memDim), using the password in keystruct. The context gctx is adopted, and updated with the algorithms used to encrypt the file. It returns an error code in case of errors. The read data are stored in origData, and their length in origLen, that can be NULL if you don't want to retrieve length. The former is allocated dinamically, so you'll want to `free()` (better, `grg_free()` ;-) it after use.

#### File encryption/decryption functions (finally ;-)

These are basically the same functions as above, but they read and save data from files instead than memory. If you work with files, please use these, because they are more integrated than operating with memory and then interface it on files, resulting in faster & safer I/O.

`int grg_validate_file (const GRG_CTX gctx, const unsigned char *path);`

> Validates a libGringotts' file (at path). In other words, it simply says if it's valid or not, and if not, why. Returns one of the error codes described above.

`int grg_update_gctx_from_file (GRG_CTX gctx, const unsigned char *path);`

> Detects the algorithms used to produce a file (at path), and updates the context gctx to contain these data.

`int grg_encrypt_file (const GRG_CTX gctx, const GRG_KEY keystruct, const unsigned char *path, const unsigned char *origData, const long origDim);`

> Writes data to a new libGringotts file. The encryption is performed with the password contained in keystruct and the settings specified in gctx; the file produced will be written to path. origData and origDim are, respectively, the data to be encoded (which can be any byte sequence, not just strings) and their length. As ever, if origDim = -1 the length is autodetected, and origData must be NULL-terminated. The return value is a libGringotts error code, to compare in the table above.

`int grg_decrypt_file (const GRG_CTX gctx, const GRG_KEY keystruct, const unsigned char *path, unsigned char **origData, long *origDim);`

> Reads data from an encrypted file located at path, using the password in keystruct. The context gctx is adopted, and updated with the algorithms used to encrypt the file. It returns an error code in case of errors. The read data are stored in origData, and their length in origLen, that can be NULL if you don't want to retrieve length. The former is allocated dinamically, so you'll want to `free()` (better, `grg_free()` ;-) it after use.

There is also a "direct" version of each of these functions, that accepts an already opened file descriptor instead of a filename. This may be desirable to avoid race conditions, i.e. when validating a file before actually opening it. Notice that these don't close the file descriptor; that operation is up to you.

`int grg_validate_file_direct (const GRG_CTX gctx, const int fd);
int grg_update_gctx_from_file_direct (GRG_CTX gctx, const int fd);
int grg_encrypt_file_direct (const GRG_CTX gctx, const GRG_KEY keystruct, const int fd, const unsigned char *origData, const long origDim);
int grg_decrypt_file_direct (const GRG_CTX gctx, const GRG_KEY keystruct, const int fd, unsigned char **origData, long *origDim);`

#### Encrypted Temporary File-related functions

`GRG_TMPFILE grg_tmpfile_gen (const GRG_CTX gctx);`

> Creates a new encrypted tempfile, and returns an handler to it. The file will be written with the encryption algorithm contained in the context at this time; any modification to it will not be reflected on the file. The file is encrypted with a random key, unavailable to the user. The handler will be used to write (once) data to the file, and retrieve them later.

`int grg_tmpfile_write (const GRG_CTX gctx, GRG_TMPFILE tf, const unsigned char *data, long data_len);`

> This function is used to write data (of length data_len, -1 as ever if data is NULL-terminated) in the temporary file represented by tf. This can be done only once. The file, after this, becomes read-only. Returns an error code as usual.

`int grg_tmpfile_read (const GRG_CTX gctx, GRG_TMPFILE tf, unsigned char **data, long *data_len);`

> This function is used to read data from a temporary file. The length is stored in data_len, but you can specify NULL if you don't want to retrieve it. This operation can be done only after a successful writing. Returns an error code as usual.

`void grg_tmpfile_close (const GRG_CTX gctx, GRG_TMPFILE tf);`

> Releases all the resources incapsulated in the tmpfile handler, in a secure way.

#### Miscellaneous utility functions

`unsigned char *grg_rnd_seq (const GRG_CTX gctx, const unsigned int size);`
`void grg_rnd_seq_direct (const GRG_CTX gctx, unsigned char *toOverwrite, const unsigned int size);`

> Returns a random byte sequence, basing on the context gctx, of size size. The `_direct` version overwrites a (valid) string provided by the user (it's his responsibility to check that string and size are coherent; if size = -1 it will be autodetected), the other allocates a new string, to `free()` afterwards.

`unsigned char grg_rnd_chr (const GRG_CTX gctx);`

> Returns a random byte.

`void grg_free (const GRG_CTX gctx, void *alloc_data, const long dim);`

> Frees allocated memory (pointed by alloc_data), overwriting it with random trash. Precious when you have sensitive data in memory, and you do want to ensure that they aren't readable in any way, afterwards. dim is the dimension of the data to wipe, or -1 if it's NULL-terminated.

`double grg_ascii_pwd_quality (const unsigned char *pwd, const long pwd_len);`

> Gives an estimation of the quality of a string password, on a scale from 0 to 1. pwd_len is the length of the password pwd; you can specify -1 if it's NULL-terminated.

`double grg_file_pwd_quality (const unsigned char *pwd_path);`

> Gives an estimation of the quality of a file (on the disk, at pwd_path), when using it as a password, on a scale from 0 to 1.

`unsigned char* grg_encode64 (const unsigned char *in, const unsigned int inlen, unsigned int *outlen);
unsigned char* grg_decode64 (const unsigned char *in, const unsigned int inlen, unsigned int *outlen);
`

> These function convert a generic sequence of bytes to/from the base64 format. Useful for email sending, xml storing, etc.

`int grg_file_shred (const char *path, const int npasses);`

> Securely wipes a file, overwriting it npasses times with random data. The data can't be recovered, once done this; be careful. This option still have some limitations, as it takes the assumption that the filesystem overwrites files in place: for some FSs this isn't true. See `man 1 shred` for details.

### An example is better than a 103 words

Confused? Don't be! It's simple! :-) Here is a couple of examples of how to tie it all together. This is a piece of code that writes data to a file:

    #include <libgringotts.h>
    #include <string.h>

    [...]

    {
    	//initialization of structures
    	GRG_CTX context = grg_context_initialize_defaults ("TRY");
    	GRG_KEY keyholder = grg_key_gen ("This is a nice password", -1);

    	//let's produce some data to encode
    	unsigned char *plaintext = "These are some valuable and important data!";
    	unsigned int txtlen = strlen (plaintext);
    	unsigned char *filepath = "/home/mano/somefile.try"; //extension is not relevant

    	//now we change some settings, and leave the others as defaults
    	grg_ctx_set_crypt_algo (context, GRG_TWOFISH);
    	grg_ctx_set_comp_algo (context, GRG_BZIP);

    	//finally, let's save!
    	grg_encrypt_file (context, keyholder, filepath, plaintext, txtlen); //txtlen could be -1

    	//always remember to clean up leftovers...
    	grg_key_free (context, keyholder);
    	grg_context_free (context);
    }

    [...]

To decode, things are even simpler:

    #include <libgringotts.h>
    #include <stdio.h>

    [...]

    {
    	//again, initialization of structures
    	//or you can reuse the same as before
    	GRG_CTX context = grg_context_initialize_defaults ("TRY");
    	GRG_KEY keyholder = grg_key_gen ("This is a nice password", -1);

    	unsigned int *filepath = "/home/mano/somefile.try";

    	//here the library will put the decoded data
    	unsigned char *plaintext;
    	long int txtlen;

    	//we are ready to perform the decryption
    	grg_decrypt_file (context, keyholder, filepath, &plaintext, &txtlen);

    	//now, data are stored in the two variables and in the
    	//context, that's changed accordingly. Let's use them...
    	printf ("Decrypted data are %d long, and are \"%s\"\n", txtlen, plaintext);
    	if (grg_ctx_get_crypt_algo (context) == GRG_TWOFISH)
    		printf ("You encrypted the file with Twofish\n");
    	else
    		printf ("You didn't encrypt the file with Twofish\n");

    	//as always, remember to clean up leftovers...
    	grg_key_free (context, keyholder);
    	grg_free (context, plaintext, txtlen);
    	grg_context_free (context);
    }

    [...]

This silly thing writes some stuff to a temporary file, then reads it.

    [...]

    {
    	//init of structs
    	GRG_CTX gctx = grg_context_initialize_defaults ("tRy");
    	GRG_TMPFILE tf = grg_tmpfile_gen (gctx);
    	unsigned char *try;
    	long int dim;

    	//writes some stupid data
    	grg_tmpfile_write (gctx, tf, "S0M3 S7UP1D DA7A", 16);

    	[...]

    	//then reads it. Simple...
    	grg_tmpfile_read (gctx, tf, &try, &dim);

    	//Freedom... freedom... freedom... ;-)
    	grg_tmpfile_close (gctx, tf);
    	grg_context_free (gctx);

    	//Use those data!
    	printf("Data were %s, of length %ld\n", try, dim);
    }

    [...]

### In depth: the file formats

A description of the inners of the libGringotts File Format follows. Please use it... in any way you like! ;-)

#### libGringotts file format, version 3

A file has this structure:

    HEADER | VERSION | CRC32 | ALGO | IV | CRC32 | DATA_LEN | DATA

- the part written in italic is encrypted;
- the part written in bold is compressed and encrypted

- HEADER: an unique file format ID [it must be 3 bytes long]
- VERSION: the file format version ("3") [1b]
- CRC32: the CRC32 of the remaining file part [4b]
  The second is used to check the correctness of the password
- ALGO is a bitfield defined as follow:

         IAAABCDD
      MSB^      ^LSB

  where:
  - I: ignored, set it to 0
  - A: encryption algorithm
  - 000 -> AES (Rjindael-128)
  - 001 -> Serpent
  - 010 -> Twofish
  - 011 -> CAST-256
  - 100 -> SAFER+
  - 101 -> LOKI97
  - 110 -> 3DES
  - 111 -> Rjindael-256
  - B: hashing algorithm
  - 0 -> SHA-1
  - 1 -> RIPEMD-160
  - C: compression type
  - 0 -> ZLib
  - 1 -> libBZ2
  - D: compression level
  - 00 -> none (0)
  - 01 -> fast (3)
  - 10 -> good (6)
  - 11 -> best (9)
  in bold the defaults (0 001 1 0 11 = 0x1b).
- IV is the Initialization Vector of the encryption function. It's randomly generated at file encryption, and its dimension varies from 8 to 32 bytes accordingly to encryption algorithm block size
- DATA_LEN: the length of the uncompressed DATA [4b]
- DATA: the data stored in this file

This way, the final file length is typically (compressed DATA)+33b, and this format allows compressed data size up to 512 Mb ( 24*8 ). Fair enough.

The data are encrypted using the given algorithm with a key derived from the password. This key is calculated hashing the password data with the chosen algorithm, for the length required by the encryption algorithm, and using the S2K algorithm `KEYGEN_S2K_SIMPLE` of MHash. The key obtained is XOR'ed with the IV, repeated as many times as needed; this way, I have a different key for each saving of the file, increasing security.

#### libGringotts file format, version 2

This was analogous to format 3, but used KEYGEN_MCRYPT for key generation; Bob Mathews found that this had a minor weakness, so I changed it. It isn't supported anymore from libGringotts 1.0.0; use Gringotts 0.7.0 to convert an old file in this format.

#### libGringotts file format, version 1

It didn't allow any choice of the algorithms; it used SHA-256 for hashing, and SERPENT for encryption.

It isn't supported anymore, starting from libGringotts 2 (Gringotts 0.6.0). If you still have some file encrypted with this, use an older Gringotts (between 0.4.2 and 0.5.9) to convert it to v2, and then Gringotts 0.7.0 to convert to v3.