Loading…

Kryptel Storage

Kryptel 7 Modifiable Storage

The purpose of this article is to provide information on Kryptel internal data formats for public scrutiny. At least a basic understanding of programming is required.

Container Format

Kryptel 7 container consists of mandatory base segment that is optionally followed by any number of fixup segments.

Base segment is, in fact, Kryptel 6 container with very few differences:

  1. Storage Handler component ID is CID_STORAGE_7 instead of CID_BASIC_STORAGE,
  2. Kryptel 6 key verification bug fixed,
  3. What was called "session key area" is now called "key associated data" (just a terminology change),
  4. Alignment data may follow the trailer

Aside from these small differences, Kryptel 7 container can be considered as Kryptel 6 container (i.e. base segment) supplemented with zero or more correction blocks (i.e fixup segments).

Base Segment Format

[Container Header]      Always present
[Data Area]             Optional, contains data blocks
[Agent Data Area]       Optional
[Directory]             Always present
[Header Copy]           Always present
[Trailer]               Always present
[Alignment Data]        If required (see 'Segment Alignment')
		

Container Header

Container header contains IDs of the components used for encryption, component initialization values, and other critical data required for opening the container.

Size Description
4 Container tag
2 Header size including header hash
2 Version of handler that created the container
2 Version of handler required to process the container
16 Storage Handler component ID (must be CID_STORAGE_7)
16 Agent component ID
16 Cipher component ID
16 Compressor component ID
16 Hash function component ID
20 Cipher parameter block
4 Compressor parameter block
12 Hash function parameter block
16 Key ID
2 Size of key associated data (if KeyID == IDENT_YUBIKEY or IDENT_YUBIKEY_PASSWORD)
. . . Key associated data (if KeyID == IDENT_YUBIKEY or IDENT_YUBIKEY_PASSWORD)
4 Number of key verification passes
HashSize Key verification block
4 Size of agent data field (0 if no agent data)
6 Agent data address (points to directory if no agent data)
HashSize Agent data HMAC (0 if no agent data)
6 Size of the directory area
BlockSize Directory area init vector (encrypt it to get the actual vector)
HashSize Directory HMAC (computed on unencrypted data)
STR Cipher name string
STR Cipher scheme string
STR Compressor name string
STR Compressor scheme string
STR Hash function name string
STR Hash function scheme string
16 MD5 hash of the header

STR represents a UTF-16 string; first word (2 bytes) contains the string length in characters. The string does not contain trailing zero. Each UNICODE character occupies 2 bytes.

The container tag, the offset to the handler component ID, and the offset to the agent component ID are constants. The container tag goes first; the position of the agent/handler component IDs must remain the same for the given tag in order for the GetContainerHandlers function to work correctly.

Note that the cipher, the compressor, and the hash function cannot be null components.

Data Area

Data area is a set of objects' data streams. Each data stream is an independent encrypted stream. Data area is never accessed sequentally so gaps are allowed.

Data recovery block

A data stream may be followed by an optional data recovery block. Note that recovery blocks are created but never accessed except during recovery scan.

Size Description
4 Random data
4 Recovery block tag
2 Size of the recovery block
6 Size of the data stream
. . . Initialization vector
16 Data stream MD5 hash
2 Size of agent-owned recovery data
. . . Agent-owned recovery data
. . . Bytes 0xE8 to align to BlockSize boundary - 4
4 CRC-32 of the previous data

Unlike data streams and the directory area, recovery blocks are encrypted as a simple sequence of blocks in CBC chaining mode, using zero initialization vector.

Agent Data Area

This field contains agent-specific data. The handler does not use these data, it just keeps them for the associated agent. The agent data are memory-based, so the agent should keep the data size reasonable.

Container Directory

The container directory is an encrypted stream. The directory data is compressed before encryption. It contains exactly one directory entry for the upper-level object (which in turn may contain any number of child objects). In some sense this object may be considered as the container itself.

Directory Entry

Size Description
2 Object start tag
16 Object ID
6 Size of object's data block (0 if no data)
6 Size of uncompressed data block (present if data size > 0)
6 Address of data block (present if data size > 0)
2 Size of associated recovery block (present if data size > 0)
BlockSize Initialization vector (present if data size > 0)
16 Data block MD5 hash (present if data size > 0)
4 Size of object's attribute block (0 if no attributes)
. . . Attribute block
. . . [Optional directory entries for child objects]
2 Object end tag

If the object has a data block, it is an encrypted stream. The stream is always compressed, however the storage client (i.e. an agent) can specify the compression level when calling IEncryptedObject::CreateStream, and that level may be set to CT_NO_COMPRESSION. The Kryptel agents use this method to turn off compression for specific files.

Header Copy

The copy of the header is used only during recovery scan if the container header is corrupted. The container verification function, however, checks the validity of the header copy.

Container Trailer

Trailer is used for container verification only. The standard Open operation does not read it.

Size Description
4 Trailer tag
2 Header size including the header hash field
6 Address of agent data area
6 Directory address
16 MD5 hash of the header
16 MD5 hash of the data area
16 MD5 hash of the agent data area
16 MD5 hash of the directory area
16 MD5 hash of the trailer

All hashes are computed on the encrypted data so that the user could verify the container integrity without entering the password. If the corresponding area is not present, the hash value is ignored (setting it to 0 is recommended).

Fixup Segments

Fixup segment contains corrections that should be applied to previous segments.

[Segment Header]        Always present
[Data Area]             Optional, contains data blocks
[Agent Data Area]       Optional
[Fixup List]            Optional
[Segment Trailer]       Always present
[Alignment Data]        If required (see 'Segment Alignment')
		

Segment Header

Size Description
4 Fixup tag
4 Size of agent data field (see note below)
6 Agent data address (points to fixup list if no agent data)
HashSize Agent data HMAC
6 Size of the fixup list in bytes (0 if no fixup records)
BlockSize Fixup list init vector(encrypt it to get the actual vector)
HashSize Fixup list HMAC
16 MD5 hash of the preceding fields

If neither agent data nor fixup list is present, then the pointer at offset 8 points to the trailer. An empty fixup segment is considered as normal; for example, a segment, which only purpose is removing the agent data, will consist of a header and a trailer only. Unused fields (HMACs/InitVector) are ignored (filling with zeros is recommended).

Note about fixup tag

There are two fixup tags: in-progress and complete tag. The in-progress tag means that the previous container modification has failed and the segment is incomplete. Such a segment should be thrown away silently. When the handler completes a modification session, the last operation it performs is replacing the in-progress tag with the complete segment tag. The complete tag does not guarantee that the segment is valid, it just informs the handler that the segment has been properly closed and should be parsed.

Note also that the other header fields do not contain any meaningful data if the tag field contains in-progress. The actual header will be written later, along with the regular segment tag (see the next note).

Note about order of writes

When the handler closes a fixup session, it must write the segment parts down in the following order:

Fixup list -> Segment trailer -> Alignment data -> Segment header
		

This way the handler can distinguish incomplete segments from corrupted ones, that is

If the tag is in-progress, the handler fixes the container silently by simply throwing this segment away (i.e. by setting EOF at the tag position). There is no risk of data loss as the previous session has not been completed (and so source files have not been deleted).

If the tag is complete, but any of the hashes does not match, then the segment is corrupted, and the user must be notified.

Fixup Agent Data Area

Depending on the value of the size field in the header, there are three possible cases:

  • Size is 0. The fixup segment contains no agent data. The next field points to the fixup list, and the agent data HMAC field is ignored.
  • Size is -1 (0xFFFFFFFF). The fixup segment contains no agent data, and the existing agent data must be removed. The next field points to the fixup list, and the agent data HMAC field is ignored.
  • Size is in the range 1 to 0xFFFFFFFE. The fixup segment contains agent data that is to replace the existing data (if present). The next field is the agent data address followed by agent data HMAC.

Fixup Data Area

Fixup data area is identical to base segment's data area, that is, it is a collection of encrypted streams (gaps are allowed). A stream may be followed by a corresponding recovery block.

Fixup List

Fixup list is a sequence of fixup records.

'Create Object' Record

Size Description
2 'Create Object' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path

'Attach Attributes' Record

Size Description
2 'Attach Attributes' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path
4 Size of the attribute block
. . . Attribute block (present if size > 0)

If size of attribute block is 0, this record removes existing attribute block.

'Attach Data' Record

Size Description
2 'Attach Data' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path
6 Size of data block (0 if no data)
6 Size of uncompressed data block (present if size > 0)
6 Address of data block (present if size > 0)
2 Size of associated recovery block (present if size > 0)
BlockSize Initialization vector (present if size > 0)
16 Data block MD5 hash (present if size > 0)

If size of data block is 0, this record removes existing data block.

'Add Object' Record

Size Description
2 'Add Object' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path
6 Size of data block (0 if no data)
6 Size of uncompressed data block (present if size > 0)
6 Address of data block (present if size > 0)
2 Size of associated recovery block (present if size > 0)
BlockSize Initialization vector (present if size > 0)
16 Data block MD5 hash (present if size > 0)
4 Size of the attribute block
. . . Attribute block (present if size > 0)

'Move Object' Record

Size Description
2 'Move Object' tag
2 Number of IDs in source object's path (including object's own ID)
16*n Source object path
2 Number of IDs in target object's path (including object's own ID)
16*n Target object path

'Delete Object' Record

Size Description
2 'Delete Object' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path

'Undelete Object' Record

Size Description
2 'Undelete Object' tag
2 Number of IDs in object's path (including object's own ID)
16*n Object path
1 1 if recursive undelete, 0 otherwise

Note about 'Attach Attributes' and 'Attach Data' records

Those records may be used on objects, which already have attributes / data streams. In this case these fixup records replace the existing attributes / data stream.

Note about 'Add Object' record

The 'Add Object' record is excessive and can be replaced with 'Create Object', possibly followed by 'Attach Attributes' and/or 'Attach Data'. The only purpose for introducing that record is reducing the size of the fixup list. It is the responsibility of the storage handler to recognize the typical sequence 'Create Object' => 'Attach Attributes' => 'Attach Data' and pack it into a single 'Add Object' record.

Note about object replacement

There are two ways to replace an existing object: the recommended one is to issue a pair of fixup records - first 'remove object', then 'add object' with a different object ID. This way the deleted object will remain accessible and recoverable.

The other method is to replace object's attribute and data; while the old object's data still remain recoverable, it is not a natural operation for a file agent (as there is nothing to 'undelete').

Segment Trailer

Trailer is used for segment verification only. The standard Open operation does not read it.

Size Description
4 Segment trailer tag
6 Address of segment header
6 Address of agent data area (points to fixup list if no agent data)
6 Fixup list address
16 MD5 hash of the segment header
16 MD5 hash of the segment data area
16 MD5 hash of the agent data area
16 MD5 hash of the fixup list area
16 MD5 hash of the preceding fields

All hashes are computed on the encrypted data so the user could verify the container integrity without entering the password. If the corresponding area is not present, the hash value is ignored (setting to 0 is recommended).

About Trailers

The reason for introducing trailer in v6 container was to provide a simple and easy-to-locate (always the last 96 bytes of the container) data structure for the verificator. However v7 trailer is not easy-to-locate anymore; even in a single-segment container the trailer is followed by an unknown number of alignment bytes. So verification of v7 container requires partial parsing of all the segment headers and there is not much sense in retaining an excessive data structure. However it has been decided to keep trailer in v7 containers for the sake of format and code commonality.

Segment Alignment

Fixup segments always start on a segment alignment boundary, which is 4096. This is the size of the largest disk sector; aligning a new segment on a disk sector boundary helps to avoid overwriting the previous segment's data when a new segment is created. Even if overwritten with the same data, the sector still might get corrupted if a disk error occurs. Aligning will exclude any write operation on the last sector of the preceding segment, making container corruption during modification much less likely.

The handler adds alignment data to align the container size to the 4k boundary. This way, the size of the resulting container is always a multiple of 4096; if it is not, then the last segment is assumed to be corrupted (see also the discussion of the 'in-progress' tag above).

Although the alignment data are not important, it is highly recommended to use zeros or another simple regular pattern. Using random data for segment alignment may raise a suspicion that the alignment block contains a backdoor or a lockpick. Kryptel 7 storage handler uses repeating hex string 'EDC7'.

Tag Values

Container tag                   0x07AA050B
Trailer tag                     0x0B0507AA

Recovery block tag              0xE0EAF0C8

Fixup in progress tag           0x70ADB050
Fixup segment tag               0x70AEB050
Segment trailer tag             0x0B0507AB

Object start tag                0x050B
Object end tag                  0x07AA

Create object tag               0xAE50
Attach attributes tag           0xAE54
Attach stream tag               0xAE55
Add object tag                  0xAE59
Move object tag                 0xAE5D
Delete object tag               0xAE5E
Undelete object tag             0xAE5F
		

Note About Deleted Objects

For version 7 encrypted storage, deletion of an object (i.e. calling IEncryptedObject::DeleteChildObject) does not actually removes the object. This operation just sets object's 'EFL_OBJECT_DELETED' flag and creates a new 'Delete Object' fixup record. The object remains fully accessible but becomes read-only; any attempt to modify the object causes an exception. The only modification operation allowed for a deleted object is undeletion (i.e. calling IEncryptedObject::UndeleteChildObject).

The storage is not guaranteed to keep objects that was added and then deleted in the same session (actually keeping ANY deleted object is not guaranteed; it is just a useful side effect of the storage organization). Specifically, Kryptel 7 storage handler discards objects, deleted during base segment creation, but keeps the objects, added and then deleted in the same fixup session.

Key Usage Notes

Although cipher and HMAC (HMACs are computed for agent data and directory, and are used for key verification) use the same key, they use it differently.

Cipher uses first no more than KeySize bytes of the key (padded with zeroes if necessary).

HMAC reverts and inverts the significant bytes of the key, pads with zeroes if necessary, and uses the first 64 bytes of the result.

For example, in the default case of 256-bit AES and SHA-512, the cipher will use the first 256 bits of the SHA-512 password hash, and HMAC will use the whole reversed and inverted password hash. In most cases this method ensures that cipher and HMAC use completely different or at least weakly related keys.

In case of user-defined binary key (IDENT_BINARY_KEY), HMAC assumes its size is 512 bits and prepares (reverts and inverts) that portion. It is responsibility of the user to pad the key with zeros up to BINARY_KEY_SIZE. Note that end-user Kryptel does not fully supports user-defined binary keys – Kryptel Browser can open a container, encrypted with a user-defined key, however creating such a container requires programmatic access to the file agent.