Key Management

All Kryptonite for Kafka modules support different options to source the required key material. You can define this with the key_source configuration parameter. The right choice primarily depends on your security posture and preferred operational model.

Overview

key_source=CONFIG           → plain keysets inline in config
key_source=CONFIG_ENCRYPTED → KEK-encrypted keysets inline in config
key_source=KMS              → plain keysets in cloud secret manager
key_source=KMS_ENCRYPTED    → KEK-encrypted keysets in cloud secret manager
key_source=NONE             → no keysets required (KMS-based envelope encryption only)

Plain keysets in configuration

The simplest and least secure mode. Plain keysets are embedded directly in the cipher_data_keys configuration parameter.

Plain keyset example

  {
    "identifier": "my-key",
    "material": {
      "primaryKeyId": 10000,
      "key": [
        {
          "keyData": {
            "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmKey",
            "value": "<BASE64_ENCODED_KEY_HERE>",
            "keyMaterialType": "SYMMETRIC"
          },
          "status": "ENABLED",
          "keyId": 10000,
          "outputPrefixType": "TINK"
        }
      ]
    }
  }

Encrypted keysets in configuration

Encrypted keysets give you moderate security while still being able to directly reference keys inside the configuration. A cloud KMS key encryption key (KEK) is used to decrypt the keysets at runtime. The plain keysets are never stored anywhere.

Requires: key_source=CONFIG_ENCRYPTED, depends on kek_type, kek_uri, kek_config, and cipher_data_keys.

Encrypted keyset example

Note, that the material field takes the encrypted keyset form:

  {
    "identifier": "my-key",
    "material": {
      "encryptedKeyset": "<ENCRYPTED_AND_BASE64_ENCODED_KEYSET_HERE>",
      "keysetInfo": {
        "primaryKeyId": 10000,
        "keyInfo": [
          {
            "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmKey",
            "status": "ENABLED",
            "keyId": 10000,
            "outputPrefixType": "TINK"
          }
        ]
      }
    }
  }

Plain keysets in cloud secret manager

Store plain keysets as secrets in a cloud secret manager to get reasonable key material protection. Modules either fetch them lazily on demand or load them eagerly during initialization with the configured cloud provider credentials.

Requires: key_source=KMS, depends on kms_type and kms_config. When using a cloud secret manager, cipher_data_keys can be deliberately set to the empty JSON array [].

Plain keyset example

For cloud secret manager usage, plain keysets must be generated in RAW format with the keyset tool.

{
  "primaryKeyId": 10000,
  "key": [
    {
      "keyData": {
        "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmKey",
        "value": "<BASE64_ENCODED_KEY_HERE>",
        "keyMaterialType": "SYMMETRIC"
      },
      "status": "ENABLED",
      "keyId": 10000,
      "outputPrefixType": "TINK"
    }
  ]
}

Encrypted keysets in cloud secret manager

The most secure mode. This setup stores keysets encrypted in a cloud secret manager, and a separate cloud KMS key encryption key decrypts them at runtime.

Requires: key_source=KMS_ENCRYPTED, depends on kms_type,kms_config, kek_type, kek_uri, and kek_config. When using a cloud secret manager cipher_data_keys can be deliberately set to the empty JSON array [].

Encrypted keyset example

For cloud secret manager usage, encrypted keysets must be generated in RAW format with the keyset tool.

{
  "encryptedKeyset": "<ENCRYPTED_AND_BASE64_ENCODED_KEYSET_HERE>",
  "keysetInfo": {
    "primaryKeyId": 10000,
    "keyInfo": [
      {
        "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmKey",
        "status": "ENABLED",
        "keyId": 10000,
        "outputPrefixType": "TINK"
      }
    ]
  }
}

No keysets

When using key_source=NONE, no Tink keysets are loaded or required. This only works if the configured cipher_algorithm is TINK/AES_GCM_ENVELOPE_KMS which is always backed by a cloud KMS key encryption key (KEK) used to wrap random, ephemeral data encryption keys (DEK). This is the right choice when you must rely only on envelope encryption and do not want to work directly with Tink keysets at all.

Learn more about KMS-based envelope encryption.

Secret naming conventions

Depending on which cloud secret manager is configured to store plain or encrypted keysets, the following secret name prefixes must be considered to distinguish plain from encrypted keysets:

Tink Keyset Format

All keyset material, regardless of the key source, must conform to Tink's plain or encrypted keyset specification.

Plain keyset examples

• AES-GCM (probabilistic encryption)

{
  "primaryKeyId": 123456789,
  "key": [
    {
      "keyData": {
        "typeUrl": "type.googleapis.com/google.crypto.tink.AesGcmKey",
        "value": "<BASE64_ENCODED_KEY_HERE>",
        "keyMaterialType": "SYMMETRIC"
      },
      "status": "ENABLED",
      "keyId": 123456789,
      "outputPrefixType": "TINK"
    }
  ]
}

• AES-GCM-SIV (deterministic encryption)

{
  "primaryKeyId": 123456789,
  "key": [
    {
      "keyData": {
        "typeUrl": "type.googleapis.com/google.crypto.tink.AesSivKey",
        "value": "<BASE64_ENCODED_KEY_HERE>",
        "keyMaterialType": "SYMMETRIC"
      },
      "status": "ENABLED",
      "keyId": 123456789,
      "outputPrefixType": "TINK"
    }
  ]
}

• FPE FF3-1 (format-preserving encryption)

{
  "primaryKeyId": 2000001,
  "key": [
    {
      "keyData": {
        "typeUrl": "io.github.hpgrahsl.kryptonite/crypto.custom.mysto.fpe.FpeKey",
        "value": "<BASE64_ENCODED_KEY_HERE>",
        "keyMaterialType": "SYMMETRIC"
      },
      "status": "ENABLED",
      "keyId": 2000001,
      "outputPrefixType": "RAW"
    }
  ]
}

Note the custom typeUrl and outputPrefixType: RAW for FPE keysets.

Key Rotation with Keysets

Tink keysets natively support more than one key. The primaryKeyId determines which key is used for new encryptions. Older keys are supposed to remain in the keyset for decryption of existing ciphertexts.

1. Generate a new keyset with more than one key (e.g., -n 2 with the Keyset Tool)
2. One of the keys in the keyset must be the primary key, initially it's the first one
3. Any new encryption operations use the currently designated primary key
4. A different key in the keyset can be promoted to become the primary key at any time
5. Once the primary key changes, old ciphertexts remain decryptable as long as the old primary key is still resolvable within the configured keyset.

See Keyset Tool for examples of generating multi-key keysets.

Key Rotation with Envelope Encryption

Envelope encryption introduces a two-level key hierarchy — a short-lived Data Encryption Key (DEK) that encrypts the actual field data, and a long-lived Key Encryption Key (KEK) that wraps the DEK. Rotation operates differently at each level, and the two envelope variants behave differently.

DEK Rotation: automatic and continuous

DEKs are ephemeral by design and rotate automatically without any manual intervention. A DEK session is considered expired when either of the following thresholds is reached:

• dek_max_encryptions — upper limit for field encrypt operations performed with the current DEK (default: 100000)
• dek_ttl_minutes — longest allowed age of the current DEK session in minutes (default: 720, i.e. 12 hours)

Whichever threshold is hit first triggers the creation of a new DEK on the next encrypt call. Old DEKs are not discarded. They remain resolvable on the decrypt path as long as their wrapped form is present in the EdekStore (KMS-based variant) or inline in the ciphertext bundle (keyset-based variant). Tune these settings to control how frequently new DEKs are generated.

KEK Rotation: on-demand and manual

KEK rotation is not automatic and must be triggered explicitly from the outside. The procedure differs between the two envelope variants: