Envelope Encryption

By default, Kryptonite for Kafka applies encryption using key material sourced directly from Tink keysets. An alternative mode is to opt in for envelope encryption. This mode randomly generates a short-lived Data Encryption Key (DEK) which is used to encrypt field data, and uses a long-lived Key Encryption Key (KEK) to wrap the DEK. The wrapped DEK then travels with the ciphertext which guarantees there is no direct relationship between the ciphertext and the long-lived KEK material.

Key benefits:

• KEK never touches field data: the KEK is only used to wrap/unwrap DEKs; all field encryption uses the short-lived DEK
• Independent DEK and KEK lifecycle: DEKs rotate automatically on a high-frequency operational schedule; the KEK rotates on a slower administrative schedule; these two concerns are fully decoupled
• Reduced blast radius: a compromised DEK only affects records encrypted during that DEK's lifetime which is configurable

Two Variants

Kryptonite for Kafka supports two envelope encryption variants which differ in where the KEK lives and how the wrapped DEK is stored alongside the ciphertext.

Keyset-based Envelope Encryption

Uses a regular Tink keyset as the KEK. The wrapped DEK is bundled directly inside the ciphertext alongside the encrypted field data, so no external store is needed.

Wire format: [4-byte wrappedDekLen | wrappedDek | dekCiphertext]

Configuration:

cipher_algorithm=TINK/AES_GCM_ENVELOPE_KEYSET
cipher_data_key_identifier=my-kek-keyset
cipher_data_keys=[{"identifier":"my-kek-keyset","material":{...}}]

The cipher_data_key_identifier and cipher_data_keys refer to the keyset acting as the KEK. All the usual key_source options apply which means you can source the KEK keyset from configuration (plain or encrypted) or from a cloud secret manager.

KMS-based Envelope Encryption

Uses a cloud KMS key as the KEK. The KEK never leaves the cloud KMS and all wrap/unwrap operations happen remotely. Because the wrapped DEK would make every ciphertext considerably larger if bundled inline, only a compact 16-byte fingerprint is embedded in the ciphertext; the actual wrapped DEK is stored externally using implementations of EdekStore for which the default is KCache/Kafka.

Wire format: [16-byte fingerprint | dekCiphertext]

Configuration:

cipher_algorithm=TINK/AES_GCM_ENVELOPE_KMS
envelope_kek_configs=[{"identifier":"my-kek","type":"GCP","uri":"gcp-kms://...","config":{"credentials":"...","projectId":"..."}}]
envelope_kek_identifier=my-kek
edek_store_config={"kafkacache.bootstrap.servers":"localhost:9092","kafkacache.topic":"_k4k_edeks"}

EdekStore Implementations

The EdekStore defaults to KCache which persists into a compacted Kafka topic to permanently store the wrapped DEKs. Each record maps a 16-byte fingerprint (a SHA-256 prefix of the wrapped DEK) to the actual wrapped DEK bytes.

Encrypt path:

1. When a new DEK session is created, Kryptonite publishes the wrapped DEK to the EdekStore before any encryption happens, and only then makes the DEK session available for use
2. Each ciphertext carries only the 16-byte fingerprint as a compact pointer to the wrapped DEK in the EdekStore

Decrypt path:

1. Extract the 16-byte fingerprint from the ciphertext
2. Check the L1 in-process cache (fingerprint → Aead): if hit, decrypt directly; no EdekStore or KMS involved
3. On L1 miss: look up the fingerprint in the EdekStore to retrieve the wrapped DEK
4. Call the KMS to unwrap the DEK, build the Aead, and populate the L1 cache for future lookups
5. Decrypt the field data with the Aead

Topic requirements:

• Must be a compacted topic — cleanup.policy=compact
• kafkacache.topic.require.compact=true applies by default; Kryptonite for Kafka will refuse to start against a non-compacted topic
• Replication factor and partition count follow standard Kafka best practices for your environment; one partition is enough in low-write-volume setups like this one (one record per DEK session)

EdekStore Configuration

The edek_store_config value is a JSON object and is supposed to contain the expected configuration based on the chosen EdekStore implementation. For KCache/Kafka it might look like this:

{
  "kafkacache.bootstrap.servers": "broker1:9092,...",
  "kafkacache.topic": "_k4k_edeks"
}

Required keys for KCache/Kafka EdekStore

Optional overrides (defaults shown):

DEK Session Lifecycle

Both variants share the same DEK session management. Instead of generating a fresh DEK for every field encryption (which would mean a KMS/keyset operation per record), a DEK session is reused across a configurable window.

A session expires when either threshold is reached first:

• dek_max_encryptions: upper limit for field encryptions with this DEK (default: 100,000)
• dek_ttl_minutes: longest allowed age of the DEK session (default: 720 = 12 hours)

When a session expires, a new DEK is generated and wrapped transparently without the need for intervention. For KMS-based envelope encryption the new wrapped DEK is published to the EdekStore before the session becomes active.

KEK configuration for KMS-based envelope encryption

The envelope_kek_configs parameter takes a JSON array of KEK entries. Each entry identifies which cloud KMS key to use as the KEK:

[
  {
    "identifier": "my-gcp-kek",
    "type": "GCP",
    "uri": "gcp-kms://projects/<project>/locations/<location>/keyRings/<ring>/cryptoKeys/<key>",
    "config": {
      "credentials": "<GCP service account JSON contents>",
      "projectId": "<project>"
    }
  }
]

[
  {
    "identifier": "my-aws-kek",
    "type": "AWS",
    "uri": "aws-kms://arn:aws:kms:<region>:<account>:key/<key-id>",
    "config": {
      "accessKey": "...",
      "secretKey": "...",
      "region": "..."
    }
  }
]

[
  {
    "identifier": "my-azure-kek",
    "type": "AZURE",
    "uri": "azure-kv://<vault-name>.vault.azure.net/keys/<key-name>",
    "config": {
      "clientId": "...",
      "tenantId": "...",
      "clientSecret": "...",
      "keyVaultUrl": "https://<vault-name>.vault.azure.net"
    }
  }
]

You can define more than one KEK entry. Each field can reference a different KEK identifier, enabling per-field or per-topic key isolation.

See Cloud KMS for provider-specific IAM permissions and setup details.