diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index e58f9a4279..2452bb0d0b 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -104,6 +104,15 @@ include::third-party:partial$nav.adoc[] ** xref:learn:security/auditing.adoc[Auditing] ** xref:learn:security/encryption-overview.adoc[Encryption] *** xref:learn:security/native-encryption-at-rest-overview.adoc[] + **** xref:learn:security/encryption-at-rest-keys.adoc[Encryption Keys] + **** xref:learn:security/per-node-per-bucket.adoc[Per-Node, Per-Bucket, and Per-Data-Type Encryption] + **** xref:learn:security/transparent-encryption-decryption.adoc[Transparent Encryption and Decryption] + **** xref:learn:security/master-password-encryption-at-rest.adoc[Master Password Encryption at Rest] + **** xref:learn:/security/encryption-key-management-services.adoc[Encryption Key Management Services] + **** xref:learn:security/multiple-KMS-multiple-keys.adoc[Multiple KMSs and Multiple Keys] + **** xref:learn:security/encryption-key-rotation-and-expiration.adoc[Encryption Key Rotation and Expiration] + **** xref:learn:security/when-CBServer-encrypts-data.adoc[When Couchbase Server Encrypts Data] + **** xref:learn:security/key-health-monitoring-alerts.adoc[Key Health Monitoring and Alerts] .Manage * xref:manage:management-overview.adoc[Overview] @@ -171,6 +180,9 @@ include::third-party:partial$nav.adoc[] **** xref:manage:manage-security/handle-certificate-errors.adoc[Certificate Error Handling] ** xref:manage:manage-security/manage-tls.adoc[Manage On-the-Wire Security] ** xref:manage:manage-security/manage-native-encryption-at-rest.adoc[] + *** xref:manage:manage-security/encrypt-data-at-rest.adoc[Encrypt Data at Rest] + *** xref:manage:manage-security/manually-drop-DEKs-reencrypt-data.adoc[Manually Drop DEKs to Re-Encrypt Data] + *** xref:manage:manage-security/manage-encryption-at-rest-keys.adoc[Manage Encryption Keys] ** xref:manage:manage-security/manage-auditing.adoc[Manage Auditing] ** xref:manage:manage-security/manage-sessions.adoc[Manage Sessions] ** xref:manage:manage-security/manage-console-access.adoc[Manage Console Access] diff --git a/modules/learn/pages/security/encryption-at-rest-keys.adoc b/modules/learn/pages/security/encryption-at-rest-keys.adoc new file mode 100644 index 0000000000..29bcaedbe0 --- /dev/null +++ b/modules/learn/pages/security/encryption-at-rest-keys.adoc @@ -0,0 +1,376 @@ += Encryption-at-Rest Keys +:description: To encrypt data at rest, you must create at least 1 encryption key. Couchbase Server uses the keys you create to encrypt data encryption keys. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +It then uses those data encryption keys to encrypt data on disk. +This two-tier system lets Couchbase Server manage the rotation and expiration of the DEKs, even when another system manages the encryption keys. + +You have 2 primary choices to make when creating an encryption key: + +* What system you want to manage the key. +You can have Couchbase Server manage encryption keys or have a third-party Key Management Service (KMS) manage them. +For more information, see xref:learn:security/encryption-at-rest-keys.adoc[Key Management Services]. + +* What data Couchbase Server can use the encryption key to encrypt. +You can restrict the key to encrypting 1 or more types of data: + +** *Other encryption keys:* You can enable an encryption-at-rest key to be a Key Encryption Key (KEK) for other encryption-at-rest keys. +KEKs are useful when you want to use multiple layers of encryption keys. +For example, you can use a KEK managed by a third-party KMS as a primary encryption key that encrypts encryption-at-rest keys managed by Couchbase Server. + +** *Buckets:* You can enable a key to encrypt all buckets or restrict it to specific buckets. + +** *Configuration:* By default, Couchbase Server uses the master password to encrypt configuration data. +You can choose to use an encryption-at-rest key instead. + +** *Logs:* You can choose to use an encryption-at-rest key to encrypt logs. +You can also use the master password instead. + +** *Audits:* As with configuration and log data, you can choose to use an encryption-at-rest key or the master password to encrypt audit data. + +** *Other:* You can enable a key to encrypt service-generated data, such as Query Service temporary spill files. + <> for more information. + +[#two-tier-encryption-model] +== Two-Tier Encryption Model (KEK to DEK) + +Couchbase Server uses a two-tier key hierarchy to encrypt data at rest: + +=== Key Encryption Keys (KEKs) +You create and manage KEKs through the Couchbase Server REST API or UI. +A KEK encrypts and decrypts 1 or more Data Encryption Keys. +You can store a KEK locally (managed by Couchbase Server) or in an external KMS such as AWS KMS, Azure Key Vault, GCP KMS, HashiCorp Vault, or a KMIP-compatible server. + +=== Data Encryption Keys (DEKs) +Couchbase Server generates and manages DEKs automatically on each node. +A DEK encrypts the actual data written to disk. +Each node maintains separate DEKs for each encryption data type (buckets, configuration, logs, audit, and other). + +This design provides the following benefits: + +Reduced external KMS calls:: +Couchbase Server only contacts the external KMS to decrypt KEKs. +After decryption, it uses the KEKs locally to encrypt and decrypt DEKs. +This reduces latency and avoids an availability dependency on the external KMS during normal operations. + +Independent key rotation:: +You can rotate KEKs and DEKs independently. +Couchbase Server automatically rotates DEKs at a configurable interval without contacting the external KMS for each rotation. + <> for more information. + +Granular data isolation:: +You can assign different KEKs to different data types. +For example, you can use 1 KEK for bucket data and a different KEK for configuration data. +In multi-tenant configurations, each tenant's bucket can use its own KEK. + <> for more information. + +IMPORTANT: Use an external KMS key only as a KEK for locally managed keys. +Do not assign an external KMS key directly to encrypt data. +Doing so causes Couchbase Server to call the external KMS for every DEK decryption, which increases latency and creates an availability dependency. + <> for more information. + +== Encrypting Bucket Data + +When using native encryption at rest to encrypt data in buckets, you choose which buckets to encrypt. +For example, you can decide to just encrypt buckets containing sensitive data such as customer information. +Then you would leave less-sensitive data unencrypted, such as product catalog data. +Encrypting just sensitive data can help reduce the overhead of encrypting and decrypting data on your cluster. + +Each bucket can have its own encryption key. +This configuration is useful in multi-tenancy configurations where each customer may want to manage their own encryption key. + +== Encrypting Audit, Logs, and Configuration Data + +In addition to data, you can encrypt audit, logs, and most configuration data. +You enable the encryption of each of these types of data separately. +You can use either an encryption-at-rest key or the master password to encrypt data. + +=== Configuration Data + +Configuration data includes cluster topology, security settings, passwords, certificates, and service configuration files. +Now, it also includes the Prometheus monitoring configuration and indexing MetaKV entries. + +By default, Couchbase Server uses the master password to encrypt configuration data. +Couchbase Server does not set the master password by default. +You must set it on each node in the cluster. +For more information, see xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password]. + +IMPORTANT: You may notice that Couchbase Server reports that the configuration data as partially encrypted using the master password, even if you have not set it. +When you first create the Couchbase Cluster, the master password is set to the empty string (`""`). +Couchbase Server uses this empty password to encrypt the configuration data. +Because the password is known, the encryption does not provide any security. +Be sure to set the master password on each node in the cluster to make sure that the configuration data is encrypted with a secure password. +You can also assign an encryption-at-rest key to encrypt configuration data instead of the master password. + +[NOTE] +==== +* Some configuration data cannot be encrypted. +This includes: + +** Bootstrap information. +** Node and internal client certificates. +** Prometheus configuration, metric data, and tokens used to gather metrics. + +* Logs of some services are not encrypted by NS Server. +For example, Analytics log files such as `analytics_debug.log` and `analytics_periodic_dump.log` remain unencrypted even when encryption-at-rest for logs is enabled. +==== + +==== Log Data + +When you enable log encryption, Couchbase Server encrypts all new log entries it writes to disk. +Now, this also includes service-specific log artifacts: + +Server log files:: +Standard Couchbase Server log files. + +First Failure Data Capture (FFDC) files:: +Diagnostic files that contain sensitive information, including xref:n1ql-rest-admin:index.adoc#Requests[completed and active requests] data and xref:n1ql-rest-admin:index.adoc#Vitals[Vitals] information. +Only FFDC files that contain sensitive data are encrypted. + +Archive request files:: +Files that persist completed query requests when xref:n1ql:n1ql-manage/monitoring-n1ql-query.adoc#sys-history[streaming of completed requests] is enabled. + +Index Service statistics logs:: +The `indexer_stats.log` and `projector_stats.log` files. + +==== Audit Data + +Audit data includes the audit event files that Couchbase Server generates. +You can assign an encryption-at-rest key or use the master password to encrypt audit data. + +[#service-generated-data] +== Service-Specific Details + +Individual services encrypt their own service-generated files when you enable encryption at rest. +Each service uses the encryption key associated with the appropriate data type. +You do not need to configure encryption separately for each service. +When you enable encryption for a data type, services that write that data type encrypt their files automatically. + +The following sections describe what each service encrypts and which encryption data type applies. + +=== Index Service (GSI) + +When you enable encryption at rest, the Index Service encrypts its data files, statistics, and metadata. +The Index Service supports encryption for all storage modes, Plasma (the default), Memory-Optimized (MOI), and BHive. + +==== What the Index Service Encrypts + +[cols="2,1"] +|=== +| Data | Encryption Data Type + +| Index data files +| Bucket + +| Index definitions +| Bucket + +| Persisted index statistics +| Bucket + +| Codebook files for vector indexes +| Bucket + +| Backup index data files +| Bucket + +| Request handle cache +| Bucket + +| Scan backfill temporary files +| Bucket + +| Indexing MetaKV entries +| Configuration + +| `indexer_stats.log` and `projector_stats.log` +| Log +|=== + +==== File Path Changes + +When you enable encryption, the Index Service removes bucket and index names from index storage path names. +This prevents the filesystem from exposing sensitive naming information. + +For example, when you enable encryption, the Index Service replaces an index path that contained the bucket name and index name with opaque identifiers. +This change applies to all new index files created after you enable encryption. + +==== Rebalance Behavior + +The Index Service preserves encryption keys during file-based rebalance. +When you use file-based rebalance to move index data between nodes, the encryption keys transfer along with the data files. +The Index Service also enforces shard cleanup during DLI (Deferred Local Index) rebalance when encryption is enabled. + +==== Observability + +You can check the encryption status of every index using a REST API endpoint. +// TODO: :info: confirm endpoint details with the Index team — a REST endpoint shows the encryption status of every bucket, vBucket, and index. + +For more information about the Index Service, see xref:learn:services-and-indexes/services/index-service.adoc[Index Service]. + +=== Search Service + +When you enable encryption at rest for a bucket, the Search Service encrypts sensitive portions of its index files for indexes associated with that bucket. + +==== What the Search Service Encrypts + +The Search Service encrypts the following data within its index files: + +* Stored fields +* Doc values +* Vector index data +* Synonyms data + +Other metadata portions of Search index files are not encrypted. +These portions do not contain sensitive data, and encrypting them would reduce Search performance. + +==== How to Configure + +You do not configure Search encryption separately. +When you enable encryption for a bucket, the Search Service automatically encrypts the sensitive portions of any Search index that maps to that bucket. +When you turn off encryption for the bucket, the Search Service stops encrypting new data. +The Search Service continues to decrypt data that was encrypted before encryption was disabled. + +==== Performance Considerations + +Encrypting Search index data adds overhead to both write and read operations. +You may observe increased latency and reduced throughput for Search queries, especially for vector search workloads. +Test your workload with encryption enabled to understand the performance impact in your environment. + +For more information about the Search Service, see xref:learn/services-and-indexes/services/search-service.adoc[Search Service]. + +=== Backup Service + +Now, the Backup Service integrates with the centralized encryption-at-rest framework. + +NOTE: The Backup Service and `cbbackupmgr` already support encrypting backup data independently using a KMS-managed repository key. +For more information, see xref:learn:services-and-indexes/services/backup-service.adoc[Backup Service]. +The changes described here are about encrypting the Backup Service's own operational files — not backup data itself. + +==== What the Backup Service Encrypts + +When you enable encryption at rest, the Backup Service encrypts the following: + +Backup log files:: +The Backup Service encrypts its log output when you enable log encryption. +This includes logs written by `cbbackupmgr` within the backup archive (under the `logs/` directory). + +Continuous backup metadata:: +When you use continuous backup (point-in-time recovery), the Backup Service encrypts metadata files that track backup state and restore points. + +==== Bucket Identifiers in Backup Paths + +Now, the Backup Service uses bucket UUIDs instead of bucket names in backup storage paths when encryption is enabled. +This prevents bucket names from being exposed through the filesystem, consistent with the Index Service's approach to path obfuscation. + +=== Query Service + +The Query Service uses Data Encryption Keys (DEKs) to encrypt sensitive data on disk. +It encrypts different types of files under different encryption data types. + +For more information, see xref:learn:services-and-indexes/services/query-service.adoc[Query Service]. + +==== Other Encryption Data Type + +When you enable "other" encryption, the Query Service encrypts temporary spill files used during query processing, such as files created for spilled sorting operations. + +For more information, see xref:manage:manage-security/manage-encryption-at-rest.adoc[Manage Native Encryption at Rest]. + +==== Bucket Encryption Data Type + +When you enable encryption for a bucket, the Query Service encrypts: + +IndexScan backfill files:: +The Index and Search Service clients create temporary backfill files when scan results exceed the capacity of the buffered channel between the client and the Query Service. +The Query Service encrypts these files with the associated bucket's encryption key. ++ +The size of the buffered channel is managed by the xref:n1ql:n1ql-manage/query-settings.adoc#scan-cap-srv[pipeline-batch] setting. + +Sequential scan spill files:: +When the Query Service performs a xref:indexes:query-without-index.adoc[sequential scan] over a bucket, it creates temporary spill files on disk. +The Query Service encrypts these files with the bucket's encryption key. + +==== Log Encryption Data Type + +When you enable log encryption, the Query Service encrypts: + +First Failure Data Capture (FFDC) files:: +Diagnostic files created when the Query Service detects incidents or conditions that may lead to failure. +Only FFDC files that contain sensitive data are encrypted, including files with xref:n1ql-rest-admin:index.adoc#Requests[completed and active requests] information and xref:n1ql-rest-admin:index.adoc#Vitals[Vitals] information. + +Archive request files:: +Files that persist completed requests when xref:n1ql:n1ql-manage/monitoring-n1ql-query.html#sys-history[streaming of completed requests] is enabled. + +==== Decrypting Query Service Files + +Use the `cbcat` CLI tool to decrypt encrypted FFDC files and archived request files. + +==== Observability + +Use the Query Admin REST API to check the encryption-at-rest status on a Query node: + +[source,console] +---- +curl -u : http://:8093/admin/encryption_at_rest +---- + +.Example response +[source,json] +---- +{ + "keys.in_use": { + "logs": ["", "c8e2a394-0498-4c37-8380-a61f434c7b80"] + } +} +---- + +`keys.in_use`:: +An object containing the DEK IDs currently in use to encrypt data on disk. +The list may include the special key ID `""` (an empty string), which indicates the presence of unencrypted data on disk. +When encryption at rest is enabled for a data type, the list always includes the current active key. + +*Required roles:* Full Admin, Security Admin, or Read-Only Security Admin. + +[#eventing-service] +=== Eventing Service + +The Eventing Service adopts the centralized encryption-at-rest framework. +When you enable encryption at rest, the Eventing Service encrypts its metadata and service-generated artifacts. + +==== What the Eventing Service Encrypts + +The Eventing Service encrypts the following when encryption is enabled: + +* Eventing function definitions and configuration documents +* Timer-related metadata +* Checkpoint data (per-vBucket processing state) +* DCP stream metadata +* Application log (applog) files — encrypted when you enable log encryption + +Eventing application log files are stored in the `/opt/couchbase/var/lib/couchbase/data/@eventing/` directory. +When you enable log encryption, the Eventing Service encrypts new log entries as it writes them. + +==== Log Rotation Requirement + +The Eventing `enable_applog_rotation` setting must remain `true` (the default) when encryption at rest is enabled. +Couchbase Server blocks attempts to disable log rotation because encryption at rest depends on it to rotate encrypted log files properly. + +If you attempt to set `enable_applog_rotation` to `false` while encryption at rest is enabled, the request is rejected. + +==== Decrypting Eventing Files + +Use the `cbcat` CLI tool to decrypt encrypted Eventing log files and metadata. +Run `cbcat` with the `--with-gosecrets` flag and the path to `gosecrets.cfg`. + +=== Diagnostic and Tool-Generated Files + +Several services generate diagnostic files that contain potentially sensitive information. +When you enable encryption at rest, these files are encrypted as they are written. + +Use the `cbcat` CLI tool to decrypt any encrypted service-generated file. +Run `cbcat` with the `--with-gosecrets` flag and the path to `gosecrets.cfg`. + +IMPORTANT: Encryption at rest does not retroactively encrypt service-generated files that were written before you enabled encryption. +Only files written after enablement are encrypted. + diff --git a/modules/learn/pages/security/encryption-key-management-services.adoc b/modules/learn/pages/security/encryption-key-management-services.adoc new file mode 100644 index 0000000000..63d787a641 --- /dev/null +++ b/modules/learn/pages/security/encryption-key-management-services.adoc @@ -0,0 +1,569 @@ += Encryption Key Management Services +:description: A key management service (KMS) creates, stores, and manages the lifecycle of encryption keys. Couchbase Server uses a KMS to manage its encryption-at-rest keys. It uses these keys to encrypt and decrypt data encryption keys (DEKs) which are the keys nodes use to encrypt and decrypt the data they store on disk. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +When you create an encryption-at-rest key, you choose which KMS you want to manage it. +Couchbase Server works with several KMSs: + +[cols="1,1,1"] +|=== +| KMS | Key Location | Minimum Version + +| <> +| Local (on each node) +| 8.0 + +| <> +| AWS +| 8.0 + +| <> +| Google Cloud Platform +| The next major release + +| <> +| Microsoft Azure +| The next major release + +| <> +| HashiCorp Vault server +| The next major release + +| <> +| KMIP-compatible server +| 8.0 +|=== + +[#couchbase-server-secret-management] +== Couchbase Server Secret Management +You can have Couchbase Server act as a KMS. +When you use Couchbase Server as a KMS, it generates, stores, and manages the encryption keys it uses to encrypt DEKs. + +When Couchbase Server manages the keys, it stores them locally in an encrypted format. +You can use the master password to encrypt encryption-at-rest keys. +For more information about the master password, see xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password]. +This password is not set by default--you must configure it for each node. +For more information about the master password, see xref:manage:manage-security/manage-system-secrets.adoc[]. + +You can also use an encryption key managed by another KMS as a key encryption key (KEK) to encrypt the keys that Couchbase Server manages. +In this configuration, Couchbase Server relies on the remote KMS to decrypt its encryption keys. +This is the recommended configuration when using an external KMS. +It limits the number of encryption and decryption requests Couchbase Server has to send to the external KMS. +After Couchbase Server has the KMS decrypt its own encryption keys, it can use them to encrypt and decrypt DEKs locally. + +[#aws-key-management-service] +== AWS Key Management Service +You can use the https://docs.aws.amazon.com/kms/latest/developerguide/overview.html[AWS Key Management Service] (AWS KMS) to manage encryption-at-rest keys. +This method enhances security by keeping the encryption keys stored in AWS rather than on the Couchbase Server cluster. +Couchbase Server sends encryption and decryption requests to the AWS KMS. + +=== Prerequisites + +Before you configure an AWS KMS encryption key in Couchbase Server, complete the following steps in AWS: + +. Create a symmetric encryption key in AWS KMS. +Record the key's Amazon Resource Name (ARN). + +. Create an IAM user or role with the following minimum permissions on the key: ++ +-- +* `kms:Encrypt` +* `kms:Decrypt` +* `kms:DescribeKey` +-- + +. Generate an access key ID and secret access key for the IAM user (or configure an instance profile if running on EC2). + +=== Integration with Couchbase Server + +When you create an encryption-at-rest key using the AWS KMS provider, you supply the following: + +[cols="1,2"] +|=== +| Parameter | Description + +| `keyARN` +| The Amazon Resource Name (ARN) of the AWS KMS key. + +| `keyRegion` +| The AWS region where the key resides (for example, `us-east-1`). + +| `authByInstanceMetadata` +| Set to `true` to use EC2 instance metadata (IMDS) for authentication instead of explicit credentials. + +| `accessKeyId` / `secretAccessKey` +| The IAM credentials for the user or role. +Not required if you use instance metadata. +|=== + +=== Considerations + +* AWS KMS enhances security by keeping key material stored in AWS rather than on the Couchbase Server cluster. + +* The cluster relies on AWS for key operations. +Disruptions in AWS or in the network between your cluster and AWS can cause errors because Couchbase Server cannot encrypt or decrypt DEKs. + +* Using AWS KMS can cause latency if Couchbase Server relies on it for many encryption keys. +For more information, see <>. + +[#gcp-cloud-kms] +== GCP Cloud KMS + +You can use https://cloud.google.com/kms[Google Cloud Key Management Service^] (Cloud KMS) to manage encryption-at-rest keys. +Key material resides entirely in GCP — it never leaves the Cloud KMS boundary. +Couchbase Server sends encrypt and decrypt API calls to Cloud KMS. + +=== Prerequisites + +Before you configure a GCP Cloud KMS encryption key in Couchbase Server, complete the following steps in GCP: + +. *Create a key ring and key.* +In the Google Cloud Console, navigate to *Security > Key Management* and create a key ring in the region closest to your Couchbase Server cluster. +Create a symmetric encrypt/decrypt key within the key ring. +Record the key's resource name, which follows this format: ++ +---- +projects//locations//keyRings//cryptoKeys/ +---- + +. *Create a service account.* +Create a service account (or use an existing one) with the following IAM role on the key: ++ +-- +* `roles/cloudkms.cryptoKeyEncrypterDecrypter` — allows the service account to encrypt and decrypt data using the key. +-- + +. *Generate a service account key file.* +Download a JSON key file for the service account. +This file contains the credentials Couchbase Server uses to authenticate with Cloud KMS. ++ +Alternatively, if your Couchbase Server nodes run on Google Compute Engine, you can use the instance's attached service account for authentication instead of a key file. + +=== Integration with Couchbase Server + +When you create an encryption-at-rest key using the GCP Cloud KMS provider, you supply the following: + +[cols="1,2"] +|=== +| Parameter | Description + +| `keyResourceName` +| The full resource name of the Cloud KMS key (for example, `projects/my-project/locations/us-central1/keyRings/my-ring/cryptoKeys/my-key`). + +| `credentialsFile` +| The path to the JSON service account key file on each Couchbase Server node. +Not required if you use instance-attached service account credentials. +|=== + +NOTE: Place the service account key file on every node in the cluster at the same path. +Couchbase Server reads this file to authenticate with Cloud KMS. + +==== Considerations + +* GCP Cloud KMS enhances security by keeping key material stored in GCP rather than on the Couchbase Server cluster. + +* The cluster relies on GCP for key operations. Disruptions in GCP or in the network between your cluster and GCP can cause errors because Couchbase Server cannot encrypt or decrypt DEKs. + +* Using GCP Cloud KMS can cause latency if Couchbase Server relies on it for many encryption keys. See <> for guidance. + +*Ensure that the GCP service account credentials file is present at the exact same file path on every node in the cluster. Couchbase Server does not distribute this file automatically. + +[#azure-key-vault] +== Azure Key Vault + +You can use https://azure.microsoft.com/en-us/products/key-vault[Microsoft Azure Key Vault^] to manage encryption-at-rest keys. +Key material resides entirely in Azure Key Vault — it never leaves the Azure boundary. +Couchbase Server sends encrypt and decrypt API calls to Azure Key Vault. + +=== Prerequisites + +Before you configure an Azure Key Vault encryption key in Couchbase Server, complete the following steps in Azure: + +. *Create a Key Vault.* +In the Azure Portal, create a Key Vault (or use an existing one). +Choose the *Standard* or *Premium* pricing tier. +The Premium tier uses Hardware Security Modules (HSMs) for key protection. +Record the vault's URI (for example, `\https://my-vault.vault.azure.net/`). + +. *Create a key.* +In the Key Vault, generate a new RSA key or EC key for encryption. +Record the key name and, optionally, the key version. ++ +NOTE: Couchbase Server uses the key for wrap/unwrap operations (envelope encryption), not direct encrypt/decrypt of data. + +. *Register an application.* +In *Microsoft Entra ID* (formerly Azure Active Directory), register an application to represent Couchbase Server. +Record the following: ++ +-- +* *Application (client) ID* +* *Directory (tenant) ID* +* *Client secret* (or client certificate) — create a new client secret under the application's *Certificates & secrets* settings and record the secret value. +-- + +. *Assign Key Vault permissions.* +Grant the registered application the following key permissions on the Key Vault: ++ +-- +* `wrapKey` +* `unwrapKey` +-- ++ +You can grant these permissions using either Key Vault access policies or Azure RBAC (the *Key Vault Crypto User* role includes both permissions). + +=== Integration with Couchbase Server + +When you create an encryption-at-rest key using the Azure Key Vault provider, you supply the following: + +[cols="1,2"] +|=== +| Parameter | Description + +| `vaultUrl` +| The Key Vault URI (for example, `\https://my-vault.vault.azure.net/`). + +| `keyName` +| The name of the key in the Key Vault. + +| `keyVersion` +| _(Optional)_ The specific key version. +If omitted, Couchbase Server uses the latest version. + +| `tenantId` +| The Microsoft Entra ID (Azure AD) tenant ID. + +| `clientId` +| The application (client) ID of the registered application. + +| `clientSecret` +| The client secret value for the registered application. + +| `azureDomain` +| _(Optional)_ The Azure domain to use. +Defaults to the global Azure environment. +Set this if you use a sovereign cloud (for example, Azure Government or Azure China). +|=== + +=== Considerations + +* Azure Key Vault follows the same external-key model as AWS KMS and GCP Cloud KMS — key material never resides on Couchbase Server nodes. +* The cluster relies on Azure for key operations. +Disruptions in Azure or in the network between your cluster and Azure can cause errors. +* The algorithm field is immutable after key creation — you cannot change the algorithm of an existing Azure Key Vault encryption key. +* No backward compatibility concerns — this is a new KMS provider. +* For best practices, see <>. + + +[#hashicorp-vault] +== HashiCorp Vault + +You can use https://www.hashicorp.com/products/vault[HashiCorp Vault^] to manage encryption-at-rest keys. +Couchbase Server uses the Vault https://developer.hashicorp.com/vault/docs/secrets/transit[Transit secrets engine^] for encrypt and decrypt operations. +Key material resides entirely in the Vault server — it never leaves the Vault boundary. + +NOTE: HashiCorp Vault is a key management system, not an at-rest encryption solution by itself. +Couchbase Server uses Vault as an external KMS to manage KEKs. +The actual data encryption is performed by Couchbase Server using DEKs. + +=== Prerequisites + +Before you configure a HashiCorp Vault encryption key in Couchbase Server, complete the following steps in Vault: + +. *Enable the Transit secrets engine.* ++ +[source,console] +---- +vault secrets enable transit +---- + +. *Create a named encryption key.* ++ +[source,console] +---- +vault write -f transit/keys/couchbase-kek type=aes256-gcm96 +---- ++ +Record the key name (for example, `couchbase-kek`). +Couchbase Server uses this name when sending encrypt and decrypt requests. + +. *Create a policy.* +Create a Vault policy that grants encrypt and decrypt access to the Transit key: ++ +[source,hcl] +---- +path "transit/encrypt/couchbase-kek" { + capabilities = ["update"] +} + +path "transit/decrypt/couchbase-kek" { + capabilities = ["update"] +} + +path "transit/keys/couchbase-kek" { + capabilities = ["read"] +} +---- + +. *Configure an authentication method.* +Couchbase Server supports the following Vault authentication methods: ++ +-- +* *Token authentication* — Provide a Vault token directly. +The token must have the policy created in the previous step attached. +* *TLS certificate authentication* — Provide a client certificate and private key. +Vault uses mutual TLS to authenticate Couchbase Server. +The private key must be in PKCS#8 format. +-- ++ +For TLS certificate authentication, configure a certificate auth method in Vault and create a certificate role: ++ +[source,console] +---- +vault auth enable cert +vault write auth/cert/certs/couchbase-role \ + display_name=couchbase \ + policies=couchbase-transit-policy \ + certificate=@/path/to/client-cert.pem +---- + +=== Integration with Couchbase Server + +When you create an encryption-at-rest key using the HashiCorp Vault provider, you supply the following: + +[cols="1,2"] +|=== +| Parameter | Description + +| `address` +| The URL of the Vault server (for example, `\https://vault.example.com:8200`). + +| `keyName` +| The name of the Transit key (for example, `couchbase-kek`). + +| `transitMountPath` +| _(Optional)_ The mount path of the Transit secrets engine. +Defaults to `transit`. + +| `authMethod` +| The authentication method: `token` or `cert`. + +| `token` +| _(Token auth only)_ The Vault token. + +| `clientCertPath` +| _(Cert auth only)_ The path to the client certificate file on each node. + +| `clientKeyPath` +| _(Cert auth only)_ The path to the client private key file (PKCS#8 format) on each node. + +| `clientKeyPassphrase` +| _(Cert auth only)_ The passphrase for the client private key. + +| `caCertPath` +| _(Optional)_ The path to the CA certificate used to verify the Vault server's TLS certificate. +If omitted, Couchbase Server uses the system CA bundle. +|=== + +=== Key Rotation + +HashiCorp Vault supports key versioning within the Transit engine. +When you rotate a Transit key in Vault: + +. Vault creates a new key version. +The key name stays the same. +. Trigger rotation of the corresponding encryption key in Couchbase Server (on-demand). +. Couchbase Server re-encrypts all keys encrypted by the Vault key using the new version. + +Previous key versions remain available in Vault for decrypting data encrypted with older versions. + +=== Considerations + +* HashiCorp Vault provides a cloud-agnostic KMS option — useful when your infrastructure spans multiple cloud providers or runs on-premises. +* The cluster relies on Vault for key operations. +Disruptions in Vault or in the network between your cluster and Vault can cause errors. +* For TLS certificate authentication, place the client certificate and key files on every node in the cluster at the same path. +* PKCS#8 format is required for the client private key. +* No backward compatibility concerns — this is a new KMS provider. +* For best practices, see <>. + +[#kmip-kms] +=== KMSs that support Key Management Interoperability Protocol (KMIP) +https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol[KMIP] is a standards protocol implemented by key management services. +Couchbase Server can work with any KMS that implements this standard. +As with AWS KMS, using a KMIP-compliant KMS enhances security by storing the encryption keys remotely instead of locally in the cluster. +It also has the same downside--Couchbase Server may report errors due to KMS downtime or network issues. + +=== Prerequisites + +Before you configure a KMIP encryption key in Couchbase Server: + +. *Set up a KMIP-compliant KMS.* +Install and configure a KMS that supports KMIP 1.2 or later. + +. *Create a symmetric key in the KMS.* +Create an AES-256 symmetric key and record its KMIP key ID. + +. *Configure mutual TLS authentication.* +KMIP uses mutual TLS for authentication between Couchbase Server and the KMS. +You need: ++ +-- +* A client certificate (`tls.crt`) issued to Couchbase Server +* A client private key (`tls.key`) in PKCS#8 format, encrypted with a passphrase +* The passphrase for the private key +* The CA certificate used to verify the KMIP server's TLS certificate (optional — you can also use the system CA bundle or the Couchbase CA bundle) +-- ++ +Place the certificate and key files on every node in the cluster at the same path. + +=== Encryption Approaches + +Couchbase Server supports two approaches for KMIP key operations: + +[cols="1,2,1"] +|=== +| Approach | Description | Key in Couchbase Memory? + +| `LocalEncrypt` (recommended) +| Couchbase Server fetches the key material from the KMIP server, caches it in memory, and performs encrypt and decrypt operations locally. +This is faster because only the initial key fetch requires a network call to the KMIP server. +| Yes — key material is cached in Couchbase Server memory. + +| `NativeEncryptDecrypt` +| Couchbase Server sends data to the KMIP server for encryption and decryption. +The key material never leaves the KMIP server. +This is more secure because the key is never present in Couchbase Server memory, but it is slower because every encrypt and decrypt operation requires a network call. +| No — key material never leaves the KMIP server. +|=== + +Choose the approach that matches your security and performance requirements: + +* Use `LocalEncrypt` when performance is a priority and you trust the security of the Couchbase Server nodes. +This approach is recommended for most deployments. +* Use `NativeEncryptDecrypt` when regulatory or compliance requirements mandate that key material must never reside outside the KMS. + +=== Integration with Couchbase Server + +When you create an encryption-at-rest key using the KMIP provider, you supply the following: + +[cols="1,2"] +|=== +| Parameter | Description + +| `host` +| The hostname or IP address of the KMIP server. + +| `port` +| The KMIP server port (typically `5696`). +Must be between 1 and 65536. + +| `keyID` +| The KMIP key ID of the symmetric key. + +| `encryptionApproach` +| `LocalEncrypt` or `NativeEncryptDecrypt`. + +| `certPath` +| The path to the client certificate file on each node. + +| `keyPath` +| The path to the client private key file (PKCS#8 format) on each node. + +| `keyPassphrase` +| The passphrase for the client private key. + +| `caSelection` +| How to verify the KMIP server's TLS certificate: `verifyWithSystemCA`, `verifyWithCouchbaseCA`, or `skipServerCertVerification`. + +| `reqTimeoutMs` +| The request timeout in milliseconds (between 1000 and 300000). +|=== + +=== Key Rotation + +To rotate a KMIP key: + +. Generate a new symmetric key in the KMIP server and record its ID. +. Update the KMIP key ID in the corresponding encryption key in Couchbase Server. +. Couchbase Server re-encrypts all keys encrypted by the old KMIP key with the new key. +Previous KMIP key IDs are saved until re-encryption is complete, then removed automatically. + +NOTE: Unlike AWS KMS (which uses transparent rotation), KMIP requires you to manually update the key ID in Couchbase Server after generating a new key in the KMIP server. + +[#bring-your-own-key] +== Bring-Your-Own-Key (BYOK) + +Couchbase Server supports a Bring-Your-Own-Key (BYOK) model through its external KMS integrations. +With BYOK, you create and manage your encryption keys in your own KMS and configure Couchbase Server to use those keys for encryption at rest. + +=== How BYOK Works + +BYOK in Couchbase Server uses envelope encryption: + +. You create a key in your external KMS (AWS KMS, Azure Key Vault, GCP Cloud KMS, HashiCorp Vault, or a KMIP server). +. You configure this external key as a KEK in Couchbase Server. +. Couchbase Server generates local encryption-at-rest keys and encrypts them with your external KEK. +. These local keys encrypt the DEKs, which encrypt the data on disk. + +Your external key never leaves your KMS. +You retain full lifecycle control: creation, rotation, suspension, and deletion. + +=== Benefits + +* *Full key control.* You manage the master key in your own infrastructure or cloud account. +No Couchbase employee or process can access your key material. +* *Compliance.* BYOK satisfies regulatory requirements that mandate customer-managed encryption keys (for example, PCI-DSS, HIPAA, SOX). +* *Centralized key management.* You can use a single KMS to manage keys for Couchbase Server and other systems in your environment. +* *Revocability.* If you disable or delete the external key in your KMS, Couchbase Server can no longer decrypt DEKs. +This provides a cryptographic "kill switch" for the data. + +=== Considerations + +IMPORTANT: If you delete or permanently disable the external key in your KMS, Couchbase Server cannot start or decrypt any data encrypted by keys that depend on it. +This action is irreversible if the KMS does not support key recovery. +Always verify that your KMS supports a soft-delete or key-disabling mechanism before relying on key deletion as a security control. + +* BYOK does not prevent authorized Couchbase Server administrators from reading data while the cluster is running. +When the cluster is operational, the data is decrypted transparently. +BYOK protects against offline access to storage media. + +* Use the recommended multi-layer key hierarchy: external key → local KEK → DEK → data. +For more information, see <>. + +* Key rotation in the external KMS must be coordinated with Couchbase Server. +After rotating a key in your KMS, trigger a corresponding rotation in Couchbase Server. +For more information, see xref:learn:security/encryption-key-rotation-expiration.adoc[]. + +[#remote-kms-best-practice] +== Remote KMS Best Practice +==== +Do not assign encryption keys managed by an external KMS (AWS KMS, GCP Cloud KMS, Azure Key Vault, HashiCorp Vault, or a KMIP KMS) directly to encrypt Couchbase Server data. + +When you assign an encryption key to encrypt a particular type of data, Couchbase Server uses the key to encrypt the DEKs each node uses to encrypt the data. +If a DEK is encrypted with a key managed by an external KMS, Couchbase Server has to send a request to the KMS to decrypt the DEK. +These requests to a remote KMS can take less than a second. +However, a starting cluster needs to decrypt all of the DEKs on all nodes. +The volume of these synchronous requests can result in a significant delay in starting the cluster. + +To avoid this issue, use an external KMS encryption key only as a KEK for encryption-at-rest keys managed by Couchbase Server. +Then assign these Couchbase Server-managed keys to encrypt data. +This method reduces the number of requests to the remote KMS. +Once Couchbase Server has the KMS decrypt its encryption keys, it can decrypt the DEKs locally. + +For more information about using multiple KMSs and multiple keys, see <<#kms-and-keys>>. +==== + +[#kms-and-keys] +== Using Multiple KMSs and Multiple Keys + +In a basic setup, you may choose to use a single encryption key managed by Couchbase Server to encrypt all data. +This configuration is easy to manage, but does not offer much flexibility. + +Couchbase Server does not limit you to a single KMS. +You can choose any KMS for each encryption key. +For example, you can choose to create 1 or more encryption keys managed by AWS KMS or a KMIP-compliant KMS. +Then use these keys as Key Encryption Keys (KEKs) to encrypt keys that Couchbase Server manages. +This method adds a layer of security to the locally managed encryption keys while reducing the number of key retrievals from the remote KMS. + + diff --git a/modules/learn/pages/security/encryption-key-rotation-expiration.adoc b/modules/learn/pages/security/encryption-key-rotation-expiration.adoc new file mode 100644 index 0000000000..17890eec1c --- /dev/null +++ b/modules/learn/pages/security/encryption-key-rotation-expiration.adoc @@ -0,0 +1,212 @@ += Encryption Key Rotation and Expiration +:description: Key rotation periodically deactivates old keys and generates new encryption keys to replace them. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +Frequent rotations limit the amount of data encrypted with any particular key. +It helps limit the exposure of data if a data breach compromises an encryption key. + +You can choose to have Couchbase Server rotate DEKs automatically. +You can also have it automatically rotate encryption keys that it manages. +Rotation of an externally managed encryption key is handled by the KMS that manage it. +AWS transparently rotates its keys, while keeping the key ID the same. +From Couchbase Server's perspective, the key does not change. +KMIP compatible KMSs have their own rotation schemes and may require you to manually rotate keys or update the key IDs in Couchbase Server after rotation. +See your KMS's documentation for more information about how it handles key rotation. + +By default, Couchbase Server automatically rotates DEKs but not the encryption keys it manages. +When Couchbase Server rotates a DEK, it generates a new DEK and uses the active encryption key to encrypt it. +You choose how frequently Couchbase Server rotates DEKs and (if you enable it) the encryption keys it manages. + +When Couchbase Server generates a new encryption-at-rest key or DEK during rotation, it does not immediately delete the deactivated key. +It keeps deactivated keys so it can decrypt the data that's still encrypted by it. +It uses the new key to encrypt data as it writes it to disk. +When rotating DEKs, Couchbase Server does not re-encrypt existing data unless it's mutated. + +Couchbase Server only deletes a deactivated DEK when either: + +* No data uses the DEK for encryption. +Once the last piece of data that relies on the deactivated DEK for decryption is either mutated or deleted, Couchbase Server deletes the unused DEK. + +* The DEK's lifetime elapses. +You can set a maximum lifetime for DEKs which limits how long Couchbase Server can keep it after rotation. +When a DEK's lifetime elapses, Couchbase Server uses the active DEK to re-encrypt data that's still encrypted with the DEK. +It then deletes the expired DEK. + +You can adjust the rotation and lifetime for encryption keys to suit your environment. + +For information on managing native encryption at rest, see xref:manage:manage-security/manage-native-encryption-at-rest.adoc[]. + +[#re-encryption-vs-key-rotation] +== Re-encryption vs Key Rotation + +Key rotation and data re-encryption are related but distinct operations. +Understanding the difference helps you choose the right operation for your security requirements. + +=== Key Rotation (Lightweight) + +When you rotate a KEK or a DEK, Couchbase Server generates new key material and re-wraps the keys below it in the hierarchy: + +---- +Rotate KEK → new KEK material generated + → all DEKs re-wrapped (decrypt with old KEK, encrypt with new KEK) + → data on disk is UNCHANGED +---- + +---- +Rotate DEK → new DEK generated + → new DEK encrypted with current KEK + → old DEK kept for reading existing data + → new data written with new DEK + → existing data on disk is UNCHANGED +---- + +Key rotation is a lightweight operation. +It changes only the key wrappers — the actual data on disk is not rewritten. +The performance impact is minimal. + +=== Data Re-encryption (Expensive) + +Data re-encryption reads all data from disk, decrypts it with the old DEK, re-encrypts it with the new DEK, and writes it back to disk. +This is an I/O-intensive operation that rewrites every piece of data for the affected data type. + +---- +Drop DEKs → new DEK generated + → ALL data read from disk + → data decrypted with old DEK + → data re-encrypted with new DEK + → data written back to disk + → old DEK deleted after all data is re-encrypted +---- + +Data re-encryption (Drop DEKs) is expensive. +The time it takes depends on the amount of data and the I/O capacity of your cluster. + +=== When to Use Each Operation + + +[cols="1,1,1"] +|=== +| Scenario | Operation | Reason + +| Regular security hygiene +| Key rotation (automatic) +| Limits the amount of data encrypted with any single key. +Lightweight and safe to run frequently. + +| Suspected key compromise +| Drop DEKs (force re-encryption) +| Ensures the compromised key is no longer used to protect any data. + +| Compliance requirement for periodic re-encryption +| Configured DEK rotation (recommended) +| Automatically re-encrypts data over time based on DEK lifetime configuration. +Only re-encrypts data that has not already been re-encrypted naturally. +If DEK rotation is not available, Drop DEKs can be used as an alternative, though it re-encrypts 100% of data on every invocation. + +| Transitioning from unencrypted to fully encrypted +| Drop DEKs (one-time) +| Encrypts all existing data that was written before encryption was enabled. +|=== + +TIP: In most environments, automatic DEK rotation is sufficient. +Reserve the Drop DEKs operation for situations where you need to guarantee that no data remains encrypted with a specific key. + +[#drop-deks-force-re-encryption] +== Drop DEKs and Force Re-encryption + +The endpoints are available for managing Data Encryption Keys (DEKs) are: + +Drop DEKs (POST /controller/dropEncryptionAtRestDeks/*) — Drops all existing DEKs and re-encrypts all data when encryption is enabled. + +Force Encryption (POST /controller/forceEncryptionAtRest/*) — Encrypts all currently unencrypted data, typically data that existed before encryption was enabled. This endpoint does not generate new keys or re-encrypt data that is already encrypted. + +=== When to use Drop DEKs: + +* After a suspected key compromise, to ensure all data is re-encrypted with new keys. + +* After disabling and re-enabling encryption, to re-encrypt all data with fresh keys. + +* To meet periodic re-encryption requirements when automatic DEK rotation is not configured. If DEK rotation is enabled, data is re-encrypted automatically and using Drop DEKs for this purpose is unnecessary. + +=== When to use Force Encryption: + +* After enabling encryption for the first time, to encrypt pre-existing data without re-encrypting data that is already encrypted. + +=== Performance Impact + +WARNING: Drop DEKs is an expensive operation. +It reads and rewrites all data for the affected data type. +The time and I/O load depend on the amount of data. +Plan this operation during maintenance windows or periods of low activity. + +For bucket data, the operation triggers compaction to re-encrypt all documents. +During re-encryption, the cluster continues to serve read and write requests, but you may observe increased I/O latency. + +=== Monitoring Re-encryption Progress + +You can monitor the progress of re-encryption using the REST API. +Check the `encryptionAtRestInfo` field in the bucket or system data status: + +* A `dataStatus` of `"partially encrypted"` indicates that re-encryption is in progress or that some data has not yet been re-encrypted. +* A `dataStatus` of `"encrypted"` indicates that all data for that type is encrypted. + +Couchbase Server also exposes the `cm_encr_at_rest_drop_deks_events_total` metric, which tracks the total number of Drop DEKs events. + +For the steps to force re-encryption, see xref:manage:manage-security/manage-encryption-at-rest.adoc#drop-deks[Drop DEKs and Re-encrypt Data]. + +[#minimum-rotation-interval] +== Minimum Rotation Interval + +Couchbase Server enforces a minimum DEK rotation interval to prevent key accumulation issues. + +=== Why a Minimum Exists + +Each time Couchbase Server rotates a DEK, it generates a new DEK and keeps the old one until either: + +* No data references it, or +* The DEK's configured lifetime expires. + +If you set the rotation interval too short, Couchbase Server generates DEKs faster than it can retire them. +This can cause the number of active DEKs to grow until the system reaches its DEK limit and reports a `too_many_deks` error, which blocks further rotation. + +The minimum rotation interval prevents this situation. + +=== Minimum Values + +The minimum DEK rotation interval is *7 days*. + +[cols="1,1,1"] +|=== +| Setting | Minimum | Default + +| DEK rotation interval +| 7 days +| 30 days + +| DEK lifetime +| Must be greater than or equal to the rotation interval +| 365 days + +| Encryption key (KEK) rotation interval +| 1 day +| Not enabled by default +|=== + +NOTE: The DEK lifetime must be greater than or equal to the rotation interval. +If you set a lifetime shorter than the rotation interval, the DEK may expire before the next rotation occurs, causing Couchbase Server to force re-encryption of all data protected by the expired DEK. +This is by design — it ensures that expired DEKs are not kept indefinitely — but it can be expensive if it happens frequently. + +=== Log Retention Rationale + +A new log file is created each time a new Data Encryption Key (DEK) is generated, triggering log file rotation. Because the system retains only a limited number of rotated log files on disk, frequent DEK rotation can result in reduced log history. + +This also applies to on-demand rotation using the drop DEKs procedure. Initiating a DEK drop triggers log rotation, and if performed frequently, it can similarly reduce the available log history. + +=== Recommendations + +* Use the default rotation interval (30 days) unless your compliance requirements mandate more frequent rotation. +* If you shorten the rotation interval, also shorten the DEK lifetime proportionally. +* Monitor the `cm_encr_at_rest_deks_in_use` metric to ensure the number of active DEKs remains stable over time. +* If you see `too_many_deks` errors in your logs, increase the rotation interval or investigate why old DEKs are not being retired. +For information on configuring rotation intervals, see the xref:manage:manage-security/manage-encryption-at-rest.adoc[Manage Native Encryption at Rest]. diff --git a/modules/learn/pages/security/key-health-monitoring-alerts.adoc b/modules/learn/pages/security/key-health-monitoring-alerts.adoc new file mode 100644 index 0000000000..e2a26e52a2 --- /dev/null +++ b/modules/learn/pages/security/key-health-monitoring-alerts.adoc @@ -0,0 +1,378 @@ += Key Health Monitoring and Alerts +:description: Couchbase Server provides metrics, alerts, key testing endpoints, and diagnostic information to help you monitor the health of your encryption-at-rest configuration. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +Use these tools to detect problems before they cause data unavailability or prevent a node from starting. + +[#periodic-key-testing] +== Periodic Key Testing + +You can test encryption keys at any time to verify that Couchbase Server can reach the KMS and encrypt and decrypt data using the key. +For keys managed by an external KMS (AWS KMS, Azure Key Vault, GCP Cloud KMS, HashiCorp Vault, or a KMIP server), testing is critical: network or credential issues can break encryption without triggering immediate alarms. + +=== How Key Testing Works + +When you test a key, Couchbase Server performs a round-trip encrypt/decrypt operation: + +. It generates a test payload. +. It encrypts the payload using the key (contacting the external KMS if applicable). +. It decrypts the encrypted payload. +. It compares the decrypted result to the original payload. + +If any step fails, the test returns an error describing the failure. +Starting with the next major release, key tests bypass the key cache. +This means a test always contacts the external KMS directly, even if the key is cached locally. +This also ensures the test accurately reflects the current state of the external KMS connection — not the cached state. + +=== When to Test Keys + +Test your encryption keys in the following situations: + +* *After configuring a new external KMS key* — to verify the connection, credentials, and permissions are correct before assigning the key to encrypt data. +* *After rotating credentials* (such as AWS access keys, Azure client secrets, or HashiCorp Vault tokens) — to verify the new credentials work. +* *After network or infrastructure changes* — to verify Couchbase Server can still reach the external KMS. +* *When investigating encryption-related alerts* — to pinpoint whether the problem is with the key configuration, the KMS, or the network. +* *Periodically as part of operational health checks* — to catch issues before they cause problems during DEK rotation or node restart. + +=== How to Test Keys + +You can test encryption keys using the Couchbase Server Web Console or the REST API. + +==== Web Console + +. In the Web Console, navigate to *Security > Encryption at Rest*. +. Select the key you want to test and click *Test Encryption Key Settings*. + +==== REST API + +Couchbase Server provides 3 test endpoints: + +[cols="1,2"] +|=== +| Endpoint | Description + +| `POST /settings/encryptionKeys//test` +| Test an existing encryption key by its ID. + +| `POST /settings/encryptionKeys/test` +| Test encryption key settings before creating the key. +Send the same JSON payload as when creating a key with `POST /settings/encryptionKeys`. + +| `PUT /settings/encryptionKeys//test` +| Test modified encryption key settings before saving them. +Send the same JSON payload as when editing a key with `PUT /settings/encryptionKeys/`. + +|=== + +*Required roles:* Full Admin or Security Admin. + +.Example: Test an existing key + +[source,console] + +---- + +curl -u : -X POST \ + http://:8091/settings/encryptionKeys//test + +---- + +.Example response (success) + +[source,json] + +---- + +{ + "result": "success" +} + +---- + +.Example response (failure) + +[source,json] + +---- + +{ + "result": "failure", + "reason": "failed to use key to generate proof: could not encrypt data error: secrets (code=Unknown): operation error KMS: Encrypt, https response error StatusCode: 400, RequestID: 6bbee74e-..., api error AccessDeniedException: ..." +} + +---- + +[#key-test-alerts] +== Key Test Alerts + +Couchbase Server generates alerts when encryption operations fail. +These alerts appear in the Web Console and can be configured to send notifications via email or other alert destinations. + +The following encryption-at-rest alerts are available: + +[cols="2,1,2"] +|=== + +| Alert | Metric | Description + +| Encryption-at-Rest Error +| `cm_encr_at_rest_generate_dek_failures_total` +| Triggered when Couchbase Server fails to generate a new DEK. +If another DEK is present, the system continues using the old DEK (possibly expired) for encryption. +If no DEK exists (first enablement), data stays unencrypted until the issue is resolved. + +| Encryption-at-Rest Error +| `cm_encryption_key_rotation_failures_total` +| Triggered when the system fails to rotate a configured encryption key (KEK). +Failed rotation does not cause immediate service disruption, but can be an early sign of an external KMS connectivity or configuration problem. + +| Encryption-at-Rest Error +| `cm_encryption_service_failures_total` +| Covers a range of encryption key management failures, including problems reading or writing keys to disk, encryption and decryption failures, corrupted key files, and key integrity check failures. +Impact depends on which key is affected — can range from partial data unavailability to complete node failure. + +| Encryption-at-Rest Key Test Failed +|`encr_at_rest_key_test_failed` +| Triggered when a periodic encryption key test fails. The system regularly tests configured encryption keys, and this alert fires if any such test does not pass. +|=== + +=== Investigating Alerts + +When you receive an encryption-at-rest alert: + +. Check the server logs (`ns_log` and debug logs) for the specific error message. +The log entry typically indicates whether the failure is caused by a misconfiguration, an expired credential, or a connectivity problem with the external KMS. + +. Test the encryption key that is configured to encrypt the affected data type. +Use the Web Console (*Security > Encryption at Rest > Test Encryption Key Settings*) or the REST API (`POST /settings/encryptionKeys//test`). + +. If the key test fails, check: + +* External KMS connectivity (can Couchbase Server reach the KMS endpoint?). + +* Credential validity (have access keys, client secrets, or tokens expired?). + +* Permission configuration (does the KMS identity have the required encrypt/decrypt permissions?). + +* Key status in the external KMS (has the key been disabled, scheduled for deletion, or rotated?). + +TIP: Set up alert notifications (email, webhook, or other destination) for encryption-at-rest alerts so you are notified promptly when issues occur. + +For information on configuring steps, see xref:manage:manage-settings/configure-alerts.adoc[Configure Alerts]. + +[#key-test-results-in-api] +== Key Test Results in API + +You can check the encryption status and health of keys programmatically using several REST API endpoints. + +=== Encryption Status Endpoints + +[cols="1,2"] + +|=== + +| Endpoint | What It Returns + +| `GET /pools/default` +| Per-node encryption-at-rest status for configuration, audit, and log data (in the `encryptionAtRestInfo` key). + +| `GET /pools/default/buckets` +| Aggregated per-bucket encryption-at-rest info (`encryptionAtRestInfo` key), and per-bucket per-node info (`bucketEncryptionAtRestInfo` key). + +| `GET /pools/default/buckets/` +| Aggregated encryption-at-rest info for a specific bucket (`encryptionAtRestInfo` key), and per-node info (`bucketEncryptionAtRestInfo` key). + +| `GET /settings/security/encryptionAtRest` +| Aggregated encryption-at-rest info for configuration, log, and audit data (in the `info` key). + +|=== + +Each endpoint returns a `dataStatus` field for each data type: + +[cols="1,2"] + +|=== + +| Value | Meaning + +| `"encrypted"` +| All data of this type is encrypted. + +| `"partiallyEncrypted"` +| Some data is encrypted and some is not. + +| `"unencrypted"` +| No data of this type is encrypted. + +|=== + +=== Query Service Encryption Status + +The Query service provides its own encryption-at-rest status endpoint: + +[source,console] + +---- + +curl -u : http://:8093/admin/encryption_at_rest + +---- + +.Example response + +[source,json] + +---- +{ + "keys.in_use": { + "logs": ["", "c8e2a394-0498-4c37-8380-a61f434c7b80"] + } +} + +---- + +The `keys.in_use` field lists the DEK IDs in use. +An empty string (`""`) indicates the presence of unencrypted data on disk. + +*Required roles:* Full Admin, Security Admin, or Read-Only Security Admin. + +[#encryption-error-alerts] +== Encryption Metrics Reference + +Couchbase Server exposes the following Prometheus metrics for encryption at rest. +You can use these metrics in Grafana dashboards, Prometheus alerting rules, or any compatible monitoring system. + +[cols="2,2"] + +|=== + +| Metric | Description + +| `cm_encr_at_rest_data_status` +| Encryption at rest status per data type. + +Values: `1` (encrypted), `0.5` (partially encrypted), `0` (unencrypted), `-1` (unknown). + +| `cm_encr_at_rest_deks_in_use` +| Number of data keys currently in use per data type. + +A steady increase over time may indicate that old DEKs are not being retired. + +| `cm_encr_at_rest_drop_deks_events_total` +| Total number of Drop DEKs events (when a DEK expires and the system re-encrypts data to stop using it). + +| `cm_encr_at_rest_generate_dek_total` +| Total number of successful DEK generation events. + +| `cm_encr_at_rest_generate_dek_failures_total` +| Total number of DEK generation failures. + +A non-zero value indicates a problem with the encryption key used to encrypt DEKs — test the key immediately. + +| `cm_encryption_key_rotation_failures_total` +| Total number of encryption key (KEK) rotation failures. + +| `cm_encryption_key_rotations_total` +| Total number of successful encryption key (KEK) rotations. + +| `cm_encryption_service_failures_total` +| Total number of encryption service failures. + +This is a broad metric covering key read/write errors, encryption/decryption failures, corrupted key files, and key integrity check failures. + +|=== + +=== Recommended Alert Rules + +[cols="2,1,1,2"] + +|=== + +| Alert Name | Expression | Threshold | Severity + +| DEK Generation Failures +| `increase(cm_encr_at_rest_generate_dek_failures_total[5m]) > 0` +| Any failure +| Critical — data may remain unencrypted + +| Key Rotation Failures +| `increase(cm_encryption_key_rotation_failures_total[5m]) > 0` +| Any failure +| Warning — early sign of KMS connectivity issue + +| Encryption Service Failures +| `increase(cm_encryption_service_failures_total[5m]) > 0` +| Any failure +| Critical — can cause data unavailability or node failure + +| DEK Accumulation +| `cm_encr_at_rest_deks_in_use > 50` +| Per data type +| Warning — old DEKs may not be retiring; check rotation interval + +| Encryption Status Degraded +| `cm_encr_at_rest_data_status < 1` +| Per data type +| Info — data is partially encrypted or unencrypted + +|=== + +[#dek-expiry-diagnostics] +== DEK Expiry in Diagnostics Endpoint + +The diagnostics endpoint `GET /diag/encryptionAtRest` provides detailed information about the encryption-at-rest state of each node, including debug information about DEKs, timers, and failed DEK-related jobs. + +Starting with the next major release, this endpoint also shows whether each DEK is expired or not. + +This enhancement helps you identify DEKs that have exceeded their configured lifetime and are pending re-encryption. + +=== Accessing the Diagnostics Endpoint + +[source,console] +---- +curl -u : http://:8091/diag/encryptionAtRest +---- + +*Required roles:* Full Admin. + +=== What the Endpoint Returns + +The diagnostics endpoint returns the following information for each node: + +DEK details:: +For each data type (buckets, configuration, logs, audit, other), the endpoint lists: + +* The DEK IDs currently in use. + +* Whether each DEK is active (the current encryption key) or deactivated (rotated out, kept for decryption). + +* Whether each DEK is expired — meaning its configured lifetime has elapsed and the system is attempting to re-encrypt data to retire it. + +* The creation and rotation timestamps for each DEK. + +Timer information:: +Shows the next scheduled DEK rotation time and the next scheduled encryption key (KEK) rotation time for each data type. + +Failed jobs:: +Lists any DEK-related operations that failed, including the failure reason and timestamp. +This is useful for diagnosing why a DEK rotation or Drop DEKs operation did not complete. + +Cached key IDs:: +Lists the IDs of encryption keys that are currently cached in memory. +Key tests bypass this cache, but the cache information is useful for understanding which keys the node is actively using. + +=== When to Use the Diagnostics Endpoint + +Use the diagnostics endpoint for troubleshooting and operational health checks: + +* *After a DEK generation or rotation failure alert* — to see which DEKs are affected and why the operation failed. + +* *When investigating partial encryption status* — to identify which DEKs are expired and pending re-encryption. + +* *During support escalations* — the output from this endpoint is included in `cbcollect_info` and provides comprehensive information for support analysis. + +NOTE: This endpoint is intended for diagnostics and troubleshooting. + +It returns internal implementation details that may change between releases. +Do not build automation that depends on the exact format of this endpoint's output. diff --git a/modules/learn/pages/security/master-password-encryption-at-rest.adoc b/modules/learn/pages/security/master-password-encryption-at-rest.adoc new file mode 100644 index 0000000000..5530ba59bc --- /dev/null +++ b/modules/learn/pages/security/master-password-encryption-at-rest.adoc @@ -0,0 +1,127 @@ += Master Password and Encryption at Rest +:description: The master password is the foundation of the encryption key hierarchy on each Couchbase Server node. It protects locally stored encryption keys and configuration data. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +By default, the master password is empty and is used to encrypt configuration data. This default configuration is not secure. To secure the encryption key hierarchy, you can choose one of the following approaches: + +* Configure a master password on each Couchbase Server node. +The master password protects locally stored encryption keys and configuration data. + +* Use an external KMS by changing the encryption method for configuration data to use an encryption key managed externally. +In this case, a master password is not required. + +== Why the Master Password Matters + +By default, Couchbase Server does not set the master password. +It uses an empty string (`""`) as the password. +This means: + +* Configuration data is encrypted, but with a known password — providing no real security. + +* The Web Console may report configuration as "partially encrypted using the master password" even + if you have not explicitly set one. + +* Locally managed KEKs (Couchbase Server Secret Management keys) are encrypted with this empty + password. + +IMPORTANT: Set the master password on *every* node in the cluster before enabling encryption at +rest. +If you do not set the master password, your encryption at rest deployment is not secure — anyone +with access to the node's file system can decrypt the locally stored keys. + +== Setting the Master Password + +You set the master password individually on each node. +The master password is per-node — each node can have a different master password, although using +the same password across all nodes simplifies operations. + +=== Using the CLI + +Use the `master-password` command to set or change the master password: + +[source,console] +---- +couchbase-cli master-password -c :8091 \ + --new-password +---- + +Couchbase Server prompts you for the new password. + +To send the master password to a node that is waiting for it during startup: + +[source,console] +---- +couchbase-cli master-password --send-password +---- + +You are prompted to enter the password. +You have 3 attempts to enter the correct password. + +NOTE: You must run the `master-password` command on the same host where Couchbase Server is +running. + +=== Using the REST API + +Use the `POST /node/controller/changeMasterPassword` method to set or change the master password: + +[source,console] +---- +curl -X POST -u : \ + http://:8091/node/controller/changeMasterPassword \ + -d newPassword= +---- + +==== Using an Environment Variable + +On all supported operating systems (except macOS), you can use the `CB_MASTER_PASSWORD` environment variable to pass the master password to a node at startup: + +[source,console] +---- +export CB_MASTER_PASSWORD= +---- +This is an alternative to using `couchbase-cli master-password --send-password`. +Set this variable before starting Couchbase Server, or restart the service after setting it. + +NOTE: On macOS, the `CB_MASTER_PASSWORD` environment variable does not work. Instead, add the export to the /`Applications/Couchbase Server.app/Contents/Resources/start-server.sh` script. + +== Master Password at Node Startup + +When you set a master password, Couchbase Server requires the password at every node startup. +This includes restarts due to reconfiguration, failover recovery, or anomalies. + +If Couchbase Server does not receive the master password, it waits indefinitely and does not +complete startup. +You must provide the password using the CLI, the REST API, or the environment variable. + +== Rotating the Master Password + +You can rotate the master password without disrupting the running cluster: + +. Use the CLI command, REST API method, or environment variable to set a new master password on + each node. +. Couchbase Server re-encrypts the data key with the new master key. +. Existing DEKs and KEKs are not affected — only the data key wrapper changes. + +You can also rotate the data key independently: + +[source,console] +---- +couchbase-cli master-password -c :8091 --rotate-data-key +---- + +Or using the REST API: + +[source,console] +---- +curl -X POST -u : \ + http://:8091/node/controller/rotateDataKey +---- + +== Configuring How the Master Password Is Obtained + +You can configure whether Couchbase Server obtains the master password from an environment variable +or from a custom script. +You can also configure the location of encrypted keys and whether you red, write and delete keys as files or through custom scripts. + +For more information, see xref:manage:manage-security/manage-system-secrets.adoc[Manage System Secrets]. diff --git a/modules/learn/pages/security/multiple-KMS-multiple-keys.adoc b/modules/learn/pages/security/multiple-KMS-multiple-keys.adoc new file mode 100644 index 0000000000..3baeffc7f8 --- /dev/null +++ b/modules/learn/pages/security/multiple-KMS-multiple-keys.adoc @@ -0,0 +1,90 @@ +[#using-multiple-kmss-and-multiple-keys] +== Using Multiple KMSs and Multiple Keys +:description: In a basic setup, you may choose to use a single encryption key managed by Couchbase Server to encrypt all data. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +This configuration is easy to manage, but does not offer much flexibility. + +Couchbase Server does not limit you to a single KMS. +You can choose any supported KMS for each encryption key. +For example, you can create one or more encryption keys managed by AWS KMS, Azure Key Vault, GCP Cloud KMS, HashiCorp Vault, or a KMIP-compliant server. +Then use these keys as Key Encryption Keys (KEKs) to encrypt keys that Couchbase Server manages locally. +This method adds a layer of security to the locally managed encryption keys while reducing the number of key retrievals from the remote KMS. + +=== Considerations for Multi-KMS Setups + +* Each external KMS is an availability dependency. +If an external KMS is unreachable, Couchbase Server cannot decrypt the KEKs managed by that KMS, which prevents decryption of the associated DEKs. +Using multiple KMSs means you have multiple availability dependencies. + +* Use the <> for each external KMS. +Assign external keys only as KEKs for locally managed keys — not directly to encrypt data. + +* Different KMSs may have different key rotation procedures. +Coordinate rotation across all KMSs to maintain consistent security posture. +See <> for more information. + +[#encryption-key-rotation-and-expiration] +== Encryption Key Rotation and Expiration + +Key rotation periodically deactivates old keys and generates new encryption keys to replace them. +Frequent rotations limit the amount of data encrypted with any particular key. +It helps limit the exposure of data if a data breach compromises an encryption key. + +You can choose to have Couchbase Server rotate DEKs automatically. +You can also have it automatically rotate encryption keys that it manages. +Rotation of an externally managed encryption key is handled by the KMS that manages it: + +[cols="1,2"] +|=== +| KMS | Rotation Behavior + +| Couchbase Server (CBSM) +| Can be rotated automatically or on-demand. +When rotated, Couchbase Server generates new key material. +After rotation, all keys encrypted by this key are re-wrapped with the new key material. + +| AWS KMS +| AWS transparently rotates key material, keeping the same key ARN. +From Couchbase Server's perspective, the key does not change. +After AWS rotates the key, you must trigger a rotation of the AWS KMS key in Couchbase Server (on-demand only) to re-wrap DEKs. + +| Azure Key Vault +| Azure supports automatic key rotation with configurable policies. +After Azure rotates the key version, trigger a rotation in Couchbase Server to re-wrap DEKs. + +| GCP Cloud KMS +| GCP supports automatic key rotation at configurable intervals. +After GCP rotates the key version, trigger a rotation in Couchbase Server to re-wrap DEKs. + +| HashiCorp Vault +| Vault supports key versioning within the Transit engine. +When you rotate a Transit key in Vault, trigger a rotation in Couchbase Server to re-wrap DEKs with the new key version. + +| KMIP +| You must manually generate a new key in the KMIP server and update the KMIP key ID in Couchbase Server. +See <> for details. +|=== + +By default, Couchbase Server automatically rotates DEKs but not the encryption keys it manages. +When Couchbase Server rotates a DEK, it generates a new DEK and uses the active encryption key to encrypt it. +You choose how frequently Couchbase Server rotates DEKs and (if you enable it) the encryption keys it manages. + +When Couchbase Server generates a new encryption-at-rest key or DEK during rotation, it does not immediately delete the deactivated key. +It keeps deactivated keys so it can decrypt the data that is still encrypted by them. +It uses the new key to encrypt data as it writes it to disk. +When rotating DEKs, Couchbase Server does not re-encrypt existing data unless you force re-encryption. + +Couchbase Server only deletes a deactivated DEK when either: + +* No data uses the DEK for encryption. +Once the last piece of data that relies on the deactivated DEK for decryption is either mutated or deleted, Couchbase Server deletes the unused DEK. + +* The DEK's lifetime elapses. +You can set a maximum lifetime for DEKs, which limits how long Couchbase Server can keep a DEK after rotation. +When a DEK's lifetime elapses, Couchbase Server uses the active DEK to re-encrypt data that is still encrypted with the expired DEK. +It then deletes the expired DEK. + +You can adjust the rotation interval and lifetime for encryption keys to suit your environment. +See xref:manage:manage-security/manage-encryption-at-rest.adoc[Manage Native Encryption at Rest] to learn how to manage these settings. diff --git a/modules/learn/pages/security/native-encryption-at-rest-overview.adoc b/modules/learn/pages/security/native-encryption-at-rest-overview.adoc index 0dc58fd647..9176ac5479 100644 --- a/modules/learn/pages/security/native-encryption-at-rest-overview.adoc +++ b/modules/learn/pages/security/native-encryption-at-rest-overview.adoc @@ -8,7 +8,7 @@ This feature is transparent to the database's users. Couchbase Server automatically decrypts data when reading it from disk and encrypts it when writing it to disk. Encryption at rest covers service-generated files such as Index, Query, Search, and Eventing artifacts. -It also supports additional key management providers such as Azure Key Vault, GCP Cloud KMS, and HashiCorp Vault, in addition to the existing AWS KMS and KMIP options. +It also supports additional key management providers such as Azure Key Vault, GCP Key Management System (KMS), and HashiCorp Vault, in addition to the existing AWS KMS and Key Management Interoperability Protocol (KMIP) options. For steps to take when managing this feature, see xref:manage:manage-security/manage-native-encryption-at-rest.adoc[]. [NOTE] @@ -24,220 +24,121 @@ For example: * Python SDK: xref:python-sdk:howtos:encrypting-using-sdk.adoc[] ==== +== What Encryption at Rest Protects -[#keys] -== Encryption-at-Rest Keys +Encryption at rest protects data that Couchbase Server persists to disk. +It does not protect data held in-memory or data in transit across the network. -To encrypt data at rest, you must create at least one encryption key. -Couchbase Server uses the keys you create directly to encrypt Data Encryption Keys (DEKs) which uses to encrypt the information it stores on disk. -This two-layer system lets Couchbase Server manage the rotation and expiration of the DEKs, even when another system manages the encryption keys. +=== Data Written to Disk -You have two primary choices to make when creating an encryption key: +When you enable encryption at rest, Couchbase Server encrypts the following categories of data as it writes them to persistent storage: -* What system you want to manage the key. -You can have Couchbase Server manage encryption keys or have a third-party Key Management Service (KMS) manage them. -See <<#kms>> for more information about choosing a KMS. +[cols="1,2,1"] +|=== +| Category | Examples | Encryption Data Type -* What data Couchbase Server can use the encryption key to encrypt. -You can restrict the key to encrypting one or more types of data: +| Customer data +| Documents stored in buckets (Magma and Couchstore storage engines), index data files +| Bucket -+ -** **Other encryption keys:** -You can enable an encryption-at-rest key to be a Key Encryption Key (KEK) for other encryption-at-rest keys. -KEKs are useful when you want to use multiple layers of encryption keys. -For example, you can use a KEK managed by a third-party KMS as a primary encryption key that encrypts encryption-at-rest keys managed by Couchbase Server. -** **Buckets:** -You can enable a key to encrypt all buckets or restrict it to specific buckets. -** **Configuration:** -By default, Couchbase Server uses the xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[master password] to encrypt configuration data. -You can choose to use an encryption-at-rest key instead. -** **Logs:** -You can choose to use an encryption-at-rest key to encrypt logs. -You can also use the master password instead. -** **Audits:** -As with configuration and log data, you can choose to use an encryption-at-rest key or the master password to encrypt audit data. +| System data +| Cluster configuration files, passwords, certificates, Prometheus configuration +| Configuration -The following sections explain these choices in greater detail. +| Logs +| Server log files, First Failure Data Capture (FFDC) files, archived request files +| Log -== Encrypting Bucket Data +| Audit records +| Audit event files generated by Couchbase Server +| Audit -When using native encryption at rest to encrypt data in buckets, you choose which buckets to encrypt. -For example, you can decide to just encrypt buckets containing sensitive data (such as customer information). -Then you would leave less-sensitive data unencrypted, such as product catalog data. -Encrypting just sensitive data can help reduce the overhead of encrypting and decrypting data on your cluster. +| Service-generated files +| Query temporary spill files, index scan backfill files, Search index data, Eventing metadata +| Bucket, Log, or Other (varies by service) +|=== -Each bucket can have its own encryption key. -This configuration is useful in multi-tenancy configurations where each customer may want to manage their own encryption key. +=== What Encryption at Rest Does Not Protect -== Encrypting Audit, Logs, and Configuration Data +* *In-memory data.* Documents cached in the managed cache and data processed in flight are not covered. -In addition to data, you can encrypt audit, logs, and most configuration data. -You enable the encryption of each of these types of data separately. -You can use either an encryption-at-rest key or the master password to encrypt data. -By default, Couchbase Server uses the master password to encrypt configuration data. -Couchbase Server does not set the master password by default. -You must set it on each node in the cluster. -See xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password] for more information about the master password. +* *Data in transit.* TLS protects data moving between nodes, or between clients and servers — not encryption at rest. -IMPORTANT: You may notice that Couchbase Server reports that he the configuration data is partially encrypted using the master password, even if you have not set it. -When you first create the Couchbase Cluster, the master password is set to the empty string (`""`). -Couchbase Server uses this empty password to encrypt the configuration data. -Because the password is known, the encryption does not provide any security. -Be sure to set the master password on each node in the cluster to ensure that the configuration data is encrypted with a secure password. +* *Pre-existing unencrypted data.* Data written before you enabled encryption remains unencrypted on disk until you mutate it, compact it, or force re-encryption. -[NOTE] -==== -* Some configuration data cannot be encrypted. -This includes: +== Differences Between Network, Field-Level, and At-Rest Encryption -** Bootstrap information. -** Node and internal client certificates. -** Prometheus configuration, metric data, and tokens used to gather metrics. +Couchbase provides multiple layers of encryption, each protecting data at a different point in its lifecycle. +These layers are complementary — enabling 1 does not replace the others. -* Logs of some services are not encrypted by NS Server. -For example, Analytics log files such as `analytics_debug.log` and `analytics_periodic_dump.log` remain unencrypted even when encryption-at-rest for logs is enabled. -==== +=== Network Encryption -[#when-data-is-encrypted] -== When Couchbase Server Encrypts Data - -When you enable encryption at rest for a bucket or a type of system data, Couchbase Server encrypts all of the data it writes. -However, it does not encrypt existing data in the bucket or already written system data. -For example, if you enable encryption at rest for logs, Couchbase Server encrypts new log entries but does not encrypt existing log entries. -Existing unencrypted data in buckets is encrypted when its mutated or compacted. - -You can use the REST API to force Couchbase Server to encrypt existing unencrypted data. -See xref:rest-api:security/encryption-at-rest/force-encryption-at-rest.adoc[] for more information. - -Similarly to enabling encryption, when you turn off encryption at rest, Couchbase Server does not immediately decrypt all encrypted data. -It stops encrypting the new data it writes but does not decrypt and rewrite the existing encrypted data. -It still decrypts encrypted data as it reads it. -It only rewrites the data in an unencrypted format when it updates the data. -You can force the decryption of data by calling the REST API to drop the DEKs for a bucket or type of system data. -See xref:rest-api:security/encryption-at-rest/drop-encryption-deks.adoc[] for more information. - -[#kms] -== Encryption Key Management Services - -A key management service (KMS) creates, stores, and manages the lifecycle of encryption keys. -Couchbase Server uses a KMS to manage its encryption-at-rest keys. -It uses these keys to encrypt and decrypt data encryption keys (DEKs) which are the keys nodes use to encrypt and decrypt the data they store on disk. - -When you create an encryption-at-rest key, you choose which KMS you want to manage it. -Couchbase Server works with several KMSs: - -Couchbase Server Secret Management:: -You can have Couchbase Server act as a KMS. -When you use Couchbase Server as a KMS, it generates, stores, and manages the encryption keys it uses to encrypt DEKs. - -+ -When Couchbase Server manages the keys, it stores them locally in an encrypted format. -You can use the master password to encrypt encryption-at-rest keys. -See xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password] for more information about the master password. -This password is not set by default--you must configure it for each node. -See xref:manage:manage-security/manage-system-secrets.adoc[] for more information about the master password. - -+ -You can also use an encryption key managed by another KMS as a key encryption key (KEK) to encrypt the keys that Couchbase Server manages. -In this configuration, Couchbase Server relies on the remote KMS to decrypt its encryption keys. -This is the recommended configuration when using an external KMS. -It limits the number of encryption and decryption requests Couchbase Server has to send to the external KMS. -After Couchbase Server has the KMS decrypt its own encryption keys, it can use them to encrypt and decrypt DEKs locally. - -AWS Key Management Service:: -You can use the https://docs.aws.amazon.com/kms/latest/developerguide/overview.html[AWS Key Management Service] (AWS KMS) to manage keys for you. -This method enhances security by keeping the encryption keys stored in AWS rather than on the Couchbase Server cluster. -Couchbase Server sends encryption and decryption requests to the AWS KMS. -One downside of using the AWS KMS is that the cluster relies on an external source for encryption. -Disruptions in AWS or the network can cause errors because the Couchbase Server cannot encrypt or decrypt data encrypted using the keys managed by AWS. -Using AWS KMS can also cause latency issues if Couchbase Server relies on it for many encryption keys. -See <<#aws-kms-caution>> for more information. - - -KMSs that support Key Management Interoperability Protocol (KMIP):: -https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol[KMIP] is a standards protocol implemented by key management services. -Couchbase Server can work with any KMS that implements this standard. -As with AWS KMS, using a KMIP-compliant KMS enhances security by storing the encryption keys remotely instead of locally in the cluster. -It also has the same downside--Couchbase Server may report errors due to KMS downtime or network issues. - - -[#aws-kms-caution] -[CAUTION] -.Remote KMS Best Practice -==== -Do not assign encryption keys managed by AWS KMS or a KMIP KMS to encrypt Couchbase Server data. -When you assign an encryption key to encrypt a particular type of data, Couchbase Server uses the key to encrypt the DEKs each node uses to encrypt the data. -If a DEK is encrypted with a key managed by an external KMS, Couchbase Server has to send a request to the KMS to decrypt the DEK. -These requests to a remote KMS can take less than a second. -However, a starting cluster needs to decrypt all of the DEKs on all nodes. -The volume of these synchronous requests can result in a significant delay in starting the cluster. - -To avoid this issue, only use AWS KMS or a KMIP KMS encryption keys to encrypt encryption-at-rest keys managed by Couchbase Server. -You then assign these Couchbase Server managed keys to encrypt data. -This method reduces the number of requests to the remote KMS. -Once Couchbase Server has the KMS decrypt its encryption keys, it can decrypt the DEKs locally. - -See <<#kms-and-keys>> for more information about using multiple KMSs and multiple keys. -==== +Network encryption protects data in transit — data moving between clients and servers, between nodes in a cluster, or between clusters (for example, during XDCR replication). + +* *Mechanism:* TLS (Transport Layer Security), with configurable minimum TLS version and cipher suites. +* *Scope:* All network communication — client-to-node, node-to-node, and cluster-to-cluster. +* *Protects against:* Eavesdropping, man-in-the-middle attacks, and data interception on the network. +* *Does not cover:* Data stored on disk or data within document fields visible to authorized users. -[#kms-and-keys] -== Using Multiple KMSs and Multiple Keys +For more information, see xref:learn:security/on-the-wire-security.adoc[On-the-Wire Security]. -In a basic setup, you may choose to use a single encryption key managed by Couchbase Server to encrypt all data. -This configuration is easy to manage, but does not offer much flexibility. +=== Field-Level Encryption + +Field-level encryption protects sensitive fields within individual documents. +The encryption and decryption happen in the application layer using Couchbase SDKs. + +* *Mechanism:* Application-side encryption using the Couchbase SDK encryption libraries. +The application encrypts specific JSON fields before writing the document and decrypts them after reading. +* *Scope:* Individual fields within documents (for example, a `credit_card_number` or `ssn` field). +The document key, metadata, and non-sensitive fields remain in plaintext. +* *Key management:* You manage keys in the application, typically through an external key store or key provider configured in the SDK. +* *Protects against:* Unauthorized access to sensitive field values — even database administrators cannot read encrypted field values without the application-level decryption key. +* *Does not cover:* Document keys, metadata, non-encrypted fields, server configuration, logs, or audit files. + + +Depending on your application's requirements, field-level encryption may be more appropriate than encrypting the entire bucket. +See the SDK documentation for your development language for more information. -Couchbase Server does not limit you to a single KMS. -You can choose any KMS for each encryption key. -For example, you can choose to create one or more encryption keys managed by AWS KMS or a KMIP-compliant KMS. -Then use these keys as Key Encryption Keys (KEKs) to encrypt keys that Couchbase Server manages. -This method adds a layer of security to the locally managed encryption keys while reducing the number of key retrievals from the remote KMS. +=== Encryption at Rest -The following diagram shows a possible configuration using a single primary encryption key hosted by AWS KMS. -This key encryption key encrypts 5 encryption-at-rest keys managed by Couchbase Server. -Each of these keys encrypt the DEKs that encrypt the different types of data that's written to disk: audit, configuration, and log data and the data stored in 2 buckets named A and B. -Each node in the cluster has Data Encryption Keys (DEKs) encrypted by the intermediate encryption keys managed by Couchbase Server. -For simplicity, the diagram only shows 2 nodes. -However, this configuration can scale to any size cluster. +Encryption at rest protects all data that Couchbase Server writes to disk — transparently, at the server level, with no application code changes required. -image::security/encryption-at-rest-key-hierarchy.svg["Diagram showing a single AWS key encrypting 5 Couchbase Server managed KEKs which in turn encrypt DEKs on each node"] +* *Mechanism:* Server-side AES-256-GCM encryption using automatically managed Data Encryption Keys (DEKs). +* *Scope:* Bucket data, cluster configuration, server logs, audit records, and service-generated files — each configurable independently. +* *Key management:* A two-tier hierarchy using Key Encryption Keys (KEKs) managed through external providers or locally, plus automatically managed DEKs. +* *Protects against:* Unauthorized access to physical or virtual storage media — stolen disks, compromised storage snapshots, or unauthorized file system access. +* *Does not cover:* Data in-memory or data in transit across the network. -You can have even more complex hierarchies where there are several keys hosted by AWS KMS or a KMIP KMS. +== Supported Editions and Versions +Native encryption at rest is available only in the Enterprise Edition of Couchbase Server. -[#rotation-expiration] -== Encryption Key Rotation and Expiration +.Encryption at Rest Feature Availability +[cols="2,1"] +|=== +| Feature | Minimum Version -Key rotation periodically deactivates old keys and generates new encryption keys to replace them. -Frequent rotations limit the amount of data encrypted with any particular key. -It helps limit the exposure of data if a data breach compromises an encryption key. +| Encryption at rest for buckets, configuration, logs, and audit +| 8.0 -You can choose to have Couchbase Server rotate DEKs automatically. -You can also have it automatically rotate encryption keys that it manages. -Rotation of an externally managed encryption key is handled by the KMS that manage it. -AWS transparently rotates its keys, while keeping the key ID the same. -From Couchbase Server's perspective, the key does not change. -KMIP compatible KMSs have their own rotation schemes and may require you to manually rotate keys or update the key IDs in Couchbase Server after rotation. -See your KMS's documentation for more information about how it handles key rotation. +| AWS KMS and KMIP key providers +| 8.0 -By default, Couchbase Server automatically rotates DEKs but not the encryption keys it manages. -When Couchbase Server rotates a DEK, it generates a new DEK and uses the active encryption key to encrypt it. -You choose how frequently Couchbase Server rotates DEKs and (if you enable it) the encryption keys it manages. +| Azure Key Vault key provider +| The next major release -When Couchbase Server generates a new encryption-at-rest key or DEK during rotation, it does not immediately delete the deactivated key. -It keeps deactivated keys so it can decrypt the data that is still encrypted by it. -It uses the new key to encrypt data as it writes it to disk. -When rotating DEKs, Couchbase Server does not re-encrypt existing data unless it's mutated. +| GCP Cloud KMS key provider +| The next major release -Couchbase Server only deletes a deactivated DEK when either: +| HashiCorp Vault key provider +| The next major release -* No data uses the DEK for encryption. -Once the last piece of data that relies on the deactivated DEK for decryption is either mutated or deleted, Couchbase Server deletes the unused DEK. +| Service-level encryption (Query, Index, Search, Eventing) +| The next major release -* The DEK's lifetime elapses. -You can set a maximum lifetime for DEKs which limits how long Couchbase Server can keep it after rotation. -When a DEK's lifetime elapses, Couchbase Server uses the active DEK to re-encrypt data that's still encrypted with the DEK. -It then deletes the expired DEK. +| "Other" encryption data type +| The next major release +|=== -You can adjust the rotation and lifetime for encryption keys to suit your environment. +NOTE: The Community Edition does not support native encryption at rest. -See xref:manage:manage-security/manage-native-encryption-at-rest.adoc[] to learn how to manage native encryption at rest. diff --git a/modules/learn/pages/security/per-node-per-bucket.adoc b/modules/learn/pages/security/per-node-per-bucket.adoc new file mode 100644 index 0000000000..b915b0c6ee --- /dev/null +++ b/modules/learn/pages/security/per-node-per-bucket.adoc @@ -0,0 +1,75 @@ += Per-Node and Per-Bucket Encryption Principles +:description: Encryption at rest in Couchbase Server is designed around two key architectural principles: encryption is managed independently on each node, and each bucket can be encrypted independently with its own key. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +== Per-Node DEK Management + +Each node in the cluster generates and manages its own Data Encryption Keys (DEKs). +DEKs are not shared between nodes. +This design means: + +* Each node independently encrypts and decrypts the data it stores on its local disk. +* If a node is compromised, only the DEKs for that node are exposed — DEKs on other nodes remain unaffected. +* When you add a new node to the cluster, Couchbase Server generates new DEKs for that node during the rebalance process. +During file-based rebalance, services such as the Data and Index services may transfer encrypted bucket DEKs along with data files between nodes. +The transferred DEKs are used for decryption purposes only. +New DEKs are generated on the target node for any subsequent encryption. + +== Per-Node KEK Storage + +Key Encryption Keys (KEKs) are configured globally for the cluster, but each node stores its own encrypted copy of each KEK on disk. +This ensures that each node can decrypt its DEKs independently, even if the node is temporarily disconnected from the rest of the cluster. + +For locally managed keys (Couchbase Server Secret Management), the key material is stored in an encrypted file on each node's disk. +The key is encrypted using either the master password or another KEK, depending on your configuration. + +For external KMS keys (AWS KMS, Azure Key Vault, GCP Cloud KMS, HashiCorp Vault, KMIP), the key material remains in the external KMS. +Each node independently contacts the external KMS to decrypt the KEK when needed — typically at node startup. + +== Per-Bucket Encryption + +You configure encryption independently for each bucket. +This means: + +* Different buckets can use different encryption keys. +* You can encrypt sensitive buckets (such as buckets containing PII) while leaving less-sensitive + buckets unencrypted. +* In multi-tenant deployments, each tenant's bucket can use its own encryption key, ensuring + cryptographic separation of tenant data. + +When you enable encryption for a bucket, Couchbase Server generates separate DEKs on each node for that bucket. +Each node uses its per-node, per-bucket DEK to encrypt data it writes to disk for that bucket. + +== Per-Data-Type Encryption + +Similarly, you configure encryption independently for each type of system data (configuration, logs, audit, and other). +Each data type has its own DEKs on each node: + +[source] +---- +Node 1 Node 2 +├── DEK: bucket-A ├── DEK: bucket-A +├── DEK: bucket-B ├── DEK: bucket-B +├── DEK: configuration ├── DEK: configuration +├── DEK: logs ├── DEK: logs +├── DEK: audit ├── DEK: audit +└── DEK: other └── DEK: other +---- + +This granular approach means you can assign different KEKs to different data types on the same cluster. +For example, you can protect bucket data with one KEK managed by AWS KMS, and protect log data with a different KEK managed locally by Couchbase Server. + +== Monitor Encryption Status + +You can check the encryption status of each data type on each node using the REST API: + +* `GET /pools/default` — Returns per-node encryption status for configuration, audit, and log data + (in the `encryptionAtRestInfo` field). +* `GET /pools/default/buckets/` — Returns aggregated encryption status for the bucket + (in the `encryptionAtRestInfo` field) and per-node encryption status + (in the `bucketEncryptionAtRestInfo` field). +* `GET /settings/security/encryptionAtRest` — Returns aggregated encryption status for + configuration, log, and audit data (in the `info` field). + +For more information, see xref:manage:manage-security/manage-encryption-at-rest.adoc[Manage Native Encryption at Rest] . diff --git a/modules/learn/pages/security/transparent-encryption-decryption.adoc b/modules/learn/pages/security/transparent-encryption-decryption.adoc new file mode 100644 index 0000000000..d972db4309 --- /dev/null +++ b/modules/learn/pages/security/transparent-encryption-decryption.adoc @@ -0,0 +1,62 @@ += Transparent Encryption and Decryption +:description: Encryption at rest is transparent to applications and users. You do not need to change your application code, SDKs, or queries to use encryption at rest. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +When encryption at rest is enabled: + +Write path:: +When Couchbase Server writes data to disk, it encrypts the data in memory using the active DEK(AES-256-GCM) and writes the encrypted result to the file system. ++ +[source] +---- +Application data → encrypt with DEK (in memory) → write encrypted data to disk +---- + +Read path:: +When Couchbase Server reads data from disk, it decrypts the data in memory using the appropriate DEK and returns the plaintext to the requesting service or client. ++ +[source] +---- +Read encrypted data from disk → decrypt with DEK (in memory) → return plaintext +---- + +== Cold Start (Node Startup) + +When a node starts, Couchbase Server must decrypt the key hierarchy before it can read any encrypted data: + +. Couchbase Server obtains the master password (from the environment variable, a prompt, or a script). +. It uses the master password to decrypt locally stored KEKs. + If a KEK is managed by an external KMS, Couchbase Server sends a decrypt request to the KMS. + This is the only time Couchbase Server contacts the external KMS during normal operation. +. It uses the decrypted KEKs to decrypt the DEKs stored on disk. +. It uses the decrypted DEKs to decrypt data as services read it from disk. + +After startup, DEKs and KEKs are cached in memory. +Couchbase Server contacts the external KMS periodically to test configured encryption keys. + +== What Transparency Means for Your Application + +* *SDKs:* No changes required. + All Couchbase SDKs work the same way whether encryption at rest is enabled or disabled. +* *Queries:* {sqlpp} queries and Full Text Search queries work identically. + The Query service and Search service decrypt data transparently as they read it from disk. +* *XDCR:* Cross Data Center Replication works transparently. + Data is decrypted on the source cluster before transmission and can be re-encrypted at rest on the destination cluster if encryption is enabled there. +* *Backup and Restore:* The Backup service and `cbbackupmgr` read decrypted data through the normal Couchbase Server APIs. + Backup repositories have their own independent encryption mechanism. + For more information, see xref:manage:manage-backup-and-restore/manage-backup-and-restore.adoc[Manage Backup and Restore]. + +== Performance Considerations + +Encryption and decryption add CPU overhead to every disk read and write operation. +The impact depends on your workload, hardware, and storage backend: + +* *Write-heavy workloads* see the most impact because every document mutation requires encryption + before the write to disk. +* *Read-heavy workloads* with high cache hit ratios see minimal impact because most reads are + served from the managed cache (in memory) rather than from disk. +* *Index operations* may see increased latency during index builds and scans that read heavily + from disk. + +Test your workload with encryption enabled to understand the performance impact in your environment. diff --git a/modules/learn/pages/security/when-CBServer-encrypts-data.adoc b/modules/learn/pages/security/when-CBServer-encrypts-data.adoc new file mode 100644 index 0000000000..9845814ecb --- /dev/null +++ b/modules/learn/pages/security/when-CBServer-encrypts-data.adoc @@ -0,0 +1,141 @@ +[#when-data-is-encrypted] +== When Couchbase Server Encrypts Data +:description: When you enable encryption at rest for a bucket or a type of system data, Couchbase Server encrypts all of the data it writes. However, it does not encrypt existing data in the bucket or already written system data. +:page-toclevels: 2 +:page-edition: Enterprise Edition + +For example, if you enable encryption at rest for logs, Couchbase Server encrypts new log entries but does not encrypt existing log entries. +Existing unencrypted data in buckets is encrypted when it's mutated or compacted. +Now, this also applies to service-generated data (the "other" data type). + +When you enable "other" encryption, services encrypt new service-generated files (such as Query spill files and Index backfill files) but do not retroactively encrypt files written before enablement. + +You can use the REST API to force Couchbase Server to encrypt existing unencrypted data. +See xref:rest-api:security/encryption-at-rest/force-encryption-at-rest.adoc[] for more information. + +See xref:manage:manage-security/manage-encryption-at-rest.adoc#force-encryption[Force Encryption of Unencrypted Data] for more information. + +=== Encryption States + + +An encryption data type can be in 1 of the following states: + + +[cols="1,2"] + + +|=== + + +| State | Meaning + + +| Encrypted + + +| All data of this type on the node is encrypted with the active DEK. + + +| Partially encrypted + + +| Some data is encrypted and some is not. + + +This is the normal state after you enable encryption until all existing data has been mutated, compacted, or force-encrypted. + + +| Unencrypted + + +| No data of this type on the node is encrypted. + + +|=== + + +You can check the encryption state using the `cm_encr_at_rest_data_status` metric or the REST API status endpoints. + + +See <> for more information. + +[#disabling-encryption] + + +=== Disabling Encryption + + +When you disable encryption at rest for a data type, Couchbase Server does not immediately decrypt all encrypted data. + + +It stops encrypting the new data it writes, but it does not decrypt and rewrite the existing encrypted data. + + +It still decrypts encrypted data as it reads it. + + +It only rewrites the data in an unencrypted format when it updates the data. + + +This means your cluster can be in a mixed state where some data is encrypted and some is not. + + +The encryption status reports as "partially encrypted" until all encrypted data has been rewritten without encryption. + + +==== Forcing Decryption + + +To force all data to be stored unencrypted after disabling encryption, use the Drop DEKs operation. + + +Dropping DEKs after disabling encryption causes Couchbase Server to: + + +. Read all encrypted data from disk. + + +. Rewrite the data without encryption. + + +. Delete the DEKs that were used to encrypt it. + + +See xref:manage:manage-security/manage-encryption-at-rest.adoc#drop-deks[Drop DEKs and Re-encrypt Data] for the steps to force decryption. + + +WARNING: Dropping DEKs is an I/O-intensive operation. + + +It reads and rewrites all data for the affected data type. + + +Plan this operation during a maintenance window or a period of low activity. + + +==== DEK Retention After Disabling + + +When you disable encryption, Couchbase Server keeps the existing DEKs so it can decrypt data that is still encrypted. + + +It does not delete DEKs until all data they protect has been rewritten without encryption (or force-decrypted via Drop DEKs). + + +==== Re-enabling Encryption + + +You can re-enable encryption at any time after disabling it. + + +When you re-enable encryption, Couchbase Server generates new DEKs and begins encrypting new data. + + +Previously encrypted data that was not decrypted remains encrypted with the old DEKs. + + +Previously decrypted data is encrypted with new DEKs as it is mutated or compacted. + + + + diff --git a/modules/manage/pages/manage-security/encrypt-data-at-rest.adoc b/modules/manage/pages/manage-security/encrypt-data-at-rest.adoc new file mode 100644 index 0000000000..b5fcc44551 --- /dev/null +++ b/modules/manage/pages/manage-security/encrypt-data-at-rest.adoc @@ -0,0 +1,157 @@ +== Encrypt Data at Rest +:description: pass:q[Once you have created an encrytion-at-rest key, you can use it to encrypt bucket data.] +:tabs: +:page-topic-type: guide +:page-toclevels: 3 +:keywords: encryption at rest, security +:page-edition: Enterprise Edition +:page-topic-type: reference + +For audit, configuration, and log data, you can use the primary password or an encryption-at-rest key. Choose based on your security and operational requirements. +You must set the master password on each node before using it for encryption at rest. +By default, the nodes do not have a master password. +See xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password] for more information. + +NOTE: Once you enable encryption for a bucket or system data, Couchbase Server begins to encrypt data as it writes it. +It does not encrypt the data that existed before you enabled encryption. +See xref:learn:security/native-encryption-at-rest-overview.adoc#when-data-is-encrypted[When Couchbase Server Encrypts Data] for more information when Couchbase Server encrypts and decrypts data. + +The following sections explain how to enable encryption for each type of data. + +=== Encrypt Bucket Data + +To encrypt a bucket, you must configure at least 1 encryption-at-rest key. +This key encrypts all buckets or a specific bucket. +See xref:manage:manage-security/manage-native-encryption-at-rest.adoc#create-keys[Creating Encryption Keys] for more information about creating encryption-at-rest keys. + +Users with xref:learn:security/roles.adoc#bucket-admin[Bucket Admin] or xref:learn:security/roles.adoc#cluster-admin[Cluster Admin] roles can enable encryption for rest for buckets as long as an encryption key exists that's allowed to encrypt the bucket. + +NOTE: Once you enable encryption for a bucket, Couchbase Server begins to encrypt data as it writes it. +It does not encrypt the data that existed in the bucket before you enabled encryption. +See xref:learn:security/native-encryption-at-rest-overview.adoc#when-data-is-encrypted[When Couchbase Server Encrypts Data] for more information when Couchbase Server encrypts and decrypts data. + +==== Encrypt a Bucket Using the Couchbase Server Web Console +. On the main menu, select menu:Buckets[]. +. You can encrypt a bucket when you create it or you can edit an existing bucket to encrypt it. ++ +-- +For an existing bucket: + +a. Click the name of the bucket you want to encrypt. +b. Click btn:[Edit]. + +For a new bucket: + +a. Click btn:[Add Bucket] to open the *Create Bucket* dialog. +-- + +. In the *Edit Bucket Settings* or *Add Data Bucket* dialog, expand the *Advanced bucket settings* section. +. Select *Enable Encryption at Rest*. +. In the *Available Encryption Keys* list, select the encryption key you want to use to encrypt the bucket. +. Edit the *DEK Rotation Interval* and *DEK Life Time* settings to configure the data encryption key rotation. +See xref::learn:security/native-encryption-at-rest-overview.adoc#rotation-expiration[Encryption Key Rotation and Expiration] for more information about these settings. +. If you are adding a bucket, configure the rest of the settings for your bucket. +See xref:manage:manage-buckets/create-bucket.adocl#create-bucket-using-couchbase-web-console[Create a Bucket with the UI] for more information about creating buckets. +. Click btn:[Add Bucket] or btn:[Save Changes] to save your changes. + +==== Encrypt a Bucket Using the REST API +When creating a bucket, you can set the `encryptionAtRestKeyId` parameter to the ID of the encryption key you want to use to encrypt the bucket: + +include::rest-api:example$encryption-at-rest/bucket-encryption-examples.adoc[tag=create-bucket] + +When updating an existing bucket, you can set the `encryptionAtRestKeyId` parameter to the ID of the encryption key you want to use to encrypt the bucket: + +include::rest-api:example$encryption-at-rest/bucket-encryption-examples.adoc[tag=alter-bucket] + +If the bucket is already encrypted, Couchbase Server re-encrypts the bucket's data encryption keys (DEKs) using the newly assigned encryption key. +If the bucket is not encrypted, Couchbase Server begins encrypting its data. + +See xref:rest-api:rest-bucket-create.adoc[] for more information about creating and updating buckets using the REST API. + +=== Encrypt Audit, Configuration, and Log Data + +You can encrypt audit, configuration, and log data using the master password or an encryption-at-rest key. +By default, Couchbase Server encrypts the configuration data using the master password. + +To make changes to the encryption settings for audit, configuration, and log data, you must have the xref:learn:security/roles.adoc#full-admin[Full Admin] or xref:learn:security/roles.adoc#security-admin[Security Admin] roles. + +==== Change Audit, Configuration, or Log Encryption Settings via the Web Console +To change the settings for audit, configuration, and log data via the Couchbase Server Web Console: + +. Click menu:Security[] on the main menu. +. Click menu:Encryption at Rest[]. +. Click btn:[Edit] under *Configuration Encryption*, *Logs Encryption*, or *Audit Encryption* depending on the type of data whose encryption settings you want to change. +. In the *Encryption at Rest* dialog, change the encryption-at-rest settings: + ++ +-- +To Disable Encryption:: +Select *Disabled* + +To Use the Master Password:: +Select *Master Password* + +To Use an Encryption-at-Rest Key:: +a. Select *Encryption Key*. + +b. Under *Available Encryption Keys*, select the encryption key you want to use. +You must have a key that you configured to encrypt the type of data you selected. + +c. Optionally change the *DEK Rotation Interval* and *DEK Life Time* settings to configure the data encryption key rotation. + +See xref::learn:security/native-encryption-at-rest-overview.adoc#rotation-expiration[Encryption Key Rotation and Expiration] for more information about these settings. +-- +. Click btn:[Save Changes] to save your changes. + +==== Change Audit, Configuration, or Log Encryption Settings via the REST API + +The REST API's `/settings/security/encryptionAtRest` endpoint lets you change the settings for audit, configuration, and log data encryption. +To change the settings, send a POST request to this endpoint with settings for each type of data you whose encryption you want to change. +The following example shows how to enabled encryption at rest for audit data using an encryption-at-rest key whose `id` is `0`: + +include::rest-api:example$encryption-at-rest/system-encryption-examples.adoc[tag=encrypt-data-with-key] + +TIP: You can find the `id` attribute of the encryption-at-rest key you want to use by sending a GET request to the `/settings/encryptionKeys` endpoint. +See xref:rest-api:security/encryption-at-rest/manage-encryption-keys.adoc#list-keys[List Encryption-at-Rest Keys] for more information about this endpoint. + +For more information about managing encryption at rest settings for audit, configuration, and log data, see xref:rest-api:security/encryption-at-rest/manage-system-encryption-at-rest.adoc[]. + +== Choosing What Data to Encrypt + +Couchbase Server lets you encrypt 4 types of data at rest. +You can enable encryption independently for each type, and you can use different encryption keys for each: + +[cols="2,3,3"] +|=== +| Data Type | Description | Encryption Options + +| Bucket data +| All data stored in a non-ephemeral bucket, including documents, indexes, and metadata written to disk. +| Encryption-at-rest key only + +| Configuration data +| Cluster and node configuration settings stored on disk. +| Master password or encryption-at-rest key + +| Log data +| Couchbase Server log files written to disk. +| Master password or encryption-at-rest key + +| Audit data +| Audit event records written to disk. +| Master password or encryption-at-rest key + +|=== + +=== Key Considerations When Deciding What to Encrypt + +* Bucket data requires an encryption-at-rest key — you cannot use the master password for bucket encryption. You must create at least one encryption-at-rest key before you can encrypt a bucket. + +* Configuration, log, and audit data can be encrypted using either the master password or an encryption-at-rest key. By default, Couchbase Server encrypts configuration data using the master password. + +* For the best security, use encryption-at-rest keys for all data types rather than relying on the master password. + +* Each encryption key must be configured with the appropriate usage permissions for the data type it will encrypt. For example, a key intended to encrypt log data must have log-encryption in its usage list. + +* When using a remote KMS (AWS KMS, GCP Cloud KMS, Azure Key Vault, HashiCorp Vault, or KMIP), create the remote key as a Key Encryption Key (KEK) only, then use it to encrypt a Couchbase Server-managed key that dire + diff --git a/modules/manage/pages/manage-security/manage-native-encryption-at-rest.adoc b/modules/manage/pages/manage-security/manage-native-encryption-at-rest.adoc index d4d06e7678..23ca294718 100644 --- a/modules/manage/pages/manage-security/manage-native-encryption-at-rest.adoc +++ b/modules/manage/pages/manage-security/manage-native-encryption-at-rest.adoc @@ -23,13 +23,13 @@ For an overview of native encryption at rest, see xref:learn:security/native-enc == Enabling Encryption -Enabling encryption at rest is a two step process: +Enabling encryption at rest is a 2 step process: -. Create at least one encryption key. +. Create at least 1 encryption key. You must use an encryption-at-rest key to encrypt bucket data. You can choose to use the xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[master password] instead of an encryption-at-rest key to encrypt audit, configuration, and log data. However, for the best security, you should use an encryption-at-rest key. -. Enable encryption for one or more types of data. +. Enable encryption for 1 or more types of data. The following sections explain these steps in greater detail. @@ -37,17 +37,20 @@ The following sections explain these steps in greater detail. == Creating Encryption Keys Couchbase Server only supports encrypting bucket data using an encryption at rest key. -Therefore, you must create at least one encryption-at-rest key to encrypt bucket data. +Therefore, you must create at least 1 encryption-at-rest key to encrypt bucket data. You should also consider creating encryption-at-rest keys when encrypting audit, configuration, and log data. === Requirements For each key you create, you must choose a Key Management Service (KMS) that maintains the key for you. -You have three options to choose from: +You have 3 options to choose from: -* Amazon's AWS KMS +* Amazon Web Services Key Management Service (AWS KMS) +* Google Cloud Key Management Service (GCP KMS) +* Azure Key Vault +* HashiCorp Vault * Any KMS that implements the Key Management Interoperability Protocol (KMIP) -* Couchbase Server +* Couchbase Server (auto-generated) See xref:learn:security/native-encryption-at-rest-overview.adoc#kms[Encryption Key Management Services] for more information. @@ -81,22 +84,12 @@ To create an encryption key using the Couchbase Server Web Console: . Click menu:Encryption at Rest[]. The *Encryption at Rest* tab opens. -+ -image::manage-security/encryption-at-rest-page.png[] - . Click btn:[Add Encryption Key] to open the *Add Encryption Key* dialog. -+ -image::manage-security/add-encryption-key-dialog.png[,400] - - . Enter a name for your key in the *Name* box. . If you want to limit what your key can encrypt, click *Configure* to expand the list of uses. Then choose what you want to your key to be able to encrypt. -+ -image::manage-security/add-encryption-key-uses.png[,300] - + If you want your key to only be able encrypt specific buckets, deselect *Data* and then select the buckets. @@ -126,6 +119,55 @@ Enable this option if your Couchbase Server cluster runs on AWS EC2 instances to -- +GCP:: ++ +-- +[start=8] +. Enter the GCP Resource Name for the cryptographic key. +This is the full resource path in the format: `projects//locations//keyRings//cryptoKeys/ +. Enter the Credentials File Path — the path on each cluster node where the GCP service account credentials JSON file is stored. +. Click Test Encryption Key Settings to verify that Couchbase Server can connect to GCP Cloud KMS and use the key. +. Click Add Encryption Key to save. + +-- + +Azure Key Vault:: ++ +-- +[start=8] +. Enter the Vault URL — the URI of your Azure Key Vault (e.g., https://my-vault.vault.azure.net). +. Enter the Key Name — the name of the key in your Azure Key Vault. +. Enter the Key Version (optional) — the specific version of the key. If omitted, the latest version is used. +. Enter the Azure Authentication Details: +.. Tenant ID — your Azure Active Directory tenant ID. +.. Client ID — the application (service principal) client ID. +.. Client Secret Path — the path on each cluster node to a file containing the client secret. +Alternatively, if Couchbase Server runs on Azure VMs, you can enable Managed Identity authentication. +. Click Test Encryption Key Settings to verify that Couchbase Server can connect to Azure Key Vault and use the key. +. Click Add Encryption Key to save. + +-- + +HashiCorp Vault:: ++ +-- +[start=8] +. Enter the Vault Address — the URL of your HashiCorp Vault server (e.g., https://vault.example.com:8200). +. Enter the Key Name — the name of the Transit secrets engine key. +. Enter the Mount Path (optional) — the mount path of the Transit secrets engine if it differs from the default transit/. +. Configure Authentication: +.. Token-based: Enter the path on each cluster node to a file containing the Vault token. +.. AppRole-based: Enter the Role ID and the path to a file containing the Secret ID. +.. TLS Certificate-based: Enter the paths to the client certificate and private key files. +. Configure TLS Settings (if applicable): +.. Enter the CA Certificate Path for verifying the Vault server's identity. +.. Choose whether to verify the Vault server's TLS certificate. +. Enter a Connection Timeout for network connections to the Vault server. +. Click Test Encryption Key Settings to verify that Couchbase Server can connect to HashiCorp Vault and use the key. +. Click Add Encryption Key to save. + +-- + KMIP:: + -- @@ -190,123 +232,12 @@ include::rest-api:example$encryption-at-rest/create-auto-generated-key-response. See xref:rest-api:security/encryption-at-rest/manage-encryption-keys.adoc#create-key[Create or Update an Encryption-at-Rest Key] for details on how to create keys using the REST API. -== Encrypt Data at Rest - -Once you have created an encrytion-at-rest key, you can use it to encrypt bucket data. -For audit, configuration, and log data, you can choose to use the master password instead of an encryption-at-rest key. -You must set the master password on each node before using it for encryption at rest. -By default, the nodes do not have a master password. -See xref:manage:manage-security/manage-system-secrets.adoc#setting-the-master-password[Setting the Master Password] for more information. - -NOTE: Once you enable encryption for a bucket or system data, Couchbase Server begins to encrypt data as it writes it. -It does not encrypt the data that existed before you enabled encryption. -See xref:learn:security/native-encryption-at-rest-overview.adoc#when-data-is-encrypted[When Couchbase Server Encrypts Data] for more information when Couchbase Server encrypts and decrypts data. - -The following sections explain how to enable encryption for each type of data. - -=== Encrypt Bucket Data - -To encrypt a bucket, you must have at least one encrytion-at-rest key configured to encrypt all buckets or the specific bucket you want to encrypt. -See <> for more information about creating encryption-at-rest keys. - -Users with xref:learn:security/roles.adoc#bucket-admin[Bucket Admin] or xref:learn:security/roles.adoc#cluster-admin[Cluster Admin] roles can enable encryption for rest for buckets as long as an encryption key exists that's allowed to encrypt the bucket. - -NOTE: Once you enable encryption for a bucket, Couchbase Server begins to encrypt data as it writes it. -It does not encrypt the data that existed in the bucket before you enabled encryption. -See xref:learn:security/native-encryption-at-rest-overview.adoc#when-data-is-encrypted[When Couchbase Server Encrypts Data] for more information when Couchbase Server encrypts and decrypts data. - -==== Encrypt a Bucket Using the Couchbase Server Web Console -. On the main menu, select menu:Buckets[]. -. You can encrypt a bucket when you create it or you can edit an existing bucket to encrypt it. -+ --- -For an existing bucket: - -a. Click the name of the bucket you want to encrypt. -a. Click btn:[Edit]. - -For a new bucket: - -a. Click btn:[Add Bucket] to open the *Create Bucket* dialog. --- - -. In the *Edit Bucket Settings* or *Add Data Bucket* dialog, expand the *Advanced bucket settings* section. -. Select *Enable Encryption at Rest*. -. In the *Available Encryption Keys* list, select the encryption key you want to use to encrypt the bucket. -. Edit the *DEK Rotation Interval* and *DEK Life Time* settings to configure the data encryption key rotation. -See xref::learn:security/native-encryption-at-rest-overview.adoc#rotation-expiration[Encryption Key Rotation and Expiration] for more information about these settings. -. If you are adding a bucket, configure the rest of the settings for your bucket. -See xref:manage:manage-buckets/create-bucket.adocl#create-bucket-using-couchbase-web-console[Create a Bucket with the UI] for more information about creating buckets. -. Click btn:[Add Bucket] or btn:[Save Changes] to save your changes. - -==== Encrypt a Bucket Using the REST API -When creating a bucket, you can set the `encryptionAtRestKeyId` parameter to the ID of the encryption key you want to use to encrypt the bucket: - -include::rest-api:example$encryption-at-rest/bucket-encryption-examples.adoc[tag=create-bucket] - -When updating an existing bucket, you can set the `encryptionAtRestKeyId` parameter to the ID of the encryption key you want to use to encrypt the bucket: - -include::rest-api:example$encryption-at-rest/bucket-encryption-examples.adoc[tag=alter-bucket] - -If the bucket is already encrypted, Couchbase Server re-encrypts the bucket's data encryption keys (DEKs) using the newly-assigned encryption key. -If the bucket is not encrypted, Couchbase Server begins encrypting its data. - -See xref:rest-api:rest-bucket-create.adoc[] for more information about creating and updating buckets using the REST API. - -=== Encrypt Audit, Configuration, and Log Data - -You can encrypt audit, configuration, and log data using the master password or an encryption-at-rest key. -By default, Couchbase Server encrypts the configuration data using the master password. - -To make changes to the encryption settings for audit, configuration, and log data, you must have the xref:learn:security/roles.adoc#full-admin[Full Admin] or xref:learn:security/roles.adoc#security-admin[Security Admin] roles. - -==== Change Audit, Configuration, or Log Encryption Settings via the Web Console -To change the settings for audit, configuration, and log data via the Couchbase Server Web Console: - -. Click menu:Security[] on the main menu. -. Click menu:Encryption at Rest[]. -. Click btn:[Edit] under *Configuration Encryption*, *Logs Encryption*, or *Audit Encryption* depending on the type of data whose encryption settings you want to change. -. In the *Encryption at Rest* dialog, change the encryption-at-rest settings: - -+ --- -To Disable Encryption:: -Select *Disabled* - -To Use the Master Password:: -Select *Master Password* - -To Use an Encryption-at-Rest Key:: -a. Select *Encryption Key*. - -a. Under *Available Encryption Keys*, select the encryption key you want to use. -You must have a key that you configured to encrypt the type of data you selected. -a. Optionally change the *DEK Rotation Interval* and *DEK Life Time* settings to configure the data encryption key rotation. -See xref::learn:security/native-encryption-at-rest-overview.adoc#rotation-expiration[Encryption Key Rotation and Expiration] for more information about these settings. --- -. Click btn:[Save Changes] to save your changes. - - -==== Change Audit, Configuration, or Log Encryption Settings via the REST API - -The REST API's `/settings/security/encryptionAtRest` endpoint lets you change the settings for audit, configuration, and log data encryption. -To change the settings, send a POST request to this endpoint with settings for each type of data you whose encryption you want to change. -The following example shows how to enabled encryption at rest for audit data using an encryption-at-rest key whose `id` is `0`: - -include::rest-api:example$encryption-at-rest/system-encryption-examples.adoc[tag=encrypt-data-with-key] - -TIP: You can find the `id` attribute of the encryption-at-rest key you want to use by sending a GET request to the `/settings/encryptionKeys` endpoint. -See xref:rest-api:security/encryption-at-rest/manage-encryption-keys.adoc#list-keys[List Encryption-at-Rest Keys] for more information about this endpoint. - -For more information about managing encryption at rest settings for audit, configuration, and log data, see xref:rest-api:security/encryption-at-rest/manage-system-encryption-at-rest.adoc[]. - == Viewing Encryption Status You can view the status of encryption at rest for a bucket, audit, configuration, and log data using the Couchbase Server Web Console or the REST API. === Viewing Encryption Status Using the Couchbase Server Web Console - To view the status of encryption at rest for a bucket using the Couchbase Server Web Console: . Click menu:Buckets[] on the main menu. @@ -314,9 +245,6 @@ To view the status of encryption at rest for a bucket using the Couchbase Server . The encryption status appears next to the *Encryption At Rest* label under the bucket's name. . For more details, hover over the eye icon next to the *Encryption At Rest* label. -+ -image::manage-security/encryption-at-rest-details.png[alt=An image of a popup showing that the buceket is fully encrypted and details of the data encryption keys (DEKS)"] - To view the status of encryption at rest for audit, configuration, and log data using the Couchbase Server Web Console: . Click menu:Security[] on the main menu. @@ -329,11 +257,11 @@ To view the status of encryption at rest for audit, configuration, and log data To view the encryption status of buckets using the REST API, send a GET request to the `/pools/default/buckets` endpoint. View the `encryptionAtRestKeyId` field in the response to see the encryption status of each bucket. If it's set to `-1`, the bucket is not encrypted. -If it's set to any other value, the bucket is encrypted and the value is the ID of the encryption key Couchbase Server uses to encrypt it. +If it's set to any other value, the bucket is encrypted and the value is the ID of the encryption key Couchbase Server uses to encrypt it. Additional details, such as the encrypted status of the data, are in the `encryptionAtRestInfo` object. See xref:rest-api:rest-bucket-summary.adoc[] for more information about the `/pools/default/buckets` endpoint. -The following example shows how to view the encryption status of the bucket bucket named `testBucket`. +The following example shows how to view the encryption status of the bucket named `testBucket`. It pipes the REST API result through the `jq` command to format and filter the output to show just the `encryptionAtRestKeyId` and `encryptionAtRestInfo` fields: [source,bash] @@ -361,210 +289,4 @@ The result of running this command looks like this: To view the encryption status of audit, configuration, and log data using the REST API, send a GET request to the `/settings/security/encryptionAtRest` endpoint. See xref:rest-api:security/encryption-at-rest/manage-system-encryption-at-rest.adoc#get-settings[Get Audit, Config, and Log Encryption-at-Rest Settings] for more information. -[#rotate-keys] -== Manually Rotate Encryption-at-rest Keys - -You can have Couchbase Server automatically rotate the encryption-at-rest keys it manages based on a schedule you set when creating the key. -You can also manually have Couchbase Server rotate a key that it manages. -To rotate an encryption-at-rest key managed by an external KMS, you must use the KMS's tools to rotate the key, then have Couchbase Server re-encrypt DEKs with the new key. - -The following sections explain the manual processes. - -=== Manually Rotate a Couchbase Server-Managed Encryption-At-Rest Key - -You can manually rotate encryption-at-rest keys that Couchbase Server manages. -You may choose to manually rotate a key if you believe it has been compromised or if you your security requirements have changed. - -When you rotate an encryption-at-rest key, Couchbase Server creates a new key and uses it to re-encrypt all DEKs that were encrypted with the previous version of the key. -This process does not re-encrypt data, as the DEKs do not change. - -To manually rotate a Couchbase-Managed encryption-at-rest key using the Couchbase Server Web Console: - -. Click menu:Security[] on the main menu. -. Click the menu:Encryption at Rest[] tab. -. Click the encryption key you want to rotate. -. Click the btn:[Rotate] button next to the key you want to rotate. - -To manually rotate an encryption-at-rest key, send a POST request to the `/controller/rotateEncryptionKey/{KEY_ID}` endpoint, where `{KEY_ID}` is the ID of the key you want to rotate. -See xref:rest-api:security/encryption-at-rest/rotate-encryption-at-rest-key.adoc[] for more information about manually rotating encryption-at-rest keys using the REST API. - -=== Manually Rotate a Encryption-at-rest Key Managed by AWS KMS - -To rotate an encryption-at-rest key managed by AWS KMS, you must use AWS's tools to rotate the key. -See https://docs.aws.amazon.com/kms/latest/developerguide/rotate-keys.html[Rotate AWS KMS keys^] to learn how to rotate keys in AWS KMS. -Once you have rotated the key in AWS KMS, have Couchbase Server re-encrypt any DEKs encrypted with the previous version of the key. - -To re-encrypt the DEKs with the rotated key using the Couchbase Server Web Console: - -. Click menu:Security[] on the main menu. -. Click the menu:Encryption at Rest[] tab. -. Click the encryption key that you rotated in AWS KMS. -. Click btn:[Rotate] next to the key you want to rotate. - -To re-encrypt the DEKs using the REST API, send a POST request to the `/controller/rotateEncryptionKey/{KEY_ID}` endpoint, where `{KEY_ID}` is the ID of the key that you rotated in AWS KMS. -See xref:rest-api:security/encryption-at-rest/rotate-encryption-at-rest-key.adoc[] for more information about manually rotating encryption-at-rest keys using the REST API. - -=== Manually Rotate an Encryption-at-rest Key Managed by a KMIP KMS - -You must use the KMS's tools to rotate the keys it manages. -Couchbase Server cannot trigger the rotation of a key managed by an external KMS. -Consult your KMS's documentation to learn how to rotate its keys. - -[NOTE] -==== -When configuring external KMS integration, make sure that all required key and certificate files are present -at the exact file paths specified during key creation. -These files must exist on every node in the cluster, including any nodes added during a rebalance operation. - -Couchbase Server does not create these files automatically. - -If any of these files are missing or located at a different path, encryption initialization or rebalance operations can fail. -For example, the Client Private Key Path (PEM format) must exist at the same specified location on all nodes before you create or edit a KMIP key. -==== - -Some KMIP KMSs require you to create a new key with a new ID when rotating a key. -In this case, when you rotate a key in the KMS, you must update the key ID in Couchbase Server. - -To update the key ID in Couchbase Server using the Couchbase Server Web Console: - -. Click menu:Security[]. -. Click menu:Encryption at Rest[]. -. Click the KMIP KMS encryption key that you rotated. -. Click the btn:[Edit]. -. In the *Edit Encryption Key* dialog, change the *KMIP Key ID* to the new key ID and click btn:[Save Encryption Key]. - -To update the key ID in Couchbase Server using the REST API, use the `/settings/encryptionKeys/{KEY_ID}` endpoint to modify the key. -The `KEY_ID` path parameter is the Couchbase Server ID of the key you want to update. -Update the `activeKey.kmipId` attribute in the `data` object to the new key ID. - -The following example updates the KMIP key ID of the encryption-at-rest key with the Couchbase Server `id` of `3` to the a KMIP key ID: - -[source,console] ----- -curl -sS -u Administrator:password \ - -X PUT http://127.0.0.1:8091/settings/encryptionKeys/3 \ - --data-binary @- <