diff --git a/assets/cockroach-labs/cockroachdb-15.0.6.tgz b/assets/cockroach-labs/cockroachdb-15.0.6.tgz new file mode 100644 index 000000000..8db8d1493 Binary files /dev/null and b/assets/cockroach-labs/cockroachdb-15.0.6.tgz differ diff --git a/assets/intel/intel-device-plugins-operator-0.32.0.tgz b/assets/intel/intel-device-plugins-operator-0.32.0.tgz new file mode 100644 index 000000000..61f93a0f5 Binary files /dev/null and b/assets/intel/intel-device-plugins-operator-0.32.0.tgz differ diff --git a/assets/intel/intel-device-plugins-qat-0.32.0.tgz b/assets/intel/intel-device-plugins-qat-0.32.0.tgz new file mode 100644 index 000000000..e14bbaef5 Binary files /dev/null and b/assets/intel/intel-device-plugins-qat-0.32.0.tgz differ diff --git a/assets/intel/intel-device-plugins-sgx-0.32.0.tgz b/assets/intel/intel-device-plugins-sgx-0.32.0.tgz new file mode 100644 index 000000000..b2cd699d5 Binary files /dev/null and b/assets/intel/intel-device-plugins-sgx-0.32.0.tgz differ diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/CONTRIBUTING.md b/charts/cockroach-labs/cockroachdb/15.0.6/CONTRIBUTING.md new file mode 100644 index 000000000..e248d72e1 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/CONTRIBUTING.md @@ -0,0 +1,14 @@ +# Contributing + +Contributions are welcome! + +For every change, please increment the `version` contained in +[Chart.yaml](https://github.com/cockroachdb/helm-charts/blob/master/cockroachdb/Chart.yaml). +The `version` roughly follows the [SEMVER](https://semver.org/) versioning +pattern. For changes which do not affect backwards compatibility, the PATCH or +MINOR version must be incremented, e.g. `4.1.3` -> `4.1.4`. For changes which +affect the backwards compatibility of the chart, the major version must be +incremented, e.g. `4.1.3` -> `5.0.0`. Examples of changes which affect backwards +compatibility include any major version releases of CockroachDB, as well as any +breaking changes to the CockroachDB chart templates. + diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/Chart.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/Chart.yaml new file mode 100644 index 000000000..28d71424d --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/Chart.yaml @@ -0,0 +1,18 @@ +annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: CockroachDB + catalog.cattle.io/kube-version: '>=1.8-0' + catalog.cattle.io/release-name: cockroachdb +apiVersion: v1 +appVersion: 24.3.4 +description: CockroachDB is a scalable, survivable, strongly-consistent SQL database. +home: https://www.cockroachlabs.com +icon: file://assets/icons/cockroachdb.png +kubeVersion: '>=1.8-0' +maintainers: +- email: helm-charts@cockroachlabs.com + name: cockroachlabs +name: cockroachdb +sources: +- https://github.com/cockroachdb/cockroach +version: 15.0.6 diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/README.md b/charts/cockroach-labs/cockroachdb/15.0.6/README.md new file mode 100644 index 000000000..f65e16f58 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/README.md @@ -0,0 +1,580 @@ + +# CockroachDB Helm Chart + +[CockroachDB](https://github.com/cockroachdb/cockroach) - the open source, cloud-native distributed SQL database. + +## Documentation + +Below is a brief overview of operating the CockroachDB Helm Chart and some specific implementation details. For additional information on deploying CockroachDB, please see: +> + +Note that the documentation requires Helm 3.0 or higher. + +## Prerequisites Details + +* Kubernetes 1.8 +* PV support on the underlying infrastructure (only if using `storage.persistentVolume`). [Docker for windows hostpath provisioner is not supported](https://github.com/cockroachdb/docs/issues/3184). +* If you want to secure your cluster to use TLS certificates for all network communication, [Helm must be installed with RBAC privileges](https://helm.sh/docs/topics/rbac/) or else you will get an "attempt to grant extra privileges" error. + +## StatefulSet Details + +* + +## StatefulSet Caveats + +* + +## Chart Details + +This chart will do the following: + +* Set up a dynamically scalable CockroachDB cluster using a Kubernetes StatefulSet. + +## Add the CockroachDB Repository + +```shell +$ helm repo add cockroachdb https://charts.cockroachdb.com/ +``` + +## Installing the Chart + +To install the chart with the release name `my-release`: + +```shell +$ helm install my-release cockroachdb/cockroachdb +``` + +Note that for a production cluster, you will likely want to override the following parameters in [`values.yaml`](values.yaml) with your own values. + +- `statefulset.resources.requests.memory` and `statefulset.resources.limits.memory` allocate memory resources to CockroachDB pods in your cluster. +- `conf.cache` and `conf.max-sql-memory` are memory limits that we recommend setting to 1/4 of the above resource allocation. When running CockroachDB, you must set these limits explicitly to avoid running out of memory. +- `storage.persistentVolume.size` defaults to `100Gi` of disk space per pod, which you may increase or decrease for your use case. +- `storage.persistentVolume.storageClass` uses the default storage class for your environment. We strongly recommend that you specify a storage class which uses an SSD. +- `tls.enabled` must be set to `yes`/`true` to deploy in secure mode. + +For more information on overriding the `values.yaml` parameters, please see: +> + +Confirm that all pods are `Running` successfully and init has been completed: + +```shell +$ kubectl get pods + +NAME READY STATUS RESTARTS AGE +my-release-cockroachdb-0 1/1 Running 0 1m +my-release-cockroachdb-1 1/1 Running 0 1m +my-release-cockroachdb-2 1/1 Running 0 1m +my-release-cockroachdb-init-k6jcr 0/1 Completed 0 1m +``` + +Confirm that persistent volumes are created and claimed for each pod: + +```shell +$ kubectl get pv + +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE +pvc-64878ebf-f3f0-11e8-ab5b-42010a8e0035 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-0 standard 51s +pvc-64945b4f-f3f0-11e8-ab5b-42010a8e0035 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-1 standard 51s +pvc-649d920d-f3f0-11e8-ab5b-42010a8e0035 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-2 standard 51s +``` + +### Running in secure mode + +In order to set up a secure cockroachdb cluster set `tls.enabled` to `yes`/`true` + +There are 3 ways to configure a secure cluster, with this chart. This all relates to how the certificates are issued: + +* Self-signer (default) +* Cert-manager +* Manual + +#### Self-signer + +This is the default behaviour, and requires no configuration beyond setting certificate durations if user wants to set custom duration. + +If you are running in this mode, self-signed certificates are created by self-signed utility for the nodes and root client and are stored in a secret. +You can look for the certificates created: +```shell +$ kubectl get secrets + +crdb-cockroachdb-ca-secret Opaque 2 23s +crdb-cockroachdb-client-secret kubernetes.io/tls 3 22s +crdb-cockroachdb-node-secret kubernetes.io/tls 3 23s +``` + + +#### Manual + +If you wish to supply the certificates to the nodes yourself set `tls.certs.provided` to `yes`/`true`. You may want to use this if you want to use a different certificate authority from the one being used by Kubernetes or if your Kubernetes cluster doesn't fully support certificate-signing requests. To use this, first set up your certificates and load them into your Kubernetes cluster as Secrets using the commands below: + +```shell +$ mkdir certs +$ mkdir my-safe-directory +$ cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key +$ cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.key +$ kubectl create secret generic cockroachdb-root --from-file=certs +secret/cockroachdb-root created +$ cockroach cert create-node --certs-dir=certs --ca-key=my-safe-directory/ca.key localhost 127.0.0.1 my-release-cockroachdb-public my-release-cockroachdb-public.my-namespace my-release-cockroachdb-public.my-namespace.svc.cluster.local *.my-release-cockroachdb *.my-release-cockroachdb.my-namespace *.my-release-cockroachdb.my-namespace.svc.cluster.local +$ kubectl create secret generic cockroachdb-node --from-file=certs +secret/cockroachdb-node created +``` + +> Note: The subject alternative names are based on a release called `my-release` in the `my-namespace` namespace. Make sure they match the services created with the release during `helm install` + +If your certificates are stored in tls secrets such as secrets generated by cert-manager, the secret will contain files named: + +* `ca.crt` +* `tls.crt` +* `tls.key` + +Cockroachdb, however, expects the files to be named like this: + +* `ca.crt` +* `node.crt` +* `node.key` +* `client.root.crt` +* `client.root.key` + +By enabling `tls.certs.tlsSecret` the tls secrets are projected on to the correct filenames, when they are mounted to the cockroachdb pods. + +#### Cert-manager + +If you wish to supply certificates with [cert-manager][3], set + +* `tls.certs.certManager` to `yes`/`true` +* `tls.certs.certManagerIssuer` to an IssuerRef (as they appear in certificate resources) pointing to a clusterIssuer or issuer, you have set up in the cluster + +Example issuer: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: cockroachdb-ca + namespace: cockroachdb +data: + tls.crt: [BASE64 Encoded ca.crt] + tls.key: [BASE64 Encoded ca.key] +type: kubernetes.io/tls +--- +apiVersion: cert-manager.io/v1alpha3 +kind: Issuer +metadata: + name: cockroachdb-cert-issuer + namespace: cockroachdb +spec: + ca: + secretName: cockroachdb-ca +``` + +## Upgrading the cluster + +### Chart version 3.0.0 and after + +Launch a temporary interactive pod and start the built-in SQL client: + +```shell +$ kubectl run cockroachdb --rm -it \ +--image=cockroachdb/cockroach \ +--restart=Never \ +-- sql --insecure --host=my-release-cockroachdb-public +``` + +> If you are running in secure mode, you will have to provide a client certificate to the cluster in order to authenticate, so the above command will not work. See [here](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) for an example of how to set up an interactive SQL shell against a secure cluster or [here](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/example-app-secure.yaml) for an example application connecting to a secure cluster. + +Set `cluster.preserve_downgrade_option`, where `$current_version` is the CockroachDB version currently running (e.g., `19.2`): + +```sql +> SET CLUSTER SETTING cluster.preserve_downgrade_option = '$current_version'; +``` + +Exit the shell and delete the temporary pod: + +```sql +> \q +``` + +Kick off the upgrade process by changing the new Docker image, where `$new_version` is the CockroachDB version to which you are upgrading: + +```shell +$ helm upgrade my-release cockroachdb/cockroachdb \ +--set image.tag=$new_version \ +--reuse-values +``` + +Kubernetes will carry out a safe [rolling upgrade](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets) of your CockroachDB nodes one-by-one. + +However, the upgrade will fail if it involves adding new Persistent Volume Claim (PVC) to the existing pods (e.g. enabling WAL Failover, pushing logs to a separate volume, etc.). +In such cases, kindly run the `scripts/upgrade_with_new_pvc.sh` script to upgrade the cluster. + +`./scripts/upgrade_with_new_pvc.sh -h` can be used for generating help on how to run the script. + +Monitor the cluster's pods until all have been successfully restarted: + +```shell +$ kubectl get pods + +NAME READY STATUS RESTARTS AGE +my-release-cockroachdb-0 1/1 Running 0 2m +my-release-cockroachdb-1 1/1 Running 0 3m +my-release-cockroachdb-2 1/1 Running 0 3m +my-release-cockroachdb-3 0/1 ContainerCreating 0 25s +my-release-cockroachdb-init-nwjkh 0/1 ContainerCreating 0 6s +``` + +```shell +$ kubectl get pods \ +-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\n"}' + +my-release-cockroachdb-0 cockroachdb/cockroach:v24.3.4 +my-release-cockroachdb-1 cockroachdb/cockroach:v24.3.4 +my-release-cockroachdb-2 cockroachdb/cockroach:v24.3.4 +my-release-cockroachdb-3 cockroachdb/cockroach:v24.3.4 +``` + +Resume normal operations. Once you are comfortable that the stability and performance of the cluster is what you'd expect post-upgrade, finalize the upgrade: + +```shell +$ kubectl run cockroachdb --rm -it \ +--image=cockroachdb/cockroach \ +--restart=Never \ +-- sql --insecure --host=my-release-cockroachdb-public +``` + +```sql +> RESET CLUSTER SETTING cluster.preserve_downgrade_option; +> \q +``` + +### Chart versions prior to 3.0.0 + +Due to a change in the label format in version 3.0.0 of this chart, upgrading requires that you delete the StatefulSet. Luckily there is a way to do it without actually deleting all the resources managed by the StatefulSet. Use the workaround below to upgrade from charts versions previous to 3.0.0: + +Get the new labels from the specs rendered by Helm: + +```shell +$ helm template -f deploy.vals.yml cockroachdb/cockroachdb -x templates/statefulset.yaml \ +| yq r - spec.template.metadata.labels + +app.kubernetes.io/name: cockroachdb +app.kubernetes.io/instance: my-release +app.kubernetes.io/component: cockroachdb +``` + +Place the new labels on all pods of the StatefulSet (change `my-release-cockroachdb-0` to the name of each pod): + +```shell +$ kubectl label pods my-release-cockroachdb-0 \ +app.kubernetes.io/name=cockroachdb \ +app.kubernetes.io/instance=my-release \ +app.kubernetes.io/component=cockroachdb +``` + +Delete the StatefulSet without deleting pods: + +```shell +$ kubectl delete statefulset my-release-cockroachdb --cascade=false +``` + +Verify that no pod is deleted and then upgrade as normal. A new StatefulSet will be created, taking over the management of the existing pods and upgrading them if needed. + +### See also + +For more information about upgrading a cluster to the latest major release of CockroachDB, see [Upgrade to CockroachDB](https://www.cockroachlabs.com/docs/stable/upgrade-cockroach-version.html). + +Note that there are sometimes backward-incompatible changes to SQL features between major CockroachDB releases. For details, see the [Upgrade Policy](https://www.cockroachlabs.com/docs/cockroachcloud/upgrade-policy). + +## Configuration + +The following table lists the configurable parameters of the CockroachDB chart and their default values. +For details see the [`values.yaml`](values.yaml) file. + +| Parameter | Description | Default | +| --------- | ----------- | ------- | +| `clusterDomain` | Cluster's default DNS domain | `cluster.local` | +| `conf.attrs` | CockroachDB node attributes | `[]` | +| `conf.cache` | Size of CockroachDB's in-memory cache | `25%` | +| `conf.cluster-name` | Name of CockroachDB cluster | `""` | +| `conf.disable-cluster-name-verification` | Disable CockroachDB cluster name verification | `no` | +| `conf.join` | List of already-existing CockroachDB instances | `[]` | +| `conf.log` | Logging configuration | `{}` | +| `conf.max-disk-temp-storage` | Max storage capacity for temp data | `0` | +| `conf.max-offset` | Max allowed clock offset for CockroachDB cluster | `500ms` | +| `conf.max-sql-memory` | Max memory to use processing SQL querie | `25%` | +| `conf.locality` | Locality attribute for this deployment | `""` | +| `conf.single-node` | Disable CockroachDB clustering (standalone mode) | `no` | +| `conf.sql-audit-dir` | Directory for SQL audit log | `""` | +| `conf.port` | WARNING this parameter is deprecated and will be removed in future version. Use `service.ports.grpc.internal.port` instead | `""` | +| `conf.http-port` | WARNING this parameter is deprecated and will be removed in future version. Use `service.ports.http.port` instead | `""` | +| `conf.path` | CockroachDB data directory mount path | `cockroach-data` | +| `conf.store.enabled` | Enable store configuration for CockroachDB | `false` | +| `conf.store.count` | Number of data stores per node | `1` | +| `conf.store.type` | CockroachDB storage type | `""` | +| `conf.store.size` | CockroachDB storage size | `""` | +| `conf.store.attrs` | CockroachDB storage attributes | `""` | +| `conf.wal-failover` | CockroachDB WAL Failover configuration | `{}` | +| `image.repository` | Container image name | `cockroachdb/cockroach` | +| `image.tag` | Container image tag | `v24.3.4` | +| `image.pullPolicy` | Container pull policy | `IfNotPresent` | +| `image.credentials` | `registry`, `user` and `pass` credentials to pull private image | `{}` | +| `statefulset.replicas` | StatefulSet replicas number | `3` | +| `statefulset.updateStrategy` | Update strategy for StatefulSet Pods | `{"type": "RollingUpdate"}` | +| `statefulset.podManagementPolicy` | `OrderedReady`/`Parallel` Pods creation/deletion order | `Parallel` | +| `statefulset.budget.maxUnavailable` | k8s PodDisruptionBudget parameter | `1` | +| `statefulset.args` | Extra command-line arguments | `[]` | +| `statefulset.env` | Extra env vars | `[]` | +| `statefulset.secretMounts` | Additional Secrets to mount at cluster members | `[]` | +| `statefulset.labels` | Additional labels of StatefulSet and its Pods | `{"app.kubernetes.io/component": "cockroachdb"}` | +| `statefulset.annotations` | Additional annotations of StatefulSet Pods | `{}` | +| `statefulset.nodeAffinity` | [Node affinity rules][2] of StatefulSet Pods | `{}` | +| `statefulset.podAffinity` | [Inter-Pod affinity rules][1] of StatefulSet Pods | `{}` | +| `statefulset.podAntiAffinity` | [Anti-affinity rules][1] of StatefulSet Pods | auto | +| `statefulset.podAntiAffinity.topologyKey` | The topologyKey for auto [anti-affinity rules][1] | `kubernetes.io/hostname` | +| `statefulset.podAntiAffinity.type` | Type of auto [anti-affinity rules][1] | `soft` | +| `statefulset.podAntiAffinity.weight` | Weight for `soft` auto [anti-affinity rules][1] | `100` | +| `statefulset.nodeSelector` | Node labels for StatefulSet Pods assignment | `{}` | +| `statefulset.priorityClassName` | [PriorityClassName][4] for StatefulSet Pods | `""` | +| `statefulset.tolerations` | Node taints to tolerate by StatefulSet Pods | `[]` | +| `statefulset.topologySpreadConstraints` | [Topology Spread Constraints rules][5] of StatefulSet Pods | auto | +| `statefulset.topologySpreadConstraints.maxSkew` | Degree to which Pods may be unevenly distributed | `1` | +| `statefulset.topologySpreadConstraints.topologyKey` | The key of node labels | `topology.kubernetes.io/zone` | +| `statefulset.topologySpreadConstraints.whenUnsatisfiable` | `ScheduleAnyway`/`DoNotSchedule` for unsatisfiable constraints | `ScheduleAnyway` | +| `statefulset.resources` | Resource requests and limits for StatefulSet Pods | `{}` | +| `statefulset.customLivenessProbe` | Custom Liveness probe | `{}` | +| `statefulset.customReadinessProbe` | Custom Rediness probe | `{}` | +| `statefulset.customStartupProbe` | Custom Startup probe | `{}` | +| `statefulset.terminationGracePeriodSeconds` | Termination grace period for CRDB statefulset pods | `300` | +| `service.ports.grpc.external.port` | CockroachDB primary serving port in Services | `26257` | +| `service.ports.grpc.external.name` | CockroachDB primary serving port name in Services | `grpc` | +| `service.ports.grpc.internal.port` | CockroachDB inter-communication port in Pods and Services | `26257` | +| `service.ports.grpc.internal.name` | CockroachDB inter-communication port name in Services | `grpc-internal` | +| `service.ports.http.port` | CockroachDB HTTP port in Pods and Services | `8080` | +| `service.ports.http.name` | CockroachDB HTTP port name in Services | `http` | +| `service.public.type` | Public Service type | `ClusterIP` | +| `service.public.labels` | Additional labels of public Service | `{"app.kubernetes.io/component": "cockroachdb"}` | +| `service.public.annotations` | Additional annotations of public Service | `{}` | +| `service.discovery.labels` | Additional labels of discovery Service | `{"app.kubernetes.io/component": "cockroachdb"}` | +| `service.discovery.annotations` | Additional annotations of discovery Service | `{}` | +| `ingress.enabled` | Enable ingress resource for CockroachDB | `false` | +| `ingress.labels` | Additional labels of Ingress | `{}` | +| `ingress.annotations` | Additional annotations of Ingress | `{}` | +| `ingress.paths` | Paths for the default host | `[/]` | +| `ingress.hosts` | CockroachDB Ingress hostnames | `[]` | +| `ingress.tls[0].hosts` | CockroachDB Ingress tls hostnames | `nil` | +| `ingress.tls[0].secretName` | CockroachDB Ingress tls secret name | `nil` | +| `prometheus.enabled` | Enable automatic monitoring of all instances when Prometheus is running | `true` | +| `serviceMonitor.enabled` | Create [ServiceMonitor](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/design.md#servicemonitor) Resource for scraping metrics using [PrometheusOperator](https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/user-guides/getting-started.md#prometheus-operator) | `false` | +| `serviceMonitor.labels` | Additional labels of ServiceMonitor | `{}` | +| `serviceMonitor.annotations` | Additional annotations of ServiceMonitor | `{}` | +| `serviceMonitor.interval` | ServiceMonitor scrape metrics interval | `10s` | +| `serviceMonitor.scrapeTimeout` | ServiceMonitor scrape timeout | `nil` | +| `serviceMonitor.tlsConfig` | Additional TLS configuration of ServiceMonitor | `{}` | +| `serviceMonitor.namespaced` | Limit ServiceMonitor to current namespace | `false` | +| `storage.hostPath` | Absolute path on host to store data | `""` | +| `storage.persistentVolume.enabled` | Whether to use PersistentVolume to store data | `yes` | +| `storage.persistentVolume.size` | PersistentVolume size | `100Gi` | +| `storage.persistentVolume.storageClass` | PersistentVolume class | `""` | +| `storage.persistentVolume.labels` | Additional labels of PersistentVolumeClaim | `{}` | +| `storage.persistentVolume.annotations` | Additional annotations of PersistentVolumeClaim | `{}` | +| `init.labels` | Additional labels of init Job and its Pod | `{"app.kubernetes.io/component": "init"}` | +| `init.jobAnnotations` | Additional annotations of the init Job itself | `{}` | +| `init.annotations` | Additional annotations of the Pod of init Job | `{}` | +| `init.affinity` | [Affinity rules][2] of init Job Pod | `{}` | +| `init.nodeSelector` | Node labels for init Job Pod assignment | `{}` | +| `init.tolerations` | Node taints to tolerate by init Job Pod | `[]` | +| `init.resources` | Resource requests and limits for the `cluster-init` container | `{}` | +| `init.terminationGracePeriodSeconds` | Termination grace period for CRDB init job | `300` | +| `tls.enabled` | Whether to run securely using TLS certificates | `no` | +| `tls.serviceAccount.create` | Whether to create a new RBAC service account | `yes` | +| `tls.serviceAccount.name` | Name of RBAC service account to use | `""` | +| `tls.copyCerts.image` | Image used in copy certs init container | `busybox` | +| `tls.copyCerts.resources` | Resource requests and limits for the `copy-certs` container | `{}` | +| `tls.certs.provided` | Bring your own certs scenario, i.e certificates are provided | `no` | +| `tls.certs.clientRootSecret` | If certs are provided, secret name for client root cert | `cockroachdb-root` | +| `tls.certs.nodeSecret` | If certs are provided, secret name for node cert | `cockroachdb-node` | +| `tls.certs.tlsSecret` | Own certs are stored in TLS secret | `no` | +| `tls.certs.selfSigner.enabled` | Whether cockroachdb should generate its own self-signed certs | `true` | +| `tls.certs.selfSigner.caProvided` | Bring your own CA scenario. This CA will be used to generate node and client cert | `false` | +| `tls.certs.selfSigner.caSecret` | If CA is provided, secret name for CA cert | `""` | +| `tls.certs.selfSigner.minimumCertDuration` | Minimum cert duration for all the certs, all certs duration will be validated against this duration | `624h` | +| `tls.certs.selfSigner.caCertDuration` | Duration of CA cert in hour | `43824h` | +| `tls.certs.selfSigner.caCertExpiryWindow` | Expiry window of CA cert means a window before actual expiry in which CA cert should be rotated | `648h` | +| `tls.certs.selfSigner.clientCertDuration` | Duration of client cert in hour | `672h | +| `tls.certs.selfSigner.clientCertExpiryWindow` | Expiry window of client cert means a window before actual expiry in which client cert should be rotated | `48h` | +| `tls.certs.selfSigner.nodeCertDuration` | Duration of node cert in hour | `8760h` | +| `tls.certs.selfSigner.nodeCertExpiryWindow` | Expiry window of node cert means a window before actual expiry in which node certs should be rotated | `168h` | +| `tls.certs.selfSigner.rotateCerts` | Whether to rotate the certs generate by cockroachdb | `true` | +| `tls.certs.selfSigner.readinessWait` | Wait time for each cockroachdb replica to become ready once it comes in running state. Only considered when rotateCerts is set to true | `30s` | +| `tls.certs.selfSigner.podUpdateTimeout` | Wait time for each cockroachdb replica to get to running state. Only considered when rotateCerts is set to true | `2m` | +| `tls.certs.certManager` | Provision certificates with cert-manager | `false` | +| `tls.certs.certManagerIssuer.group` | IssuerRef group to use when generating certificates | `cert-manager.io` | +| `tls.certs.certManagerIssuer.kind` | IssuerRef kind to use when generating certificates | `Issuer` | +| `tls.certs.certManagerIssuer.name` | IssuerRef name to use when generating certificates | `cockroachdb` | +| `tls.certs.certManagerIssuer.caCertDuration` | Duration of CA cert in hour | `43824h` | +| `tls.certs.certManagerIssuer.caCertExpiryWindow` | Expiry window of CA cert means a window before actual expiry in which CA cert should be rotated | `648h` | +| `tls.certs.certManagerIssuer.clientCertDuration` | Duration of client cert in hours | `672h` | +| `tls.certs.certManagerIssuer.clientCertExpiryWindow` | Expiry window of client cert means a window before actual expiry in which client cert should be rotated | `48h` | +| `tls.certs.certManagerIssuer.nodeCertDuration` | Duration of node cert in hours | `8760h` | +| `tls.certs.certManagerIssuer.nodeCertExpiryWindow` | Expiry window of node certificates means a window before actual expiry in which node certs should be rotated. | `168h` | +| `tls.selfSigner.image.repository` | Image to use for self signing TLS certificates | `cockroachlabs-helm-charts/cockroach-self-signer-cert`| +| `tls.selfSigner.image.tag` | Image tag to use for self signing TLS certificates | `0.1` | +| `tls.selfSigner.image.pullPolicy` | Self signing TLS certificates container pull policy | `IfNotPresent` | +| `tls.selfSigner.image.credentials` | `registry`, `user` and `pass` credentials to pull private image | `{}` | +| `networkPolicy.enabled` | Enable NetworkPolicy for CockroachDB's Pods | `no` | +| `networkPolicy.ingress.grpc` | Whitelist resources to access gRPC port of CockroachDB's Pods | `[]` | +| `networkPolicy.ingress.http` | Whitelist resources to access gRPC port of CockroachDB's Pods | `[]` | + + +Override the default parameters using the `--set key=value[,key=value]` argument to `helm install`. + +Alternatively, a YAML file that specifies custom values for the parameters can be provided while installing the chart. For example: + +```shell +$ helm install my-release -f my-values.yaml cockroachdb/cockroachdb +``` + +> **Tip**: You can use the default [values.yaml](values.yaml) + +## Deep dive + +### Connecting to the CockroachDB cluster + +Once you've created the cluster, you can start talking to it by connecting to its `-public` Service. CockroachDB is PostgreSQL wire protocol compatible, so there's a [wide variety of supported clients](https://www.cockroachlabs.com/docs/install-client-drivers.html). As an example, we'll open up a SQL shell using CockroachDB's built-in shell and play around with it a bit, like this (likely needing to replace `my-release-cockroachdb-public` with the name of the `-public` Service that was created with your installed chart): + +```shell +$ kubectl run cockroach-client --rm -it \ +--image=cockroachdb/cockroach \ +--restart=Never \ +-- sql --insecure --host my-release-cockroachdb-public +``` +``` +Waiting for pod default/cockroach-client to be running, status is Pending, +pod ready: false +If you don't see a command prompt, try pressing enter. +root@my-release-cockroachdb-public:26257> SHOW DATABASES; ++--------------------+ +| Database | ++--------------------+ +| information_schema | +| pg_catalog | +| system | ++--------------------+ +(3 rows) +root@my-release-cockroachdb-public:26257> CREATE DATABASE bank; +CREATE DATABASE +root@my-release-cockroachdb-public:26257> CREATE TABLE bank.accounts (id INT +PRIMARY KEY, balance DECIMAL); +CREATE TABLE +root@my-release-cockroachdb-public:26257> INSERT INTO bank.accounts VALUES +(1234, 10000.50); +INSERT 1 +root@my-release-cockroachdb-public:26257> SELECT * FROM bank.accounts; ++------+---------+ +| id | balance | ++------+---------+ +| 1234 | 10000.5 | ++------+---------+ +(1 row) +root@my-release-cockroachdb-public:26257> \q +Waiting for pod default/cockroach-client to terminate, status is Running +pod "cockroach-client" deleted +``` + +> If you are running in secure mode, you will have to provide a client certificate to the cluster in order to authenticate, so the above command will not work. See [here](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/client-secure.yaml) for an example of how to set up an interactive SQL shell against a secure cluster or [here](https://github.com/cockroachdb/cockroach/blob/master/cloud/kubernetes/example-app-secure.yaml) for an example application connecting to a secure cluster. + +### Cluster health + +Because our pod spec includes regular health checks of the CockroachDB processes, simply running `kubectl get pods` and looking at the `STATUS` column is sufficient to determine the health of each instance in the cluster. + +If you want more detailed information about the cluster, the best place to look is the Admin UI. + +### Accessing the Admin UI + +If you want to see information about how the cluster is doing, you can try pulling up the CockroachDB Admin UI by port-forwarding from your local machine to one of the pods (replacing `my-release-cockroachdb-0` with the name of one of your pods: + +```shell +$ kubectl port-forward my-release-cockroachdb-0 8080 +``` + +You should then be able to access the Admin UI by visiting in your web browser. + +### Failover + +If any CockroachDB member fails, it is restarted or recreated automatically by the Kubernetes infrastructure, and will re-join the cluster automatically when it comes back up. You can test this scenario by killing any of the CockroachDB pods: + +```shell +$ kubectl delete pod my-release-cockroachdb-1 +``` + +```shell +$ kubectl get pods -l "app.kubernetes.io/instance=my-release,app.kubernetes.io/component=cockroachdb" + +NAME READY STATUS RESTARTS AGE +my-release-cockroachdb-0 1/1 Running 0 5m +my-release-cockroachdb-2 1/1 Running 0 5m +``` + +After a while: + +```shell +$ kubectl get pods -l "app.kubernetes.io/instance=my-release,app.kubernetes.io/component=cockroachdb" + +NAME READY STATUS RESTARTS AGE +my-release-cockroachdb-0 1/1 Running 0 5m +my-release-cockroachdb-1 1/1 Running 0 20s +my-release-cockroachdb-2 1/1 Running 0 5m +``` + +You can check the state of re-joining from the new pod's logs: + +```shell +$ kubectl logs my-release-cockroachdb-1 + +[...] +I161028 19:32:09.754026 1 server/node.go:586 [n1] node connected via gossip and +verified as part of cluster {"35ecbc27-3f67-4e7d-9b8f-27c31aae17d6"} +[...] +cockroachdb-0.my-release-cockroachdb.default.svc.cluster.local:26257 +build: beta-20161027-55-gd2d3c7f @ 2016/10/28 19:27:25 (go1.7.3) +admin: http://0.0.0.0:8080 +sql: +postgresql://root@my-release-cockroachdb-1.my-release-cockroachdb.default.svc.cluster.local:26257?sslmode=disable +logs: cockroach-data/logs +store[0]: path=cockroach-data +status: restarted pre-existing node +clusterID: {35ecbc27-3f67-4e7d-9b8f-27c31aae17d6} +nodeID: 2 +[...] +``` + +### NetworkPolicy + +To enable NetworkPolicy for CockroachDB, install [a networking plugin that implements the Kubernetes NetworkPolicy spec](https://kubernetes.io/docs/tasks/administer-cluster/declare-network-policy#before-you-begin), and set `networkPolicy.enabled` to `yes`/`true`. + +For Kubernetes v1.5 & v1.6, you must also turn on NetworkPolicy by setting the `DefaultDeny` Namespace annotation. Note: this will enforce policy for _all_ pods in the Namespace: + +```shell +$ kubectl annotate namespace default "net.beta.kubernetes.io/network-policy={\"ingress\":{\"isolation\":\"DefaultDeny\"}}" +``` + +For more precise policy, set `networkPolicy.ingress.grpc` and `networkPolicy.ingress.http` rules. This will only allow pods that match the provided rules to connect to CockroachDB. + +### Scaling + +Scaling should be managed via the `helm upgrade` command. After resizing your cluster on your cloud environment (e.g., GKE or EKS), run the following command to add a pod. This assumes you scaled from 3 to 4 nodes: + +```shell +$ helm upgrade \ +my-release \ +cockroachdb/cockroachdb \ +--set statefulset.replicas=4 \ +--reuse-values +``` + +Note, that if you are running in secure mode (`tls.enabled` is `yes`/`true`) and increase the size of your cluster, you will also have to approve the CSR (certificate-signing request) of each new node (using `kubectl get csr` and `kubectl certificate approve`). + +[1]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity +[2]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity +[3]: https://cert-manager.io/ +[4]: https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass +[5]: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/ diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/app-readme.md b/charts/cockroach-labs/cockroachdb/15.0.6/app-readme.md new file mode 100644 index 000000000..8fcc1fd6f --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/app-readme.md @@ -0,0 +1,9 @@ +# CockroachDB Chart + +CockroachDB is a Distributed SQL database that runs natively in Kubernetes. It gives you resilient, horizontal scale across multiple clouds with always-on availability and data partitioned by location. + +CockroachDB scales horizontally without reconfiguration or need for a massive architectural overhaul. Simply add a new node to the cluster and CockroachDB takes care of the underlying complexity. + + - Scale by simply adding new nodes to a CockroachDB cluster + - Automate balancing and distribution of ranges, not shards + - Optimize server utilization evenly across all nodes diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/NOTES.txt b/charts/cockroach-labs/cockroachdb/15.0.6/templates/NOTES.txt new file mode 100644 index 000000000..13b421f62 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/NOTES.txt @@ -0,0 +1,50 @@ +CockroachDB can be accessed via port {{ .Values.service.ports.grpc.external.port }} at the +following DNS name from within your cluster: + +{{ template "cockroachdb.fullname" . }}-public.{{ .Release.Namespace }}.svc.cluster.local + +Because CockroachDB supports the PostgreSQL wire protocol, you can connect to +the cluster using any available PostgreSQL client. + +{{- if not .Values.tls.enabled }} + +For example, you can open up a SQL shell to the cluster by running: + + kubectl run -it --rm cockroach-client \ + --image=cockroachdb/cockroach \ + --restart=Never \ + {{- if .Values.networkPolicy.enabled }} + --labels="{{ template "cockroachdb.fullname" . }}-client=true" \ + {{- end }} + --command -- \ + ./cockroach sql --insecure --host={{ template "cockroachdb.fullname" . }}-public.{{ .Release.Namespace }} + +From there, you can interact with the SQL shell as you would any other SQL +shell, confident that any data you write will be safe and available even if +parts of your cluster fail. +{{- else }} + +Note that because the cluster is running in secure mode, any client application +that you attempt to connect will either need to have a valid client certificate +or a valid username and password. +{{- end }} + +{{- if and (.Values.networkPolicy.enabled) (not (empty .Values.networkPolicy.ingress.grpc)) }} + +Note: Since NetworkPolicy is enabled, the only Pods allowed to connect to this +CockroachDB cluster are: + +1. Having the label: "{{ template "cockroachdb.fullname" . }}-client=true" + +2. Matching the following rules: {{- toYaml .Values.networkPolicy.ingress.grpc | nindent 0 }} +{{- end }} + +Finally, to open up the CockroachDB admin UI, you can port-forward from your +local machine into one of the instances in the cluster: + + kubectl port-forward -n {{ .Release.Namespace }} {{ template "cockroachdb.fullname" . }}-0 {{ index .Values.conf `http-port` | int64 }} + +Then you can access the admin UI at http{{ if .Values.tls.enabled }}s{{ end }}://localhost:{{ index .Values.conf `http-port` | int64 }}/ in your web browser. + +For more information on using CockroachDB, please see the project's docs at: +https://www.cockroachlabs.com/docs/ diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/_helpers.tpl b/charts/cockroach-labs/cockroachdb/15.0.6/templates/_helpers.tpl new file mode 100644 index 000000000..3670fccc7 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/_helpers.tpl @@ -0,0 +1,352 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "cockroachdb.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 56 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "cockroachdb.fullname" -}} +{{- if .Values.fullnameOverride -}} + {{- .Values.fullnameOverride | trunc 56 | trimSuffix "-" -}} +{{- else -}} + {{- $name := default .Chart.Name .Values.nameOverride -}} + {{- if contains $name .Release.Name -}} + {{- .Release.Name | trunc 56 | trimSuffix "-" -}} + {{- else -}} + {{- printf "%s-%s" .Release.Name $name | trunc 56 | trimSuffix "-" -}} + {{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create a default fully qualified app name for cluster scope resource. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name with release namespace appended at the end. +*/}} +{{- define "cockroachdb.clusterfullname" -}} +{{- if .Values.fullnameOverride -}} + {{- printf "%s-%s" .Values.fullnameOverride .Release.Namespace | trunc 56 | trimSuffix "-" -}} +{{- else -}} + {{- $name := default .Chart.Name .Values.nameOverride -}} + {{- if contains $name .Release.Name -}} + {{- printf "%s-%s" .Release.Name .Release.Namespace | trunc 56 | trimSuffix "-" -}} + {{- else -}} + {{- printf "%s-%s-%s" .Release.Name $name .Release.Namespace | trunc 56 | trimSuffix "-" -}} + {{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "cockroachdb.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 56 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create the name of the ServiceAccount to use. +*/}} +{{- define "cockroachdb.serviceAccount.name" -}} +{{- if .Values.statefulset.serviceAccount.create -}} + {{- default (include "cockroachdb.fullname" .) .Values.statefulset.serviceAccount.name -}} +{{- else -}} + {{- default "default" .Values.statefulset.serviceAccount.name -}} +{{- end -}} +{{- end -}} + +{{/* +Return the appropriate apiVersion for NetworkPolicy. +*/}} +{{- define "cockroachdb.networkPolicy.apiVersion" -}} +{{- if semverCompare ">=1.4-0, <=1.7-0" .Capabilities.KubeVersion.Version -}} + {{- print "extensions/v1beta1" -}} +{{- else if semverCompare "^1.7-0" .Capabilities.KubeVersion.Version -}} + {{- print "networking.k8s.io/v1" -}} +{{- end -}} +{{- end -}} + +{{/* +Return the appropriate apiVersion for StatefulSets +*/}} +{{- define "cockroachdb.statefulset.apiVersion" -}} +{{- if semverCompare "<1.12-0" .Capabilities.KubeVersion.Version -}} + {{- print "apps/v1beta1" -}} +{{- else -}} + {{- print "apps/v1" -}} +{{- end -}} +{{- end -}} + +{{/* +Return CockroachDB store expression +*/}} +{{- define "cockroachdb.conf.store" -}} + {{- $isInMemory := eq (.Values.conf.store.type | toString) "mem" -}} + {{- $persistentSize := empty .Values.conf.store.size | ternary .Values.storage.persistentVolume.size .Values.conf.store.size -}} + + {{- $store := dict -}} + {{- $_ := set $store "type" ($isInMemory | ternary "type=mem" "") -}} + {{- if eq .Args.idx 0 -}} + {{- $_ := set $store "path" ($isInMemory | ternary "" (print "path=" .Values.conf.path)) -}} + {{- else -}} + {{- $_ := set $store "path" ($isInMemory | ternary "" (print "path=" .Values.conf.path "-" (add1 .Args.idx))) -}} + {{- end -}} + {{- $_ := set $store "size" (print "size=" ($isInMemory | ternary .Values.conf.store.size $persistentSize)) -}} + {{- $_ := set $store "attrs" (empty .Values.conf.store.attrs | ternary "" (print "attrs=" .Values.conf.store.attrs)) -}} + + {{- compact (values $store) | sortAlpha | join "," -}} +{{- end -}} + +{{/* +Define the default values for the certificate selfSigner inputs +*/}} +{{- define "selfcerts.fullname" -}} + {{- printf "%s-%s" (include "cockroachdb.fullname" .) "self-signer" | trunc 56 | trimSuffix "-" -}} +{{- end -}} + +{{- define "rotatecerts.fullname" -}} + {{- printf "%s-%s" (include "cockroachdb.fullname" .) "rotate-self-signer" | trunc 56 | trimSuffix "-" -}} +{{- end -}} + +{{- define "selfcerts.minimumCertDuration" -}} + {{- if .Values.tls.certs.selfSigner.minimumCertDuration -}} + {{- print (.Values.tls.certs.selfSigner.minimumCertDuration | trimSuffix "h") -}} + {{- else }} + {{- $minCertDuration := min (sub (.Values.tls.certs.selfSigner.clientCertDuration | trimSuffix "h" ) (.Values.tls.certs.selfSigner.clientCertExpiryWindow | trimSuffix "h")) (sub (.Values.tls.certs.selfSigner.nodeCertDuration | trimSuffix "h") (.Values.tls.certs.selfSigner.nodeCertExpiryWindow | trimSuffix "h")) -}} + {{- print $minCertDuration -}} + {{- end }} +{{- end -}} + +{{/* +Define the cron schedules for certificate rotate jobs and converting from hours to valid cron string. +We assume that each month has 31 days, hence the cron job may run few days earlier in a year. In a cron schedule, +we can not set a cron of more than a year, hence we try to run the cron in such a way that the cron run comes to +as close possible to the expiry window. However, it is possible that cron may run earlier than the expiry window. +*/}} +{{- define "selfcerts.caRotateSchedule" -}} +{{- $tempHours := sub (.Values.tls.certs.selfSigner.caCertDuration | trimSuffix "h") (.Values.tls.certs.selfSigner.caCertExpiryWindow | trimSuffix "h") -}} +{{- $days := "*" -}} +{{- $months := "*" -}} +{{- $hours := mod $tempHours 24 -}} +{{- if not (eq $hours $tempHours) -}} +{{- $tempDays := div $tempHours 24 -}} +{{- $days = mod $tempDays 31 -}} +{{- if not (eq $days $tempDays) -}} +{{- $days = add $days 1 -}} +{{- $tempMonths := div $tempDays 31 -}} +{{- $months = mod $tempMonths 12 -}} +{{- if not (eq $months $tempMonths) -}} +{{- $months = add $months 1 -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- if ne (toString $months) "*" -}} +{{- $months = printf "*/%s" (toString $months) -}} +{{- else -}} +{{- if ne (toString $days) "*" -}} +{{- $days = printf "*/%s" (toString $days) -}} +{{- else -}} +{{- if ne $hours 0 -}} +{{- $hours = printf "*/%s" (toString $hours) -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- printf "0 %s %s %s *" (toString $hours) (toString $days) (toString $months) -}} +{{- end -}} + +{{- define "selfcerts.clientRotateSchedule" -}} +{{- $tempHours := int64 (include "selfcerts.minimumCertDuration" .) -}} +{{- $days := "*" -}} +{{- $months := "*" -}} +{{- $hours := mod $tempHours 24 -}} +{{- if not (eq $hours $tempHours) -}} +{{- $tempDays := div $tempHours 24 -}} +{{- $days = mod $tempDays 31 -}} +{{- if not (eq $days $tempDays) -}} +{{- $days = add $days 1 -}} +{{- $tempMonths := div $tempDays 31 -}} +{{- $months = mod $tempMonths 12 -}} +{{- if not (eq $months $tempMonths) -}} +{{- $months = add $months 1 -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- if ne (toString $months) "*" -}} +{{- $months = printf "*/%s" (toString $months) -}} +{{- else -}} +{{- if ne (toString $days) "*" -}} +{{- $days = printf "*/%s" (toString $days) -}} +{{- else -}} +{{- if ne $hours 0 -}} +{{- $hours = printf "*/%s" (toString $hours) -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- printf "0 %s %s %s *" (toString $hours) (toString $days) (toString $months) -}} +{{- end -}} + +{{/* +Define the appropriate validations for the certificate selfSigner inputs +*/}} + +{{/* +Validate that if caProvided is true, then the caSecret must not be empty and secret must be present in the namespace. +*/}} +{{- define "cockroachdb.tls.certs.selfSigner.caProvidedValidation" -}} +{{- if .Values.tls.certs.selfSigner.caProvided -}} +{{- if eq "" .Values.tls.certs.selfSigner.caSecret -}} + {{ fail "CA secret can't be empty if caProvided is set to true" }} +{{- else -}} + {{- if not (lookup "v1" "Secret" .Release.Namespace .Values.tls.certs.selfSigner.caSecret) }} + {{ fail "CA secret is not present in the release namespace" }} + {{- end }} +{{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Validate that if caCertDuration or caCertExpiryWindow must not be empty and caCertExpiryWindow must be greater than +minimumCertDuration. +*/}} +{{- define "cockroachdb.tls.certs.selfSigner.caCertValidation" -}} +{{- if not .Values.tls.certs.selfSigner.caProvided -}} +{{- if or (not .Values.tls.certs.selfSigner.caCertDuration) (not .Values.tls.certs.selfSigner.caCertExpiryWindow) }} + {{ fail "CA cert duration or CA cert expiry window can not be empty" }} +{{- else }} +{{- if gt (int64 (include "selfcerts.minimumCertDuration" .)) (int64 (.Values.tls.certs.selfSigner.caCertExpiryWindow | trimSuffix "h")) -}} + {{ fail "CA cert expiration window should not be less than minimum Cert duration" }} +{{- end -}} +{{- if gt (int64 (include "selfcerts.minimumCertDuration" .)) (sub (.Values.tls.certs.selfSigner.caCertDuration | trimSuffix "h") (.Values.tls.certs.selfSigner.caCertExpiryWindow | trimSuffix "h")) -}} + {{ fail "CA cert Duration minus CA cert expiration window should not be less than minimum Cert duration" }} +{{- end -}} +{{- end -}} +{{- end }} +{{- end -}} + +{{/* +Validate that if clientCertDuration must not be empty and it must be greater than minimumCertDuration. +*/}} +{{- define "cockroachdb.tls.certs.selfSigner.clientCertValidation" -}} +{{- if or (not .Values.tls.certs.selfSigner.clientCertDuration) (not .Values.tls.certs.selfSigner.clientCertExpiryWindow) }} + {{ fail "Client cert duration can not be empty" }} +{{- else }} +{{- if lt (sub (.Values.tls.certs.selfSigner.clientCertDuration | trimSuffix "h") (.Values.tls.certs.selfSigner.clientCertExpiryWindow | trimSuffix "h")) (int64 (include "selfcerts.minimumCertDuration" .)) }} + {{ fail "Client cert duration minus client cert expiry window should not be less than minimum Cert duration" }} +{{- end }} +{{- end }} +{{- end -}} + +{{/* +Validate that nodeCertDuration must not be empty and nodeCertDuration minus nodeCertExpiryWindow must be greater than minimumCertDuration. +*/}} +{{- define "cockroachdb.tls.certs.selfSigner.nodeCertValidation" -}} +{{- if or (not .Values.tls.certs.selfSigner.nodeCertDuration) (not .Values.tls.certs.selfSigner.nodeCertExpiryWindow) }} + {{ fail "Node cert duration can not be empty" }} +{{- else }} +{{- if lt (sub (.Values.tls.certs.selfSigner.nodeCertDuration | trimSuffix "h") (.Values.tls.certs.selfSigner.nodeCertExpiryWindow | trimSuffix "h")) (int64 (include "selfcerts.minimumCertDuration" .))}} + {{ fail "Node cert duration minus node cert expiry window should not be less than minimum Cert duration" }} +{{- end }} +{{- end }} +{{- end -}} + +{{/* +Validate that if user enabled tls, then either self-signed certificates or certificate manager is enabled +*/}} +{{- define "cockroachdb.tlsValidation" -}} +{{- if .Values.tls.enabled -}} +{{- if and .Values.tls.certs.selfSigner.enabled .Values.tls.certs.certManager -}} + {{ fail "Can not enable the self signed certificates and certificate manager at the same time" }} +{{- end -}} +{{- if and (not .Values.tls.certs.selfSigner.enabled) (not .Values.tls.certs.certManager) -}} + {{- if not .Values.tls.certs.provided -}} + {{ fail "You have to enable either self signed certificates or certificate manager, if you have enabled tls" }} + {{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} + + +{{- define "cockroachdb.tls.certs.selfSigner.validation" -}} +{{ include "cockroachdb.tls.certs.selfSigner.caProvidedValidation" . }} +{{ include "cockroachdb.tls.certs.selfSigner.caCertValidation" . }} +{{ include "cockroachdb.tls.certs.selfSigner.clientCertValidation" . }} +{{ include "cockroachdb.tls.certs.selfSigner.nodeCertValidation" . }} +{{- end -}} + +{{- define "cockroachdb.securityContext.versionValidation" }} +{{- /* Allow using `securityContext` for custom images. */}} +{{- if ne "cockroachdb/cockroach" .Values.image.repository -}} + {{ print true }} +{{- else -}} +{{- if semverCompare ">=22.1.2" .Values.image.tag -}} + {{ print true }} +{{- else -}} +{{- if semverCompare ">=21.2.13, <22.1.0" .Values.image.tag -}} + {{ print true }} +{{- else -}} + {{ print false }} +{{- end }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Validate the log configuration. +*/}} +{{- define "cockroachdb.conf.log.validation" -}} +{{- if and (not .Values.conf.log.enabled) .Values.conf.log.persistentVolume.enabled -}} + {{ fail "Persistent volume for logs can only be enabled if logging is enabled" }} +{{- end -}} +{{- if and .Values.conf.log.persistentVolume.enabled (dig "file-defaults" "dir" "" .Values.conf.log.config) -}} +{{- if not (hasPrefix (printf "/cockroach/%s" .Values.conf.log.persistentVolume.path) (dig "file-defaults" "dir" "" .Values.conf.log.config)) }} + {{ fail "Log configuration should use the persistent volume if enabled" }} +{{- end -}} +{{- end -}} +{{- end -}} + +{{- define "cockroachdb.storage.hostPath.computation" -}} +{{- if hasSuffix "/" .Values.storage.hostPath -}} + {{- printf "%s-%d/" (dir .Values.storage.hostPath) (add1 .Args.idx) | quote -}} +{{- else -}} + {{- printf "%s-%d" .Values.storage.hostPath (add1 .Args.idx) | quote -}} +{{- end -}} +{{- end -}} + +{{/* +Validate the store count configuration. +*/}} +{{- define "cockroachdb.conf.store.validation" -}} + {{- if and (not .Values.conf.store.enabled) (ne (int .Values.conf.store.count) 1) -}} + {{ fail "Store count should be 1 when disabled" }} + {{- end -}} +{{- end -}} + +{{/* +Validate the WAL failover configuration. +*/}} +{{- define "cockroachdb.conf.wal-failover.validation" -}} + {{- with index .Values.conf `wal-failover` -}} + {{- if not (mustHas .value (list "" "disabled" "among-stores")) -}} + {{- if not (hasPrefix "path=" (.value | toString)) -}} + {{ fail "Invalid WAL failover configuration value. Expected either of '', 'disabled', 'among-stores' or 'path='" }} + {{- end -}} + {{- end -}} + {{- if eq .value "among-stores" -}} + {{- if or (not $.Values.conf.store.enabled) (eq (int $.Values.conf.store.count) 1) -}} + {{ fail "WAL failover among stores requires store enabled with count greater than 1" }} + {{- end -}} + {{- end -}} + {{- if hasPrefix "path=" (.value | toString) -}} + {{- if not .persistentVolume.enabled -}} + {{ fail "WAL failover to a side disk requires a persistent volume" }} + {{- end -}} + {{- if and (not (hasPrefix (printf "/cockroach/%s" .persistentVolume.path) (trimPrefix "path=" .value))) (not (hasPrefix .persistentVolume.path (trimPrefix "path=" .value))) -}} + {{ fail "WAL failover to a side disk requires a path to the mounted persistent volume" }} + {{- end -}} + {{- end -}} + {{- end -}} +{{- end -}} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/backendconfig.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/backendconfig.yaml new file mode 100644 index 000000000..2edc88619 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/backendconfig.yaml @@ -0,0 +1,21 @@ +{{- if .Values.iap.enabled }} +apiVersion: cloud.google.com/v1beta1 +kind: BackendConfig +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + iap: + enabled: true + oauthclientCredentials: + secretName: {{ template "cockroachdb.fullname" . }}.iap + timeoutSec: 120 +{{- end }} \ No newline at end of file diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.ca.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.ca.yaml new file mode 100644 index 000000000..4043fafb0 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.ca.yaml @@ -0,0 +1,33 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.certManager }} + {{- if .Values.tls.certs.certManagerIssuer.isSelfSignedIssuer }} +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ template "cockroachdb.fullname" . }}-ca-cert + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + duration: {{ .Values.tls.certs.certManagerIssuer.caCertDuration }} + renewBefore: {{ .Values.tls.certs.certManagerIssuer.caCertExpiryWindow }} + isCA: true + secretName: {{ .Values.tls.certs.caSecret }} + privateKey: + algorithm: ECDSA + size: 256 + commonName: root + subject: + organizations: + - Cockroach + issuerRef: + name: {{ .Values.tls.certs.certManagerIssuer.name }} + kind: {{ .Values.tls.certs.certManagerIssuer.kind }} + group: {{ .Values.tls.certs.certManagerIssuer.group }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.client.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.client.yaml new file mode 100644 index 000000000..dd0272f3e --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.client.yaml @@ -0,0 +1,40 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.certManager }} +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ template "cockroachdb.fullname" . }}-root-client + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + duration: {{ .Values.tls.certs.certManagerIssuer.clientCertDuration }} + renewBefore: {{ .Values.tls.certs.certManagerIssuer.clientCertExpiryWindow }} + usages: + - digital signature + - key encipherment + - client auth + privateKey: + algorithm: RSA + size: 2048 + commonName: root + subject: + organizations: + - Cockroach + secretName: {{ .Values.tls.certs.clientRootSecret }} + issuerRef: + {{- if .Values.tls.certs.certManagerIssuer.isSelfSignedIssuer }} + name: {{ template "cockroachdb.fullname" . }}-ca-issuer + kind: Issuer + group: cert-manager.io + {{- else }} + name: {{ .Values.tls.certs.certManagerIssuer.name }} + kind: {{ .Values.tls.certs.certManagerIssuer.kind }} + group: {{ .Values.tls.certs.certManagerIssuer.group }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.issuer.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.issuer.yaml new file mode 100644 index 000000000..5cf579ff9 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.issuer.yaml @@ -0,0 +1,20 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.certManager }} + {{- if .Values.tls.certs.certManagerIssuer.isSelfSignedIssuer }} +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: {{ template "cockroachdb.fullname" . }}-ca-issuer + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + ca: + secretName: {{ .Values.tls.certs.caSecret }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.node.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.node.yaml new file mode 100644 index 000000000..05e909d0b --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/certificate.node.yaml @@ -0,0 +1,50 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.certManager }} +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: {{ template "cockroachdb.fullname" . }}-node + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + duration: {{ .Values.tls.certs.certManagerIssuer.nodeCertDuration }} + renewBefore: {{ .Values.tls.certs.certManagerIssuer.nodeCertExpiryWindow }} + usages: + - digital signature + - key encipherment + - server auth + - client auth + privateKey: + algorithm: RSA + size: 2048 + commonName: node + subject: + organizations: + - Cockroach + dnsNames: + - "localhost" + - "127.0.0.1" + - {{ printf "%s-public" (include "cockroachdb.fullname" .) | quote }} + - {{ printf "%s-public.%s" (include "cockroachdb.fullname" .) .Release.Namespace | quote }} + - {{ printf "%s-public.%s.svc.%s" (include "cockroachdb.fullname" .) .Release.Namespace .Values.clusterDomain | quote }} + - {{ printf "*.%s" (include "cockroachdb.fullname" .) | quote }} + - {{ printf "*.%s.%s" (include "cockroachdb.fullname" .) .Release.Namespace | quote }} + - {{ printf "*.%s.%s.svc.%s" (include "cockroachdb.fullname" .) .Release.Namespace .Values.clusterDomain | quote }} + secretName: {{ .Values.tls.certs.nodeSecret }} + issuerRef: + {{- if .Values.tls.certs.certManagerIssuer.isSelfSignedIssuer }} + name: {{ template "cockroachdb.fullname" . }}-ca-issuer + kind: Issuer + group: cert-manager.io + {{- else }} + name: {{ .Values.tls.certs.certManagerIssuer.name }} + kind: {{ .Values.tls.certs.certManagerIssuer.kind }} + group: {{ .Values.tls.certs.certManagerIssuer.group }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrole.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrole.yaml new file mode 100644 index 000000000..6b8a3dc5f --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrole.yaml @@ -0,0 +1,19 @@ +{{- if and .Values.tls.enabled (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager) }} +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "cockroachdb.clusterfullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +rules: + - apiGroups: ["certificates.k8s.io"] + resources: ["certificatesigningrequests"] + verbs: ["create", "get", "watch"] +{{- end }} \ No newline at end of file diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrolebinding.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrolebinding.yaml new file mode 100644 index 000000000..3c18694ef --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/clusterrolebinding.yaml @@ -0,0 +1,23 @@ +{{- if and .Values.tls.enabled (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager) }} +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "cockroachdb.clusterfullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: {{ template "cockroachdb.clusterfullname" . }} +subjects: + - kind: ServiceAccount + name: {{ template "cockroachdb.serviceAccount.name" . }} + namespace: {{ .Release.Namespace | quote }} +{{- end }} \ No newline at end of file diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-ca-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-ca-certSelfSigner.yaml new file mode 100644 index 000000000..4cd53900c --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-ca-certSelfSigner.yaml @@ -0,0 +1,62 @@ +{{- if and .Values.tls.enabled (and .Values.tls.certs.selfSigner.enabled (not .Values.tls.certs.selfSigner.caProvided)) }} + {{- if .Values.tls.certs.selfSigner.rotateCerts }} + {{- if .Capabilities.APIVersions.Has "batch/v1/CronJob" }} +apiVersion: batch/v1 + {{- else }} +apiVersion: batch/v1beta1 + {{- end }} +kind: CronJob +metadata: + name: {{ template "rotatecerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} +spec: + schedule: {{ template "selfcerts.caRotateSchedule" . }} + jobTemplate: + spec: + backoffLimit: 1 + template: + metadata: + {{- with .Values.tls.selfSigner.labels }} + labels: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.annotations }} + annotations: {{- toYaml . | nindent 12 }} + {{- end }} + spec: + restartPolicy: Never + {{- with .Values.tls.selfSigner.affinity }} + affinity: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.tolerations }} + tolerations: {{- toYaml . | nindent 12 }} + {{- end }} + containers: + - name: cert-rotate-job + image: "{{ .Values.tls.selfSigner.image.registry }}/{{ .Values.tls.selfSigner.image.repository }}:{{ .Values.tls.selfSigner.image.tag }}" + imagePullPolicy: "{{ .Values.tls.selfSigner.image.pullPolicy }}" + args: + - rotate + - --ca + - --ca-duration={{ .Values.tls.certs.selfSigner.caCertDuration }} + - --ca-expiry={{ .Values.tls.certs.selfSigner.caCertExpiryWindow }} + - --ca-cron={{ template "selfcerts.caRotateSchedule" . }} + - --readiness-wait={{ .Values.tls.certs.selfSigner.readinessWait }} + - --pod-update-timeout={{ .Values.tls.certs.selfSigner.podUpdateTimeout }} + env: + - name: STATEFULSET_NAME + value: {{ template "cockroachdb.fullname" . }} + - name: NAMESPACE + value: {{ .Release.Namespace }} + - name: CLUSTER_DOMAIN + value: {{ .Values.clusterDomain}} + serviceAccountName: {{ template "rotatecerts.fullname" . }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-client-node-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-client-node-certSelfSigner.yaml new file mode 100644 index 000000000..d500cbeb6 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/cronjob-client-node-certSelfSigner.yaml @@ -0,0 +1,69 @@ +{{- if and .Values.tls.certs.selfSigner.enabled .Values.tls.certs.selfSigner.rotateCerts }} + {{- if .Capabilities.APIVersions.Has "batch/v1/CronJob" }} +apiVersion: batch/v1 + {{- else }} +apiVersion: batch/v1beta1 + {{- end }} +kind: CronJob +metadata: + name: {{ template "rotatecerts.fullname" . }}-client + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} +spec: + schedule: {{ template "selfcerts.clientRotateSchedule" . }} + jobTemplate: + spec: + backoffLimit: 1 + template: + metadata: + {{- with .Values.tls.selfSigner.labels }} + labels: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.annotations }} + annotations: {{- toYaml . | nindent 12 }} + {{- end }} + spec: + restartPolicy: Never + {{- with .Values.tls.selfSigner.affinity }} + affinity: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.tls.selfSigner.tolerations }} + tolerations: {{- toYaml . | nindent 12 }} + {{- end }} + containers: + - name: cert-rotate-job + image: "{{ .Values.tls.selfSigner.image.registry }}/{{ .Values.tls.selfSigner.image.repository }}:{{ .Values.tls.selfSigner.image.tag }}" + imagePullPolicy: "{{ .Values.tls.selfSigner.image.pullPolicy }}" + args: + - rotate + {{- if .Values.tls.certs.selfSigner.caProvided }} + - --ca-secret={{ .Values.tls.certs.selfSigner.caSecret }} + {{- else }} + - --ca-duration={{ .Values.tls.certs.selfSigner.caCertDuration }} + - --ca-expiry={{ .Values.tls.certs.selfSigner.caCertExpiryWindow }} + {{- end }} + - --client + - --client-duration={{ .Values.tls.certs.selfSigner.clientCertDuration }} + - --client-expiry={{ .Values.tls.certs.selfSigner.clientCertExpiryWindow }} + - --node + - --node-duration={{ .Values.tls.certs.selfSigner.nodeCertDuration }} + - --node-expiry={{ .Values.tls.certs.selfSigner.nodeCertExpiryWindow }} + - --node-client-cron={{ template "selfcerts.clientRotateSchedule" . }} + - --readiness-wait={{ .Values.tls.certs.selfSigner.readinessWait }} + - --pod-update-timeout={{ .Values.tls.certs.selfSigner.podUpdateTimeout }} + env: + - name: STATEFULSET_NAME + value: {{ template "cockroachdb.fullname" . }} + - name: NAMESPACE + value: {{ .Release.Namespace }} + - name: CLUSTER_DOMAIN + value: {{ .Values.clusterDomain}} + serviceAccountName: {{ template "rotatecerts.fullname" . }} + {{- end}} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/ingress.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/ingress.yaml new file mode 100644 index 000000000..2fa6373c8 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/ingress.yaml @@ -0,0 +1,90 @@ +{{- if .Values.ingress.enabled -}} +{{- $paths := .Values.ingress.paths -}} +{{- $ports := .Values.service.ports -}} +{{- $fullName := include "cockroachdb.fullname" . -}} +{{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }} +apiVersion: networking.k8s.io/v1 +{{- else if $.Capabilities.APIVersions.Has "networking.k8s.io/v1beta1/Ingress" }} +apiVersion: networking.k8s.io/v1beta1 +{{- else -}} +apiVersion: extensions/v1beta1 +{{- end }} +kind: Ingress +metadata: +{{- if or .Values.ingress.annotations .Values.iap.enabled }} + annotations: + {{- range $key, $value := .Values.ingress.annotations }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- if .Values.iap.enabled }} + kubernetes.io/ingress.class: "gce" + kubernetes.io/ingress.allow-http: "false" + {{- end }} +{{- end }} + name: {{ $fullName }}-ingress + namespace: {{ .Release.Namespace }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ $.Release.Name | quote }} + app.kubernetes.io/managed-by: {{ $.Release.Service | quote }} +{{- if .Values.ingress.labels }} +{{- toYaml .Values.ingress.labels | nindent 4 }} +{{- end }} +spec: + rules: + {{- if .Values.ingress.hosts }} + {{- range $host := .Values.ingress.hosts }} + - host: {{ $host }} + http: + paths: + {{- range $path := $paths }} + - path: {{ $path | quote }} + {{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }} + {{- if $.Values.iap.enabled }} + pathType: ImplementationSpecific + {{- else }} + pathType: Prefix + {{- end }} + {{- end }} + backend: + {{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }} + service: + name: {{ $fullName }}-public + port: + name: {{ $ports.http.name | quote }} + {{- else }} + serviceName: {{ $fullName }}-public + servicePort: {{ $ports.http.name | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- else }} + - http: + paths: + {{- range $path := $paths }} + - path: {{ $path | quote }} + {{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }} + {{- if $.Values.iap.enabled }} + pathType: ImplementationSpecific + {{- else }} + pathType: Prefix + {{- end }} + {{- end }} + backend: + {{- if $.Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" }} + service: + name: {{ $fullName }}-public + port: + name: {{ $ports.http.name | quote }} + {{- else }} + serviceName: {{ $fullName }}-public + servicePort: {{ $ports.http.name | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- if .Values.ingress.tls }} + tls: +{{- toYaml .Values.ingress.tls | nindent 4 }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-certSelfSigner.yaml new file mode 100644 index 000000000..54ed2cad3 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-certSelfSigner.yaml @@ -0,0 +1,83 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ template "selfcerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + annotations: + # This is what defines this resource as a hook. Without this line, the + # job is considered part of the release. + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "4" + "helm.sh/hook-delete-policy": hook-succeeded,hook-failed + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} +spec: + template: + metadata: + name: {{ template "selfcerts.fullname" . }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.tls.selfSigner.labels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.annotations }} + annotations: {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- if and .Values.tls.certs.selfSigner.securityContext.enabled }} + securityContext: + seccompProfile: + type: "RuntimeDefault" + runAsGroup: 1000 + runAsUser: 1000 + fsGroup: 1000 + runAsNonRoot: true + {{- end }} + restartPolicy: Never + {{- with .Values.tls.selfSigner.affinity }} + affinity: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.tolerations }} + tolerations: {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: cert-generate-job + image: "{{ .Values.tls.selfSigner.image.registry }}/{{ .Values.tls.selfSigner.image.repository }}:{{ .Values.tls.selfSigner.image.tag }}" + imagePullPolicy: "{{ .Values.tls.selfSigner.image.pullPolicy }}" + args: + - generate + {{- if .Values.tls.certs.selfSigner.caProvided }} + - --ca-secret={{ .Values.tls.certs.selfSigner.caSecret }} + {{- else }} + - --ca-duration={{ .Values.tls.certs.selfSigner.caCertDuration }} + - --ca-expiry={{ .Values.tls.certs.selfSigner.caCertExpiryWindow }} + {{- end }} + - --client-duration={{ .Values.tls.certs.selfSigner.clientCertDuration }} + - --client-expiry={{ .Values.tls.certs.selfSigner.clientCertExpiryWindow }} + - --node-duration={{ .Values.tls.certs.selfSigner.nodeCertDuration }} + - --node-expiry={{ .Values.tls.certs.selfSigner.nodeCertExpiryWindow }} + env: + - name: STATEFULSET_NAME + value: {{ template "cockroachdb.fullname" . }} + - name: NAMESPACE + value: {{ .Release.Namespace | quote }} + - name: CLUSTER_DOMAIN + value: {{ .Values.clusterDomain}} + {{- if and .Values.tls.certs.selfSigner.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + {{- end }} + serviceAccountName: {{ template "selfcerts.fullname" . }} +{{- end}} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-cleaner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-cleaner.yaml new file mode 100644 index 000000000..1503ac459 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job-cleaner.yaml @@ -0,0 +1,70 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ template "selfcerts.fullname" . }}-cleaner + namespace: {{ .Release.Namespace | quote }} + annotations: + # This is what defines this resource as a hook. Without this line, the + # job is considered part of the release. + "helm.sh/hook": pre-delete + "helm.sh/hook-delete-policy": hook-succeeded,hook-failed + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} +spec: + backoffLimit: 1 + template: + metadata: + name: {{ template "selfcerts.fullname" . }}-cleaner + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.tls.selfSigner.labels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.annotations }} + annotations: {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- if and .Values.tls.certs.selfSigner.securityContext.enabled }} + securityContext: + seccompProfile: + type: "RuntimeDefault" + runAsGroup: 1000 + runAsUser: 1000 + fsGroup: 1000 + runAsNonRoot: true + {{- end }} + restartPolicy: Never + {{- with .Values.tls.selfSigner.affinity }} + affinity: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.tls.selfSigner.tolerations }} + tolerations: {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: cleaner + image: "{{ .Values.tls.selfSigner.image.registry }}/{{ .Values.tls.selfSigner.image.repository }}:{{ .Values.tls.selfSigner.image.tag }}" + imagePullPolicy: "{{ .Values.tls.selfSigner.image.pullPolicy }}" + args: + - cleanup + - --namespace={{ .Release.Namespace }} + env: + - name: STATEFULSET_NAME + value: {{ template "cockroachdb.fullname" . }} + {{- if and .Values.tls.certs.selfSigner.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + {{- end }} + serviceAccountName: {{ template "rotatecerts.fullname" . }} +{{- end}} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/job.init.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job.init.yaml new file mode 100644 index 000000000..dbc1eaa17 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/job.init.yaml @@ -0,0 +1,303 @@ +{{ $isClusterInitEnabled := and (eq (len .Values.conf.join) 0) (not (index .Values.conf `single-node`)) }} +{{ $isDatabaseProvisioningEnabled := .Values.init.provisioning.enabled }} +{{- if or $isClusterInitEnabled $isDatabaseProvisioningEnabled }} + {{ template "cockroachdb.tlsValidation" . }} +kind: Job +apiVersion: batch/v1 +metadata: + name: {{ template "cockroachdb.fullname" . }}-init + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.init.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + annotations: + helm.sh/hook: post-install,post-upgrade + helm.sh/hook-delete-policy: before-hook-creation + {{- with .Values.init.jobAnnotations }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + template: + metadata: + labels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.init.labels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.init.annotations }} + annotations: {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- if eq (include "cockroachdb.securityContext.versionValidation" .) "true" }} + {{- if and .Values.init.securityContext.enabled }} + securityContext: + seccompProfile: + type: "RuntimeDefault" + runAsGroup: 1000 + runAsUser: 1000 + fsGroup: 1000 + runAsNonRoot: true + {{- end }} + {{- end }} + restartPolicy: OnFailure + terminationGracePeriodSeconds: {{ .Values.init.terminationGracePeriodSeconds }} + {{- if or .Values.image.credentials (and .Values.tls.enabled .Values.tls.selfSigner.image.credentials (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager)) }} + imagePullSecrets: + {{- if .Values.image.credentials }} + - name: {{ template "cockroachdb.fullname" . }}.db.registry + {{- end }} + {{- if and .Values.tls.enabled .Values.tls.selfSigner.image.credentials (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager) }} + - name: {{ template "cockroachdb.fullname" . }}.self-signed-certs.registry + {{- end }} + {{- end }} + serviceAccountName: {{ template "cockroachdb.serviceAccount.name" . }} + {{- if .Values.tls.enabled }} + initContainers: + - name: copy-certs + image: {{ .Values.tls.copyCerts.image | quote }} + imagePullPolicy: {{ .Values.tls.selfSigner.image.pullPolicy | quote }} + command: + - /bin/sh + - -c + - "cp -f /certs/* /cockroach-certs/; chmod 0400 /cockroach-certs/*.key" + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + {{- if and .Values.init.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + {{- end }} + volumeMounts: + - name: client-certs + mountPath: /cockroach-certs/ + - name: certs-secret + mountPath: /certs/ + {{- with .Values.tls.copyCerts.resources }} + resources: {{- toYaml . | nindent 12 }} + {{- end }} + {{- end }} + {{- with .Values.init.affinity }} + affinity: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.init.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.init.tolerations }} + tolerations: {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: cluster-init + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy | quote }} + # Run the command in an `while true` loop because this Job is bound + # to come up before the CockroachDB Pods (due to the time needed to + # get PersistentVolumes attached to Nodes), and sleeping 5 seconds + # between attempts is much better than letting the Pod fail when + # the init command does and waiting out Kubernetes' non-configurable + # exponential back-off for Pod restarts. + # Command completes either when cluster initialization succeeds, + # or when cluster has been initialized already. + command: + - /bin/bash + - -c + - >- + {{- if $isClusterInitEnabled }} + initCluster() { + while true; do + local output=$( + set -x; + + /cockroach/cockroach init \ + {{- if .Values.tls.enabled }} + --certs-dir=/cockroach-certs/ \ + {{- else }} + --insecure \ + {{- end }} + {{- with index .Values.conf "cluster-name" }} + --cluster-name={{.}} \ + {{- end }} + --host={{ template "cockroachdb.fullname" . }}-0.{{ template "cockroachdb.fullname" . -}} + :{{ .Values.service.ports.grpc.internal.port | int64 }} \ + {{- if .Values.init.pcr.enabled -}} + {{- if .Values.init.pcr.isPrimary }} + --virtualized \ + {{- else }} + --virtualized-empty \ + {{- end }} + {{- end }} + 2>&1); + + local exitCode="$?"; + echo $output; + + if [[ "$output" =~ .*"Cluster successfully initialized".* || "$output" =~ .*"cluster has already been initialized".* ]]; then + break; + fi + + echo "Cluster is not ready to be initialized, retrying in 5 seconds" + sleep 5; + done + } + + initCluster; + {{- end }} + + {{- if $isDatabaseProvisioningEnabled }} + provisionCluster() { + while true; do + /cockroach/cockroach sql \ + {{- if .Values.tls.enabled }} + --certs-dir=/cockroach-certs/ \ + {{- else }} + --insecure \ + {{- end }} + --host={{ template "cockroachdb.fullname" . }}-0.{{ template "cockroachdb.fullname" . -}} + :{{ .Values.service.ports.grpc.internal.port | int64 }} \ + --execute=" + {{- range $clusterSetting, $clusterSettingValue := .Values.init.provisioning.clusterSettings }} + SET CLUSTER SETTING {{ $clusterSetting }} = '${{ $clusterSetting | replace "." "_" }}_CLUSTER_SETTING'; + {{- end }} + + {{- range $user := .Values.init.provisioning.users }} + CREATE USER IF NOT EXISTS {{ $user.name }} WITH + {{- if $user.password }} + PASSWORD '${{ $user.name }}_PASSWORD' + {{- else }} + PASSWORD null + {{- end }} + {{ join " " $user.options }} + ; + {{- end }} + + {{- range $database := .Values.init.provisioning.databases }} + CREATE DATABASE IF NOT EXISTS {{ $database.name }} + {{- if $database.options }} + {{ join " " $database.options }} + {{- end }} + ; + + {{- range $owner := $database.owners }} + GRANT ALL ON DATABASE {{ $database.name }} TO {{ $owner }}; + {{- end }} + + {{- range $owner := $database.owners_with_grant_option }} + GRANT ALL ON DATABASE {{ $database.name }} TO {{ $owner }} WITH GRANT OPTION; + {{- end }} + + {{- if $database.backup }} + CREATE SCHEDULE IF NOT EXISTS {{ $database.name }}_scheduled_backup + FOR BACKUP DATABASE {{ $database.name }} INTO '{{ $database.backup.into }}' + + {{- if $database.backup.options }} + WITH {{ join "," $database.backup.options }} + {{- end }} + RECURRING '{{ $database.backup.recurring }}' + {{- if $database.backup.fullBackup }} + FULL BACKUP '{{ $database.backup.fullBackup }}' + {{- else }} + FULL BACKUP ALWAYS + {{- end }} + + {{- if and $database.backup.schedule $database.backup.schedule.options }} + WITH SCHEDULE OPTIONS {{ join "," $database.backup.schedule.options }} + {{- end }} + ; + {{- end }} + {{- end }} + " + &>/dev/null; + + local exitCode="$?"; + + if [[ "$exitCode" -eq "0" ]] + then break; + fi + + sleep 5; + done + + echo "Provisioning completed successfully"; + } + + provisionCluster; + {{- end }} + env: + {{- $secretName := printf "%s-init" (include "cockroachdb.fullname" .) }} + {{- range $user := .Values.init.provisioning.users }} + {{- if $user.password }} + - name: {{ $user.name }}_PASSWORD + valueFrom: + secretKeyRef: + name: {{ $secretName }} + key: {{ $user.name }}-password + {{- end }} + {{- end }} + {{- range $clusterSetting, $clusterSettingValue := .Values.init.provisioning.clusterSettings }} + {{- if $clusterSettingValue }} + - name: {{ $clusterSetting | replace "." "_" }}_CLUSTER_SETTING + valueFrom: + secretKeyRef: + name: {{ $secretName }} + key: {{ $clusterSetting | replace "." "-" }}-cluster-setting + {{- end }} + {{- end }} + {{- if .Values.tls.enabled }} + volumeMounts: + - name: client-certs + mountPath: /cockroach-certs/ + {{- end }} + {{- with .Values.init.resources }} + resources: {{- toYaml . | nindent 12 }} + {{- end }} + {{- if and .Values.init.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: ["ALL"] + {{- end }} + {{- if .Values.tls.enabled }} + volumes: + - name: client-certs + emptyDir: {} + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }} + - name: certs-secret + {{- if or .Values.tls.certs.tlsSecret .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }} + projected: + sources: + - secret: + {{- if .Values.tls.certs.selfSigner.enabled }} + name: {{ template "cockroachdb.fullname" . }}-client-secret + {{ else }} + name: {{ .Values.tls.certs.clientRootSecret }} + {{ end -}} + items: + - key: ca.crt + path: ca.crt + mode: 0400 + - key: tls.crt + path: client.root.crt + mode: 0400 + - key: tls.key + path: client.root.key + mode: 0400 + {{- else }} + secret: + secretName: {{ .Values.tls.certs.clientRootSecret }} + defaultMode: 0400 + {{- end }} + {{- end }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/networkpolicy.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/networkpolicy.yaml new file mode 100644 index 000000000..d41afa32b --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/networkpolicy.yaml @@ -0,0 +1,59 @@ +{{- if .Values.networkPolicy.enabled }} +kind: NetworkPolicy +apiVersion: {{ template "cockroachdb.networkPolicy.apiVersion" . }} +metadata: + name: {{ template "cockroachdb.serviceAccount.name" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + podSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 6 }} + {{- end }} + ingress: + - ports: + - port: grpc + {{- with .Values.networkPolicy.ingress.grpc }} + from: + # Allow connections via custom rules. + {{- toYaml . | nindent 8 }} + # Allow client connection via pre-considered label. + - podSelector: + matchLabels: + {{ template "cockroachdb.fullname" . }}-client: "true" + # Allow other CockroachDBs to connect to form a cluster. + - podSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 14 }} + {{- end }} + {{- if gt (.Values.statefulset.replicas | int64) 1 }} + # Allow init Job to connect to bootstrap a cluster. + - podSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.init.labels }} + {{- toYaml . | nindent 14 }} + {{- end }} + {{- end }} + {{- end }} + # Allow connections to admin UI and for Prometheus. + - ports: + - port: http + {{- with .Values.networkPolicy.ingress.http }} + from: {{- toYaml . | nindent 8 }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/poddisruptionbudget.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/poddisruptionbudget.yaml new file mode 100644 index 000000000..f707e4054 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/poddisruptionbudget.yaml @@ -0,0 +1,26 @@ +kind: PodDisruptionBudget +{{- if or (.Capabilities.APIVersions.Has "policy/v1") (semverCompare ">=1.21-0" .Capabilities.KubeVersion.Version) }} +apiVersion: policy/v1 +{{- else }} +apiVersion: policy/v1beta1 +{{- end }} +metadata: + name: {{ template "cockroachdb.fullname" . }}-budget + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + selector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 6 }} + {{- end }} + maxUnavailable: {{ .Values.statefulset.budget.maxUnavailable | int64 }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certRotateSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certRotateSelfSigner.yaml new file mode 100644 index 000000000..f0e2b90ce --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certRotateSelfSigner.yaml @@ -0,0 +1,27 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "rotatecerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["create", "get", "update", "delete"] + - apiGroups: ["apps"] + resources: ["statefulsets"] + verbs: ["get"] + resourceNames: + - {{ template "cockroachdb.fullname" . }} + - apiGroups: [""] + resources: ["pods"] + verbs: ["delete", "get"] +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certSelfSigner.yaml new file mode 100644 index 000000000..1cbaab3dd --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role-certSelfSigner.yaml @@ -0,0 +1,33 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "selfcerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + annotations: + # This is what defines this resource as a hook. Without this line, the + # job is considered part of the release. + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "2" + "helm.sh/hook-delete-policy": hook-succeeded,hook-failed + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +rules: + - apiGroups: [""] + resources: ["secrets"] + verbs: ["create", "get", "update", "delete"] + - apiGroups: ["apps"] + resources: ["statefulsets"] + verbs: ["get"] + resourceNames: + - {{ template "cockroachdb.fullname" . }} + - apiGroups: [""] + resources: ["pods"] + verbs: ["delete", "get"] +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/role.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role.yaml new file mode 100644 index 000000000..ebe5ce8ae --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/role.yaml @@ -0,0 +1,23 @@ +{{- if .Values.tls.enabled }} +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +rules: + - apiGroups: [""] + resources: ["secrets"] + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager }} + verbs: ["get"] + {{- else }} + verbs: ["create", "get"] + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certRotateSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certRotateSelfSigner.yaml new file mode 100644 index 000000000..c1a45f797 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certRotateSelfSigner.yaml @@ -0,0 +1,23 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "rotatecerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ template "rotatecerts.fullname" . }} +subjects: + - kind: ServiceAccount + name: {{ template "rotatecerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certSelfSigner.yaml new file mode 100644 index 000000000..5725d02a4 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding-certSelfSigner.yaml @@ -0,0 +1,29 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "selfcerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + annotations: + # This is what defines this resource as a hook. Without this line, the + # job is considered part of the release. + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "3" + "helm.sh/hook-delete-policy": hook-succeeded,hook-failed + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ template "selfcerts.fullname" . }} +subjects: + - kind: ServiceAccount + name: {{ template "selfcerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding.yaml new file mode 100644 index 000000000..00d9f9a55 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/rolebinding.yaml @@ -0,0 +1,23 @@ +{{- if .Values.tls.enabled }} +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ template "cockroachdb.fullname" . }} +subjects: + - kind: ServiceAccount + name: {{ template "cockroachdb.serviceAccount.name" . }} + namespace: {{ .Release.Namespace | quote }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.backendconfig.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.backendconfig.yaml new file mode 100644 index 000000000..61103060a --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.backendconfig.yaml @@ -0,0 +1,25 @@ +{{- if .Values.iap.enabled }} +kind: Secret +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" . }}.iap + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +type: Opaque +data: + {{- if eq "" .Values.iap.clientId }} + {{ fail "iap.clientID can't be empty if iap.enabled is set to true" }} + {{- end }} + client_id: {{ .Values.iap.clientId | b64enc }} + {{- if eq "" .Values.iap.clientSecret }} + {{ fail "iap.clientSecret can't be empty if iap.enabled is set to true" }} + {{- end }} + client_secret: {{ .Values.iap.clientSecret | b64enc }} +{{- end }} \ No newline at end of file diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.logconfig.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.logconfig.yaml new file mode 100644 index 000000000..40b929ae7 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.logconfig.yaml @@ -0,0 +1,19 @@ +{{- if .Values.conf.log.enabled }} +kind: Secret +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" . }}-log-config + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +type: Opaque +stringData: + log-config.yaml: | + {{- toYaml .Values.conf.log.config | nindent 4 }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.registry.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.registry.yaml new file mode 100644 index 000000000..a054069fb --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secret.registry.yaml @@ -0,0 +1,23 @@ +{{- range $name, $cred := dict "db" (.Values.image.credentials) "init-certs" (.Values.tls.selfSigner.image.credentials) }} +{{- if not (empty $cred) }} +{{- if or (and (eq $name "init-certs") $.Values.tls.enabled) (ne $name "init-certs") }} +--- +kind: Secret +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" $ }}.{{ $name }}.registry + namespace: {{ $.Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" $ }} + app.kubernetes.io/name: {{ template "cockroachdb.name" $ }} + app.kubernetes.io/instance: {{ $.Release.Name | quote }} + app.kubernetes.io/managed-by: {{ $.Release.Service | quote }} + {{- with $.Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +type: kubernetes.io/dockerconfigjson +data: + .dockerconfigjson: {{ printf `{"auths":{%s:{"auth":"%s"}}}` ($cred.registry | quote) (printf "%s:%s" $cred.username $cred.password | b64enc) | b64enc | quote }} +{{- end }} +{{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/secrets.init.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secrets.init.yaml new file mode 100644 index 000000000..4d13a35ff --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/secrets.init.yaml @@ -0,0 +1,20 @@ +{{- if .Values.init.provisioning.enabled }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ template "cockroachdb.fullname" . }}-init + namespace: {{ .Release.Namespace | quote }} +type: Opaque +stringData: + +{{- range $user := .Values.init.provisioning.users }} +{{- if $user.password }} + {{ $user.name }}-password: {{ $user.password | quote }} +{{- end }} +{{- end }} + +{{- range $clusterSetting, $clusterSettingValue := .Values.init.provisioning.clusterSettings }} + {{ $clusterSetting | replace "." "-" }}-cluster-setting: {{ $clusterSettingValue | quote }} +{{- end }} + +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.discovery.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.discovery.yaml new file mode 100644 index 000000000..8fe2a427a --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.discovery.yaml @@ -0,0 +1,64 @@ +# This service only exists to create DNS entries for each pod in +# the StatefulSet such that they can resolve each other's IP addresses. +# It does not create a load-balanced ClusterIP and should not be used directly +# by clients in most circumstances. +kind: Service +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.service.discovery.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + annotations: + # Use this annotation in addition to the actual field below because the + # annotation will stop being respected soon, but the field is broken in + # some versions of Kubernetes: + # https://github.com/kubernetes/kubernetes/issues/58662 + service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" + # Enable automatic monitoring of all instances when Prometheus is running + # in the cluster. + {{- if .Values.prometheus.enabled }} + prometheus.io/scrape: "true" + prometheus.io/path: _status/vars + prometheus.io/port: {{ .Values.service.ports.http.port | quote }} + {{- end }} + {{- with .Values.service.discovery.annotations }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + clusterIP: None + # We want all Pods in the StatefulSet to have their addresses published for + # the sake of the other CockroachDB Pods even before they're ready, since they + # have to be able to talk to each other in order to become ready. + publishNotReadyAddresses: true + ports: + {{- $ports := .Values.service.ports }} + # The main port, served by gRPC, serves Postgres-flavor SQL, inter-node + # traffic and the CLI. + - name: {{ $ports.grpc.external.name | quote }} + port: {{ $ports.grpc.external.port | int64 }} + targetPort: grpc + {{- if ne ($ports.grpc.internal.port | int64) ($ports.grpc.external.port | int64) }} + - name: {{ $ports.grpc.internal.name | quote }} + port: {{ $ports.grpc.internal.port | int64 }} + targetPort: grpc + {{- end }} + # The secondary port serves the UI as well as health and debug endpoints. + - name: {{ $ports.http.name | quote }} + port: {{ $ports.http.port | int64 }} + targetPort: http + selector: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.public.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.public.yaml new file mode 100644 index 000000000..251e9ab08 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/service.public.yaml @@ -0,0 +1,55 @@ +# This Service is meant to be used by clients of the database. +# It exposes a ClusterIP that will automatically load balance connections +# to the different database Pods. +kind: Service +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" . }}-public + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.service.public.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- if or .Values.service.public.annotations .Values.tls.enabled .Values.iap.enabled }} + annotations: + {{- with .Values.service.public.annotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- if .Values.tls.enabled }} + service.alpha.kubernetes.io/app-protocols: '{"http":"HTTPS"}' + {{- end }} + {{- if .Values.iap.enabled }} + beta.cloud.google.com/backend-config: '{"default": "{{ template "cockroachdb.fullname" . }}"}' + {{- end }} + {{- end }} +spec: + type: {{ .Values.service.public.type | quote }} + ports: + {{- $ports := .Values.service.ports }} + # The main port, served by gRPC, serves Postgres-flavor SQL, inter-node + # traffic and the CLI. + - name: {{ $ports.grpc.external.name | quote }} + port: {{ $ports.grpc.external.port | int64 }} + targetPort: grpc + {{- if ne ($ports.grpc.internal.port | int64) ($ports.grpc.external.port | int64) }} + - name: {{ $ports.grpc.internal.name | quote }} + port: {{ $ports.grpc.internal.port | int64 }} + targetPort: grpc + {{- end }} + # The secondary port serves the UI as well as health and debug endpoints. + - name: {{ $ports.http.name | quote }} + port: {{ $ports.http.port | int64 }} + targetPort: http + selector: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceMonitor.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceMonitor.yaml new file mode 100644 index 000000000..42f2390b4 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceMonitor.yaml @@ -0,0 +1,54 @@ +{{- $serviceMonitor := .Values.serviceMonitor -}} +{{- $ports := .Values.service.ports -}} +{{- if $serviceMonitor.enabled }} +apiVersion: monitoring.coreos.com/v1 +kind: ServiceMonitor +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- if $serviceMonitor.labels }} + {{- toYaml $serviceMonitor.labels | nindent 4 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- if $serviceMonitor.annotations }} + annotations: + {{- toYaml $serviceMonitor.annotations | nindent 4 }} + {{- end }} +spec: + selector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.service.discovery.labels }} + {{- toYaml . | nindent 6 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 6 }} + {{- end }} + namespaceSelector: + {{- if $serviceMonitor.namespaced }} + matchNames: + - {{ .Release.Namespace }} + {{- else }} + any: true + {{- end }} + endpoints: + - port: {{ $ports.http.name | quote }} + path: /_status/vars + {{- if $serviceMonitor.interval }} + interval: {{ $serviceMonitor.interval }} + {{- end }} + {{- if $serviceMonitor.scrapeTimeout }} + scrapeTimeout: {{ $serviceMonitor.scrapeTimeout }} + {{- end }} + {{- if .Values.serviceMonitor.tlsConfig }} + tlsConfig: {{ toYaml .Values.serviceMonitor.tlsConfig | nindent 6 }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certRotateSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certRotateSelfSigner.yaml new file mode 100644 index 000000000..a27cba921 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certRotateSelfSigner.yaml @@ -0,0 +1,22 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} + {{ template "cockroachdb.tls.certs.selfSigner.validation" . }} +kind: ServiceAccount +apiVersion: v1 +metadata: + name: {{ template "rotatecerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- if .Values.tls.certs.selfSigner.svcAccountAnnotations }} + annotations: + {{- with .Values.tls.certs.selfSigner.svcAccountAnnotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certSelfSigner.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certSelfSigner.yaml new file mode 100644 index 000000000..3ce2d63e9 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount-certSelfSigner.yaml @@ -0,0 +1,25 @@ +{{- if and .Values.tls.enabled .Values.tls.certs.selfSigner.enabled }} + {{ template "cockroachdb.tls.certs.selfSigner.validation" . }} +kind: ServiceAccount +apiVersion: v1 +metadata: + name: {{ template "selfcerts.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + annotations: + # This is what defines this resource as a hook. Without this line, the + # job is considered part of the release. + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": hook-succeeded,hook-failed + {{- with .Values.tls.certs.selfSigner.svcAccountAnnotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount.yaml new file mode 100644 index 000000000..3af9be9aa --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/serviceaccount.yaml @@ -0,0 +1,21 @@ +{{- if .Values.statefulset.serviceAccount.create }} +kind: ServiceAccount +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.serviceAccount.name" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- if .Values.statefulset.serviceAccount.annotations }} + annotations: + {{- with .Values.statefulset.serviceAccount.annotations }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/statefulset.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/statefulset.yaml new file mode 100644 index 000000000..5be883940 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/statefulset.yaml @@ -0,0 +1,563 @@ +{{ template "cockroachdb.conf.log.validation" . }} +{{ template "cockroachdb.conf.store.validation" . }} +kind: StatefulSet +apiVersion: {{ template "cockroachdb.statefulset.apiVersion" . }} +metadata: + name: {{ template "cockroachdb.fullname" . }} + namespace: {{ .Release.Namespace | quote }} + labels: + helm.sh/chart: {{ template "cockroachdb.chart" . }} + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + app.kubernetes.io/managed-by: {{ .Release.Service | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + serviceName: {{ template "cockroachdb.fullname" . }} + replicas: {{ .Values.statefulset.replicas | int64 }} + updateStrategy: {{- toYaml .Values.statefulset.updateStrategy | nindent 4 }} + podManagementPolicy: {{ .Values.statefulset.podManagementPolicy | quote }} + selector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 6 }} + {{- end }} + template: + metadata: + labels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.statefulset.annotations }} + annotations: {{- toYaml . | nindent 8 }} + {{- end }} + spec: + {{- if or .Values.image.credentials (and .Values.tls.enabled .Values.tls.selfSigner.image.credentials (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager)) }} + imagePullSecrets: + {{- if .Values.image.credentials }} + - name: {{ template "cockroachdb.fullname" . }}.db.registry + {{- end }} + {{- if and .Values.tls.enabled .Values.tls.selfSigner.image.credentials (not .Values.tls.certs.provided) (not .Values.tls.certs.certManager) }} + - name: {{ template "cockroachdb.fullname" . }}.self-signed-certs.registry + {{- end }} + {{- end }} + serviceAccountName: {{ template "cockroachdb.serviceAccount.name" . }} + {{- if .Values.tls.enabled }} + initContainers: + - name: copy-certs + image: {{ .Values.tls.copyCerts.image | quote }} + imagePullPolicy: {{ .Values.tls.selfSigner.image.pullPolicy | quote }} + command: + - /bin/sh + - -c + - "cp -f /certs/* /cockroach-certs/; chmod 0400 /cockroach-certs/*.key" + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + {{- if .Values.statefulset.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + privileged: false + readOnlyRootFilesystem: true + {{- end }} + volumeMounts: + - name: certs + mountPath: /cockroach-certs/ + - name: certs-secret + mountPath: /certs/ + {{- with .Values.tls.copyCerts.resources }} + resources: {{- toYaml . | nindent 12 }} + {{- end }} + {{- range $ic := .Values.statefulset.initContainers }} + - {{- toYaml $ic | nindent 10 }} + {{ with $.Values.statefulset.volumeMounts}} + volumeMounts: + {{- toYaml . | nindent 12 }} + {{- end }} + {{- end }} + {{- end }} + {{- if or .Values.statefulset.nodeAffinity .Values.statefulset.podAffinity .Values.statefulset.podAntiAffinity }} + affinity: + {{- with .Values.statefulset.nodeAffinity }} + nodeAffinity: {{- toYaml . | nindent 10 }} + {{- end }} + {{- with .Values.statefulset.podAffinity }} + podAffinity: {{- toYaml . | nindent 10 }} + {{- end }} + {{- if .Values.statefulset.podAntiAffinity }} + podAntiAffinity: + {{- if .Values.statefulset.podAntiAffinity.type }} + {{- if eq .Values.statefulset.podAntiAffinity.type "hard" }} + requiredDuringSchedulingIgnoredDuringExecution: + - topologyKey: {{ .Values.statefulset.podAntiAffinity.topologyKey }} + labelSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 18 }} + {{- end }} + {{- else if eq .Values.statefulset.podAntiAffinity.type "soft" }} + preferredDuringSchedulingIgnoredDuringExecution: + - weight: {{ .Values.statefulset.podAntiAffinity.weight | int64 }} + podAffinityTerm: + topologyKey: {{ .Values.statefulset.podAntiAffinity.topologyKey }} + labelSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 20 }} + {{- end }} + {{- end }} + {{- else }} + {{- toYaml .Values.statefulset.podAntiAffinity | nindent 10 }} + {{- end }} + {{- end }} + {{- end }} + {{- if semverCompare ">=1.16-0" .Capabilities.KubeVersion.Version }} + topologySpreadConstraints: + - labelSelector: + matchLabels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.statefulset.labels }} + {{- toYaml . | nindent 12 }} + {{- end }} + {{- with .Values.statefulset.topologySpreadConstraints }} + maxSkew: {{ .maxSkew }} + topologyKey: {{ .topologyKey }} + whenUnsatisfiable: {{ .whenUnsatisfiable }} + {{- end }} + {{- end }} + {{- with .Values.statefulset.nodeSelector }} + nodeSelector: {{- toYaml . | nindent 8 }} + {{- end }} + {{- if .Values.statefulset.priorityClassName }} + priorityClassName: {{ .Values.statefulset.priorityClassName }} + {{- end }} + {{- with .Values.statefulset.tolerations }} + tolerations: {{- toYaml . | nindent 8 }} + {{- end }} + # No pre-stop hook is required, a SIGTERM plus some time is all that's + # needed for graceful shutdown of a node. + terminationGracePeriodSeconds: {{ .Values.init.terminationGracePeriodSeconds }} + containers: + - name: db + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy | quote }} + args: + - shell + - -ecx + # The use of qualified `hostname -f` is crucial: + # Other nodes aren't able to look up the unqualified hostname. + # + # `--join` CLI flag is hardcoded to exactly 3 Pods, because: + # 1. Having `--join` value depending on `statefulset.replicas` + # will trigger undesired restart of existing Pods when + # StatefulSet is scaled up/down. We want to scale without + # restarting existing Pods. + # 2. At least one Pod in `--join` is enough to successfully + # join CockroachDB cluster and gossip with all other existing + # Pods, even if there are 3 or more Pods. + # 3. It's harmless for `--join` to have 3 Pods even for 1-Pod + # clusters, while it gives us opportunity to scale up even if + # some Pods of existing cluster are down (for whatever reason). + # See details explained here: + # https://github.com/helm/charts/pull/18993#issuecomment-558795102 + - >- + exec /cockroach/cockroach + {{- if index .Values.conf `single-node` }} + start-single-node + {{- else }} + start --join= + {{- if .Values.conf.join }} + {{- join `,` .Values.conf.join -}} + {{- else }} + {{- range $i, $_ := until 3 -}} + {{- if gt $i 0 -}},{{- end -}} + ${STATEFULSET_NAME}-{{ $i }}.${STATEFULSET_FQDN}:{{ $.Values.service.ports.grpc.internal.port | int64 -}} + {{- end -}} + {{- end }} + {{- with index .Values.conf `cluster-name` }} + --cluster-name={{ . }} + {{- if index $.Values.conf `disable-cluster-name-verification` }} + --disable-cluster-name-verification + {{- end }} + {{- end }} + {{- end }} + --advertise-host=$(hostname).${STATEFULSET_FQDN} + {{- if .Values.tls.enabled }} + --certs-dir=/cockroach/cockroach-certs/ + {{- else }} + --insecure + {{- end }} + {{- with .Values.conf.attrs }} + --attrs={{ join `:` . }} + {{- end }} + {{- if index .Values.conf `http-port` }} + --http-port={{ index .Values.conf `http-port` | int64 }} + {{- else }} + --http-port={{ index .Values.service.ports.http.port | int64 }} + {{- end }} + {{- if .Values.conf.port }} + --port={{ .Values.conf.port | int64 }} + {{- else }} + --port={{ .Values.service.ports.grpc.internal.port | int64 }} + {{- end }} + --cache={{ .Values.conf.cache }} + {{- with index .Values.conf `max-disk-temp-storage` }} + --max-disk-temp-storage={{ . }} + {{- end }} + {{- with index .Values.conf `max-offset` }} + --max-offset={{ . }} + {{- end }} + --max-sql-memory={{ index .Values.conf `max-sql-memory` }} + {{- with .Values.conf.locality }} + --locality={{ . }} + {{- end }} + {{- with index .Values.conf `sql-audit-dir` }} + --sql-audit-dir={{ . }} + {{- end }} + {{- if .Values.conf.store.enabled }} + {{- range $idx := until (int .Values.conf.store.count) }} + {{- $_ := set $ "Args" (dict "idx" $idx) }} + --store={{ include "cockroachdb.conf.store" $ }} + {{- end }} + {{- end }} + {{- with index .Values.conf `wal-failover` `value` }} + {{- template "cockroachdb.conf.wal-failover.validation" $ }} + --wal-failover={{ . }} + {{- end }} + {{- if .Values.conf.log.enabled }} + --log-config-file=/cockroach/log-config/log-config.yaml + {{- else }} + --logtostderr={{ .Values.conf.logtostderr }} + {{- end }} + {{- range .Values.statefulset.args }} + {{ . }} + {{- end }} + env: + - name: STATEFULSET_NAME + value: {{ template "cockroachdb.fullname" . }} + - name: STATEFULSET_FQDN + value: {{ template "cockroachdb.fullname" . }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }} + - name: COCKROACH_CHANNEL + value: kubernetes-helm + {{- with .Values.statefulset.env }} + {{- toYaml . | nindent 12 }} + {{- end }} + ports: + - name: grpc + {{- if .Values.conf.port }} + containerPort: {{ .Values.conf.port | int64 }} + {{- else }} + containerPort: {{ .Values.service.ports.grpc.internal.port | int64 }} + {{- end }} + protocol: TCP + - name: http + {{- if index .Values.conf `http-port` }} + containerPort: {{ index .Values.conf `http-port` | int64 }} + {{- else }} + containerPort: {{ index .Values.service.ports.http.port | int64 }} + {{- end }} + protocol: TCP + volumeMounts: + {{- range $i := until (int .Values.conf.store.count) }} + {{- if eq $i 0 }} + - name: datadir + mountPath: /cockroach/{{ $.Values.conf.path }}/ + {{- else }} + - name: datadir-{{ add1 $i }} + mountPath: /cockroach/{{ $.Values.conf.path }}-{{ add1 $i }}/ + {{- end }} + {{- end }} + {{- with index .Values.conf `wal-failover` `persistentVolume` }} + {{- if .enabled }} + - name: failoverdir + mountPath: /cockroach/{{ .path }}/ + {{- end }} + {{- end }} + {{- if .Values.tls.enabled }} + - name: certs + mountPath: /cockroach/cockroach-certs/ + {{- if .Values.tls.certs.provided }} + - name: certs-secret + mountPath: /cockroach/certs/ + {{- end }} + {{- end }} + {{- range .Values.statefulset.secretMounts }} + - name: {{ printf "secret-%s" . | quote }} + mountPath: {{ printf "/etc/cockroach/secrets/%s" . | quote }} + readOnly: true + {{- end }} + {{- if .Values.conf.log.enabled }} + - name: log-config + mountPath: /cockroach/log-config + readOnly: true + {{- end }} + {{- if .Values.conf.log.persistentVolume.enabled }} + - name: logsdir + mountPath: /cockroach/{{ .Values.conf.log.persistentVolume.path }}/ + {{- end }} + {{- with .Values.statefulset.volumeMounts }} + {{ toYaml . | nindent 12 }} + {{- end }} + {{- if .Values.statefulset.customStartupProbe }} + startupProbe: + {{ toYaml .Values.statefulset.customStartupProbe | nindent 12 }} + {{- end }} + livenessProbe: + {{- if .Values.statefulset.customLivenessProbe }} + {{ toYaml .Values.statefulset.customLivenessProbe | nindent 12 }} + {{- else }} + httpGet: + path: /health + port: http + {{- if .Values.tls.enabled }} + scheme: HTTPS + {{- end }} + initialDelaySeconds: 30 + periodSeconds: 5 + {{- end }} + readinessProbe: + {{- if .Values.statefulset.customReadinessProbe }} + {{ toYaml .Values.statefulset.customReadinessProbe | nindent 12 }} + {{- else }} + httpGet: + path: /health?ready=1 + port: http + {{- if .Values.tls.enabled }} + scheme: HTTPS + {{- end }} + initialDelaySeconds: 10 + periodSeconds: 5 + failureThreshold: 2 + {{- end }} + {{- if eq (include "cockroachdb.securityContext.versionValidation" .) "true" }} + {{- if .Values.statefulset.securityContext.enabled }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + privileged: false + readOnlyRootFilesystem: true + {{- end }} + {{- end }} + {{- with .Values.statefulset.resources }} + resources: {{- toYaml . | nindent 12 }} + {{- end }} + volumes: + {{- range $i := until (int .Values.conf.store.count) }} + {{- if eq $i 0 }} + - name: datadir + {{- if $.Values.storage.persistentVolume.enabled }} + persistentVolumeClaim: + claimName: datadir + {{- else if $.Values.storage.hostPath }} + hostPath: + path: {{ $.Values.storage.hostPath | quote }} + {{- else }} + emptyDir: {} + {{- end }} + {{- else }} + - name: datadir-{{ add1 $i }} + {{- if $.Values.storage.persistentVolume.enabled }} + persistentVolumeClaim: + claimName: datadir-{{ add1 $i }} + {{- else if $.Values.storage.hostPath }} + {{- $_ := set $ "Args" (dict "idx" $i) }} + hostPath: + path: {{ include "cockroachdb.storage.hostPath.computation" $ }} + {{- else }} + emptyDir: {} + {{- end }} + {{- end }} + {{- end }} + {{- with index .Values.conf `wal-failover` }} + {{- if .value }} + - name: failoverdir + {{- if .persistentVolume.enabled }} + persistentVolumeClaim: + claimName: failoverdir + {{- else }} + emptyDir: {} + {{- end }} + {{- end }} + {{- end }} + {{- with .Values.statefulset.volumes }} + {{ toYaml . | nindent 8 }} + {{- end }} + {{- if .Values.tls.enabled }} + - name: certs + emptyDir: {} + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }} + - name: certs-secret + {{- if or .Values.tls.certs.tlsSecret .Values.tls.certs.certManager .Values.tls.certs.selfSigner.enabled }} + projected: + sources: + - secret: + {{- if .Values.tls.certs.selfSigner.enabled }} + name: {{ template "cockroachdb.fullname" . }}-node-secret + {{ else }} + name: {{ .Values.tls.certs.nodeSecret }} + {{ end -}} + items: + - key: ca.crt + path: ca.crt + mode: 256 + - key: tls.crt + path: node.crt + mode: 256 + - key: tls.key + path: node.key + mode: 256 + {{- else }} + secret: + secretName: {{ .Values.tls.certs.nodeSecret }} + defaultMode: 256 + {{- end }} + {{- end }} + {{- end }} + {{- range .Values.statefulset.secretMounts }} + - name: {{ printf "secret-%s" . | quote }} + secret: + secretName: {{ . | quote }} + {{- end }} + {{- if .Values.conf.log.enabled }} + - name: log-config + secret: + secretName: {{ template "cockroachdb.fullname" . }}-log-config + {{- end }} + {{- if .Values.conf.log.enabled }} + - name: logsdir + {{- if .Values.conf.log.persistentVolume.enabled }} + persistentVolumeClaim: + claimName: logsdir + {{- else }} + emptyDir: {} + {{- end }} + {{- end }} + {{- if eq (include "cockroachdb.securityContext.versionValidation" .) "true" }} + {{- if and .Values.securityContext.enabled }} + securityContext: + seccompProfile: + type: "RuntimeDefault" + fsGroup: 1000 + runAsGroup: 1000 + runAsUser: 1000 + runAsNonRoot: true + {{- end }} + {{- end }} +{{- if or .Values.storage.persistentVolume.enabled (index .Values.conf `wal-failover` `persistentVolume` `enabled`) .Values.conf.log.persistentVolume.enabled }} + volumeClaimTemplates: + {{- if .Values.storage.persistentVolume.enabled }} + {{- range $i := until (int .Values.conf.store.count) }} + - metadata: + {{- if eq $i 0 }} + name: datadir + {{- else }} + name: datadir-{{ add1 $i }} + {{- end }} + labels: + app.kubernetes.io/name: {{ template "cockroachdb.name" $ }} + app.kubernetes.io/instance: {{ $.Release.Name | quote }} + {{- with $.Values.storage.persistentVolume.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with $.Values.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with $.Values.storage.persistentVolume.annotations }} + annotations: {{- toYaml . | nindent 10 }} + {{- end }} + spec: + accessModes: ["ReadWriteOnce"] + {{- if $.Values.storage.persistentVolume.storageClass }} + {{- if (eq "-" $.Values.storage.persistentVolume.storageClass) }} + storageClassName: "" + {{- else }} + storageClassName: {{ $.Values.storage.persistentVolume.storageClass | quote}} + {{- end }} + {{- end }} + resources: + requests: + storage: {{ $.Values.storage.persistentVolume.size | quote }} + {{- end }} + {{- end }} + {{- with index .Values.conf `wal-failover` }} + {{- if .persistentVolume.enabled }} + - metadata: + name: failoverdir + labels: + app.kubernetes.io/name: {{ template "cockroachdb.name" $ }} + app.kubernetes.io/instance: {{ $.Release.Name | quote }} + {{- with .persistentVolume.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with $.Values.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with .persistentVolume.annotations }} + annotations: {{- toYaml . | nindent 10 }} + {{- end }} + spec: + accessModes: ["ReadWriteOnce"] + {{- with .persistentVolume.storageClass }} + {{- if eq "-" . }} + storageClassName: "" + {{- else }} + storageClassName: {{ . | quote}} + {{- end }} + {{- end }} + resources: + requests: + storage: {{ .persistentVolume.size | quote }} + {{- end }} + {{- end }} + {{- if .Values.conf.log.persistentVolume.enabled }} + - metadata: + name: logsdir + labels: + app.kubernetes.io/name: {{ template "cockroachdb.name" . }} + app.kubernetes.io/instance: {{ .Release.Name | quote }} + {{- with .Values.conf.log.persistentVolume.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with .Values.labels }} + {{- toYaml . | nindent 10 }} + {{- end }} + {{- with .Values.conf.log.persistentVolume.annotations }} + annotations: {{- toYaml . | nindent 10 }} + {{- end }} + spec: + accessModes: ["ReadWriteOnce"] + {{- if .Values.conf.log.persistentVolume.storageClass }} + {{- if (eq "-" .Values.conf.log.persistentVolume.storageClass) }} + storageClassName: "" + {{- else }} + storageClassName: {{ .Values.conf.log.persistentVolume.storageClass | quote}} + {{- end }} + {{- end }} + resources: + requests: + storage: {{ .Values.conf.log.persistentVolume.size | quote }} + {{- end }} +{{- end }} diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/templates/tests/client.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/templates/tests/client.yaml new file mode 100644 index 000000000..8656b8ed6 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/templates/tests/client.yaml @@ -0,0 +1,65 @@ +kind: Pod +apiVersion: v1 +metadata: + name: {{ template "cockroachdb.fullname" . }}-test + namespace: {{ .Release.Namespace | quote }} +{{- if .Values.networkPolicy.enabled }} + labels: + {{ template "cockroachdb.fullname" . }}-client: "true" +{{- end }} + annotations: + helm.sh/hook: test-success +spec: + restartPolicy: Never +{{- if .Values.image.credentials }} + imagePullSecrets: + - name: {{ template "cockroachdb.fullname" . }}.db.registry +{{- end }} + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager }} + volumes: + - name: client-certs + {{- if or .Values.tls.certs.tlsSecret .Values.tls.certs.certManager }} + projected: + sources: + - secret: + name: {{ .Values.tls.certs.clientRootSecret }} + items: + - key: ca.crt + path: ca.crt + mode: 0400 + - key: tls.crt + path: client.root.crt + mode: 0400 + - key: tls.key + path: client.root.key + mode: 0400 + {{- else }} + secret: + secretName: {{ .Values.tls.certs.clientRootSecret }} + defaultMode: 0400 + {{- end }} + {{- end }} + containers: + - name: client-test + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy | quote }} + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager }} + volumeMounts: + - name: client-certs + mountPath: /cockroach-certs + {{- end }} + command: + - /cockroach/cockroach + - sql + {{- if or .Values.tls.certs.provided .Values.tls.certs.certManager }} + - --certs-dir + - /cockroach-certs + {{- else }} + - --insecure + {{- end}} + - --host + - {{ template "cockroachdb.fullname" . }}-public.{{ .Release.Namespace }} + - --port + - {{ .Values.service.ports.grpc.external.port | quote }} + - -e + - SHOW DATABASES; diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/values.schema.json b/charts/cockroach-labs/cockroachdb/15.0.6/values.schema.json new file mode 100644 index 000000000..b23c47974 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/values.schema.json @@ -0,0 +1,97 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "properties": { + "tls": { + "type": "object", + "properties": { + "certs": { + "type": "object", + "properties": { + "selfSigner": { + "type": "object", + "required": ["enabled", "caProvided"], + "properties": { + "enabled": { + "type": "boolean" + }, + "caProvided": { + "type": "boolean" + } + }, + "if": { + "properties": { + "enabled": { + "const": true + } + } + }, + "then": { + "if": { + "properties": { + "caProvided": { + "const": false + } + } + }, + "then": { + "properties": { + "caCertDuration" : { + "type": "string", + "pattern": "^[0-9]*h$" + }, + "caCertExpiryWindow": { + "type": "string", + "pattern": "^[0-9]*h$" + } + } + }, + "properties": { + "clientCertDuration": { + "type": "string", + "pattern": "^[0-9]*h$" + }, + "clientCertExpiryWindow": { + "type": "string", + "pattern": "^[0-9]*h$" + }, + "nodeCertDuration": { + "type": "string", + "pattern": "^[0-9]*h$" + }, + "nodeCertExpiryWindow": { + "type": "string", + "pattern": "^[0-9]*h$" + }, + "rotateCerts": { + "type": "boolean" + } + } + } + } + } + }, + "selfSigner": { + "type": "object", + "properties": { + "image": { + "type": "object", + "required": ["repository", "tag", "pullPolicy"], + "properties": { + "repository": { + "type": "string" + }, + "tag": { + "type": "string" + }, + "pullPolicy": { + "type": "string", + "pattern": "^(Always|Never|IfNotPresent)$" + } + } + } + } + } + } + } + } +} \ No newline at end of file diff --git a/charts/cockroach-labs/cockroachdb/15.0.6/values.yaml b/charts/cockroach-labs/cockroachdb/15.0.6/values.yaml new file mode 100644 index 000000000..17bf8cc63 --- /dev/null +++ b/charts/cockroach-labs/cockroachdb/15.0.6/values.yaml @@ -0,0 +1,713 @@ +# Generated file, DO NOT EDIT. Source: build/templates/values.yaml +# Overrides the chart name against the label "app.kubernetes.io/name: " placed on every resource this chart creates. +nameOverride: "" + +# Override the resource names created by this chart which originally is generated using release and chart name. +fullnameOverride: "" + +image: + repository: cockroachdb/cockroach + tag: v24.3.4 + pullPolicy: IfNotPresent + credentials: {} + # registry: docker.io + # username: john_doe + # password: changeme + + +# Additional labels to apply to all Kubernetes resources created by this chart. +labels: {} + # app.kubernetes.io/part-of: my-app + + +# Cluster's default DNS domain. +# You should overwrite it if you're using a different one, +# otherwise CockroachDB nodes discovery won't work. +clusterDomain: cluster.local + + +conf: + # An ordered list of CockroachDB node attributes. + # Attributes are arbitrary strings specifying machine capabilities. + # Machine capabilities might include specialized hardware or number of cores + # (e.g. "gpu", "x16c"). + attrs: [] + # - x16c + # - gpu + + # Total size in bytes for caches, shared evenly if there are multiple + # storage devices. Size suffixes are supported (e.g. `1GB` and `1GiB`). + # A percentage of physical memory can also be specified (e.g. `.25`). + cache: 25% + + # Sets a name to verify the identity of a cluster. + # The value must match between all nodes specified via `conf.join`. + # This can be used as an additional verification when either the node or + # cluster, or both, have not yet been initialized and do not yet know their + # cluster ID. + # To introduce a cluster name into an already-initialized cluster, pair this + # option with `conf.disable-cluster-name-verification: yes`. + cluster-name: "" + + # Tell the server to ignore `conf.cluster-name` mismatches. + # This is meant for use when opting an existing cluster into starting to use + # cluster name verification, or when changing the cluster name. + # The cluster should be restarted once with `conf.cluster-name` and + # `conf.disable-cluster-name-verification: yes` combined, and once all nodes + # have been updated to know the new cluster name, the cluster can be restarted + # again with `conf.disable-cluster-name-verification: no`. + # This option has no effect if `conf.cluster-name` is not specified. + disable-cluster-name-verification: false + + # The addresses for connecting a CockroachDB nodes to an existing cluster. + # If you are deploying a second CockroachDB instance that should join a first + # one, use the below list to join to the existing instance. + # Each item in the array should be a FQDN (and port if needed) resolvable by + # new Pods. + join: [] + + # New logging configuration. + log: + enabled: false + # https://www.cockroachlabs.com/docs/v21.1/configure-logs + config: + # file-defaults: + # dir: /cockroach/cockroach-logs + # fluent-defaults: + # format: json-fluent + # sinks: + # stderr: + # channels: [DEV] + persistentVolume: + # If enabled, then a PersistentVolumeClaim will be created and + # used to store CockroachDB's logs. + enabled: false + # CockroachDB's logs volume mount path. This gets prepended with + # `/cockroach/` in the stateful set. The `conf.log.config` should have + # `file-defaults.dir` to specify the log path and should reference the + # mounted volume. + path: cockroach-logs + size: 10Gi + # If defined, then `storageClassName: `. + # If set to "-", then `storageClassName: ""`, which disables dynamic + # provisioning. + # If undefined or empty (default), then no `storageClassName` spec is + # set, so the default provisioner will be chosen (gp2 on AWS, standard + # on GKE, AWS & OpenStack). + storageClass: "" + # Additional labels to apply to the created PersistentVolumeClaims. + labels: {} + # Additional annotations to apply to the created PersistentVolumeClaims. + annotations: {} + + # Logs at or above this threshold to STDERR. Ignored when "log" is enabled + logtostderr: INFO + + # Maximum storage capacity available to store temporary disk-based data for + # SQL queries that exceed the memory budget (e.g. join, sorts, etc are + # sometimes able to spill intermediate results to disk). + # Accepts numbers interpreted as bytes, size suffixes (e.g. `32GB` and + # `32GiB`) or a percentage of disk size (e.g. `10%`). + # The location of the temporary files is within the first store dir. + # If expressed as a percentage, `max-disk-temp-storage` is interpreted + # relative to the size of the storage device on which the first store is + # placed. The temp space usage is never counted towards any store usage + # (although it does share the device with the first store) so, when + # configuring this, make sure that the size of this temp storage plus the size + # of the first store don't exceed the capacity of the storage device. + # If the first store is an in-memory one (i.e. `type=mem`), then this + # temporary "disk" data is also kept in-memory. + # A percentage value is interpreted as a percentage of the available internal + # memory. + # max-disk-temp-storage: 0GB + + # Maximum allowed clock offset for the cluster. If observed clock offsets + # exceed this limit, servers will crash to minimize the likelihood of + # reading inconsistent data. Increasing this value will increase the time + # to recovery of failures as well as the frequency of uncertainty-based + # read restarts. + # Note, that this value must be the same on all nodes in the cluster. + # In order to change it, all nodes in the cluster must be stopped + # simultaneously and restarted with the new value. + # max-offset: 500ms + + # Maximum memory capacity available to store temporary data for SQL clients, + # including prepared queries and intermediate data rows during query + # execution. Accepts numbers interpreted as bytes, size suffixes + # (e.g. `1GB` and `1GiB`) or a percentage of physical memory (e.g. `.25`). + max-sql-memory: 25% + + # An ordered, comma-separated list of key-value pairs that describe the + # topography of the machine. Topography might include country, datacenter + # or rack designations. Data is automatically replicated to maximize + # diversities of each tier. The order of tiers is used to determine + # the priority of the diversity, so the more inclusive localities like + # country should come before less inclusive localities like datacenter. + # The tiers and order must be the same on all nodes. Including more tiers + # is better than including fewer. For example: + # locality: country=us,region=us-west,datacenter=us-west-1b,rack=12 + # locality: country=ca,region=ca-east,datacenter=ca-east-2,rack=4 + # locality: planet=earth,province=manitoba,colo=secondary,power=3 + locality: "" + + # Run CockroachDB instances in standalone mode with replication disabled + # (replication factor = 1). + # Enabling this option makes the following values to be ignored: + # - `conf.cluster-name` + # - `conf.disable-cluster-name-verification` + # - `conf.join` + # + # WARNING: Enabling this option makes each deployed Pod as a STANDALONE + # CockroachDB instance, so the StatefulSet does NOT FORM A CLUSTER. + # Don't use this option for production deployments unless you clearly + # understand what you're doing. + # Usually, this option is intended to be used in conjunction with + # `statefulset.replicas: 1` for temporary one-time deployments (like + # running E2E tests, for example). + single-node: false + + # If non-empty, create a SQL audit log in the specified directory. + sql-audit-dir: "" + + # WARNING this parameter is deprecated and will be removed in a future version. Use `.service.ports.grpc.internal.port` instead + port: "" + + # WARNING this parameter is deprecated and will be removed in a future version. Use `.service.ports.http.port` instead + http-port: "" + + # CockroachDB's data mount path. + # For multi-store configuration, the path for each store is evaluated as: + # Store 1: cockroach-data + # Store 2: cockroach-data-2 + # Store N: cockroach-data-N + path: cockroach-data + + # CockroachDB's storage configuration https://www.cockroachlabs.com/docs/v21.1/cockroach-start.html#storage + # Uses --store flag + store: + enabled: false + # Number of data stores per node. + # For multi-store configuration, set this to a value greater than 1. + count: 1 + # Should be empty or 'mem' + type: + # Required for type=mem. If type and size is empty - storage.persistentVolume.size is used + size: + # Arbitrary strings, separated by colons, specifying disk type or capability + attrs: + + # CockroachDB's WAL failover configuration: + # https://www.cockroachlabs.com/docs/stable/cockroach-start#write-ahead-log-wal-failover + # Uses `--wal-failover` flag + wal-failover: + # The value to be passed to the `--wal-failover` flag. + # Possible configurations: + # 1. ``: If empty, `--wal-failover` is not passed to cockroach start. + # 2. `disabled`: Disables WAL failover. + # 3. `among-stores`: Enables WAL failover among multiple stores. This requires + # `conf.store.count` to be greater than 1. + # 4. `path=`: Enables WAL failover to a side disk. This requires + # a persistent volume should be mounted at this path (e.g. `path=/cockroach/cockroach-failover`). + value: + + persistentVolume: + # If enabled, then a PersistentVolumeClaim will be created and + # used for WAL failover as a side disk. + # https://www.cockroachlabs.com/docs/v24.3/wal-failover#provision-a-single-store-cluster-and-side-disk-for-wal-failover + enabled: false + # Mount path for the side disk. This gets prepended with `/cockroach/` in the stateful set. + path: cockroach-failover + size: 25Gi + # If defined, then `storageClassName: `. + # If set to "-", then `storageClassName: ""`, which disables dynamic + # provisioning. + # If undefined or empty (default), then no `storageClassName` spec is + # set, so the default provisioner will be chosen (gp2 on AWS, standard + # on GKE, AWS & OpenStack). + storageClass: "" + # Additional labels to apply to the created PersistentVolumeClaims. + labels: {} + # Additional annotations to apply to the created PersistentVolumeClaims. + annotations: {} + +statefulset: + replicas: 3 + updateStrategy: + type: RollingUpdate + podManagementPolicy: Parallel + budget: + maxUnavailable: 1 + + # List of additional command-line arguments you want to pass to the + # `cockroach start` command. + args: [] + # - --disable-cluster-name-verification + + # List of extra environment variables to pass into container + env: [] + # - name: COCKROACH_ENGINE_MAX_SYNC_DURATION + # value: "24h" + + # List of Secrets names in the same Namespace as the CockroachDB cluster, + # which shall be mounted into `/etc/cockroach/secrets/` for every cluster + # member. + secretMounts: [] + + # Additional labels to apply to this StatefulSet and all its Pods. + labels: + app.kubernetes.io/component: cockroachdb + + # Additional annotations to apply to the Pods of this StatefulSet. + annotations: {} + + # Affinity rules for scheduling Pods of this StatefulSet on Nodes. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity + nodeAffinity: {} + # Inter-Pod Affinity rules for scheduling Pods of this StatefulSet. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity + podAffinity: {} + # Anti-affinity rules for scheduling Pods of this StatefulSet. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity + # You may either toggle options below for default anti-affinity rules, + # or specify the whole set of anti-affinity rules instead of them. + podAntiAffinity: + # The topologyKey to be used. + # Can be used to spread across different nodes, AZs, regions etc. + topologyKey: kubernetes.io/hostname + # Type of anti-affinity rules: either `soft`, `hard` or empty value (which + # disables anti-affinity rules). + type: soft + # Weight for `soft` anti-affinity rules. + # Does not apply for other anti-affinity types. + weight: 100 + + # Node selection constraints for scheduling Pods of this StatefulSet. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector + nodeSelector: {} + + # PriorityClassName given to Pods of this StatefulSet + # https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass + priorityClassName: "" + + # Taints to be tolerated by Pods of this StatefulSet. + # https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ + tolerations: [] + + # https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/ + topologySpreadConstraints: + maxSkew: 1 + topologyKey: topology.kubernetes.io/zone + whenUnsatisfiable: ScheduleAnyway + + # Uncomment the following resources definitions or pass them from + # command line to control the CPU and memory resources allocated + # by Pods of this StatefulSet. + resources: {} + # limits: + # cpu: 100m + # memory: 512Mi + # requests: + # cpu: 100m + # memory: 512Mi + + # terminationGracePeriodSeconds is the duration in seconds the Pod needs to terminate gracefully. + terminationGracePeriodSeconds: 300 + + # Custom Liveness probe + # https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-a-liveness-http-request + customLivenessProbe: {} + # httpGet: + # path: /health + # port: http + # scheme: HTTPS + # initialDelaySeconds: 30 + # periodSeconds: 5 + + # Custom Rediness probe + # https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-readiness-probes + customReadinessProbe: {} + # httpGet: + # path: /health + # port: http + # scheme: HTTPS + # initialDelaySeconds: 30 + # periodSeconds: 5 + + # Custom Startup Probe + # https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-startup-probes + customStartupProbe: {} + # httpGet: + # path: /health + # port: http + # scheme: HTTPS + # initialDelaySeconds: 30 + # periodSeconds: 5 + + securityContext: + enabled: true + + serviceAccount: + # Specifies whether this ServiceAccount should be created. + create: true + # The name of this ServiceAccount to use. + # If not set and `create` is `true`, then service account is auto-generated. + # If not set and `create` is `false`, then it uses default service account. + name: "" + # Additional serviceAccount annotations (e.g. for attaching AWS IAM roles to pods) + annotations: {} + + # initContainers allows you to add additional containers to cockroachdb statefulset. + initContainers: [] +# - name: "fetch-metadata" +# image: "badouralix/curl-jq" +# command: +# - "sh" +# - "-c" +# - "curl -s -H \"Metadata:true\" --noproxy \"*\" \"http://169.254.169.254/metadata/instance?api-version=2021-02-01\" | jq '.' > /metadata/instance_metadata.json" +# resources: {} +# # requests: +# # cpu: "10m" +# # memory: "128Mi" +# # limits: +# # cpu: "10m" +# # memory: "128Mi" +# securityContext: +# allowPrivilegeEscalation: false +# capabilities: +# drop: +# - ALL +# privileged: false +# readOnlyRootFilesystem: true + + # volumeMounts are mounted on the same path in the main crdb container and all init containers. + volumeMounts: [] +# - name: metadata +# mountPath: /metadata + + # volumes allows you to add additional volumes to cockroachdb statefulset. + volumes: [] +# - name: metadata +# emptyDir: {} + +service: + ports: + # You can set a different external and internal gRPC ports and their name. + grpc: + external: + port: 26257 + name: grpc + # If the port number is different than `external.port`, then it will be + # named as `internal.name` in Service. + internal: + # CockroachDB's port to listen to inter-communications and client connections. + port: 26257 + # If using Istio set it to `cockroach`. + name: grpc-internal + http: + # CockroachDB's port to listen to HTTP requests. + port: 8080 + name: http + + # This Service is meant to be used by clients of the database. + # It exposes a ClusterIP that will automatically load balance connections + # to the different database Pods. + public: + type: ClusterIP + # Additional labels to apply to this Service. + labels: + app.kubernetes.io/component: cockroachdb + # Additional annotations to apply to this Service. + annotations: {} + + # This service only exists to create DNS entries for each pod in + # the StatefulSet such that they can resolve each other's IP addresses. + # It does not create a load-balanced ClusterIP and should not be used directly + # by clients in most circumstances. + discovery: + # Additional labels to apply to this Service. + labels: + app.kubernetes.io/component: cockroachdb + # Additional annotations to apply to this Service. + annotations: {} + +# CockroachDB's ingress for web ui. +ingress: + enabled: false + labels: {} + annotations: {} + # kubernetes.io/ingress.class: nginx + # cert-manager.io/cluster-issuer: letsencrypt + paths: [/] + hosts: [] + # - cockroachlabs.com + tls: [] + # - hosts: [cockroachlabs.com] + # secretName: cockroachlabs-tls + +prometheus: + enabled: true + +securityContext: + enabled: true + +# CockroachDB's Prometheus operator ServiceMonitor support +serviceMonitor: + enabled: false + labels: {} + annotations: {} + interval: 10s + # scrapeTimeout: 10s + # Limits the ServiceMonitor to the current namespace if set to `true`. + namespaced: false + + # tlsConfig: TLS configuration to use when scraping the endpoint. + # Of type: https://github.com/coreos/prometheus-operator/blob/main/Documentation/api.md#tlsconfig + tlsConfig: {} + +# CockroachDB's data persistence. +# If neither `persistentVolume` nor `hostPath` is used, then data will be +# persisted in ad-hoc `emptyDir`. +storage: + # Absolute path on host to store CockroachDB's data. + # If not specified, then `emptyDir` will be used instead. + # If specified, but `persistentVolume.enabled` is `true`, then has no effect. + hostPath: "" + + # If `enabled` is `true` then a PersistentVolumeClaim will be created and + # used to store CockroachDB's data, otherwise `hostPath` is used. + persistentVolume: + enabled: true + + size: 100Gi + + # If defined, then `storageClassName: `. + # If set to "-", then `storageClassName: ""`, which disables dynamic + # provisioning. + # If undefined or empty (default), then no `storageClassName` spec is set, + # so the default provisioner will be chosen (gp2 on AWS, standard on + # GKE, AWS & OpenStack). + storageClass: "" + + # Additional labels to apply to the created PersistentVolumeClaims. + labels: {} + # Additional annotations to apply to the created PersistentVolumeClaims. + annotations: {} + + +# Kubernetes Job which initializes multi-node CockroachDB cluster. +# It's not created if `statefulset.replicas` is `1`. +init: + # Additional labels to apply to this Job and its Pod. + labels: + app.kubernetes.io/component: init + + # Additional annotations to apply to this Job. + jobAnnotations: {} + + # Additional annotations to apply to the Pod of this Job. + annotations: {} + + # Affinity rules for scheduling the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity + affinity: {} + + # Node selection constraints for scheduling the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector + nodeSelector: {} + + # Taints to be tolerated by the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ + tolerations: [] + + # The init Pod runs at cluster creation to initialize CockroachDB. It finishes + # quickly and doesn't continue to consume resources in the Kubernetes + # cluster. Normally, you should leave this section commented out, but if your + # Kubernetes cluster uses Resource Quotas and requires all pods to specify + # resource requests or limits, you can set those here. + resources: {} + # requests: + # cpu: "10m" + # memory: "128Mi" + # limits: + # cpu: "10m" + # memory: "128Mi" + + # terminationGracePeriodSeconds is the duration in seconds the Pod needs to terminate gracefully. + terminationGracePeriodSeconds: 300 + + securityContext: + enabled: true + + # Setup Physical Cluster Replication (PCR) between primary and standby cluster. + # If isPrimary is set to true, the CockroachDB cluster created is the primary cluster. + # If isPrimary is set to false, the CockroachDB cluster created is the standby cluster. + pcr: + enabled: false + # isPrimary: true + + provisioning: + enabled: false + # https://www.cockroachlabs.com/docs/stable/cluster-settings.html + clusterSettings: + # cluster.organization: "'FooCorp - Local Testing'" + # enterprise.license: "'xxxxx'" + users: [] + # - name: + # password: + # # https://www.cockroachlabs.com/docs/stable/create-user.html#parameters + # options: [LOGIN] + databases: [] + # - name: + # # https://www.cockroachlabs.com/docs/stable/create-database.html#parameters + # options: [encoding='utf-8'] + # owners: [] + # # https://www.cockroachlabs.com/docs/stable/grant.html#parameters + # owners_with_grant_option: [] + # # Backup schedules are not idemponent for now and will fail on next run + # # https://github.com/cockroachdb/cockroach/issues/57892 + # backup: + # into: s3:// + # # Enterprise-only option (revision_history) + # # https://www.cockroachlabs.com/docs/stable/create-schedule-for-backup.html#backup-options + # options: [revision_history] + # recurring: '@always' + # # Enterprise-only feature. Remove this value to use `FULL BACKUP ALWAYS` + # fullBackup: '@daily' + # schedule: + # # https://www.cockroachlabs.com/docs/stable/create-schedule-for-backup.html#schedule-options + # options: [first_run = 'now'] + + +# Whether to run securely using TLS certificates. +tls: + enabled: true + copyCerts: + image: busybox + certs: + # Bring your own certs scenario. If provided, tls.init section will be ignored. + provided: false + # Secret name for the client root cert. + clientRootSecret: cockroachdb-root + # Secret name for node cert. + nodeSecret: cockroachdb-node + # Secret name for CA cert + caSecret: cockroach-ca + # Enable if the secret is a dedicated TLS. + # TLS secrets are created by cert-mananger, for example. + tlsSecret: false + # Enable if the you want cockroach db to create its own certificates + selfSigner: + # If set, the cockroach db will generate its own certificates + enabled: true + # Run selfSigner as non-root + securityContext: + enabled: true + # If set, the user should provide the CA certificate to sign other certificates. + caProvided: false + # It holds the name of the secret with caCerts. If caProvided is set, this can not be empty. + caSecret: "" + # Minimum Certificate duration for all the certificates, all certs duration will be validated against this. + minimumCertDuration: 624h + # Duration of CA certificates in hour + caCertDuration: 43800h + # Expiry window of CA certificates means a window before actual expiry in which CA certs should be rotated. + caCertExpiryWindow: 648h + # Duration of Client certificates in hour + clientCertDuration: 672h + # Expiry window of client certificates means a window before actual expiry in which client certs should be rotated. + clientCertExpiryWindow: 48h + # Duration of node certificates in hour + nodeCertDuration: 8760h + # Expiry window of node certificates means a window before actual expiry in which node certs should be rotated. + nodeCertExpiryWindow: 168h + # If set, the cockroachdb cert selfSigner will rotate the certificates before expiry. + rotateCerts: true + # Wait time for each cockroachdb replica to become ready once it comes in running state. Only considered when rotateCerts is set to true + readinessWait: 30s + # Wait time for each cockroachdb replica to get to running state. Only considered when rotateCerts is set to true + podUpdateTimeout: 2m + # ServiceAccount annotations for selfSigner jobs (e.g. for attaching AWS IAM roles to pods) + svcAccountAnnotations: {} + + # Use cert-manager to issue certificates for mTLS. + certManager: false + # Specify an Issuer or a ClusterIssuer to use, when issuing + # node and client certificates. The values correspond to the + # issuerRef specified in the certificate. + certManagerIssuer: + group: cert-manager.io + kind: Issuer + name: cockroachdb + # Make it false when you are providing your own CA issuer + isSelfSignedIssuer: true + # Duration of CA certificates in hour + caCertDuration: 43800h + # Expiry window of CA certificates means a window before actual expiry in which CA certs should be rotated. + caCertExpiryWindow: 648h + # Duration of Client certificates in hours + clientCertDuration: 672h + # Expiry window of client certificates means a window before actual expiry in which client certs should be rotated. + clientCertExpiryWindow: 48h + # Duration of node certificates in hours + nodeCertDuration: 8760h + # Expiry window of node certificates means a window before actual expiry in which node certs should be rotated. + nodeCertExpiryWindow: 168h + + selfSigner: + # Additional labels to apply to the Pod of this Job. + labels: {} + + # Additional annotations to apply to the Pod of this Job. + annotations: {} + + # Affinity rules for scheduling the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity + affinity: {} + + # Node selection constraints for scheduling the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector + nodeSelector: {} + + # Taints to be tolerated by the Pod of this Job. + # https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ + tolerations: [] + + # Image Placeholder for the selfSigner utility. This will be changed once the CI workflows for the image is in place. + image: + repository: cockroachlabs-helm-charts/cockroach-self-signer-cert + tag: "1.5" + pullPolicy: IfNotPresent + credentials: {} + registry: gcr.io + # username: john_doe + # password: changeme + +networkPolicy: + enabled: false + + ingress: + # List of sources which should be able to access the CockroachDB Pods via + # gRPC port. Items in this list are combined using a logical OR operation. + # Rules for allowing inter-communication are applied automatically. + # If empty, then connections from any Pod is allowed. + grpc: [] + # - podSelector: + # matchLabels: + # app.kubernetes.io/name: my-app-django + # app.kubernetes.io/instance: my-app + + # List of sources which should be able to access the CockroachDB Pods via + # HTTP port. Items in this list are combined using a logical OR operation. + # If empty, then connections from any Pod is allowed. + http: [] + # - namespaceSelector: + # matchLabels: + # project: my-project + +# To put the admin interface behind Identity Aware Proxy (IAP) on Google Cloud Platform +# make sure to set ingress.paths: ['/*'] +iap: + enabled: false + # Create Google Cloud OAuth credentials and set client id and secret + # clientId: + # clientSecret: diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/.helmignore b/charts/intel/intel-device-plugins-operator/0.32.0/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/Chart.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/Chart.yaml new file mode 100644 index 000000000..79f56a937 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/Chart.yaml @@ -0,0 +1,13 @@ +annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel Device Plugins Operator + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-operator +apiVersion: v2 +appVersion: 0.32.0 +description: A Helm chart for Intel Device Plugins Operator for Kubernetes +icon: file://assets/icons/intel-device-plugins-operator.png +kubeVersion: '>=1.19-0' +name: intel-device-plugins-operator +type: application +version: 0.32.0 diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/LICENSE b/charts/intel/intel-device-plugins-operator/0.32.0/LICENSE new file mode 100644 index 000000000..9aa5290eb --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/LICENSE @@ -0,0 +1,14 @@ +Copyright 2023 Intel Corporation +SPDX-License-Identifier: Apache-2.0 + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/README.md b/charts/intel/intel-device-plugins-operator/0.32.0/README.md new file mode 100644 index 000000000..0fae0970e --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/README.md @@ -0,0 +1,60 @@ +# Intel Device Plugins Operator Helm Chart + +[Intel Device Plugins for Kubernetes](https://github.com/intel/intel-device-plugins-for-kubernetes) Helm charts for installing the operator. Operator installation is manadtory after which each device plugin can be installed via its own Helm chart. +## Prerequisites +- [cert-manager](https://cert-manager.io/docs/installation/helm) +- [Node Feature Discovery NFD](https://kubernetes-sigs.github.io/node-feature-discovery/master/get-started/deployment-and-usage.html) [optional] + +## Get Helm Repository Info +``` +helm repo add intel https://intel.github.io/helm-charts/ +helm repo update +``` + +You can execute `helm search repo intel` command to see pulled charts [optional]. + +## Install Helm Chart +CRDs of the device plugin operator are installed as part of the chart. + +``` +helm install device-plugin-operator intel/intel-device-plugins-operator [flags] +``` + +## Upgrade Chart +``` +helm upgrade device-plugin-operator intel/intel-device-plugins-operator [flags] +``` +CRDs are not upgraded. + +## Uninstall Chart +``` +helm uninstall device-plugin-operator +``` +CRDs are not uninstalled. + +## Configuration +See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments: + +```console +helm show values intel/intel-device-plugins-operator +``` + +You may also run `helm show values` on this chart's dependencies for additional options. + +|parameter| value | +|---------|-----------| +| `manager.image.hub` | `intel` | +| `manager.image.tag` | `` | +| `manager.devices` | `` | +| `privateRegistry.registryUrl` | `` | +| `privateRegistry.registryUser` | `` | +| `privateRegistry.registrySecret` | `` | +| `pullPolicy` | `IfNotPresent` | + +Defining `manager.devices` with a name-bool dictionary allows enabling only certain devices. The following will enable only fpga and gpu devices: +``` +manager: + devices: + fpga: true + gpu: true +``` diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dlbdeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dlbdeviceplugins.yaml new file mode 100644 index 000000000..bfd11bfde --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dlbdeviceplugins.yaml @@ -0,0 +1,190 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: dlbdeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: DlbDevicePlugin + listKind: DlbDevicePluginList + plural: dlbdeviceplugins + singular: dlbdeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + DlbDevicePlugin is the Schema for the dlbdeviceplugins API. It represents + the DLB device plugin responsible for advertising Intel DLB hardware resources to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: DlbDevicePluginSpec defines the desired state of DlbDevicePlugin. + properties: + image: + description: Image is a container image with DLB device plugin executable. + type: string + initImage: + description: InitImage is a container image with a script that initializes + devices. + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: DlbDevicePluginStatus defines the observed state of DlbDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dsadeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dsadeviceplugins.yaml new file mode 100644 index 000000000..f964961fa --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_dsadeviceplugins.yaml @@ -0,0 +1,200 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: dsadeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: DsaDevicePlugin + listKind: DsaDevicePluginList + plural: dsadeviceplugins + singular: dsadeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + DsaDevicePlugin is the Schema for the dsadeviceplugins API. It represents + the DSA device plugin responsible for advertising Intel DSA hardware resources to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: DsaDevicePluginSpec defines the desired state of DsaDevicePlugin. + properties: + image: + description: Image is a container image with DSA device plugin executable. + type: string + initImage: + description: InitImage is an initcontainer image to configure and + enable DSA devices and workqueues with idxd-config (accel-config) + utility + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + provisioningConfig: + description: ProvisioningConfig is a ConfigMap used to pass the DSA + devices and workqueues configuration into idxd-config initcontainer. + type: string + sharedDevNum: + description: SharedDevNum is a number of containers that can share + the same DSA device. + minimum: 1 + type: integer + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: DsaDevicePluginStatus defines the observed state of DsaDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_fpgadeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_fpgadeviceplugins.yaml new file mode 100644 index 000000000..b4e6a99f2 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_fpgadeviceplugins.yaml @@ -0,0 +1,197 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: fpgadeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: FpgaDevicePlugin + listKind: FpgaDevicePluginList + plural: fpgadeviceplugins + singular: fpgadeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + FpgaDevicePlugin is the Schema for the fpgadeviceplugins API. It represents + the FPGA device plugin responsible for advertising Intel FPGA hardware resources to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: FpgaDevicePluginSpec defines the desired state of FpgaDevicePlugin. + properties: + image: + description: Image is a container image with FPGA device plugin executable. + type: string + initImage: + description: InitImage is a container image with tools used to initialize + the host before starting FPGA workloads on it. + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + mode: + description: Mode is a mode of the plugin's operation. + enum: + - af + - region + - regiondevel + type: string + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: FpgaDevicePluginStatus defines the observed state of FpgaDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_gpudeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_gpudeviceplugins.yaml new file mode 100644 index 000000000..4dd89c0f1 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_gpudeviceplugins.yaml @@ -0,0 +1,214 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: gpudeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: GpuDevicePlugin + listKind: GpuDevicePluginList + plural: gpudeviceplugins + singular: gpudeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + GpuDevicePlugin is the Schema for the gpudeviceplugins API. It represents + the GPU device plugin responsible for advertising Intel GPU hardware resources to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: GpuDevicePluginSpec defines the desired state of GpuDevicePlugin. + properties: + enableMonitoring: + description: |- + EnableMonitoring enables the monitoring resource ('i915_monitoring') + which gives access to all GPU devices on given node. Typically used with Intel XPU-Manager. + type: boolean + image: + description: Image is a container image with GPU device plugin executable. + type: string + initImage: + description: InitImage is a container image with tools (e.g., GPU + NFD source hook) installed on each node. + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + preferredAllocationPolicy: + description: |- + PreferredAllocationPolicy sets the mode of allocating GPU devices on a node. + See documentation for detailed description of the policies. Only valid when SharedDevNum > 1 is set. + Not applicable with ResourceManager. + enum: + - balanced + - packed + - none + type: string + resourceManager: + description: ResourceManager handles the fractional resource management + for multi-GPU nodes. Enable only for clusters with GPU Aware Scheduling. + type: boolean + sharedDevNum: + description: SharedDevNum is a number of containers that can share + the same GPU device. + minimum: 1 + type: integer + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: GpuDevicePluginStatus defines the observed state of GpuDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_iaadeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_iaadeviceplugins.yaml new file mode 100644 index 000000000..beb5c64a7 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_iaadeviceplugins.yaml @@ -0,0 +1,199 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: iaadeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: IaaDevicePlugin + listKind: IaaDevicePluginList + plural: iaadeviceplugins + singular: iaadeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + IaaDevicePlugin is the Schema for the iaadeviceplugins API. It represents + the IAA device plugin responsible for advertising Intel IAA hardware resources to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: IaaDevicePluginSpec defines the desired state of IaaDevicePlugin. + properties: + image: + description: Image is a container image with IAA device plugin executable. + type: string + initImage: + description: InitImage is an initcontainer image to configure and + enable IAA devices and workqueues with accel-config utility + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + provisioningConfig: + description: ProvisioningConfig is a ConfigMap used to pass the IAA + configuration into idxd initcontainer. + type: string + sharedDevNum: + description: SharedDevNum is a number of containers that can share + the same IAA device. + minimum: 1 + type: integer + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: IaaDevicePluginStatus defines the observed state of IaaDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_qatdeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_qatdeviceplugins.yaml new file mode 100644 index 000000000..a9cb80dc7 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_qatdeviceplugins.yaml @@ -0,0 +1,230 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: qatdeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: QatDevicePlugin + listKind: QatDevicePluginList + plural: qatdeviceplugins + singular: qatdeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + QatDevicePlugin is the Schema for the qatdeviceplugins API. It represents the QAT device + plugin responsible for advertising Intel QuickAssist Technology hardware resources + to the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: QatDevicePluginSpec defines the desired state of QatDevicePlugin. + properties: + dpdkDriver: + description: DpdkDriver is a DPDK device driver for configuring the + QAT device. + enum: + - igb_uio + - vfio-pci + type: string + image: + description: Image is a container image with QAT device plugin executable. + type: string + initImage: + description: InitImage is a container image with a script that initialize + devices. + type: string + kernelVfDrivers: + description: KernelVfDrivers is a list of VF device drivers for the + QuickAssist devices in the system. + items: + description: KernelVfDriver is a VF device driver for QuickAssist + devices. + enum: + - dh895xccvf + - c6xxvf + - c3xxxvf + - d15xxvf + - 4xxxvf + - 420xxvf + - c4xxxvf + type: string + type: array + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + maxNumDevices: + description: MaxNumDevices is a maximum number of QAT devices to be + provided to the QuickAssist device plugin + minimum: 1 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + preferredAllocationPolicy: + description: |- + PreferredAllocationPolicy sets the mode of allocating QAT devices on a node. + See documentation for detailed description of the policies. + enum: + - balanced + - packed + type: string + provisioningConfig: + description: ProvisioningConfig is a ConfigMap used to pass the configuration + of QAT devices into qat initcontainer. + type: string + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: QatDevicePluginStatus defines the observed state of QatDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_sgxdeviceplugins.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_sgxdeviceplugins.yaml new file mode 100644 index 000000000..33823b089 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/deviceplugin.intel.com_sgxdeviceplugins.yaml @@ -0,0 +1,201 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: sgxdeviceplugins.deviceplugin.intel.com +spec: + group: deviceplugin.intel.com + names: + kind: SgxDevicePlugin + listKind: SgxDevicePluginList + plural: sgxdeviceplugins + singular: sgxdeviceplugin + scope: Cluster + versions: + - additionalPrinterColumns: + - jsonPath: .status.desiredNumberScheduled + name: Desired + type: integer + - jsonPath: .status.numberReady + name: Ready + type: integer + - jsonPath: .spec.nodeSelector + name: Node Selector + type: string + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + name: v1 + schema: + openAPIV3Schema: + description: |- + SgxDevicePlugin is the Schema for the sgxdeviceplugins API. It represents + the SGX device plugin responsible for advertising SGX device nodes to + the kubelet. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: SgxDevicePluginSpec defines the desired state of SgxDevicePlugin. + properties: + enclaveLimit: + description: EnclaveLimit is a number of containers that can share + the same SGX enclave device. + minimum: 1 + type: integer + image: + description: Image is a container image with SGX device plugin executable. + type: string + initImage: + description: |- + InitImage is a container image with tools (i.e., SGX NFD source hook) installed on each node. + Recommendation is to leave this unset and prefer the SGX NodeFeatureRule instead. + type: string + logLevel: + description: LogLevel sets the plugin's log level. + minimum: 0 + type: integer + nodeSelector: + additionalProperties: + type: string + description: NodeSelector provides a simple way to constrain device + plugin pods to nodes with particular labels. + type: object + provisionLimit: + description: ProvisionLimit is a number of containers that can share + the same SGX provision device. + minimum: 1 + type: integer + tolerations: + description: Specialized nodes (e.g., with accelerators) can be Tainted + to make sure unwanted pods are not scheduled on them. Tolerations + can be set for the plugin pod to neutralize the Taint. + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + type: object + status: + description: SgxDevicePluginStatus defines the observed state of SgxDevicePlugin. + properties: + controlledDaemonSet: + description: ControlledDaemoSet references the DaemonSet controlled + by the operator. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: |- + If referring to a piece of an object instead of an entire object, this string + should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. + For example, if the object reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container that triggered + the event) or if no container name is specified "spec.containers[2]" (container with + index 2 in this pod). This syntax is chosen only to have some well-defined way of + referencing a part of an object. + type: string + kind: + description: |- + Kind of the referent. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + type: string + namespace: + description: |- + Namespace of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ + type: string + resourceVersion: + description: |- + Specific resourceVersion to which this reference is made, if any. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + uid: + description: |- + UID of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids + type: string + type: object + x-kubernetes-map-type: atomic + desiredNumberScheduled: + description: |- + The total number of nodes that should be running the device plugin + pod (including nodes correctly running the device plugin pod). + format: int32 + type: integer + nodeNames: + description: The list of Node names where the device plugin pods are + running. + items: + type: string + type: array + numberReady: + description: |- + The number of nodes that should be running the device plugin pod and have one + or more of the device plugin pod running and ready. + format: int32 + type: integer + required: + - desiredNumberScheduled + - numberReady + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_acceleratorfunctions.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_acceleratorfunctions.yaml new file mode 100644 index 000000000..b0bca116c --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_acceleratorfunctions.yaml @@ -0,0 +1,68 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: acceleratorfunctions.fpga.intel.com +spec: + group: fpga.intel.com + names: + kind: AcceleratorFunction + listKind: AcceleratorFunctionList + plural: acceleratorfunctions + shortNames: + - af + singular: acceleratorfunction + scope: Namespaced + versions: + - name: v2 + schema: + openAPIV3Schema: + description: |- + AcceleratorFunction is a specification for an Accelerator Function resource + provided by a FPGA-based programmable hardware accelerator. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: AcceleratorFunctionSpec contains actual specs for AcceleratorFunction. + properties: + afuId: + pattern: ^[0-9a-f]{8,40}$ + type: string + interfaceId: + pattern: ^[0-9a-f]{8,32}$ + type: string + mode: + pattern: ^af|region$ + type: string + required: + - afuId + - interfaceId + - mode + type: object + status: + description: AcceleratorFunctionStatus is an empty object used to satisfy + operator-sdk. + type: object + required: + - spec + type: object + served: true + storage: true diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_fpgaregions.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_fpgaregions.yaml new file mode 100644 index 000000000..061863672 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/crds/fpga.intel.com_fpgaregions.yaml @@ -0,0 +1,59 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.1 + name: fpgaregions.fpga.intel.com +spec: + group: fpga.intel.com + names: + kind: FpgaRegion + listKind: FpgaRegionList + plural: fpgaregions + shortNames: + - fpga + singular: fpgaregion + scope: Namespaced + versions: + - name: v2 + schema: + openAPIV3Schema: + description: |- + FpgaRegion is a specification for a FPGA region resource which can be programmed + with a bitstream. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + spec: + description: FpgaRegionSpec contains actual specs for FpgaRegion. + properties: + interfaceId: + pattern: ^[0-9a-f]{8,32}$ + type: string + required: + - interfaceId + type: object + status: + description: FpgaRegionStatus is an empty object used to satisfy operator-sdk. + type: object + required: + - spec + type: object + served: true + storage: true diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/templates/NOTES.txt b/charts/intel/intel-device-plugins-operator/0.32.0/templates/NOTES.txt new file mode 100644 index 000000000..7b8b5d604 --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/templates/NOTES.txt @@ -0,0 +1,6 @@ +Thank you for installing {{ .Chart.Name }}. + +The next step would be to install the device (plugin) specific chart. + +Friendly note about CRDs. Make sure to manually update CRDs if +they have changed. CRDs are not updated with helm by default. diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/templates/operator.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/templates/operator.yaml new file mode 100644 index 000000000..0dfb3f8fb --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/templates/operator.yaml @@ -0,0 +1,731 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: inteldeviceplugins-leader-election-role + namespace: {{ .Release.Namespace | quote }} +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - list + - watch + - create + - update + - patch + - delete +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + creationTimestamp: null + name: inteldeviceplugins-gpu-manager-role +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - list +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + creationTimestamp: null + name: inteldeviceplugins-manager-role +rules: +- apiGroups: + - "" + resources: + - nodes/proxy + verbs: + - get + - list +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - create + - delete + - get + - list + - watch +- apiGroups: + - apps + resources: + - daemonsets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create +- apiGroups: + - coordination.k8s.io + resourceNames: + - d1c7b6d5.intel.com + resources: + - leases + verbs: + - get + - update +- apiGroups: + - deviceplugin.intel.com + resources: + - dlbdeviceplugins + - dsadeviceplugins + - fpgadeviceplugins + - gpudeviceplugins + - iaadeviceplugins + - qatdeviceplugins + - sgxdeviceplugins + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - deviceplugin.intel.com + resources: + - dlbdeviceplugins/finalizers + - dsadeviceplugins/finalizers + - fpgadeviceplugins/finalizers + - gpudeviceplugins/finalizers + - iaadeviceplugins/finalizers + - qatdeviceplugins/finalizers + - sgxdeviceplugins/finalizers + verbs: + - update +- apiGroups: + - deviceplugin.intel.com + resources: + - dlbdeviceplugins/status + - dsadeviceplugins/status + - fpgadeviceplugins/status + - gpudeviceplugins/status + - iaadeviceplugins/status + - qatdeviceplugins/status + - sgxdeviceplugins/status + verbs: + - get + - patch + - update +- apiGroups: + - fpga.intel.com + resources: + - acceleratorfunctions + - fpgaregions + verbs: + - get + - list + - watch +- apiGroups: + - rbac.authorization.k8s.io + resources: + - clusterrolebindings + verbs: + - create + - delete + - get + - list + - watch +- apiGroups: + - security.openshift.io + resourceNames: + - privileged + resources: + - securitycontextconstraints + verbs: + - use +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: inteldeviceplugins-metrics-reader +rules: +- nonResourceURLs: + - /metrics + verbs: + - get +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: inteldeviceplugins-auth-role +rules: +- apiGroups: + - authentication.k8s.io + resources: + - tokenreviews + verbs: + - create +- apiGroups: + - authorization.k8s.io + resources: + - subjectaccessreviews + verbs: + - create +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: inteldeviceplugins-leader-election-rolebinding + namespace: {{ .Release.Namespace | quote }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: inteldeviceplugins-leader-election-role +subjects: +- kind: ServiceAccount + name: default + namespace: {{ .Release.Namespace | quote }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: inteldeviceplugins-manager-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: inteldeviceplugins-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: {{ .Release.Namespace | quote }} +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: inteldeviceplugins-auth-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: inteldeviceplugins-auth-role +subjects: +- kind: ServiceAccount + name: default + namespace: {{ .Release.Namespace | quote }} +--- +apiVersion: v1 +kind: Service +metadata: + labels: + control-plane: controller-manager + name: inteldeviceplugins-controller-manager-metrics-service + namespace: {{ .Release.Namespace | quote }} +spec: + ports: + - name: https + port: 8443 + targetPort: https + selector: + control-plane: controller-manager +--- +apiVersion: v1 +kind: Service +metadata: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} +spec: + ports: + - port: 443 + targetPort: 9443 + selector: + control-plane: controller-manager +--- +{{- if .Values.privateRegistry.registrySecret }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ .Release.Name }}-operator-private-registry +type: kubernetes.io/dockerconfigjson +data: + .dockerconfigjson: {{ printf "{\"auths\": {\"%s\": {\"auth\": \"%s\"}}}" .Values.privateRegistry.registryUrl (printf "%s:%s" .Values.privateRegistry.registryUser .Values.privateRegistry.registrySecret | b64enc) | b64enc }} +{{- end }} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + control-plane: controller-manager + name: inteldeviceplugins-controller-manager + namespace: {{ .Release.Namespace | quote }} +spec: + replicas: 1 + selector: + matchLabels: + control-plane: controller-manager + template: + metadata: + labels: + control-plane: controller-manager + spec: + {{- if .Values.privateRegistry.registrySecret }} + imagePullSecrets: + - name: {{ .Release.Name }}-operator-private-registry + {{- end }} + containers: + - args: + - "--metrics-bind-address=:8443" + - "--metrics-secure" + - "--health-probe-bind-address=:8081" + - "--leader-elect" + {{- if .Values.manager.devices }} + {{- range $key, $value := .Values.manager.devices }} + {{- if $value }} + - "--devices={{- $key }}" + {{- end }} + {{- end }} + {{- end }} + {{- if .Values.controllerExtraArgs }} + {{- with .Values.controllerExtraArgs }} + {{- tpl . $ | trim | nindent 8 }} + {{- end }} + {{- end }} + env: + - name: DEVICEPLUGIN_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + image: "{{ .Values.manager.image.hub }}/intel-deviceplugin-operator:{{ .Values.manager.image.tag | default .Chart.AppVersion }}" + imagePullPolicy: {{ .Values.manager.image.pullPolicy }} + livenessProbe: + httpGet: + path: /healthz + port: 8081 + initialDelaySeconds: 15 + periodSeconds: 20 + readinessProbe: + httpGet: + path: /readyz + port: 8081 + initialDelaySeconds: 5 + periodSeconds: 10 + name: manager + ports: + - containerPort: 9443 + name: webhook-server + protocol: TCP + resources: + {{- toYaml .Values.resources | nindent 10 }} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + runAsGroup: 65532 + runAsNonRoot: true + runAsUser: 65532 + seccompProfile: + type: RuntimeDefault + volumeMounts: + - mountPath: /tmp/k8s-webhook-server/serving-certs + name: cert + readOnly: true + nodeSelector: {{- .Values.nodeSelector | toYaml | nindent 8 }} + serviceAccountName: default + terminationGracePeriodSeconds: 10 + volumes: + - name: cert + secret: + defaultMode: 420 + secretName: webhook-server-cert + tolerations: {{ .Values.tolerations | toYaml | nindent 8 }} +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: inteldeviceplugins-serving-cert + namespace: {{ .Release.Namespace | quote }} +spec: + dnsNames: + - inteldeviceplugins-webhook-service.{{ .Release.Namespace }}.svc + - inteldeviceplugins-webhook-service.{{ .Release.Namespace }}.svc.cluster.local + issuerRef: + kind: Issuer + name: inteldeviceplugins-selfsigned-issuer + secretName: webhook-server-cert +--- +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: inteldeviceplugins-selfsigned-issuer + namespace: {{ .Release.Namespace | quote }} +spec: + selfSigned: {} +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: MutatingWebhookConfiguration +metadata: + annotations: + cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/inteldeviceplugins-serving-cert + name: inteldeviceplugins-mutating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-dlbdeviceplugin + failurePolicy: Fail + name: mdlbdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - dlbdeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-dsadeviceplugin + failurePolicy: Fail + name: mdsadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - dsadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-fpgadeviceplugin + failurePolicy: Fail + name: mfpgadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - fpgadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-gpudeviceplugin + failurePolicy: Fail + name: mgpudeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - gpudeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-iaadeviceplugin + failurePolicy: Fail + name: miaadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - iaadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-qatdeviceplugin + failurePolicy: Fail + name: mqatdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - qatdeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate-deviceplugin-intel-com-v1-sgxdeviceplugin + failurePolicy: Fail + name: msgxdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - sgxdeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /pods + failurePolicy: Ignore + name: fpga.mutator.webhooks.intel.com + rules: + - apiGroups: + - "" + apiVersions: + - v1 + operations: + - CREATE + resources: + - pods + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /mutate--v1-pod + failurePolicy: Ignore + name: sgx.mutator.webhooks.intel.com + reinvocationPolicy: IfNeeded + rules: + - apiGroups: + - "" + apiVersions: + - v1 + operations: + - CREATE + resources: + - pods + sideEffects: None +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + annotations: + cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/inteldeviceplugins-serving-cert + name: inteldeviceplugins-validating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-dlbdeviceplugin + failurePolicy: Fail + name: vdlbdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - dlbdeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-dsadeviceplugin + failurePolicy: Fail + name: vdsadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - dsadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-fpgadeviceplugin + failurePolicy: Fail + name: vfpgadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - fpgadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-gpudeviceplugin + failurePolicy: Fail + name: vgpudeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - gpudeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-iaadeviceplugin + failurePolicy: Fail + name: viaadeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - iaadeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-qatdeviceplugin + failurePolicy: Fail + name: vqatdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - qatdeviceplugins + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: inteldeviceplugins-webhook-service + namespace: {{ .Release.Namespace | quote }} + path: /validate-deviceplugin-intel-com-v1-sgxdeviceplugin + failurePolicy: Fail + name: vsgxdeviceplugin.kb.io + rules: + - apiGroups: + - deviceplugin.intel.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - sgxdeviceplugins + sideEffects: None diff --git a/charts/intel/intel-device-plugins-operator/0.32.0/values.yaml b/charts/intel/intel-device-plugins-operator/0.32.0/values.yaml new file mode 100644 index 000000000..0cd35585f --- /dev/null +++ b/charts/intel/intel-device-plugins-operator/0.32.0/values.yaml @@ -0,0 +1,33 @@ +nodeSelector: + kubernetes.io/arch: amd64 + +manager: + image: + hub: intel + tag: "" + pullPolicy: IfNotPresent + + # supported devices by the operator + devices: + # dlb: true + # dsa: true + # fpga: true + # gpu: true + # iaa: true + # qat: true + # sgx: true + +privateRegistry: + registryUrl: "" + registryUser: "" + registrySecret: "" + +resources: + limits: + cpu: 100m + memory: 120Mi + requests: + cpu: 100m + memory: 100Mi + +tolerations: [] diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/.helmignore b/charts/intel/intel-device-plugins-qat/0.32.0/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/Chart.yaml b/charts/intel/intel-device-plugins-qat/0.32.0/Chart.yaml new file mode 100644 index 000000000..619ac5357 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/Chart.yaml @@ -0,0 +1,13 @@ +annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel QAT Device Plugin + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-qat +apiVersion: v2 +appVersion: 0.32.0 +description: A Helm chart for Intel QAT Device Plugin +icon: file://assets/icons/intel-device-plugins-qat.png +kubeVersion: '>=1.19-0' +name: intel-device-plugins-qat +type: application +version: 0.32.0 diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/LICENSE b/charts/intel/intel-device-plugins-qat/0.32.0/LICENSE new file mode 100644 index 000000000..9aa5290eb --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/LICENSE @@ -0,0 +1,14 @@ +Copyright 2023 Intel Corporation +SPDX-License-Identifier: Apache-2.0 + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/README.md b/charts/intel/intel-device-plugins-qat/0.32.0/README.md new file mode 100644 index 000000000..04b0b5232 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/README.md @@ -0,0 +1,50 @@ +# Intel QAT Device Plugin Helm Chart + +## Get Helm Repository Info +``` +helm repo add intel https://intel.github.io/helm-charts/ +helm repo update +``` + +You can execute `helm search repo intel` command to see pulled charts [optional]. + +## Dependencies + +QAT Device Plugin depends on Node Feature Discovery (NFD). See NFD's Helm install page [here](https://kubernetes-sigs.github.io/node-feature-discovery/v0.12/deployment/helm.html?highlight=helm#deployment). If you do not want to use NFD in you cluster, you'll need to change the nodeSelector in the [values](values.yaml) file to match nodes with QAT device. + +## Install Helm Chart +``` +helm install qat-device-plugin intel/intel-device-plugins-qat [flags] +``` + +## Upgrade Chart +``` +helm upgrade qat-device-plugin intel/intel-device-plugins-qat [flags] +``` + +## Uninstall Chart +``` +helm uninstall qat-device-plugin +``` + +## Configuration +See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments: + +```console +helm show values intel/intel-device-plugins-qat +``` + +You may also run `helm show values` on this chart's dependencies for additional options. + +|parameter| value | +|---------|-----------| +| `image.hub` | `intel` | +| `image.tag` | `` | +| `initImage.hub` | `intel` | +| `initImage.tag` | `` | +| `dpdkDriver` | `vfio-pci` | +| `kernelVfDrivers` | `4xxxvf`, `420xxvf` | +| `maxNumDevices` | `128` | +| `logLevel` | `4` | +| `nodeFeatureRule` | `true` | +| `tolerations` | `` | diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/questions.yaml b/charts/intel/intel-device-plugins-qat/0.32.0/questions.yaml new file mode 100644 index 000000000..74461ffa8 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/questions.yaml @@ -0,0 +1,6 @@ +questions: +- variable: nodeFeatureRule + default: false + type: boolean + label: Enable Node Feature Discovery feature labels + description: "When Node Feature Discovery (NFD) is deployed, enable QAT node labeling using NFD feature rules." diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/templates/NOTES.txt b/charts/intel/intel-device-plugins-qat/0.32.0/templates/NOTES.txt new file mode 100644 index 000000000..c5615c64c --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/templates/NOTES.txt @@ -0,0 +1 @@ +Thank you for installing {{ .Chart.Name }}. diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/templates/qat.yaml b/charts/intel/intel-device-plugins-qat/0.32.0/templates/qat.yaml new file mode 100644 index 000000000..b569f3d28 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/templates/qat.yaml @@ -0,0 +1,53 @@ +{{- /* +based on +deployments/operator/samples/deviceplugin_v1_qatdeviceplugin.yaml +*/}} + +apiVersion: deviceplugin.intel.com/v1 +kind: QatDevicePlugin +metadata: + name: {{ .Values.name }} + annotations: {{ toYaml .Values.annotations | nindent 4 }} +spec: + image: "{{ .Values.image.hub }}/intel-qat-plugin:{{ .Values.image.tag | default .Chart.AppVersion }}" + initImage: "{{ .Values.initImage.hub }}/intel-qat-initcontainer:{{ .Values.initImage.tag | default .Chart.AppVersion }}" + dpdkDriver: {{ .Values.dpdkDriver }} + kernelVfDrivers: + {{- range .Values.kernelVfDrivers }} + - {{ . }} + {{- end }} + maxNumDevices: {{ .Values.maxNumDevices }} + logLevel: {{ .Values.logLevel }} + nodeSelector: {{ .Values.nodeSelector | toYaml | nindent 4 }} + tolerations: {{- .Values.tolerations | toYaml | nindent 4 }} + +--- +{{ if eq .Values.nodeFeatureRule true }} +apiVersion: nfd.k8s-sigs.io/v1alpha1 +kind: NodeFeatureRule +metadata: + name: intel-dp-qat-device +spec: + rules: + - name: "intel.qat" + labels: + "intel.feature.node.kubernetes.io/qat": "true" + matchFeatures: + - feature: pci.device + matchExpressions: + vendor: {op: In, value: ["8086"]} + device: {op: In, value: ["37c8", "4940", "4942", "4944", "4946"]} + class: {op: In, value: ["0b40"]} + - feature: kernel.loadedmodule + matchExpressions: + intel_qat: {op: Exists} + matchAny: + - matchFeatures: + - feature: kernel.loadedmodule + matchExpressions: + vfio_pci: {op: Exists} + - matchFeatures: + - feature: kernel.enabledmodule + matchExpressions: + vfio-pci: {op: Exists} +{{ end }} diff --git a/charts/intel/intel-device-plugins-qat/0.32.0/values.yaml b/charts/intel/intel-device-plugins-qat/0.32.0/values.yaml new file mode 100644 index 000000000..98ca374e4 --- /dev/null +++ b/charts/intel/intel-device-plugins-qat/0.32.0/values.yaml @@ -0,0 +1,23 @@ +name: qatdeviceplugin-sample + +image: + hub: intel + tag: "" + +initImage: + hub: intel + tag: "" + +dpdkDriver: vfio-pci +kernelVfDrivers: + - 4xxxvf + - 420xxvf +maxNumDevices: 128 +logLevel: 4 + +nodeSelector: + intel.feature.node.kubernetes.io/qat: 'true' + +tolerations: + +nodeFeatureRule: true diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/.helmignore b/charts/intel/intel-device-plugins-sgx/0.32.0/.helmignore new file mode 100644 index 000000000..0e8a0eb36 --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/Chart.yaml b/charts/intel/intel-device-plugins-sgx/0.32.0/Chart.yaml new file mode 100644 index 000000000..caa52fce1 --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/Chart.yaml @@ -0,0 +1,13 @@ +annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel SGX Device Plugin + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-sgx +apiVersion: v2 +appVersion: 0.32.0 +description: A Helm chart for Intel SGX Device Plugin +icon: file://assets/icons/intel-device-plugins-sgx.png +kubeVersion: '>=1.19-0' +name: intel-device-plugins-sgx +type: application +version: 0.32.0 diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/LICENSE b/charts/intel/intel-device-plugins-sgx/0.32.0/LICENSE new file mode 100644 index 000000000..9aa5290eb --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/LICENSE @@ -0,0 +1,14 @@ +Copyright 2023 Intel Corporation +SPDX-License-Identifier: Apache-2.0 + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/README.md b/charts/intel/intel-device-plugins-sgx/0.32.0/README.md new file mode 100644 index 000000000..0cbd391ac --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/README.md @@ -0,0 +1,40 @@ +# Intel SGX Device Plugin Helm Chart + +## Get Helm Repository Info +``` +helm repo add intel https://intel.github.io/helm-charts/ +helm repo update +``` + +You can execute `helm search repo intel` command to see pulled charts [optional]. + +## Install Helm Chart +``` +helm install sgx-device-plugin intel/intel-device-plugins-sgx [flags] +``` +## Upgrade Chart +``` +helm upgrade sgx-device-plugin intel/intel-device-plugins-sgx [flags] +``` + +## Uninstall Chart +``` +helm uninstall sgx-device-plugin +``` + +## Configuration +See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments: + +```console +helm show values intel/intel-device-plugins-sgx +``` + +You may also run `helm show values` on this chart's dependencies for additional options. + +|parameter| value | +|---------|-----------| +| `image.hub` | `intel` | +| `image.tag` | `` | +| `enclaveLimit` | `110` | +| `provisionLimit` | `110` | +| `logLevel` | `4` | \ No newline at end of file diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/questions.yaml b/charts/intel/intel-device-plugins-sgx/0.32.0/questions.yaml new file mode 100644 index 000000000..402e94820 --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/questions.yaml @@ -0,0 +1,6 @@ +questions: +- variable: nodeFeatureRule + default: false + type: boolean + label: Enable Node Feature Discovery feature labels + description: "When Node Feature Discovery (NFD) is deployed, enable SGX node labeling using NFD feature rules." diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/templates/sgx.yaml b/charts/intel/intel-device-plugins-sgx/0.32.0/templates/sgx.yaml new file mode 100644 index 000000000..5fde596ad --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/templates/sgx.yaml @@ -0,0 +1,43 @@ +{{- /* +based on +deployments/operator/samples/deviceplugin_v1_sgxdeviceplugin.yaml +*/}} + +apiVersion: deviceplugin.intel.com/v1 +kind: SgxDevicePlugin +metadata: + name: {{ .Values.name }} + annotations: {{ toYaml .Values.annotations | nindent 4 }} +spec: + image: "{{ .Values.image.hub }}/intel-sgx-plugin:{{ .Values.image.tag | default .Chart.AppVersion }}" + enclaveLimit: {{ .Values.enclaveLimit }} + provisionLimit: {{ .Values.provisionLimit }} + logLevel: {{ .Values.logLevel }} + nodeSelector: {{- .Values.nodeSelector | toYaml | nindent 4 }} + tolerations: {{- .Values.tolerations | toYaml | nindent 4 }} + +--- +{{ if eq .Values.nodeFeatureRule true }} +apiVersion: nfd.k8s-sigs.io/v1alpha1 +kind: NodeFeatureRule +metadata: + name: intel-dp-sgx-device +spec: + rules: + - name: "intel.sgx" + labels: + "intel.feature.node.kubernetes.io/sgx": "true" + extendedResources: + sgx.intel.com/epc: "@cpu.security.sgx.epc" + matchFeatures: + - feature: cpu.cpuid + matchExpressions: + SGX: {op: Exists} + SGXLC: {op: Exists} + - feature: cpu.security + matchExpressions: + sgx.enabled: {op: IsTrue} + - feature: kernel.config + matchExpressions: + X86_SGX: {op: Exists} +{{ end }} diff --git a/charts/intel/intel-device-plugins-sgx/0.32.0/values.yaml b/charts/intel/intel-device-plugins-sgx/0.32.0/values.yaml new file mode 100644 index 000000000..5da974c99 --- /dev/null +++ b/charts/intel/intel-device-plugins-sgx/0.32.0/values.yaml @@ -0,0 +1,16 @@ +name: sgxdeviceplugin-sample + +image: + hub: intel + tag: "" + +enclaveLimit: 110 +provisionLimit: 110 +logLevel: 4 + +nodeSelector: + intel.feature.node.kubernetes.io/sgx: 'true' + +tolerations: + +nodeFeatureRule: true \ No newline at end of file diff --git a/index.yaml b/index.yaml index 942734a52..2966ddef7 100644 --- a/index.yaml +++ b/index.yaml @@ -6612,6 +6612,28 @@ entries: - assets/cloudcasa/cloudcasa-3.4.1.tgz version: 3.4.1 cockroachdb: + - annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: CockroachDB + catalog.cattle.io/kube-version: '>=1.8-0' + catalog.cattle.io/release-name: cockroachdb + apiVersion: v1 + appVersion: 24.3.4 + created: "2025-02-01T00:01:49.027707406Z" + description: CockroachDB is a scalable, survivable, strongly-consistent SQL database. + digest: 2907478e8dd26f3845ac03b175178ff0a1f1986115f26550f72f5d3c92bc6d9c + home: https://www.cockroachlabs.com + icon: file://assets/icons/cockroachdb.png + kubeVersion: '>=1.8-0' + maintainers: + - email: helm-charts@cockroachlabs.com + name: cockroachlabs + name: cockroachdb + sources: + - https://github.com/cockroachdb/cockroach + urls: + - assets/cockroach-labs/cockroachdb-15.0.6.tgz + version: 15.0.6 - annotations: catalog.cattle.io/certified: partner catalog.cattle.io/display-name: CockroachDB @@ -18872,6 +18894,23 @@ entries: - assets/instana/instana-agent-1.2.60.tgz version: 1.2.60 intel-device-plugins-operator: + - annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel Device Plugins Operator + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-operator + apiVersion: v2 + appVersion: 0.32.0 + created: "2025-02-01T00:01:50.031539943Z" + description: A Helm chart for Intel Device Plugins Operator for Kubernetes + digest: 34fa2e0464af3ab4307475b456017902fa4fa2590d957ab2d17f39127272ca5a + icon: file://assets/icons/intel-device-plugins-operator.png + kubeVersion: '>=1.19-0' + name: intel-device-plugins-operator + type: application + urls: + - assets/intel/intel-device-plugins-operator-0.32.0.tgz + version: 0.32.0 - annotations: catalog.cattle.io/certified: partner catalog.cattle.io/display-name: Intel Device Plugins Operator @@ -19003,6 +19042,23 @@ entries: - assets/intel/intel-device-plugins-operator-0.26.1.tgz version: 0.26.1 intel-device-plugins-qat: + - annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel QAT Device Plugin + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-qat + apiVersion: v2 + appVersion: 0.32.0 + created: "2025-02-01T00:01:50.033435395Z" + description: A Helm chart for Intel QAT Device Plugin + digest: 40e8891ee8cd10bac8ddf39b52c305cc1d921fb2840e5ce62e38c331a5cb21f0 + icon: file://assets/icons/intel-device-plugins-qat.png + kubeVersion: '>=1.19-0' + name: intel-device-plugins-qat + type: application + urls: + - assets/intel/intel-device-plugins-qat-0.32.0.tgz + version: 0.32.0 - annotations: catalog.cattle.io/certified: partner catalog.cattle.io/display-name: Intel QAT Device Plugin @@ -19134,6 +19190,23 @@ entries: - assets/intel/intel-device-plugins-qat-0.26.1.tgz version: 0.26.1 intel-device-plugins-sgx: + - annotations: + catalog.cattle.io/certified: partner + catalog.cattle.io/display-name: Intel SGX Device Plugin + catalog.cattle.io/kube-version: '>=1.19-0' + catalog.cattle.io/release-name: intel-device-plugins-sgx + apiVersion: v2 + appVersion: 0.32.0 + created: "2025-02-01T00:01:50.03509248Z" + description: A Helm chart for Intel SGX Device Plugin + digest: 3b51b3cf5ae1388c3a132cb35d4c44eb479c4e18182e9ec2de07f5c02a3e6a22 + icon: file://assets/icons/intel-device-plugins-sgx.png + kubeVersion: '>=1.19-0' + name: intel-device-plugins-sgx + type: application + urls: + - assets/intel/intel-device-plugins-sgx-0.32.0.tgz + version: 0.32.0 - annotations: catalog.cattle.io/certified: partner catalog.cattle.io/display-name: Intel SGX Device Plugin @@ -49053,4 +49126,4 @@ entries: urls: - assets/netfoundry/ziti-host-1.5.1.tgz version: 1.5.1 -generated: "2025-01-31T00:01:45.881547877Z" +generated: "2025-02-01T00:01:48.430240952Z"