From 9c13e046a988ce352a36ac8ced02acc79c8d1ac9 Mon Sep 17 00:00:00 2001 From: Uwe Krueger Date: Mon, 12 Jun 2023 12:47:51 +0200 Subject: [PATCH] incorporate spec changes (#56) - new access method for helm repositories - component descriptor normalization according to RFC8785 (JCS) - version mapping for mapping OCM versions to OCI tags. **What this PR does / why we need it**: **Which issue(s) this PR fixes**: Fixes # **Special notes for your reviewer**: **Release note**: ```feature user ``` --- doc/appendix/A/OCIRegistry/README.md | 16 ++ doc/appendix/B/README.md | 5 +- doc/appendix/B/helm.md | 42 +++++ .../B/{ociArtefact.md => ociArtifact.md} | 0 doc/appendix/E/README.md | 18 +-- .../E/{ociArtefact.md => ociArtifact.md} | 0 doc/glossary.md | 10 +- .../componentdescriptor_normalization.md | 149 ++++++++++-------- doc/specification/formats/normalization.md | 2 +- 9 files changed, 167 insertions(+), 75 deletions(-) create mode 100644 doc/appendix/B/helm.md rename doc/appendix/B/{ociArtefact.md => ociArtifact.md} (100%) rename doc/appendix/E/{ociArtefact.md => ociArtifact.md} (100%) diff --git a/doc/appendix/A/OCIRegistry/README.md b/doc/appendix/A/OCIRegistry/README.md index c312b8c..9ec2c94 100644 --- a/doc/appendix/A/OCIRegistry/README.md +++ b/doc/appendix/A/OCIRegistry/README.md @@ -89,6 +89,22 @@ access specification. + +## Version Mapping + +The Open Component Model supports version names according to [semantic versioning](https://semver.org/). +The tags used to represent versions in the [OCI specification](https://github.com/opencontainers/distribution-spec/blob/main/spec.md#pulling-manifests) do not allow to directly use semantic version names as tags, becase the plus (`+`) character is not supported. Therefore, the open component model version names have to be mapped +to OCI-compliant tag names. + +The followinmg mapping for version is used, here: +- the optional plus `+` character used to attach build information in semantic versions is mapped to the sequence (`.build-`) + +Mapping tags back to versions uses the following mappings: +- the last character sequence (`.build-`) is mapped to a plus (`+`) character. + +This way the formal parts of a pre-release of semantic version (separated by dots) are kept +unchanged. The build/metadata suffix of a semantic version is just added as optional last pre-release part, where the prefix `build-` is used to indicate its meaning as metadata suffix. + ## Blob Mappings Local blobs with an OCI artifact media type will implicitly be mapped to a regular diff --git a/doc/appendix/B/README.md b/doc/appendix/B/README.md index e75625a..728136f 100644 --- a/doc/appendix/B/README.md +++ b/doc/appendix/B/README.md @@ -12,7 +12,8 @@ required to identity the blob and its location. The following access method types are centrally defined: - [localBlob](localBlob.md) an artifact stored along with the component version -- [ociArtifact](ociArtefact.md) an artifact in a repository of an OCI registry +- [ociArtifact](ociArtifact.md) an artifact in a repository of an OCI registry - [ociBlob](ociBlob.md) a blob in a repository of an OCI registry -- [gitHub](gitHub.md) a GitHub commit +- [helm](helm.md) a Helm chart stored in a Helm Repository +- [gitHub](gitHub.md) a commit in a GitHub-based Git repository - [s3](s3.md) a blob stored in an AWS S3 bucket diff --git a/doc/appendix/B/helm.md b/doc/appendix/B/helm.md new file mode 100644 index 0000000..a2db8f5 --- /dev/null +++ b/doc/appendix/B/helm.md @@ -0,0 +1,42 @@ + +# Access Method `helm` - Helm Repository Access + + +### Synopsis +``` +type: helm/v1 +``` + +Provided blobs use the following media type: attribute `application/vnd.cncf.helm.chart.content.v1.tar+gzip` + +### Description +This method implements the access of a Helm chart stored in a Helm chart repository. + +Supported specification version is `v1` + +### Specification Versions + +#### Version `v1` + +The type specific specification fields are: + +- **`helmRepository`** *string* + + Helm repository URL. + +- **`helmChart`** *string* + + The name of the Helm chart and its version separated by a colon. + +- **`caCert`** *string* + + An optional TLS root certificate. + +- **`keyring`** *string* + + An optional keyring used to verify the chart. + +### Related + +Helm charts can be stored as OCI artifacts in OCI registries, also. The access to those +helm charts is described by the access method [`ociArtifact](ociArtifact.md). \ No newline at end of file diff --git a/doc/appendix/B/ociArtefact.md b/doc/appendix/B/ociArtifact.md similarity index 100% rename from doc/appendix/B/ociArtefact.md rename to doc/appendix/B/ociArtifact.md diff --git a/doc/appendix/E/README.md b/doc/appendix/E/README.md index e7e3f1b..547bc6a 100644 --- a/doc/appendix/E/README.md +++ b/doc/appendix/E/README.md @@ -2,14 +2,14 @@ The following [artifact types](../../specification/formats/types.md#artifact-types) are centrally defined: -| TYPE | VALUE | DESCRIPTION | -| ------------- | ------------------------------- | ----------------------------- | -| OCI Artifact | [`ociArtifact`](ociArtefact.md) | A generic OCI artifact following the [open containers image specification](https://github.com/opencontainers/image-spec/blob/main/spec.md) | -| OCI Image | [`ociImage`](ociImage.md) | An OCI image or image list | -| Helm Chart | [`helmChart`](helmChart.md) | A Helm Chart stored as OCI artifact or as tar blob (`mediaType` tar) | -| Blob | [`blob`](blob.md) | Any anonymous untyped blob data | -| Filesystem | [`fileSystem`](fileSystem.md) | Some filesystem content (tar, tgz) | -| GitOps | [`gitOpsTemplate`](gitOpsTemplate.md) | Filesystem content (tar, tgz) used as GitOps Template, e.g. to set up a git repo used for continuous deployment (for example flux) | +| TYPE | VALUE | DESCRIPTION | +|--------------------|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------| +| OCI Artifact | [`ociArtifact`](ociArtifact.md) | A generic OCI artifact following the [open containers image specification](https://github.com/opencontainers/image-spec/blob/main/spec.md) | +| OCI Image | [`ociImage`](ociImage.md) | An OCI image or image list | +| Helm Chart | [`helmChart`](helmChart.md) | A Helm Chart stored as OCI artifact or as tar blob (`mediaType` tar) | +| Blob | [`blob`](blob.md) | Any anonymous untyped blob data | +| Filesystem Content | [`filesystem` `directoryTree`](fileSystem.md) | Some filesystem content (typically provided by a *tar* or *tgz* archive). The blob's mime type specifies the concrete format. | +| GitOps | [`gitOpsTemplate`](gitOpsTemplate.md) | Filesystem content (tar, tgz) used as GitOps Template, e.g. to set up a git repo used for continuous deployment (for example flux) | For centrally defined artifact types, there might be special support in the standard OCM library and tool set. For example, there is a dedicated downloader @@ -39,7 +39,7 @@ must deliver such a blob, also. This basically means, whenever a new resource type is supported, corresponding blob formats must be defined for this type. Type-agnostic access types, like [`localBlob`](../B/localBlob.md) or [`ociBlob`](../B/ociBlob.md) just deal with those blobs, they never need to know anything about their internal -format. But specific access methods, e.g. the [`ociArtecat`](../B/ociArtifact.md) +format. But specific access methods, e.g. the [`ociArtifact`](../B/ociArtifact.md) method may provide dedicated blob formats. These blob formats may depend on the combination of artifact type and access type. diff --git a/doc/appendix/E/ociArtefact.md b/doc/appendix/E/ociArtifact.md similarity index 100% rename from doc/appendix/E/ociArtefact.md rename to doc/appendix/E/ociArtifact.md diff --git a/doc/glossary.md b/doc/glossary.md index d6fa9b5..c896afa 100644 --- a/doc/glossary.md +++ b/doc/glossary.md @@ -1,6 +1,6 @@ # Glossary -[A](#a)   B   [C](#c)   [D](#d)   [E](#e)   F   G   H   [I](#i)   J   K   [L](#l)   [M](#m)   [N](#n)   [O](#o)   [P](#p)   Q   [R](#r)   [S](#s)   [T](#t)   U   V   W   X   Y   Z +[A](#a)   B   [C](#c)   [D](#d)   [E](#e)   F   G   [H](#h)   [I](#i)   J   K   [L](#l)   [M](#m)   [N](#n)   [O](#o)   [P](#p)   Q   [R](#r)   [S](#s)   [T](#t)   U   V   W   X   Y   Z ## A @@ -106,6 +106,14 @@ behaviour of such elements. additional parts of the identity of an [element](#elemid) of a [component version](#compvers) described as dedicated attributes in a [component descriptor](#compdesc). +## H + +#### [Helm Chart](appendix/B/helm.md) +A set of files describing the deplyoment of a Kubernetes application using +the [Helm](https://helm.sh/) deployment mechanism. + +see [element identity](#elemid), [component identity](#compid), [component version identity](#compversid) + ## I #### Identity diff --git a/doc/specification/formats/componentdescriptor_normalization.md b/doc/specification/formats/componentdescriptor_normalization.md index 753030f..946f192 100644 --- a/doc/specification/formats/componentdescriptor_normalization.md +++ b/doc/specification/formats/componentdescriptor_normalization.md @@ -22,7 +22,92 @@ normalizations. - `jsonNormalisationV2`: This is the new format. which is independent of the chosen representation format of the component descriptor. -## Generic Normalization format +The normalization process is divided into two steps: + +- *extraction of the signature relevant information from the component descriptor* + + The result is basically a JSON object, which decsribed the relevant information. + +- *normalization of the resulting JSON object* + + Here, the object is serialized to a unique and reproducable byte sequence, which is finally used to determine the digest. + + There are two such normalization methods: + - `jsonNormalisationV1` uses an [OCM specific representation](#generic-normalization-format) of the JSON object. + - `jsonNormalisationV2` uses a standard scheme according to [RFC8785 (JCS)](https://www.rfc-editor.org/rfc/rfc8785). + + +## `jsonNormalisationV1` vs `jsonNormalisationV2` + +The `JsonNormalisationV1` serialization format is based on the serialization +format of the component descriptor. It uses an appropriate JSON object containing the relevant fields as contained in the component descriptors's serialization. The format version fields are included. Therefore, the normalized form is depending on the chosen serialization format. Changing this format version would result in different digests. The resulting JSON object is serialized with the [OCM specific scheme](#generic-normalization-format) + +`JsonNormalisationV2` strictly uses only the relevant component descriptor +information according to the field specification shown below. It is independent of the serialization format used to store the component decsriptor in some storage backend. Therefore, the calculated digest is finally independent of the serialization format chosen for storing the component descriptor in a storage backend. + +Additionally, it uses the JCS scheme for uniquely serializing the resulting +JSON object. + +## Relevant information in Component Descriptors + +A component descriptor contains static information and +information, which may change over time (for example, the access methods +specifications might be changed during a transport step). A digest should be +stable even after a transport and therefore should only hash static +information. Therefore, a component descriptor is transformed to format +containing only immutable fields, which are finally relevant for the signing +process to assure the data integrity. + +Relevant fields and their mapping to the normalized data structure for `JsonNormalisationV2`: +- Component Name: mapped to `component.name` +- Component Version: mapped to `component.version` +- Component Labels: mapped to `component.labels` (see [Labels](#labels)]) +- Component Provider: mapped to `component.provider` +- Resources: mapped to `component.resources`, always empty list enforced, without the source references (see [Labels](#labels)] and [Access Methods](#access-methods)]) +- Sources: mapped to `component.sources`, always empty list enforced, (see [Labels](#labels)] and [Access Methods](#access-methods)]) +- References: mapped to `component.references`, always empty list enforced, (see [Labels](#labels)]) + +### Access Methods + +Access method specifications are completely ignored. +A resource/source is ignored, if the access method type is `none`. + +## Labels + +Labels are removed before signing but can be marked with a special boolean +property `signing`. This property indicates that the label should be +signing-relevant and therefore part of the digest. As a consequence such +labels cannot be changed anymore during the lifecycle of a component version +any may only describe static information. +The structure of signing-relevant labels is preserved from the component +descriptor version `v2`. + +Example: + +```yaml +labels: +- name: label1 + value: foo +- name: label2 + value: bar + signing: true +``` + +`label1` will be excluded from the digest, `label2` will be included. +The label value is taken as it is, preserving a potentially deeply nested structure. + +## Exclude Resources from Normalization/Signing + +If a resource should not be part of the normalization and later signing, the resource needs a special digest in the following format: + +```yaml +digest: + hashAlgorithm: NO-DIGEST + normalisationAlgorithm: EXCLUDE-FROM-SIGNATURE + value: NO-DIGEST +``` + +## Generic Normalization Format The generic format is based on a data structure consisting of dictionaries, lists and simple values (like strings and integers). @@ -220,64 +305,4 @@ are all normalized to: ```json [] -``` - -## Relevant information in Component Descriptors. - -A component descriptor contains signature relevant information and -information, which may change. For example, the access methods specifications -might be changed during atransport step. - -Relevant fields and their mapping to the normalized data structure for `JsonNormalisationV2`: -- Component Name: mapped to `component.name` -- Component Version: mapped to `component.version` -- Component Labels: mapped to `component.labels` (see [Labels](#labels)]) -- Component Provider: mapped to `component.provider` -- Resources: mapped to `component.resources`, always empty list enforced, without the source references (see [Labels](#labels)] and [Access Methods](#access-methods)]) -- Sources: mapped to `component.sources`, always empty list enforced, (see [Labels](#labels)] and [Access Methods](#access-methods)]) -- References: mapped to `component.references`, always empty list enforced, (see [Labels](#labels)]) - -### Access Methods - -Access method specifications are completely ignored. -A resource/source is ignored, if the access method type is `none`. - -## Labels - -Labels are removed before signing but can be marked with a special boolean -property `signing` not to be removed and thus be part of the signature. -The structure of signing-relevant labels is preserved from the component -descriptor version `v2`. - -Example: - -```yaml -labels: -- name: label1 - value: foo -- name: label2 - value: bar - signing: true -``` - -`label1` will be excluded from the signature, `label2` will be included. -The label values is takes as it is, preserving a potentially deep structure. - -## Exclude Resource from Normalisation/Signing - -If a resource should not be part of the normalisation and later signing, the resource needs a special digest in the following format: - -```yaml -digest: - hashAlgorithm: NO-DIGEST - normalisationAlgorithm: EXCLUDE-FROM-SIGNATURE - value: NO-DIGEST -``` - -# `jsonNormalisationV1` vs `jsonNormalisationV2` - -The `JsonNormalisationV1` serialization format is based on the serialization -format of the component descriptor. The format version fields are included - -`JsonNormalisationV2` strictly uses only the relevant component descriptor -information according to the field specification shown above. +``` \ No newline at end of file diff --git a/doc/specification/formats/normalization.md b/doc/specification/formats/normalization.md index 911536a..7dbaa1c 100644 --- a/doc/specification/formats/normalization.md +++ b/doc/specification/formats/normalization.md @@ -5,7 +5,7 @@ content of described artifacts must be incorporated. The metadata of a component version is described by a [component-descriptor](../../specification/elements/README.md#component-descriptor). It contains volatile information, also, e.g. the artifact access specification, which might change during [transports](../../introduction/transports.md). Therefore, to calculate a digest, the component descriptor has to be transformed -to an immutable technical reprsentation containing only signature relevant information. THis process is called [*component descriptor normalization*](componentdescriptor_normalization.md). +to an immutable technical representation containing only static signature relevant information. This process is called [*component descriptor normalization*](componentdescriptor_normalization.md). To cover the content of the artifacts described by a component version, a digest for the artifact content has to be calculated, also, and incorporated into the