Skip to content

Commit

Permalink
incorporate spec changes (#56)
Browse files Browse the repository at this point in the history
- new access method for helm repositories
- component descriptor normalization according to RFC8785 (JCS)
- version mapping for mapping OCM versions to OCI tags.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes**:
Fixes #

**Special notes for your reviewer**:

**Release note**:
<!--  Write your release note:
1. Enter your release note in the below block.
2. If no release note is required, just write "NONE" within the block.

Format of block header: <category> <target_group>
Possible values:
- category:       breaking|feature|bugfix|doc|other
- target_group:   user|operator|developer|dependency
-->
```feature user

```
  • Loading branch information
mandelsoft authored Jun 12, 2023
1 parent 9fc863b commit 9c13e04
Show file tree
Hide file tree
Showing 9 changed files with 167 additions and 75 deletions.
16 changes: 16 additions & 0 deletions doc/appendix/A/OCIRegistry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,22 @@ access specification.

</div>


## Version Mapping

The Open Component Model supports version names according to [semantic versioning](https://semver.org/).
The tags used to represent versions in the [OCI specification](https://github.com/opencontainers/distribution-spec/blob/main/spec.md#pulling-manifests) do not allow to directly use semantic version names as tags, becase the plus (`+`) character is not supported. Therefore, the open component model version names have to be mapped
to OCI-compliant tag names.

The followinmg mapping for version is used, here:
- the optional plus `+` character used to attach build information in semantic versions is mapped to the sequence (`.build-`)

Mapping tags back to versions uses the following mappings:
- the last character sequence (`.build-`) is mapped to a plus (`+`) character.

This way the formal parts of a pre-release of semantic version (separated by dots) are kept
unchanged. The build/metadata suffix of a semantic version is just added as optional last pre-release part, where the prefix `build-` is used to indicate its meaning as metadata suffix.

## Blob Mappings

Local blobs with an OCI artifact media type will implicitly be mapped to a regular
Expand Down
5 changes: 3 additions & 2 deletions doc/appendix/B/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ required to identity the blob and its location.
The following access method types are centrally defined:

- [localBlob](localBlob.md) an artifact stored along with the component version
- [ociArtifact](ociArtefact.md) an artifact in a repository of an OCI registry
- [ociArtifact](ociArtifact.md) an artifact in a repository of an OCI registry
- [ociBlob](ociBlob.md) a blob in a repository of an OCI registry
- [gitHub](gitHub.md) a GitHub commit
- [helm](helm.md) a Helm chart stored in a Helm Repository
- [gitHub](gitHub.md) a commit in a GitHub-based Git repository
- [s3](s3.md) a blob stored in an AWS S3 bucket
42 changes: 42 additions & 0 deletions doc/appendix/B/helm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@

# Access Method `helm` - Helm Repository Access


### Synopsis
```
type: helm/v1
```

Provided blobs use the following media type: attribute `application/vnd.cncf.helm.chart.content.v1.tar+gzip`

### Description
This method implements the access of a Helm chart stored in a Helm chart repository.

Supported specification version is `v1`

### Specification Versions

#### Version `v1`

The type specific specification fields are:

- **`helmRepository`** *string*

Helm repository URL.

- **`helmChart`** *string*

The name of the Helm chart and its version separated by a colon.

- **`caCert`** *string*

An optional TLS root certificate.

- **`keyring`** *string*

An optional keyring used to verify the chart.

### Related

Helm charts can be stored as OCI artifacts in OCI registries, also. The access to those
helm charts is described by the access method [`ociArtifact](ociArtifact.md).
File renamed without changes.
18 changes: 9 additions & 9 deletions doc/appendix/E/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@

The following [artifact types](../../specification/formats/types.md#artifact-types) are centrally defined:

| TYPE | VALUE | DESCRIPTION |
| ------------- | ------------------------------- | ----------------------------- |
| OCI Artifact | [`ociArtifact`](ociArtefact.md) | A generic OCI artifact following the [open containers image specification](https://github.com/opencontainers/image-spec/blob/main/spec.md) |
| OCI Image | [`ociImage`](ociImage.md) | An OCI image or image list |
| Helm Chart | [`helmChart`](helmChart.md) | A Helm Chart stored as OCI artifact or as tar blob (`mediaType` tar) |
| Blob | [`blob`](blob.md) | Any anonymous untyped blob data |
| Filesystem | [`fileSystem`](fileSystem.md) | Some filesystem content (tar, tgz) |
| GitOps | [`gitOpsTemplate`](gitOpsTemplate.md) | Filesystem content (tar, tgz) used as GitOps Template, e.g. to set up a git repo used for continuous deployment (for example flux) |
| TYPE | VALUE | DESCRIPTION |
|--------------------|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
| OCI Artifact | [`ociArtifact`](ociArtifact.md) | A generic OCI artifact following the [open containers image specification](https://github.com/opencontainers/image-spec/blob/main/spec.md) |
| OCI Image | [`ociImage`](ociImage.md) | An OCI image or image list |
| Helm Chart | [`helmChart`](helmChart.md) | A Helm Chart stored as OCI artifact or as tar blob (`mediaType` tar) |
| Blob | [`blob`](blob.md) | Any anonymous untyped blob data |
| Filesystem Content | [`filesystem` `directoryTree`](fileSystem.md) | Some filesystem content (typically provided by a *tar* or *tgz* archive). The blob's mime type specifies the concrete format. |
| GitOps | [`gitOpsTemplate`](gitOpsTemplate.md) | Filesystem content (tar, tgz) used as GitOps Template, e.g. to set up a git repo used for continuous deployment (for example flux) |

For centrally defined artifact types, there might be special support in the
standard OCM library and tool set. For example, there is a dedicated downloader
Expand Down Expand Up @@ -39,7 +39,7 @@ must deliver such a blob, also.
This basically means, whenever a new resource type is supported,
corresponding blob formats must be defined for this type. Type-agnostic access types, like [`localBlob`](../B/localBlob.md) or [`ociBlob`](../B/ociBlob.md)
just deal with those blobs, they never need to know anything about their internal
format. But specific access methods, e.g. the [`ociArtecat`](../B/ociArtifact.md)
format. But specific access methods, e.g. the [`ociArtifact`](../B/ociArtifact.md)
method may provide dedicated blob formats.

These blob formats may depend on the combination of artifact type and access type.
Expand Down
File renamed without changes.
10 changes: 9 additions & 1 deletion doc/glossary.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Glossary

[A](#a) &nbsp; B &nbsp; [C](#c) &nbsp; [D](#d) &nbsp; [E](#e) &nbsp; F &nbsp; G &nbsp; H &nbsp; [I](#i) &nbsp; J &nbsp; K &nbsp; [L](#l) &nbsp; [M](#m) &nbsp; [N](#n) &nbsp; [O](#o) &nbsp; [P](#p) &nbsp; Q &nbsp; [R](#r) &nbsp; [S](#s) &nbsp; [T](#t) &nbsp; U &nbsp; V &nbsp; W &nbsp; X &nbsp; Y &nbsp; Z
[A](#a) &nbsp; B &nbsp; [C](#c) &nbsp; [D](#d) &nbsp; [E](#e) &nbsp; F &nbsp; G &nbsp; [H](#h) &nbsp; [I](#i) &nbsp; J &nbsp; K &nbsp; [L](#l) &nbsp; [M](#m) &nbsp; [N](#n) &nbsp; [O](#o) &nbsp; [P](#p) &nbsp; Q &nbsp; [R](#r) &nbsp; [S](#s) &nbsp; [T](#t) &nbsp; U &nbsp; V &nbsp; W &nbsp; X &nbsp; Y &nbsp; Z

## A

Expand Down Expand Up @@ -106,6 +106,14 @@ behaviour of such elements.
additional parts of the identity of an [element](#elemid) of a [component version](#compvers)
described as dedicated attributes in a [component descriptor](#compdesc).

## H

#### [Helm Chart](appendix/B/helm.md)<a id="helmchart"/>
A set of files describing the deplyoment of a Kubernetes application using
the [Helm](https://helm.sh/) deployment mechanism.

see [element identity](#elemid), [component identity](#compid), [component version identity](#compversid)

## I

#### Identity<a id="identity"/>
Expand Down
149 changes: 87 additions & 62 deletions doc/specification/formats/componentdescriptor_normalization.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,92 @@ normalizations.
- `jsonNormalisationV2`: This is the new format. which is independent of the
chosen representation format of the component descriptor.

## Generic Normalization format
The normalization process is divided into two steps:

- *extraction of the signature relevant information from the component descriptor*

The result is basically a JSON object, which decsribed the relevant information.

- *normalization of the resulting JSON object*

Here, the object is serialized to a unique and reproducable byte sequence, which is finally used to determine the digest.

There are two such normalization methods:
- `jsonNormalisationV1` uses an [OCM specific representation](#generic-normalization-format) of the JSON object.
- `jsonNormalisationV2` uses a standard scheme according to [RFC8785 (JCS)](https://www.rfc-editor.org/rfc/rfc8785).


## `jsonNormalisationV1` vs `jsonNormalisationV2`

The `JsonNormalisationV1` serialization format is based on the serialization
format of the component descriptor. It uses an appropriate JSON object containing the relevant fields as contained in the component descriptors's serialization. The format version fields are included. Therefore, the normalized form is depending on the chosen serialization format. Changing this format version would result in different digests. The resulting JSON object is serialized with the [OCM specific scheme](#generic-normalization-format)

`JsonNormalisationV2` strictly uses only the relevant component descriptor
information according to the field specification shown below. It is independent of the serialization format used to store the component decsriptor in some storage backend. Therefore, the calculated digest is finally independent of the serialization format chosen for storing the component descriptor in a storage backend.

Additionally, it uses the JCS scheme for uniquely serializing the resulting
JSON object.

## Relevant information in Component Descriptors

A component descriptor contains static information and
information, which may change over time (for example, the access methods
specifications might be changed during a transport step). A digest should be
stable even after a transport and therefore should only hash static
information. Therefore, a component descriptor is transformed to format
containing only immutable fields, which are finally relevant for the signing
process to assure the data integrity.

Relevant fields and their mapping to the normalized data structure for `JsonNormalisationV2`:
- Component Name: mapped to `component.name`
- Component Version: mapped to `component.version`
- Component Labels: mapped to `component.labels` (see [Labels](#labels)])
- Component Provider: mapped to `component.provider`
- Resources: mapped to `component.resources`, always empty list enforced, without the source references (see [Labels](#labels)] and [Access Methods](#access-methods)])
- Sources: mapped to `component.sources`, always empty list enforced, (see [Labels](#labels)] and [Access Methods](#access-methods)])
- References: mapped to `component.references`, always empty list enforced, (see [Labels](#labels)])

### Access Methods

Access method specifications are completely ignored.
A resource/source is ignored, if the access method type is `none`.

## Labels

Labels are removed before signing but can be marked with a special boolean
property `signing`. This property indicates that the label should be
signing-relevant and therefore part of the digest. As a consequence such
labels cannot be changed anymore during the lifecycle of a component version
any may only describe static information.
The structure of signing-relevant labels is preserved from the component
descriptor version `v2`.

Example:

```yaml
labels:
- name: label1
value: foo
- name: label2
value: bar
signing: true
```
`label1` will be excluded from the digest, `label2` will be included.
The label value is taken as it is, preserving a potentially deeply nested structure.

## Exclude Resources from Normalization/Signing

If a resource should not be part of the normalization and later signing, the resource needs a special digest in the following format:

```yaml
digest:
hashAlgorithm: NO-DIGEST
normalisationAlgorithm: EXCLUDE-FROM-SIGNATURE
value: NO-DIGEST
```

## Generic Normalization Format

The generic format is based on a data structure consisting of dictionaries, lists and
simple values (like strings and integers).
Expand Down Expand Up @@ -220,64 +305,4 @@ are all normalized to:

```json
[]
```

## Relevant information in Component Descriptors.

A component descriptor contains signature relevant information and
information, which may change. For example, the access methods specifications
might be changed during atransport step.

Relevant fields and their mapping to the normalized data structure for `JsonNormalisationV2`:
- Component Name: mapped to `component.name`
- Component Version: mapped to `component.version`
- Component Labels: mapped to `component.labels` (see [Labels](#labels)])
- Component Provider: mapped to `component.provider`
- Resources: mapped to `component.resources`, always empty list enforced, without the source references (see [Labels](#labels)] and [Access Methods](#access-methods)])
- Sources: mapped to `component.sources`, always empty list enforced, (see [Labels](#labels)] and [Access Methods](#access-methods)])
- References: mapped to `component.references`, always empty list enforced, (see [Labels](#labels)])

### Access Methods

Access method specifications are completely ignored.
A resource/source is ignored, if the access method type is `none`.

## Labels

Labels are removed before signing but can be marked with a special boolean
property `signing` not to be removed and thus be part of the signature.
The structure of signing-relevant labels is preserved from the component
descriptor version `v2`.

Example:

```yaml
labels:
- name: label1
value: foo
- name: label2
value: bar
signing: true
```
`label1` will be excluded from the signature, `label2` will be included.
The label values is takes as it is, preserving a potentially deep structure.

## Exclude Resource from Normalisation/Signing

If a resource should not be part of the normalisation and later signing, the resource needs a special digest in the following format:

```yaml
digest:
hashAlgorithm: NO-DIGEST
normalisationAlgorithm: EXCLUDE-FROM-SIGNATURE
value: NO-DIGEST
```

# `jsonNormalisationV1` vs `jsonNormalisationV2`

The `JsonNormalisationV1` serialization format is based on the serialization
format of the component descriptor. The format version fields are included

`JsonNormalisationV2` strictly uses only the relevant component descriptor
information according to the field specification shown above.
```
2 changes: 1 addition & 1 deletion doc/specification/formats/normalization.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ content of described artifacts must be incorporated.

The metadata of a component version is described by a [component-descriptor](../../specification/elements/README.md#component-descriptor). It contains volatile information, also, e.g. the artifact access specification, which might change during [transports](../../introduction/transports.md).
Therefore, to calculate a digest, the component descriptor has to be transformed
to an immutable technical reprsentation containing only signature relevant information. THis process is called [*component descriptor normalization*](componentdescriptor_normalization.md).
to an immutable technical representation containing only static signature relevant information. This process is called [*component descriptor normalization*](componentdescriptor_normalization.md).

To cover the content of the artifacts described by a component version, a digest
for the artifact content has to be calculated, also, and incorporated into the
Expand Down

0 comments on commit 9c13e04

Please sign in to comment.