Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add pod security standards (restricted) in CEL expressions - Part 3 #776

Merged
merged 9 commits into from
Jan 12, 2024

Conversation

MariamFahmy98
Copy link
Contributor

@MariamFahmy98 MariamFahmy98 commented Oct 2, 2023

Related Issue(s)

Closes #773

Description

This PR adds Kyverno policies written in CEL expressions for pod security standards (restricted).

Checklist

  • I have read the policy contribution guidelines.
  • I have added test manifests and resources covering both positive and negative tests that prove this policy works as intended.
  • I have added the artifacthub-pkg.yml file and have verified it is complete and correct.

@MariamFahmy98 MariamFahmy98 changed the title feat:feat: add pod security standards (restricted) in CEL expressions feat:feat: add pod security standards (restricted) in CEL expressions - Part 3 Oct 2, 2023
@MariamFahmy98 MariamFahmy98 changed the title feat:feat: add pod security standards (restricted) in CEL expressions - Part 3 feat: add pod security standards (restricted) in CEL expressions - Part 3 Oct 2, 2023
@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch from 655029c to 98411e6 Compare October 2, 2023 14:50
@chipzoller chipzoller self-requested a review October 26, 2023 14:10
@chipzoller
Copy link
Contributor

The require-run-as-nonroot policy here is failing kuttl tests.

@chipzoller
Copy link
Contributor

Per Slack discussion, if validate.message does not work currently for CEL expression Kyverno policies, the field should be removed.

@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch from be23f9f to 2162ee4 Compare November 7, 2023 12:17
@MariamFahmy98
Copy link
Contributor Author

Regarding the failed test require-run-as-nonroot, I got different results when using kyverno test.

  1. In case of using kyverno test CLI, all tests pass as expected. (The same resources are used for kuttl tests as well):
Loading test  ( ../policies/pod-security-cel/restricted/require-run-as-nonroot/kyverno-test.yaml ) ...
  Loading values/variables ...
  Loading policies ...
  Loading resources ...
  Applying 1 policy to 75 resources ...
  Checking results ...

│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│────────│
│ ID (75)  │ POLICY                 │ RULE            │ RESOURCE                    │ RESULT │ REASON │
│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│────────│
│        1 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob01        │ Pass   │ Ok     │
│        2 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob02        │ Pass   │ Ok     │
│        3 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob03        │ Pass   │ Ok     │
│        4 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob04        │ Pass   │ Ok     │
│        5 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob05        │ Pass   │ Ok     │
│        6 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob06        │ Pass   │ Ok     │
│        7 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob07        │ Pass   │ Ok     │
│        8 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob08        │ Pass   │ Ok     │
│        9 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob09        │ Pass   │ Ok     │
│       10 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob10        │ Pass   │ Ok     │
│       11 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob11        │ Pass   │ Ok     │
│       12 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob12        │ Pass   │ Ok     │
│       13 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob13        │ Pass   │ Ok     │
│       14 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob14        │ Pass   │ Ok     │
│       15 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob15        │ Pass   │ Ok     │
│       16 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment01  │ Pass   │ Ok     │
│       17 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment02  │ Pass   │ Ok     │
│       18 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment03  │ Pass   │ Ok     │
│       19 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment04  │ Pass   │ Ok     │
│       20 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment05  │ Pass   │ Ok     │
│       21 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment06  │ Pass   │ Ok     │
│       22 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment07  │ Pass   │ Ok     │
│       23 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment08  │ Pass   │ Ok     │
│       24 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment09  │ Pass   │ Ok     │
│       25 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment10  │ Pass   │ Ok     │
│       26 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment11  │ Pass   │ Ok     │
│       27 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment12  │ Pass   │ Ok     │
│       28 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment13  │ Pass   │ Ok     │
│       29 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment14  │ Pass   │ Ok     │
│       30 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment15  │ Pass   │ Ok     │
│       31 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod01                │ Pass   │ Ok     │
│       32 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod02                │ Pass   │ Ok     │
│       33 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod03                │ Pass   │ Ok     │
│       34 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod04                │ Pass   │ Ok     │
│       35 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod05                │ Pass   │ Ok     │
│       36 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod06                │ Pass   │ Ok     │
│       37 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod07                │ Pass   │ Ok     │
│       38 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod08                │ Pass   │ Ok     │
│       39 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod09                │ Pass   │ Ok     │
│       40 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod10                │ Pass   │ Ok     │
│       41 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod11                │ Pass   │ Ok     │
│       42 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod12                │ Pass   │ Ok     │
│       43 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod13                │ Pass   │ Ok     │
│       44 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod14                │ Pass   │ Ok     │
│       45 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod15                │ Pass   │ Ok     │
│       46 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob01       │ Pass   │ Ok     │
│       47 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob02       │ Pass   │ Ok     │
│       48 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob03       │ Pass   │ Ok     │
│       49 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob04       │ Pass   │ Ok     │
│       50 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob05       │ Pass   │ Ok     │
│       51 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob06       │ Pass   │ Ok     │
│       52 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob07       │ Pass   │ Ok     │
│       53 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob08       │ Pass   │ Ok     │
│       54 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob09       │ Pass   │ Ok     │
│       55 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob10       │ Pass   │ Ok     │
│       56 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment01 │ Pass   │ Ok     │
│       57 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment02 │ Pass   │ Ok     │
│       58 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment03 │ Pass   │ Ok     │
│       59 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment04 │ Pass   │ Ok     │
│       60 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment05 │ Pass   │ Ok     │
│       61 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment06 │ Pass   │ Ok     │
│       62 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment07 │ Pass   │ Ok     │
│       63 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment08 │ Pass   │ Ok     │
│       64 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment09 │ Pass   │ Ok     │
│       65 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment10 │ Pass   │ Ok     │
│       66 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod01               │ Pass   │ Ok     │
│       67 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod02               │ Pass   │ Ok     │
│       68 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod03               │ Pass   │ Ok     │
│       69 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod04               │ Pass   │ Ok     │
│       70 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod05               │ Pass   │ Ok     │
│       71 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod06               │ Pass   │ Ok     │
│       72 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod07               │ Pass   │ Ok     │
│       73 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod08               │ Pass   │ Ok     │
│       74 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod09               │ Pass   │ Ok     │
│       75 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod10               │ Pass   │ Ok     │
│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│────────│


Test Summary: 75 tests passed and 0 tests failed
  1. In case of running it as a kuttl test, the following resources are created although it shouldn't:
    • badpod01
    • badpod05
    • badpod13
    • baddeployment01
    • baddeployment05
    • baddeployment13
    • badcronjob01
    • badcronjob05
    • badcronjob13

The main issue is in the second CEL expression. When I tried to remove it from the policy, the kyverno test fails but kuttl tests pass!

  1. In case of using kyverno test: (there's a bug in the output!)
Loading test  ( ../policies/pod-security-cel/restricted/require-run-as-nonroot/kyverno-test.yaml ) ...
  Loading values/variables ...
  Loading policies ...
  Loading resources ...
  Applying 1 policy to 75 resources ...
  Checking results ...

│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│─────────────────────│
│ ID (75)  │ POLICY                 │ RULE            │ RESOURCE                    │ RESULT │ REASON              │
│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│─────────────────────│
│        1 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob01        │ Pass   │ Want fail, got pass │
│        2 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob02        │ Pass   │ Ok                  │
│        3 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob03        │ Pass   │ Ok                  │
│        4 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob04        │ Pass   │ Ok                  │
│        5 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob05        │ Pass   │ Want fail, got pass │
│        6 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob06        │ Pass   │ Ok                  │
│        7 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob07        │ Pass   │ Ok                  │
│        8 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob08        │ Pass   │ Ok                  │
│        9 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob09        │ Pass   │ Ok                  │
│       10 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob10        │ Pass   │ Ok                  │
│       11 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob11        │ Pass   │ Ok                  │
│       12 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob12        │ Pass   │ Ok                  │
│       13 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob13        │ Pass   │ Want fail, got pass │
│       14 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob14        │ Pass   │ Ok                  │
│       15 │ require-run-as-nonroot │ run-as-non-root │ CronJob/badcronjob15        │ Pass   │ Ok                  │
│       16 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment01  │ Pass   │ Want fail, got pass │
│       17 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment02  │ Pass   │ Ok                  │
│       18 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment03  │ Pass   │ Ok                  │
│       19 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment04  │ Pass   │ Ok                  │
│       20 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment05  │ Pass   │ Want fail, got pass │
│       21 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment06  │ Pass   │ Ok                  │
│       22 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment07  │ Pass   │ Ok                  │
│       23 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment08  │ Pass   │ Ok                  │
│       24 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment09  │ Pass   │ Ok                  │
│       25 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment10  │ Pass   │ Ok                  │
│       26 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment11  │ Pass   │ Ok                  │
│       27 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment12  │ Pass   │ Ok                  │
│       28 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment13  │ Pass   │ Want fail, got pass │
│       29 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment14  │ Pass   │ Ok                  │
│       30 │ require-run-as-nonroot │ run-as-non-root │ Deployment/baddeployment15  │ Pass   │ Ok                  │
│       31 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod01                │ Pass   │ Want fail, got pass │
│       32 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod02                │ Pass   │ Ok                  │
│       33 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod03                │ Pass   │ Ok                  │
│       34 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod04                │ Pass   │ Ok                  │
│       35 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod05                │ Pass   │ Want fail, got pass │
│       36 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod06                │ Pass   │ Ok                  │
│       37 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod07                │ Pass   │ Ok                  │
│       38 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod08                │ Pass   │ Ok                  │
│       39 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod09                │ Pass   │ Ok                  │
│       40 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod10                │ Pass   │ Ok                  │
│       41 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod11                │ Pass   │ Ok                  │
│       42 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod12                │ Pass   │ Ok                  │
│       43 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod13                │ Pass   │ Want fail, got pass │
│       44 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod14                │ Pass   │ Ok                  │
│       45 │ require-run-as-nonroot │ run-as-non-root │ Pod/badpod15                │ Pass   │ Ok                  │
│       46 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob01       │ Pass   │ Ok                  │
│       47 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob02       │ Pass   │ Ok                  │
│       48 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob03       │ Pass   │ Ok                  │
│       49 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob04       │ Pass   │ Ok                  │
│       50 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob05       │ Pass   │ Ok                  │
│       51 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob06       │ Pass   │ Ok                  │
│       52 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob07       │ Pass   │ Ok                  │
│       53 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob08       │ Pass   │ Ok                  │
│       54 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob09       │ Pass   │ Ok                  │
│       55 │ require-run-as-nonroot │ run-as-non-root │ CronJob/goodcronjob10       │ Pass   │ Ok                  │
│       56 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment01 │ Pass   │ Ok                  │
│       57 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment02 │ Pass   │ Ok                  │
│       58 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment03 │ Pass   │ Ok                  │
│       59 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment04 │ Pass   │ Ok                  │
│       60 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment05 │ Pass   │ Ok                  │
│       61 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment06 │ Pass   │ Ok                  │
│       62 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment07 │ Pass   │ Ok                  │
│       63 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment08 │ Pass   │ Ok                  │
│       64 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment09 │ Pass   │ Ok                  │
│       65 │ require-run-as-nonroot │ run-as-non-root │ Deployment/gooddeployment10 │ Pass   │ Ok                  │
│       66 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod01               │ Pass   │ Ok                  │
│       67 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod02               │ Pass   │ Ok                  │
│       68 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod03               │ Pass   │ Ok                  │
│       69 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod04               │ Pass   │ Ok                  │
│       70 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod05               │ Pass   │ Ok                  │
│       71 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod06               │ Pass   │ Ok                  │
│       72 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod07               │ Pass   │ Ok                  │
│       73 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod08               │ Pass   │ Ok                  │
│       74 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod09               │ Pass   │ Ok                  │
│       75 │ require-run-as-nonroot │ run-as-non-root │ Pod/goodpod10               │ Pass   │ Ok                  │
│──────────│────────────────────────│─────────────────│─────────────────────────────│────────│─────────────────────│


Test Summary: 75 tests passed and 0 tests failed
  1. In case of running it as a kuttl test, it passes!

@chipzoller
Copy link
Contributor

So is the second expression wrong? And is there one or two bugs here?

@MariamFahmy98
Copy link
Contributor Author

The 2nd expression is required because it basically checks that either of spec.securityContext.runAsNonRoot or spec.containers.runAsNonRoot/spec.initContainers.runAsNonRoot/spec.ephemeralContainers.runAsNonRoot must be set to true.

For example, the following pod should fail:

apiVersion: v1
kind: Pod
metadata:
  name: badpod01
spec:
  containers:
  - name: container01
    image: busybox:1.35

The 1st expression will be evaluated to true, whereas the 2nd one will be evaluated to false so it is required.

The kyverno test result is expected but I haven't figured out why running the kuttl test fails! Actually, why we have two different results? I need to investigate in it.

@chipzoller
Copy link
Contributor

Have you tried the CEL playground to make sure this is being evaluated correctly?

@chipzoller
Copy link
Contributor

Were you able to get the last policy written?

@eddycharly
Copy link
Member

@MariamFahmy98 createdAt is a required field in artifacthub-pkg.yaml files

@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch 3 times, most recently from b83bfbb to 99f229b Compare December 4, 2023 09:14
@MariamFahmy98
Copy link
Contributor Author

MariamFahmy98 commented Dec 4, 2023

For testing purposes, I extracted the expression that causes failures into a separate policy and tested it against three bad pods.

  1. policy.yaml:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-run-as-nonroot
spec:
  validationFailureAction: Enforce
  rules:
    - name: run-as-non-root
      match:
        any:
        - resources:
            kinds:
              - Pod
      validate:
        cel:
          expressions:
            - expression: >-
                has(object.spec.securityContext) ||
                (object.spec.containers.all(container, has(container.securityContext) && has(container.securityContext.runAsNonRoot))
                &&
                (!has(object.spec.initContainers) || object.spec.initContainers.all(container, has(container.securityContext) && has(container.securityContext.runAsNonRoot)))
                &&
                (!has(object.spec.ephemeralContainers) || object.spec.ephemeralContainers.all(container, has(container.securityContext) && has(container.securityContext.runAsNonRoot))))
              message: >-
                Running as root is not allowed. Either the field spec.securityContext.runAsNonRoot
                must be set to `true`, or the fields spec.containers[*].securityContext.runAsNonRoot,
                spec.initContainers[*].securityContext.runAsNonRoot, and spec.ephemeralContainers[*].securityContext.runAsNonRoot
                must be set to `true`.
  1. resources.yaml:
apiVersion: v1
kind: Pod
metadata:
  name: badpod01
  namespace: default
spec:
  containers:
  - name: container01
    image: busybox:1.35
---
apiVersion: v1
kind: Pod
metadata:
  name: badpod05
spec:
  containers:
  - name: container01
    image: busybox:1.35
  - name: container02
    image: busybox:1.35
---
apiVersion: v1
kind: Pod
metadata:
  name: badpod13
spec:
  initContainers:
  - name: initcontainer01
    image: busybox:1.35
  - name: initcontainer02
    image: busybox:1.35
  containers:
  - name: container01
    image: busybox:1.35
    securityContext:
      runAsNonRoot: true

Upon investigation, here is what I have discovered so far:

  1. I tried to apply the policy using kyverno apply CLI , and all pods fail as expected:
summary:
  error: 0
  fail: 3
  pass: 0
  skip: 0
  warn: 0
  1. I tried to run the policy in Kyverno playground and all pods fail as we expect.
  2. I tried to apply both policy and resource in the cluster, and all pods were created. This is the main issue.
  3. I tried the CEL expression itself in the CEL playground to check the result and the expression was evaluated to false against all pods.
  4. I tried to create the VAP, and its binding to check the policy results as well, and pods were created.

Conclusion:

  1. The expression is evaluated as false in the CEL playground, which means the expression is correct.
  2. The bad pods fail as expected in the Kyverno CLI and playground.
  3. The bad pods were created successfully as a result of applying Kyverno policy and VAP.

I started a discussion thread in #sig-api-machinery-cel to see why VAP fails to block the creation of the pod.

@MariamFahmy98
Copy link
Contributor Author

@chipzoller - I think it would be better to remove this policy temporarily to get this PR into the main, and I will create a separate issue for this failed policy for more discussion.

@chipzoller
Copy link
Contributor

Are we close to getting this in?

@chipzoller
Copy link
Contributor

Ping @MariamFahmy98

@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch 2 times, most recently from 81b5e22 to 69c3b67 Compare January 2, 2024 15:33
@JimBugwadia JimBugwadia self-requested a review January 4, 2024 23:33
@JimBugwadia
Copy link
Member

@chipzoller - any additional comments or feedback?

Copy link
Contributor

@chipzoller chipzoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value of the policies.kyverno.io/category annotation should be Pod Security Standards (Restricted) in CEL to be consistent with the CEL Baseline policies

@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch from ae4cfd0 to a5da65d Compare January 5, 2024 07:18
Copy link
Contributor

@chipzoller chipzoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for missing this earlier and requesting more changes, but I also see the annotation policies.kyverno.io/minversion: 1.11.0 is missing from these while it is present on the other CEL policies for PSS.

@MariamFahmy98 MariamFahmy98 force-pushed the pod-security-cel-part-3 branch from 5683283 to 26727eb Compare January 12, 2024 13:41
@JimBugwadia
Copy link
Member

@chipzoller - anything else required to merge this?

Copy link
Contributor

@chipzoller chipzoller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice job, @MariamFahmy98!

@chipzoller chipzoller merged commit 15bb3bc into kyverno:main Jan 12, 2024
115 of 116 checks passed
@MariamFahmy98 MariamFahmy98 deleted the pod-security-cel-part-3 branch January 12, 2024 22:21
swastik959 pushed a commit to swastik959/policies that referenced this pull request Jan 18, 2024
…rt 3 (kyverno#776)

* feat: add pod security standards (restricted) in CEL expressions

Signed-off-by: Mariam Fahmy <[email protected]>

* fix: use cel.expression.message instead of validate.message

Signed-off-by: Mariam Fahmy <[email protected]>

* chore: update artifacthub-pkg.yaml

Signed-off-by: Mariam Fahmy <[email protected]>

* remove the failed policy

Signed-off-by: Mariam Fahmy <[email protected]>

* fix the name field in artifacthub-pkg.yaml

Signed-off-by: Mariam Fahmy <[email protected]>

* fix chainsaw tests

Signed-off-by: Mariam Fahmy <[email protected]>

* fix a lint issue

Signed-off-by: Mariam Fahmy <[email protected]>

* fix the value of policies.kyverno.io/category

Signed-off-by: Mariam Fahmy <[email protected]>

* fix: add the minversion annotation

Signed-off-by: Mariam Fahmy <[email protected]>

---------

Signed-off-by: Mariam Fahmy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Sample] Add policies using CEL expressions for Pod Security Standards
4 participants