Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run a cronjob that generates monthly openshift usage report. #45

Merged
merged 1 commit into from
Mar 4, 2024
Merged

Run a cronjob that generates monthly openshift usage report. #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions k8s/base/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
resources:
- daily-openshift-metrics-collector-cronjob.yaml
- produce-report-cronjob.yaml
- metrics-downloader-configmap.yaml
9 changes: 9 additions & 0 deletions k8s/base/metrics-downloader-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: metrics-downloader
data:
script.sh: |
DIRECTORY_NAME=$(date --date="$(date +%Y-%m-15) -1 month" +'data_%Y-%m')
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Date command document says this (run info date):

The fuzz in units can cause problems with relative items. For
example, '2003-07-31 -1 month' might evaluate to 2003-07-01, because
2003-06-31 is an invalid date. To determine the previous month more
reliably, you can ask for the month before the 15th of the current
month. For example:

 $ date -R
 Thu, 31 Jul 2003 13:02:39 -0700
 $ date --date='-1 month' +'Last month was %B?'
 Last month was July?
 $ date --date="$(date +%Y-%m-15) -1 month" +'Last month was %B!'
 Last month was June!

echo $DIRECTORY_NAME
aws s3 cp s3://openshift-metrics/$DIRECTORY_NAME/ /data --endpoint-url=https://s3.us-east-005.backblazeb2.com --recursive
66 changes: 66 additions & 0 deletions k8s/base/produce-report-cronjob.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
apiVersion: batch/v1
kind: CronJob
metadata:
name: produce-report
spec:
schedule: "0 13 1 * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: daily-openshift-metrics-collector
image: ghcr.io/cci-moc/openshift-usage-scripts:main
env:
- name: S3_OUTPUT_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: nerc-invoices-b2-bucket
key: access-key-id
- name: S3_OUTPUT_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: nerc-invoices-b2-bucket
key: secret-access-key
- name: CLIENT_ID
valueFrom:
secretKeyRef:
name: keycloak-client
key: client-id
- name: CLIENT_SECRET
valueFrom:
secretKeyRef:
name: keycloak-client
key: client-secret
volumeMounts:
- name: data-volume
mountPath: /data
command: ["/bin/sh", "-c", "cd /data && python /app/openshift_metrics/merge.py /data/*.json --upload-to-s3"]
initContainers:
- name: download-metrics
image: amazon/aws-cli
command: ["/bin/sh", "-c", "/script/script.sh"]
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: openshift-metrics-b2-bucket
key: access-key-id
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: openshift-metrics-b2-bucket
key: secret-access-key
volumeMounts:
- name: data-volume
mountPath: /data
- name: metrics-downloader
mountPath: /script
volumes:
- name: data-volume
emptyDir: {}
- name: metrics-downloader
configMap:
name: metrics-downloader
defaultMode: 0555
restartPolicy: OnFailure
8 changes: 8 additions & 0 deletions k8s/overlay/dev/keycloak-client.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: keycloak-client
type: Opaque
data:
client-id: id
client-secret: secret
2 changes: 2 additions & 0 deletions k8s/overlay/dev/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,6 @@ resources:
- ../../base
- openshift-metrics-b2-bucket.yaml
- metrics-reader-token.yaml
- nerc-invoices-b2-bucket.yaml
- keycloak-client.yaml
namespace: xdmod-reader
8 changes: 8 additions & 0 deletions k8s/overlay/dev/nerc-invoices-b2-bucket.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Secret
metadata:
name: nerc-invoices-b2-bucket
type: Opaque
data:
access-key-id: 1234
secret-access-key: secret
Loading