From 8d71caa2b1826fb9747d5ac685ddbf25c192dbb2 Mon Sep 17 00:00:00 2001 From: Victor BERTIN Date: Wed, 23 Jun 2021 18:11:58 +0200 Subject: [PATCH 1/2] Correct and update HTTPS documentation This tutorial allows to set up the client and the Hawkbit server to use HTTPS from the start. --- hawkbit-https.md | 194 ++++++++++++++++++++++++++++------------------- 1 file changed, 114 insertions(+), 80 deletions(-) diff --git a/hawkbit-https.md b/hawkbit-https.md index 9608d68..1b85547 100644 --- a/hawkbit-https.md +++ b/hawkbit-https.md @@ -1,131 +1,118 @@ # Deploying HTTPS on Hawkbit -In this document, we will see how to deploy https on Hawkbit and how to configure it on your -targets with FullMetalUpdate. +In this document, we will see how to set up HTTPS transactions on Hawkbit and how to configure it on your +targets. The final goal of this process is to make your OTA updates secure. ## Setting up the server ### Getting a certificate Like every other HTTPS service, Hawkbit requires a signed certificate from a Certificate -Authority to comply with SSL protocols. We will not through how to get one, as there -plenty of resources already available on them. Hawkbit also requires the private key -linked to the certificate. +Authority to comply with SSL protocols. -**Warning:** this setup will not work with self-signed certificates. Please reach out to -us [on gitter][gitter] if you need to configure self-signed certificates. - -### Modifying the docker-compose - -Go to the cloud directory (`fullmetalupdate-cloud-demo`) and edit `docker-composer.yml`. -Go the `hawkbit:`, and change all the `8080` to `8443` in the `expose` and `ports` -subsections. - -Exit, and restart the server using `./StopServer.sh` and `./StartServer.sh`. The server is -not yet configured for https. - -### Getting the server's CONTAINER ID - -Before setting HTTPS up, you need to take note of the Hawkbit's server container id. You -must have run `./StartServer.sh` from the `fullmetalupdate-cloud-demo` directory. - -Run this command: +The following lines will allow you to get a self signed certificate **suitable for test environments only**. ``` -docker ps -a +sudo apt-get update && sudo apt-get upgrade +sudo apt-get install openssh +openssl req -x509 -newkey rsa:4096 -keyout privateKey.pem -out certificate.crt -days 3650 -nodes +openssl pkcs12 -export -out keyStore.p12 -inkey privateKey.pem -in certificate.crt ``` +You will need a passphrase for `privateKey.pem`, which we will denote as #key-passphrase, as well as an export password for the `keyStore.p12`, which we will denote as #p12-password. -And take note of the `fullmetalupdate/hawkbit`'s CONTAINER ID. We will denote it as -`` in this document. +**Warning:** Password of `keyStore.p12` (ie, #p12-password) must be at least 6 characters long and contains letters AND numbers. -### Setting up the certificates for Hawkbit +You will then obtain a certificate `certificate.crt` (the public key is included into this certificate), a private key `privateKey.pem` and a pkcs12 binary `keyStore.p12`. this last one is a special kind of file storing cryptographic objects, in our case your certificate as well as your private key. -Hawkbit requires a single certificate file generated from the `pem` and the `key` files, -assuming that the `pem` file is the certificate, and the `key` file is the private key -linked to it. +**Warning:** This setup will not work for prod environments. Please reach out to +us [on gitter][gitter] if you need to configure proper certificates. -First, create a [`p12` file][wiki_p12] from the last two file: +**Coming soon:** Proper certificate generation with Let's Encrypt and certbot python tool. -``` -openssl pkcs12 -export -out cert.p12 -in CA.pem -inkey CA.key -``` -`CA.pem` and `CA.key` are the certificate and the private key. Enter the private key's -passphrase, which we will denote as `` in this document, and type in the -export password, which we will denote as ``. **Warning**: this password must -be at least 6 character long, and contain letters and numbers. +### Modifying the docker-compose -This will create a `cert.p12` file. Copy this file to the currently running container : +Go to the cloud directory (`fullmetalupdate-cloud-demo`) and do : -``` -docker cp cert.p12 :/opt/hawkbit/ -``` +`git checkout add-https` + +This branch holds everything you need to set up Hawkbit to work with HTTPS. We will come back to this repo later. ### Converting the `p12` file into a `jks` file The Hawkbit container embeds `keytool`, the tool used to manage the Java KeyStore (JKS). -This is what is used by Hawkbit to manage certificates. - -Run a shell in the container by executing: +JavaKeyStore objects are used by Hawkbit (note that Hawkbit client is written in Java) to manage certificates. We have to use it because Hawkbit does not seem to be able to deal with pkcs12 files. -``` -docker exec -it /bin/sh -``` +Go in a the directoy where you generated `privateKey.pem`, `certificate.crt` and `keyStore.p12`. -Execute `ls`: you should see the `cert.p12` file generated above. Execute the following -command to convert the `p12` file into a `jks` file: +You can now generate the jks file : ``` -keytool -importkeystore -srckeystore cert.p12 -srcstoretype pkcs12 \ - -destkeystore cert.jks -deststoretype pkcs12 \ - -alias 1 +keytool -importkeystore -srckeystore keyStore.p12 -srcstoretype pkcs12 \ + -destkeystore keyStore.jks -deststoretype pkcs12 \ + -alias 1 -deststorepass adminVBW ``` -First, enter the destination keystore password : **this must be the same as -``**. +First, enter the destination keystore password : **this must be the same as #p12-password **. Then, enter the source keystore password : #p12-password. -Then, enter the source keystore password : ``. +You finally get a JKS file as `keyStore.kjs`, which you can use to configure Hawkbit. -You can now exit the container by executing `exit`. +### Building new docker image fullmetalupdate/build-yocto:v2.0 -### Setting up Hawkbit's options for SSL/TLS +#### Configuring Hawkbit's options for SSL/TLS -Create an `application.properties` file which contains these parameters : +Go to your FullMetalupdate local repo `dockerfiles` and configure `hawkbit/application.properties` file with the appropriate parameters : ``` -# UI demo account +1) UI demo account hawkbit.server.ui.demo.password=admin hawkbit.server.ui.demo.user=admin hawkbit.server.ui.demo.tenant=DEFAULT -# User Security +2) User Security security.user.name=admin security.user.password=admin -# SSL/TLS Configuration +3) SSL/TLS Configuration server.port=8443 hawkbit.artifact.url.protocols.download-http.protocol=https hawkbit.artifact.url.protocols.download-http.port=8443 security.require-ssl=true server.use-forward-headers=true -server.ssl.key-store=/opt/hawkbit/cert.jks -server.ssl.key-password= -server.ssl.key-store-password= +server.ssl.key-store=/opt/hawkbit/keyStore.jks +server.ssl.key-password=#p12-password +server.ssl.key-store-password=#p12-password ``` -Replace `` accordingly. +Don't forget to replace #p12-password accordingly. -Copy this file the currently running container : +#### Setting up the build +You will need to generate a new docker image based on a Dockerfile specially written to set up HTTPS on Hawkbit. This step can take up to 1 hour depending on your computer. ``` -docker cp application.properties :/opt/hawkbit/ +git clone https://github.com/FullMetalUpdate/dockerfiles.git +git checkout add-https +mv /home/user/path/to/keyStore.jks hawkbit/ +docker build -t "fullmetalupdate/hawkbit:v2.0" hawkbit/Dockerfile ``` -This will replace the existing `application.properties`. If you already configured it -please `sh` into the container and append the previous content instead. +**Note:** When the container is running, you can access it by running `docker exec -it fullmetalupdate-cloud-demo_hawkbit_1 sh`. This will give you the hand on the container by opening a bourne shell inside the container. + +At this point, the Hawkbit server is ready to be started. + +Go to the cloud directory (`fullmetalupdate-cloud-demo`) and do : + +`./StartServer` + +Eventually if it is the first time you configure your target, you can lauch : + +`./ConfigureServer` + +Finally : + +`firefox https://localhost:8443 &` -The server is all set up! One last step: stop the server (`./StopServer`) and start it -again (`./StartServer`). You can now connect to `https://.local:8443`. +This last command will open the Hawkbit Web Client, working well in HTTPS! But your embedded target is not ready yet to support HTTPS OTA updates. ## Setting up the targets @@ -136,24 +123,53 @@ Before proceeding further, **close any instanciation of the build-yocto docker** 1. Go to your Yocto directory (`fullmetalupdate-yocto-demo`) - 1. Edit the `config.cfg.sample` and make the following changes : + 2. Edit the `config.cfg` file and make the following changes : * change `hawkbit_url_port` to `8443` * change `hawkbit_ssl` to `true` - 1. Exit, and execute `bash ConfigureBuild.sh` + 3. Move your certificate (after renaiming it) into Yocto build container : - 1. Execute `./Startbuild.sh fullmetalupdate-os`. It should build the fullmetalupdate +``` +mv /path/to/certificate.crt /path/to/ca-certificates.crt +docker cp /path/to/ca-certificates.crt \ + fullmetalupdate/hawkbit:v2.0:/data/yocto/build/tmp/fullmetalupdate-os/work/x86_65-linux/curl-native/7.69.1-r0/recipe-sysroot-native/etc/ssl/certs +``` + + 4. Modify a few things to leverage SSL/TLS security (our current certificate is self signed, and whether your browser, curl, or the HTTP API do not like it and fail to compile or execute beacuse of that) : + +First, add `-k` option to `curl` bash command in `curl_post()` Yocto function in `build/yocto/source/meta-fullmetalupdate/classes/fullmetalupdae.bbclass` file. + +Then, go to FMU client repo (`fullmetalupdate/`) and modify the source code in `fullmetalupdate.py`: +``` +@@ -76,8 +76,8 @@ async def main(): + logging.basicConfig(level=LOG_LEVEL, + format='%(asctime)s %(levelname)-8s %(message)s', + datefmt='%Y-%m-%d %H:%M:%S') +- +- async with aiohttp.ClientSession() as session: ++ connTCP = aiohttp.TCPConnector(verify_ssl=False) ++ async with aiohttp.ClientSession(connector=connTCP) as session: + client = FullMetalUpdateDDIClient(session, HOST, SSL, TENANT_ID, TARGET_NAME, + AUTH_TOKEN, ATTRIBUTES) +``` + + 5. Execute `./Startbuild.sh fullmetalupdate-os`. It should build the fullmetalupdate client again. Then you can flash the new image to your target, which will contain the new settings. - 1. After starting the target, you should see your device's IP in the target's - information panel + 6. After starting the target, you should see your device's new IP in the target's information panel, which we will denote as #IP. + + 7. To finish the deployment, you can use the send-file script furnished in fullmetalupdate/dev-scripts to apply the modification done to FMU client source code. + +`./send-file.sh fullmetalupdate.py #IP` + +After this final step, you can start deploying your apps updates and OS updates as usual, but by benefiting from the security of HTTPS. ### Configuration on the already deployed target > Note : these changes will be permanent only if you made the changes on the build system -> as described in the last section. Otherwise, the following changes will be lost on the +> as described in the last section. Otherwise, the following changes, made to the target, will be lost on the > next OS update. If you have already deployed your target and have remote access to it, please follow these @@ -169,9 +185,27 @@ steps : * change `hawkbit_url_port` to `8443` * change `hawkbit_ssl` to `true` - 1. Restart the FullMetalUpdate client by executing `systemctl restart fullmetalupdate` + 1. go to FMU client repo (`fullmetalupdate/`) and modify the source code in `fullmetalupdate.py`: +``` +@@ -76,8 +76,8 @@ async def main(): + logging.basicConfig(level=LOG_LEVEL, + format='%(asctime)s %(levelname)-8s %(message)s', + datefmt='%Y-%m-%d %H:%M:%S') +- +- async with aiohttp.ClientSession() as session: ++ connTCP = aiohttp.TCPConnector(verify_ssl=False) ++ async with aiohttp.ClientSession(connector=connTCP) as session: + client = FullMetalUpdateDDIClient(session, HOST, SSL, TENANT_ID, TARGET_NAME, + AUTH_TOKEN, ATTRIBUTES) +``` + + 1. After starting the target, you should see your device's new IP in the target's information panel, which we will denote as #IP. + + 1. To finish the deployment, you can use the send-file script furnished in fullmetalupdate/dev-scripts to apply the modification done to FMU client source code. + +`./send-file.sh fullmetalupdate.py #IP` - 1. You should see your device's IP in the target's information panel +You can verify with `journalctl -f | grep fullmetalupdate.sh` that your FMU client is working properly again, in HTTPS. [wiki_p12]: https://fr.wikipedia.org/wiki/PKCS12 From 25f3702794cc7359e9524abe1b28719e9e79a10b Mon Sep 17 00:00:00 2001 From: Victor BERTIN Date: Mon, 28 Jun 2021 18:00:12 +0200 Subject: [PATCH 2/2] Add explanations in HTTPS documentation to ease Hawkbit Container configuration --- hawkbit-https.md | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/hawkbit-https.md b/hawkbit-https.md index 1b85547..f476d1c 100644 --- a/hawkbit-https.md +++ b/hawkbit-https.md @@ -22,9 +22,9 @@ You will need a passphrase for `privateKey.pem`, which we will denote as #key-pa **Warning:** Password of `keyStore.p12` (ie, #p12-password) must be at least 6 characters long and contains letters AND numbers. -You will then obtain a certificate `certificate.crt` (the public key is included into this certificate), a private key `privateKey.pem` and a pkcs12 binary `keyStore.p12`. this last one is a special kind of file storing cryptographic objects, in our case your certificate as well as your private key. +You will then obtain a certificate `certificate.crt` (the public key is included into this certificate), a private key `privateKey.pem` and a pkcs12 binary `keyStore.p12`. This last one is a special kind of file storing cryptographic objects, in our case your certificate as well as your private key. -**Warning:** This setup will not work for prod environments. Please reach out to +**Warning:** This setup will not work for production environments. Please reach out to us [on gitter][gitter] if you need to configure proper certificates. **Coming soon:** Proper certificate generation with Let's Encrypt and certbot python tool. @@ -40,7 +40,7 @@ This branch holds everything you need to set up Hawkbit to work with HTTPS. We w ### Converting the `p12` file into a `jks` file -The Hawkbit container embeds `keytool`, the tool used to manage the Java KeyStore (JKS). +`keytool`is a tool used to manage Java KeyStore (JKS) files. JavaKeyStore objects are used by Hawkbit (note that Hawkbit client is written in Java) to manage certificates. We have to use it because Hawkbit does not seem to be able to deal with pkcs12 files. Go in a the directoy where you generated `privateKey.pem`, `certificate.crt` and `keyStore.p12`. @@ -48,20 +48,31 @@ Go in a the directoy where you generated `privateKey.pem`, `certificate.crt` and You can now generate the jks file : ``` +sudo apt-get install keytool keytool -importkeystore -srckeystore keyStore.p12 -srcstoretype pkcs12 \ -destkeystore keyStore.jks -deststoretype pkcs12 \ - -alias 1 -deststorepass adminVBW + -alias 1 -deststorepass #p12-password ``` -First, enter the destination keystore password : **this must be the same as #p12-password **. Then, enter the source keystore password : #p12-password. +First, enter the destination keystore password : **this must be the same as #p12-password**. Then, enter the source keystore password : #p12-password. -You finally get a JKS file as `keyStore.kjs`, which you can use to configure Hawkbit. +You finally get a JKS file as `keyStore.jks`, which you can use to configure Hawkbit. ### Building new docker image fullmetalupdate/build-yocto:v2.0 #### Configuring Hawkbit's options for SSL/TLS -Go to your FullMetalupdate local repo `dockerfiles` and configure `hawkbit/application.properties` file with the appropriate parameters : +Go to your FullMetalupdate local repo `dockerfiles` (or clone it on our Github) and configure `hawkbit/application.properties` file with the appropriate parameters. + +**1st solution : Use config.sh script** + +Move your JavaKeyStore file to the `hawkbit` directory. + +`mv /home/user/path/to/keyStore.jks hawkbit/` + +Run `config.sh` script and give #p12-password when it is required. + +**2nd solution : By hand** ``` 1) UI demo account @@ -83,7 +94,6 @@ server.ssl.key-store=/opt/hawkbit/keyStore.jks server.ssl.key-password=#p12-password server.ssl.key-store-password=#p12-password ``` - Don't forget to replace #p12-password accordingly. #### Setting up the build @@ -92,7 +102,6 @@ You will need to generate a new docker image based on a Dockerfile specially wri ``` git clone https://github.com/FullMetalUpdate/dockerfiles.git git checkout add-https -mv /home/user/path/to/keyStore.jks hawkbit/ docker build -t "fullmetalupdate/hawkbit:v2.0" hawkbit/Dockerfile ``` @@ -133,14 +142,14 @@ Before proceeding further, **close any instanciation of the build-yocto docker** ``` mv /path/to/certificate.crt /path/to/ca-certificates.crt docker cp /path/to/ca-certificates.crt \ - fullmetalupdate/hawkbit:v2.0:/data/yocto/build/tmp/fullmetalupdate-os/work/x86_65-linux/curl-native/7.69.1-r0/recipe-sysroot-native/etc/ssl/certs + fullmetalupdate/hawkbit:v2.0:/data/yocto/build/tmp/fullmetalupdate-os/work/x86_65-linux/curl-native/7.69.1-r0/ recipe-sysroot-native/etc/ssl/certs ``` 4. Modify a few things to leverage SSL/TLS security (our current certificate is self signed, and whether your browser, curl, or the HTTP API do not like it and fail to compile or execute beacuse of that) : First, add `-k` option to `curl` bash command in `curl_post()` Yocto function in `build/yocto/source/meta-fullmetalupdate/classes/fullmetalupdae.bbclass` file. -Then, go to FMU client repo (`fullmetalupdate/`) and modify the source code in `fullmetalupdate.py`: +Secondly, go to FMU client repo (`fullmetalupdate/`, a repo you can clone on FullMetalUpdate Github) and modify the source code in `fullmetalupdate.py`: ``` @@ -76,8 +76,8 @@ async def main(): logging.basicConfig(level=LOG_LEVEL, @@ -158,9 +167,9 @@ Then, go to FMU client repo (`fullmetalupdate/`) and modify the source code in ` client again. Then you can flash the new image to your target, which will contain the new settings. - 6. After starting the target, you should see your device's new IP in the target's information panel, which we will denote as #IP. + 6. After starting the target, you should see your device's new IP in the target's information panel on Hawkbit Web Client, which we will denote as #IP. - 7. To finish the deployment, you can use the send-file script furnished in fullmetalupdate/dev-scripts to apply the modification done to FMU client source code. + 7. To finish the deployment, you can use the `send-file` script furnished in FullMetalUpdate/dev-scripts to apply the modification done to FMU client source code. `./send-file.sh fullmetalupdate.py #IP` @@ -177,15 +186,14 @@ steps : 1. Remount `/usr` as read-write by executing: `mount -o remount,rw /usr` - 1. Edit the configuration file by executing `vi - /usr/fullmetalupdate/rauc_hawkbit/config.cfg` + 1. Edit the configuration file by executing `vi /usr/fullmetalupdate/rauc_hawkbit/config.cfg` 1. Make the following changes : * change `hawkbit_url_port` to `8443` * change `hawkbit_ssl` to `true` - 1. go to FMU client repo (`fullmetalupdate/`) and modify the source code in `fullmetalupdate.py`: + 1. go to FMU client repo (`fullmetalupdate/`, a repo you can clone on FullMetalUpdate Github) and modify the source code in `fullmetalupdate.py`: ``` @@ -76,8 +76,8 @@ async def main(): logging.basicConfig(level=LOG_LEVEL,