diff --git a/README.md b/README.md index ef0c96d79..8c1be21ed 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,6 @@ cd cmd/mdatagen go install ``` - Before starting development on the NGINX Agent, it is important to download and install the necessary tool and dependencies required by the NGINX Agent. You can do this by running the following `make` command: ``` make install-tools @@ -49,13 +48,36 @@ Build NGINX Agent apk package: OSARCH= make local-apk-package ``` +### Testing NGINX Agent + +#### Unit tests +To run unit tests and check that there is enough test coverage run the following +``` +make unit-test coverge +``` +To check for race conditions, the unit tests can also be run with a race condition detector +``` +make race-condition-test +``` + +#### Integration tests +To run integration tests, run the following +``` +make integration-test +``` + +#### Testing with a mock management plane +For testing command operations, there is a mock management gRPC server that can be used. See here: [mock management gRPC server](test/mock/grpc/README.md) \ +For testing metrics, there is a mock management OTel collector that can be used. See here: [mock management OTel collector](test/mock/collector/README.md) + + ## NGINX Agent Technical Specifications -## Supported Distributions +### Supported Distributions NGINX Agent can run in most environments. For a list of supported distributions, see the [NGINX Technical Specs](https://docs.nginx.com/nginx/technical-specs/#supported-distributions) guide. -## Supported Deployment Environments +### Supported Deployment Environments NGINX Agent can be deployed in the following environments: @@ -64,11 +86,11 @@ NGINX Agent can be deployed in the following environments: - Public Cloud: AWS, Google Cloud Platform, and Microsoft Azure - Virtual Machine -## Supported NGINX Product Versions +### Supported NGINX Product Versions NGINX Agent works with all supported versions of NGINX Open Source and NGINX Plus. -## Sizing Recommendations +### Sizing Recommendations Minimum system sizing recommendations for NGINX Agent: TBD diff --git a/test/mock/collector/README.md b/test/mock/collector/README.md new file mode 100644 index 000000000..92a53afc1 --- /dev/null +++ b/test/mock/collector/README.md @@ -0,0 +1,32 @@ +# Mock Management OTel Collector + +There are 3 images that need to be built in order to use the mock management OTel collector +* Agent with NGINX Plus image +* Agent with NGINX OSS image +* Custom OTel collector image + +To build these images run the following +``` +make local-deb-package build-test-plus-image build-test-oss-image build-mock-collector-image +``` + +To start run everything run the following +``` +make run-mock-management-otel-collector +``` + +Once everything is started there should be 5 containers running +``` +8e6df6d0bc73 localhost/nginx_plus_agent_ubuntu_22.04:latest 4 minutes ago Up 4 minutes 80/tcp, 443/tcp mock-collector-agent-with-nginx-plus +a65a7efaf2b3 localhost/nginx_oss_agent_ubuntu_22.04:latest 4 minutes ago Up 4 minutes 80/tcp, 443/tcp mock-collector-agent-with-nginx-oss +bf0f247991c0 localhost/mock-collector:latest go run main.go 4 minutes ago Up 4 minutes 0.0.0.0:4320->4317/tcp, 0.0.0.0:9775->9090/tcp mock-collector-otel-collector +67bb7bde6392 docker.io/prom/prometheus:latest --config.file=/et... 4 minutes ago Up 4 minutes 0.0.0.0:9090->9090/tcp, 9090/tcp mock-collector-prometheus +a83a997eb652 docker.io/grafana/grafana:latest 4 minutes ago Up 4 minutes 0.0.0.0:3002->3000/tcp, 3000/tcp mock-collector-grafana +``` + +To view the metrics, the grafana UI can be used by accessing this URL http://localhost:3002/login (Note: username/password is admin/admin) + +To stop everything run the following +``` +make stop-mock-management-otel-collector +``` diff --git a/test/mock/grpc/README.md b/test/mock/grpc/README.md new file mode 100644 index 000000000..d45069315 --- /dev/null +++ b/test/mock/grpc/README.md @@ -0,0 +1,305 @@ +# Mock Management gRPC Server + +To start the server, run the following command +``` +make run-mock-management-grpc-server +``` + +By default, this will start a HTTP and gRPC server on a random port. \ +It will also save config files to the `/tmp` by default. \ +To override these behaviours the following environment variables can be set to override them +``` +MOCK_MANAGEMENT_PLANE_GRPC_ADDRESS=127.0.0.1:9091 +MOCK_MANAGEMENT_PLANE_API_ADDRESS=127.0.0.1:9092 +MOCK_MANAGEMENT_PLANE_CONFIG_DIRECTORY=/tmp/ +``` + +Before starting the NGINX Agent, update the agent configuration with the following command config block +``` +command: + server: + host: localhost + port: 9091 +``` + +To interact with the mock management gRPC server, it also starts a HTTP server with the following endpoints +``` +GET http://127.0.0.1:9092/api/v1/connection + +GET http://127.0.0.1:9092/api/v1/status + +GET http://127.0.0.1:9092/api/v1/health + +GET http://127.0.0.1:9092/api/v1/responses + +POST http://127.0.0.1:9092/api/v1/requests + +POST http://127.0.0.1:9092/api/v1/instance//config/apply +``` + +# Endpoints +## GET /api/v1/connection +Used to check if the NGINX Agent successfully created connection with the management plane. \ +Example response: +``` +{ + "messageMeta": { + "correlationId": "0d777e07-bbdf-4ce9-a8d2-22b1e8383984", + "messageId": "4300d60e-0e4e-4761-a934-d93623e07b90", + "timestamp": "2024-10-25T13:14:36.028654Z" + }, + "resource": { + "hostInfo": { + "hostId": "14bb13db-347d-33b4-92f6-0e3d33e0d840", + "hostname": "example.com", + "releaseInfo": { + "codename": "darwin", + "id": "darwin", + "name": "Standalone Workstation", + "version": "23.6.0", + "versionId": "14.6.1" + } + }, + "instances": [ + { + "instanceConfig": { + "agentConfig": { + "command": {}, + "features": [ + "configuration", + "connection", + "metrics", + "file-watcher" + ], + "file": {}, + "metrics": {} + } + }, + "instanceMeta": { + "instanceId": "9da17ae7-aadc-3dff-8299-fbce82ec0175", + "instanceType": "INSTANCE_TYPE_AGENT" + }, + "instanceRuntime": { + "binaryPath": "/var/usr/bin/nginx-agent", + "configPath": "/etc/nginx-agent/nginx-agent.conf", + "processId": 31877 + } + }, + { + "instanceMeta": { + "instanceId": "6cb1a2bc-7552-33b1-9e7c-cb6658b82ebb", + "instanceType": "INSTANCE_TYPE_NGINX", + "version": "1.27.0" + }, + "instanceRuntime": { + "binaryPath": "/usr/local/bin/nginx", + "configPath": "/usr/local/etc/nginx/nginx.conf", + "instanceChildren": [ + { + "processId": 703 + } + ], + "nginxRuntimeInfo": { + "accessLogs": [ + "/usr/local/var/log/nginx/access.log" + ], + "dynamicModules": [ + "http_addition_module", + "http_auth_request_module", + "http_dav_module", + "http_degradation_module", + "http_flv_module", + "http_gunzip_module", + "http_gzip_static_module", + "http_mp4_module", + "http_random_index_module", + "http_realip_module", + "http_secure_link_module", + "http_slice_module", + "http_ssl_module", + "http_stub_status_module", + "http_sub_module", + "http_v2_module", + "http_v3_module", + "mail_ssl_module", + "stream_realip_module", + "stream_ssl_module", + "stream_ssl_preread_module" + ], + "errorLogs": [ + "/usr/local/var/log/nginx/error.log" + ], + "stubStatus": "http://127.0.0.1:8084/apitesst" + }, + "processId": 595 + } + } + ], + "resourceId": "14bb13db-347d-33b4-92f6-0e3d33e0d840" + } +} +``` +## GET /api/v1/status +Used to check if the NGINX Agent successfully sent a data plane status update to the management plane. \ +Example response: +``` +{ + "messageMeta": { + "correlationId": "0d777e07-bbdf-4ce9-a8d2-22b1e8383984", + "messageId": "4300d60e-0e4e-4761-a934-d93623e07b90", + "timestamp": "2024-10-25T13:14:36.028654Z" + }, + "resource": { + "hostInfo": { + "hostId": "14bb13db-347d-33b4-92f6-0e3d33e0d840", + "hostname": "example.com", + "releaseInfo": { + "codename": "darwin", + "id": "darwin", + "name": "Standalone Workstation", + "version": "23.6.0", + "versionId": "14.6.1" + } + }, + "instances": [ + { + "instanceConfig": { + "agentConfig": { + "command": {}, + "features": [ + "configuration", + "connection", + "metrics", + "file-watcher" + ], + "file": {}, + "metrics": {} + } + }, + "instanceMeta": { + "instanceId": "9da17ae7-aadc-3dff-8299-fbce82ec0175", + "instanceType": "INSTANCE_TYPE_AGENT" + }, + "instanceRuntime": { + "binaryPath": "/var/usr/bin/nginx-agent", + "configPath": "/etc/nginx-agent/nginx-agent.conf", + "processId": 31877 + } + }, + { + "instanceMeta": { + "instanceId": "6cb1a2bc-7552-33b1-9e7c-cb6658b82ebb", + "instanceType": "INSTANCE_TYPE_NGINX", + "version": "1.27.0" + }, + "instanceRuntime": { + "binaryPath": "/usr/local/bin/nginx", + "configPath": "/usr/local/etc/nginx/nginx.conf", + "instanceChildren": [ + { + "processId": 703 + } + ], + "nginxRuntimeInfo": { + "accessLogs": [ + "/usr/local/var/log/nginx/access.log" + ], + "dynamicModules": [ + "http_addition_module", + "http_auth_request_module", + "http_dav_module", + "http_degradation_module", + "http_flv_module", + "http_gunzip_module", + "http_gzip_static_module", + "http_mp4_module", + "http_random_index_module", + "http_realip_module", + "http_secure_link_module", + "http_slice_module", + "http_ssl_module", + "http_stub_status_module", + "http_sub_module", + "http_v2_module", + "http_v3_module", + "mail_ssl_module", + "stream_realip_module", + "stream_ssl_module", + "stream_ssl_preread_module" + ], + "errorLogs": [ + "/usr/local/var/log/nginx/error.log" + ], + "stubStatus": "http://127.0.0.1:8084/apitesst" + }, + "processId": 595 + } + } + ], + "resourceId": "14bb13db-347d-33b4-92f6-0e3d33e0d840" + } +} +``` +## GET /api/v1/health +Used to check if the NGINX Agent successfully sent a data plane health update to the management plane. \ +Example response: +``` +{ + "instanceHealths": [ + { + "instanceHealthStatus": "INSTANCE_HEALTH_STATUS_HEALTHY", + "instanceId": "6cb1a2bc-7552-33b1-9e7c-cb6658b82ebb" + } + ], + "messageMeta": { + "correlationId": "ba95cf3b-793a-4761-a843-c85c7562faca", + "messageId": "7cd63160-c408-4ab6-8639-5909453c8574", + "timestamp": "2024-10-25T13:14:37.894919Z" + } +} +``` +## GET /api/v1/responses +Used to check if what responses the management plane received from the NGINX Agent on the Subscribe rpc stream. +Example response: +``` +[ + { + "message_meta": { + "message_id": "0971692c-c0bf-4b67-a03b-ef13b3c3ea9b", + "correlation_id": "87e3c12f-eaf7-4f54-a27c-c91fe75d44a6", + "timestamp": { + "seconds": 1729862076, + "nanos": 98825000 + } + }, + "command_response": { + "status": 1, + "message": "Successfully updated all files" + } + } +] +``` +## POST /api/v1/requests +Used to send management plane requests over the Subscribe rpc stream to the NGINX Agent. \ +Example request body: +``` +{ + "message_meta": { + "message_id": "e2254df9-8edd-4900-91ce-88782473bcb9", + "correlation_id": "9673f3b4-bf33-4d98-ade1-ded9266f6818", + "timestamp": "2023-01-15T01:30:15.01Z" + }, + "health_request": {} +} +``` +## POST /api/v1/instance/\/config/apply +Used to send management plane config apply request over the Subscribe rpc stream to the NGINX Agent for a particular data plane instance. + +The config files that you need to change to perform a config apply are located here `/tmp/config//`. + +The `` and `` can be determined from the response of the `/api/v1/connection` endpoint. \ +In the example above the `instance id` would be `6cb1a2bc-7552-33b1-9e7c-cb6658b82ebb` and the `` would be `/etc/nginx-agent/nginx-agent.conf`. \ +So the full path to the file used by the mock management plane would be `/tmp/config/6cb1a2bc-7552-33b1-9e7c-cb6658b82ebb/etc/nginx-agent/nginx.conf`. + +Simply edit this file and then perform a POST request against the `/api/v1/instance//config/apply` endpoint to execute a config apply request. +