Merge pull request #9 from mia-platform/add-docs

docs/10_overview.md

@@ -0,0 +1,78 @@
+# Integration Connector Agent
+
+The Integration Connector Agent connects to external sources and keeps changes
+synchronized between the source and one or more sinks. It also allows for data
+processing before sending it to the sinks.
+
+## Key Features
+
+- **Connect to External Sources**: Monitor and retrieve changes from various external data sources.
+- **Update Sinks**: Propagate detected changes to one or more sinks to keep data up-to-date.
+- **Data Processing**: Configure processors to transform or process data before sending it to the sinks.
+
+## Architecture
+
+Ideal for synchronizing data between different platforms or systems, ensuring
+real-time updates across all configured sinks.
+
+The following image shows the architecture of the Integration Connector Agent.
+
+![architecture](./img/architecture.svg)
+
+### Use Cases
+
+Here will be described some use cases with the related configuration file.
+
+#### Integration with Jira with MongoDB Sink
+
+The following configuration file shows how to integrate Jira with the MongoDB sink, performing a mapper processing.
+
+The pipeline workflow:
+
+1. the Jira source is configured with an authentication secret taken from the environment variable `JIRA_SECRET`;
+1. The input Jira issue event will be mapped to the output event:
+   - `key` will be the issue.key field of the Jira event;
+   - `summary` will be the issue.fields.summary field of the Jira event;
+   - `createdAt` will be the issue.fields.created field of the Jira event;
+   - `description` will be the issue.fields.description field of the Jira event;
+1. The MongoDB sink is configured with the URL taken from the environment variable `MONGO_URL`
+to save data into the collection `jira-issues`.
+
+<details>
+<summary>Jira + MongoDB Sink Configuration</summary>
+
+```json
+{
+	"integrations": [
+		{
+			"type": "jira",
+			"authentication": {
+				"secret": {
+					"fromEnv": "JIRA_SECRET"
+				}
+			},
+			"processors": [
+				{
+					"type": "mapper",
+					"outputEvent": {
+						"key": "{{ issue.key }}",
+						"summary": "{{ issue.fields.summary }}",
+						"createdAt": "{{ issue.fields.created }}",
+						"description": "{{ issue.fields.description }}"
+					}
+				}
+			],
+			"sinks": [
+				{
+					"type": "mongo",
+					"url": {
+						"fromEnv": "MONGO_URL"
+					},
+					"collection": "jira-issues"
+				}
+			]
+		}
+	]
+}
+</details>
+```

docs/20_install.md

@@ -0,0 +1,84 @@
+# How to install
+
+To deploy the service, you need to configure some environment variables and a configuration file.
+
+## Environment Variables
+
+The following environment variables are required to run the application.
+
+| Variable                | Required | Default | Description              |
+|-------------------------|----------|---------|--------------------------|
+| LOG_LEVEL               | ❌       | info    | Log level                 |
+| HTTP_PORT               | ❌       | 8080    | HTTP port                 |
+| HTTP_ADDRESS            | ❌       | 0.0.0.0 | HTTP address exposed by the server  |
+| SERVICE_PREFIX          | ❌       | /       | Prefix for the service endpoints (except for the status and documentation ones) |
+| DELAY_SHUTDOWN_SECONDS  | ❌       | 10      | Delay in seconds before the server shutdown. This could be important to correctly enabling the graceful shutdown in Kubernetes environments |
+| CONFIGURATION_PATH      |✅        |         | The path where it is the configuration file |
+
+## Configuration
+
+The configuration is needed to define the service behavior: the source to integrate
+with, how to process the data, and the sink to store the data.
+
+It is possible to set more than one integration. Each integration is a data pipeline
+which starts from a source, passes through one or more processors, and store data in one or more sink.
+
+To view the possible configurations for each [source](./sources/10_overview.md),
+[processor](./processors/10_overview.md), and [sink](./sinks/10_overview.md), see the related documentation.
+
+### Example Configuration
+
+The following is an example of a configuration for integrate source `Jira` with a processor `mapper` in the `MongoDB` sink.
+
+```json
+{
+  "integrations": [
+    {
+      "type": "jira",
+      "authentication": {
+        "secret": {
+          "fromFile": "testdata/secret"
+        }
+      },
+      "processors": [
+        {
+          "type": "mapper",
+          "outputEvent": {
+            "key": "{{ issue.key }}",
+            "summary": "{{ issue.fields.summary }}",
+            "createdAt": "{{ issue.fields.created }}",
+            "description": "{{ issue.fields.description }}"
+          }
+        }
+      ],
+      "sinks": [
+        {
+          "type": "mongo",
+          "url": {
+            "fromEnv": "TEST_LOAD_SERVICE_MONGO_URL"
+          },
+          "collection": "my-collection"
+        }
+      ]
+    }
+  ]
+}
+```
+
+### Configuration Entities
+
+#### SecretSource
+
+The `SecretSource` is a way to define a secret in the configuration file. It is a JSON object with the following fields:
+
+```json
+{
+  "fromEnv": "ENV_NAME",
+  "fromFile": "file/path"
+}
+```
+
+Only one of the two fields should be defined. The `fromEnv` field is used to get the secret from an environment variable,
+while the `fromFile` field is used to get the secret from a file.
+
+If both are defined, the `fromEnv` field will be used.