Skip to content

Commit

Permalink
Standardize and add SSL settings (#168)
Browse files Browse the repository at this point in the history 
This commit made the plugin SSL settings consistent with the naming convention defined in the meta issue: elastic/logstash#14905.

It added the following SSL settings:
ssl_enabled: Enable/disable the SSL settings. If not provided, the value is inferred from the hosts' scheme
ssl_certificate: OpenSSL-style X.509 certificate file to authenticate the client
ssl_key: OpenSSL-style RSA private key that corresponds to the ssl_certificate
ssl_truststore_path: The JKS truststore to validate the server's certificate
ssl_truststore_type: The format of the truststore file
ssl_truststore_password: The truststore password
ssl_keystore_path: The keystore used to present a certificate to the server
ssl_keystore_type: The format of the keystore file
ssl_keystore_password: The keystore password
ssl_cipher_suites: The list of cipher suites to use
ssl_supported_protocols: Supported protocols with versions
ssl_verification_mode: Defines how to verify the certificates presented by another party in the TLS connection

And deprecated:
ssl in favor of ssl_enabled
ca_file in favor of ssl_certificate_authorities
keystore in favor of ssl_keystore_path
keystore_password in favor of ssl_keystore_password
  • Loading branch information
@edmocosta
edmocosta authored Mar 10, 2023
1 parent 6ff00f0 commit 4da7f62
Show file tree
Hide file tree
Showing 9 changed files with 669 additions and 71 deletions.
11 changes: 8 additions & 3 deletions .ci/logstash-run.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,14 @@ wait_for_es() {
}

if [[ "$INTEGRATION" != "true" ]]; then
jruby -rbundler/setup -S rspec -fd -t ~integration spec/filters
bundle exec rspec --format=documentation spec/filters --tag ~integration --tag ~secure_integration
else
extra_tag_args="-t integration"
if [[ "$SECURE_INTEGRATION" == "true" ]]; then
extra_tag_args="--tag secure_integration"
else
extra_tag_args="--tag ~secure_integration --tag integration"
fi

wait_for_es
jruby -rbundler/setup -S rspec -fd $extra_tag_args -t es_version:$ELASTIC_STACK_VERSION spec/filters/integration
bundle exec rspec --format=documentation $extra_tag_args --tag update_tests:painless --tag es_version:$ELASTIC_STACK_VERSION spec/filters/integration
fi
20 changes: 20 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,23 @@
## 3.15.0
- Added SSL settings for: [#168](https://github.com/logstash-plugins/logstash-filter-elasticsearch/pull/168)
- `ssl_enabled`: Enable/disable the SSL settings. If not provided, the value is inferred from the hosts scheme
- `ssl_certificate`: OpenSSL-style X.509 certificate file to authenticate the client
- `ssl_key`: OpenSSL-style RSA private key that corresponds to the `ssl_certificate`
- `ssl_truststore_path`: The JKS truststore to validate the server's certificate
- `ssl_truststore_type`: The format of the truststore file
- `ssl_truststore_password`: The truststore password
- `ssl_keystore_path`: The keystore used to present a certificate to the server
- `ssl_keystore_type`: The format of the keystore file
- `ssl_keystore_password`: The keystore password
- `ssl_cipher_suites`: The list of cipher suites to use
- `ssl_supported_protocols`: Supported protocols with versions
- `ssl_verification_mode`: Defines how to verify the certificates presented by another party in the TLS connection
- Reviewed and deprecated SSL settings to comply with Logstash's naming convention
- Deprecated `ssl` in favor of `ssl_enabled`
- Deprecated `ca_file` in favor of `ssl_certificate_authorities`
- Deprecated `keystore` in favor of `ssl_keystore_path`
- Deprecated `keystore_password` in favor of `ssl_keystore_password`

## 3.14.0
- Added support for configurable retries with new `retry_on_failure` and `retry_on_status` options [#160](https://github.com/logstash-plugins/logstash-filter-elasticsearch/pull/160)

Expand Down
214 changes: 189 additions & 25 deletions docs/index.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,14 +121,13 @@ The `monitoring` permission at cluster level is necessary to perform periodic co
[id="plugins-{type}s-{plugin}-options"]
==== Elasticsearch Filter Configuration Options

This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> and the <<plugins-{type}s-{plugin}-deprecated-options>> described later.

[cols="<,<,<",options="header",]
|=======================================================================
|Setting |Input type|Required
| <<plugins-{type}s-{plugin}-aggregation_fields>> |<<hash,hash>>|No
| <<plugins-{type}s-{plugin}-api_key>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-ca_file>> |a valid filesystem path|No
| <<plugins-{type}s-{plugin}-ca_trusted_fingerprint>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-cloud_auth>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-cloud_id>> |<<string,string>>|No
Expand All @@ -143,11 +142,22 @@ This plugin supports the following configuration options plus the <<plugins-{typ
| <<plugins-{type}s-{plugin}-query_template>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-result_size>> |<<number,number>>|No
| <<plugins-{type}s-{plugin}-retry_on_failure>> |<<number,number>>|No
| <<plugins-{type}s-{plugin}-result_on_status_>> |<<number,number list>>|No
| <<plugins-{type}s-{plugin}-retry_on_status>> |<<number,number list>>|No
| <<plugins-{type}s-{plugin}-sort>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|No
| <<plugins-{type}s-{plugin}-keystore>> |a valid filesystem path|No
| <<plugins-{type}s-{plugin}-keystore_password>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-ssl>> |<<boolean,boolean>>|__Deprecated__
| <<plugins-{type}s-{plugin}-ssl_certificate>> |<<path,path>>|No
| <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> |list of <<path,path>>|No
| <<plugins-{type}s-{plugin}-ssl_cipher_suites>> |list of <<string,string>>|No
| <<plugins-{type}s-{plugin}-ssl_enabled>> |<<boolean,boolean>>|No
| <<plugins-{type}s-{plugin}-ssl_key>> |<<path,path>>|No
| <<plugins-{type}s-{plugin}-ssl_keystore_password>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-ssl_keystore_path>> |<<path,path>>|No
| <<plugins-{type}s-{plugin}-ssl_keystore_type>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-ssl_supported_protocols>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-ssl_truststore_password>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-ssl_truststore_path>> |<<path,path>>|No
| <<plugins-{type}s-{plugin}-ssl_truststore_type>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-ssl_verification_mode>> |<<string,string>>, one of `["full", "none"]`|No
| <<plugins-{type}s-{plugin}-tag_on_failure>> |<<array,array>>|No
| <<plugins-{type}s-{plugin}-user>> |<<string,string>>|No
|=======================================================================
Expand Down Expand Up @@ -182,19 +192,11 @@ Example:
* There is no default value for this setting.

Authenticate using Elasticsearch API key. Note that this option also requires
enabling the `ssl` option.
enabling the <<plugins-{type}s-{plugin}-ssl_enabled>> option.

Format is `id:api_key` where `id` and `api_key` are as returned by the
Elasticsearch {ref}/security-api-create-api-key.html[Create API key API].

[id="plugins-{type}s-{plugin}-ca_file"]
===== `ca_file`

* Value type is <<path,path>>
* There is no default value for this setting.

SSL Certificate Authority file

[id="plugins-{type}s-{plugin}-ca_trusted_fingerprint"]
===== `ca_trusted_fingerprint`

Expand Down Expand Up @@ -364,30 +366,140 @@ Which HTTP Status codes to consider for retries (in addition to connection error

Comma-delimited list of `<field>:<direction>` pairs that define the sort order

[id="plugins-{type}s-{plugin}-ssl"]
===== `ssl`
[id="plugins-{type}s-{plugin}-ssl_certificate"]
===== `ssl_certificate`
* Value type is <<path,path>>
* There is no default value for this setting.

* Value type is <<boolean,boolean>>
* Default value is `false`
SSL certificate to use to authenticate the client. This certificate should be an OpenSSL-style X.509 certificate file.

SSL
NOTE: This setting can be used only if <<plugins-{type}s-{plugin}-ssl_key>> is set.

[id="plugins-{type}s-{plugin}-keystore"]
===== `keystore`
[id="plugins-{type}s-{plugin}-ssl_certificate_authorities"]
===== `ssl_certificate_authorities`

* Value type is a list of <<path,path>>
* There is no default value for this setting

The .cer or .pem files to validate the server's certificate.

NOTE: You cannot use this setting and <<plugins-{type}s-{plugin}-ssl_truststore_path>> at the same time.

[id="plugins-{type}s-{plugin}-ssl_cipher_suites"]
===== `ssl_cipher_suites`
* Value type is a list of <<string,string>>
* There is no default value for this setting

The list of cipher suites to use, listed by priorities.
Supported cipher suites vary depending on the Java and protocol versions.


[id="plugins-{type}s-{plugin}-ssl_enabled"]
===== `ssl_enabled`

* Value type is <<boolean,boolean>>
* There is no default value for this setting.

Enable SSL/TLS secured communication to Elasticsearch cluster.
Leaving this unspecified will use whatever scheme is specified in the URLs listed in <<plugins-{type}s-{plugin}-hosts>> or extracted from the <<plugins-{type}s-{plugin}-cloud_id>>.
If no explicit protocol is specified plain HTTP will be used.

[id="plugins-{type}s-{plugin}-ssl_key"]
===== `ssl_key`
* Value type is <<path,path>>
* There is no default value for this setting.

The keystore used to present a certificate to the server. It can be either .jks or .p12
OpenSSL-style RSA private key that corresponds to the <<plugins-{type}s-{plugin}-ssl_certificate>>.

[id="plugins-{type}s-{plugin}-keystore_password"]
===== `keystore_password`
NOTE: This setting can be used only if <<plugins-{type}s-{plugin}-ssl_certificate>> is set.

[id="plugins-{type}s-{plugin}-ssl_keystore_password"]
===== `ssl_keystore_password`

* Value type is <<password,password>>
* There is no default value for this setting.

Set the keystore password

[id="plugins-{type}s-{plugin}-ssl_keystore_path"]
===== `ssl_keystore_path`

* Value type is <<path,path>>
* There is no default value for this setting.

The keystore used to present a certificate to the server.
It can be either `.jks` or `.p12`

NOTE: You cannot use this setting and <<plugins-{type}s-{plugin}-ssl_certificate>> at the same time.

[id="plugins-{type}s-{plugin}-ssl_keystore_type"]
===== `ssl_keystore_type`

* Value can be any of: `jks`, `pkcs12`
* If not provided, the value will be inferred from the keystore filename.

The format of the keystore file. It must be either `jks` or `pkcs12`.

[id="plugins-{type}s-{plugin}-ssl_supported_protocols"]
===== `ssl_supported_protocols`

* Value type is <<string,string>>
* Allowed values are: `'TLSv1.1'`, `'TLSv1.2'`, `'TLSv1.3'`
* Default depends on the JDK being used. With up-to-date Logstash, the default is `['TLSv1.2', 'TLSv1.3']`.
`'TLSv1.1'` is not considered secure and is only provided for legacy applications.

List of allowed SSL/TLS versions to use when establishing a connection to the Elasticsearch cluster.

For Java 8 `'TLSv1.3'` is supported only since **8u262** (AdoptOpenJDK), but requires that you set the
`LS_JAVA_OPTS="-Djdk.tls.client.protocols=TLSv1.3"` system property in Logstash.

NOTE: If you configure the plugin to use `'TLSv1.1'` on any recent JVM, such as the one packaged with Logstash,
the protocol is disabled by default and needs to be enabled manually by changing `jdk.tls.disabledAlgorithms` in
the *$JDK_HOME/conf/security/java.security* configuration file. That is, `TLSv1.1` needs to be removed from the list.

[id="plugins-{type}s-{plugin}-ssl_truststore_password"]
===== `ssl_truststore_password`

* Value type is <<password,password>>
* There is no default value for this setting.

Set the truststore password

[id="plugins-{type}s-{plugin}-ssl_truststore_path"]
===== `ssl_truststore_path`

* Value type is <<path,path>>
* There is no default value for this setting.

The truststore to validate the server's certificate.
It can be either `.jks` or `.p12`.

NOTE: You cannot use this setting and <<plugins-{type}s-{plugin}-ssl_certificate_authorities>> at the same time.

[id="plugins-{type}s-{plugin}-ssl_truststore_type"]
===== `ssl_truststore_type`

* Value can be any of: `jks`, `pkcs12`
* If not provided, the value will be inferred from the truststore filename.

The format of the truststore file. It must be either `jks` or `pkcs12`.

[id="plugins-{type}s-{plugin}-ssl_verification_mode"]
===== `ssl_verification_mode`

* Value can be any of: `full`, `none`
* Default value is `full`

Defines how to verify the certificates presented by another party in the TLS connection:

`full` validates that the server certificate has an issue date that’s within
the not_before and not_after dates; chains to a trusted Certificate Authority (CA), and
has a hostname or IP address that matches the names within the certificate.

`none` performs no certificate validation.

WARNING: Setting certificate verification to `none` disables many security benefits of SSL/TLS, which is very dangerous. For more information on disabling certificate verification please read https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf

[id="plugins-{type}s-{plugin}-tag_on_failure"]
===== `tag_on_failure`

Expand All @@ -405,5 +517,57 @@ Tags the event on failure to look up previous log event information. This can be
Basic Auth - username


[id="plugins-{type}s-{plugin}-deprecated-options"]
==== Elasticsearch Filter Deprecated Configuration Options

This plugin supports the following deprecated configurations.

WARNING: Deprecated options are subject to removal in future releases.

[cols="<,<,<",options="header",]
|=======================================================================
|Setting|Input type|Replaced by
| <<plugins-{type}s-{plugin}-ca_file>> |a valid filesystem path|<<plugins-{type}s-{plugin}-ssl_certificate_authorities>>
| <<plugins-{type}s-{plugin}-keystore>> |a valid filesystem path|<<plugins-{type}s-{plugin}-ssl_keystore_path>>
| <<plugins-{type}s-{plugin}-keystore_password>> |<<password,password>>|<<plugins-{type}s-{plugin}-ssl_keystore_password>>
|=======================================================================

[id="plugins-{type}s-{plugin}-ca_file"]
===== `ca_file`
deprecated[3.15.0, Replaced by <<plugins-{type}s-{plugin}-ssl_certificate_authorities>>]

* Value type is <<path,path>>
* There is no default value for this setting.

SSL Certificate Authority file

[id="plugins-{type}s-{plugin}-ssl"]
===== `ssl`
deprecated[3.15.0, Replaced by <<plugins-{type}s-{plugin}-ssl_enabled>>]

* Value type is <<boolean,boolean>>
* Default value is `false`

SSL

[id="plugins-{type}s-{plugin}-keystore"]
===== `keystore`
deprecated[3.15.0, Replaced by <<plugins-{type}s-{plugin}-ssl_keystore_path>>]

* Value type is <<path,path>>
* There is no default value for this setting.

The keystore used to present a certificate to the server. It can be either .jks or .p12

[id="plugins-{type}s-{plugin}-keystore_password"]
===== `keystore_password`
deprecated[3.15.0, Replaced by <<plugins-{type}s-{plugin}-ssl_keystore_password>>]

* Value type is <<password,password>>
* There is no default value for this setting.

Set the keystore password


[id="plugins-{type}s-{plugin}-common-options"]
include::{include_path}/{type}.asciidoc[]
Loading

0 comments on commit 4da7f62

Please sign in to comment.