Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for source mode to various tracks #692

Merged
merged 12 commits into from
Oct 25, 2024

Conversation

martijnvg
Copy link
Member

@martijnvg martijnvg commented Oct 14, 2024

Add support for source mode to elastic/logs, elastic/security and http_logs.

@martijnvg martijnvg added the backport-to-8.15 Automatically backport to 8.15 branch label Oct 14, 2024
"mappings": {
{% if source_mode %}
"_source": {
"mapping.source.mode": {{ source_mode | tojson }}
Copy link
Contributor

@salvatore-campagna salvatore-campagna Oct 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this...Did we introduce something I am not aware of?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a silly mistake on my side.

@@ -221,6 +221,7 @@ The following parameters are available:
* `lifecycle` (default: unset to fall back on Serverless detection) - Specifies the lifecycle management feature to use for data streams. Use `ilm` for index lifecycle management or `dlm` for data lifecycle management. By default, `dlm` will be used for benchmarking Serverless Elasticsearch.
* `workflow-request-cache` (default: `true`) - Explicit control of request cache query parameter in searches executed in a workflow. This can be further overriden at an operation level with `request-cache` parameter.
* `synthetic_source_keep` (default: unset) - Allows overriding the default synthetic source behaviour for all field types with the following values: `none` (equivalent to unset) - no source is stored, `arrays` - source stored as is only for multi-value (array) fields.
* `source_mode` (default: unset) - Specified the source mode to be used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "Specifies"?

@@ -85,6 +85,7 @@ The following parameters are available:
* `corpora_uri_base` (default: `https://rally-tracks.elastic.co`) - Specify the base location of the datasets used by this track.
* `index_mode` (default: unset) - A parameter meant to be used internally which defines one of the available indexing modes, "standard", "logsdb" or "time_series". If not set, "standard" is used.
* `synthetic_source_keep` (default: unset) - Allows overriding the default synthetic source behaviour for all field types with the following values: `none` (equivalent to unset) - no source is stored, `arrays` - source stored as is only for multi-value (array) fields.
* `source_mode` (default: unset) - Specified the source mode to be used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit same here and in other places

@@ -20,6 +20,9 @@
{%- else %}
"_source": {
"enabled": {{ source_enabled | default(true) | tojson }}
{% if source_mode %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the setting missing in this track?

@salvatore-campagna
Copy link
Contributor

LGTM: I just left a few nits and a question about http_logs.

@martijnvg martijnvg changed the title Add support index.mapping.source.mode setting to various tracks Add support for stored source mode to various tracks Oct 25, 2024
@martijnvg martijnvg changed the title Add support for stored source mode to various tracks Add support for source mode to various tracks Oct 25, 2024
@martijnvg martijnvg merged commit ae63824 into elastic:master Oct 25, 2024
13 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 25, 2024
Add support for source mode to elastic/logs, elastic/security and http_logs.

(cherry picked from commit ae63824)
Copy link

💚 All backports created successfully

Status Branch Result
8.15

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details

martijnvg pushed a commit that referenced this pull request Oct 30, 2024
Add support for source mode to elastic/logs, elastic/security and http_logs.
Backport of #692 to 8.15 branch.

(cherry picked from commit ae63824)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-to-8.15 Automatically backport to 8.15 branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants