Merge branch 'main' into findingsAPIDocumentation

.github/vale/styles/Vocab/OpenSearch/Words/accept.txt

@@ -26,6 +26,7 @@ Boolean
 Dev
 [Dd]iscoverability
 Distro
+[Dd]ownvote(s|d)?
 [Dd]uplicative
 [Ee]gress
 [Ee]num
@@ -122,6 +123,7 @@ stdout
 [Ss]ubvector
 [Ss]ubwords?
 [Ss]uperset
+[Ss]yslog
 tebibyte
 [Tt]emplated
 [Tt]okenization
@@ -138,6 +140,7 @@ tebibyte
 [Uu]nregister(s|ed|ing)?
 [Uu]pdatable
 [Uu]psert
+[Uu]pvote(s|d)?
 [Ww]alkthrough
 [Ww]ebpage
 xy

.github/workflows/vale.yml

@@ -20,4 +20,5 @@ jobs:
           reporter: github-pr-check
           filter_mode: added
           vale_flags: "--no-exit"
-          version: 2.28.0
+          version: 2.28.0
+        continue-on-error: true

TERMS.md

@@ -236,6 +236,8 @@ Do not use *disable* to refer to users.
 
 Always hyphenated. Don’t use _double click_.
 
+**downvote**
+
 **dropdown list**
 
 **due to**
@@ -586,6 +588,10 @@ Use % in headlines, quotations, and tables or in technical copy.
 
 An agent and REST API that allows you to query numerous performance metrics for your cluster, including aggregations of those metrics, independent of the Java Virtual Machine (JVM).
 
+**plaintext, plain text**
+
+Use *plaintext* only to refer to nonencrypted or decrypted text in content about encryption. Use *plain text* to refer to ASCII files.
+
 **please**
 
 Avoid using except in quoted text.
@@ -700,6 +706,8 @@ Never hyphenated. Use _startup_ as a noun (for example, “The following startup
 
 **Stochastic Gradient Descent (SGD)**
 
+**syslog**
+
 ## T
 
 **term frequency–inverse document frequency (TF–IDF)**
@@ -746,6 +754,8 @@ A storage tier that you can use to store and analyze your data with Elasticsearc
 
 Hyphenate as adjectives. Use instead of *top left* and *top right*, unless the field name uses *top*. For example, "The upper-right corner."
 
+**upvote**
+
 **US**
 
 No periods, as specified in the Chicago Manual of Style.

_api-reference/index-apis/force-merge.md

@@ -72,6 +72,7 @@ The following table lists the available query parameters. All query parameters a
 | `ignore_unavailable` | Boolean | If `true`, OpenSearch ignores missing or closed indexes. If `false`, OpenSearch returns an error if the force merge operation encounters missing or closed indexes. Default is `false`. |
 | `max_num_segments` | Integer | The number of larger segments into which smaller segments are merged. Set this parameter to `1` to merge all segments into one segment. The default behavior is to perform the merge as necessary. |
 | `only_expunge_deletes` | Boolean | If `true`, the merge operation only expunges segments containing a certain percentage of deleted documents. The percentage is 10% by default and is configurable in the `index.merge.policy.expunge_deletes_allowed` setting. Prior to OpenSearch 2.12, `only_expunge_deletes` ignored the `index.merge.policy.max_merged_segment` setting. Starting with OpenSearch 2.12, using `only_expunge_deletes` does not produce segments larger than `index.merge.policy.max_merged_segment` (by default, 5 GB). For more information, see [Deleted documents](#deleted-documents). Default is `false`. |
+| `primary_only` | Boolean | If set to `true`, then the merge operation is performed only on the primary shards of an index. This can be useful when you want to take a snapshot of the index after the merge is complete. Snapshots only copy segments from the primary shards. Merging the primary shards can reduce resource consumption. Default is `false`. |
 
 #### Example request: Force merge a specific index
 
@@ -101,6 +102,13 @@ POST /.testindex-logs/_forcemerge?max_num_segments=1
 ```
 {% include copy-curl.html %}
 
+#### Example request: Force merge primary shards
+
+```json
+POST /.testindex-logs/_forcemerge?primary_only=true
+```
+{% include copy-curl.html %}
+
 #### Example response
 
 ```json

_automating-configurations/api/create-workflow.md

@@ -7,9 +7,6 @@ nav_order: 10
 
 # Create or update a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 Creating a workflow adds the content of a workflow template to the flow framework system index. You can provide workflows in JSON format (by specifying `Content-Type: application/json`) or YAML format (by specifying `Content-Type: application/yaml`). By default, the workflow is validated to help identify invalid configurations, including:
 
 * Workflow steps requiring an OpenSearch plugin that is not installed.
@@ -19,6 +16,8 @@ Creating a workflow adds the content of a workflow template to the flow framewor
 
 To obtain the validation template for workflow steps, call the [Get Workflow Steps API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-steps/).
 
+You can include placeholder expressions in the value of workflow step fields. For example, you can specify a credential field in a template as `openAI_key: '${{ openai_key }}'`. The expression will be substituted with the user-provided value during provisioning, using the format `${{ <value> }}`. You can pass the actual key as a parameter using the [Provision Workflow API]({{site.url}}{{site.baseurl}}/automating-configurations/api/provision-workflow/) or using this API with the `provision` parameter set to `true`.
+
 Once a workflow is created, provide its `workflow_id` to other APIs.
 
 The `POST` method creates a new workflow. The `PUT` method updates an existing workflow. 
@@ -59,12 +58,13 @@ POST /_plugins/_flow_framework/workflow?validation=none
 ```
 {% include copy-curl.html %}
 
-The following table lists the available query parameters. All query parameters are optional.
+The following table lists the available query parameters. All query parameters are optional. User-provided parameters are only allowed if the `provision` parameter is set to `true`.
 
 | Parameter | Data type | Description |
 | :--- | :--- | :--- |
 | `provision` | Boolean | Whether to provision the workflow as part of the request. Default is `false`. |
 | `validation` | String | Whether to validate the workflow. Valid values are `all` (validate the template) and `none` (do not validate the template). Default is `all`. |
+| User-provided substitution expressions | String | Parameters matching substitution expressions in the template. Only allowed if `provision` is set to `true`. Optional. If `provision` is set to `false`, you can pass these parameters in the [Provision Workflow API query parameters]({{site.url}}{{site.baseurl}}/automating-configurations/api/provision-workflow/#query-parameters). |
 
 ## Request fields
 

_automating-configurations/api/delete-workflow.md

@@ -7,9 +7,6 @@ nav_order: 80
 
 # Delete a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 When you no longer need a workflow template, you can delete it by calling the Delete Workflow API. 
 
 Note that deleting a workflow only deletes the stored template but does not deprovision its resources.  

_automating-configurations/api/deprovision-workflow.md

@@ -7,9 +7,6 @@ nav_order: 70
 
 # Deprovision a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 When you no longer need a workflow, you can deprovision its resources. Most workflow steps that create a resource have corresponding workflow steps to reverse that action. To retrieve all resources currently created for a workflow, call the [Get Workflow Status API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-status/). When you call the Deprovision Workflow API, resources included in the `resources_created` field of the Get Workflow Status API response will be removed using a workflow step corresponding to the one that provisioned them.
 
 The workflow executes the provisioning workflow steps in reverse order. If failures occur because of resource dependencies, such as preventing deletion of a registered model if it is still deployed, the workflow attempts retries.

_automating-configurations/api/get-workflow-status.md

@@ -7,9 +7,6 @@ nav_order: 40
 
 # Get a workflow status
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 [Provisioning a workflow]({{site.url}}{{site.baseurl}}/automating-configurations/api/provision-workflow/) may take a significant amount of time, particularly when the action is associated with OpenSearch indexing operations. The Get Workflow State API permits monitoring of the provisioning deployment status until it is complete.
 
 ## Path and HTTP methods

_automating-configurations/api/get-workflow-steps.md

@@ -7,10 +7,7 @@ nav_order: 50
 
 # Get workflow steps
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
-OpenSearch validates workflows by using the validation template that lists the required inputs, generated outputs, and required plugins for all steps. For example, for the `register_remote_model` step, the validation template appears as follows:
+This API returns a list of workflow steps, including their required inputs, outputs, default timeout values, and required plugins. For example, for the `register_remote_model` step, the Get Workflow Steps API returns the following information:
 
 ```json
 {
@@ -28,36 +25,52 @@ OpenSearch validates workflows by using the validation template that lists the r
     ]
   }
 }
-```
-
-The Get Workflow Steps API retrieves this file.   
+``` 
 
 ## Path and HTTP methods
 
 ```json
 GET /_plugins/_flow_framework/workflow/_steps
+GET /_plugins/_flow_framework/workflow/_step?workflow_step=<step_name>
 ``` 
 
+## Query parameters
+
+The following table lists the available query parameters. All query parameters are optional.
+
+| Parameter | Data type | Description |
+| :--- | :--- | :--- |
+| `workflow_step` | String | The name of the step to retrieve. Specify multiple step names as a comma-separated list. For example, `create_connector,delete_model,deploy_model`. |
+
 #### Example request
 
+To fetch all workflow steps, use the following request:
+
 ```json
 GET /_plugins/_flow_framework/workflow/_steps
+``` 
+{% include copy-curl.html %}
+
+To fetch specific workflow steps, pass the step names to the request as a query parameter:
+
+```json
+GET /_plugins/_flow_framework/workflow/_step?workflow_step=create_connector,delete_model,deploy_model
 ```
 {% include copy-curl.html %}
 
 
 #### Example response
 
-OpenSearch responds with the validation template containing the steps. The order of fields in the returned steps may not exactly match the original JSON but will function identically.
+OpenSearch responds with the workflow steps. The order of fields in the returned steps may not exactly match the original JSON but will function identically.
 
 To retrieve the template in YAML format, specify `Content-Type: application/yaml` in the request header:
 
 ```bash
-curl -XGET "http://localhost:9200/_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50" -H 'Content-Type: application/yaml'
+curl -XGET "http://localhost:9200/_plugins/_flow_framework/workflow/_steps" -H 'Content-Type: application/yaml'
 ```
 
 To retrieve the template in JSON format, specify `Content-Type: application/json` in the request header:
 
 ```bash
-curl -XGET "http://localhost:9200/_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50" -H 'Content-Type: application/json'
+curl -XGET "http://localhost:9200/_plugins/_flow_framework/workflow/_steps" -H 'Content-Type: application/json'
 ```

_automating-configurations/api/get-workflow.md

@@ -7,9 +7,6 @@ nav_order: 20
 
 # Get a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 The Get Workflow API retrieves the workflow template.   
 
 ## Path and HTTP methods

_automating-configurations/api/index.md

@@ -8,9 +8,6 @@ has_toc: false
 
 # Workflow APIs
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 OpenSearch supports the following workflow APIs:
 
 * [Create or update workflow]({{site.url}}{{site.baseurl}}/automating-configurations/api/create-workflow/)

_automating-configurations/api/provision-workflow.md

@@ -7,9 +7,6 @@ nav_order: 30
 
 # Provision a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 Provisioning a workflow is a one-time setup process usually performed by a cluster administrator to create resources that will be used by end users.  
 
 The `workflows` template field may contain multiple workflows. The workflow with the `provision` key can be executed with this API. This API is also executed when the [Create or Update Workflow API]({{site.url}}{{site.baseurl}}/automating-configurations/api/create-workflow/) is called with the `provision` parameter set to `true`.
@@ -31,10 +28,39 @@ The following table lists the available path parameters.
 | :--- | :--- | :--- |
 | `workflow_id` | String | The ID of the workflow to be provisioned. Required. |
 
-#### Example request
+## Query parameters
+
+If you have included a substitution expression in the template, you may pass it as a query parameter or as a string value of a request body field. For example, if you specified a credential field in a template as `openAI_key: '${{ openai_key }}'`, then you can include the `openai_key` parameter as a query parameter or body field so it can be substituted during provisioning. For example, the following request provides a query parameter:
+
+```json
+POST /_plugins/_flow_framework/workflow/<workflow_id>/_provision?<parameter>=<value>
+```
+
+| Parameter | Data type | Description |
+| :--- | :--- | :--- |
+| User-provided substitution expressions | String | Parameters matching substitution expressions in the template. Optional. |
+
+#### Example requests
+
+```json
+POST /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_provision
+```
+{% include copy-curl.html %}
+
+The following request substitutes the expression `${{ openai_key }}` with the value "12345" using a query parameter:
+
+```json
+POST /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_provision?openai_key=12345
+```
+{% include copy-curl.html %}
+
+The following request substitutes the expression `${{ openai_key }}` with the value "12345" using the request body:
 
 ```json
 POST /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_provision
+{
+  "openai_key" : "12345"
+}
 ```
 {% include copy-curl.html %}
 

_automating-configurations/api/search-workflow-state.md

@@ -7,9 +7,6 @@ nav_order: 65
 
 # Search for a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 You can search for resources created by workflows by matching a query to a field. The fields you can search correspond to those returned by the [Get Workflow Status API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-status/).
 
 ## Path and HTTP methods

_automating-configurations/api/search-workflow.md

@@ -7,9 +7,6 @@ nav_order: 60
 
 # Search for a workflow
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 You can retrieve created workflows with their `workflow_id` or search for workflows by using a query matching a field. You can use the `use_case` field to search for similar workflows.
 
 ## Path and HTTP methods

_automating-configurations/index.md

@@ -11,9 +11,6 @@ redirect_from: /automating-configurations/
 **Introduced 2.12**
 {: .label .label-purple }
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 You can automate complex OpenSearch setup and preprocessing tasks by providing templates for common use cases. For example, automating machine learning (ML) setup tasks streamlines the use of OpenSearch ML offerings.
 
 In OpenSearch 2.12, configuration automation is limited to ML tasks.

_automating-configurations/workflow-settings.md

@@ -6,9 +6,6 @@ nav_order: 30
 
 # Workflow settings
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 The following keys represent configurable workflow settings.
 
 |Setting	|Data type	|Default value	|Description	|

_automating-configurations/workflow-steps.md

@@ -6,9 +6,6 @@ nav_order: 10
 
 # Workflow steps
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/flow-framework/issues/475).    
-{: .warning}
-
 _Workflow steps_ form basic "building blocks" for process automation. Most steps directly correspond to OpenSearch or plugin API operations, such as CRUD operations on machine learning (ML) connectors, models, and agents. Some steps simplify the configuration by reusing the body expected by these APIs across multiple steps. For example, once you configure a _tool_, you can use it with multiple _agents_.  
 
 ## Workflow step fields
@@ -42,6 +39,9 @@ The following table lists the workflow step types. The `user_inputs` fields for
 |`register_agent`	|[Register Agent API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/)	|Registers an agent as part of the ML Commons Agent Framework.	|
 |`delete_agent`	|[Delete Agent API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/)	|Deletes an agent.	|
 |`create_tool`	|No API	| A special-case non-API step encapsulating the specification of a tool for an agent in the ML Commons Agent Framework. These will be listed as `previous_node_inputs` for the appropriate register agent step, with the value set to `tools`.	|
+|`create_index`|[Create Index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/)     | Creates a new OpenSearch index. The inputs include `index_name`, which should be the name of the index to be created, and `configurations`, which contains the payload body of a regular REST request for creating an index.
+|`create_ingest_pipeline`|[Create Ingest Pipeline]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/) | Creates or updates an ingest pipeline. The inputs include `pipeline_id`, which should be the ID of the pipeline, and `configurations`, which contains the payload body of a regular REST request for creating an ingest pipeline.
+|`create_search_pipeline`|[Create Search Pipeline]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/creating-search-pipeline/) | Creates or updates a search pipeline. The inputs include `pipeline_id`, which should be the ID of the pipeline, and `configurations`, which contains the payload body of a regular REST request for creating a search pipeline.
 
 ## Additional fields