Skip to content

Commit

Permalink
Merge pull request #2998 from spryker/profuel-patch-5
Browse files Browse the repository at this point in the history
Update configure-elasticsearch.md
  • Loading branch information
andriitserkovnyi authored Jan 3, 2025
2 parents b71a51a + 90e1554 commit 3c501f6
Show file tree
Hide file tree
Showing 22 changed files with 79 additions and 51 deletions.
4 changes: 2 additions & 2 deletions _includes/checking-elasticsearch-indices-via-a-cli.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
### Check ElasticSearch connection details
### Check Elasticsearch connection details

Check `SPRYKER_SEARCH_HOST ` and `SPRYKER_SEARCH_PORT` values as follows:

{% include checking-a-service-connection-configuration.md %}

### Check the health and status of ElasticSearch indices
### Check the health and status of Elasticsearch indices

1. Connect to the environment’s VPN.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
1. In the AWS Management Console, go to **Services** > **Amazon OpenSearch Service**.
2. Select the desired environment.
3. On the page of the environment, check the following:
- *Domain status:* If the status is *Active*, ElasticSearch is running.
- *Cluster health*: If the status is *Green* or *Yellow*, ElasticSearch works properly.
- *Domain status:* If the status is *Active*, Elasticsearch is running.
- *Cluster health*: If the status is *Green* or *Yellow*, Elasticsearch works properly.

4. Below the *General information* pane, switch to the **Cluster health** tab.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ class ProductPageSearchConfig extends SprykerProductPageSearchConfig

{% info_block warningBox "Verification" %}

Make sure that abstract products that can be added to cart have the `add_to_cart_sku` field in the ElasticSearch document.
Make sure that abstract products that can be added to cart have the `add_to_cart_sku` field in the Elasticsearch document.

{% endinfo_block %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Spryker\Zed\ProductPageSearch\Communication\Plugin\Search\ProductPageMapPlugin

## Upgrading from version 2.* to version 3.*

`ProductPageSearch` 3.0.0 got separate search index for Concrete Products. It includes database table and ElasticSearch index.
`ProductPageSearch` 3.0.0 got separate search index for Concrete Products. It includes database table and Elasticsearch index.

*Estimated migration time: ~2h*

Expand All @@ -83,7 +83,7 @@ vendor/bin/console transfer:generate
vendor/bin/console search:setup
```

4. Sync concrete products data with ElasticSearch:
4. Sync concrete products data with Elasticsearch:

```bash
vendor/bin/console data:import:product-concrete
Expand Down
2 changes: 1 addition & 1 deletion docs/ca/dev/create-and-restore-database-backups.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ This opens the **Jobs** page where you can see the progress of restoration. Wait
Restored DB is created and modified.


### Re-sync data from the restored database to ElasticSearch and Redis
### Re-sync data from the restored database to Elasticsearch and Redis

1. Connect to OpenVPN.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ The following alerts are configured by default for all environments.

</div>

### ElasticSearch
### Elasticsearch

<div class="width-100">

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ redirect_from:
---
Zed didn't restart after a deployment or you can't access it.

To troubleshoot this issue, you need to go through all the stages of information flow. The default information flow is: Front end > Gateway(optional) > Zed > ElastiCache, ElasticSearch, RMQ, and RDS.
To troubleshoot this issue, you need to go through all the stages of information flow. The default information flow is: Front end > Gateway(optional) > Zed > ElastiCache, Elasticsearch, RMQ, and RDS.

![information flow diagram](https://spryker.s3.eu-central-1.amazonaws.com/cloud-docs/_includes/informatin-flow-diagram.png)

Expand Down Expand Up @@ -85,15 +85,15 @@ To check Redis system information via a CLI, do the following.
{% include checking-redis-system-information-via-a-cli.md %} <!-- To edit, see /_includes/checking-redis-system-information-via-a-cli.md -->


## 6. Check ElasticSearch status
## 6. Check Elasticsearch status

Check ElasticSearch status via AWS Management Console:
Check Elasticsearch status via AWS Management Console:

{% include checking-elasticsearch-status-via-aws-management-console.md %} <!-- To edit, see /_includes/checking-elasticsearch-status-via-aws-management-console.md -->

## 7. Check ElasticSearch indices
## 7. Check Elasticsearch indices

To check ElasticSearch indices via a CLI, do the following.
To check Elasticsearch indices via a CLI, do the following.

{% include checking-elasticsearch-indices-via-a-cli.md %} <!-- To edit, see /_includes/checking-elasticsearch-indices-via-a-cli.md -->

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: Tutorial — Troubleshooting unavailable Yves or Glue
description: Troubleshoot unavailable Yves or Glue in Spryker by checking logs, ECS services, Redis, and ElasticSearch to identify and resolve issues quickly.
description: Troubleshoot unavailable Yves or Glue in Spryker by checking logs, ECS services, Redis, and Elasticsearch to identify and resolve issues quickly.
template: troubleshooting-guide-template
last_updated: Oct 6, 2023
redirect_from:
- /docs/cloud/dev/spryker-cloud-commerce-os/troubleshooting/troubleshooting-tutorials/tutorial-troubleshooting-unavailable-yves-or-glue.html
---
Yves or Glue didn't restart after a deployment or you can't access it.

To troubleshoot this issue, you need to go through all the stages of information flow. The default information flow is front end > Yves or Glue > ElastiCache, ElasticSearch, and Zed.
To troubleshoot this issue, you need to go through all the stages of information flow. The default information flow is front end > Yves or Glue > ElastiCache, Elasticsearch, and Zed.

![information flow diagram](https://spryker.s3.eu-central-1.amazonaws.com/cloud-docs/_includes/informatin-flow-diagram.png)

Expand Down Expand Up @@ -43,14 +43,14 @@ Check Redis system information via a CLI as follows.

{% include checking-redis-system-information-via-a-cli.md %} <!-- To edit, see /_includes/checking-redis-system-information-via-a-cli.md -->

## 6. Check ElasticSearch status
## 6. Check Elasticsearch status

Check ElasticSearch status via AWS Management Console:
Check Elasticsearch status via AWS Management Console:

{% include checking-elasticsearch-status-via-aws-management-console.md %} <!-- To edit, see /_includes/checking-elasticsearch-status-via-aws-management-console.md -->

## 7. Check ElasticSearch indices
## 7. Check Elasticsearch indices

Check ElasticSearch status via a CLI:
Check Elasticsearch status via a CLI:

{% include checking-elasticsearch-indices-via-a-cli.md %} <!-- To edit, see /_includes/checking-elasticsearch-indices-via-a-cli.md -->
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ related:
link: docs/scos/dev/back-end-development/data-manipulation/data-publishing/synchronization-behavior-enabling-multiple-mappings.html
---

Sometimes it's needed to manually [synchronize or re-syncrhonize](/docs/dg/dev/backend-development/data-manipulation/data-publishing/publish-and-synchronize-repeated-export.html#published-data-re-generation) the published model data with Redis or ElasticSearch. To do that, you need to implement a synchronization plugin.
Sometimes it's needed to manually [synchronize or re-syncrhonize](/docs/dg/dev/backend-development/data-manipulation/data-publishing/publish-and-synchronize-repeated-export.html#published-data-re-generation) the published model data with Redis or Elasticsearch. To do that, you need to implement a synchronization plugin.

Follow these steps to implement and register a synchronization plugin.

Expand Down Expand Up @@ -115,7 +115,7 @@ The method descriptions:
* `HelloWorldSynchronizationDataRepositoryPlugin::getResourceName()`—defines a resource name of the storage or search module for key generation.
* `HelloWorldSynchronizationDataRepositoryPlugin::hasStore()`—defines if the entity implements a multi-store concept.
* `HelloWorldSynchronizationDataRepositoryPlugin::getData()`—retrieves a collection of sync transfers based on the provided offset and limit.
* `HelloWorldSynchronizationDataRepositoryPlugin::getParams()`—defines additional sync parameters for Redis or ElasticSearch.
* `HelloWorldSynchronizationDataRepositoryPlugin::getParams()`—defines additional sync parameters for Redis or Elasticsearch.
* `HelloWorldSynchronizationDataRepositoryPlugin::getQueueName()`—defines a queue name for synchonization.
* `HelloWorldSynchronizationDataRepositoryPlugin::getSynchronizationQueuePoolName()`—defines the name of the synchronization queue pool for broadcasting messages.

Expand Down
2 changes: 1 addition & 1 deletion docs/dg/dev/development-getting-started-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ To learn about the module versioning approach in Spryker, see [Semantic Versioni

1. Define how to manage the settings in the configuration files with [Configuration management](/docs/dg/dev/backend-development/data-manipulation/configuration-management.html).
2. [Configure services](/docs/dg/dev/integrate-and-configure/configure-services.html).
3. [Configure ElasticSearch](/docs/pbc/all/search/{{site.version}}/base-shop/tutorials-and-howtos/configure-elasticsearch.html).
3. [Configure Elasticsearch](/docs/pbc/all/search/{{site.version}}/base-shop/tutorials-and-howtos/configure-elasticsearch.html).
4. [Configure queue](/docs/dg/dev/backend-development/data-manipulation/queue/queue.html).
5. [Configure stores](/docs/dg/dev/internationalization-and-multi-store/set-up-multiple-stores.html#configure-stores).
6. [Set up cronjobs](/docs/dg/dev/backend-development/cronjobs/cronjobs.html).
Expand Down
8 changes: 4 additions & 4 deletions docs/dg/dev/integrate-and-configure/configure-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,18 +218,18 @@ docker/sdk clean-data
docker/sdk up --build --data
```

## ElasticSearch
## Elasticsearch

[Elasticsearch](https://www.elastic.co/elasticsearch/) is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.

For more information, see the following documents:

* [Configure Elasticsearch](/docs/pbc/all/search/{{site.version}}/base-shop/tutorials-and-howtos/configure-elasticsearch.html)—describes ElastciSearch configuration in Spryker.
* [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html)—provides detailed information about ElasticSearch.
* [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html)—provides detailed information about Elasticsearch.

### Configure ElasticSearch
### Configure Elasticsearch

In `deploy.*.yml`, adjust the `services` section to open the port used for accessing ElasticSearch:
In `deploy.*.yml`, adjust the `services` section to open the port used for accessing Elasticsearch:
```yaml
services:
search:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ redirect_from:
Spryker comes pre-configured with the following services by default:

* RabbitMQ
* ElasticSearch
* Elasticsearch
* Redis
* MariaDB
* Jenkins
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ To create a new content form plugin, follow these steps:
* `getTypeKey()`—returns a string with the name of your content item—for example, Foo.
* `getTermKey()`—returns a string displaying the term for this content type in database—for example, `Foo`, `Foo List` or `Foo Query`. In database, a content type can have different term representations. Correspondingly, there are different ways of getting information about content. For example:
* `Foo List`—product list IDs.
* `Foo Query`—product query as part of SQL/ElasticSearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `Foo Query`—product query as part of SQL/Elasticsearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `getForm()`—a form class name with a namespace which is displayed on the **Content create** or **Content edit** pages.
* `getTransferObject()`—maps form data to a content term transfer object—for example, `ContentFooTermTransfer`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ To create a new content form plugin, follow these steps:
* `getTypeKey()`—returns a string with the name of your content item—for example, Foo.
* `getTermKey()`—returns a string displaying the term for this content type in database—for example, `Foo`, `Foo List` or `Foo Query`. In database, a content type can have different term representations. Correspondingly, there are different ways of getting information about content. For example:
* `Foo List`—product list IDs.
* `Foo Query`—product query as part of SQL/ElasticSearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `Foo Query`—product query as part of SQL/Elasticsearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `getForm()`—a form class name with a namespace which is displayed on the **Content create** or **Content edit** pages.
* `getTransferObject()`—maps form data to a content term transfer object—for example, `ContentFooTermTransfer`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ To create a new content form plugin, follow these steps:
* `getTypeKey()`—returns a string with the name of your content item—for example, Foo.
* `getTermKey()`—returns a string displaying the term for this content type in database—for example, `Foo`, `Foo List` or `Foo Query`. In database, a content type can have different term representations. Correspondingly, there are different ways of getting information about content. For example:
* `Foo List`—product list IDs.
* `Foo Query`—product query as part of SQL/ElasticSearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `Foo Query`—product query as part of SQL/Elasticsearch query. This value is displayed in the Back Office, in **Content Management&nbsp;<span aria-label="and then">></span> Content Items**.
* `getForm()`—a form class name with a namespace which is displayed on the **Content create** or **Content edit** pages.
* `getTransferObject()`—maps form data to a content term transfer object—for example, `ContentFooTermTransfer`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,9 +230,9 @@ class SearchRouter implements SearchRouterInterface
}
```

If you want to use ElasticSearch for specific cases, you have to create a plugin on the project level
If you want to use Elasticsearch for specific cases, you have to create a plugin on the project level

**ElasticSearchHandlerPlugin**
**ElasticsearchHandlerPlugin**

```php
<?php
Expand All @@ -250,7 +250,7 @@ use Spryker\Client\Search\Dependency\Plugin\QueryInterface;
/**
* @method \Pyz\Client\Search\SearchFactory getFactory()
*/
class ElasticSearchHandlerPlugin extends AbstractPlugin
class ElasticsearchHandlerPlugin extends AbstractPlugin
{
/**
* @param \Spryker\Client\Search\Dependency\Plugin\QueryInterface $searchQuery
Expand Down Expand Up @@ -290,7 +290,7 @@ The Fact-Finder Ng module contains plugins for choosing search, suggestion, or n

namespace Pyz\Client\Search;

use Pyz\Client\Search\Plugin\ElasticSearchHandlerPlugin;
use Pyz\Client\Search\Plugin\ElasticsearchHandlerPlugin;
use Spryker\Client\Kernel\Container;
use Spryker\Client\Search\SearchDependencyProvider as SprykerSearchDependencyProvider;
use SprykerEco\Client\FactFinderNg\Plugin\FactFinderNgNavigationHandlerPlugin;
Expand Down Expand Up @@ -325,7 +325,7 @@ class SearchDependencyProvider extends SprykerSearchDependencyProvider
new FactFinderNgNavigationHandlerPlugin(),
new FactFinderNgSearchHandlerPlugin(),
new FactFinderNgSuggestHandlerPlugin(),
new ElasticSearchHandlerPlugin(),
new ElasticsearchHandlerPlugin(),
];
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ class SearchRouter implements SearchRouterInterface
}
```

If you want to use ElasticSearch for specific cases, you have to create a plugin on the project level
If you want to use Elasticsearch for specific cases, you have to create a plugin on the project level

**ElasticSearchHandlerPlugin**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ class SearchRouter implements SearchRouterInterface
}
```

If you want to use ElasticSearch for specific cases, you have to create a plugin on the project level
If you want to use Elasticsearch for specific cases, you have to create a plugin on the project level

**ElasticSearchHandlerPlugin**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Price import flow:
When enabling Spryker to handle such a number of prices, the following challenges occur:

1. 25,000,000 prices are imported in two separate price dimensions.
2. A product can have about 40,000 prices. This results in overpopulated product abstract search documents: each document aggregates prices of abstract products and all related concrete products. Each price is represented as an indexed field in the search document. Increasing the number of indexed fields slows `ElasticSearch(ES)` down. Just for comparison, the [recommended limit](https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping.html#mapping-limit-settings) is 1,000.
2. A product can have about 40,000 prices. This results in overpopulated product abstract search documents: each document aggregates prices of abstract products and all related concrete products. Each price is represented as an indexed field in the search document. Increasing the number of indexed fields slows `Elasticsearch(ES)` down. Just for comparison, the [recommended limit](https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping.html#mapping-limit-settings) is 1,000.
3. Overloaded product abstract search documents cause issues with memory limit and slow down [Publish and Synchronization](/docs/dg/dev/backend-development/data-manipulation/data-publishing/publish-and-synchronization.html). The average document size is bigger than 1&nbsp;MB.
4. When more than 100 product abstract search documents are processed at a time, the payload gets above 100&nbsp;MB, and ES rejects queries. [AWS native service](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html) does not allow changing this limit.

Expand Down Expand Up @@ -94,7 +94,7 @@ In AWS, the `http.max_content_length` ES limit defines the maximum payload size
The evaluated solutions are as follows:

1. ES join field type.
This ES functionality is similar to the classical joins in relational databases. This solution solves your problem faster and with less effort. To learn about the implementation of this solution, see [ElasticSearch join data type: Implementation](#elasticsearch-join-field-type-implementation). Also, have a look at the other evaluated solutions as they may be more appropriate in your particular case.
This ES functionality is similar to the classical joins in relational databases. This solution solves your problem faster and with less effort. To learn about the implementation of this solution, see [Elasticsearch join data type: Implementation](#elasticsearch-join-field-type-implementation). Also, have a look at the other evaluated solutions as they may be more appropriate in your particular case.
<br>Documentation: [Join field type](https://www.elastic.co/guide/en/elasticsearch/reference/current/parent-join.html)
2. Multi sharding with the `_routing` field.
The idea is to avoid indexing problems by sharing big documents between shards. Breaking a huge index into smaller ones makes it easier for the search to index data. The solution is complex and does not solve the payload issues.
Expand All @@ -106,7 +106,7 @@ The evaluated solutions are as follows:
- [Script score query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#script-score-query-ex-request)
- [Chapter 12. Full-Text Search](https://www.postgresql.org/docs/9.5/textsearch.html)

## ElasticSearch Join field type: Implementation
## Elasticsearch Join field type: Implementation

To solve the ES indexing issue, we reduced the size of product abstract documents, which reduced dynamic mapping properties.

Expand Down Expand Up @@ -175,7 +175,7 @@ To implement the solution, follow these steps:

These two documents can be viewed as two tables with a foreign key in terms of relational databases.

### ElasticSearch join data type feature: Side effects
### Elasticsearch join data type feature: Side effects

The side effects of this solution are the following:

Expand Down
Loading

0 comments on commit 3c501f6

Please sign in to comment.