ES operation and maintenance common commands, common troubleshooting commands, es cluster health, cluster node allocation information, slice allocation information, slice unallocated problem diagnosis, cluster open password access

catalogs

Querying the health of a cluster

View allocation information for all nodes in the cluster

Total document count for query clusters/indexes

Querying the cluster’s slice allocation information

Query the number of indexes in the cluster, the number of documents, or which indexes are included in the cluster

Querying the cluster’s snapshot storage information

Viewing Cluster Status Information

Viewing Cluster Statistics

View node properties for all nodes in the cluster

Reasons for query slice unallocated

Cluster Enable Password Access

---

Querying the health of a cluster

• Green: all slices available
• Yellow: at least one replica is unavailable, but all primary slices are available
• Red: at least one primary slice is unavailable and data is incomplete

GET _cat/health

GET _cluster/health

{
  "cluster_name" : "my_cluster", cluster name 
  "status" : "yellow", cluster health value
  "timed_out" : false, whether to time out or not
  "number_of_nodes" : 3, number of indexed primary slices
  "number_of_data_nodes" : 3, number of data nodes
  "active_primary_shards" : 37, number of active primary shards
  "active_shards" : 65, number of active shards
  "relocating_shards" : 0, number of shards in migration
  "initializing_shards" : 0, number of shards in initialization
  "unassigned_shards" : 3, number of unassigned shards
  "delayed_unassigned_shards" : 0, number of delayed unassigned shards
  "number_of_pending_tasks" : 0, number of cluster-level changes that have not yet been implemented
  "number_of_in_flight_fetch" : 0, number of outstanding fetches
  "task_max_waiting_in_queue_millis" : 0, time since the earliest started task waited for execution
  "active_shards_percent_as_number" : 95.58823529411765 Ratio of active shards in the cluster, expressed as a percentage
}

View allocation information for all nodes in the cluster

GET _cat/nodes?v

Obtain the allocation of each node, including obtaining its assigned IP address, resource utilization information, roles, etc.

Total document count for query clusters/indexes

GET _cat/count

Calculate the total number of documents in the cluster

GET _cat/count?v
// Return the results:
epoch      timestamp count
1654141686 03:48:06  13342

Calculate the total number of documents in the product index.

GET _cat/count/product?v
// Return results
epoch      timestamp count
1654141931 03:52:11  13

This command can be replaced by the _count API

GET product/_count
// Return results
{
  "count" : 13,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

Querying the cluster’s slice allocation information

GET _cat/shards?v

Used to see where a slice is located, including which node it is on and the IP address of the node it is on. Common usage scenarios are: index lifecycle management, data flow, etc.

Query the number of indexes in the cluster, the number of documents, or which indexes are included in the cluster

GET _cat/indices?v

This command is generally used to view and mass what indexes are included. Queryable information includes but is not limited to

• State of health of the index
• Open and closed state of the index
• Index name
• Number of indexed documents
• Number of deleted documents
• Number of master slices
• Number of replica slices

Querying the cluster’s snapshot storage information

GET /_cat/snapshots?v

Used to get the snapshot backup information query for indexed data, and can also be used to specify the snapshot name for querying:

GET /_cat/snapshots/<repository>

Viewing Cluster Status Information

GET _cluster/state

For querying cluster status metadata information, including:

• Set of nodes in a cluster
• All cluster-level settings
• Information about the indexes in the cluster, including their mapping and settings
• The location of all the slices in the cluster.

Viewing Cluster Statistics

GET _cluster/stats

This command queries to retrieve statistical information from a cluster-wide perspective.API Returns basic index metrics (number of slices, storage size, memory usage) and information about the current nodes that make up the cluster (number, role, operating system, jvm version, memory usage, cpu, and installed plug-ins).

• Index to slice mapping
• Documentation Technical Statistics
• fielddata
• Cache Information
• Paragraph Document Information
• Index Field Type Statistics
• Parser statistics
• Cluster node role statistics

View node properties for all nodes in the cluster

GET _cat/nodeattrs

The most common way to view node attributes is to view the node’s custom attributes, which are used for node tagging and are often used in the following scenarios:

• Hot and cold cluster deployment
• Highly Available Cluster Architecture Deployment
• Segmented allocation strategy deployment

Reason for query slice unallocated

View belowmy_indexReason for unallocated replica slice number 0.

GET _cluster/allocation/explain
{
  "index": "my_index",
  "shard": 0,
  "primary": false
}

Output results:

{
      "node_id" : "bgLGquyZSeOKbFBTJECjOQ",
      "node_name" : "node1",
      "transport_address" : "127.0.0.1:9302",
      "node_attributes" : {
        "rack" : "rack1",
        "xpack.installed" : "true",
        "transform.node" : "false"
      },
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "filter",
          "decision" : "NO",
          "explanation" : """node does not match index setting [index.routing.allocation.require] filters [_name:"node3"]"""
        }
      ]
    }

Error reason: the node does not match the index setting [index. Routing. Allocation. The require] filters [_name: “node3”]

Below are the trouble codes and descriptions for all the reasons why the slice is not assigned, 12 in total:

* :: ALLOCATION_FAILED: Unallocated due to slice allocation failure
* CLUSTER_RECOVERED: Unallocated due to full cluster recovery.
* DANGLING_INDEX_IMPORTED: Unallocated due to import of dangling index.
* EXISTING_INDEX_RESTORED: Unallocated due to restoration to closed index.
* INDEX_CREATED: Unallocated due to index creation by API.
* INDEX_REOPENED: Unallocated due to open closed index.
* NEW_INDEX_RESTORED: Unallocated due to restore to new index.
* NODE_LEFT: de-allocation due to the node hosting it leaving the cluster.
* :: REALLOCATED_REPLICA: Identify better replica locations and cancel existing replica assignments.
* REINITIALIZED: when the fragment is moved from "start" back to "initialized".
* :: REPLICA_ADDED: Unallocated due to explicit addition of replica copy.
* REROUTE_CANCELLED: Distribution canceled as a result of an explicit cancel re-route command.

Different causes of the problem require different solutions. To address the above causes of the problem, it is sufficient to turn off the slice filter:

PUT my_index/_settings
{
  "index.routing.allocation.require._name": null
}

Cluster Enable Password Access

1) Assuming there are three es servers, generating TLS and authentication on server 1 will generate the file elastic-certificates.p12 under config, pass this file to the config directory of the other two nodes, paying attention to the file permissions:

-> bin/elasticsearch-certutil cert -out config/elastic-certificates.p12 -pass ""

2) Upload the elastic-certificates.p12 file in the config directory to the config directory of the other 2 servers and restart the ElasticSearch service, which needs to be executed on all 3 servers;

3) Set the password and execute the command on Server 1:

bin/elasticsearch-setup-passwords interactive

Then follow the prompts to enter the passwords of elastic, apm_system, kibana, logstash_system, beats_system, remote_monitoring_user, and then the data will be automatically synchronized to the other 2 nodes.

4) Visit ES again in your browser, http 9200, and you’ll see that you need a username and password to access it.