Add verbose_pipeline documentation in using-search-pipeline.md

Signed-off-by: Junwei Dai <[email protected]>

_search-plugins/search-pipelines/using-search-pipeline.md

@@ -14,6 +14,7 @@ You can use a search pipeline in the following ways:
 - [Specify an existing pipeline](#specifying-an-existing-search-pipeline-for-a-request) for a request.
 - [Use a temporary pipeline](#using-a-temporary-search-pipeline-for-a-request) for a request.
 - Set a [default pipeline](#default-search-pipeline) for all requests in an index.
+- Debugging search pipeline with the [verbose pipeline](#debugging-search-pipeline)  parameter.
 
 ## Specifying an existing search pipeline for a request
 
@@ -236,3 +237,172 @@ PUT /my_index/_settings
 }
 ```
 {% include copy-curl.html %}
+
+## Debugging search pipeline
+
+The `verbose_pipeline` parameter helps visualize and debug data flow across search pipeline processors. It aids troubleshooting, optimizing pipeline configurations, and ensuring transparency for search request and response transformations.
+
+The `verbose_pipeline` parameter provides a detailed view of how data is passed through and transformed by each processor in the search pipeline. This functionality is supported in all three ways of using a search pipeline:
+
+- [Specifying an existing pipeline for a request](#specifying-an-existing-search-pipeline-for-a-request).
+- [Using a temporary pipeline for a request](#using-a-temporary-search-pipeline-for-a-request).
+- [Setting a default pipeline for all requests in an index](#default-search-pipeline).
+
+To enable the `verbose_pipeline` feature, simply add `verbose_pipeline=true` as a query parameter in your search request.
+
+### Example request with verbose pipeline
+
+You can specify the `verbose_pipeline` parameter in the query to enable detailed debugging. The `verbose_pipeline` parameter works seamlessly across all search pipeline configurations:
+
+#### Default Search Pipeline
+
+To use `verbose_pipeline` with a default search pipeline, set the pipeline as the default in the index settings and include `verbose_pipeline=true` in the query:
+
+```json
+PUT /my_index/_settings
+{
+  "index.search.default_pipeline": "my_pipeline"
+}
+```
+{% include copy-curl.html %}
+
+```json
+GET /my_index/_search?verbose_pipeline=true
+```
+{% include copy-curl.html %}
+
+#### Specified Search Pipeline by ID
+To use verbose_pipeline with a specific search pipeline, specify the pipeline ID along with verbose_pipeline=true in the query:
+```json
+GET /my_index/_search?search_pipeline=my_pipeline&verbose_pipeline=true
+```
+{% include copy-curl.html %}
+
+#### Temporary Search Pipeline
+To use verbose_pipeline with a temporary search pipeline, define the pipeline directly in the request body and include verbose_pipeline=true in the query:
+```json
+POST /my_index/_search?verbose_pipeline=true
+{
+  "query": {
+    "match": { "text_field": "some search text" }
+  },
+  "search_pipeline": {
+    "request_processors": [
+      {
+        "filter_query": {
+          "query": { "term": { "visibility": "public" } }
+        }
+      }
+    ],
+    "response_processors": [
+      {
+        "collapse": {
+          "field": "category"
+        }
+      }
+    ]
+  }
+}
+```
+{% include copy-curl.html %}
+### Example response with verbose pipeline
+
+When the verbose_pipeline parameter is enabled, the response contains an additional `processor_results` field that details the transformations applied by each processor in the pipeline:
+
+<details open markdown="block">
+  <summary>
+    Response
+  </summary>
+  {: .text-delta}
+
+```json
+{
+  "took": 40,
+  "timed_out": false,
+  "_shards": {
+    "total": 1,
+    "successful": 1,
+    "skipped": 0,
+    "failed": 0
+  },
+  "hits": {
+    "total": {
+      "value": 1,
+      "relation": "eq"
+    },
+    "max_score": 0.0,
+    "hits": [
+      {
+        "_index": "my_index",
+        "_id": "1",
+        "_score": 0.0,
+        "_source": {
+          "notification": "This is a public message",
+          "visibility": "public"
+        }
+      }
+    ]
+  },
+  "processor_results": [
+    {
+      "processor_name": "filter_query",
+      "tag": "tag1",
+      "duration_millis": 1094542,
+      "status": "success",
+      "input_data": {
+        "verbose_pipeline": true
+      },
+      "output_data": {
+        "verbose_pipeline": true,
+        "query": {
+          "bool": {
+            "filter": [
+              {
+                "term": {
+                  "visibility": {
+                    "boost": 1.0,
+                    "value": "public"
+                  }
+                }
+              }
+            ],
+            "adjust_pure_negative": true,
+            "boost": 1.0
+          }
+        }
+      }
+    },
+    {
+      "processor_name": "rename_field",
+      "duration_millis": 1336958,
+      "status": "success",
+      "input_data": [
+        {
+          "_index": "my_index",
+          "_id": "1",
+          "_score": 0.0,
+          "_source": {
+            "message": "This is a public message",
+            "visibility": "public"
+          }
+        }
+      ],
+      "output_data": [
+        {
+          "_index": "my_index",
+          "_id": "1",
+          "_score": 0.0,
+          "_source": {
+            "notification": "This is a public message",
+            "visibility": "public"
+          }
+        }
+      ]
+    }
+  ]
+}
+```
+</details>
+
+
+