[BUG]bulk api → Request size exceeded 10485760 bytes

[ichige]

What is the bug?

After upgrading from 2.3.1 to 2.4 the other day, I started getting the following exception in the bulk api.

OpenSearch\Common\Exceptions\BadRequest400Exception: {"Message":"Request size exceeded 10485760 bytes"}

How can one reproduce the bug?

Simply execute the bulk method on the same Client.

What is the expected behavior?

In previous implementations, an instance of endpoint was created with each call.

        if (is_null($this->endpoint)) {
            $serializer = $this->serializer;

            $this->endpoint = function ($class) use ($serializer) {
                $fullPath = '\\OpenSearch\\Endpoints\\' . $class;

                $reflection = new ReflectionClass($fullPath);
                $constructor = $reflection->getConstructor();

                if ($constructor && $constructor->getParameters()) {
                    return new $fullPath($serializer);
                } else {
                    return new $fullPath();
                }
            };
        }

The new version returns the same instance, so body is inherited.

    public function getEndpoint(string $class): AbstractEndpoint
    {
        if (!isset($this->endpoints[$class])) {
            $this->endpoints[$class] = $this->createEndpoint($class);
        }

        return $this->endpoints[$class];
    }

I hope it comes across as a translation from Japanese to English in deepl.

[shyim]

The new version returns the same instance, so body is inherited.

About which body we are talking. There is no internal state where the body is buffered 🤔

[dblock]

@ichige do you have a repro? try to turn this into a failing unit test?

[ichige]

@shyim

We use Laravel.
Here is a simple way to reproduce it.

        /** @var OpenSearchClient $client */
        $client = App::make(OpenSearchClient::class);
        $client->bulk([
            'body' => [
                ['index' => ['_index' => 'example', '_id' => 'id-01']],
                ['data' => Str::random(10)]
            ]
        ]);
        $client->bulk([
            'body' => [
                ['index' => ['_index' => 'example', '_id' => 'id-02']],
                ['data' => Str::random(10)]
            ]
        ]);

Use Laravel's dump function to check the body of the Bullk Endpoint.

    public function bulk(array $params = [])
    {
        $index = $this->extractArgument($params, 'index');
        $body = $this->extractArgument($params, 'body');

        $endpoint = $this->endpointFactory->getEndpoint(\OpenSearch\Endpoints\Bulk::class);
        $endpoint->setParams($params);
        $endpoint->setIndex($index);
        $endpoint->setBody($body);
        // check the body
        dump($endpoint->getBody());

        return $this->performRequest($endpoint);
    }

This is the result.

thanks.

[dblock]

@ichige Turn it into a unit test and maybe a fix?

[ichige]

@dblock

Turn it into a unit test and maybe a fix?

I am sorry, but I am not in the habit of creating Unit Tests, so I am afraid I cannot help you.