Modified data silently truncated when using pass_original=True with multiple post_load methods

[ghostwheel42]

Hi,
this happens when a pre_load(pass_many=True) hook appends items to the the input data collection, due to the usage of zip in _invoke_processors called by post_load(pass_many=False):

marshmallow/src/marshmallow/schema.py

Lines 1214 to 1218 in 3247666

	if pass_original:
	data = [
	processor(item, original, many=many, **kwargs)
	for item, original in zip(data, original_data)
	]

It is arguable if modifying the input data in this way should be allowed, but it is not forbidden in the docs. I think using itertools.zip_longest would be a good choice here. Raising an exception if len(data) != len(original_data) is another option.

Newly added items will not be validated too:

marshmallow/src/marshmallow/schema.py

Lines 1172 to 1174 in 3247666

	if many and not pass_many:
	for idx, (item, orig) in enumerate(zip(data, original_data)):
	self._run_validator(

I think itertools.zip_longest should be used here.

Cheers. Alex

[deckar01]

I was unable to reproduce this with pre_load, then post_load, but was able to when they were both post_load.

from marshmallow import Schema, fields, pre_load, post_load

class Test(Schema):
  foo = fields.Str()

  @post_load(pass_many=True)
  def setup(self, data, **kwargs):
    data.append({'foo': 'test'})
    print(data)
    return data

  @post_load(pass_many=False, pass_original=True)
  def test(self, data, original_data, **kwargs):
    print(data, original_data)
    return data

print(Test().load([{'foo': 'bar'}], many=True))
# [{'foo': 'bar'}, {'foo': 'test'}]
# {'foo': 'bar'} {'foo': 'bar'}
# [{'foo': 'bar'}]

[ghostwheel42]

@deckar01 You are correct, I can't recreate this with pre_load anymore.
I found this a week ago and seem to have remembered incorrectly.

[sloria]

fixed in #2797