Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MultiMapper fails with single-parameter mappers #70

Open
jkleint opened this issue Mar 18, 2013 · 1 comment
Open

MultiMapper fails with single-parameter mappers #70

jkleint opened this issue Mar 18, 2013 · 1 comment

Comments

@jkleint
Copy link
Contributor

jkleint commented Mar 18, 2013

If I write a map function with the alternative low-level single-parameter interface, then give it to MultiMapper:

import dumbo
from dumbo.lib import MultiMapper
from dumbo.decor import primary

@primary
def mapper(keysvals):
    for key, val in keysvals:
        yield key, val

if __name__ == "__main__":
    multimapper = MultiMapper()
    multimapper.add("primary", mapper)
    dumbo.run(multimapper)

It tries to call it with two parameters:

...
  File "build/bdist.linux-x86_64/egg/dumbo/core.py", line 322, in run
  File "build/bdist.linux-x86_64/egg/dumbo/util.py", line 49, in dumpcode
  File "build/bdist.linux-x86_64/egg/dumbo/core.py", line 314, in <genexpr>
  File "build/bdist.linux-x86_64/egg/dumbo/core.py", line 445, in mapfunc_iter
  File "build/bdist.linux-x86_64/egg/dumbo/lib/__init__.py", line 180, in __call__

TypeError: mapper() takes exactly 1 argument (2 given)

It looks like MultiMapper doesn't know about the alternative interface.

@a4tunado
Copy link

In general cases mapper callable object should support 2-parameters interface for ex.

def mapper(key, value):
    yield key, value.split('\t')

Where value is usually a single value or a line but not a generator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants