Explicitly define p64/u64 functions for IDE support

[peace-maker]

Instead of dynamically generating the packing and unpacking helper functions using setattr(module, name, routine) at runtime, unroll the loop and explicitly declare all 8 helpers.

This allows IDEs to know about the function statically without running the code.

Fixes #1657

[Arusekk]

You missed u16/p16 and probably other stuff as well.

[peace-maker]

The diff view is weird for this patch. I think there are all 8 pack and unpack functions which were previously generated by the loop.

Maybe looking at the file view works better to scroll through the functions?

[Arusekk]

I would prefer less boilerplate; for example

@make_unpacker
def u16(data): ...
@make_unpacker
def u32(data): ...

or

u16 = make_unpacker(16)
u32 = make_unpacker(32)

with a clever decorator figuring out the correct context defaults from function name alone and filling in the docstrings and stripping the kwargs in the wrapper.

[peace-maker]

I haven't tried those yet, but I'd be surprised if the documentation would show up when it's generated like that in IDEs. I'm all for a more compact solution though, so will try.

[peace-maker]

I've tried both ways now. u16 = make_unpacker(16) doesn't show any parameters of the function in the IDE, only makes it show up as a possible function to call.

A decorator kinda works by showing the parameters, but doesn't show the docstring in the IDE.

This works, but doesn't show the other documented parameters:

@_make_user_packer
def p8(number):
    # type: (int) -> bytes
    pass

This shows the parameters as well, but still doesn't know about the docstring:

@_make_user_packer
def p8(number, endianness = None, sign = None, **kwargs):
    # type: (...) -> bytes
    pass

I'd include the type hints to make the stubs as useful as possible, since it's all about IDE support anyways. I'd prefer the verbose but most IDE friendly implementation by writing out the docstrings as it is right now in the PR, but it's your call.

[Hexrotor]

Will this be merged? My vscode really looks annoy 😂

[peace-maker]

For context here's my try at implementing it using a decorator. This does make the functions show up in my IDE but doesn't capture the docstring.

import functools

return_types     = {'p': 'bytes', 'u': 'int'}
op_verbs         = {'p': 'pack', 'u': 'unpack'}
arg_name         = {'p': 'number', 'u': 'data'}
arg_doc          = {'p': 'number (int): Number to convert',
                    'u': 'data (bytes): Byte string to convert'}
rv_doc           = {'p': 'The packed number as a byte string',
                    'u': 'The unpacked number'}

def _make_user_packer(function):
    mod = sys.modules[__name__]
    name = function.__name__
    op = name[0]
    size = int(name[1:])

    ls = getattr(mod, "_%sls" % (name))
    lu = getattr(mod, "_%slu" % (name))
    bs = getattr(mod, "_%sbs" % (name))
    bu = getattr(mod, "_%sbu" % (name))

    @LocalNoarchContext
    @functools.wraps(function)
    def routine(*args):
        endian = context.endian if len(args) < 2 else args[1]
        signed = context.signed if len(args) < 3 else args[2]
        return {("little", True  ): ls,
                ("little", False):  lu,
                ("big",    True  ): bs,
                ("big",    False):  bu}[endian, signed](args[0], 3)

    routine.__name__ = name
    routine.__doc__  = """%s%s(%s, endianness, sign, ...) -> %s

    %ss an %s-bit integer

    Arguments:
        %s
        endianness (str): Endianness of the converted integer ("little"/"big")
        sign (str): Signedness of the converted integer ("unsigned"/"signed")
        kwargs (dict): Arguments passed to context.local(), such as
            ``endian`` or ``signed``.

    Returns:
        %s
    """ % (op, size, arg_name[op], return_types[op], op_verbs[op].title(), size, arg_doc[op], rv_doc[op])
    return routine

@_make_user_packer
def p8(number, endianness = None, sign = None, **kwargs):
    # type: (...) -> bytes
    pass

@_make_user_packer
def p16(number, endianness = None, sign = None, **kwargs):
    # type: (...) -> bytes
    pass

@_make_user_packer
def p32(number, endianness = None, sign = None, **kwargs):
    # type: (...) -> bytes
    pass

@_make_user_packer
def p64(number, endianness = None, sign = None, **kwargs):
    # type: (...) -> bytes
    pass

@_make_user_packer
def u8(data, endianness = None, sign = None, **kwargs):
    # type: (...) -> int
    pass

@_make_user_packer
def u16(data, endianness = None, sign = None, **kwargs):
    # type: (...) -> int
    pass

@_make_user_packer
def u32(data, endianness = None, sign = None, **kwargs):
    # type: (...) -> int
    pass

@_make_user_packer
def u64(data, endianness = None, sign = None, **kwargs):
    # type: (...) -> int
    pass

[peace-maker]

Per our discussion at 37c3, I'll merge this now since it's a free quality of life improvement for frequently-used functions when writing exploits using an IDE and we didn't come up with any shorter implementation or nice code generation solution @Arusekk . There is no way to forward to other documentation like @see to deduplicate stuff first.