pybind11 syntax alternative

[pavlis]

I was trying to solve a pybind11 problem I was having, the details of which aren't relevant to this discussion, and ran across this interesting example:

#include "pybind11/pybind11.h"
#include "pybind11/stl.h"

#include "numpy/arrayobject.h"
#include "numpy/arrayobject.h"
#include "ndarray/pybind11.h"

#include "lsst/pex/exceptions/python/Exception.h"

#include "lsst/TMPL/ExampleOne.h"

namespace py = pybind11;
using namespace pybind11::literals;

namespace lsst {
namespace tmpl {

PYBIND11_PLUGIN(exampleOne) {
    py::module mod("exampleOne");

    if (_import_array() < 0) {
            PyErr_SetString(PyExc_ImportError, "numpy.core.multiarray failed to import");
            return nullptr;
    };

    pex::exceptions::python::declareException<ExampleError, pex::exceptions::RuntimeError>(
            mod, "ExampleError", "RuntimeError");

    py::class_<ExampleOne, std::shared_ptr<ExampleOne>> clsExampleOne(mod, "ExampleOne");

    py::enum_<ExampleOne::State>(clsExampleOne, "State")
        .value("RED", ExampleOne::State::RED)
        .value("ORANGE", ExampleOne::State::ORANGE)
        .value("GREEN", ExampleOne::State::GREEN)
        .export_values();

    clsExampleOne.def(py::init<>());
    clsExampleOne.def(py::init<std::string const&, ExampleOne::State>(), "fileName"_a, "state"_a=ExampleOne::State::RED);
    clsExampleOne.def(py::init<ExampleOne const&, bool>(), "other"_a, "deep"_a=true); // Copy constructor

    clsExampleOne.def("getState", &ExampleOne::getState);
    clsExampleOne.def("setState", &ExampleOne::setState);
    clsExampleOne.def_property("state", &ExampleOne::getState, &ExampleOne::setState);
    clsExampleOne.def("computeSomething", &ExampleOne::computeSomething);
    clsExampleOne.def("computeSomethingElse",
                      (double (ExampleOne::*)(int, double) const) & ExampleOne::computeSomethingElse,
                      "myFirstParam"_a, "mySecondParam"_a);
    clsExampleOne.def("computeSomethingElse", (double (ExampleOne::*)(int, std::string) const) &ExampleOne::computeSomethingElse, "myFirstParam"_a, "anotherParam"_a="foo");
    clsExampleOne.def("computeSomeVector", &ExampleOne::computeSomeVector);
    clsExampleOne.def("doSomethingWithArray", &ExampleOne::doSomethingWithArray);
    clsExampleOne.def_static("initializeSomething", &ExampleOne::initializeSomething);

    clsExampleOne.def("__eq__", &ExampleOne::operator==, py::is_operator());
    clsExampleOne.def("__ne__", &ExampleOne::operator!=, py::is_operator());
    clsExampleOne.def("__iadd__", &ExampleOne::operator+=);
    clsExampleOne.def("__add__", [](ExampleOne const & self, ExampleOne const & other) { return self + other; }, py::is_operator());

    return mod.ptr();
}

}}  // lsst::tmpl

I only point it out because although it is a bit more verbose, the syntax is far less obscure than the way our files are laid out following guidance from the pybind11 docs. For readers not familiar with this issue, the pybind11 docs would have laid this out (in a more skeleton form) more like this:

...
py::class_<ExampleOne, std::shared_ptr<ExampleOne>> (mod, "ExampleOne")
  .def(py::init<>())
  .def(py::init<std::string const&, ExampleOne::State>(), "fileName"_a, "state"_a=ExampleOne::State::RED)
      ...
   .def("__add__", [](ExampleOne const & self, ExampleOne const & other) { return self + other; }, py::is_operator())
;
...

I suspect we would have fewer syntax headaches if we used the structure in the example above compared to the way the pybind11 docs suggest doing this. Specifying the ".def" items and individual C++ code lines would reduce the problems with getting the C++ punctuation (i.e. closing {}, [], and () pairs) correct. Early on I especially struggled with that because I hadn't yet realized how a single misplaced punctuation could cause a cascade with pages of compiler errors. Following this syntax might reduce such errors.

This may be something to consider retrofitting down the road to help maintain the C++ code base. Add comments as you see fit below. If you all concur I think we should add this picky job to our development agenda. Taking on the job would be a good task a new developer who needed to learn pybind11. The act of editing the existing pybind11 code to this alternative structure would be a good tool for teaching all the oddities of pybind11.