Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support building debian packages for PANDA and pip packages for PyPANDA #1399

Merged
merged 7 commits into from
Dec 19, 2023

Conversation

AndrewFasano
Copy link
Contributor

@AndrewFasano AndrewFasano commented Dec 16, 2023

This PR aims to give us a sane way to distribute PANDA and PyPANDA.

Debian packaging

Add support for creating debian packages for PANDA using our existing docker containers.

Running ./panda/debian/setup.sh Ubuntu [VERSION] with VERSION set to 18.04/20.04/22.04 will create a pandare.deb package that can be installed with apt install pandare.deb. This PR does not integrate this with CI but I think that's something we should explore in the future.

PyPANDA packaging

This PR refactors the PyPANDA install to remove the panda binaries, libraries and plugins and limits it to just python scripts, the qcows.json, and the autogenerated headers. As a result the package is about 50x smaller and no longer gets out of sync with the host (non-python) version of PANDA.

Instead of including these binaries in the package, this now uses the binaries at the expected paths from the debian package package. If a user installs with python3 setup.py develop it should continue to behave as normal and pull libpanda libraries and compiled plugins from the panda build path.

To create a source dist in panda/panda/python/core you run:

python3 setup.py install
python3 setup.py sdist
# distribution created in ./dist/pandare-[version].tar.gz

Known issues

  • The PyPANDA package used to copy llvm .bc files which might be required for taint. Maybe those should be installed by the debian packages? I'm not sure where they're used.

Example

The following dockerfile installs panda and pypanda in Ubuntu 20.04 from the panda .deb and the pypanda sdist:

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

# Install panda deb
COPY pandare.deb /host/
RUN apt update && apt install -q -y /host/pandare.deb

# Install pypanda
COPY pandare-0.1.2.0.tar.gz /host/
RUN pip install /host/pandare-0.1.2.0.tar.gz

# Download a qcow and test PyPANDA
COPY syscalls.py /host
RUN python3 -m pandare.qcows i386
RUN python3 /host/syscalls.py i386

@AndrewFasano AndrewFasano changed the title Remove binary blobs from PyPANDA install Add support for building debian packages for ubuntu and small PyPANDA packages Dec 17, 2023
@AndrewFasano AndrewFasano marked this pull request as ready for review December 17, 2023 04:49
@AndrewFasano AndrewFasano changed the title Add support for building debian packages for ubuntu and small PyPANDA packages Support building debian packages for PANDA and pip packages for PyPANDA Dec 17, 2023
@AndrewFasano
Copy link
Contributor Author

The CI failure here found an unrelated bug introduced in #1327 when PLogReader is unavailable. I added a fix to this PR

@AndrewFasano AndrewFasano merged commit 8b9c676 into dev Dec 19, 2023
8 checks passed
@AndrewFasano AndrewFasano deleted the deb_pypanda_install branch December 19, 2023 18:14
@AndrewFasano AndrewFasano mentioned this pull request Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant