Developer Guide

Intro Github Guide: How to Develop QuEmb on Github:

This guide will walk you through the first steps you should take to set up the QuEmb repository in your local device, contribute to the development, and push the changes to the remote repository.

First, some key Git terms:

• local repository refers to the version of the code you have downloaded on your computer. These changes are only saved locally.
• remote repository refers to the version of the code up on Github.
• main is the main branch of our code. This is the working, reliable version with the well-tested features
• feature_branches are the in-progress branches which are home to changes that you and others make. Everytime you want to change or add something to the code, make a new branch! These are deleted when your changes are finished and merged back with the main branch. As you may notice from the name, each feature branch should aim to add one feature (or fix) at a time.
• issues are raised when you run into a problem. Try to be as specific as possible when logging any problems you see!
• and more... (rebase, diff, stash, tags/versions, linters, and more...)

For QuEmb, Github is our provider for remote repository. In addition to providing this remote space, Github has functionalities and tools we use to facilitate collaborative code development.

Now, let's set up a local copy of QuEmb in your local directory!

Cloning QuEmb

In order to set up a copy of the QuEmb repository in your machine, we clone the repository. In your development environment, use:

git clone [email protected]:troyvvgroup/quemb.git

This will prepare a local repository in the current directory.

Now, before anything else, we should set up the pre-commit hook for the local repository. Run:

git config --local core.hooksPath .githooks/

activate the pre-commit hook. Then whenever you commit, this will run fast sanity checks such as the test for trailing white space and if ruff complains about anything.¹ This pre-commit script protects you from making changes not allowed in the QuEmb development environment.

---

¹ In the rare case that you want to consciously ignore this check upon commiting just pass --no-verify to git commit. This is not recommended and sooner or later you will have to adhere to all these checks when doing a pull request to merge into main.

What's your local Git status?

When in the directory of your clone, to see the status of your branch of code (i.e. have you made changes to the code?), use:

git status

To see what branch of the code you are on, use:

git branch

How to keep your local Git updated with the remote repo?

To keep your local Git of the QuEmb software up-to-date with the remote version, use:

git fetch <remote>

In most cases, this is simply origin, the remote code location. This will not update your local code, but it will download all of the changes and commits locally for you to review. You can fetch a specific branch, using:

git fetch <remote> <branch>

You can then merge the fetched code (now located locally, at origin/<branch> rather than origin <branch>) using:

git merge origin/<branch>

This will change your local code so that it matches the fetched remote version. To keep your local version of the code up-to-date with the remote version (i.e. keep updating the code to include commits from other work and contributors which have been pushed to the main branch), use:

git pull origin <remote>

This will fetch and merge any changes into your local code, together in one step. If you have your local branch tracking the remote one, you can simply use git pull instead.

How to manage new branches?

If you want to make a new branch, or if you have made changes locally and want to make them a new branch, use:

git checkout -b <new_feature_branch>

This will save any of those changes, at your current code progress point, on a new local branch (i.e. not on the remote repo, but locally on your computer). You can switch branches, once your changes are stashed or saved to a different branch, simply using:

git checkout <diff_feature_branch>

These changes all happen with your local code version, not changes to the remote repo. To add the new branch to the remote repo, and set up tracking between your local branch and the new remote one, use:

git push -u origin <feature_branch>

How to start making changes to the remote repo?

When you make changes to a given branch of the code, you need to commit those changes to keep a log of any progress, saving changes in the local repo. Any files that you are committing must first be staged, by telling git which files go in the commit. You can see which files were marked for your commit using 'git status'.

git add <file> stages a code (as-is! not any later changes you make to the file) to be committed, while git rm <file> un-stages it.

You can stage the whole code for a commit simply using git add . at the head of the directory. You then can commit your staged changes using:

git commit -m "<your message here>"

where the -m flag lets you leave a message. Note that you must leave helpful commit messages. Having a healthy log of git history allows other developers who read the commit history in the future to understand how the code was developed by multiple developers.

If you do not need to be as careful in your staging (i.e. you would like to commit all changes), you can use:

git commit -am "<your message here>"

Note that you will need to add any new files you added to the branch, with git add <new_file>, when doing this.

After committing any changes, you will have to make sure your local branch with its changes can be successfully pushed and merged into the remote one. You should have been periodically pulling from the remote repo to make sure the branches "agree" and are as up-to-date as possible.

To save and share your changes on the remote repo, push them to their corresponding remote branch:

git push origin <feature_branch>

This will push all of the commits and their logs to the remote feature branch saved on Github in the remote repository. If someone else worked on your feature branch and pushed their changes since your last push, Git will complain about the mismatch of the local and remote HEAD. If this happens, you should fetch and merge the remote branch as introduced earlier. The merge can sometimes happen automatically, but other times will require you to go through by hand. In this case, git will allow you to hand-pick the correct changes between the versions it marked with '>>>>>>>>' in your code.

How to contribute your changes to the main branch in the remote repository?

Once you have made all of changes you see fit, tested them (using the different checks below), and would like to add them to the main branch of the code, make a pull request on Github to merge your feature branch with the main one. You should describe the changes made in the request, including addressing any issues that you have fixed with your changes. You can assign reviewer(s) to go through and approve the request before the code can be merged into the main branch and the branch is deleted. As configured, QuEmb requires a reviewer to approve PRs before they can be merged to protect the main branch.

If the main branch on the remote repository has changed since you last sync-ed it with your local feature branch, Github will automatically ask you to re-sync the branch when you submit your Pull Request. This can be either done on the web interface, or you can fetch the remote main branch and merge it to your feature branch as shown in the next section.

How to stay synchronized with main

If you branched off from main and while doing your changes someone successfully merged their code into main via a pull request then you have to incorporate these changes into your code.

Assuming you are in your feature_branch and want to synchronize it, the recommended way is

git pull origin main

which pulls from the remote (origin) and merges into your feature_branch.¹ If there are no merge conflicts the merge-commit will be automatically created. If, however, there are merge conflicts, git will tell you, you have to manually resolve them and then explicitly commit the changes. The command

git diff --name-only --diff-filter=U --relative

shows you all files with a merge conflict, which you can handily combine with your text-editor of choice such as:

vi `git diff --name-only --diff-filter=U --relative`

In this files search for >>>>>>>>>> and you will see both possible versions of the conflict, which you have to manually combine, or decide on one. Note, there are also many GUI programs available to help you with resolving merge conflicts.

If everything is resolved, just git add the files and git commit them.

Intro Linting, etc.: How do we check our code?

We use ruff, pytest, and mypy in different ways to test our code. First, to reiterate from above, make sure to install the pre-commit Github hook on install, which will do some checks before allowing you to push to Github:

git config --local core.hooksPath .githooks/

activate the pre-commit hook.

ruff

We use the linter, ruff, to find mistakes and simplify our python code. This allows us to also follow and enforce (roughly) the same style formatting. ruff helps us avoid keywords that go nowhere, import unneccessary code, and more. To use ruff on your local machine, simply install:

pip install ruff

To re-format the code in a uniform style, type:

ruff format <file or directory>

to perform the static code-analysis run:

ruff check <file or directory>

ruff check --fix will change those issues that can be safely fixed automatically. The remaining issues have to be corrected manually.

pytest

We have a number of tests written in src/tests, designed to test different parts of the code. We require these to still run correctly before committing new code, and we encourage adding them while writing new code features.

You can use pytest on your machine by installing:

pip install pytest

and running:

pytest <python script or directory>

mypy

We further have started using mypy as an additional typing and error-finding tool. Install:

pip install mypy

and make sure you have all of the requirements:

pip install -r tests/static_analysis_requirements.txt

Running:

mypy <file or directory>

will run mypy checks and raise warnings or errors. Adding detailed typing, ala mypy, is not required in the code, but can be a useful tool.

Useful bit of code: If you run into issues running mypy on your local computer, you can try:

git clean -x -d -f --dry-run

to remove the cached files locally that could be causing issues. --dry-run lists which files would be removed without making any changes. Removing that flag and running git clean -x -d -f removes those cached files.