Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release v4.6.0 #6796

Merged
merged 7 commits into from
Feb 14, 2025
Merged

release v4.6.0 #6796

merged 7 commits into from
Feb 14, 2025

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented Jan 23, 2025

Release checklist:

Copied from #6538 with a few changes.

deadline

February 14, 2025 (ref: #6791)

before merge

after merge

code to update tags (click me)
NEW_VERSION="4.6.0"
git checkout master
git pull upstream master
git fetch upstream --tags
git tag -d stable
git push upstream :refs/tags/stable
git tag stable
git tag "v${NEW_VERSION}"
git push upstream stable "v${NEW_VERSION}"

after the release is created by Azure DevOps job

  • Add release notes from release-drafter to the automatically-created GitHub release, convert it from Draft to published.
  • Manually attach arm64 macOS wheel to the release
  • Upload release to CRAN
  • Upload release to test PyPI
  • Upload release to PyPI.
code to publish to PyPI (summary)
mkdir /tmp/lgb-release
cd /tmp/lgb-release

NEW_VERSION="4.6.0"

# NOTE: requires a GitHub personal access token with "repo" scope
gh release download \
    --repo microsoft/LightGBM \
    --dir ./artifacts \
    --pattern 'lightgbm*-py3-*.whl' \
    --pattern "lightgbm-${NEW_VERSION}.tar.gz" \
    "v${NEW_VERSION}"

# NOTE: manually downloaded arm64 wheels from CI

# config docs: https://packaging.python.org/en/latest/specifications/pypirc/
twine upload \
    -r testpypi \
    ./artifacts/*

(gh is the GitHub CLI, see https://cli.github.com/manual/gh_release_download)

Then confirmed that installing the latest wheel works.

pip install -i https://test.pypi.org/simple/ "lightgbm==${NEW_VERSION}"
python ./examples/python-guide/logistic_regression.py

Then pushed them to real PyPI.

cd /tmp/lgb-release

twine upload \
    ./artifacts/*
code to open that dev version PR (click me)
RELEASE_PR_NUMBER="6439"
RELEASE_VERSION=$(cat ./VERSION.txt)
DEV_VERSION="${RELEASE_VERSION}.99"

git checkout -b ci/dev-version

echo "${DEV_VERSION}" > ./VERSION.txt
sed \
    -i .bak \
    "s|${RELEASE_VERSION}|${DEV_VERSION}|g" \
    .appveyor.yml

sed \
    -i .bak \
    "s|version = \"${RELEASE_VERSION}\"|version = \"${DEV_VERSION}\"|g" \
    ./python-package/pyproject.toml

sed \
    -i .bak \
    's|mode\: release|mode\: unreleased|g' \
    ./R-package/pkgdown/_pkgdown.yml

docker run \
    --rm \
    -v $(pwd):/opt/LightGBM \
    -w /opt/LightGBM \
    ubuntu:22.04 \
    ./R-package/recreate-configure.sh

git add \
    ./.appveyor.yml \
    ./R-package/configure \
    ./R-package/pkgdown/_pkgdown.yml \
    ./VERSION.txt \
    ./python-package/pyproject.toml

commit_msg="bump development version to ${DEV_VERSION}"
git commit -m "${commit_msg}"
git push upstream ci/dev-version

gh pr create \
    --repo microsoft/LightGBM \
    --base 'master' \
    --label 'maintenance' \
    --title "${commit_msg}" \
    --body "now that v${RELEASE_VERSION} has been released: #${RELEASE_PR_NUMBER}"

Notes for Reviewers

I believe this should be v4.6.0 instead of v4.5.1 because of some small breaking changes:

See https://github.com/microsoft/LightGBM/releases.

@jameslamb jameslamb changed the title release v4.6.0 WIP: release v4.6.0 Jan 23, 2025
@jameslamb
Copy link
Collaborator Author

/gha run r-configure

@jameslamb
Copy link
Collaborator Author

I think it's time to start another release!

We need to at least release the R package by February 14th, to stay on CRAN (#6791). But even without that, it's been about 6 months since v4.5.0 and a lot of good changes have piled up.

I've listed the PRs / issues that I think we should try to get into the release. Please comment if there are others you want to try to get in.

@guolinke @shiyu1994 @StrikerRUS @jmoralez @borchero @btrotta

@jameslamb
Copy link
Collaborator Author

jameslamb commented Jan 23, 2025

/gha run r-configure

This failed with the following:

{
  "message": "The 'Microsoft Open Source' enterprise forbids access via a personal access tokens (classic) if the token's lifetime is greater than 365 days. Please adjust your token's lifetime at the following URL: https://github.com/settings/tokens/503169623",
  "documentation_url": "https://docs.github.com/rest/repos/repos#create-a-repository-dispatch-event",
  "status": "403"
}

(build link)

@shiyu1994 can you please fix that in the repo settings? I can update R-package/configure manually and push it, but we'll need to get comment-triggered workflows working again to run the valgrind checks.

@jameslamb
Copy link
Collaborator Author

Add release branch to RTD versions,

Did this. Here's the first build: https://readthedocs.org/projects/lightgbm/builds/26974345/

@StrikerRUS
Copy link
Collaborator

I guess we need to replace old WORKFLOW token with a new one that has 365 days lifetime.
Refer to #3439 (comment) for required token permissions.

@jameslamb
Copy link
Collaborator Author

@shiyu1994 I tried messaging you in the maintainer Slack as well, trying again here in case you check GitHub more frequently. Can you please help us with the workflow token issues here (described in the comments above)?

@shiyu1994
Copy link
Collaborator

I cannot access the URL https://github.com/settings/tokens/503169623. I will figure it out tomorrow.

@StrikerRUS
Copy link
Collaborator

@shiyu1994 Do you have rights to create new tokens in the repo? It should be enough...

@shiyu1994
Copy link
Collaborator

/gha run r-configure

@shiyu1994
Copy link
Collaborator

/gha run r-configure

@shiyu1994
Copy link
Collaborator

I think it works now.

@jameslamb
Copy link
Collaborator Author

jameslamb commented Feb 5, 2025

/gha run r-valgrind

Status: failure ❌.

@jameslamb
Copy link
Collaborator Author

Thanks @shiyu1994 , looks like that worked!

This revealed that the valgrind workflow is broken for other, unrelated reasons. Put up #6816 to try to fix it.

@jameslamb
Copy link
Collaborator Author

jameslamb commented Feb 10, 2025

/gha run r-valgrind

Workflow R valgrind tests has been triggered! 🚀
https://github.com/microsoft/LightGBM/actions/runs/13233666956

Status: success ✔️.

@jameslamb jameslamb changed the title WIP: release v4.6.0 release v4.6.0 Feb 13, 2025
@jameslamb jameslamb marked this pull request as ready for review February 13, 2025 01:44
@jameslamb
Copy link
Collaborator Author

Add release branch to RTD versions, trigger a new build, check docs

Docs look good to me after merging the last required PR (#6714)

With that, I think we are ready to release!

@guolinke @shiyu1994 @StrikerRUS @jmoralez @borchero can you please approve?

Remember that we have to get a release up to CRAN by February 14th, Vienna time... so in about 19 hours 😬 (#6791)

Very sorry for the time pressure, I tried to get this ready further ahead of that deadline 😭

I'll merge this once we have at least 2 reviews (including at least 1 from either @guolinke or @shiyu1994). If we don't have enough approvals to merge normally by around 15 hours from now, I'll just upload the R package to CRAN.

@jameslamb
Copy link
Collaborator Author

Remember that we have to get a release up to CRAN by February 14th, Vienna time... so in about 19 hours 😬 (#6791)

If we don't have enough approvals to merge normally by around 15 hours from now, I'll just upload the R package to CRAN.

It's currently around 10:30p in Vienna. I have some other obligations later today, so we are really out of time to at least release the R package.

I built it on my laptop today and submitted to CRAN. I'm sorry to do that without approvals on this PR, but I thought it was ok to avoid disrupting all of {lightgbm}'s R users installing from CRAN.

Built locally like this:

git clone \
    --recursive \
    --branch release/v4.6.0 \
    -o upstream \
    [email protected]:microsoft/LightGBM.git

cd ./LightGBM
sh build-cran-package.sh

And tested it locally as one last smoke test.

R CMD check --as-cran ./lightgbm_4.6.0.tar.gz
'R CMD check' results (click me)
* using log directory ‘/Users/jlamb/repos/releasing-lightgbm/LightGBM/lightgbm.Rcheck’
* using R version 4.4.2 (2024-10-31)
* using platform: aarch64-apple-darwin20
* R was compiled by
    Apple clang version 14.0.0 (clang-1400.0.29.202)
    GNU Fortran (GCC) 12.2.0
* running under: macOS Sonoma 14.4.1
* using session charset: UTF-8
* using option ‘--as-cran’
* checking for file ‘lightgbm/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘lightgbm’ version ‘4.6.0’
* package encoding: UTF-8
* checking CRAN incoming feasibility ... [5s/33s] Note_to_CRAN_maintainers
Maintainer: ‘James Lamb <[email protected]>’
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘lightgbm’ can be installed ... [121s/123s] OK
* used C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.3.9.4)’
* used SDK: ‘MacOSX14.4.sdk’
* checking C++ specification ... OK
  Not all R platforms support C++17
* checking installed package size ... NOTE
  installed size is  7.9Mb
  sub-directories of 1Mb or more:
    libs   7.2Mb
* checking package directory ... OK
* checking for future file timestamps ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking use of S3 registration ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of ‘data’ directory ... OK
* checking data for non-ASCII characters ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking use of SHLIB_OPENMP_*FLAGS in Makefiles ... OK
* checking pragmas in C/C++ headers and code ... OK
* checking compilation flags used ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking examples with --run-donttest ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* checking HTML version of manual ... OK
* checking for non-standard things in the check directory ... OK
* checking for detritus in the temp directory ... OK
* DONE

Status: 1 NOTE
See
  ‘/Users/jlamb/repos/releasing-lightgbm/LightGBM/lightgbm.Rcheck/00check.log’
for details.
image

@StrikerRUS
Copy link
Collaborator

@jameslamb
Sorry, I saw your call for review earlier but wasn't able to approve. My new notebook at work wasn't signed in and turned out that GitHub stopped sending 2FA sms in Russia (my 2FA recovery codes were also at home). So I was able to approve only when I got back home.

@StrikerRUS
Copy link
Collaborator

StrikerRUS commented Feb 14, 2025

At master branch we have the following R linting errors:

[1] Total linting issues found: 15
[[1]]
/home/runner/work/LightGBM/LightGBM/build_r.R:124:9: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
      , paste0(parsed_args[["make_args"]], collapse = "\", \"")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[2]]
/home/runner/work/LightGBM/LightGBM/build_r.R:170:31: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
      cmd <- paste0(cmd, " ", paste0(args, collapse = " "))
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[3]]
/home/runner/work/LightGBM/LightGBM/build_r.R:429:35: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
  cmd <- paste0(install_cmd, " ", paste0(install_args, collapse = " "))
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[4]]
/home/runner/work/LightGBM/LightGBM/R-package/R/callback.R:70:10: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
  return(paste0(msg, collapse = "  "))
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[5]]
/home/runner/work/LightGBM/LightGBM/R-package/R/lgb.convert_with_rules.R:8:17: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
                paste0(class(x), collapse = ",")
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[6]]
/home/runner/work/LightGBM/LightGBM/R-package/R/lgb.Dataset.R:460:24: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
        merged_name <- paste0(as.list(private$colnames), collapse = "\t")
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[7]]
/home/runner/work/LightGBM/LightGBM/R-package/R/utils.R:37:12: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
    val <- paste0(
           ^~~~~~~

[[8]]
/home/runner/work/LightGBM/LightGBM/R-package/R/utils.R:49:13: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
    pair <- paste0(c(param_names[[i]], val), collapse = "=")
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[9]]
/home/runner/work/LightGBM/LightGBM/R-package/R/utils.R:58:10: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
  return(paste0(ret, collapse = " "))
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~

[[10]]
/home/runner/work/LightGBM/LightGBM/R-package/R/utils.R:118:12: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
      "[", paste0(interaction_constraints[[j]] - 1L, collapse = ","), "]"
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[11]]
/home/runner/work/LightGBM/LightGBM/R-package/src/install.libs.R:54:31: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
      cmd <- paste0(cmd, " ", paste0(args, collapse = " "))
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

[[12]]
/home/runner/work/LightGBM/LightGBM/R-package/tests/testthat/test_lgb.Booster.R:891:35: warning: [string_boundary] Use == to check for an exact string match. Doing so is more readable and more efficient.
        start_indx <- which(grepl("^feature_importances\\:$", file_lines)) + 1L
                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~

[[13]]
/home/runner/work/LightGBM/LightGBM/R-package/tests/testthat/test_lgb.Booster.R:[958](https://github.com/microsoft/LightGBM/actions/runs/13293119956/job/37118487419#step:3:959):31: warning: [string_boundary] Use == to check for an exact string match. Doing so is more readable and more efficient.
    start_indx <- which(grepl("^parameters\\:$", file_lines)) + 1L
                              ^~~~~~~~~~~~~~~~~

[[14]]
/home/runner/work/LightGBM/LightGBM/R-package/tests/testthat/test_lgb.Booster.R:1535:29: warning: [string_boundary] Use == to check for an exact string match. Doing so is more readable and more efficient.
      expect_true(any(grepl("^LightGBM Model$", printed_txt)))
                            ^~~~~~~~~~~~~~~~~~

[[15]]
/home/runner/work/LightGBM/LightGBM/R-package/tests/testthat/test_parameters.R:21:29: warning: [paste] Use paste(), not paste0(), to collapse a character vector when sep= is not used.
        , feature_penalty = paste0(feature_penalties, collapse = ",")
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I suggest we fix them in this PR and turn this PR into 4.6.1 release. Then go through the normal release process including CRAN as we won't have deadlines there. 4.6.0 release will remain published only at CRAN.

UPD: Sorry, didn't see those errors are already fixed #6714 (comment).

Copy link
Collaborator

@borchero borchero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go from my POV :)

@jameslamb
Copy link
Collaborator Author

jameslamb commented Feb 14, 2025

No problem @StrikerRUS , I'm sorry you had to go through that! Thank you for all your reviews to help get the last couple PRs into this release, it was really really helpful (as always).

And thanks for looking @borchero! I hope you'll have more time for LightGBM for the next release cycle, we could really use your help (for example with #5739).

I'd still like either @guolinke or @shiyu1994 to also approve before we merge this and continue with the release.

@jameslamb
Copy link
Collaborator Author

Thanks friends! I'll merge this and push a tag, to trigger the CI job that produces release artifacts.

I'll come back and continue the release (publishing packages to PyPI, etc.) later tonight my time.

@jameslamb jameslamb merged commit d02a01a into master Feb 14, 2025
49 checks passed
@jameslamb jameslamb deleted the release/v4.6.0 branch February 14, 2025 20:27
@jameslamb
Copy link
Collaborator Author

Push v* tag to trigger GitHubRelease action at Azure Pipelines.
Update stable tag at GitHub.

Just did this, using the code in the description of this PR.

image

ref: https://github.com/microsoft/LightGBM/tags

That triggered this run: https://dev.azure.com/lightgbm-ci/lightgbm-ci/_build/results?buildId=17596&view=results

Which succeeded 🎉 , and created this draft release: https://github.com/microsoft/LightGBM/releases/tag/untagged-0fb853a31a098d52f907

@jameslamb
Copy link
Collaborator Author

Add new tag to RTD versions and trigger a new build.

Done. Build: https://readthedocs.org/projects/lightgbm/builds/27196379/

Trigger a new build of the stable version docs on readthedocs (see #6673 (comment))

Done. Build: https://readthedocs.org/projects/lightgbm/builds/27196384/

Remove the release branch release/v{n}.{n}.{n} from RTD versions

Done.

@jameslamb
Copy link
Collaborator Author

Add release notes from release-drafter to the automatically-created GitHub release, convert it from Draft to published.

Done: https://github.com/microsoft/LightGBM/releases/tag/v4.6.0

Manually attach arm64 macOS wheel to the release

Done. Copied from here: https://github.com/microsoft/LightGBM/actions/runs/13336935748

@jameslamb
Copy link
Collaborator Author

Upload release to test PyPI

Done, and tested installing from there (on my M2 Mac) following the steps in "code to publish to PyPI" in the description of this PR.

Upload release to PyPI.

Done: https://pypi.org/project/lightgbm/#files

@jameslamb
Copy link
Collaborator Author

Upload release to CRAN

This was done earlier: #6796 (comment)

Update cran-comments.md when new release is accepted on CRAN.

put up another PR incrementing development version here in GitHub

The R package was accepted to CRAN, and so far checks are succeeding. For example, the gcc-15 job that was failing (the main reason we had to release by February 14th) has been removed from https://cran.r-project.org/web/checks/check_results_lightgbm.html 🎉

Screenshot 2025-02-14 at 10 10 38 PM

And a few precompiled binaries have already been posted:

Screenshot 2025-02-14 at 10 11 50 PM

ref: https://cran.r-project.org/web/packages/lightgbm/index.html

So I've updated cran-comments.md on the PR that also updates the dev version: #6826

@jameslamb
Copy link
Collaborator Author

update conda-forge recipe

Opened conda-forge/lightgbm-feedstock#70. Also included the new scikit-learn floor from #6651 there.

@jameslamb
Copy link
Collaborator Author

Update version and commit hash in Homebrew formula (docs)

Looks like a bot in homebrew-core triggered a PR for this shortly after I pushed the new tag here. And it's already merged: Homebrew/homebrew-core#207744

@jameslamb
Copy link
Collaborator Author

Upload release to NuGet.

Done: https://www.nuget.org/packages/LightGBM

Also opened #6827 to document something I've noticed in the last couple releases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants