Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve docs for GPSS #7

Merged
merged 15 commits into from
May 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ This package implements slice sampling algorithms accessible through the `Abstra
For general usage, please refer to [here](https://turinglang.org/SliceSampling.jl/dev/general/).

## Implemented Algorithms
- [Univariate slice sampling](https://turinglang.org/SliceSampling.jl/dev/univariate_slice/) algorithms with coordinate-wise Gibbs sampling by R. Neal [^N2003].
- [Latent slice sampling](https://turinglang.org/SliceSampling.jl/dev/latent_slice/) by Li and Walker[^LW2023]
- [Gibbsian polar slice sampling](https://turinglang.org/SliceSampling.jl/dev/gibbs_polar/) by P. Schär, M. Habeck, and D. Rudolf[^SHR2023].
- Univariate slice sampling ([Slice](https://turinglang.org/SliceSampling.jl/dev/univariate_slice/)) algorithms with coordinate-wise Gibbs sampling by R. Neal [^N2003].
- Latent slice sampling ([LSS](https://turinglang.org/SliceSampling.jl/dev/latent_slice/)) by Li and Walker[^LW2023]
- Gibbsian polar slice sampling ([GPSS](https://turinglang.org/SliceSampling.jl/dev/gibbs_polar/)) by P. Schär, M. Habeck, and D. Rudolf[^SHR2023].

## Example with Turing Models
This package supports the [Turing](https://github.com/TuringLang/Turing.jl) probabilistic programming framework:
Expand Down
23 changes: 15 additions & 8 deletions docs/src/gibbs_polar.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ However, unlike ESS, GPSS is applicable to any target distribution.


## Description
For a $$d$$-dimensional target distribution, GPSS utilizes the following augmented target distribution:
For a $$d$$-dimensional target distribution $$\pi$$, GPSS utilizes the following augmented target distribution:
```math
\begin{aligned}
p(x, T) &= \varrho_{\pi}^{(0)}(x) \varrho_{\pi}^{(1)}(x) \, \operatorname{Uniform}\left(T; 0, \varrho^1(x)\right) \\
Expand All @@ -26,7 +26,7 @@ In a high-level view, GPSS operates a Gibbs sampler in the following fashion:
T_n &\sim \operatorname{Uniform}\left(0, \varrho^{(1)}\left(x_{n-1}\right)\right) \\
\theta_n &\sim \operatorname{Uniform}\left\{ \theta \in \mathbb{S}^{d-1} \mid \varrho^{(1)}\left(r_{n-1} \theta\right) > T_n \right\} \\
r_n &\sim \operatorname{Uniform}\left\{ r \in \mathbb{R}_{\geq 0} \mid \varrho^{(1)}\left(r \theta_n\right) > T_n \right\} \\
x &= \theta r,
x_n &= \theta_n r_n,
\end{aligned}
```
where $$T_n$$ is the usual acceptance threshold auxiliary variable, while $$\theta$$ and $$r$$ are the sampler states in polar coordinates.
Expand All @@ -51,7 +51,9 @@ GibbsPolarSlice
```

## Demonstration
As illustrated in the original paper, GPSS shows good performance on heavy-tailed targets despite being a multivariate slice sampler:
As illustrated in the original paper, GPSS shows good performance on heavy-tailed targets despite being a multivariate slice sampler.
Consider a 10-dimensional Student-$$t$$ target with 1-degree of freedom (this corresponds to a multivariate Cauchy):

```@example gpss
using Distributions
using Turing
Expand All @@ -64,12 +66,17 @@ using Plots
end
model = demo()

n_samples = 10000
chain = sample(model, externalsampler(GibbsPolarSlice(10)), n_samples; initial_params=ones(10))
histogram(chain[:,1,:], xlims=[-10,10])
savefig("cauchy_gpss.svg")
n_samples = 1000
latent_chain = sample(model, externalsampler(LatentSlice(10)), n_samples; initial_params=ones(10))
polar_chain = sample(model, externalsampler(GibbsPolarSlice(10)), n_samples; initial_params=ones(10))
stephist( rand(TDist(1), 10000), bins=-10:1:10, normed=true, label="true", linewidth=3)
stephist!(latent_chain[:,1,:], bins=-10:1:10, fill=true, alpha=0.5, normed=true, label="LSS")
stephist!(polar_chain[:,1,:], bins=-10:1:10, fill=true, alpha=0.5, normed=true, label="GPSS")
savefig("student_latent_gpss.svg")
```
![](cauchy_gpss.svg)
![](student_latent_gpss.svg)

Clearly, for 1000 samples, GPSS is mixing much quicker than the [latent slice sampler](@ref latent) (LSS) at a similar per-iteration cost.


[^SHR2023]: Schär, P., Habeck, M., & Rudolf, D. (2023, July). Gibbsian polar slice sampling. In International Conference on Machine Learning.
Expand Down
Loading