-
Notifications
You must be signed in to change notification settings - Fork 422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multivariate Gaussian Problem with Assumptions on Covariance #791
Comments
Actually, it's because it's not quite symmetric (if you wrap it in a If we proceed with #307, we could allow something like
Though what that would do internally isn't quite clear (i.e. how should the covariance component of |
This is actually a good argument for getting rid of PDMats, so as to make it easier for custom factorizations (see #688 (comment)). If all you're doing is sampling, then the most efficient way to store it is as a If you want to compute |
How is 𝐉'𝐖𝐉, that is, 2×2 Array{Float64,2}: not symmetric? |
Floating point rounding error:
|
Oh wow! Thanks! Could we maybe always wrap this around? At least for errors of rounding magnitude? I am pretty sure that I am not the only one who ended up one afternoon cursing and wondering whether something was wrong in my math as it happened only for 1/230 cases... |
Allover, I am so in love with Julia & Distributions.jl. But getting such kinks out would be really good, I want to make my Julia code so nice that I can avoid doing pseudo-code and doing without a Symmetric wrapper would be really nice in this respect. |
Unfortunately it's far from trivial to figure out what an appropriate tolerance should be. We should at least give a more informative error though (i.e. suggest the |
@simonbyrne is there something to do on this one? Add a |
The warning would probably go in the PDMats constructor. |
I am using Optim together with Distributions for Laplace approximations and run into the same issues where the estimated covariance matrix very often have rounding errors and is not exactly symmetric. This causes the call to |
I found https://github.com/timholy/PositiveFactorizations.jl to be helpful for dealing with numerical issues arising from estimated covariance matrices. You should be able to compute the (approximate) Cholesky decomposition with PositiveFactorizations manually, and then construct a PDMat that you can use a covariance matrix for MvNormal. |
Symmetry issues aside, the fact remains that |
PDMatsExtras adds support for psd matrices to PDMats. You should be able to use it to wrap your psd covariance matrix in a format that MvNormal accepts. |
@devmotion: quick question on usage of using PDMats, PDMatsExtras
m = rand(2,2); # some 2x2 matrix
pd = PDMat(a*a'); # some PD matrix
e = [0, 1];
psd = PSDMat(Symmetric(a*Diagonal(e)*a')); # some PSD matrix
eigmin(psd) # check PSD
X_A_Xt(pd, a) # all good
X_A_Xt(psd, a) #! not defined Should I be importing both |
I am not familiar with PDMatsExtras (have neither used it nor contributed to it) but to me it seems this is a bug in PDMatsExtras and the definitions in https://github.com/invenia/PDMatsExtras.jl/blob/2c689308b846d69cfbced00d4108f77105a60de3/src/psd_mat.jl#L109-L132 should be qualified (i.e., they should define |
Thanks and sorry for the making the assumption you were a contributor. |
I am working on a few tutorials in machine learning as well as papers. One recurrent theme in these algorithms is that there are linear transformations applied to Gaussian variables, e.g.,
δu = rand(MvNormal([0,0],𝐖));
δx = 𝐉*δu;
However, in most cases, we aim to transform the covariance matrix and use
δx = rand(MvNormal([0,0],𝐉'𝐖𝐉));
instead (that happens in really well-known things such as Kalman filters). It also causes a problem with Julia's implementation of Gaussians as ,𝐉'𝐖𝐉 is not always positive definite and with
𝐉 = [-0.707805 0.706408; 0.706408 0.707805];
𝐖 = diagm(0=>[1,0.1])/20;
our Method 1 would work but Method 2 would fail as 𝐉'𝐖𝐉 is only semi-positive definite. This case of high practical relevance! May I appeal to changing the requirement from positive definite to semi-positive definite?
I would like to use Distributions.jl directly instead of having to create a wrapper around it!
The text was updated successfully, but these errors were encountered: