Windows dependencies

[Mistuke]

What's the minimum supported GHC version for hmatrix?

I've noticed the dependencies for Windows are as follows:

    if os(windows)
        if flag(openblas)
            extra-libraries:    libopenblas, libgcc_s_seh-1, libgfortran, libquadmath-0
        else
            extra-libraries:    blas lapack

This essentially limits hmatrix to only dynamic linking (because these names will force the finding of a dll in some cases) and also only to x86_64 GCC because of libgcc_s_seh-1 (32 bit GCCs that we use in GHC use different exception handling mechanisms, so it has libgcc_s_dw2-1.dll instead).

Ideally this should just be

    if os(windows)
        if flag(openblas)
            extra-libraries:    openblas, gcc, gfortran, quadmath
        else
            extra-libraries:    blas lapack

But whether this will work or not will depend on whether or which minimum GHC is supported.

Also as a side note, you won't be able to use hmatrix reliably in ghci until the next major release of GHC (8.4) if the C files are compiled using -O3 which turns on SSE vectorization, at the moment you'll get random segfaults (see https://ghc.haskell.org/trac/ghc/ticket/13617).

[albertoruiz]

Not sure about the minimum version, I usually test only on the latest ones. And unfortunately I cannot test on windows, sorry. But pull requests are welcome!

[rossng]

I've also been trying to get hmatrix building on Windows so I can run grenade. I think it's working now!

I documented the steps I took in some detail on my fork. Would much appreciate it if someone (@Mistuke?) could look at these and have a go on their machine before I make a pull request.

In order to use my local build of hmatrix for grenade, I bumped the version numbers of hmatrix to 0.18.1.2 and hmatrix-gsl to 0.18.0.2 and made the following changes to grenade's stack.yaml:

• In packages:, add lines:

- {relative_path_to_hmatrix}/packages/base
- {relative_path_to_hmatrix}/packages/gsl

• In extra-deps, add lines:

- hmatrix-0.18.1.2
- hmatrix-gsl-0.18.0.2

[Mistuke]

Hi @rossng,

A couple of comments:

• why not just install mingw-w64-x86_64-openblas from msys2, that way you don't have to compile it. (also canonical way of writing arch independent way of these packages is mingw-$(uname -m)-openblas
• Also to install libgfortran you can just install libgfortran mingw-w64-$(uname -m)-gcc-libgfortran, the file being just 210k means I can probably also bundle it with GHC as fortran is likely used a lot for these sort of things

If you install gcc-fortran now, you will get libgfortran version 7 or later. In libgfortran 7, the DLL changed from libgfortran-3.dll to libgfortran-4.dll. The reference to libgfortran was updated in hmatrix 0.18.1.0, but this doesn't seem to work on Windows. Here are some steps that might resolve errors about missing libraries

libgfortran is actually correct, though the lib prefix is unneeded. It's supposed to be an import library which will point to the correct dll. However GHC 8.2 doesn't recognize them unless the extension is .lib or .dll.a but in this case it's .a. This is corrected in GHC 8.4. So while for now you need gfortran-4 for ghci, this is a bug.

Also even though your tests ran, they're not stable in GHCi. hmatrix uses SSE vectorization, which has a 16 byte alignment requirement. Currently the GHCi loader only does 8 byte alignments. So your chances of ending up on a 16 byte aligned border is 50%. If you do, it'll work, if you don't, it'll segfault. The fix for this was supposed to be in GHC 8.4, https://phabricator.haskell.org/D3915 but because of the new shorter deadlines it'll now be pushed to GHC 8.6.

[kewp]

@rossng Thanks for the run-down. In your INSTALL.md you say

make the following changes to the following hmatrix Cabal files

How did you do this - Did you edit the local versions that stack pulled? Where are those stored?

[rossng]

@kewp I think I had cloned the git repository and was just trying build hmatrix from that. As such, I could just edit the files directly and run stack build. If you do manage to get it to work then you can use a local package source to include it in other Stack projects.

I gave up on this after reading Mistuke's comment - looks like the fix he linked to is still in progress and hopefully going into GHC 8.8.1 (March 2019). I'm not familiar with GHC internals so you might want comment from someone a bit more qualified!

[Mistuke]

@rossng It's been committed, it'll be in 8.8. I hadn't been able to get anyone to review the changes in time for 8.6 sadly.

[kewp]

Huh. Well I managed to get things working. Didn't need to do anything special with libfortran...

[kewp]

Thanks @rossng. I did something similar and managed to get it working

[Magalame]

Did anyone here have an issue with Sundials?

[mpilgrem]

EDIT: The steps below have been revised in the light of @Mistuke's subsequent comments.

I have been trying to build hmatrix-0.20.0.0 on Windows 10 version 2004 using stack. I can do that if:

1. I install the mingw-w64-x86_64-openblas package, in the MSYS2 provided by stack. That involved:

a. Running msys2_shell.cmd, found in the msys2-20180531 folder of the folder returned by stack path --programs, to open the MSYS2 terminal window.

b. Running pacman -S mingw-w64-x86_64-openblas in the MSYS2 terminal window, to install the package.

2. In a local clone of the hmatrix repository, edit packages\base\hmatrix.cabal (a) to remove the dependencies on libgfortran and libquadmath-0 libraries and (b) (EDIT) specify the import libraries (so openblas rather than libopenblas and gcc_s rather than libgcc_s_seh-1), if the openblas flag is true. So:

if os(windows)
        if flag(openblas)
            extra-libraries:    libopenblas, libgcc_s_seh-1, libgfortran, libquadmath-0
        else
            extra-libraries:    blas lapack

becomes

if os(windows)
        if flag(openblas)
            extra-libraries:    openblas, gcc_s
        else
            extra-libraries:    blas lapack

The dependency on libgfortan seems to be particularly problematic, if included. EDIT: MSYS2 package mingw-w64-x86_64-gcc-fortran provides import libraries libgfortran.a and libgfortran.dll.a in folder /mingw64/lib/gcc/x86_64-w64-mingw32/10.2.0.

3. In packages\base, build with:

stack --resolver nightly-2020-07-24 build --flag hmatrix:openblas

I use the GHC 8.10.1 resolver, because GHC 8.8.3 is broken on Windows 10 version 2004. However, --resolver lts-16.6 (GHC 8.8.3) also works here.

It seems to build, but the error/warning below is reported - I do not know its significance:

src\Internal\C\vector-aux.c: In function 'urandom':

src\Internal\C\vector-aux.c:983:11: error:
     warning: implicit declaration of function 'rand_s'; did you mean 'rand'? [-Wimplicit-function-declaration]
         err = rand_s(&number);
               ^~~~~~
               rand
    |
983 |     err = rand_s(&number);
    |           ^

[Mistuke]

See my previous comment. You do not ever link against a dll. Ever. You link against its import library or you can violate the program's code model.

Binutils allows this because of historical reasons and for 32-bit code it didn't matter if it was supposed to use strict bindings. But do not ever link against a dll. See my very long post on the topic explaining why.. https://hub.zhox.com/posts/why-doesnt-ghc-on-windows-find-my-dll/

[mpilgrem]

Further to my comment above, if I take the following additional steps, I can build all of the packages in the hmatrix repository on Windows 10 version 2004 using stack:

4. I also install the mingw-w64-x86_64-gsl and mingw-w64-x86_64-glpk packages, in the MSYS2 provided by stack (see Step 1):

pacman -S mingw-w64-x86_64-gsl
pacman -S mingw-w64-x86_64-glpk

5. In the local clone of the hmatrix repository (see Step 2), edit packages\gsl\hmatrix-gsl.cabal to change the dependency from gsl-0 to gsl (see also Why hmatrix-gsl needs gsl-0 #312). So:

    if os(windows)
        extra-libraries: gsl-0

becomes

    if os(windows)
        extra-libraries: gsl

6. In that local clone, edit examples\examples.cabal to remove the upper limit on the base package. So:

build-depends:       base >=4.10 && <4.11,

becomes

build-depends:       base >=4.10 && <5,

7. In root folder hmatrix, build using stack with:

stack --resolver nightly-2020-07-24 build --flag hmatrix:openblas

8. I can then test with:

stack --resolver nightly-2020-07-24 ghci --flag hmatrix:openblas
Configuring GHCi with the following packages: examples, hmatrix, hmatrix-glpk, hmatrix-gsl, hmatrix-special, hmatrix-tests
GHCi, version 8.10.1: https://www.haskell.org/ghc/  :? for help
...
Ok, 81 modules loaded.
...
> Numeric.LinearAlgebra.Tests.runTests 20
------ index
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ mult Double
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ sub-trans
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ ctrans
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ lu
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ inv (linearSolve)
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ luSolve
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ ldlSolve
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ cholSolve
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ luSolveLS
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests (5% trivial).
+++ OK, passed 100 tests (5% trivial).
------ pinv (linearSolveSVD)
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ det
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ svd
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ eig
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ geig
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ nullSpace
+++ OK, passed 100 tests (59% trivial).
+++ OK, passed 100 tests (50% trivial).
------ qr
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ hess
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ schur
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ chol
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ expm
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ vector operations - Double
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ read . show
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
+++ OK, passed 100 tests.
------ some unit tests
Cases: 52  Tried: 52  Errors: 0  Failures: 0

[mpilgrem]

@Mistuke, I have read your comment and post, but I may not understand fully its implications for a user of hmatrix getting it to build on Windows 10.

I note the following:

[1] mingw-w64-x86_64-openblas, mingw-w64-x86_64-gsl and mingw-w64-x86_64-glpk do provide *.a or *.dll.a files (or lib*.a and lib*.dll.a files) at a location that stack adds (/mingw64/lib)

[2] I think MSYS2 only provides /mingw64/bin/libgcc_s_seh-1.dll (via a dependency of mingw-w64-x86_64-openblas)

[3] In the case of the (apparently unnecessary) libgfortran and libquadmath-0, I think MSYS2 only provides /mingw64/bin/libgfortran-5.dll (here) and /mingw64/bin/libquadmath-0.dll (here).

In your post, you write: "So what is one supposed to do when you only have a DLL? Create an import library from the Microsoft DEF file." Is the implication that a user should, perhaps, take the MSYS2-provided *.dll files (where the import library files are not available) and use tools to create import library files? In that regard, I found this post: Generate a DEF file from a DLL.

[idontgetoutmuch]

@mpilgrem and @Mistuke thanks for working on this

[Mistuke]

@mpilgrem well..

@Mistuke, I have read your comment and post, but I may not understand fully its implications for a user of hmatrix getting it to build on Windows 10.

I note the following:

[1] mingw-w64-x86_64-openblas, mingw-w64-x86_64-gsl and mingw-w64-x86_64-glpk do provide *.a or *.dll.a files (or lib*.a and lib*.dll.a files) at a location that stack adds (/mingw64/lib)

Correct, your import libraries are compile time libraries so are in lib not bin. That's the thing you need to have in your library search path. Anything in bin is only required for running, not for linking. If you want libfoo you specify as extra-library foo without the lib prefix. Providing this prefix removes choice from the linker and is usually used when you want to force one way of linking over the other.

by using foo you give the linker (and user) the choice between static and dynamic linking if both a .a and .dll.a or .lib is provided.

In other words, this allows users to choose how they want to link, further it also preserves the binding mode the developer intended. If the developer wanted lazy binding and you link directly against a DLL you force strict binding. Unlike Linux, the shared library does not contain a PLT and GOT section inside the library itself. This is externalized in the import libraries.

[2] I think MSYS2 only provides /mingw64/bin/libgcc_s_seh-1.dll (via a dependency of mingw-w64-x86_64-openblas)

It provides libgcc.a and libgcc_s.a so you should use gcc or gcc_s. Also modern GHCs already add gcc always for stack checking. That's because static linking is preferred here and libgcc even though it's GPL has a runtime exception clause in the license.

libgcc_s_seh-1.dll is wrong because aside from forcing a link against a very specific DLL, it also forces linking against one specific EH (Exception Handling) model. The 32-bit versions of GCC are configured with a different EH model for instance so this dll won't exist.

This is why the import libraries exist. They allow the linker to be able to resolve these differences while making it so that you don't have to change how you compile your program.

[3] In the case of the (apparently unnecessary) libgfortran and libquadmath-0, I think MSYS2 only provides /mingw64/bin/libgfortran-5.dll (here) and /mingw64/bin/libquadmath-0.dll (here).

Those packages are intended to provide the runtime dependencies of libgfortran and libquadmath. If you want the developer versions to link against then you need https://packages.msys2.org/package/mingw-w64-x86_64-gcc-fortran and https://packages.msys2.org/package/mingw-w64-x86_64-gcc which also give you the related header files. libquadmath is a dependency of GCC so you already have its import libraries as soon as you have a working C compiler.

In your post, you write: "So what is one supposed to do when you only have a DLL? Create an import library from the Microsoft DEF file." Is the implication that a user should, perhaps, take the MSYS2-provided *.dll files (where the import library files are not available) and use tools to create import library files? In that regard, I found this post: Generate a DEF file from a DLL.

Yes, but no library that is intended to be used by developers as a link rather than a runtime target will ship without an import library (a .dll.a or .lib). You just need the correct packages. This is why typically you have -devel packages which are intended to provide the extra things developers need to use the package whereas the non-devel packages contain the things the user needs to run something written using the package.

so like I mentioned in my original post, you don't need the prefixes or suffixes for the libraries. The search order of libraries on Windows is documented https://sourceware.org/binutils/docs/ld/WIN32.html and it's

          libxxx.dll.a
          xxx.dll.a
          libxxx.a
          xxx.lib
          cygxxx.dll (*)
          libxxx.dll
          xxx.dll

by prefixing it with lib you prevent it from being able to find lib style import libraries, because typically those are not named with lib in the name.

[mpilgrem]

@Mistuke, thank you for your comprehensive further explanation. I have changed the Step 2 in my original comment, in the light of it. I will propose a pull request, that reflects these steps.

[mpilgrem]

Actually, after a bit of experimentation, I think the dependency on gcc_s is also unnecessary. The following works without it:

stack --resolver nightly-2020-07-27 ghci --flag hmatrix:openblas
...
> Numeric.LinearAlgebra.Tests.runTests 20
...
> Numeric.GSL.Tests.runTests 20
...