hGetContents: invalid argument (invalid byte sequence) #96

nh2 · 2014-11-07T21:58:27Z

Running hlint v1.9.11 on my file containing unicode characters, I get:

hlint: myfile.hs: hGetContents: invalid argument (invalid byte sequence)

The problem is explained by @hvr here: finnsson/template-helper#2

The text was updated successfully, but these errors were encountered:

ndmitchell · 2014-11-07T22:00:34Z

Can you try passing the --utf8 flag and see if that fixes it? From @hvr's comment it sounds like that should at least be the default, if not the only option.

nh2 · 2014-11-07T22:02:28Z

I should have said, I'm already using the --utf8 flag.

nh2 · 2014-11-07T22:03:08Z

By the way, it's failing in this file: https://github.com/ganeti/ganeti/blob/master/test/hs/Test/Ganeti/Attoparsec.hs#L45

ndmitchell · 2014-11-07T22:04:04Z

OK, thanks for the test case, I'll take a look (probably Monday - my parents are in town this weekend).

nh2 · 2014-11-07T22:10:16Z

I noticed that it works well with 1.8.57, but breaks in 1.8.58 and above.

nh2 · 2014-11-07T22:23:21Z

My git bisect says that it's a96570b.

ndmitchell · 2014-11-07T22:25:16Z

Thanks. Then I almost certainly just screwed up the flags and UTF8 no longer gets passed down as far as it should. Hopefully easy to fix.

nh2 · 2014-11-07T22:29:12Z

Bisection with git bisect run:

% git bisect good v1.8.57
% git bisect bad v1.8.58
Bisecting: 11 revisions left to test after this (roughly 4 steps)
[edee5669a5d7a003a4d8f0ddf0467c3277821745] Update Acknowledgements in README
niklas@niklas ~/src/hlint (git)-[edee566...|bisect] % git bisect run sh -c 'cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs'
running sh -c cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs
Resolving dependencies...
In order, the following will be installed:
hlint-1.8.57 (latest: 1.9.11) (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Installed hlint-1.8.57
No suggestions
Bisecting: 5 revisions left to test after this (roughly 3 steps)
[876f1af8ace66b0fa8e73e88f2a8c7be434b1a01] #21, strip C style comments
running sh -c cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs
Resolving dependencies...
In order, the following will be installed:
hlint-1.8.57 (latest: 1.9.11) (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Installed hlint-1.8.57
hlint: Attoparsec.hs: hGetContents: invalid argument (invalid byte sequence)
Bisecting: 2 revisions left to test after this (roughly 2 steps)
[8360f4532189589ebfe80d0664a393aad1fbe0ee] Require cpphs-1.18.1 which fixes a pattern match error that seemingly Happy now trips over
running sh -c cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs
Resolving dependencies...
In order, the following will be installed:
hlint-1.8.57 (latest: 1.9.11) (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Installed hlint-1.8.57
No suggestions
Bisecting: 0 revisions left to test after this (roughly 1 step)
[32b0fb45a65203bc6c80cceae6b3237eda18f538] Indent an else slightly
running sh -c cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs
Resolving dependencies...
In order, the following will be installed:
hlint-1.8.57 (latest: 1.9.11) (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Installed hlint-1.8.57
hlint: Attoparsec.hs: hGetContents: invalid argument (invalid byte sequence)
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[a96570bd43e33545e994e2ec55cb8c9b4b8b0254] Move to cmdargs more completely
running sh -c cabal install --disable-library-profiling --disable-documentation && LANG= $HOME/.cabal/bin/hlint --utf8  Attoparsec.hs
Resolving dependencies...
In order, the following will be installed:
hlint-1.8.57 (latest: 1.9.11) (reinstall)
Warning: Note that reinstalls are always dangerous. Continuing anyway...
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Failed to install hlint-1.8.57
Build log ( $HOME/.cabal/logs/hlint-1.8.57.log ):
Configuring hlint-1.8.57...
Building hlint-1.8.57...
Preprocessing library hlint-1.8.57...
[35 of 36] Compiling HLint            ( src/HLint.hs, dist/build/HLint.o )

src/HLint.hs:71:12:
    Unexpected semi-colons in conditional:
        if null files then error "No files found"; else runHints
                                                          (cmd {cmdFiles = files}) flags
    Perhaps you meant to use DoAndIfThenElse?
cabal: Error: some packages failed to install:
hlint-1.8.57 failed during the building phase. The exception was:
ExitFailure 1
a96570bd43e33545e994e2ec55cb8c9b4b8b0254 is the first bad commit
commit a96570bd43e33545e994e2ec55cb8c9b4b8b0254
Author: Neil Mitchell <[email protected]>
Date:   Mon Feb 24 18:18:55 2014 +0000

    Move to cmdargs more completely

:040000 040000 7cdc20b6de6a1bff0abe624f52f280e5ecc606d4 e5e26268124c7b0cf6db045a807c58d98f8a3f89 M  src
bisect run success
  50.93s user 5.18s system 100% cpu 55.813 total

ndmitchell · 2014-11-08T19:12:01Z

Looking into it more, I think --encoding=UTF-8 should still work, its just the --utf8 flag that got broken. I also don't see this on Windows since it seems to default to something that works for the above file, which is probably why I haven't spotted it broken before now - a Travis test case should fix that going forward though.

ndmitchell · 2014-11-09T10:20:19Z

I fixed the utf8 flag, but also made utf8 the default, so it should still work if you omit the flag. Released as hlint-1.9.12.

ndmitchell · 2014-11-09T10:21:14Z

BTW, thanks for your efforts tracking this down - made it easy enough that I could fix it quickly while my parents popped out :). I haven't used git bisect run before, only ever done it manually, so I'll have to give that a try at some point.

This uses `cabal configure` to determine which exact dependency versions we are compiling against, and ensures that these versions are used by passing -package-id flags to GHC. The `cabal configure` step makes the build fail before compiling / type checking if the user tries to compile against a dependency version we don't support; before, this case led to type errors which were not clearly user errors. This fixes issue ganeti#988. The output of `cabal configure` is also used to generate MIN_VERSION_* macros. MIN_VERSION_* macros are the standard way to build CPP dependency switches in Haskell packages, and they replace our custom macros (like PARALLEL3 and NO_REGEX_PCRE) which had to be hand-built for each dependency. We can now query the version of any Haskell dependency without having to manually add a flag via autoconf. All ghc and hlint invocations were adjusted to take these macros into account. This change introduces a Haskell-build-time dependency on cabal-install (for `cabal configure`) and the Cabal API (for obtaining the configured dependency versions and generating the macros). Any cabal version since Debian Squeeze is supported. Note that our use of Cabal does not imply any downloading of dependencies at build time, hermetic builds are unaffected by this change. For developers we now require hlint >= 1.8.60, to make use of its --cpp-file option. However, hlint >= 1.9.12 is recommended since for hlint >= 1.8.58 && < 1.9.12 the --utf8 flag is non-functional (see ndmitchell/hlint#96); this can be worked though by using the equivalent `--encoding=UTF-8` flag.

This uses `cabal configure` to determine which exact dependency versions we are compiling against, and ensures that these versions are used by passing -package-id flags to GHC. The `cabal configure` step makes the build fail before compiling / type checking if the user tries to compile against a dependency version we don't support; before, this case led to type errors which were not clearly user errors. This fixes issue #988. The output of `cabal configure` is also used to generate MIN_VERSION_* macros. MIN_VERSION_* macros are the standard way to build CPP dependency switches in Haskell packages, and they replace our custom macros (like PARALLEL3 and NO_REGEX_PCRE) which had to be hand-built for each dependency. We can now query the version of any Haskell dependency without having to manually add a flag via autoconf. All ghc and hlint invocations were adjusted to take these macros into account. This change introduces a Haskell-build-time dependency on cabal-install (for `cabal configure`) and the Cabal API (for obtaining the configured dependency versions and generating the macros). Any cabal version since Debian Squeeze is supported. Note that our use of Cabal does not imply any downloading of dependencies at build time, hermetic builds are unaffected by this change. For developers we now require hlint >= 1.8.60, to make use of its --cpp-file option. However, hlint >= 1.9.12 is recommended since for hlint >= 1.8.58 && < 1.9.12 the --utf8 flag is non-functional (see ndmitchell/hlint#96); this can be worked though by using the equivalent `--encoding=UTF-8` flag. Signed-off-by: Niklas Hambuechen <[email protected]> Reviewed-by: Klaus Aehlig <[email protected]>

We now require cabal-install and the Cabal API for building, and hlint >= 1.9.12 since oder versions either did not support the --cpp-file option or had a bug regarding the --utf8 option (which we use; see ndmitchell/hlint#96).

We now require cabal-install and the Cabal API for building, and hlint >= 1.9.12 since oder versions either did not support the --cpp-file option or had a bug regarding the --utf8 option (which we use; see ndmitchell/hlint#96). Signed-off-by: Niklas Hambuechen <[email protected]> Reviewed-by: Helga Velroyen <[email protected]>

See: ndmitchell/hlint#96

xor-xor · 2015-06-21T19:02:19Z

I'm still getting this error with hlint 1.9.21 and 1.9.20 (1.9.19 works fine, though).

ndmitchell · 2015-06-21T20:57:40Z

@xor-xor I'm unaware of any changes that happened with 1.9.20. Can you tell me exactly what you are running, what the error is etc.

xor-xor · 2015-06-22T06:59:01Z

It's weird... I can't reproduce this problem today. Maybe because I've made some changes to the file that I was hlinting (and yes, I forgot to make a copy of it when I encountered this problem - but I think that there was nothing special about this file). The error message was:

hlint: myfile.hs: hGetContents: invalid argument (invalid byte sequence)

(I'm on OS X 10.9.5 w/ Haskell Platform 2014.2.0.0; hlint installed in a sandbox)

If I manage to reproduce that somehow, I'll let you know.

Sorry for the noise! 😉

ndmitchell · 2015-06-22T16:52:14Z

I always appreciate bug reports, even if they can't reproduce later, so thanks! Any chance you were using CPP with include files? If not, let's ignore it til it resurfaces.

ndmitchell added a commit that referenced this issue Nov 9, 2014

#96, fix the --utf8 flag

9767d5a

ndmitchell closed this as completed Nov 9, 2014

nh2 mentioned this issue Dec 11, 2014

hGetContents: invalid argument (invalid byte sequence) google/pull-request-mailer#22

Open

obcode pushed a commit to ob-cs-hm-edu/fun-Blatt01 that referenced this issue May 20, 2015

fix hlints encoding problem

e6bdcd5

See: ndmitchell/hlint#96

lierdakil mentioned this issue Aug 29, 2016

Use HLint3.hlint in Lint DanielG/ghc-mod#829

Merged

andregpss mentioned this issue Jan 9, 2020

Lint error: hGetContents: invalid argument (invalid byte sequence) (Win10 + Atom) haskell/haskell-ide-engine#1560

Closed

ndmitchell mentioned this issue Jun 3, 2020

should --json output always be utf-8? #1030

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hGetContents: invalid argument (invalid byte sequence) #96

hGetContents: invalid argument (invalid byte sequence) #96

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 8, 2014

ndmitchell commented Nov 9, 2014

ndmitchell commented Nov 9, 2014

xor-xor commented Jun 21, 2015

ndmitchell commented Jun 21, 2015

xor-xor commented Jun 22, 2015

ndmitchell commented Jun 22, 2015

hGetContents: invalid argument (invalid byte sequence) #96

hGetContents: invalid argument (invalid byte sequence) #96

Comments

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 7, 2014

nh2 commented Nov 7, 2014

ndmitchell commented Nov 8, 2014

ndmitchell commented Nov 9, 2014

ndmitchell commented Nov 9, 2014

xor-xor commented Jun 21, 2015

ndmitchell commented Jun 21, 2015

xor-xor commented Jun 22, 2015

ndmitchell commented Jun 22, 2015