Now that 4.1 is released, I can work on the Linux port once again #74

felix91gr · 2018-03-31T04:23:18Z

As per a weird Foundation bug in 4.0, we couldn't compile SKD on Linux. Now that 4.1 is out, that Foundation bug is also fixed. I should be able to go back to working on the port now. If anyone wants to help, feel free! Below is what's left to do:

State of the port

Dependencies compile and pass tests on Linux
SKD compiles on Linux
SKD passes tests on Linux

The test suite actually crashes. My current guess is that it's due to some forced unwrappings done in SKD that may be non-Nil on Darwin, but Nil on Linux. We shall see.

After our tests pass, we should be almost done! Here's hoping we can get the port running before Swift 5.0 gets released ✊

felix91gr · 2018-04-01T15:40:36Z

If I’m right with the force-unwrapping issue, then the culprit might be Foundation. Here’s the current status of the Linux implementation. I’m gonna investigate what Foundation types we’re using. If they’re incomplete on Linux, that should be a red flag.

felix91gr · 2018-04-02T04:22:52Z

List of suspicious (or faulty) files

Foundation is incomplete on Linux, therefore I'm writing here all of the files that I find in SKD that use functionality that is either Mostly Complete, Incomplete or Unimplemented as per status page,

Suspicious

Here are the lines that use Mostly Complete parts of Foundation, with the name of the library they use:

Surely Faulty

I couldn't find any use of Incomplete pieces of Foundation.

Definitely Faulty

I couldn't find any use of Unimplemented pieces of Foundation.

Annex: Clean Files

These are the files for which I couldn't find usage of anything that isn't marked as Complete in Foundation's Status Page. In other words, everything they use is marked as Complete, and therefore shouldn't be the source of any errors regarding Foundation.

/Completer/Completer.swift
/Extensions/Path+.swift
/Project/ProjectType.swift

Files that apparently didn't need `Foundation` to begin with

These files still let the project compile after I erased their import Foundation line.

/Extensions/Dictionary+.swift
/Project/ProjectError.swift

felix91gr · 2018-04-02T04:53:37Z

Actually, before we do that, I found that this line in particular crashes. It might be unrelated to Foundation.

The crash is related to XCodeEdit. It's very weird.

felix91gr · 2018-04-02T19:11:45Z

Here's how it's crashing for me:

Have a clean directory:

felix@felix-X550LD ~/D/P/SKD> git clone https://github.com/terhechte/SourceKittenDaemon/ cleanSlate_1

Run make test:

Here's the expected output. The building procedure just works.

felix@felix-X550LD ~/D/P/SKD> cd cleanSlate_1/
felix@felix-X550LD ~/D/P/S/cleanSlate_1> make test
make: agvtool: Command not found
find: Arguments to -type should contain only one letter
FIXTURE_PATH="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures" \
FIXTURE_PROJECT_DIR="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project" \
FIXTURE_PROJECT_FILE_PATH="/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project/Fixture.xcodeproj" \
swift test
Fetching https://github.com/Carthage/Commandant.git
Fetching https://github.com/jpsim/SourceKitten.git
Fetching https://github.com/envoy/Embassy.git
Fetching https://github.com/felix91gr/XcodeEdit.git
Fetching https://github.com/felix91gr/FileSystemWatcher.git
Fetching https://github.com/antitypical/Result.git
Fetching https://github.com/drmohundro/SWXMLHash.git
Fetching https://github.com/jpsim/Yams.git
Fetching https://github.com/norio-nomura/Clang_C.git
Fetching https://github.com/norio-nomura/SourceKit.git
Fetching https://github.com/felix91gr/inotify.git
Cloning https://github.com/Carthage/Commandant.git
Resolving https://github.com/Carthage/Commandant.git at 0.12.0
Cloning https://github.com/felix91gr/XcodeEdit.git
Resolving https://github.com/felix91gr/XcodeEdit.git at 1.1.2
Cloning https://github.com/jpsim/Yams.git
Resolving https://github.com/jpsim/Yams.git at 0.4.1
Cloning https://github.com/norio-nomura/SourceKit.git
Resolving https://github.com/norio-nomura/SourceKit.git at 1.0.1
Cloning https://github.com/envoy/Embassy.git
Resolving https://github.com/envoy/Embassy.git at 4.0.2
Cloning https://github.com/felix91gr/inotify.git
Resolving https://github.com/felix91gr/inotify.git at 1.0.2
Cloning https://github.com/norio-nomura/Clang_C.git
Resolving https://github.com/norio-nomura/Clang_C.git at 1.0.2
Cloning https://github.com/drmohundro/SWXMLHash.git
Resolving https://github.com/drmohundro/SWXMLHash.git at 4.6.0
Cloning https://github.com/antitypical/Result.git
Resolving https://github.com/antitypical/Result.git at 3.2.4
Cloning https://github.com/felix91gr/FileSystemWatcher.git
Resolving https://github.com/felix91gr/FileSystemWatcher.git at 1.1.2
Cloning https://github.com/jpsim/SourceKitten.git
Resolving https://github.com/jpsim/SourceKitten.git at 0.18.4
Compile CYaml src/writer.c
Compile CYaml src/scanner.c
Compile CYaml src/reader.c
Compile CYaml src/parser.c
Compile CYaml src/emitter.c
Compile CYaml src/api.c
Compile inotify inotify_wrapper.c
Compile Swift Module 'XcodeEdit' (5 sources)
Compile Swift Module 'SWXMLHash' (3 sources)
Compile Swift Module 'Result' (2 sources)
Compile Swift Module 'Embassy' (29 sources)
Compile Swift Module 'FileSystemWatcher' (1 sources)

Then it gives me a bunch of warnings related to Swift 4.1...

/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/XcodeEdit.git--3656023722064191949/Sources/Extensions.swift:19:17: warning: 'flatMap' is deprecated: Please use compactMap(_:) for the case where closure returns an optional value
    return self.flatMap { $0 as? T }
                ^
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/XcodeEdit.git--3656023722064191949/Sources/Extensions.swift:19:17: note: use 'compactMap(_:)' instead
    return self.flatMap { $0 as? T }
                ^~~~~~~
                compactMap
Compile Swift Module 'Yams' (15 sources)
Compile Swift Module 'Commandant' (10 sources)
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Embassy.git-6935331987155158314/Sources/TCPSocket.swift:221:49: warning: overlapping accesses to 'address', but modification requires exclusive access; consider copying to a local variable
        return try withUnsafeMutablePointer(to: &address) { pointer in
                                                ^~~~~~~~
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Embassy.git-6935331987155158314/Sources/TCPSocket.swift:231:34: note: conflicting access is here
            switch Int32(address.ss_family) {
                         ~~~~~~~~^~~~~~~~~
/home/felix/Documents/Programming/SKD/cleanSlate_1/.build/checkouts/Commandant.git-3714899713033202887/Sources/Commandant/ArgumentParser.swift:71:19: warning: 'characters' is deprecated: Please use String or Substring directly

And then it finishes compiling:

Compile Swift Module 'sourcekittend' (2 sources)
Compile Swift Module 'SourceKittenDaemonTests' (7 sources)
Linking ./.build/x86_64-unknown-linux/debug/sourcekittend
Compile Swift Module 'SourceKittenDaemonPackageTests' (1 sources)
Linking ./.build/x86_64-unknown-linux/debug/SourceKittenDaemonPackageTests.xctest

So far so good. But now is when it fails:

Test Suite 'All tests' started at 2018-04-02 19:02:54.140
Test Suite 'debug.xctest' started at 2018-04-02 19:02:54.294
Test Suite 'CompleterTests' started at 2018-04-02 19:02:54.294
Test Case 'CompleterTests.testCompletingAConstructor' started at 2018-04-02 19:02:54.294
[DEBUG] Checking setUp func in CompleterTests
[DEBUG] Starting super.setUp...
[DEBUG] Finished!. Creating a test var of type xcodeprojFixturePath...
[DEBUG] Worked!. Creating ProjectType from xcodeprojFixturePath...
[DEBUG] [EXTRA] Printing ProjectType
some(SourceKittenDaemon.ProjectType.project("/home/felix/Documents/Programming/SKD/cleanSlate_1/Tests/SourceKittenDaemonTests/Fixtures/Project/Fixture.xcodeproj"))
[DEBUG] [EXTRA] Success!
[DEBUG] ProjectType is in order. Poking project var...
[DEBUG] The error seems to be in the following line.
[DEBUG] It will fail on the 'public convenience init(propertyListData data: Data)' method,
[DEBUG] On XCProjectFile.swift
Fatal error: 'try!' expression unexpectedly raised an error: Data in .pbxproj file not in expected format: file /home/buildnode/jenkins/workspace/oss-swift-4.1-package-linux-ubuntu-16_04/swift/stdlib/public/core/ErrorType.swift, line 184
Current stack trace:
0    libswiftCore.so                    0x00007fe51cd3f220 _swift_stdlib_reportFatalErrorInFile + 221
1    libswiftCore.so                    0x00007fe51caae33c <unavailable> + 1368892
2    libswiftCore.so                    0x00007fe51cce7f12 <unavailable> + 3702546
3    libswiftCore.so                    0x00007fe51cce9379 <unavailable> + 3707769
4    libswiftCore.so                    0x00007fe51caada36 <unavailable> + 1366582
5    libswiftCore.so                    0x00007fe51cce7cdb <unavailable> + 3701979
6    libswiftCore.so                    0x00007fe51caada36 <unavailable> + 1366582
7    libswiftCore.so                    0x00007fe51cc1ae69 <unavailable> + 2862697
8    libswiftCore.so                    0x00007fe51caeb700 swift_unexpectedError + 318
Exited with signal code 4
Makefile:28: recipe for target 'test' failed
make: *** [test] Error 1

The [DEBUG] messages are debug prints that I put there. They only get compiled if we're on Linux.

XCProjectFile.swift is the one that makes the test suite crash. It's part of XCodeEdit (previously called XCode.Swift). I don't know why it crashes. This is the line that makes it crash

felix91gr · 2018-04-02T19:20:18Z

Regarding XCodeEdit / XCode.Swift:

They changed names when they hit v1.0
I was updating us to use it, and made a fork in the meantime because we had some API issues with the version that was then current
They have since released v2.0 and v2.1. Maybe they updated the API such that we can rely on them better.

I'll also ask about this crash. I don't understand what's the reason and they probably know.

felix91gr self-assigned this Mar 31, 2018

felix91gr added the help wanted label Mar 31, 2018

felix91gr added this to the Linux port milestone Mar 31, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Now that 4.1 is released, I can work on the Linux port once again #74

Now that 4.1 is released, I can work on the Linux port once again #74

felix91gr commented Mar 31, 2018 •

edited

Loading

felix91gr commented Apr 1, 2018

felix91gr commented Apr 2, 2018

felix91gr commented Apr 2, 2018 •

edited

Loading

felix91gr commented Apr 2, 2018 •

edited

Loading

felix91gr commented Apr 2, 2018 •

edited

Loading

Now that 4.1 is released, I can work on the Linux port once again #74

Now that 4.1 is released, I can work on the Linux port once again #74

Comments

felix91gr commented Mar 31, 2018 • edited Loading

State of the port

felix91gr commented Apr 1, 2018

felix91gr commented Apr 2, 2018

List of suspicious (or faulty) files

Suspicious

Surely Faulty

Definitely Faulty

Annex: Clean Files

Files that apparently didn't need Foundation to begin with

felix91gr commented Apr 2, 2018 • edited Loading

felix91gr commented Apr 2, 2018 • edited Loading

felix91gr commented Apr 2, 2018 • edited Loading

felix91gr commented Mar 31, 2018 •

edited

Loading

Files that apparently didn't need `Foundation` to begin with

felix91gr commented Apr 2, 2018 •

edited

Loading

felix91gr commented Apr 2, 2018 •

edited

Loading

felix91gr commented Apr 2, 2018 •

edited

Loading