Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: normalize line endings #105

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Djaytan
Copy link

@Djaytan Djaytan commented Jan 9, 2025

This change is similar to Incendo/cloud#786.

Introduction

When cloning and building the project from a Windows machine, here is the error encountered:

Execution failed for task ':cloud-kotlin-coroutines:spotlessKotlinCheck'.
  > The following files had format violations:
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\extension\command-builder-extensions.kt
            @@ -1,47 +1,47 @@
            -//\n
            -// MIT License\n
            -//\n
            -// Copyright (c) 2024 Incendo\n
            -//\n
            -// Permission is hereby granted, free of charge, to any person obtaining a copy\n
            -// of this software and associated documentation files (the "Software"), to deal\n
            -// in the Software without restriction, including without limitation the rights\n
            -// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n
            -// copies of the Software, and to permit persons to whom the Software is\n
            -// furnished to do so, subject to the following conditions:\n
            -//\n
            -// The above copyright notice and this permission notice shall be included in all\n
            -// copies or substantial portions of the Software.\n
            -//\n
            -// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n
            -// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n
            -// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n
            -// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n
            -// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n
            -// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n
            -// SOFTWARE.\n
            -//\n
            -package org.incendo.cloud.kotlin.coroutines.extension\n
            -\n
            -import kotlinx.coroutines.CoroutineScope\n
            -import kotlinx.coroutines.GlobalScope\n
            -import org.incendo.cloud.Command\n
            -import org.incendo.cloud.kotlin.coroutines.SuspendingExecutionHandler\n
            -import kotlin.coroutines.CoroutineContext\n
            -import kotlin.coroutines.EmptyCoroutineContext\n
            -\n
            -/**\n
            - * Specify a suspending command execution handler.\n
            - *\n
            - * @param scope coroutine scope\n
            - * @param context coroutine context\n
            - * @param handler suspending handler\n
            - * @return modified copy of this [Command.Builder]\n
            - * @see Command.Builder.handler\n
            - * @see SuspendingExecutionHandler\n
            - */\n
            -public fun <C : Any> Command.Builder<C>.suspendingHandler(\n
            -    scope: CoroutineScope = GlobalScope,\n
            -    context: CoroutineContext = EmptyCoroutineContext,\n
            -    handler: SuspendingExecutionHandler<C>\n
            -): Command.Builder<C> = handler(SuspendingExecutionHandler.createCommandExecutionHandler(scope, context, handler))\n
            +//\r\n
        ... (46 more lines that didn't fit)
    Violations also present in:
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\extension\mutable-command-builder-extensions.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingArgumentParser.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingExecutionHandler.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingSuggestionProvider.kt
        src\test\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingArgumentParserTest.kt
        src\test\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingHandlerTest.kt
    Run 'gradlew.bat :cloud-kotlin-coroutines:spotlessApply' to fix these violations.

=> Gradle build scan report can be found here.

Encountering any error is a surprise when knowing that no modification has been made so far.

Investigations

When running the following command:

$ ./gradlew :spotlessApply

I realize that files' line endings are converted from LF to CRLF, which is not something we want since otherwise it will just push the problem to Unix/Mac/whatever users, thus not solving the real issue in the end.

The weird thing is that only Kotlin and YAML files are impacted! No issue on Java files.

When reading a bit about the Spotless plugin's behavior, I can find the following relevant section: link. We can learn that, by default, Spotless rely on the .gitattributes file in order to align itself with Git decisions:

The default value is GIT_ATTRIBUTES_FAST_ALLSAME, and we highly recommend that you do not change this value. Git has opinions about line endings, and if Spotless and git disagree, then you're going to have a bad time.

Fair enough, but that doesn't tell us precisely what FAST_ALLSAME really mean. I had to search in the code itself to find the relevant description helping to better understand the behavior behind this option.

Uses the same line endings as Git, and assumes that every single file being formatted will have the same line ending.

And when digging even more in the Spotless' code, we can find that only the first encountered attributes line is taken into account it seems (assuming that the provided iterator preserve files order).

Since the first line of the .gitattributes define LF line endings, we can assume that this will be the retained value. This explains why there is no issue with Java files, I guess. But this doesn't explain why we encounter the issue with Kotlin and YAML files.

Finally, I decided to stop the investigations here since I was not able to spot the issue, even by trying to dig into the cloud-build-logic and indra projects. I tried to adjust some Git configs on my side as well like core.autocrlf and core.eol, but it doesn't affect the outcome and the error was still triggered. It just convinces me even more to rely on the proposed solution since the underlying tools don't seem reliable enough to support platform-agnostic setup.

Solution: normalizing line endings to LF in the repository

I tend to favor .gitattributes files systematically in order to avoid issues no matter the user Git config or the project setup specificities. I even defined my own GitHub Gist for easy copy/paste. This is working fine in all setup projects so far, therefore I really think it's reliable without adding maintenance overhead.

Then I tried to run the following command:

$ git add --renormalize .

But nothing changed, meaning that all files already have normalized line endings, which is great!

## Introduction

When cloning and building the project from a Windows machine, here is the error encountered:

```
Execution failed for task ':cloud-kotlin-coroutines:spotlessKotlinCheck'.
  > The following files had format violations:
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\extension\command-builder-extensions.kt
            @@ -1,47 +1,47 @@
            -//\n
            -// MIT License\n
            -//\n
            -// Copyright (c) 2024 Incendo\n
            -//\n
            -// Permission is hereby granted, free of charge, to any person obtaining a copy\n
            -// of this software and associated documentation files (the "Software"), to deal\n
            -// in the Software without restriction, including without limitation the rights\n
            -// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n
            -// copies of the Software, and to permit persons to whom the Software is\n
            -// furnished to do so, subject to the following conditions:\n
            -//\n
            -// The above copyright notice and this permission notice shall be included in all\n
            -// copies or substantial portions of the Software.\n
            -//\n
            -// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n
            -// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n
            -// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n
            -// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n
            -// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n
            -// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n
            -// SOFTWARE.\n
            -//\n
            -package org.incendo.cloud.kotlin.coroutines.extension\n
            -\n
            -import kotlinx.coroutines.CoroutineScope\n
            -import kotlinx.coroutines.GlobalScope\n
            -import org.incendo.cloud.Command\n
            -import org.incendo.cloud.kotlin.coroutines.SuspendingExecutionHandler\n
            -import kotlin.coroutines.CoroutineContext\n
            -import kotlin.coroutines.EmptyCoroutineContext\n
            -\n
            -/**\n
            - * Specify a suspending command execution handler.\n
            - *\n
            - * @param scope coroutine scope\n
            - * @param context coroutine context\n
            - * @param handler suspending handler\n
            - * @return modified copy of this [Command.Builder]\n
            - * @see Command.Builder.handler\n
            - * @see SuspendingExecutionHandler\n
            - */\n
            -public fun <C : Any> Command.Builder<C>.suspendingHandler(\n
            -    scope: CoroutineScope = GlobalScope,\n
            -    context: CoroutineContext = EmptyCoroutineContext,\n
            -    handler: SuspendingExecutionHandler<C>\n
            -): Command.Builder<C> = handler(SuspendingExecutionHandler.createCommandExecutionHandler(scope, context, handler))\n
            +//\r\n
        ... (46 more lines that didn't fit)
    Violations also present in:
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\extension\mutable-command-builder-extensions.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingArgumentParser.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingExecutionHandler.kt
        src\main\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingSuggestionProvider.kt
        src\test\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingArgumentParserTest.kt
        src\test\kotlin\org\incendo\cloud\kotlin\coroutines\SuspendingHandlerTest.kt
    Run 'gradlew.bat :cloud-kotlin-coroutines:spotlessApply' to fix these violations.
```

=> Gradle build scan report can be found here: https://scans.gradle.com/s/5hczi6ibxdqry/projects.

Encountering any error is a surprise when knowing that no modification has been made so far.

## Investigations

When running the following command:

```
$ ./gradlew :spotlessApply
```

I realize that files' line endings are converted from LF to CRLF, which is not something we want since otherwise it will just push the problem to Unix/Mac/whatever users, thus not solving the real issue in the end.

The weird thing is that only Kotlin and YAML files are impacted! No issue on Java files.

When reading a bit about the Spotless plugin's behavior, I can find the following relevant section: https://github.com/diffplug/spotless/tree/main/plugin-gradle#line-endings-and-encodings-invisible-stuff.

We can learn that, by default, Spotless rely on the `.gitattributes` file in order to align itself with Git decisions:

> The default value is `GIT_ATTRIBUTES_FAST_ALLSAME`, and *we highly recommend that you **do not change** this value*. Git has opinions about line endings, and if Spotless and git disagree, then you're going to have a bad time.

Fair enough, but that doesn't tell us precisely what `FAST_ALLSAME` really mean. I had to search in the code itself to find the relevant description helping to better understand the behavior behind this option: https://github.com/diffplug/spotless/blob/8219f37abfc8817fcd30571552501c12b01558eb/lib/src/main/java/com/diffplug/spotless/LineEnding.java#L41

> Uses the same line endings as Git, and assumes that every single file being formatted will have the same line ending.

And when digging even more in the Spotless' code, we can find that only the first encountered attributes line is taken into account it seems (assuming that the provided iterator preserve files order): https://github.com/diffplug/spotless/blob/8219f37abfc8817fcd30571552501c12b01558eb/lib-extra/src/main/java/com/diffplug/spotless/extra/GitAttributesLineEndings.java#L95.

Since the first line of the `.gitattributes` define `LF` line endings, we can assume that this will be the retained value. This explains why there is no issue with Java files, I guess. But this doesn't explain why we encounter the issue with Kotlin and YAML files.

Finally, I decided to stop the investigations here since I was not able to spot the issue, even by trying to dig into the `cloud-build-logic` and `indra` projects. I tried to adjust some Git configs on my side as well like `core.autocrlf` and `core.eol`, but it doesn't affect the outcome and the error was still triggered. It just convinces me even more to rely on the proposed solution since the underlying tools don't seem reliable enough to support platform-agnostic setup.

## Solution: normalizing line endings to `LF` in the repository

I tend to favor `.gitattributes` files systematically in order to avoid issues no matter the user Git config or the project setup specificities. I even defined my own GitHub Gist for easy copy/paste: https://gist.github.com/Djaytan/44fe2d1a16c83d48e4e3db6612c7a346. This is working fine in all setup projects so far, therefore I really think it's reliable without adding maintenance overhead.

Then I tried to run the following command:

```
$ git add --renormalize .
```

But nothing changed, meaning that all files already have normalized line endings, which is great!
@Djaytan Djaytan force-pushed the chore/normalize-line-endings branch from c6e57e9 to dd14ae4 Compare January 9, 2025 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant