Configuring experiments as "enable-record-ecosystem-meta=false" in dependabot@2 doesn't prevent associated logs to be displayed

[emmanwl]

Describe the bug
I run a dependabot@2 (2.40.1110) task and I am setting experiments to disable enable-record-ecosystem-meta by providing
experiments: 'enable-record-ecosystem-meta=false' as a task input.

In the task log, I still can see errors traces like:

• cli | *** | unexpected output type: record_ecosystem_data

• updater | *** | ERROR <job_update_0_npm_and_yarn_all> Failed to record ecosystem meta after 3 retries

I was supposing that disabling this experiment would result in not displaying those lines: the thing is those retries are time consuming from what I see.

Not sure this is a bug but I am missing something obviously.

I have done the same test by providing the V2 task a similar input (but using underscores between words instead of dashes since I found references to both in the documentation):
experiments: 'enable_record_ecosystem_meta=false' but it produces the same result.

Categorization

• This is not a permissions issue (Seek help at 403 error when Dependabot tries to create a pull request #1245)

Repository
NA

To Reproduce
Steps to reproduce the behavior:

1. Set experiments: 'enable-record-ecosystem-meta=false' as a dependabot@2 task input
2. Run the task, notice that the new experiments value(s) was/were correctly passed into parameter but that the "faulty" log lines remain.

Expected behavior
I would expect those log lines not to be displayed (and presumably the underlying retries not to be performed)

Logs and screenshots
See extension troubleshooting for more on how to collect additional diagnostic logs.

Extension (please complete the following information):

• Host: [Azure DevOps Service]
• Version [e.g. Dev19.M247.1]

Additional context
None.

[SKoningScanplan]

Encountering the same issue. PRs are still created but the errors make the ADO pipeline fail.

[rhyskoedijk]

@emmanwl @SKoningScanplan thanks for reporting, could one of you please provide the relevant parts of you dependabot.yml for this update, so I can attempt to reproduce. At minimum, I need to know which ecosystem is being used.

[SKoningScanplan]

@rhyskoedijk Here you go:

version: 2
updates:
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "weekly"
    ignore:
      - dependency-name: "mlflow*"
      - dependency-name: "numpy"
      - dependency-name: "setuptools"
    groups:
      azure-ml-deps:
        patterns:
          - "azureml-*"
          - "inference-schema"
          - "azure-ai-ml"
    target-branch: "dependabot-collection-branch"
    open-pull-requests-limit: 5

[emmanwl]

Hello,

here is mine @rhyskoedijk (not same ecosystem as @SKoningScanplan):

version: 2
updates:
  - package-ecosystem: npm
     directory: "/"
     azureDevopsAccessToken: '$(System.AccessToken)'
     versioning-strategy: 'auto'
     open-pull-requests-limit: 10
     commit-message:
       prefix: "build"
       include: "scope"
     registries:
       - my-corp-npm-registry
registries:
  my-corp-npm-registry:
    type: npm-registry
    url: 'https://some-path/'
    username: some-user-name
    password: some-user-password

I have tried another task execution, the retries I mentionned are actually performed and the task never ends; indeed it's timed out after running during nearly 7 hours:
We stopped hearing from agent self-hosted-agent-XXX... verify the agent machine is running and has a healthy internet connection...."
I can't tell since when this behaviour arises, it has not always been the case, that I am sure of.

[leloscheidt]

Hello, here is my dependabot.yml file and I got the same issue as @emmanwl , but my pipeline expires in 1hr. No PRs could be created in that time:

version: 2
updates:  
  - package-ecosystem: "npm"    
       directory: "/"    
       schedule:      
         interval: "daily"    
      target-branch: "master"  

I also tried what was suggested and added the experiments: 'enable-record-ecosystem-meta=false' but still the same error.

[rhyskoedijk]

Explicitly setting it to false will not work currently (bug); To disable it, you'll need to do experiments: 'none' or set your experiment list to any value that does not contain "enable-record-ecosystem-meta".

[SKoningScanplan]

@rhyskoedijk Where do I need to put the experiments: 'none' setting? I tried setting it both on the same level as package-ecosystem and on top-level, yet it seems to be ignored.

From my pipeline logs in both cases described above:

Experiments: {
  [...]
  'enable-record-ecosystem-meta': true
}

[rhyskoedijk]

In your pipeline; e.g.

steps:
- task: dependabot@2
  inputs:
    experiments: 'none'

[SKoningScanplan]

@rhyskoedijk This way the workaround works, thanks.

[rhyskoedijk]

Just to update this, I'm pretty sure this is a bug and I intend to fix it in the next few days; using enable-record-ecosystem-meta=false is valid and should work. Until a fix is submitted, you have to use the workaround above to disable it.

[leloscheidt]

@rhyskoedijk

experiments: 'none'

Using this solve the issue for me. Thanks! :)

[emmanwl]

@leloscheidt, when you say that it's working, do you actually mean that by using the indicated tweak, you are not anymore having the whole task timed out over one hour (as mentionned earlier) ?

[rhyskoedijk]

@emmanwl there are many contributing factors to pipeline time outs; If you are using NPM and were seeing a lot of "Failed to record ecosystem meta after 3 retries" errors in the logs, then the above work around will reduce the pipeline run time.

Timeouts may still happen if you have a very large and outdated project however and there are other issues reported in this project and dependabot-core around timeouts which are still a work in progress.

[emmanwl]

@rhyskoedijk, it's crystal clear, I was asking just out of curiosity and to validate/invalidate my initial assumption. I was hoping that the behaviour I described was actually not tied to a regression from the dependabot-core code but was more a matter of configuration.
Thanks for the fix anyway!
Will test it tomorrow.

[leloscheidt]

@emmanwl sorry, I was not clear

Yes, using experiments: 'none' made the pipeline end in 3 minutes.

[emmanwl]

Test performed, it completed in three hours, with two errors related to missing package descriptors.

It completed anyway, thanks !