Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stack overflow exception in TextContainer.GetPrefixLocation #583

Open
jhutchings1 opened this issue Apr 23, 2024 · 3 comments
Open

Stack overflow exception in TextContainer.GetPrefixLocation #583

jhutchings1 opened this issue Apr 23, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@jhutchings1
Copy link

Describe the bug
I ran ApplicationInspector against the dependabot-core repo, and it crashed with a stackoverflow exception.

To Reproduce
Steps to reproduce the behavior:

  1. git clone https://github.com/dependabot/dependabot-core
  2. appinspector analyze -s dependabot-core -N

Expected behavior
Application inspector runs and outputs the analysis

Operating Environment (please complete the following information):

  • Application Inspector Version: ApplicationInspector.CLI 1.9.22+911e9d3be0
  • OS: Ubuntu (GitHub Codespaces environment)

Additional context
Here's a snippet of the log before the crash (there are 40k instances of this first line before it dies)

   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.GetPrefixLocation(Int32, Int32, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.PopulateCommentedStatesInternal(Int32, System.String, System.String, Boolean)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.PopulateCommentedState(Int32)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.IsCommented(Int32)
   at Microsoft.ApplicationInspector.RulesEngine.TextContainer.ScopeMatch(System.Collections.Generic.IList`1<Microsoft.ApplicationInspector.RulesEngine.PatternScope>, Microsoft.ApplicationInspector.RulesEngine.Boundary)
   at Microsoft.ApplicationInspector.RulesEngine.OatExtensions.OatSubstringIndexOperation+<GetMatches>d__4.MoveNext()
   at System.Linq.Enumerable+SelectEnumerableIterator`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.ValueTuple`2[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Collections.Generic.List`1[[System.ValueTuple`2[[System.Int32, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].AddRange(System.Collections.Generic.IEnumerable`1<System.ValueTuple`2<Int32,System.__Canon>>)
   at Microsoft.ApplicationInspector.RulesEngine.OatExtensions.OatSubstringIndexOperation.SubstringIndexOperationDelegate(Microsoft.CST.OAT.Clause, System.Object, System.Object, System.Collections.Generic.IEnumerable`1<Microsoft.CST.OAT.ClauseCapture>)
   at Microsoft.CST.OAT.Analyzer.GetClauseCapture(Microsoft.CST.OAT.Clause, System.Object, System.Object, System.Collections.Generic.IEnumerable`1<Microsoft.CST.OAT.ClauseCapture>)
   at Microsoft.CST.OAT.Analyzer.Evaluate(System.String[], System.Collections.Generic.List`1<Microsoft.CST.OAT.Clause>, System.Object, System.Object, System.Collections.Generic.IEnumerable`1<Microsoft.CST.OAT.ClauseCapture>)
   at Microsoft.CST.OAT.Analyzer+<>c__DisplayClass34_1.<Evaluate>g__EvaluateLambda|1()
   at Microsoft.CST.OAT.Analyzer.Evaluate(System.String[], System.Collections.Generic.List`1<Microsoft.CST.OAT.Clause>, System.Object, System.Object, System.Collections.Generic.IEnumerable`1<Microsoft.CST.OAT.ClauseCapture>)
   at Microsoft.CST.OAT.Analyzer.GetCapture(Microsoft.CST.OAT.Rule, System.Object, System.Object)
   at Microsoft.CST.OAT.Analyzer+<>c__DisplayClass22_0.<GetCaptures>b__0(Microsoft.CST.OAT.Rule)
   at System.Threading.Tasks.Parallel+<>c__DisplayClass43_0`2[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].<PartitionerForEachWorker>b__1(System.Collections.IEnumerator ByRef, Int64, Boolean ByRef)
   at System.Threading.Tasks.TaskReplicator+Replica.Execute()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(System.Threading.Tasks.Task, Boolean)
   at System.Threading.Tasks.TaskScheduler.TryRunInline(System.Threading.Tasks.Task, Boolean)
   at System.Threading.Tasks.Task.InternalRunSynchronously(System.Threading.Tasks.TaskScheduler, Boolean)
   at System.Threading.Tasks.TaskReplicator.Run[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](ReplicatableUserAction`1<System.__Canon>, System.Threading.Tasks.ParallelOptions, Boolean)
   at System.Threading.Tasks.Parallel.PartitionerForEachWorker[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Concurrent.Partitioner`1<System.__Canon>, System.Threading.Tasks.ParallelOptions, System.Action`1<System.__Canon>, System.Action`2<System.__Canon,System.Threading.Tasks.ParallelLoopState>, System.Action`3<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64>, System.Func`4<System.__Canon,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func`5<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64,System.__Canon,System.__Canon>, System.Func`1<System.__Canon>, System.Action`1<System.__Canon>)
   at System.Threading.Tasks.Parallel.ForEachWorker[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Threading.Tasks.ParallelOptions, System.Action`1<System.__Canon>, System.Action`2<System.__Canon,System.Threading.Tasks.ParallelLoopState>, System.Action`3<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64>, System.Func`4<System.__Canon,System.Threading.Tasks.ParallelLoopState,System.__Canon,System.__Canon>, System.Func`5<System.__Canon,System.Threading.Tasks.ParallelLoopState,Int64,System.__Canon,System.__Canon>, System.Func`1<System.__Canon>, System.Action`1<System.__Canon>)
   at System.Threading.Tasks.Parallel.ForEach[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](System.Collections.Generic.IEnumerable`1<System.__Canon>, System.Action`1<System.__Canon>)
   at Microsoft.CST.OAT.Analyzer+<GetCaptures>d__22.MoveNext()
   at Microsoft.ApplicationInspector.RulesEngine.RuleProcessor.AnalyzeFile(Microsoft.ApplicationInspector.RulesEngine.TextContainer, Microsoft.CST.RecursiveExtractor.FileEntry, Microsoft.ApplicationInspector.RulesEngine.LanguageInfo, System.Collections.Generic.IEnumerable`1<System.String>, Int32)
   at Microsoft.ApplicationInspector.RulesEngine.RuleProcessor.AnalyzeFile(System.String, Microsoft.CST.RecursiveExtractor.FileEntry, Microsoft.ApplicationInspector.RulesEngine.LanguageInfo, System.Collections.Generic.IEnumerable`1<System.String>, Int32)
   at Microsoft.ApplicationInspector.RulesEngine.RuleProcessor.AnalyzeFile(Microsoft.CST.RecursiveExtractor.FileEntry, Microsoft.ApplicationInspector.RulesEngine.LanguageInfo, System.Collections.Generic.IEnumerable`1<System.String>, Int32)
   at Microsoft.ApplicationInspector.Commands.AnalyzeCommand+<>c__DisplayClass15_1.<PopulateRecords>g__ProcessLambda|2()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
@jhutchings1 jhutchings1 added the bug Something isn't working label Apr 23, 2024
@jhutchings1
Copy link
Author

jhutchings1 commented Apr 25, 2024 •
edited
Loading

Interestingly, this one doesn't reproduce if you enable verbose logging.

Sorry, that doesn't seem to be true. It didn't reproduce with this command:
appinspector analyze -v verbose -s dependabot-core -l ./log.txt

but it did with this one:
appinspector analyze -v verbose -s dependabot-core -l ./log.txt -f sarif

@gfs
Copy link
Contributor

gfs commented Apr 29, 2024

Thanks for your report and the follow up info. That might indicate its somehow specifically in the sarif export path. I'll investigate this week.

@gfs
Copy link
Contributor

gfs commented Jul 29, 2024

Forgot to update the ticket, but this was fixed in the codebase with e2e13ee, though I don't think we have a release out with this fix available yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gfs @jhutchings1