Add diagram to show repocop dependencies (#1386)

guardian · Jan 13, 2025 · ba102d5 · ba102d5
1 parent dd6ae61
commit ba102d5
Show file tree

Hide file tree

Showing 2 changed files with 113 additions and 0 deletions.
diff --git a/packages/repocop/JobDependencies.md b/packages/repocop/JobDependencies.md
@@ -0,0 +1,111 @@
+# Job Dependencies
+This diagram shows the flow of data and dependencies that RepoCop relies on as well as the downstream events that rely
+on RepoCop.
+
+```mermaid
+flowchart LR
+    %% Data stores
+    awsAccountData[(aws_organizations_accounts)]
+    awsStackData[(aws_cloudformation_stacks)]
+    awsFsbpFindingData[(aws_securityhub_findings)]
+    ghLangData[(github_languages)]
+    ghRepoData[(github_repositories)]
+    ghBranchData[(github_repository_branches)]
+    ghTeamData[(github_teams)]
+    ghTeamRepoData[(github_team_repositories)]
+    snykIssueData[(snyk_issues)]
+    snykProjectData[(snyk_projects)]
+    galaxyTeamData[(galaxies_teams_table)]
+    repoCopVulnData[(repocop_vulnerabilities)]
+    fsbpVulnData[(cloudbuster_fsbp_vulnerabilities)]
+    obligatronResultData[(obligatron_results)]
+    
+    %% Data views
+    awsAccountView[(aws_accounts)]
+    repoOwnerView[(view_repo_ownership)]
+    
+    %% Notifications
+    vulnNotice[[SNS: Vulnerability digest]]
+    fsbpNotice[[SNS: FSBP findings digest]]
+    ghInteractiveRepoNotice[[SNS: Interactive repo missing topic]]
+    ghBranchProtectNotice[[SNS: Branch protection]]
+    ghProdRepoNotice[[SNS: Production repo missing topic]]
+    ghDepGraphMissingNotice[[SNS: Dependency graph submission action missing]]
+    
+    %% Other writes
+    ghDepGraphPR[[Dependency graph submission workflow PRs]]
+    ghProdRepo[[Repos updated with Prod topic]]
+    ghBranchProtect[[Repos updated with default branch protection]]
+    
+    %% Processes
+    awsWeeklyLoader[CQ AWS weekly sync<br>ECS task<br>scheduled SAT 16:00]
+    awsDailyLoader[CQ AWS daily sync<br>ECS task<br>scheduled daily 22:00]
+    awsFreqLoader[CQ AWS frequent sync<br>ECS task<br>scheduled every 3 hours]
+    ghWeeklyLoader[CQ Github weekly sync<br>ECS task<br>scheduled MON 10:00]
+    ghDailyLoader[CQ Github daily sync<br>ECS task<br>scheduled daily 00:00]
+    snykLoader[CQ Snyk sync<br>ECS task<br>scheduled daily 06:00]
+    ghLangLoader[CQ Github languages sync<br>ECS task<br>scheduled every 7 days]
+    galaxiesLoader[CQ Galaxies sync<br>ECS task<br>scheduled every 1 day]
+    repocop[RepoCop<br>lambda<br>scheduled MON-FRI 03:00]
+    depGraphIntegrator[Dependency Graph Integrator<br>lambda<br>SNS trigger]
+    cloudbuster[CloudBuster<br>lambda<br>scheduled MON-FRI 03:00]
+    obligatronTagging[Obligatron<br>Tagging obligation<br>lambda<br>scheduled daily 09:00]
+    obligatronDeps[Obligatron<br>Dependencies obligation<br>lambda<br>scheduled daily 10:00]
+    obligatronVulns[Obligatron<br>AWS Vulnerabilities obligation<br>lambda<br>scheduled daily 11:00]
+    
+    %% Dependencies
+    
+    %% Data loading
+    awsWeeklyLoader --> awsAccountData
+    awsFreqLoader --> awsStackData
+    awsDailyLoader --> awsFsbpFindingData
+    ghLangLoader --> ghLangData
+    ghDailyLoader --> ghRepoData
+    ghDailyLoader --> ghBranchData
+    ghWeeklyLoader --> ghTeamData
+    ghWeeklyLoader --> ghTeamRepoData
+    snykLoader --> snykIssueData
+    snykLoader --> snykProjectData
+    galaxiesLoader --> galaxyTeamData
+    awsAccountData --> awsAccountView
+    ghTeamRepoData --> repoOwnerView
+    ghTeamData --> repoOwnerView
+    galaxyTeamData --> repoOwnerView
+    
+    %% RepoCop
+    awsStackData --> repocop
+    ghLangData --> repocop
+    ghRepoData --> repocop
+    ghBranchData --> repocop
+    ghTeamData --> repocop
+    snykIssueData --> repocop
+    snykProjectData --> repocop
+    repoOwnerView --> repocop
+    repocop --> repoCopVulnData
+    repocop --> ghDepGraphMissingNotice
+    repocop --> ghInteractiveRepoNotice
+    repocop --> ghBranchProtectNotice
+    repocop --> ghBranchProtect
+    repocop --> vulnNotice
+    repocop --> ghProdRepoNotice
+    repocop --> ghProdRepo
+    
+    %% Dependency graph integrator
+    ghDepGraphMissingNotice --> depGraphIntegrator
+    depGraphIntegrator --> ghDepGraphPR
+    
+    %% Cloud Buster
+    awsFsbpFindingData --> cloudbuster
+    cloudbuster --> fsbpVulnData
+    cloudbuster --> fsbpNotice
+    
+    %% Obligatron
+    awsFsbpFindingData --> obligatronTagging
+    awsAccountView --> obligatronTagging
+    obligatronTagging --> obligatronResultData
+    ghRepoData --> obligatronDeps
+    repoCopVulnData --> obligatronDeps
+    obligatronDeps --> obligatronResultData
+    awsFsbpFindingData --> obligatronVulns
+    obligatronVulns --> obligatronResultData
+```
diff --git a/packages/repocop/README.md b/packages/repocop/README.md
@@ -9,6 +9,8 @@ It is deployed as an AWS Lambda, and powers:
 - The [interactive-monitor](../interactive-monitor/README.md) lambda
 - The [dependency-graph-integrator](../dependency-graph-integrator/README.md) lambda
 
+See the [job dependencies diagram](JobDependencies.md).
+
 ## Running RepoCop locally
 
 From the root of the repo: