Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/np introspector #25

Open
wants to merge 168 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
558aa60
start of new version
Dec 12, 2024
2138b04
now creating
Dec 12, 2024
2badf08
machine image
Dec 12, 2024
51a0906
update
Dec 12, 2024
fdac521
now running
Dec 13, 2024
5ed453c
using ami
Dec 13, 2024
f29b5df
bugfix egress
Dec 13, 2024
f381197
adding updates and proposed changes
Dec 13, 2024
5c7e3a2
work in progress
Dec 13, 2024
7f1fd3a
starting
Dec 13, 2024
b9a7e36
moving towards asg support
Dec 14, 2024
d8898a4
now trying again with medium
Dec 14, 2024
7e7f8f9
many sizes
Dec 14, 2024
7f40ba1
now creating more sizes
Dec 14, 2024
107ae6a
removing too small
Dec 14, 2024
68c39a9
alb
Dec 16, 2024
790bb6b
create tg
Dec 16, 2024
8c10213
create tg
Dec 16, 2024
bb62580
applied
Dec 16, 2024
f9d0d56
group coming online
Dec 16, 2024
a135fe8
plan
Dec 16, 2024
b561619
wip
Dec 16, 2024
7e4cf29
update
Dec 16, 2024
1cf2295
health check
Dec 16, 2024
c6fd42c
update
Dec 16, 2024
049c6ef
start of swarms deploy
Dec 16, 2024
05ed0bd
start of swarms
Dec 16, 2024
248221c
now applying
Dec 17, 2024
1063aed
now applying
Dec 17, 2024
06f9a86
adding readme
Dec 17, 2024
790758e
starting cognito
Dec 17, 2024
e671f30
apply
Dec 17, 2024
3cd307a
appplied
Dec 17, 2024
e64a39a
google
Dec 17, 2024
e43831e
Delete accounts/swarms/.terraform.lock.hcl
jmikedupont2 Dec 17, 2024
ea2a611
more scopes
Dec 17, 2024
0e86ce3
adding ses
Dec 17, 2024
a2bff67
starting on ses verification
Dec 17, 2024
464cc1c
insecure listener and updated instructions
Dec 18, 2024
334207b
building ami
Dec 18, 2024
79ec0b3
now larger drive
Dec 18, 2024
ada7442
new ami asg
Dec 18, 2024
2e59cc2
loading
Dec 18, 2024
c38d394
new installer
Dec 18, 2024
45f8b38
update
Dec 18, 2024
13621be
remove the slow asg
Dec 18, 2024
06a8b41
simple autoscale policy
Dec 19, 2024
eedf499
swarmdeploy
Dec 19, 2024
cac4c79
now just hacking this for now
Dec 19, 2024
554f901
bugfix: use the install in the root dir
Dec 19, 2024
f708fad
proof of godaddy working
Dec 19, 2024
b6f8961
secrets loading
Dec 19, 2024
e8aa000
version 1
Dec 19, 2024
aa37da0
fix the yaml
Dec 19, 2024
cc6d1e8
call swarms
Dec 19, 2024
57d111e
notes from todays meeting
Dec 19, 2024
3011848
Update README.md
jmikedupont2 Dec 19, 2024
b9cd606
Update README.md
jmikedupont2 Dec 19, 2024
112773e
test
Dec 19, 2024
c512375
first version of call swarms calling
Dec 19, 2024
69de293
update job version
Dec 19, 2024
3c4a68b
starting on a new autoscaling group
Dec 19, 2024
ea44b5f
customer test
Dec 19, 2024
eca5690
working it seems
Dec 19, 2024
db93c71
now adding a call swarms.sh
Dec 20, 2024
749092d
update
Dec 20, 2024
dcf219e
fixing bug
Dec 20, 2024
b6b8460
removing test code
Dec 20, 2024
d482d1c
deploy
Dec 20, 2024
832898b
start of the observability
Dec 21, 2024
76dea16
applied
Dec 21, 2024
08e85bd
allow decrypt
Dec 21, 2024
06b9a22
wip
Dec 21, 2024
3c23717
work in progress
Dec 21, 2024
dd2b6fd
call swarms
Dec 21, 2024
c849407
gitignore
Dec 21, 2024
45427cd
fixing more perms
Dec 21, 2024
fcbecca
wip
Dec 22, 2024
d0b08da
report
Dec 22, 2024
1cb80d1
adding branch name
Dec 22, 2024
151c3e8
now fixing the logging config
Dec 22, 2024
aee8856
swap
Dec 22, 2024
41a652d
update the code to fetch then checkout
Dec 22, 2024
f1da299
pdev
Dec 22, 2024
c930829
pdev
Dec 22, 2024
9dd1a41
pdev
Dec 22, 2024
cb9a924
adding main
Dec 22, 2024
f4326f3
starting with docker
Dec 23, 2024
3899d69
update readme
Dec 23, 2024
4342004
normalizing the names
Dec 23, 2024
42a947f
adding status
Dec 23, 2024
b2c2099
top level module
Dec 24, 2024
198627d
tofu apply -target module.call_swarms.aws_ssm_document.deploy-docker
Dec 24, 2024
4c8eba2
example logs
Dec 24, 2024
0761940
creating readme
Dec 24, 2024
b96bfdd
creating readme
Dec 24, 2024
9fa1b3a
fix fmt
Dec 24, 2024
8138b4a
now its pulling
Dec 24, 2024
02bbed9
resize
Dec 25, 2024
f71554b
max price 0.0275
Dec 25, 2024
81564cb
fmt
Dec 25, 2024
66dda6e
adding spot
Dec 25, 2024
ecb23ac
Delete environments/swarms-aws-agent-api/dev/us-east-1/Readme.md~
jmikedupont2 Dec 26, 2024
ddb49a5
moving traffic source attachments
Dec 26, 2024
0357e1b
fixing size
Dec 26, 2024
45768a5
increase size
Dec 26, 2024
cc23a5a
removing spot for now
Dec 26, 2024
00452be
fixing branch to be constant in user data
Dec 27, 2024
a045f4b
creating cname
Dec 27, 2024
aee30ad
creating new mcs subcomponent
Dec 27, 2024
3de651f
tofu fmt
Dec 27, 2024
41ef75c
fixing path name
Dec 27, 2024
a792f31
fixing branch
Dec 28, 2024
038b0b7
changing health check path https://github.com/The-Swarm-Corporation/M…
Dec 28, 2024
73bb419
parser now producing better results
Dec 28, 2024
6b959d0
report
Dec 28, 2024
679e81e
now generating new code
Dec 29, 2024
9784e4f
generated first version
Dec 29, 2024
4ab9ced
adding in documentation
Dec 31, 2024
1ff4b4e
getlogs
Dec 31, 2024
67998a8
creating mcs dev
Dec 31, 2024
bcc9ee1
adding mcs dev
Dec 31, 2024
e12be67
update
Dec 31, 2024
eb322e3
Merge branch 'feature/ses' into feature/mcs
Jan 9, 2025
d4b64fb
applied
Jan 9, 2025
edce93d
step one create ssm infra
Jan 9, 2025
faf3eb5
adding eliza
Jan 9, 2025
0982e88
applied
Jan 9, 2025
723bfad
fixing name of log file
Jan 9, 2025
d4a3eaa
work in progress starting to boot
Jan 10, 2025
27ee585
applied
Jan 10, 2025
97d36b0
new sizes
Jan 10, 2025
c9a23d4
renaming
Jan 12, 2025
8db2f0f
wip
Jan 12, 2025
50ed993
adding in bogus azs
Jan 12, 2025
03d3b0d
adding owners
Jan 13, 2025
cc47c09
removing my key
Jan 13, 2025
2fef929
update readme
Jan 13, 2025
f09fb87
comment out kms id
Jan 13, 2025
8b46a9a
keypairs
Jan 13, 2025
a0f7676
fixing
Jan 13, 2025
c5be841
move to branch
Jan 14, 2025
4eb3ee8
allow github to create repositories
Jan 14, 2025
a62af86
wip
Jan 14, 2025
ecd5950
adding more detailed instructions
Jan 14, 2025
316a26e
adding ai notes back in
Jan 14, 2025
88ef79a
tofu
Jan 14, 2025
83c5524
Merge branch 'feature/aitokenteam' into feature/codebuild
Jan 14, 2025
24b2c90
adding codebuild
Jan 14, 2025
7c751db
notes
Jan 15, 2025
164c02d
bugfix
Jan 15, 2025
02933b3
v1 of solfnum meme infra plan
Jan 20, 2025
9c707b9
plan with one click
Jan 20, 2025
1de1e7f
parking for jump
Jan 20, 2025
66e00d4
now applied with terraform
Jan 21, 2025
6b012ca
lagging changes
Jan 21, 2025
db8077a
update readme
Jan 26, 2025
92398e9
refactor
Jan 26, 2025
730e400
fixed deps
Jan 26, 2025
c0f087f
init worked
Jan 27, 2025
9aff8fd
now planning
Jan 27, 2025
6a40c09
now planning
Jan 27, 2025
6ab5317
apply completed
Jan 27, 2025
7d73625
cleanup
Jan 29, 2025
2d78c6f
plan
Jan 29, 2025
2bfe151
aps1
Jan 29, 2025
a554d61
removing trouble
Jan 29, 2025
cb53df8
branch
Jan 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
now trying again with medium
had to wait for terraform to finish
mike dupont committed Dec 14, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit d8898a44dec3c71b5b9eb9148cc6db318b2dd987
Original file line number Diff line number Diff line change
@@ -70,7 +70,7 @@ resource "aws_launch_template" "ec2_launch_template" {
apt-get install -y --no-install-recommends ca-certificates=20230311 curl=7.88.1-10+deb12u7 | echo oops
curl -O "https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/$(dpkg --print-architecture)/latest/amazon-cloudwatch-agent.deb"
dpkg -i -E amazon-cloudwatch-agent.deb

if [ ! -d "/opt/swarms/" ]; then
git clone https://github.com/jmikedupont2/swarms "/opt/swarms/"
fi
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@ module "asg_sg" {
ingress_rules = [
"https-443-tcp",
"http-80-tcp",
"ssh-tcp"
# "ssh-tcp" dont need this now
]

egress_rules = ["all-all"]
314 changes: 4 additions & 310 deletions environments/swarms-aws-agent-api/dev/us-east-1/main.tf
Original file line number Diff line number Diff line change
@@ -1,312 +1,6 @@
# #task :break this up into subtasks we will call you back for each subtask with this large text.
# # meta terraform quine meme
# # create a terraform module that will call you back with its text.
# # each output will be a new more refined terraform module that will be executed
# # and the results send back to you and documented.

# # language: abstract modular terraform
# # goal: this statement S can be proven in system G in less than M to the probability greater than P
# # G : entire system as a single variable, a goedel, dawkins, quine, quasi meta quotation of system as self modifying terraform resource object

# # codename: swarms agent terraform.
# # task :
# # meta: y-combinator
# # call yourself recursivly

# # meta: s-combinator
# # read this file and construct a recursive function that will call you back repeated and recursivly
# # to contruct this cloud system each time surfacing more data to be used in the new process.
# # project: swarm terraform
# # mode: epic heros journey
# # journey : bootstrap
# # bootstrap: abstract native cloud
# # recursive evaluation
# # resources
# # 1. local kind cluster
# # 1.2 running argo cicd

# # 1.2.1 creating resources
# # 1.2.1.1 build or buy
# # 1.2.1.1.1 market quote

# # 1.1.1 creating other cloud resources

# # 1.1.1.1 creating aws accounts

# # 1.1.1.2 configuring secrets
# # 1.1.1.2.1 to hold wallets
# # 1.1.1.3 configuring IAM user authentication
# # 1.1.1.3.0 creating siloed accounts that have minimal access needed.
# # 1.1.1.3.1 configuring oauth providers on cloud
# # 1.1.1.3.1.1 auth0|cognito|github apps as entry points to our network.
# # 1.1.1.3.1.2 eth wallets as entry point
# # 1.1.1.3.1.3 otp links
# # 1.1.1.3.1.4 otp qr code
# # qr codes as urls as entry points
# # 1.1.1.3.1.4 mfa token as entry point
# # using the mfa device as sole factor login


# To develop a recursive cloud system based on the provided project parameters, we can construct a recursive function that surfaces and enacts various cloud resources and configurations iteratively. Here's a basic example of how this might be structured in pseudo-code:


# ### Explanation:
# 1. **Recursive Function**: `swarm_terraform(level)` serves as the main function that sets up resources based on the current recursion depth.
# 2. **Setup Functions**: Helper functions are defined for local clusters, CI/CD, AWS accounts, secrets, and IAM.
# 3. **Level-Based Logic**: The creation of resources and configurations is determined based on the recursion level, allowing the system to build incrementally.
# 4. **Termination Condition**: The recursion stops at a defined `MAX_LEVEL` to prevent infinite execution.

# ### Steps to Execution:
# - Implement specific logic within each helper function as per your project needs.
# - Adjust the `MAX_LEVEL` variable based on how detailed or expansive you want the recursive processing to be.

# This structure provides a solid foundation for building out the cloud infrastructure recursively, driven by the initial parameters you've outlined.


# To express this concept in a modular Terraform-style pseudo-language, you may want to define a resource that captures the essence of your statement S while incorporating the probabilistic and self-referential properties of system G. Here’s a concise example:

# ```hcl
# resource "godel_dawkins_quine" "system_G" {
# name = "G"
# probability = var.P
# threshold = var.M

# statement {
# content = "S can be proven in system G"
# complexity = calculate_complexity(content) # define this function for complexity measure
# proven_in_time_less_than_M = complexity < var.M
# }

# self_modifying {
# enabled = true
# modifications = ["update_probability", "redefine_threshold"]
# }
# }

# variable "P" {
# description = "Probability threshold"
# type = number
# default = 0.95
# }

# variable "M" {
# description = "Maximum time for proof"
# type = number
# default = 100 # example limit on time
# }
# ```




# It seems there might have been a misunderstanding or you are looking for further clarification. Here's an extended Terraform module example that incorporates recursive processes and configurations for cloud resources based on your initial request:

# ```hcl
# terraform {
# required_providers {
# aws = {
# source = "hashicorp/aws"
# version = "~> 3.0"
# }
# }
# }

# variable "max_levels" {
# description = "Maximum level of recursion"
# type = number
# default = 3
# }

# resource "aws_iam_user" "swarm_user" {
# count = var.max_levels

# name = "swarm_user_${count.index}"

# tags = {
# Level = count.index
# }
# }

# resource "aws_secretsmanager_secret" "swarm_secret" {
# count = var.max_levels

# name = "swarm_secret_${count.index}"

# description = "Secret for level ${count.index}"
# }

# resource "aws_cognito_user_pool" "swarm_user_pool" {
# count = var.max_levels

# name = "swarm_user_pool_${count.index}"
# alias_attributes = ["email"]

# lambda_config {
# pre_sign_up = aws_lambda_function.pre_sign_up[count.index].arn
# }
# }

# resource "aws_lambda_function" "pre_sign_up" {
# count = var.max_levels

# function_name = "pre_sign_up_${count.index}"
# runtime = "nodejs14.x"

# handler = "index.handler"
# source_code_hash = filebase64sha256("path_to_your_lambda_zip_${count.index}.zip")

# role = aws_iam_role.lambda_exec.arn
# }

# resource "aws_iam_role" "lambda_exec" {
# name = "lambda_exec_role"

# assume_role_policy = jsonencode({
# Version = "2012-10-17"
# Statement = [{
# Action = "sts:AssumeRole"
# Principal = {
# Service = "lambda.amazonaws.com"
# }
# Effect = "Allow"
# Sid = ""
# }]
# })
# }

# output "swarm_users" {
# value = aws_iam_user.swarm_user[*].name
# }

# output "swarm_secrets" {
# value = aws_secretsmanager_secret.swarm_secret[*].name
# }

# output "swarm_user_pools" {
# value = aws_cognito_user_pool.swarm_user_pool[*].id
# }
# ```

# ### Key Components:

# - **Recursive Resource Creation**: The use of the `count` parameter simulates recursion by creating multiple instances of resources based on the specified `max_levels`.

# - **IAM Users and Secrets**: Creates IAM users and AWS secrets for each level.

# - **Cognito User Pools**: Sets up a user pool for authentication at each recursion depth.

# - **Outputs**: Outputs the created IAM users, secrets, and Cognito user pools, allowing you to track the resources created at each level.

# ### Execution:
# - Adjust `max_levels` as needed to control the depth of resource creation.
# - Ensure that the paths to any Lambda function code or specific policies are set correctly.

# This example captures the essence of a meta Terraform module while following the recursive and self-referential principles you've outlined. Let me know how you would like to expand or modify this!


# It seems you're looking for further examples or a continuation of the Terraform module design. Here's a more comprehensive modular approach that emphasizes recursion and the creation of various cloud resources, broken down into subtasks for clarity:

# ### Main Terraform Configuration

# ```hcl
# module "swarm_terraform" {
# source = "./modules/swarm" # Define the module source
# level = 1 # Start at recursion level 1
# max_levels = var.max_levels # Maximum recursion depth
# aws_region = var.aws_region # AWS region variable
# }

# variable "max_levels" {
# description = "Maximum depth for resource creation"
# type = number
# default = 3
# }

# variable "aws_region" {
# description = "AWS region to deploy resources"
# type = string
# default = "us-east-1"
# }
# ```

# ### Module: `modules/swarm/main.tf`

# ```hcl
# resource "aws_iam_user" "swarm_user" {
# count = var.level <= var.max_levels ? 1 : 0

# name = "swarm_user_${var.level}"

# tags = {
# Level = var.level
# }
# }

# resource "aws_secretsmanager_secret" "swarm_secret" {
# count = var.level <= var.max_levels ? 1 : 0

# name = "swarm_secret_${var.level}"
# }

# resource "aws_lambda_function" "pre_sign_up" {
# count = var.level <= var.max_levels ? 1 : 0

# function_name = "pre_sign_up_${var.level}"
# runtime = "nodejs14.x"
# handler = "index.handler"

# # Assuming the source code is provided
# source_code_hash = filebase64sha256("path_to_your_lambda_zip_${var.level}.zip")
# role = aws_iam_role.lambda_exec.arn
# }

# resource "aws_iam_role" "lambda_exec" {
# name = "lambda_exec_role"

# assume_role_policy = jsonencode({
# Version = "2012-10-17"
# Statement = [{
# Action = "sts:AssumeRole"
# Principal = {
# Service = "lambda.amazonaws.com"
# }
# Effect = "Allow"
# Sid = ""
# }]
# })
# }

# # Recursive module call to create resources for the next level
# module "swarm_terraform_next" {
# source = "./main.tf"
# level = var.level + 1
# max_levels = var.max_levels
# aws_region = var.aws_region
# depends_on = [aws_iam_user.swarm_user, aws_secretsmanager_secret.swarm_secret, aws_lambda_function.pre_sign_up]
# }
# ```

# ### Explanation:

# 1. **Modular Structure**: This approach divides the resources into a module, making it reusable and expandable.

# 2. **Recursion Handling**: The module checks if the current `level` is less than or equal to `max_levels`. If it is, it creates the corresponding resources and calls itself for the next level.

# 3. **IAM User and Lambda Function**: Each recurred level creates new IAM users, Secrets Manager secrets, and Lambda functions, maintaining the recursion.

# 4. **Outputs**: Outputs can be included to retrieve created resource details at the end of processing, although they are not explicitly shown in this snippet.

# ### Execution Steps:
# - Ensure you have the appropriate IAM permissions and AWS configuration.
# - Adjust the `max_levels` variable as desired, and ensure the paths to any Lambda function source code are correctly defined.
# - Use `terraform apply` to trigger the recursion and monitor resource creation.

# This format should give you a robust foundation for developing a complex, recursive Terraform setup. If you need further adjustments or detailed explanations on specific elements, let me know!

#module "vpc" {
# source = "./components/vpc"
#}
locals {
# instance_type = "t3.large"
instance_type = "t3.medium"
ami = "ami-0e2c8caa4b6378d8c"
name = "swarms"
region = "us-east-1"
@@ -326,7 +20,7 @@ module "kp" {
}

module "lt" {
instance_type = "t3.medium"
instance_type = local.instance_type
security_group_id = module.security.security_group_id
source = "./components/launch_template"
}
@@ -335,7 +29,7 @@ module "lt" {
module "asg" {
source = "./components/autoscaling_group"
security_group_id = module.security.security_group_id
instance_type = "t3.medium"
instance_type = local.instance_type
launch_template_id = module.lt.launch_template_id
}

Loading