Bug: config-helper write mishap with conditional replacements

[molleweide]

In the example below, which is from my setup.bash file, upon auto-rewriting the file, dorothy removed the second instance of GO_INSTALL when it attempted to move strong box to setup-util which resulted in only the closing parenthesis being left.

BEFORE:

if "$DOROTHY/commands/is-mac"; then
	GO_INSTALL=(
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit # manual install -> `go get -u -v github.com/gennaro-tedesco/boilit`
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/cloudflare/utahfs/cmd/utahfs-client
		# https://github.com/maaslalani/nap # cli snippets manager
	)
else
	GO_INSTALL=(
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/rfjakob/gocryptfs: use `setup-util-gocryptfs` instead, as this version lags behind
	)
fi

AFTER:

if "$DOROTHY/commands/is-mac"; then
  GO_INSTALL=(
  	'changkun.de/x/rmtrash'
  	'github.com/gennaro-tedesco/boilit'
  	'github.com/x-motemen/ghq@latest'
  )
else
   ### SYNTAX ERROR HERE... ###
    changkun.de/x/rmtrash
    github.com/gennaro-tedesco/boilit
    github.com/uw-labs/strongbox
    github.com/x-motemen/ghq@latest
    # github.com/rfjakob/gocryptfs: use `setup-util-gocryptfs` instead, as this version lags behind
  )
fi

[balupton]

Can you provide the entire before, after, and expected

If you want to have a go at writing a test for it and fixing this, it would be config-helper. Seems like the regex needs to be changed to shortest match instead of longest match, which should be done via a question mark character

[molleweide]

Yeah perfect, i'll look into and study the config helper file. i suspected that was the root cause

[balupton]

Any news?

[molleweide]

I spent a lot of time this week just studying related dorothy commands. Reading up on shit.

If one has a large multi conditional in setup.bash, eg.

if mac
...
elif wsl
...
elif ubuntu
...
elif linux
...

How do we want this to be handled I am thinking about?? I havent concluded anything because I need to play around more.

There can potentially be a lot of similar looking PKG_INSTALL sections.

---

[balupton]

https://github.com/bevry/dorothy/blob/master/docs/scripting/conventions.md#use-if-not-magic

[molleweide]

Hmm, i just realized that i dont even know exactly why i had the is-mac statement in the first place - IIRC it had something to do with gocrypfs but now since it is moved into setup-util I dont have the problem anymore.

I have been sketching on some solutions for conditionals but since I dont know why i had the conditional to begin with, I am going to remove it for my configs, and then fixing this bug doesn't make any sense.

closing..

ofc there might still be some things that need to be modified with config-helper but we'll cross that bridge when..

However, I have learnt a lot from digging into this.

[balupton]

I think the bug still needs to be fixed. I think the best option will be to rewrite config-helper from a ripgrep, sd, teip combo into an AWK script instead.

[molleweide]

If we do a big rewrite we should/have to consider this
https://ast-grep.github.io/
https://ast-grep.github.io/guide/quick-start.html
https://ast-grep.github.io/catalog/typescript/#typescript <<< since I know you are an expert in TS.

It allows for making syntax based queries on the code instead of using regex type solutions.

Bash is supported as builtin
https://ast-grep.github.io/reference/languages.html

playground:
https://ast-grep.github.io/playground.html

treesitter:
https://tree-sitter.github.io/tree-sitter/
https://github.com/tree-sitter/tree-sitter/wiki/List-of-parsers << quite many parsers exist now

VSCode extension: https://marketplace.visualstudio.com/items?itemName=ast-grep.ast-grep-vscode#overview

---

https://www.youtube.com/results?search_query=ast-grep
interview with creator: https://www.youtube.com/watch?v=Xs_NDy1U2Fk

[balupton]

ast-grep seems perfect, however it would then become an essential dorothy dependency and thus constrain which platforms dorothy is available on, furthermore the regexps are already done and a gawk solution, which we already have to make an essential dependency — #239 — will still eliminate the need for 3 rust tools (sd, ripgrep, teip).

I will look into ast-grep's platform support, if it does support all our targets (WSL2, amd64, arm64, risc5, apple intel, apple silicon) then will be fine to become an essential dependency, in addition to the gawk essential dep requirement.

[balupton]

It doesn't support RISC5 at this point:

• https://github.com/ast-grep/ast-grep/releases/tag/0.26.3
• no issues on their issue tracker about it

[balupton]

ast-grep will actually be fine for riscv, as it provides itself via cargo compilation, which is already what we do for all our other rust essential deps.

gawk will not be suitable as a replacement, as we use very advanced regexps which rust regex is suitable for and is what we already use, which gawk does not. However, there is frawk:
https://github.com/ezrosent/frawk

So will go with ast-grep or frawk.

[balupton]

Frawk doesn't provide any pre-built binaries:
ezrosent/frawk#53

[balupton]

For the OP bug:

before:

GO_INSTALL=()
if is-mac; then
	GO_LINTING_INSTALL='yes'
	GO_INSTALL+=(
		'changkun.de/x/rmtrash'
		# 'github.com/cloudflare/utahfs/cmd/utahfs-client'
	)
fi

if "$DOROTHY/commands/is-mac"; then
	GO_INSTALL=(
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit # manual install -> `go get -u -v github.com/gennaro-tedesco/boilit`
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/cloudflare/utahfs/cmd/utahfs-client
		# https://github.com/maaslalani/nap # cli snippets manager
	)
else
	GO_INSTALL=(
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/rfjakob/gocryptfs: use `setup-util-gocryptfs` instead, as this version lags behind
	)
fi

after setup-go:

GO_INSTALL=(
	'changkun.de/x/rmtrash'
	'github.com/gennaro-tedesco/boilit'
	'github.com/x-motemen/ghq@latest'
)
if is-mac; then
	GO_LINTING_INSTALL='yes'
	GO_INSTALL+=(
		'changkun.de/x/rmtrash'
		# 'github.com/cloudflare/utahfs/cmd/utahfs-client'
	)
fi

if "$DOROTHY/commands/is-mac"; then
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit # manual install -> `go get -u -v github.com/gennaro-tedesco/boilit`
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/cloudflare/utahfs/cmd/utahfs-client
		# https://github.com/maaslalani/nap # cli snippets manager
	)
else
		changkun.de/x/rmtrash
		github.com/gennaro-tedesco/boilit
		github.com/uw-labs/strongbox
		github.com/x-motemen/ghq@latest
		# github.com/rfjakob/gocryptfs: use `setup-util-gocryptfs` instead, as this version lags behind
	)
fi

Here is the intended behaviour:

before:

GO_INSTALL=(
	# github.com/cloudflare/utahfs/cmd/utahfs-client
	# github.com/rfjakob/gocryptfs: use `setup-util-gocryptfs` instead, as this version lags behind
	# https://github.com/maaslalani/nap # cli snippets manager
	changkun.de/x/rmtrash
	github.com/gennaro-tedesco/boilit # manual install -> `go get -u -v github.com/gennaro-tedesco/boilit`
	github.com/uw-labs/strongbox
	github.com/x-motemen/ghq@latest
)

after:

GO_INSTALL=(
	'changkun.de/x/rmtrash'
	'github.com/gennaro-tedesco/boilit'
	'github.com/x-motemen/ghq@latest'
)

Here is the call:

config-helper --file=/Users/balupton/.local/share/dorothy/user/config/setup.bash -- --field=GO_INSTALL '--array=changkun.de/x/rmtrash
github.com/gennaro-tedesco/boilit
github.com/x-motemen/ghq@latest' --field=SETUP_UTILS '--array=1password-cli
aria2
bash
bat
bottom
curl
delta
deno
fish
git
gocryptfs
jq
nu
obs
prettier
ripgrep
sd
strongbox
tealdeer
teip
tree
vim'

[balupton]

I will fix this by just fixing the regular expressions in config-helper. @molleweide if you want to experiment with ast-grep I'll leave that for you.

[balupton]

Fixed in dev, but not ideal. Will need to document and communicate the complexity to the user.