Update template to newest version

[nicholasRutherford]

Description

I noticed in #5 that this plugin no longer works with decky. So I looked at the latest https://github.com/SteamDeckHomebrew/decky-plugin-template version and tried to port your changes over to it. I'm hoping that this would fix whatever went wrong.

Testing

Unfortunately I can't build this because I'm on an ARM mac, so hopefully someone else can build it and test it for me :(

PS

Thanks for creating this plugin! The lack of TTS was really inconvenient so I'm hoping this will be working again soon 🤞

[Gaspadlo]

Looking at the authors activity on Github / Social media, one might wonder whether he is "still around" at all.

There potentially should be no problem with forking and making a new PR into the decky-plugin-database repo, right? (Ofc with credits given where credits are due).

On a side note - I am still flabbergasted, that "Speech-to-text" on a Steamdeck is not a thing. (Be it officially implemented by Valve, or more actively demanded by the community.) The on-screen keyboard is serviceable for occasional input prompts, but for MP games it is a major pain in the B to use efficiently.

Edit: I had to change some things here and there in build-zip.sh to get it to build under Ubuntu in WSL and it seems to be working fine. (I also added own button binding, since the original trigger button combination is just too much).
Custom GUI binding would be nice.

(My bindings: L5 to start listening - as I have bound that to an Enter in WoW, R5 to stop listening - and also bound to escape, If I wanted to cancel the message... I also added a helping variable "listening" so it won't needlessly spam me with notifications)

[Gaspadlo]

@nicholasRutherford Oh lol... Sorry, I re-read just now your post, that you can't build it. Here is a built dist
(Just extract its contents into /home/deck/homebrew/plugins/decky-dictation folder, you'll probably have to sudo copy it there, since that folder is managed by a decky-loader at higher privileges)
You should be able to customize binding even in the dist index.js (just like in index.tsx)

[cboiangiu]

@nicholasRutherford @Gaspadlo Changes seem to be looking good and I am glad y'all been able to test it and make sure it works. Due to some personal issues I haven't been able to give this project the love it deserves unfortunately.
I will take care of this tomorrow and hopefully have this merged. We'd still have to fix the dependency of the model file since it's too large to ship it within the package. Decky documentation should help with having this file be downloaded at install time for the user.

[cboiangiu]

I have fixed the build scripts to properly use the decky cli, which just got a macOS release 2 days ago.

@nicholasRutherford You should be able to try and build this on macOS with the build task from VSCode. Currently I am stuck on this error though:

INFO [decky::plugin] Looking for package.json...
INFO [decky::plugin] Looking for plugin.json...
INFO [decky::plugin] Looking for deck.json...
INFO [decky::cli::plugin::build] Creating temporary build directory
INFO [decky::cli::plugin::build] Building plugin
INFO [decky::cli::plugin::build] Building backend
[+] Building 1.0s (10/10) FINISHED                                                                                                                                                                     docker:orbstack
 => [internal] load build definition from Dockerfile                                                                                                                                                              0.0s
 => => transferring dockerfile: 529B                                                                                                                                                                              0.0s
 => [internal] load metadata for ghcr.io/steamdeckhomebrew/holo-base:latest                                                                                                                                       0.9s
 => [auth] steamdeckhomebrew/holo-base:pull token for ghcr.io                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                 0.0s
 => => transferring context: 2B                                                                                                                                                                                   0.0s
 => [1/5] FROM ghcr.io/steamdeckhomebrew/holo-base:latest@sha256:26666d74d8a5e55b521c972a601772f0b4f9ff95c8b48ecc6732c26f9c109dcd                                                                                 0.0s
 => CACHED [2/5] RUN pacman -Sydd --noconfirm --dbpath /var/lib/pacman python-pip git wget unzip                                                                                                                  0.0s
 => CACHED [3/5] RUN pip3 install vosk --target=/vosk                                                                                                                                                             0.0s
 => CACHED [4/5] RUN git clone https://github.com/ideasman42/nerd-dictation.git                                                                                                                                   0.0s
 => CACHED [5/5] RUN cd nerd-dictation && wget https://alphacephei.com/kaldi/models/vosk-model-small-en-us-0.15.zip && unzip vosk-model-small-en-us-0.15.zip && rm vosk-model-small-en-us-0.15.zip && mv vosk-mo  0.0s
 => exporting to image                                                                                                                                                                                            0.0s
 => => exporting layers                                                                                                                                                                                           0.0s
 => => writing image sha256:1c6698adc262a63cc3d1ba0de529cdbfa23753ca5961aaf780c89e00c5677efa                                                                                                                      0.0s
 => => naming to docker.io/library/decky-dictation                                                                                                                                                                0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
cp: cannot create directory '/backend/out/vosk': Permission denied
Error: Failed to build backend. There might be more information in the output above.

Caused by:
    child status was: exit status: 1

Will wait for @nicholasRutherford's feedback and in case he has the same issues, I will open an issue on the decky cli in the hopes that we can figure out why this is happening. I had success creating files in other directories in the backend dir, but the out dir seems to have permission problems somehow, even though other directories that I had success writing to, also had the same permissions drwxr-xr-x.

[cboiangiu]

@Gaspadlo Could you please also give the VSCode build scripts a go under WSL? Just curious to see if we can get past this error.

[Gaspadlo]

@cboiangiu Hey there, I don't use VSCode but a phpstorm (basically a webstorm + php from JetBrains).
I have not meddled around with permissions (although I also had problems with writing into /tmp dir) - So I just changed the absolute paths to be relative to the project's directory (which makes a mess inside the project, but the final built .zip package was there in the output)

I also changed "gfind" to "find", wich should be the same gnu find util, but aliased diffrently under Ubuntu??

I think the correct solution should be using a tmp folder inside the docker container, that should get thrown away after the script is finished and the container is shut down. (I am no docker nor bash scripts expert. I leave setting them up in the projects to the devops guys).

#!/bin/bash

# Copied from https://github.com/Epictek/DeckyStream/blob/master/build-zip.sh#L7

plugin="decky-dictation"
docker_name="backend-decky-dictation"

dockerfile_exists="false"
entrypoint_exists="false"
docker_name="backend-decky-dictation"
# [ -d $PWD/backend ] && echo "$(ls -lla $PWD/backend | grep Dockerfile)"
[ -f $PWD/backend/Dockerfile ] && dockerfile_exists=true
[ -f $PWD/backend/entrypoint.sh ] && entrypoint_exists=true

#build backend
if [[ "$dockerfile_exists" == "true" ]]; then
  echo "Grabbing provided dockerfile."
  echo "Building provided Dockerfile."
  docker build -f $PWD/backend/Dockerfile -t "$docker_name" .
  mkdir -p ./tmp/output/$plugin/backend/out
  # check entrypoint script exists
  if [[ "$entrypoint_exists" == "true" ]]; then
    echo "Running docker image "$docker_name" with provided entrypoint script."
    docker run --rm -i -v $PWD/backend:/backend -v ./tmp/output/$plugin/backend/out:/backend/out --entrypoint /backend/entrypoint.sh "$docker_name"
    mkdir -p ./tmp/output/$plugin/bin
    cp -rv ./tmp/output/$plugin/backend/out/. ./tmp/output/$plugin/bin
  else
    echo "Running docker image "$docker_name" with entrypoint script specified in Dockerfile."
    docker run --rm -i -v $PWD/backend:/backend -v ./tmp/output/$plugin/backend/out:/backend/out "$docker_name"
    mkdir -p ./tmp/output/$plugin/bin
    cp -rv ./tmp/output/$plugin/backend/out/. ./tmp/output/$plugin/bin
  fi
  docker image rm "$docker_name"
  echo "Built $plugin backend"
# Dockerfile doesn't exist but entrypoint script does, run w/ default image
elif [[ "$dockerfile_exists" == "false" && "$entrypoint_exists" == "true" ]]; then
  echo "Grabbing default docker image and using provided entrypoint script."
  docker run --rm -i -v $PWD/backend:/backend -v ./tmp/output/$plugin/backend/out:/backend/out ghcr.io/steamdeckhomebrew/holo-base:latest
  mkdir -p ./tmp/output/$plugin/bin
  cp ./tmp/output/$plugin/backend/out/. ./tmp/output/$plugin/bin
  echo "Built $plugin backend"
else
  echo "Plugin $plugin does not have a backend"
fi

#build frontend
docker run --rm -i -v $PWD:/plugin -v ./tmp/output/$plugin:/out ghcr.io/steamdeckhomebrew/builder:latest
echo Built $plugin frontend
ls -lla ./tmp/output/$plugin

#make zip
mkdir -p ./tmp/zips/
mkdir -p ./tmp/output/
cd ./tmp/output/${plugin}
zipname=${plugin}.zip
echo $plugin
# Names of the optional files (the license can either be called license or license.md, not both)
# (head is there to take the first file, because we're assuming there's only a single license file)
license="$(find . -maxdepth 1 -type f \( -iname "license" -o -iname "license.md" \) -printf '%P\n' | head -n 1)"
readme="$(find . -maxdepth 1 -type f -iname 'readme.md' -printf '%P\n')"
haspython="$(find . -maxdepth 1 -type f -name '*.py' -printf '%P\n')"
# Check if plugin has a bin folder, if so, add "bin" and it's contents to root dir
hasbin="$(find . -maxdepth 1 -type d -name 'bin' -printf '%P\n')"
# Check if plugin has a defaults folder, if so, add "default" contents to root dir
hasdefaults="$(find . -maxdepth 1 -type d -name 'defaults' -printf '%P\n')"
#          if [[ "${{ secrets.STORE_ENV }}" == "testing" ]]; then
#            long_sha="${{ github.event.pull_request.head.sha || github.sha }}"
#            sha=$(echo $long_sha | cut -c1-7)
#            cat $plugin/package.json | jq --arg jqsha "$sha" '.version |= . + "-" + $jqsha' | sudo tee $plugin/$sha-package.json
#            sudo mv $plugin/$sha-package.json $plugin/package.json
#          fi
# Add required plugin files (and directory) to zip file
echo "dist plugin.json package.json"
zip -r $zipname "dist" "plugin.json" "package.json"

if [ ! -z "$hasbin" ]; then
  ls -al bin
  echo "/bin"
  zip -r $zipname "bin"
fi
if [ ! -z "$haspython" ]; then
  echo "*.py"
  find . -maxdepth 1 -type f -name '*.py' -exec zip -r $zipname {} \;
fi
if [ ! -z "$hasdefaults" ]; then
  export workingdir=$PWD
  cd defaults
  export plugin="$plugin"
  export zipname="$zipname"
  if [ ! -f "defaults.txt" ]; then
    find . -mindepth 1 -type d,f -name '*' -exec bash -c '
                for object do
                  outdir="./tmp/output"
                  name="$(basename $object)"
                  # echo "object = $object, name = $name"
                  if [ -e "$object" ]; then
                    sudo mv "$object" $outdir/$plugin/$name
                    moved="$?"
                    # echo "moved = $moved"
                    cd $workingdir
                    if [ "$moved" = "0" ]; then
                      zip -r $zipname $plugin/$name
                    fi
                  fi
                done
              ' find-sh {} +
  else
    if [[ ! "$plugin" =~ "plugin-template" ]]; then
      printf "${red}defaults.txt found in defaults folder, please remove either defaults.txt or the defaults folder.${end}\n"
    else
      printf "plugin template, allowing defaults.txt\n"
    fi
  fi
  cd "$workingdir"
fi
# Check if other files exist, and if they do, add them
echo "license:$plugin/$license readme:$plugin/$readme"
if [ ! -z "$license" ]; then
  zip -r $zipname "$license"
fi
if [ ! -z "$readme" ]; then
  zip -r $zipname "$readme"
fi

[cboiangiu]

@Gaspadlo You can find the build scripts inside .vscode. Try to run them from the plugin working directory as such . ./.vscode/setup.sh. The order should be setup.sh, config.sh, build.sh. The build.sh script in particular uses the decky cli instead of the build-zip.sh.

Edit: Make sure to run setup.sh as such setup.sh linux-x86_64

[Gaspadlo]

@cboiangiu oh well... Might be some WSL shenanigans:

• setup.sh - PLATFORM did not resolve for me, had to manually set it to "linux-x86_64" and put $PLATFORM on line 5 into quotation marks
• build.sh ends with "libssl.so.3 not found"

Seems like my WSL Ubuntu 20.4 LTS image is quite old and does not have up-to-date libssl package available. Probably a good idea to make a new fresh Ubuntu WSL instance after 2 - 3 years. (It's gonna be pain migrating all project... sigh)

(at least the old build script with relative paths still works for me :) )

[cboiangiu]

I have indeed tested the old build script and it does seem to create a good looking zip file, but the disclaimers within the setup.sh script seem to be pushing decky cli quite hard. Not sure atm whether plugin store submission requires the use of it or we can still go with the old build script approach.

In the meantime I will do a bit more cleanup and review for this PR. Will also push the remote binary fixes here. I have also removed the model zip file which bloated the plugin with about 40 MB. Sorry for that 😅

[cboiangiu]

Oh also, if y'all have any suggestions for non overlapping SteamOS button bindings, please let me know. I wish to land on something that doesn't take over any single button uses (so we'll most likely need a combo). If we can also have it working for external controllers lacking extra back buttons, that would be a huge plus.

[Gaspadlo]

Well the ideal solution would obviously be via GUI user-definable persistent bindings...
(Per game would be even bigger feature)

When I tried putting a debug notifications in the index.tsx to see codes for the upper paddle button (L4 & R4) - (If I am not mistaken) - these buttons can't be caught for some reason the way they are being listened to right now, yes?

My main problem with the Steam+B+L2/R2 combo was that the combination of Steam+B+someting triggers some kind "halfway jumped out of the game/application" state and even when the game remains in front - I always lose my custom steaminput bindings in WoW - so I'd have to open Steam menu and press "resume game" to have GameMode "acknowledge" my custom SteamInput bindings again - That wasn't ideal, so I just set them to the bottom paddle buttons myself, since I use those only for enter and escape anyway.

(Plus just pressing L5 - talking - pressing L5 - and then R5 felt just so smooth and seamless - the vosk model could use some improvements - I was stuck at saying word "Test" - being registered as "Best" - In the end I had to say it with a higher pitch and mouth wide open, to register the word as "test" - maybe some of the larger model would perform better, but we are talking about ~40/50MB VS GBs + I am not sure about the increase of computational requirement with the larger model - like if the larger model would have a noticable impact on the game being played... Well at the very least the RAM requirement, because I guess it loads the whole file into the memory, right? PS: I am aware that is beyond the scope of this plugin and we are simply working with what the community has created and the lightweight vosk model is the latest one available :) )

For what the vosk + nerd dictation does right now in a combination with this plugin - it still seems like a magic and the occasional "voice-typos" are bearable, since others usually understands what I was trying to "say-type" anyway. If it's way off, I just press "stop + esc" and try again - still miles faster than trackpad typing.

[cboiangiu]

@nicholasRutherford Any chance we could give this a try under macOS? The issue is currently being investigated here.

[cboiangiu]

The issue I had with cli/decky has been fixed.
I will push the fix to this repo and will finally be able to give this a try on my Steam Deck 🚀

[cboiangiu]

All seems to work fine on my end.
Hotkeys have been changed to L5 and R5. There is also a toggle to enable or disable the plugin, in case you want to use the buttons for something else on some games. The toggle won't persist across reboots, but to be honest I wouldn't want it to anyways. I think of the toggle as something you enable when you play your text chat heavy games and you disable when you exit them. Added a note about per game hotkeys as a nice to have. This should do for now.

Last thing I want to do is move the model to a remote binary and we should hopefully be able to merge this and move forward with an update submission.

[jreggett]

Hi, does this addon still work? Would love love love to get this working for me. Doesn’t appear to be in Decky, I’m guessing I have to install it manually? My understanding is that means I extract the zip file to the homebrew addons file. That doesn’t appear to work, I guess I don’t have the permissions. From what I read elsewhere I need to “sudo copy” it there with konsole (whatever the hell that means). Does that sound accurate? Any tips on getting it run for me?

Thank you!