Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate example code #112

Open
alexsb opened this issue Feb 6, 2025 · 1 comment
Open

Validate example code #112

alexsb opened this issue Feb 6, 2025 · 1 comment

Comments

@alexsb
Copy link
Contributor

alexsb commented Feb 6, 2025

We recently found several examples of outdated or buggy code snippets on the website (see #111, #109).

We should somehow avoid that and have a way to automatically catch that.

There are two cases. Case 1 is where the study contains a fully working config, like described in #111. There, we should load that example from code and have it in the study repo as one of the demos.

Case 2 is more difficult: when we have snippets. Those snippets often have shortcuts, e.g., "..." in them. I can't really see a good way to address this.

@mcnuttandrew
Copy link

mcnuttandrew commented Feb 6, 2025

For case 1: i think you could probably just have some sort of integration check that checks all of them to see if they pass the lint and maybe run? like this seems like a job for the CI rather than trying to merge everything into the demos

For case 2: trickier! when i've needed to do stuff like this in the past i've tagged things that i want to be under automation watch. One could imagine doing something like this in the types

js<fullExample>
{
  "type": "website",
  "path": "<study-name>/website.html",
  "parameters": {
    "barData": [0.32, 0.01, 1.2, 1.3, 0.82, 0.4, 0.3]
  }
  "response": [
    {
      "id": "barChart",
      "prompt": "Your selected answer:",
      "location": "belowStimulus",
      "type": "reactive"
    }
  ],
}

And then extract the things marked with <fullExample> and check em. This means that the checking is opt-in, and you can specify things like what to check, like <fullExample:json> and so forth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants