Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): Add shortest init command #297

Merged
merged 22 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,20 @@ If helpful, [here's a short video](https://github.com/anti-work/shortest/issues/

### Installation

```bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual instructions can be removed. The description below outlines the steps, in case someone would like to do it manually.

npm install -D @antiwork/shortest
# or
pnpm add -D @antiwork/shortest
```
Use the `shortest init` command to streamline the setup process in a new or existing project.

Add `.shortest/` to your `.gitignore` (where Shortest stores screenshots and caching of each test run):
The `shortest init` command will:

```bash
echo ".shortest/" >> .gitignore
```sh
npx @antiwork/shortest init
```

This will:
- Automatically install the `@antiwork/shortest` package as a dev dependency if it is not already installed
- Create a default `shortest.config.ts` file with boilerplate configuration
- Generate a `.env.local` file (unless present) with placeholders for required environment variables, such as `ANTHROPIC_API_KEY`
- Add `.env.local` and `.shortest/` to `.gitignore`

### Quick start

1. Determine your test entry and add your Anthropic API key in config file: `shortest.config.ts`
Expand Down
18 changes: 10 additions & 8 deletions packages/shortest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,20 @@ AI-powered natural language end-to-end testing framework.

### Installation

```bash
npm install -D @antiwork/shortest
# or
pnpm add -D @antiwork/shortest
```
Use the `shortest init` command to streamline the setup process in a new or existing project.

Add `.shortest/` to your `.gitignore` (where Shortest stores screenshots and caching of each test run):
The `shortest init` command will:

```bash
echo ".shortest/" >> .gitignore
```sh
npx @antiwork/shortest init
```

This will:
- Automatically install the `@antiwork/shortest` package as a dev dependency if it is not already installed
- Create a default `shortest.config.ts` file with boilerplate configuration
- Generate a `.env.local` file (unless present) with placeholders for required environment variables, such as `ANTHROPIC_API_KEY`
- Add `.env.local` and `.shortest/` to `.gitignore`

### Quick start

1. Determine your test entry and add your Anthropic API key in config file: `shortest.config.ts`
Expand Down
3 changes: 2 additions & 1 deletion packages/shortest/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
},
"files": [
"dist",
"dist/cli"
"dist/cli",
"src/shortest.config.ts.example"
],
"scripts": {
"build": "rimraf dist && pnpm build:types && pnpm build:js && pnpm build:cli",
Expand Down
3 changes: 2 additions & 1 deletion packages/shortest/src/ai/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import Anthropic from "@anthropic-ai/sdk";
import pc from "picocolors";
import { BashTool } from "../browser/core/bash-tool";
import { BrowserTool } from "../browser/core/browser-tool";
import { CONFIG_FILENAME } from "../constants";
import { ToolResult } from "../types";
import { AIConfig, RequestBash, RequestComputer } from "../types/ai";
import { CacheAction, CacheStep } from "../types/cache";
Expand All @@ -17,7 +18,7 @@ export class AIClient {
constructor(config: AIConfig, debugMode: boolean = false) {
if (!config.apiKey) {
throw new Error(
"Anthropic API key is required. Set it in shortest.config.ts or ANTHROPIC_API_KEY env var",
`Anthropic API key is required. Set it in ${CONFIG_FILENAME} or ANTHROPIC_API_KEY env var`,
);
}

Expand Down
3 changes: 2 additions & 1 deletion packages/shortest/src/browser/integrations/github.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import dotenv from "dotenv";
import { authenticator } from "otplib";
import { ENV_LOCAL_FILENAME } from "../../constants";
import { BrowserToolInterface } from "../../types/browser";

export class GitHubTool {
Expand All @@ -16,7 +17,7 @@ export class GitHubTool {
};

constructor(secret?: string) {
dotenv.config({ path: [".env", ".env.local"] });
dotenv.config({ path: [".env", ENV_LOCAL_FILENAME] });

this.totpSecret = secret || process.env.GITHUB_TOTP_SECRET || "";

Expand Down
13 changes: 9 additions & 4 deletions packages/shortest/src/cli/bin.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/usr/bin/env node

import pc from "picocolors";
import { getConfig } from "..";
import { GitHubTool } from "../browser/integrations/github";
import { CONFIG_FILENAME, ENV_LOCAL_FILENAME } from "../constants";
import { TestRunner } from "../core/runner";

process.removeAllListeners("warning");
Expand Down Expand Up @@ -42,7 +42,7 @@ ${pc.bold("Options:")}
--no-cache Disable caching (storing browser actions between tests)

${pc.bold("Authentication:")}
--secret=<key> GitHub TOTP secret key (or use .env.local)
--secret=<key> GitHub TOTP secret key (or use ${ENV_LOCAL_FILENAME})

${pc.bold("Examples:")}
${pc.dim("# Run all tests")}
Expand All @@ -58,7 +58,7 @@ ${pc.bold("Examples:")}
shortest --github-code --secret=<OTP_SECRET>

${pc.bold("Environment Setup:")}
Required variables in .env.local:
Required variables in ${ENV_LOCAL_FILENAME}:
- ANTHROPIC_API_KEY Required for AI test execution
- GITHUB_TOTP_SECRET Required for GitHub authentication
- GITHUB_USERNAME GitHub login credentials
Expand Down Expand Up @@ -111,6 +111,11 @@ function isValidArg(arg: string): boolean {
async function main() {
const args = process.argv.slice(2);

if (args[0] === "init") {
await require("../commands/init").default();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracted all the logic to its own module.

process.exit(0);
}

if (args.includes("--help") || args.includes("-h")) {
showHelp();
process.exit(0);
Expand Down Expand Up @@ -157,7 +162,7 @@ async function main() {
console.error(pc.dim(error.message));
console.error(
pc.dim(
"\nMake sure you have a valid shortest.config.ts with all required fields:",
`\nMake sure you have a valid ${CONFIG_FILENAME} with all required fields:`,
),
);
console.error(pc.dim(" - headless: boolean"));
Expand Down
124 changes: 124 additions & 0 deletions packages/shortest/src/commands/init/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
import { execSync } from "child_process";
import { readFile, writeFile } from "node:fs/promises";
import { join } from "path";
import { fileURLToPath } from "url";
import { detect, resolveCommand } from "package-manager-detector";
import pc from "picocolors";
import { CONFIG_FILENAME, ENV_LOCAL_FILENAME } from "../../constants";
import { addToEnv } from "../../utils/add-to-env";
import { addToGitignore } from "../../utils/add-to-gitignore";

export default async function main() {
console.log(pc.blue("Setting up Shortest..."));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blue doesn't look great on a dark terminal. We should consider improving the contrast at some point.


try {
const packageJson = await getPackageJson();
if (
packageJson?.dependencies?.["@antiwork/shortest"] ||
packageJson?.devDependencies?.["@antiwork/shortest"]
) {
console.log(pc.green("✔ Package already installed"));
return;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important to exit early if the package is already installed, otherwise there would be some considerable effort to ensure the install doesn't overwrite any existing data.

} else {
console.log("Installing @antiwork/shortest...");
const installCmd = await getInstallCmd();
execSync(installCmd, { stdio: "inherit" });
console.log(pc.green("✔ Dependencies installed"));
}

const configPath = join(process.cwd(), CONFIG_FILENAME);
const exampleConfigPath = join(
fileURLToPath(new URL("../../src", import.meta.url)),
`${CONFIG_FILENAME}.example`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified the logic to use an example file.

);

const exampleConfig = await readFile(exampleConfigPath, "utf8");
await writeFile(configPath, exampleConfig, "utf8");
console.log(pc.green(`✔ ${CONFIG_FILENAME} created`));

const envResult = await addToEnv(process.cwd(), {
ANTHROPIC_API_KEY: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's worth adding Mailosaur at this time, although other ENVs can be easily added in the future.

value: "your_value_here",
comment: "Shortest variables",
},
});
if (envResult.error) {
console.error(
pc.red(`Failed to update ${ENV_LOCAL_FILENAME}`),
envResult.error,
);
} else if (envResult.added.length > 0) {
const added = envResult.added.join(", ");
const skipped = envResult.skipped.join(", ");
const detailsString = [
added ? `${added} added` : "",
skipped ? `${skipped} skipped` : "",
]
.filter(Boolean)
.join(", ");
console.log(
pc.green(
`✔ ${ENV_LOCAL_FILENAME} ${envResult.wasCreated ? "created" : "updated"} (${detailsString})`,
),
);
}

const resultGitignore = await addToGitignore(process.cwd(), [
".env*.local",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not worth the effort trying to group with other env*-like values already present (e.g. .env from a fresh Next.js install).

".shortest/",
]);
if (resultGitignore.error) {
console.error(
pc.red("Failed to update .gitignore"),
resultGitignore.error,
);
} else {
console.log(
pc.green(
`✔ .gitignore ${resultGitignore.wasCreated ? "created" : "updated"}`,
),
);
}

console.log(pc.green("\nInitialization complete! Next steps:"));
console.log(`1. Update ${ENV_LOCAL_FILENAME} with your values`);
console.log("2. Create your first test file: example.test.ts");
console.log("3. Run tests with: npx/pnpm test example.test.ts");
} catch (error) {
console.error(pc.red("Initialization failed:"), error);
process.exit(1);
}
}

export const getPackageJson = async () => {
try {
return JSON.parse(
await readFile(join(process.cwd(), "package.json"), "utf8"),
);
} catch {}
};

export const getInstallCmd = async () => {
const packageManager = (await detect()) || { agent: "npm", version: "" };
const packageJson = await getPackageJson();
if (packageJson?.packageManager) {
const [name] = packageJson.packageManager.split("@");
if (["pnpm", "yarn", "bun"].includes(name)) {
packageManager.agent = name;
}
}

const command = resolveCommand(packageManager.agent, "install", [
"@antiwork/shortest",
"--save-dev",
]);

if (!command) {
throw new Error(`Unsupported package manager: ${packageManager.agent}`);
}

const cmdString = `${command.command} ${command.args.join(" ")}`;
console.log(pc.dim(cmdString));

return cmdString;
};
2 changes: 2 additions & 0 deletions packages/shortest/src/constants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export const CONFIG_FILENAME = "shortest.config.ts";
export const ENV_LOCAL_FILENAME = ".env.local";
11 changes: 6 additions & 5 deletions packages/shortest/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { join } from "path";
import dotenv from "dotenv";
import { expect as jestExpect } from "expect";
import { APIRequest } from "./browser/core/api-request";
import { CONFIG_FILENAME, ENV_LOCAL_FILENAME } from "./constants";
import { TestCompiler } from "./core/compiler";
import {
TestFunction,
Expand Down Expand Up @@ -42,7 +43,7 @@ if (!global.__shortest__) {
global.expect = global.__shortest__.expect;

dotenv.config({ path: join(process.cwd(), ".env") });
dotenv.config({ path: join(process.cwd(), ".env.local") });
dotenv.config({ path: join(process.cwd(), ENV_LOCAL_FILENAME) });
}

function validateConfig(config: Partial<ShortestConfig>) {
Expand All @@ -56,7 +57,7 @@ function validateConfig(config: Partial<ShortestConfig>) {

if (missingFields.length > 0) {
throw new Error(
`Missing required fields in shortest.config.ts:\n` +
`Missing required fields in ${CONFIG_FILENAME}:\n` +
missingFields.map((field) => ` - ${field}`).join("\n"),
);
}
Expand All @@ -66,10 +67,10 @@ export async function initialize() {
if (globalConfig) return globalConfig;

dotenv.config({ path: join(process.cwd(), ".env") });
dotenv.config({ path: join(process.cwd(), ".env.local") });
dotenv.config({ path: join(process.cwd(), ENV_LOCAL_FILENAME) });

const configFiles = [
"shortest.config.ts",
CONFIG_FILENAME,
"shortest.config.js",
"shortest.config.mjs",
];
Expand Down Expand Up @@ -97,7 +98,7 @@ export async function initialize() {
}

throw new Error(
"No config file found. Create shortest.config.ts in your project root.\n" +
`No config file found. Create ${CONFIG_FILENAME} in your project root.\n` +
"Required fields:\n" +
" - headless: boolean\n" +
" - baseUrl: string\n" +
Expand Down
8 changes: 8 additions & 0 deletions packages/shortest/src/shortest.config.ts.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import type { ShortestConfig } from "@antiwork/shortest";

export default {
headless: false,
baseUrl: "http://localhost:3000",
testPattern: "**/*.test.ts",
anthropicKey: process.env.ANTHROPIC_API_KEY,
} satisfies ShortestConfig;
63 changes: 63 additions & 0 deletions packages/shortest/src/utils/add-to-env.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { readFile, writeFile } from "node:fs/promises";
import os from "os";
import { join } from "path";
import { ENV_LOCAL_FILENAME } from "../constants";

type EnvResult = {
added: string[];
skipped: string[];
wasCreated: boolean;
error?: Error;
};

export async function addToEnv(
path: string,
entries: Record<string, { value: string; comment?: string }>,
): Promise<EnvResult> {
const result: EnvResult = {
added: [],
skipped: [],
wasCreated: false,
};

try {
const envPath = join(path, ENV_LOCAL_FILENAME);
let envContent = await readFile(envPath, "utf8").catch(() => null);
result.wasCreated = envContent === null;
envContent = envContent ?? "";
const EOL = envContent.includes("\r\n") ? "\r\n" : os.EOL;

const existingEntries = new Map(
envContent
.split(EOL)
.filter((line) => line.trim() && !line.startsWith("#"))
.map((line) => {
const [key] = line.split("=");
return [key.trim(), true];
}),
);

let content = envContent;
for (const [key, { value, comment }] of Object.entries(entries)) {
if (existingEntries.has(key)) {
result.skipped.push(key);
continue;
}

const needsEol = content.length > 0 && !content.endsWith(EOL);
if (comment) {
content += `${needsEol ? EOL : ""}# ${comment}${EOL}`;
}
content += `${needsEol && !comment ? EOL : ""}${key}=${value}${EOL}`;
result.added.push(key);
}

if (result.added.length > 0) {
await writeFile(envPath, content);
}
} catch (error) {
result.error = error as Error;
}

return result;
}
Loading
Loading