Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement basic full-text-search #1359

Merged
merged 6 commits into from
Oct 15, 2022
Merged

Implement basic full-text-search #1359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3e1eb52
Add 'isLatest' field to reduce array-contains uses
justinfagnani Oct 13, 2022
9a80564
Implement basic full-text-search
justinfagnani Oct 8, 2022
0897748
Address feedback
justinfagnani Oct 14, 2022
da9845a
Merge branch 'main' into elements-query
justinfagnani Oct 14, 2022
7589b7d
Add more tests
justinfagnani Oct 14, 2022
7b80145
Remove console log
justinfagnani Oct 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 1 addition & 20 deletions packages/catalog-api/src/lib/schema.graphql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,8 @@ type Query {
"""
Queries custom elements in the entire catalog, from the latest version of each
package.

Eventually this will have more query parameters, and use some sort of ranking
algorithm, otherwise the order will just be defined by the database
implementation.

NOT IMPLEMENTED
DO_NOT_LAUNCH
"""
elements(distTag: String = "latest", limit: Int): [CustomElement!]!

"""
Retrieves the custom element data for a single element.

NOT IMPLEMENTED
DO_NOT_LAUNCH
"""
element(
packageName: String!
elementName: String!
tag: String = "latest"
): CustomElement
elements(query: String, limit: Int): [CustomElement!]!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we perhaps also want a strict lookup like the element API we had before, or will we instead support search prefixes like package:foo element:bar?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the package() query gives us what we had with element() - and we can make some of the fields like customElements into queries if we want that.

I think this query should grow to add both search operators and possibly subqueries/fields that get you from element to package here.

One consideration is how to implement structured + text search. If we move to use a search service do we do both Firestore + search service queries and mix them? If a search service offer structured fields do we delegate all searches to it?

}

type Mutation {
Expand Down
2 changes: 2 additions & 0 deletions packages/catalog-server/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,15 @@
"@koa/router": "^12.0.0",
"@types/koa__router": "^12.0.0",
"@types/koa-bodyparser": "^4.3.8",
"@types/natural": "^5.1.1",
"@webcomponents/catalog-api": "0.0.0",
"@webcomponents/custom-elements-manifest-tools": "0.0.0",
"custom-elements-manifest": "^2.0.0",
"firebase": "^9.6.10",
"firebase-admin": "^11.0.0",
"graphql-helix": "^1.13.0",
"koa-bodyparser": "^4.3.0",
"natural": "^5.2.3",
"node-fetch": "^3.2.3",
"npm-registry-fetch": "^13.1.0",
"semver": "^7.3.7"
Expand Down
13 changes: 11 additions & 2 deletions packages/catalog-server/src/lib/catalog.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,8 +272,7 @@ export class Catalog {

console.log('Writing custom elements...');
await this.#repository.writeCustomElements(
packageName,
version,
packageVersionMetadata,
customElements,
versionDistTags,
author
Expand Down Expand Up @@ -307,11 +306,21 @@ export class Catalog {
return this.#repository.getPackageVersion(packageName, version);
}

/**
* Gets the custom elements for a package
*/
async getCustomElements(
packageName: string,
version: string,
tagName: string | undefined
): Promise<Array<CustomElement>> {
return this.#repository.getCustomElements(packageName, version, tagName);
}

async queryElements({query, limit}: {query?: string; limit?: number}) {
// TODO (justinfagnani): The catalog should parse out GitHub-style search
justinfagnani marked this conversation as resolved.
Show resolved Hide resolved
// operators (like "author:yogibear") and pass structured + text queries
// to the repository
return this.#repository.queryElements({query, limit});
}
}
Loading