You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a user, I would like to easily access a crawler for the web. This would allow me to easily research content from the web and grow the knowledge graph. Right now, the crawler is in the graph engine's codebase and that should be moved out since it is not a part of the graph.
Engineering
The separate crawler project uses spider-rs but we can continue building with the simple HTTP request based crawler till we need spider-rs. Let us migrate the simple crawler from within the pixlieai_rs to the crawler project.
The text was updated successfully, but these errors were encountered:
As a user, I would like to easily access a crawler for the web. This would allow me to easily research content from the web and grow the knowledge graph. Right now, the crawler is in the graph engine's codebase and that should be moved out since it is not a part of the graph.
Engineering
The separate
crawler
project usesspider-rs
but we can continue building with the simple HTTP request based crawler till we needspider-rs
. Let us migrate the simple crawler from within thepixlieai_rs
to thecrawler
project.The text was updated successfully, but these errors were encountered: