Add samples for Headless PrPr (single node) #156
Conversation
| # Headless Container Runtime Jobs | ||
|
|
||
| ## Setup | ||
|
|
There was a problem hiding this comment.
Let's add an intro section here so we put these steps into context? Like "The remote ML execution framework relies on SPCS jobs (link to doc) to execute the user's code inside the Container Runtime environment..."
There was a problem hiding this comment.
Also let's inform the user that these below steps are steps to get your environment in place to use our remote execution framework
There was a problem hiding this comment.
Let's do that in a separate document. I'm envisioning this README as a super concise "getting started" guide with the assumption that the user already wants to use headless. Overall we'd probably want each of these as separate documents:
- Overview - provide background, motivations, and introduce concepts. Similar to https://docs.snowflake.com/en/developer-guide/snowflake-ml/container-runtime-ml
- API Reference - e.g. https://docs.snowflake.com/en/developer-guide/snowpark-ml/reference/latest/index
- Quick start (i.e. this README) - short technical guide to get user up and running in <5 minutes, with pointers to additional resources for more advanced usage
- Tutorials (e.g.
pytorch-cifar10/README.mdandxgb-loan-apps/README.md) - full end-to-end walkthroughs
The intro sections would go in No.1 (Overview). WDYT?
| @@ -0,0 +1,260 @@ | |||
| # Headless Container Runtime Jobs | |||
There was a problem hiding this comment.
Fine for now. But let's talk to marketing and consider a more appropriate name
| INSTANCE_FAMILY = CPU_X64_S -- See https://docs.snowflake.com/en/sql-reference/sql/create-compute-pool | ||
| ``` | ||
|
|
||
| ### Function Dispatch |
There was a problem hiding this comment.
I'd also include an intro here.
Maybe you can pull from the PRD. So say something like "Users who are looking to gain the benefits offered by Snowflake's Container Runtime for ML (link to docs), including flexibility with packages, choice of CPU vs GPUs, and ability to use distributed APIs to scale your workloads, but want to do so from their own IDE can instrument their code to execute remotely..."
| ) | ||
| ``` | ||
|
|
||
| ### Airflow Integration |
There was a problem hiding this comment.
Here as well. Let's provide an intro for why this is different than any other Airflow integration. i.e. Building an ML pipeline so that steps in the workflow can execute in the Container Runtime () with benefits such as ...
Add guides for getting started with Headless Runtime PrPr