Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise Viz docs content to add intersphinx linking #1628

Merged
merged 24 commits into from
Nov 13, 2023
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f7d9aea
something to start off
stichbury Nov 8, 2023
5960089
Fix permalink icon
astrojuanlu Nov 8, 2023
c030361
Revise some links to framework docs
stichbury Nov 8, 2023
a442790
Merge branch 'viz-docs-update' of https://github.com/kedro-org/kedro-…
stichbury Nov 8, 2023
329caf1
Add live tracking demo to open in new tab
stichbury Nov 8, 2023
ebba904
Remove code to load old CSS
stichbury Nov 8, 2023
44bec27
[Draft, not ready] Update docs front page following design review (#1…
stichbury Nov 8, 2023
945e1e1
A few content improvements
stichbury Nov 8, 2023
c1d7ad9
Update layout.html
vladimir-mck Nov 8, 2023
eaed0d0
Update layout.html
vladimir-mck Nov 8, 2023
9fbea4c
Delete docs/source/kedro-horizontal-color-on-light.svg
vladimir-mck Nov 8, 2023
b2e67bf
Update conf.py
vladimir-mck Nov 9, 2023
ddc8ec5
Update layout.html
stichbury Nov 9, 2023
c26e025
Update Shareable Viz docs with sections on creating access key and bi…
NeroOkwa Nov 9, 2023
5929057
Update docs/source/conf.py
stichbury Nov 10, 2023
7c94911
Update docs/source/preview_datasets.md
stichbury Nov 10, 2023
5c30d85
Update docs/source/visualise_charts_with_matplotlib.md
stichbury Nov 10, 2023
d65db3c
Update docs/source/share_kedro_viz.md
stichbury Nov 10, 2023
7a45793
Update docs/source/visualise_charts_with_plotly.md
stichbury Nov 10, 2023
667fc94
Update docs/source/preview_datasets.md
stichbury Nov 10, 2023
830d462
Merge branch 'main' into viz-docs-update
stichbury Nov 10, 2023
4b8e914
Suggested variable removal
stichbury Nov 10, 2023
9ad1646
Merge branch 'viz-docs-update' of https://github.com/kedro-org/kedro-…
stichbury Nov 10, 2023
5e471d6
Fix links
stichbury Nov 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 33 additions & 26 deletions docs/source/_templates/layout.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,44 +5,51 @@
{%- set _logo_url = logo_url|default(pathto('_static/' + (logo or ""), 1))
%} {%- set _root_doc = root_doc|default(master_doc) %}
<a href="{{ pathto(_root_doc) }}">
{% if not theme_logo_only %}{{ project }}{% endif %} {%- if logo or
logo_url %}
<img src="{{ _logo_url }}" class="logo" alt="{{ _('Logo') }}" />
{%- endif %}
<img src="https://raw.githubusercontent.com/kedro-org/kedro-brand-identity/main/horizontal/color/kedro-horizontal-color-on-light.svg" class="logo" alt="{{ _('Logo') }}" />
</a>

{%- if theme_display_version %} {%- set nav_version = version %} {%- if
READTHEDOCS and current_version %} {%- set nav_version = current_version %}
{%- endif %} {%- if nav_version %}
<div class="wy-header-version">{{ nav_version }}</div>
{%- endif %} {%- endif %}
</div>

<div class="wy-main-nav">
<a href="https://docs.kedro.org/" >Kedro</a>
<a href="https://docs.kedro.org">Kedro</a>
<a href="/" class="active">Kedro-Viz</a>
<a href="#">Kedro-Datasets</a>
<a href="https://docs.kedro.org/projects/kedro-datasets">Kedro-Datasets</a>
</div>

{%- include "searchbox.html" %}
</div>

{% endblock %}

{%- block extrahead %}
{% endblock %} {%- block extrahead %}
<script type="text/javascript">
window.heap=window.heap||[],heap.load=function(e,t){window.heap.appid=e,window.heap.config=t=t||{};var r=document.createElement("script");r.type="text/javascript",r.async=!0,r.src="https://cdn.heapanalytics.com/js/heap-"+e+".js";var a=document.getElementsByTagName("script")[0];a.parentNode.insertBefore(r,a);for(var n=function(e){return function(){heap.push([e].concat(Array.prototype.slice.call(arguments,0)))}},p=["addEventProperties","addUserProperties","clearEventProperties","identify","resetIdentity","removeEventProperty","setEventProperties","track","unsetEventProperty"],o=0;o<p.length;o++)heap[p[o]]=n(p[o])};

heap.load({{ "522942930" }});

// Wait for DOM to load and clone main nav into the sidebar
document.addEventListener("DOMContentLoaded", function() {
var sidebarSearch = document.querySelector('.wy-side-nav-search div[role="search"]');
sidebarSearch.parentNode.insertBefore(document.querySelector('.wy-main-nav').cloneNode(true), sidebarSearch);
});
window.heap=window.heap||[],heap.load=function(e,t){window.heap.appid=e,window.heap.config=t=t||{};var r=document.createElement("script");r.type="text/javascript",r.async=!0,r.src="https://cdn.heapanalytics.com/js/heap-"+e+".js";var a=document.getElementsByTagName("script")[0];a.parentNode.insertBefore(r,a);for(var n=function(e){return function(){heap.push([e].concat(Array.prototype.slice.call(arguments,0)))}},p=["addEventProperties","addUserProperties","clearEventProperties","identify","resetIdentity","removeEventProperty","setEventProperties","track","unsetEventProperty"],o=0;o<p.length;o++)heap[p[o]]=n(p[o])};

heap.load({{ "522942930" }});

// Wait for DOM to load and clone main nav into the sidebar
document.addEventListener("DOMContentLoaded", function() {
var sidebarSearch = document.querySelector('.wy-side-nav-search div[role="search"]');
sidebarSearch.parentNode.insertBefore(document.querySelector('.wy-main-nav').cloneNode(true), sidebarSearch);

// Prepend close button to mobile header
var closeBtn = document.createElement("span");
closeBtn.dataset.toggle = 'wy-nav-top';
closeBtn.classList.add('wy-nav-top-close');
document.querySelector('.wy-nav-top').prepend(closeBtn);

// Prepend open button to mobile header
var openBtn = document.createElement("span");
openBtn.dataset.toggle = 'wy-nav-top';
openBtn.classList.add('wy-nav-top-open');
document.querySelector('.wy-nav-top').prepend(openBtn);

</script>
<link rel="stylesheet" href="https://kedro-shared-files.s3.eu-west-2.amazonaws.com/qb1-sphinx-rtd.css" type="text/css"
<link
rel="stylesheet"
href="https://kedro-shared-files.s3.eu-west-2.amazonaws.com/qb1-sphinx-rtd.css"
type="text/css"
/>
<link rel="stylesheet" href="https://kedro-shared-files.s3.eu-west-2.amazonaws.com/theme-overrides.css" type="text/css"
<link
rel="stylesheet"
href="https://kedro-shared-files.s3.eu-west-2.amazonaws.com/theme-overrides.css"
type="text/css"
/>
{% endblock %}
17 changes: 11 additions & 6 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@

# -- Project information -----------------------------------------------------

project = "kedro-viz"
author = "kedro-viz"
project = "kedro"
author = "kedro"
stichbury marked this conversation as resolved.
Show resolved Hide resolved
stichbury marked this conversation as resolved.
Show resolved Hide resolved

# The short X.Y version.
version = re.match(r"^([0-9]+\.[0-9]+).*", release).group(1)
Expand Down Expand Up @@ -53,6 +53,7 @@
myst_heading_anchors = 2

intersphinx_mapping = {
"kedro": ("https://docs.kedro.org/en/stable/", None),
stichbury marked this conversation as resolved.
Show resolved Hide resolved
"kedro-datasets": ("https://docs.kedro.org/projects/kedro-datasets/en/kedro-datasets-1.7.1/", None),
}

Expand All @@ -62,6 +63,7 @@
# a list of builtin themes.
#
html_theme = "sphinx_rtd_theme"
here = Path(__file__).parent.absolute()
stichbury marked this conversation as resolved.
Show resolved Hide resolved


# Theme options are theme-specific and customise the look and feel of a theme
Expand All @@ -79,8 +81,11 @@
}


def _override_permalinks_icon(app):
# https://github.com/readthedocs/sphinx_rtd_theme/issues/98#issuecomment-1503211439
app.config.html_permalinks_icon = "¶"


def setup(app):
app.add_css_file("css/qb1-sphinx-rtd.css")
# fix a bug with table wraps in Read the Docs Sphinx theme:
# https://rackerlabs.github.io/docs-rackspace/tools/rtd-tables.html
app.add_css_file("css/theme-overrides.css")
app.connect("builder-inited", _override_permalinks_icon)

8 changes: 4 additions & 4 deletions docs/source/experiment_tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ The metadata you store may include:

You can use Kedro-Viz experiment tracking to store and access results, and to share them with others for comparison. Storage can be local or remote, such as cloud storage on AWS S3.

Kedro's [experiment tracking demo](https://demo.kedro.org/experiment-tracking) enables you to explore the experiment tracking capabilities of Kedro-Viz.
The <a href="https://demo.kedro.org/" target="_blank" rel="noopener noreferrer">experiment tracking demo</a> enables you to explore the experiment tracking capabilities of Kedro-Viz.

![](./images/experiment-tracking_demo.gif)

Expand All @@ -35,11 +35,11 @@ The choice of experiment tracking tool depends on your use case and choice of co
- **MLflow** - You can combine MLflow with Kedro by using [`kedro-mlflow`](https://kedro-mlflow.readthedocs.io/en/stable/) if you require experiment tracking, model registry and/or model serving capabilities or have access to Managed MLflow within the Databricks ecosystem.
- **Neptune** - If you require experiment tracking and model registry functionality, improved visualisation of metrics and support for collaborative data science, you may consider [`kedro-neptune`](https://docs.neptune.ai/integrations/kedro/) for your workflow.

[We support a growing list of integrations](https://docs.kedro.org/en/stable/extend_kedro/plugins.html).
{doc}`We support a growing list of integrations<kedro:extend_kedro/plugins>`.

## Set up a project

This section describes the steps necessary to set up experiment tracking and access logged metrics, using the [spaceflights tutorial](https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html) with a version of Kedro equal to or higher than 0.18.4, and a version of Kedro-Viz equal to or higher than 5.2.
This section describes the steps necessary to set up experiment tracking and access logged metrics, using the {doc}`spaceflights tutorial<kedro:tutorial/spaceflights_tutorial>` with a version of Kedro equal to or higher than 0.18.4, and a version of Kedro-Viz equal to or higher than 5.2.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that all these {doc} references are opened in a new window, is that on purpose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't 😆 although I'm asked for it often enough and I think it's quite a reasonable thing to do. How to control this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good question 😅 I'll do some digging


There are three steps to enable experiment tracking features with Kedro-Viz. We illustrate how to:

Expand All @@ -48,7 +48,7 @@ There are three steps to enable experiment tracking features with Kedro-Viz. We
- [Modify your nodes and pipelines to output those metrics](#modify-your-nodes-and-pipelines-to-log-metrics)

### Install Kedro and Kedro-Viz
To use this tutorial code, you must already have [installed Kedro](https://docs.kedro.org/en/stable/get_started/install.html) and [Kedro-Viz](./kedro-viz_visualisation.md). You can confirm the versions you have installed by running `kedro info`
To use this tutorial code, you must already have {doc}`installed Kedro<kedro:get_started/install>` and [Kedro-Viz](./kedro-viz_visualisation.md). You can confirm the versions you have installed by running `kedro info`
stichbury marked this conversation as resolved.
Show resolved Hide resolved

```{note}
The example code uses a version of Kedro-Viz `>6.2.0`.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 8 additions & 8 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
```{image} https://raw.githubusercontent.com/kedro-org/kedro/main/static/img/kedro_banner.png
:alt: Kedro logo
:class: kedro-logo
```

# Welcome to Kedro-Viz documentation!

<p align="center">

![Kedro-Viz Pipeline Visualisation](https://raw.githubusercontent.com/kedro-org/kedro-viz/main/.github/img/banner.png)

Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project.
</p>

Kedro-Viz is an interactive development tool for visualising data science pipelines built with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also enables users to view and compare different experiment runs within their Kedro project.

Kedro-Viz features include:

✨ Complete visualisation of a Kedro project and its pipelines.
🎨 Support for light & dark themes out of the box.
Support for light & dark themes out of the box.
🚀 Scalable to big pipelines with hundreds of nodes.
🔎 Highly interactive, filterable and searchable.
🔬 Focus mode for modular pipeline visualisation.
Expand All @@ -20,7 +20,7 @@ Kedro-Viz features include:
♻️ Autoreload on code change.
🧪 Support for experiment tracking and comparing runs in a Kedro project.


Take a look at the <a href="https://demo.kedro.org/" target="_blank" rel="noopener noreferrer">live demo</a> for a preview of Kedro-Viz.

```{toctree}
:caption: Learn about Kedro-Viz
Expand Down
19 changes: 10 additions & 9 deletions docs/source/kedro-viz_visualisation.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
# Visualise the spaceflights project
# How to visualise a Kedro project


This section assumes you are familiar with the basic Kedro concepts described in the [spaceflights tutorial](https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html). If you have not yet worked through the tutorial, you can still follow this example.
This section assumes you are familiar with the basic Kedro concepts described in the {doc}`spaceflights tutorial<kedro:tutorial/spaceflights_tutorial>`. If you have not yet worked through the tutorial, you can still follow this example.

If you haven't installed Kedro [follow the documentation to get set up](https://docs.kedro.org/en/stable/get_started/install.html).

Then, in your terminal window, navigate to the folder you want to store the project.
If you haven't installed Kedro {doc}`follow the documentation to get set up<kedro:get_started/install>`. Then, in your terminal window, navigate to the folder you want to store the project.

Generate a copy of the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights):
stichbury marked this conversation as resolved.
Show resolved Hide resolved


```bash
kedro new --starter=spaceflights
```
Expand Down Expand Up @@ -39,10 +38,10 @@ You should see the following:

![](./images/pipeline_visualisation.png)

If a visualisation panel opens up and a pipeline is not visible, refresh the view, and check that your tutorial project code is complete if you've not generated it from the starter template. If you still don't see the visualisation, the Kedro community can help:
If a visualisation panel opens up and a pipeline is not visible, refresh the view. If you still don't see the visualisation, the Kedro community can help:

* use the [#questions channel](https://slack.kedro.org/) on our Slack channel to ask the community for help
* search the [searchable archive of Slack discussions](https://linen-slack.kedro.org/)
* search the [online archive of Slack discussions](https://linen-slack.kedro.org/)

To exit the visualisation, close the browser tab. To regain control of the terminal, enter `^+c` on Mac or `Ctrl+c` on Windows or Linux machines.

Expand All @@ -62,7 +61,7 @@ The `autoreload` flag reflects changes to the project as they happen. For exampl

## Visualise layers

By convention, a [pipeline can be defined as having different layers](https://docs.kedro.org/en/stable/resources/glossary.html#layers-data-engineering-convention) according to how data is processed, which makes it easier to collaborate.
By convention, a [pipeline can be defined as having different layers](https://docs.kedro.org/en/stable/resources/glossary.html#layers-data-engineering-convention) according to how data is processed.

For example, the [data engineering convention](https://towardsdatascience.com/the-importance-of-layered-thinking-in-data-engineering-a09f685edc71) labels datasets according to the stage of the pipeline (e.g. whether the data has been cleaned).

Expand Down Expand Up @@ -151,7 +150,7 @@ The visualisation now includes the layers:

## Share a pipeline visualisation

You can share a the pipeline structure within a Kedro-Viz visualisation as a JSON file from the terminal:
You can share a pipeline structure within a Kedro-Viz visualisation as a JSON file from the terminal:

```bash
kedro viz --save-file=my_shareable_pipeline.json
Expand All @@ -164,3 +163,5 @@ To visualise the shared file, type the following to load it from the terminal:
```bash
kedro viz --load-file=my_shareable_pipeline.json
```

You can also share a complete project visualisation, described in more detail on [the following page](./share_kedro_viz).
3 changes: 3 additions & 0 deletions docs/source/kedro_logo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 2 additions & 4 deletions docs/source/preview_datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,9 @@

This page describes how to preview data from different datasets in a Kedro project with Kedro-Viz. Dataset preview was introduced in Kedro-Viz version 6.3.0, which offers preview for `CSVDatasets` and `ExcelDatasets`.

We use the [spaceflights tutorial](https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html) to demonstrate how to add data preview for the `customer`, `shuttle` and `reviews` datasets. Even if you have not yet worked through the tutorial, you can still follow this example; you'll need to use the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights) to generate a copy of the project with working code in place.
We use the {doc}`spaceflights tutorial<kedro:tutorial/spaceflights_tutorial>` to demonstrate how to add data preview for the `customer`, `shuttle` and `reviews` datasets. Even if you have not yet worked through the tutorial, you can still follow this example; you'll need to use the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights) to generate a copy of the project with working code in place.
stichbury marked this conversation as resolved.
Show resolved Hide resolved

If you haven't installed Kedro [follow the documentation to get set up](https://docs.kedro.org/en/stable/get_started/install.html).

Then, in your terminal window, navigate to the folder you want to store the project.
If you haven't installed Kedro {doc}`follow the documentation to get set up<kedro:get_started/install>`. Then, in your terminal window, navigate to the folder you want to store the project.

Generate a copy of the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights):
stichbury marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
33 changes: 31 additions & 2 deletions docs/source/share_kedro_viz.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
Kedro-Viz sharing was introduced in version 6.6.0.
```

This page describes how to publish Kedro-Viz so you can share it with others. It uses the [spaceflights tutorial](https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html) as an example.
This page describes how to publish Kedro-Viz so you can share it with others. It uses the {doc}`spaceflights tutorial<kedro:tutorial/spaceflights_tutorial>` as an example.

If you haven't installed Kedro [follow the documentation to get set up](https://docs.kedro.org/en/stable/get_started/install.html). In your terminal window, navigate to the folder you want to store the project.
If you haven't installed Kedro {doc}`follow the documentation to get set up<kedro:get_started/install>`. In your terminal window, navigate to the folder you want to store the project.

If you have not yet worked through the tutorial, use the [Kedro starter for spaceflights](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights) to generate the project with working code in place. Type the following in your terminal:
stichbury marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -39,6 +39,29 @@ pip install -r src/requirements.txt

You can host your Kedro-Viz project on Amazon S3. You must first create an S3 bucket and then enable static website hosting. To do so, follow the [AWS tutorial](https://docs.aws.amazon.com/AmazonS3/latest/userguide/HostingWebsiteOnS3Setup.html) to configure a static website on Amazon S3.


Once the S3 bucket is created, you'll need to create an Identity and Access Management (IAM) user account, user group, and generate the corresponding access keys. To do so:

Sign in to the [AWS Management Console](https://console.aws.amazon.com/s3/) and create an IAM user account.
For more information, see the official AWS documentation about [IAM Identities](https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html).

![](./images/kedro_viz_share_credentials1.png)

Create a user group from the IAM dashboard, ensuring the user group has full access to the AWS S3 policy.

![](./images/kedro_viz_share_credentials2.png)

For more information, see the official AWS documentation about [IAM user groups](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html?icmpid=docs_iam_help_panel).

Add the IAM user to the user group (this is only possible if the group has been created).

![](./images/kedro_viz_share_credentials3.png)

Select the user, then select `Create access key`. Follow the steps and create your keys.

![](./images/kedro_viz_share_credentials4.png)


Once that's completed, you'll need to set your AWS credentials as environment variables in your terminal window, as shown below:

```bash
Expand Down Expand Up @@ -69,3 +92,9 @@ Here's an example of the flow:
All permissions and access control are controlled by AWS. It's up to you, the user, if you want to allow anyone to see your project or limit access to certain IP addresses, users, or groups.

You can control who can view your visualisation using [bucket and user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-iam-policies.html) or [access control lists](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acls.html). See the official AWS documentation for more information.

## Billing

You pay for storing objects in your S3 buckets. The amount you pay depends on your objects’ size, how long you stored the object during the month, and the storage class.

See the official [AWS documentation](https://aws.amazon.com/s3/pricing/?nc=sn&loc=4) for more information.
Loading