Skip to content

Commit

Permalink
docs: sync wandb
Browse files Browse the repository at this point in the history
  • Loading branch information
Zeyi-Lin committed Jan 17, 2025
1 parent a033579 commit 910a97f
Show file tree
Hide file tree
Showing 5 changed files with 163 additions and 25 deletions.
Binary file added assets/ig-wandb-4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion en/guide_cloud/general/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Github: https://github.com/SwanHubX/SwanLab
## v0.4.3 - 2025.1.17

**🚀 New Features**
- Added `swanlab.sync_wandb()`: Supports synchronizing metrics to SwanLab when using Weights&Biases for experiment tracking.
- Added `swanlab.sync_wandb()`: Supports synchronizing metrics to SwanLab when using Weights&Biases for experiment tracking. [Docs](/en/guide_cloud/integration/integration-wandb.md)
- Added framework integration: Configuration items will now record the framework being used.

**Optimizations**
Expand Down
106 changes: 87 additions & 19 deletions en/guide_cloud/integration/integration-wandb.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,114 @@
# Weights & Biases

[Weights & Biases](https://github.com/wandb/wandb) (Wandb) is an experiment tracking, model optimization, and collaboration platform for machine learning and deep learning projects. W&B provides powerful tools to log and visualize experiment results, helping data scientists and researchers better manage and share their work.
[Weights & Biases](https://github.com/wandb/wandb) (Wandb) is a platform for experiment tracking, model optimization, and collaboration in machine learning and deep learning projects. W&B provides powerful tools for recording and visualizing experimental results, helping data scientists and researchers better manage and share their work.

![wandb](/assets/ig-wandb.png)

You can use `swanlab convert` to convert existing projects on Wandb into SwanLab projects.
**You can synchronize projects from Wandb to SwanLab in two ways:**

1. **Synchronized Tracking**: If your current project uses wandb for experiment tracking, you can use the `swanlab.sync_wandb()` command to synchronize metrics to SwanLab while running the training script.
2. **Convert Existing Projects**: If you want to copy a project from wandb to SwanLab, you can use `swanlab convert` to convert an existing project on Wandb to a SwanLab project.

::: info
In the current version, only scalar charts are supported for conversion.
The current version only supports converting scalar charts.
:::

## Find Your Project, Entity, and Run ID
[[toc]]

## 1. Synchronized Tracking

### 1.1 Add the `sync_wandb` Command

Add the `swanlab.sync_wandb()` command anywhere before `wandb.init()` in your code to synchronize wandb metrics to SwanLab during training.

```python
import swanlab

swanlab.sync_wandb()

...

wandb.init()
```

In the above code, `wandb.init()` will simultaneously initialize swanlab, with the project name, experiment name, and configuration matching the `project`, `name`, and `config` in `wandb.init()`. Therefore, you do not need to manually initialize swanlab.

### 1.2 Alternative Approach

Another approach is to manually initialize swanlab first, then run the wandb code.

```python
import swanlab

swanlab.init(...)
swanlab.sync_wandb()

...

wandb.init()
```

In this approach, the project name, experiment name, and configuration will match the `project`, `experiment_name`, and `config` in `swanlab.init()`. The `project` and `name` in the subsequent `wandb.init()` will be ignored, and the `config` will be updated in `swanlab.config`.

### 1.3 Test Code

The `project`, `entity`, and `runid` are required for the conversion (runid is optional).
Location of `project` and `entity`:
```python
import wandb
import random
import swanlab

swanlab.sync_wandb()
# swanlab.init(project="sync_wandb")

wandb.init(
project="test",
config={"a": 1, "b": 2},
name="test",
)

epochs = 10
offset = random.random() / 5
for epoch in range(2, epochs):
acc = 1 - 2 ** -epoch - random.random() / epoch - offset
loss = 2 ** -epoch + random.random() / epoch + offset

wandb.log({"acc": acc, "loss": loss})
```

![alt text](/assets/ig-wandb-4.png)

## 2. Convert Existing Projects

### 2.1 Locate Your Project, Entity, and Run ID on wandb.ai

The project, entity, and run ID are required for conversion (run ID is optional).
The location of the project and entity:
![alt text](/assets/ig-wandb-2.png)

Location of `runid`:
The location of the run ID:

![alt text](/assets/ig-wandb-3.png)

## Method 1: Command Line Conversion
### 2.2 Method 1: Command Line Conversion

First, ensure that you are logged into Wandb in the current environment and have access to the target project.
First, ensure that you are logged into wandb in the current environment and have access to the target project.

Conversion command line:
Conversion command:

```bash
swanlab convert -t wandb --wb-project [WANDB_PROJECT_NAME] --wb-entity [WANDB_ENTITY]
```

Supported parameters are as follows:
Supported parameters:

- `-t`: Conversion type, options include `wandb` and `tensorboard`.
- `--wb-project`: The name of the Wandb project to be converted.
- `--wb-entity`: The space name where the Wandb project is located.
- `--wb-runid`: The ID of the Wandb Run (an experiment under the project).
- `-t`: Conversion type, options are wandb and tensorboard.
- `--wb-project`: The name of the wandb project to be converted.
- `--wb-entity`: The space name where the wandb project is located.
- `--wb-runid`: The ID of the wandb Run (a specific experiment under the project).

If `--wb-runid` is not filled in, all Runs under the specified project will be converted; if filled in, only the specified Run will be converted.
If `--wb-runid` is not provided, all Runs under the specified project will be converted; if provided, only the specified Run will be converted.

## Method 2: Conversion Within Code
### 2.3 Method 2: Conversion Within Code

```python
from swanlab.converter import WandbConverter
Expand All @@ -48,5 +117,4 @@ wb_converter = WandbConverter()
# wb_runid is optional
wb_converter.run(wb_project="WANDB_PROJECT_NAME", wb_entity="WANDB_USERNAME")
```

This method achieves the same effect as the command line conversion.
The effect is the same as command line conversion.
2 changes: 1 addition & 1 deletion zh/guide_cloud/general/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Github: https://github.com/SwanHubX/SwanLab
## v0.4.3 - 2025.1.17

**🚀新增功能**
- 新增`swanlab.sync_wandb()`:支持使用Weights&Biases跟踪实验时,同步指标到SwanLab
- 新增`swanlab.sync_wandb()`:支持使用Weights&Biases跟踪实验时,同步指标到SwanLab[文档](/guide_cloud/integration/integration-wandb.md)
- 新增在使用框架集成时,配置项将记录所使用的框架

**优化**
Expand Down
78 changes: 74 additions & 4 deletions zh/guide_cloud/integration/integration-wandb.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,83 @@

![wandb](/assets/ig-wandb.png)

你可以使用swanlab convert将Wandb上已存在的项目转换成SwanLab项目。
**你可以用两种方式将Wandb上的项目同步到SwanLab:**

1. **同步跟踪**:如果你现在的项目使用了wandb进行实验跟踪,你可以使用`swanlab.sync_wandb()`命令,在运行训练脚本时同步记录指标到SwanLab。
2. **转换已存在的项目**:如果你想要将wandb上的项目复制到SwanLab,你可以使用`swanlab convert`,将Wandb上已存在的项目转换成SwanLab项目。

::: info
在当前版本暂仅支持转换标量图表。
:::

## 找到你的projecy、entity和runid
[[toc]]


## 1. 同步跟踪

### 1.1 添加sync_wandb命令

在你的代码执行`wandb.init()`之前的任何位置,添加一行`swanlab.sync()`命令,即可在训练时同步wandb的指标到SwanLab。

```python
import swanlab

swanlab.sync_wandb()

...

wandb.init()
```

在上述这种代码写法中,`wandb.init()`的同时会初始化swanlab,项目名、实验名和配置和`wandb.init()`中的`project``name``config`一致,因此你不需要再手动初始化swanlab。

### 1.2 另一种写法

另一种用法是先手动初始化swanlab,再运行wandb的代码。

```python
import swanlab

swanlab.init(...)
swanlab.sync_wandb()

...

wandb.init()
```

在这种写法中,项目名、实验名、配置和`swanlab.init()`中的`project``experiment_name``config`一致,而后续`wandb.init()`中的`project``name`会被忽略,`config`会更新进`swanlab.config`中。

### 1.3 测试代码

```python
import wandb
import random
import swanlab

swanlab.sync_wandb()
# swanlab.init(project="sync_wandb")

wandb.init(
project="test",
config={"a": 1, "b": 2},
name="test",
)

epochs = 10
offset = random.random() / 5
for epoch in range(2, epochs):
acc = 1 - 2 ** -epoch - random.random() / epoch - offset
loss = 2 ** -epoch + random.random() / epoch + offset

wandb.log({"acc": acc, "loss": loss})
```

![alt text](/assets/ig-wandb-4.png)

## 2. 转换已存在的项目

### 2.1 找到你在wandb.ai上的projecy、entity和runid

projecy、entity和runid是转换所需要的(runid是可选的)。
project和entity的位置:
Expand All @@ -20,7 +90,7 @@ runid的位置:

![alt text](/assets/ig-wandb-3.png)

## 方式一:命令行转换
### 2.2 方式一:命令行转换

首先,需要确保当前环境下,你已登录了wandb,并有权限访问目标项目。

Expand All @@ -39,7 +109,7 @@ swanlab convert -t wandb --wb-project [WANDB_PROJECT_NAME] --wb-entity [WANDB_EN

如果不填写`--wb-runid`,则会将指定项目下的全部Run进行转换;如果填写,则只转换指定的Run。

## 方式二:代码内转换
### 2.3 方式二:代码内转换

```python
from swanlab.converter import WandbConverter
Expand Down

0 comments on commit 910a97f

Please sign in to comment.