Skip to content

Commit

Permalink
DOC Small fixes for HQQ and section title (#1986)
Browse files Browse the repository at this point in the history
Changed:

- Helper section had placeholder title
- `device` is not a valid argument to `from_pretrained`
- Excess empty lines
- Move helpers section
  • Loading branch information
BenjaminBossan authored Aug 2, 2024
1 parent 46f7897 commit 9a08782
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 12 deletions.
7 changes: 3 additions & 4 deletions docs/source/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,6 @@
title: Troubleshooting
- local: developer_guides/checkpoint
title: PEFT checkpoint format
- local: package_reference/helpers
title: Helpers

- title: 🤗 Accelerate integrations
sections:
Expand Down Expand Up @@ -114,11 +112,12 @@
title: VeRA
- local: package_reference/fourierft
title: FourierFT

title: Adapters
- sections:
- local: package_reference/merge_utils
title: Model merge
- local: package_reference/helpers
title: Helpers
title: Utilities
title: API reference

9 changes: 2 additions & 7 deletions docs/source/developer_guides/quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,13 +168,11 @@ model = get_peft_model(model, config)

The models that is quantized using Half-Quadratic Quantization of Large Machine Learning Models ([HQQ](https://mobiusml.github.io/hqq_blog/)) support LoRA adapter tuning. To tune the quantized model, you'll need to install the `hqq` library with: `pip install hqq`.

```py
```python
from hqq.engine.hf import HQQModelForCausalLM

quantized_model = HQQModelForCausalLM.from_quantized(save_dir_or_hfhub, device='cuda')

peft_config = LoraConfig(...)

quantized_model = get_peft_model(quantized_model, peft_config)
```

Expand All @@ -184,11 +182,8 @@ Or using transformers version that is compatible with HQQ (e.g. by installing it
from transformers import HqqConfig, AutoModelForCausalLM

quant_config = HqqConfig(nbits=4, group_size=64)

quantized_model = AutoModelForCausalLM.from_pretrained(save_dir_or_hfhub, device='cuda', quantization_config=quant_config)

quantized_model = AutoModelForCausalLM.from_pretrained(save_dir_or_hfhub, device_map=device_map, quantization_config=quant_config)
peft_config = LoraConfig(...)

quantized_model = get_peft_model(quantized_model, peft_config)
```

Expand Down
2 changes: 1 addition & 1 deletion docs/source/package_reference/helpers.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
rendered properly in your Markdown viewer.
-->

# Document Title
# Helper methods

A collection of helper functions for PEFT.

Expand Down

0 comments on commit 9a08782

Please sign in to comment.