Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
doncamilom committed Feb 29, 2024
1 parent 067f65f commit 20f6fa9
Showing 1 changed file with 27 additions and 7 deletions.
34 changes: 27 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,24 +59,44 @@ chem_model.run("What is the molecular weight of tylenol?")

### 💻 Running using local LLMs.

ChemCrow also supports the use of local LLMs, powered by GPT4All, which can be run on a laptop.
ChemCrow also supports the use of local LLMs, through either GPT4All, or HuggingFace's [TGI](https://huggingface.co/docs/text-generation-inference/index).

A list of supported models is provided [here](https://gpt4all.io/index.html).
#### GPT4All

To run using GPT4All, you will need to download one of the [supported models](https://gpt4all.io/index.html).

```python
from chemcrow.agents import ChemCrow

chem_model = ChemCrow(
model_type='gpt4all',
model="./models/mistral-7b-instruct-v0.1.Q4_0.gguf",
tools_model="./models/mistral-7b-instruct-v0.1.Q4_0.gguf",
temp=0.1, verbose=False, max_tokens=100, n_ctx=2048
temp=0.1,
max_tokens=100,
verbose=False,
)
output = chem_model.run("What is the molecular weight of tylenol?")
```

#### TGI

>>> output
>>> The molecular weight of acetaminophen is 151.17 g/mol ...
The other option is Text Generation Interface. This allows you to serve a model and run inference as an API.
To deploy a model, you will need docker. Run it as explained [here](https://huggingface.co/docs/text-generation-inference/quicktour).

```python
from chemcrow.agents import ChemCrow

agent = ChemCrow(
model_type='tgi',
model_server_url='http://server-ip-address:8080',
temp=0.3,
max_tokens=40,
max_iterations=3,
).agent_executor
```

The advantage of TGI is improved efficiency, plus easy access to any model available in HuggingFace.


## ✅ Citation
Bran, Andres M., et al. "ChemCrow: Augmenting large-language models with chemistry tools." arXiv preprint arXiv:2304.05376 (2023).

Expand Down

0 comments on commit 20f6fa9

Please sign in to comment.