Issue about custom LLM. The example file dosn't work! #15

Buzeg · 2024-11-14T12:29:59Z

When I use the official example to use my own LLM, such as GLM-4-Plus, it prompts an error

liukidar · 2024-11-15T01:52:57Z

Hello there! os.getenv() is a function, so you should do model=os.getenv("...") and similar for the other parameters. Let me know if that fixes the issue.

Buzeg · 2024-11-15T06:36:42Z

Hello there! os.getenv() is a function, so you should do model=os.getenv("...") and similar for the other parameters. Let me know if that fixes the issue.

Hello! Thanks for your reply! But is remind me OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
I set a .env file
os.environ["embed_api_key"] = "****************************" os.environ["OPENAI_API_KEY"] = "*************************" os.environ["base_url"] = "https://open.bigmodel.cn/api/paas/v4" os.environ["llm_model"] = "glm-4-flash" os.environ["embed_model"] = "embedding-3"

And load_dotenv() return TRUE
How can I fix this problem! Really appreciate!
@liukidar

lawcompany-SH · 2024-11-16T12:52:49Z

Set .env file with below format. see https://pypi.org/project/python-dotenv/

embed_api_key=*******
OPENAI_API_KEY=******
base_url=https://open.bigmodel.cn/api/paas/v4

Buzeg · 2024-11-17T12:23:09Z

Set .env file with below format. see https://pypi.org/project/python-dotenv/
embed_api_key=*******
OPENAI_API_KEY=******
base_url=https://open.bigmodel.cn/api/paas/v4

It still return error.
.env file

embed_api_key = ***********************
OPENAI_API_KEY = ************************
base_url = https://open.bigmodel.cn/api/paas/v4
llm_model = glm-4-plus
embed_model = embedding-3

And

working_dir = r"D:\Implement\CodeData\fast-graphrag\fast"

grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(
            model=os.getenv("llm_model"),
            base_url=os.getenv("base_url"),
            api_key=os.getenv("OPENAI_API_KEY")
        ),
        embedding_service=OpenAIEmbeddingService(
            model=os.getenv("embed_model"),
            base_url=os.getenv("base_url"),
            api_key=os.getenv("embed_api_key"),
            embedding_dim=512
        )
    )
)

return

{
	"name": "OpenAIError",
	"message": "The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable",
	"stack": "---------------------------------------------------------------------------
OpenAIError                               Traceback (most recent call last)
Cell In[7], line 9
      1 working_dir = r\"D:\\Implement\\CodeData\\fast-graphrag\\fast\"
      3 grag = GraphRAG(
      4     working_dir=working_dir,
      5     domain=DOMAIN,
      6     example_queries=\"\
\".join(QUERIES),
      7     entity_types=ENTITY_TYPES,
      8     config=GraphRAG.Config(
----> 9         llm_service=OpenAILLMService(
     10             model=os.getenv(\"llm_model\"),
     11             base_url=os.getenv(\"base_url\"),
     12             api_key=os.getenv(\"OPENAI_API_KEY\")
     13         ),
     14         embedding_service=OpenAIEmbeddingService(
     15             model=os.getenv(\"embed_model\"),
     16             base_url=os.getenv(\"base_url\"),
     17             api_key=os.getenv(\"embed_api_key\"),
     18             embedding_dim=512
     19         )
     20     )
     21 )

File <string>:6, in __init__(self, model, base_url, api_key)

File d:\\Implement\\Anaconda3\\envs\\fastgr\\Lib\\site-packages\\fast_graphrag\\_llm\\_llm_openai.py:34, in OpenAILLMService.__post_init__(self)
     32 def __post_init__(self):
     33     logger.debug(\"Initialized OpenAILLMService with patched OpenAI client.\")
---> 34     self.llm_async_client: instructor.AsyncInstructor = instructor.from_openai(AsyncOpenAI(api_key=self.api_key))

File d:\\Implement\\Anaconda3\\envs\\fastgr\\Lib\\site-packages\\openai\\_client.py:319, in AsyncOpenAI.__init__(self, api_key, organization, project, base_url, timeout, max_retries, default_headers, default_query, http_client, _strict_response_validation)
    317     api_key = os.environ.get(\"OPENAI_API_KEY\")
    318 if api_key is None:
--> 319     raise OpenAIError(
    320         \"The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable\"
    321     )
    322 self.api_key = api_key
    324 if organization is None:

OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable"
}

@ @lawcompany-SH

liukidar · 2024-11-17T19:24:38Z

Hello, I am unable to replicate your issue, would you mind sharing the full code you're using? you can use generic values for api_keys as real ones are not necessary to initialise the model.

tmceld · 2024-11-18T20:39:25Z

I think i am hitting the same problem, with Ollama:

from typing import List

from dotenv import load_dotenv

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAIEmbeddingService, OpenAILLMService

load_dotenv()

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of shoes in Caledonian Road?",
    "How does the setting of London contribute to the story's themes?",
    "Describe the chain of events that leads to Bykov's demise.",
    "What does Capmbell's Mother represent in this story?",
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

api_key = "ollama"
working_dir = "./examples/"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(
            model="llama3.2:latest",
            base_url="http://localhost:11434/v1",
            api_key=api_key,
        ),
        embedding_service=OpenAIEmbeddingService(
            model="nomic-embed-text",
            base_url="http://localhost:11434/api/embeddings/",
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./Caledonian Road_ From the award-winning au - Andrew O'Hagan.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Campbell?").response)

Giving error:

 python test.py
Traceback (most recent call last):
  File "/Users/toast/Developer/ai/fast-graphrag/test.py", line 28, in <module>
    config=GraphRAG.Config(
           ^^^^^^^^^^^^^^^^
  File "<string>", line 10, in __init__
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/fast_graphrag/__init__.py", line 69, in <lambda>
    DefaultVectorStorageConfig(embedding_dim=DefaultEmbeddingService().embedding_dim)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 8, in __init__
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 115, in __post_init__
    self.embedding_async_client: AsyncOpenAI = AsyncOpenAI(api_key=self.api_key)
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/openai/_client.py", line 319, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

tmceld · 2024-11-18T21:16:16Z

So i got around the above error, by exporting an OPENAI_API_KEY but not sure why i needed to do this.

I now have further errors with embedding, and am wondering given the status of embedding on ollama - has anyone actually got this to work?

liukidar · 2024-11-18T22:50:18Z

I think i am hitting the same problem, with Ollama:

from typing import List

from dotenv import load_dotenv

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAIEmbeddingService, OpenAILLMService

load_dotenv()

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of shoes in Caledonian Road?",
    "How does the setting of London contribute to the story's themes?",
    "Describe the chain of events that leads to Bykov's demise.",
    "What does Capmbell's Mother represent in this story?",
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

api_key = "ollama"
working_dir = "./examples/"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(
            model="llama3.2:latest",
            base_url="http://localhost:11434/v1",
            api_key=api_key,
        ),
        embedding_service=OpenAIEmbeddingService(
            model="nomic-embed-text",
            base_url="http://localhost:11434/api/embeddings/",
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./Caledonian Road_ From the award-winning au - Andrew O'Hagan.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Campbell?").response)

Giving error:

 python test.py
Traceback (most recent call last):
  File "/Users/toast/Developer/ai/fast-graphrag/test.py", line 28, in <module>
    config=GraphRAG.Config(
           ^^^^^^^^^^^^^^^^
  File "<string>", line 10, in __init__
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/fast_graphrag/__init__.py", line 69, in <lambda>
    DefaultVectorStorageConfig(embedding_dim=DefaultEmbeddingService().embedding_dim)
                                             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 8, in __init__
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 115, in __post_init__
    self.embedding_async_client: AsyncOpenAI = AsyncOpenAI(api_key=self.api_key)
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/toast/Developer/ai/fast-graphrag/.venv/lib/python3.11/site-packages/openai/_client.py", line 319, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

Just to be sure, are you using the pip version or this cloned repo? It may be a bug that we fixed but didn't push to pypi.

liukidar · 2024-11-18T22:51:29Z

So i got around the above error, by exporting an OPENAI_API_KEY but not sure why i needed to do this.

I now have further errors with embedding, and am wondering given the status of embedding on ollama - has anyone actually got this to work?

Mmmh, looking here it seems it is supported ollama/ollama#2416 ? But indeed we should clarify this better.
In the post they also suggest to look at this: https://github.com/severian42/GraphRAG-Local-UI/blob/main/embedding_proxy.py

tmceld · 2024-11-19T08:48:29Z

Just to be sure, are you using the pip version or this cloned repo? It may be a bug that we fixed but didn't push to pypi.

Ah, yes i am using the pip installed version

liukidar · 2024-11-19T16:37:39Z

Let me know if using the repo directly fixes the problem

jon-torres · 2024-11-22T12:58:21Z

I am experiencing a similar issue with Azure. But it's throwing resource not found. I have installed fast-graphrag with pip.

Linux mint 22
python 3.11

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

model = "gpt-4o"
base_url = "https://<resource>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxxxxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=model,
            base_url=base_url,
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.48s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x715107d72c50 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 47, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

liukidar · 2024-11-22T17:59:24Z

I am experiencing a similar issue with Azure. But it's throwing resource not found. I have installed fast-graphrag with pip.

Linux mint 22 python 3.11

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

model = "gpt-4o"
base_url = "https://<resource>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxxxxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=model,
            base_url=base_url,
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.48s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x715107d72c50 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 47, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

From your example it looks like you're using the same model and base_url for both LLM and Embedder, I'm not sure that's supported by Azure, I would doulbe check that. Specifically, the 404 is telling you that the LLM base_url is invalid.

jon-torres · 2024-11-22T18:15:54Z

I am experiencing a similar issue with Azure. But it's throwing resource not found. I have installed fast-graphrag with pip.
Linux mint 22 python 3.11

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

model = "gpt-4o"
base_url = "https://<resource>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxxxxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=model,
            base_url=base_url,
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.48s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x715107d72c50 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 47, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

From your example it looks like you're using the same model and base_url for both LLM and Embedder, I'm not sure that's supported by Azure, I would doulbe check that. Specifically, the 404 is telling you that the LLM base_url is invalid.

Hey, I appreciate the answer. Is the OpenAIEmbeddingService needed? I was trying to replicate the example in the main README.md but using this custom.py because from what I understand it's needed to explicit pass the endpoint API if you are not using openAI directly, right?

liukidar · 2024-11-22T18:17:48Z

Yes, both llm and embedder are necessary, but normally they have different models and urls. I will also clarify this in the example.

jon-torres · 2024-11-22T19:31:24Z

Yes, both llm and embedder are necessary, but normally they have different models and urls. I will also clarify this in the example.

Thank you for making it clear, but the error seems to persist even with a different deploy for the embedder.

code:

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

emb_model = "text-embedding-3-small"
emb_url = "https://<DEPLOYNAME>.openai.azure.com/openai/deployments/text-embedding-3-small/embeddings?api-version=2023-05-15"
emb_key = "5xxxx"

model = "gpt-4o"
base_url = "https://<DEPLOYNAME>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=emb_model,
            base_url=emb_url,
            api_key=emb_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.38s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x73ce66217a10 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 44, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

liukidar · 2024-11-26T17:50:25Z

Hello, for both errors it looks like the url you're using doesn't point to a valid model. Can you try to instantiate the OpenAILLMService and use it directly? it provides a method "send_message".

21m1n · 2024-12-27T07:31:29Z

this is how i run it with ollama:

import instructor 


grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(
            model="llama3", base_url="http://localhost:11434/v1", api_key="ollama", mode=instructor.Mode.JSON
        ),
        embedding_service=OpenAIEmbeddingService(
            model="nomic-embed-text",
            base_url="http://localhost:11434/v1",
            api_key="ollama",
            embedding_dim=768,  
        ),
    ),
)

idreesghazi · 2025-01-06T10:00:26Z

I am experiencing a similar issue with Azure. But it's throwing resource not found. I have installed fast-graphrag with pip.
Linux mint 22 python 3.11

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

model = "gpt-4o"
base_url = "https://<resource>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxxxxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=model,
            base_url=base_url,
            api_key=api_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.48s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x715107d72c50 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 47, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

From your example it looks like you're using the same model and base_url for both LLM and Embedder, I'm not sure that's supported by Azure, I would doulbe check that. Specifically, the 404 is telling you that the LLM base_url is invalid.

Hey, I appreciate the answer. Is the OpenAIEmbeddingService needed? I was trying to replicate the example in the main README.md but using this custom.py because from what I understand it's needed to explicit pass the endpoint API if you are not using openAI directly, right?

While using Azure, you have to exactly specify the base_url of your model

base_url_llm = https://<resource>.openai.azure.com/openai/deployments/gpt-4o

And for the embed model, specify it like

base_url_embed = https://<resource>.openai.azure.com/openai/deployments/text-embedding-3-large

Make sure that these models are correctly deployed on Azure
Also regarding that api_version, currently if you look inside the fast-graph library, it take api_version from the environmental variables, so you must set your environmental variables like this OPENAI_API_VERSION = "your-version"
Hopefully this helps :D

wangzhen38 · 2025-01-09T09:25:43Z

Yes, both llm and embedder are necessary, but normally they have different models and urls. I will also clarify this in the example.

Thank you for making it clear, but the error seems to persist even with a different deploy for the embedder.

code:

from fast_graphrag import GraphRAG
from fast_graphrag._llm import OpenAILLMService, OpenAIEmbeddingService

DOMAIN = "Analyze this story and identify the characters. Focus on how they interact with each other, the locations they explore, and their relationships."

EXAMPLE_QUERIES = [
    "What is the significance of Christmas Eve in A Christmas Carol?",
    "How does the setting of Victorian London contribute to the story's themes?",
    "Describe the chain of events that leads to Scrooge's transformation.",
    "How does Dickens use the different spirits (Past, Present, and Future) to guide Scrooge?",
    "Why does Dickens choose to divide the story into \"staves\" rather than chapters?"
]

ENTITY_TYPES = ["Character", "Animal", "Place", "Object", "Activity", "Event"]

emb_model = "text-embedding-3-small"
emb_url = "https://<DEPLOYNAME>.openai.azure.com/openai/deployments/text-embedding-3-small/embeddings?api-version=2023-05-15"
emb_key = "5xxxx"

model = "gpt-4o"
base_url = "https://<DEPLOYNAME>.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview"
api_key = "5xxxx"

working_dir="./book_example"
grag = GraphRAG(
    working_dir=working_dir,
    domain=DOMAIN,
    example_queries="\n".join(EXAMPLE_QUERIES),
    entity_types=ENTITY_TYPES,
    config=GraphRAG.Config(
        llm_service=OpenAILLMService(model=model, base_url=base_url, api_key=api_key),
         embedding_service=OpenAIEmbeddingService(
            model=emb_model,
            base_url=emb_url,
            api_key=emb_key,
            embedding_dim=512,  # the output embedding dim of the chosen model
        ),
    ),
)

with open("./book.txt") as f:
    grag.insert(f.read())

print(grag.query("Who is Scrooge?").response)

Extracting data:   0%|                                    | 0/1 [00:00<?, ?it/s]Error during information extraction from document: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Extracting data: 100%|████████████████████████████| 1/1 [00:09<00:00,  9.38s/it]
Error during query: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 222, in retry_async
    response: ChatCompletion = await func(*args, **kwargs)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1633, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1838, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1532, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/openai/_base_client.py", line 1633, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 217, in retry_async
    async for attempt in max_retries:
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/tenacity/__init__.py", line 419, in exc_check
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x73ce66217a10 state=finished raised NotFoundError>]

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jontorres/Documents/development/loro/fast_graphrag/graphrag_test.py", line 44, in <module>
    print(grag.query("Who is Scrooge?").response)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 150, in query
    return get_event_loop().run_until_complete(_query())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 146, in _query
    raise e
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 142, in _query
    answer = await self.async_query(query, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_graphrag.py", line 168, in async_query
    extracted_entities = await self.information_extraction_service.extract_entities_from_query(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_services/_information_extraction.py", line 46, in extract_entities_from_query
    entities, _ = await format_and_send_prompt(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_base.py", line 40, in format_and_send_prompt
    return await llm.send_message(prompt=formatted_prompt, response_model=response_model, **args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_utils.py", line 45, in wait_func
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/fast_graphrag/_llm/_llm_openai.py", line 80, in send_message
    llm_response: GTResponseModel = await self.llm_async_client.chat.completions.create(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/client.py", line 387, in create
    return await self.create_fn(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/patch.py", line 161, in new_create_async
    response = await retry_async(
               ^^^^^^^^^^^^^^^^^^
  File "/home/jontorres/anaconda3/envs/loro/lib/python3.11/site-packages/instructor/retry.py", line 248, in retry_async
    raise InstructorRetryException(
instructor.exceptions.InstructorRetryException: Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}

This commit can solve your problem：#62

liukidar mentioned this issue Jan 2, 2025

Using Fast GraphRAG with Ollama and other LLM inference providers #41

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue about custom LLM. The example file dosn't work! #15

Issue about custom LLM. The example file dosn't work! #15

Buzeg commented Nov 14, 2024 •

edited

Loading

liukidar commented Nov 15, 2024

Buzeg commented Nov 15, 2024

lawcompany-SH commented Nov 16, 2024

Buzeg commented Nov 17, 2024 •

edited

Loading

liukidar commented Nov 17, 2024

tmceld commented Nov 18, 2024 •

edited

Loading

tmceld commented Nov 18, 2024

liukidar commented Nov 18, 2024

liukidar commented Nov 18, 2024 •

edited

Loading

tmceld commented Nov 19, 2024

liukidar commented Nov 19, 2024 •

edited

Loading

jon-torres commented Nov 22, 2024

liukidar commented Nov 22, 2024

jon-torres commented Nov 22, 2024

liukidar commented Nov 22, 2024

jon-torres commented Nov 22, 2024

liukidar commented Nov 26, 2024

21m1n commented Dec 27, 2024

idreesghazi commented Jan 6, 2025 •

edited

Loading

wangzhen38 commented Jan 9, 2025

Issue about custom LLM. The example file dosn't work! #15

Issue about custom LLM. The example file dosn't work! #15

Comments

Buzeg commented Nov 14, 2024 • edited Loading

liukidar commented Nov 15, 2024

Buzeg commented Nov 15, 2024

lawcompany-SH commented Nov 16, 2024

Buzeg commented Nov 17, 2024 • edited Loading

liukidar commented Nov 17, 2024

tmceld commented Nov 18, 2024 • edited Loading

tmceld commented Nov 18, 2024

liukidar commented Nov 18, 2024

liukidar commented Nov 18, 2024 • edited Loading

tmceld commented Nov 19, 2024

liukidar commented Nov 19, 2024 • edited Loading

jon-torres commented Nov 22, 2024

liukidar commented Nov 22, 2024

jon-torres commented Nov 22, 2024

liukidar commented Nov 22, 2024

jon-torres commented Nov 22, 2024

liukidar commented Nov 26, 2024

21m1n commented Dec 27, 2024

idreesghazi commented Jan 6, 2025 • edited Loading

wangzhen38 commented Jan 9, 2025

Buzeg commented Nov 14, 2024 •

edited

Loading

Buzeg commented Nov 17, 2024 •

edited

Loading

tmceld commented Nov 18, 2024 •

edited

Loading

liukidar commented Nov 18, 2024 •

edited

Loading

liukidar commented Nov 19, 2024 •

edited

Loading

idreesghazi commented Jan 6, 2025 •

edited

Loading