Add Waii connector

[wangdatan]

Description

This PR adds Waii connector, which allows users to generate SQL queries, get an understanding of their databases, compare/describe queries, and get performance insights for the most expensive queries.

Type of Change

Please delete options that are not relevant.

• New Loader/Tool
• Bug fix / Smaller change
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• [] Added new unit/integration tests
• Added new notebook (that tests end-to-end)
• I stared at the code and made sure it makes sense

Suggested Checklist:

• I have added a library.json file if a new loader/tool was added
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• [] I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I ran make format; make lint to appease the lint gods

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[wangdatan]

@jerryjliu , I just created a patch to use Waii as a tool from Llama. I think be able to ask questions about complex query generation, diff queries, describe dataset, do perf analysis, etc. Which I think it is pretty cool.

Here's notebook examples: https://github.com/wangdatan/llama-hub/blob/main/llama_hub/tools/notebooks/waii.ipynb

Looking forward to hear your feedback about it.

If you or anybody want to test this out yourself, I will be happy to share test account / API-key for you to test.

[EmanuelCampos]

@wangdatan can you fix the tests, please? need to be lazy imports to the tests pass

[wangdatan]

@EmanuelCampos , thank you for the review. I just updated lazy imports.

[jerryjliu]

@wangdatan sorry for the delay on this, will review soon!

[jerryjliu]

this looks great! some comments:

• it might be good to turn verbose=True in the notebook, will be good for users to see what tools are specifically being called for these user queries
• you have a lot of tool functions in the tool spec, which 1) thanks for doing all this, 2) the main risk is more tools tends to confuse agents. is there a subset of this that you think agents must know to operate effectively? e.g. i'm trying to get intuition on when the agent would call fn's like "performance_analyze", "transcode", etc. But you've thought about this more than ihave

[jerryjliu]

oops didn't mean to approve yet, lmk your thoughts

[wangdatan]

@jerryjliu thank you for reviewing.

Both are good suggestions.

For 1. I will update it so the user can see what happens behind-the-scene.

For 2. My intention was, that the Waii LlamaIndex tool is something users can use to finish most of their SQL tasks, no matter if it is query generation, query transcode, or performance analysis. But I think some of the use cases are somehow orthogonal to each other. Maybe we can split it into multiple tools (I'm open to your suggestions):

a. waii: handles:

• Query generation
• Get and summarize query run result
• Describe dataset (Based on the user pattern we saw, users like to do dataset discovery / search together with query generation).
• Users can also use diff queries and describe queries to get a better understanding of existing queries.

I think these use cases are relatively separate from each other. Do you think if LLM (I tried to use GPT-4 as agent, it seems handles these use cases pretty well).

b. waii-perf: handles performance analysis, it is commonly used, but user can load the tool separately.

But since waii-perf is just one single function call, I'm not sure if it is worthwhile to separate it out.

[jerryjliu]

@wangdatan thanks for the detailed response. I think it's ok to keep as one tool spec - actually this may be one us to let users "pick" the functions they want to use with to_tool_list instead of automatically giving all tools to the agent

[jerryjliu]

Let me know when it's ready for review and I can approve!

[wangdatan]

Thank you @jerryjliu . I think it is ready for review. Let me know if I missed anything since this is my first LlamaHub PR 😃

[jerryjliu]

Thank you @jerryjliu . I think it is ready for review. Let me know if I missed anything since this is my first LlamaHub PR 😃

did you update with verbose=True?

[wangdatan]

Oops, I forgot, I will do it tonight

[wangdatan]

@jerryjliu I just added verbose=True to the notebook example.

[wangdatan]

Thank you for reviewing and merging it!