Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redis persistence store not working #520

Closed
1 of 2 tasks
cheesecake100201 opened this issue Nov 25, 2024 · 3 comments
Closed
1 of 2 tasks

Redis persistence store not working #520

cheesecake100201 opened this issue Nov 25, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@cheesecake100201
Copy link
Contributor

System Info

Collecting environment information...
PyTorch version: 2.2.2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.6.6 (x86_64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.14 (main, May 6 2024, 14:47:20) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] onnxruntime==1.19.2
[pip3] torch==2.2.2
[conda] numpy 1.26.4 pypi_0 pypi
[conda] torch 2.2.2 pypi_0 pypi

Information

  • The official example scripts
  • My own modified scripts

🐛 Describe the bug

When you execute a turn after creating agent_id and session_id, the error comes as zrangebylex method throws an error with the error being that 1 argument is missing. This is happening because we are not using sorted_sets, we are just setting the values using keys. Because of that we do not have a name for the sorted set, so its causing an error.

Error logs

INFO:     ::1:58967 - "POST /alpha/agents/turn/create HTTP/1.1" 200 OK
Traceback (most recent call last):
  File "/Users/sarthakdeshpande/Desktop/untitled folder 2/llama-stack/llama_stack/distribution/server/server.py", line 158, in sse_generator
    async for item in event_gen:
  File "/Users/sarthakdeshpande/Desktop/untitled folder 2/llama-stack/llama_stack/providers/inline/agents/meta_reference/agents.py", line 138, in _create_agent_turn_streaming
    async for event in agent.create_and_execute_turn(request):
  File "/Users/sarthakdeshpande/Desktop/untitled folder 2/llama-stack/llama_stack/providers/inline/agents/meta_reference/agent_instance.py", line 157, in create_and_execute_turn
    turns = await self.storage.get_session_turns(request.session_id)
  File "/Users/sarthakdeshpande/Desktop/untitled folder 2/llama-stack/llama_stack/providers/inline/agents/meta_reference/persistence.py", line 73, in get_session_turns
    values = await self.kvstore.range(
  File "/Users/sarthakdeshpande/Desktop/untitled folder 2/llama-stack/llama_stack/providers/utils/kvstore/redis/redis.py", line 52, in range
    return await self.redis.zrangebylex(start_key, end_key)
TypeError: SortedSetCommands.zrangebylex() missing 1 required positional argument: 'max'

Expected behavior

No error, just turn should be created

@cheesecake100201
Copy link
Contributor Author

PR #521

@aidando73
Copy link
Contributor

Yeah I'm running into this error as well

@yanxi0830 yanxi0830 added the bug Something isn't working label Dec 10, 2024
dineshyv pushed a commit that referenced this issue Jan 12, 2025
# What does this PR do?

In short, provide a summary of what this PR does and why. Usually, the
relevant context should be present in a linked issue.

- [Currently redis as a kvstore is bugged, as the range method uses
zrangebylex method. zrangebylex method is used when it is a sorted set
but we are storing the value using .set method in the redis. This causes
an error. Another issue is that zrangebylex method takes 3 args but only
2 are mentioned in the range method. This causes a runtime error. That
method has been replaced with the current implementation in the PR ]
Addresses issue (#520 )


## Test Plan

Please describe:
 - tests you ran to verify your changes with result summaries.
 - provide instructions so it can be reproduced.
`python llama_stack/apis/agents/client.py localhost 8001 tools_llama_3_1
meta-llama/Llama-3.1-70B-Instruct`
<img width="1711" alt="Screenshot 2024-11-25 at 2 59 55 PM"
src="https://github.com/user-attachments/assets/c2551555-bc73-4427-b09b-c86d6deb2956">
<img width="634" alt="Screenshot 2024-11-25 at 3 00 33 PM"
src="https://github.com/user-attachments/assets/a087718f-fc2a-424b-b096-4ecad08a07bf">

Have used redis in the run.yaml file as well for the persistence_store.
Also enable_session_persistence turned to True for this test.
Have also tested this in a jupyter notebook to make sure the current
flow does not work through multiple turns in the same session.

## Sources

Please link relevant resources if necessary.


## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the
other checks if that's the case).
- [x] Ran pre-commit to handle lint / formatting issues.
- [x] Read the [contributor
guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md),
      Pull Request section?
- [ ] Updated relevant documentation.
- [ ] Wrote necessary unit or integration tests.
@hardikjshah
Copy link
Contributor

closing this issue since the PR is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants