Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot "Download prompt and parameters" for DeepSeek-R1-Distill-Qwen-32B #1667

Open
evalstate opened this issue Jan 26, 2025 · 0 comments
Open
Labels
bug Something isn't working

Comments

@evalstate
Copy link
Contributor

Bug description

500 Server Error raised when navigating to /conversation/[id]/message/[id]/prompt for DeepSeek-R1-Distill-Qwen-32B Chats.

Steps to reproduce

Use the "Download prompt and parameters button" on a message in a DS-R1 conversation.

Screenshots

Context

Happens on chatui, reproduced locally.

Logs

The issue arises from calling the @huggingface/jinja constructor with the below template.

 error: {
      "stack":
          Error: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.
              at expect (..../node_modules/@huggingface/transformers/dist/transformers.mjs:537:13)
              at parseJinjaStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:569:9)
              at parseAny (...../node_modules/@huggingface/transformers/dist/transformers.mjs:546:16)
              at parseIfStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:618:17)
              at parseJinjaStatement (...../node_modules/@huggingface/transformers/dist/transformers.mjs:573:18)

Specs

Config

Notes

This is the chat template returned that causes the error:

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<|User|>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<|Assistant|><|tool▁calls▁begin|><|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<|tool▁call▁begin|>' + tool['type'] + '<|tool▁sep|>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<|tool▁call▁end|>'}}{{'<|tool▁calls▁end|><|end▁of▁sentence|>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<|tool▁outputs▁end|>' + message['content'] + '<|end▁of▁sentence|>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<|Assistant|>' + content + '<|end▁of▁sentence|>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<|tool▁outputs▁begin|><|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<|tool▁output▁begin|>' + message['content'] + '<|tool▁output▁end|>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<|tool▁outputs▁end|>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<|Assistant|>'}}{% endif %}
@evalstate evalstate added the bug Something isn't working label Jan 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant