Skip to content

Commit

Permalink
Add [End] to parser cleanup tags
Browse files Browse the repository at this point in the history
Signed-off-by: Abhishek Bhandwaldar <[email protected]>
  • Loading branch information
abhi1092 authored Nov 18, 2024
1 parent 09cdd0a commit 6a88422
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/instructlab/sdg/pipelines/full/knowledge.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ blocks:
parsing_pattern: '\[(?:Question|QUESTION)\]\s*(.*?)\s*\[(?:Answer|ANSWER)\]\s*(.*?)\s*(?=\[(?:Question|QUESTION)\]|$)'
parser_cleanup_tags:
- "[END]"
- "[End]"
gen_kwargs:
max_tokens: 2048
drop_duplicates:
Expand Down

0 comments on commit 6a88422

Please sign in to comment.