Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2 new API's, and Reasoning model support #423

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

Sweaterdog
Copy link

Adds 2 new API's, a fix for googles latest models, and a fix for Deepseek-R1 support.

GLHF.chat was free, and can host models from huggingface, Hyperbolic Is free and hosts hand-picked models, like Deepseek-R1

Google released Gemini-2.0-flash-thinking, which wasn't supported by the old gemini.js file.

Deepseek-R1 uses specific reasoning tags ( and ) Which needs a specific method so the commands the model is reasoning with does not become part of the response.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Uploaded the changes to mindcraft that allows Gemini thinking models, and glhf.chat and hyperbolic api's

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add Deepseek-R1 Support.

Deepseek-R1 uses "<think> [Thoughts] </think>" and the response can be cut short if it reasons through a command and it's syntax, so we cut it short before returning the response.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@Sweaterdog Sweaterdog marked this pull request as ready for review January 27, 2025 01:01

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added deepseek-r1 support

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added Deepseek-R1 support, such as the qwen2.5 32b distill
@Sweaterdog Sweaterdog changed the title GLHF.chat, and Hyperbolic API's, as well as Gemini-2.0-thinking support, and Deepseek-R1 Ollama support 2 new API's, and Reasoning model support Jan 28, 2025
@Sweaterdog
Copy link
Author

Changes made:
Allowed Deepseek-r1 support through the following providers: Huggingface, Hyperbolic, Groqcloud, and Ollama
Made changes to Gemini.js to allow the new thinking model (The endpoint in node_modules/@google/generative-ai/dist/index.js still needs to be changed from "v1" to "v1beta")

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed small error that would endlessly retry groqcloud response if Deepseek-R1 was chosen
@uukelele-scratch
Copy link
Contributor

I'm getting "SyntaxError: Unexpected token 'else'" (line 60 of prompter.js).

Here are lines 50-81, src/agent/prompter.js.

if (typeof chat === 'string' || chat instanceof String) {
            chat = {model: chat};
            if (chat.model.includes('gemini'))
                chat.api = 'google';
            else if (chat.model.includes('gpt') || chat.model.includes('o1'))
                chat.api = 'openai';
            else if (chat.model.includes('claude'))
                chat.api = 'anthropic';
            else if (chat.model.includes('huggingface/'))
                chat.api = "huggingface";
            else if (chat.model.includes('hf:'))
                chat.api = "glhf";
            else if (chat.model.includes('hyperbolic:')|| chat.model.includes('hb:'))
                chat.api = "hyperbolic";
            else if (chat.model.includes('meta/') || chat.model.includes('mistralai/') || chat.model.includes('replicate/'))
            else if (chat.model.includes('meta/') || chat.model.includes('replicate/')) // why's this here?
                chat.api = 'replicate';
            else if (chat.model.includes('mistralai/') || chat.model.includes("mistral/"))
                chat.api = 'mistral';
            else if (chat.model.includes("groq/") || chat.model.includes("groqcloud/"))
                chat.api = 'groq';
            else if (chat.model.includes('novita/'))
                chat.api = 'novita';
             else if (chat.model.includes('qwen'))
                 chat.api = 'qwen';
            else if (chat.model.includes('grok'))
                chat.api = 'xai';
            else if (chat.model.includes('deepseek'))
                chat.api = 'deepseek';
            else
                chat.api = 'ollama';
        }

The line
else if (chat.model.includes('meta/') || chat.model.includes('replicate/'))
should be removed.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Removed custom logging

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed bug in prompter.js where replicate Ai was checked twice
@MaxRobinsonTheGreat
Copy link
Collaborator

sorry, I just did a refactor to models/prompter and gemini specifically. if you merge with main I will take a look

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Add files via upload

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@Sweaterdog
Copy link
Author

Boom, I fixed all of the merge issues and this should now be up to date!

@uukelele-scratch
Copy link
Contributor

(The endpoint in node_modules/@google/generative-ai/dist/index.js still needs to be changed from "v1" to "v1beta")

In package.json, you can see:

...
"scripts": {
        "postinstall": "patch-package",
        "start": "node main.js"
    },
...

Perhaps you could add a patch that changed v1 to v1beta in node_modules/@google/generative-ai/dist/index.mjs

@Sweaterdog
Copy link
Author

Thanks for reminding me! I will fix that ASAP

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Made it so the endpoint for Gemini models is now v1beta instead of v1, allowing access to all new models

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@Sweaterdog
Copy link
Author

Okay, I added the patch file to the code repository

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed the Gemini 2.0 patch

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed Local Model usage with reasoning
@Sweaterdog
Copy link
Author

Fixed the reasoning handling for some models, and fixed the google patch

@Sweaterdog
Copy link
Author

This Pull Request is fully ready to be merged to the latest version.

@@ -129,6 +131,10 @@ export class Prompter {
model_profile.api = 'mistral';
else if (profile.model.includes("groq/") || profile.model.includes("groqcloud/"))
profile.api = 'groq';
else if (chat.model.includes('hf:'))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting an error here. Does chat need to be "profile" here?

MindServer running on port 8080
Client connected
Registering agents: andy
Agents registered
Starting agent with profile: ./andy.json
Starting agent initialization with profile: ./andy.json
Initializing action manager...
Initializing prompter...
Agent start failed with error
[ReferenceError: chat is not defined
at Prompter._selectAPI (file:///D:/Games/Minecraft/ai/src/mindcraft/src/models/prompter.js:134:18)
at new Prompter (file:///D:/Games/Minecraft/ai/src/mindcraft/src/models/prompter.js:59:39)
at Agent.start (file:///D:/Games/Minecraft/ai/src/mindcraft/src/agent/agent.js:33:29)
at file:///D:/Games/Minecraft/ai/src/mindcraft/src/process/init_agent.js:58:21
at file:///D:/Games/Minecraft/ai/src/mindcraft/src/process/init_agent.js:64:3
at ModuleJob.run (node:internal/modules/esm/module_job:271:25)
at async onImport.tracePromise.proto (node:internal/modules/esm/loader:547:26)
at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:98:5)]
Failed to start agent process:
[ReferenceError: chat is not defined
at Prompter._selectAPI (file:///D:/Games/Minecraft/ai/src/mindcraft/src/models/prompter.js:134:18)
at new Prompter (file:///D:/Games/Minecraft/ai/src/mindcraft/src/models/prompter.js:59:39)
at Agent.start (file:///D:/Games/Minecraft/ai/src/mindcraft/src/agent/agent.js:33:29)
at file:///D:/Games/Minecraft/ai/src/mindcraft/src/process/init_agent.js:58:21
at file:///D:/Games/Minecraft/ai/src/mindcraft/src/process/init_agent.js:64:3
at ModuleJob.run (node:internal/modules/esm/module_job:271:25)
at async onImport.tracePromise.proto (node:internal/modules/esm/loader:547:26)
at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:98:5)]
Agent process exited with code 1 and signal null
Agent process ./andy.json exited too quickly and will not be restarted.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed chat.model typos

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@Sweaterdog
Copy link
Author

Fixed the typos in prompter.js, should be working now

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed prompting for Embedding models

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed a minor error in the patch file for Gemini 2.0 models
@Sweaterdog
Copy link
Author

Just reviewed the current Pull Request, and everything works, it is now, guaranteed to be ready to be merged

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Fixed README.md

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants