v1.1.8

Changelog

(2023-12-26) Python Version 1.1.8, NPM Version 1.1.2

Features

• Added support for YAML file format in addition to JSON for improved readability of AIConfigs: (#583)
• python-sdk: Added optional param in add_prompt() method to specify index where to add prompt (#599)
• eval: Added generalized metric builder for creating your own metric evaluation class (#513)
• python-sdk: Supported using default model if no prompt model is provided (#600)
• python-sdk: Refactored update_model() method to take in model name and settings as separate arguments (#507)
• python-sdk: Supported additional types in Gemini model parser. Now includes list of strings, Content string, and Content struct: (#532)
• extensions: Added callback handlers to HuggingFace extensions (#597)
• python-sdk: Pinned google-generativeai to version 0.3.1 on Gemini model parser (#534)
• Added explicit output types to the ExecuteResult.data schema. Freeform also still supported (#589)

Bug Fixes / Tasks

• Checked for null in system prompt (#541)
• Converted protobuf to dict to fix pydantic BaseModel errors on Gemini (#558)(#558)
• Fixed issue where we were overwriting a single prompt output instead of creating a new one in batch execution (#566)
• Unpinned requests==2.30.0 dependency and using https instead of http in load_from_workbook() method (#582)
• typescript-sdk: Created automated test for typescript save() API (#198)

Documentation

• OpenAI Prompt Engineering Guide: https://openai-prompt-guide.streamlit.app/
• Chain-of-Verification Demo: https://chain-of-verification.streamlit.app/

v1.1.6

Changelog

Python Version 1.1.6 (2023-18-11)

NPM Version 1.1.1 (2023-18-11)

Full Changelog:

Features

• python-sdk: Created model parser extension for Google’s Gemini (#478, cookbook)
• Added attachment field to PromptInput schema to support non-text input data (ex: image, audio) (#473)
• python-sdk: Created batch execution interface using config.run_batch() (#469)
• Added model parser for HuggingFace text2Image tasks (#460)
• Updated evaluation metric values to be any arbitrary type, not just floats, & renamed fields for easier understanding (#484, #437)
• Merged aiconfig-extension-hugging-face-transformers into aiconfig-extension-hugging-face where all Hugging Face tasks will now be supported (#498, README)

Bug Fixes

• Fixed caching issue where re-running the same prompt caused nondeterministic behavior (#491)
• typescript-sdk: Pinned OpenAI dependency to to 4.11.1 to have a stable API surface (#524)
• typescript-sdk: Removed redundant PromptWithOutputs type (#508)

Documentation

• Refactored and shortened README (#493)
• Created table of supported models (#501)
• Updated cookbooks with explicit instructions on how to set API keys (#441)

v1.1.5

Changelog

Python Version 1.1.5 (2023-12-11)

NPM Version 1.0.8 (2023-12-11)

Full Changelog:

Features

• python-sdk: Evaluation harness for AIConfig. Supports text input/output evaluation with native AIConfig Interface (tutorial) as well as an integration with Promptfoo (tutorial). See README for details
• python-sdk: Support for PaLM Text as a core model Parser
• typescript-sdk: Support for PaLM Text as a core model parser (8902bef)

PaLM.Text.Example.mov

Bug Fixes

• python-extension HuggingFace Transformers Extension: Fixed bug where we're not properly appending outputs for multiple return sequences (49da477)
• python-extension HuggingFace Transformers Extension: Fixed a bug that defaulted model to GPT-2. (1c28f7c)

Extensions

• python-extension: Extension for LLama-Guard using pytorch (86cf687)
• • LLama Guard (Cookbook)

v1.1.4

Changelog

Python Version 1.1.4 (2023-12-04)

NPM Version 1.0.7 (2023-12-04)

Full Changelog:

Features

• python-sdk: DALL-E Model Parser (4753f21)
• python-sdk: Updated OpenAI Introspection Wrapper - Now more user-friendly with complete module mocking for easier import replacement, enhancing overall usability. (143c3dd)

Bug Fixes

• sdk: Updated add_prompt to rename prompt name if a different name is passed in (a29d5f87)
• typescript-sdk: Updated Metadata Field to be optional (cb5fdc5)

Better Engineering

• python-tests: Higher fidelity Test script, performs a complete build for testing (04fc5a5)
• tests: added a github action script testing for main(74e0c15)
• python-sdk: Added linter to python-sdk

Documentation

• readme: add readme and license within the python extension directory(450012c)
• cookbooks Updated Cookbooks' compatibility with latest aiconfig releases

Extensions

• python-extension: Extension for HuggingFace Text Generation with transformers (222bf6e)
• python-extension: Extension for LLama 2.0
• typescript-extension: Extension for LLama 2.0