OpenAI and Bedrock Preview #971

lrafeei · 2023-11-10T00:25:44Z

Overview

Merges Bedrock and OpenAI branches
Adds ML metrics for OpenAI and Bedrock

* Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com>

* Add openai sync instrumentation. * Remove commented code. * Test cleanup. * Add request/ response IDs. * Fixups. * Add conversation ID to message events. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add sync instrumentation for OpenAI embeddings. * Remove comments. * Clean up embedding event dictionary. * Update response_time to duration. * Linting fixes. * [Mega-Linter] Apply linters fixes * Trigger tests --------- Co-authored-by: umaannamalai <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]>

* Instrument acreate's for open ai async * Remove duplicated vendor * Re-use expected & input payloads in tests

* Attach non InferenceEvents to APM entity * Validate both resource payloads * Add tests for non-inference events * Add OpenAI sync embedding instrumentation (#938) * Add sync instrumentation for OpenAI embeddings. * Remove comments. * Clean up embedding event dictionary. * Update response_time to duration. * Linting fixes. * [Mega-Linter] Apply linters fixes * Trigger tests --------- Co-authored-by: umaannamalai <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Fixup: test names --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: umaannamalai <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add 4096 char truncation for ML events. * Add max attr check. * Fixup. * Fix character length ml event test. * Ignore test_ml_events.py for Py2. * Cleanup custom event if checks. * Add import statement. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add framework metric for OpenAI. * [Mega-Linter] Apply linters fixes * Trigger tests * Fix missing version info. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: umaannamalai <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add ml tests for outside transaction. * Update validator. * Add ML flag to application code path for record_ml_event.

* Add AWS Bedrock testing infrastructure * Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Remove OpenAI references --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Add example tests and mock error responses * Set invalid api key in auth error test Co-authored-by: Timothy Pansino <[email protected]> --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Timothy Pansino <[email protected]>

* Add openai sync instrumentation. * Remove commented code. * Initial openai error commit * Add example tests and mock error responses * Changes to attribute collection * Change error tests to match mock server * [Mega-Linter] Apply linters fixes * Trigger tests * Add dt_enabled decorator to error tests * Add embedded and async error tests * [Mega-Linter] Apply linters fixes * Trigger tests * Add http.statusCode to span before notice_error call * Report number of messages in error trace even if 0 * Revert notice_error and add _nr_message attr * Remove enabled_ml_settings as not needed * Add stats engine _nr_message test * [Mega-Linter] Apply linters fixes * Trigger tests * Revert black formatting in unicode/byte messages --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: lrafeei <[email protected]> Co-authored-by: hmstepanek <[email protected]>

* Add AWS Bedrock testing infrastructure * Squashed commit of the following: commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Squashed commit of the following: commit 182c7a8 Author: Uma Annamalai <[email protected]> Date: Fri Oct 13 10:12:55 2023 -0700 Add request/ response IDs. commit f6d13f8 Author: Uma Annamalai <[email protected]> Date: Thu Oct 12 13:23:39 2023 -0700 Test cleanup. commit d057663 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:23:00 2023 -0700 Remove commented code. commit dd29433 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:19:01 2023 -0700 Add openai sync instrumentation. commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * TEMP * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Bedrock titan extraction nearly complete * Cleaning up titan bedrock implementation * TEMP * Tests for bedrock passing Co-authored-by: Lalleh Rafeei <[email protected]> * Cleaned up titan testing Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Parametrized bedrock testing * Add support for AI21-J2 models * Change to dynamic no conversation id events * Drop all openai refs * [Mega-Linter] Apply linters fixes * Adding response_id and response_model * Drop python 3.7 tests for Hypercorn (#954) * Apply suggestions from code review * Remove unused import --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]>

* Add AWS Bedrock testing infrastructure * Squashed commit of the following: commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Squashed commit of the following: commit 182c7a8 Author: Uma Annamalai <[email protected]> Date: Fri Oct 13 10:12:55 2023 -0700 Add request/ response IDs. commit f6d13f8 Author: Uma Annamalai <[email protected]> Date: Thu Oct 12 13:23:39 2023 -0700 Test cleanup. commit d057663 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:23:00 2023 -0700 Remove commented code. commit dd29433 Author: Uma Annamalai <[email protected]> Date: Tue Oct 10 10:19:01 2023 -0700 Add openai sync instrumentation. commit 2834663 Author: Timothy Pansino <[email protected]> Date: Mon Oct 9 17:42:05 2023 -0700 OpenAI Mock Backend (#929) * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Pin flask version for flask restx tests. (#931) * Ignore new redis methods. (#932) Co-authored-by: Lalleh Rafeei <[email protected]> * Remove approved paths * Update CI Image (#930) * Update available python versions in CI * Update makefile with overrides * Fix default branch detection for arm builds --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Only get package version once (#928) * Only get package version once * Add disconnect method * Add disconnect method --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add datalib dependency for embedding testing. * Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * Add mock external openai server * Add mocked OpenAI server fixtures * Set up recorded responses. * Clean mock server to depend on http server * Linting * Remove approved paths * Add mocking for embedding endpoint * [Mega-Linter] Apply linters fixes * Add ratelimit headers * [Mega-Linter] Apply linters fixes * Add datalib dependency for embedding testing. --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> commit db63d45 Author: Uma Annamalai <[email protected]> Date: Mon Oct 2 15:31:38 2023 -0700 Add OpenAI Test Infrastructure (#926) * Add openai to tox * Add OpenAI test files. * Add test functions. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <mergify[bot]@users.noreply.github.com> * TEMP * Bedrock titan extraction nearly complete * Cleaning up titan bedrock implementation * TEMP * Tests for bedrock passing Co-authored-by: Lalleh Rafeei <[email protected]> * Cleaned up titan testing Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> * Parametrized bedrock testing * Add support for AI21-J2 models * Change to dynamic no conversation id events * Add cohere model * Remove openai instrumentation from this branch * Remove OpenAI from newrelic/config.py --------- Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]>

* AWS Bedrock embedding instrumentation * Correct symbol name

Co-authored-by: Timothy Pansino <[email protected]>

* Initial file migration * Enable DT on all span tests * Add pytest skip for older botocore versions * Fixup: app name merge conflict --------- Co-authored-by: Hannah Stepanek <[email protected]>

* Pin openai below 1.0 * Fixup

* Add get_ai_message_ids & message id capturing * Add tests * Remove generator * Add tests for conversation id unset * Add error code to mocked responses * Remove bedrock tests --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Uma Annamalai <[email protected]>

* Pin openai below 1.0 * Fixup * Add ingest_source to events * Remove duplicate test file * Handle 0.32.0.post1 version in tests (#963) --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Implement record_ai_feedback API. * [Mega-Linter] Apply linters fixes * Change API name to record_ai_feedback_event. * Fix API naming. * Rename to record_llm_feedback_event and get_llm_message_ids. * [Mega-Linter] Apply linters fixes * Address review feedback. * Update test structure. * [Mega-Linter] Apply linters fixes * Bump tests. --------- Co-authored-by: umaannamalai <[email protected]>

* Cache Package Version Lookups (#946) * Cache _get_package_version * Add Python 2.7 support to get_package_version caching * [Mega-Linter] Apply linters fixes * Bump tests --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> * Fix Redis Generator Methods (#947) * Fix scan_iter for redis * Replace generator methods * Update instance info instrumentation * Remove mistake from uninstrumented methods * Add skip condition to asyncio generator tests * Add skip condition to asyncio generator tests --------- Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Automatic RPM System Updates (#948) * Checkout old action * Adding RPM action * Add dry run * Incorporating action into workflow * Wire secret into custom action * Enable action * Correct action name * Fix syntax * Fix quoting issues * Drop pre-verification. Does not work on python * Fix merge artifact * Drop python 3.7 tests for Hypercorn (#954) * Fix pyenv installation for devcontainer (#936) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Remove duplicate kafka import hook (#956) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Initial bedrock error tracing commit * Handle 0.32.0.post1 version in tests (#963) * Add status code to mock bedrock server * Updating error response recording logic * Work on bedrock errror tracing * Chat completion error tracing * Adding embedding error tracing * Delete comment * Update moto --------- Co-authored-by: SlavaSkvortsov <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]>

github-actions · 2023-11-10T00:27:17Z

🦙 MegaLinter status: ❌ ERROR

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ PYTHON	bandit	4		0	5.24s
✅ PYTHON	black	22	4	0	4.93s
❌ PYTHON	flake8	22		3	1.21s
✅ PYTHON	isort	22	4	0	0.38s
❌ PYTHON	pylint	22		13	12.42s
✅ YAML	prettier	3	1	0	1.29s
✅ YAML	v8r	3		0	3.93s
✅ YAML	yamllint	3		0	0.48s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by

codecov-commenter · 2023-11-10T00:51:46Z

Codecov Report

Attention: 32 lines in your changes are missing coverage. Please review.

Comparison is base (b2e9e74) 81.79% compared to head (5d50120) 82.01%.

Files	Patch %	Lines
newrelic/hooks/mlmodel_openai.py	89.10%	8 Missing and 14 partials ⚠️
newrelic/hooks/external_botocore.py	94.51%	4 Missing and 5 partials ⚠️
newrelic/api/transaction.py	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                      Coverage Diff                       @@
##           develop-ai-limited-preview     #971      +/-   ##
==============================================================
+ Coverage                       81.79%   82.01%   +0.22%     
==============================================================
  Files                             190      191       +1     
  Lines                           19597    20013     +416     
  Branches                         3405     3473      +68     
==============================================================
+ Hits                            16029    16414     +385     
- Misses                           2588     2599      +11     
- Partials                          980     1000      +20

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tests/external_botocore/test_bedrock_chat_completion.py

hmstepanek

Looks good!

umaannamalai

🚀

umaannamalai and others added 30 commits October 2, 2023 15:31

Update OpenAI testing infra to match bedrock (#939)

ee42082

Instrument acreate's for open-ai (#935)

51e7362

* Instrument acreate's for open ai async * Remove duplicated vendor * Re-use expected & input payloads in tests

Add truncation support for ML events recorded outside txns. (#949)

7b3fe08

* Add ml tests for outside transaction. * Update validator. * Add ML flag to application code path for record_ml_event.

AWS Bedrock Embedding Instrumentation (#957)

d478b0d

* AWS Bedrock embedding instrumentation * Correct symbol name

Add support for bedrock claude (#960)

1803b64

Co-authored-by: Timothy Pansino <[email protected]>

Combine Botocore Tests (#959)

277d0a5

* Initial file migration * Enable DT on all span tests * Add pytest skip for older botocore versions * Fixup: app name merge conflict --------- Co-authored-by: Hannah Stepanek <[email protected]>

Pin openai tests to below 1.0 (#962)

78b8402

* Pin openai below 1.0 * Fixup

Merge branch 'develop-bedrock-instrumentation' into ai-preview

c602c2e

Add ingest source to openai events (#961)

02ad97d

* Pin openai below 1.0 * Fixup * Add ingest_source to events * Remove duplicate test file * Handle 0.32.0.post1 version in tests (#963) --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Handle 0.32.0.post1 version in tests (#963)

336fa5b

Merge branch 'develop-openai-instrumentation' into ai-preview

ca6006e

Initial merge commit

b7cd20a

Update moto

c5845af

Test for Bedrock embeddings metrics

dcbdabe

Merge branch 'develop-bedrock-instrumentation' into ai-preview

47cdfad

lrafeei added 3 commits November 9, 2023 15:52

Fix expected chat completion tests

b17e7a3

Merge branch 'develop-openai-instrumentation' into ai-preview

e2985d2

Merge branch 'main' into ai-preview

ec5c27b

mergify bot added the tests-failing label Nov 10, 2023

mergify bot removed the tests-failing label Nov 10, 2023

Remove commented out code

2f7253b

lrafeei marked this pull request as ready for review November 10, 2023 18:12

lrafeei requested a review from a team November 10, 2023 18:12

hmstepanek changed the base branch from main to develop-ai-limited-preview November 10, 2023 22:42

hmstepanek changed the base branch from develop-ai-limited-preview to develop-bedrock-instrumentation November 11, 2023 00:50

mergify bot added the merge-conflicts label Nov 11, 2023

hmstepanek requested changes Nov 11, 2023

View reviewed changes

tests/external_botocore/test_bedrock_chat_completion.py Outdated Show resolved Hide resolved

mergify bot mentioned this pull request Nov 11, 2023

Ai preview #972

Closed

hmstepanek changed the base branch from develop-bedrock-instrumentation to develop-openai-instrumentation November 11, 2023 01:48

mergify bot added tests-failing and removed merge-conflicts labels Nov 11, 2023

Correct Bedrock metric name

5d50120

lrafeei requested a review from hmstepanek November 13, 2023 21:53

hmstepanek changed the base branch from develop-openai-instrumentation to develop-ai-limited-preview November 13, 2023 23:30

hmstepanek approved these changes Nov 13, 2023

View reviewed changes

umaannamalai approved these changes Nov 14, 2023

View reviewed changes

umaannamalai merged commit 498126a into develop-ai-limited-preview Nov 14, 2023
43 of 46 checks passed

umaannamalai deleted the ai-preview branch November 14, 2023 22:31

mergify bot removed the tests-failing label Nov 14, 2023

umaannamalai added this to the v9.8.0 milestone Mar 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OpenAI and Bedrock Preview #971

OpenAI and Bedrock Preview #971

lrafeei commented Nov 10, 2023

github-actions bot commented Nov 10, 2023 •

edited

Loading

codecov-commenter commented Nov 10, 2023 •

edited

Loading

hmstepanek left a comment

umaannamalai left a comment

OpenAI and Bedrock Preview #971

OpenAI and Bedrock Preview #971

Conversation

lrafeei commented Nov 10, 2023

Overview

github-actions bot commented Nov 10, 2023 • edited Loading

🦙 MegaLinter status: ❌ ERROR

codecov-commenter commented Nov 10, 2023 • edited Loading

Codecov Report

hmstepanek left a comment

Choose a reason for hiding this comment

umaannamalai left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 10, 2023 •

edited

Loading

codecov-commenter commented Nov 10, 2023 •

edited

Loading