Added AI Agent Semantic Convention #1739
Conversation
gyliu513
force-pushed
the
agent-sc
branch
8 times, most recently
from
e814811 to
7b7efce
Compare
Signed-off-by: Guangya Liu <[email protected]>
gyliu513
force-pushed
the
agent-sc
branch
5 times, most recently
from
5628dbe to
9904399
Compare
| --- | ||
|
|
||
| `gen_ai.operation.name` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used. | ||
|
|
||
| | Value | Description | Stability | | ||
| |---|---|---| | ||
| | `chat` | Chat completion operation such as [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) |  | | ||
| | `create_agent` | Create GenAI agent |  | |
There was a problem hiding this comment.
do we need to trace create_agent? not that I have a strong opinion about this. But, it may not be that valuable.
There was a problem hiding this comment.
@karthikscale3 this is important if we have multiple agents, with multiple agents, the agents may have some strict sequence to create, to run etc.
| @@ -17,6 +17,9 @@ This document defines the attributes used to describe telemetry in the context o | |||
|
|
|||
| | Attribute | Type | Description | Examples | Stability | | |||
| |---|---|---|---|---| | |||
| | <a id="gen-ai-agent-description" href="#gen-ai-agent-description">`gen_ai.agent.description`</a> | string | Free-form description of the GenAI agent provided by the application. | `Helps with math problems`; `Generates fiction stories` |  | | |||
There was a problem hiding this comment.
Do we also want the instruction given to the agent?
There was a problem hiding this comment.
Yes, usually the instruction will act as prompt for the models.
There was a problem hiding this comment.
@TaoChenOSU do you mean we need to add a new attribute for instruction?
There was a problem hiding this comment.
Yes, maybe in the create agent span. Some services embed the instruction into a large prompt, such as Bedrock agent.
| @@ -33,6 +36,9 @@ This document defines the attributes used to describe telemetry in the context o | |||
| | <a id="gen-ai-response-model" href="#gen-ai-response-model">`gen_ai.response.model`</a> | string | The name of the model that generated the response. | `gpt-4-0613` |  | | |||
| | <a id="gen-ai-system" href="#gen-ai-system">`gen_ai.system`</a> | string | The Generative AI product as identified by the client or server instrumentation. [3] | `openai` |  | | |||
| | <a id="gen-ai-token-type" href="#gen-ai-token-type">`gen_ai.token.type`</a> | string | The type of token being counted. | `input`; `output` |  | | |||
| | <a id="gen-ai-tool-call-id" href="#gen-ai-tool-call-id">`gen_ai.tool.call.id`</a> | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` |  | | |||
| | <a id="gen-ai-tool-name" href="#gen-ai-tool-name">`gen_ai.tool.name`</a> | string | Name of the tool utilized by the agent. | `Flights` |  | | |||
There was a problem hiding this comment.
Models can sometimes call multiple tools in parallel. Maybe this needs to be a list?
There was a problem hiding this comment.
yes, let me check how to update this to a list.
| @@ -33,6 +36,9 @@ This document defines the attributes used to describe telemetry in the context o | |||
| | <a id="gen-ai-response-model" href="#gen-ai-response-model">`gen_ai.response.model`</a> | string | The name of the model that generated the response. | `gpt-4-0613` |  | | |||
| | <a id="gen-ai-system" href="#gen-ai-system">`gen_ai.system`</a> | string | The Generative AI product as identified by the client or server instrumentation. [3] | `openai` |  | | |||
| | <a id="gen-ai-token-type" href="#gen-ai-token-type">`gen_ai.token.type`</a> | string | The type of token being counted. | `input`; `output` |  | | |||
| | <a id="gen-ai-tool-call-id" href="#gen-ai-tool-call-id">`gen_ai.tool.call.id`</a> | string | The tool call identifier. | `call_mszuSIzqtI65i1wAUOE8w5H4` |  | | |||
There was a problem hiding this comment.
Could we consider recording the tools available to the agents or simple the models? This is also requested in this issue: #1760
There was a problem hiding this comment.
I think this should belong to event with https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md#event-gen_aitoolmessage , we probably need to update event to reflect this.
There was a problem hiding this comment.
The tool message event is different as it records the responses from invoking the tools. We currently don't have a way to record what tools are available to the model or agent to invoke.
|
|
||
| ## Spans | ||
|
|
||
| ### Create Agent Span |
There was a problem hiding this comment.
Should we also consider a span for agent invocation?
Fixes #1732
Changes
Please provide a brief description of the changes here.
Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.
Merge requirement checklist
[chore]