From b57bf32b04b4b914148d9622dd01a3d901dcbc07 Mon Sep 17 00:00:00 2001 From: Cleo Schneider Date: Wed, 12 Feb 2025 21:32:18 +0000 Subject: [PATCH 1/3] docs(observability): Add advanced configuration, authentication, and telemetry collection docs to observability --- docs/_guides.yaml | 6 + docs/observability/advanced-configuration.md | 125 +++++++ docs/observability/authentication.md | 102 ++++++ docs/observability/telemetry-collection.md | 282 ++++++++++++++ docs/plugins/firebase.md | 28 +- docs/plugins/google-cloud.md | 366 +------------------ 6 files changed, 520 insertions(+), 389 deletions(-) create mode 100644 docs/observability/advanced-configuration.md create mode 100644 docs/observability/authentication.md create mode 100644 docs/observability/telemetry-collection.md diff --git a/docs/_guides.yaml b/docs/_guides.yaml index a6aab46e3..179ec9a52 100644 --- a/docs/_guides.yaml +++ b/docs/_guides.yaml @@ -73,6 +73,12 @@ toc: - heading: Observing AI workflows - title: Getting Started path: /docs/genkit/observability/getting-started + - title: Authentication + path: /docs/genkit/observability/authentication + - title: Advance Configuration + path: /docs/genkit/observability/advanced-configuration + - title: Telemetry Collection + path: /docs/genkit/observability/telemetry-collection - title: Troubleshooting path: /docs/genkit/observability/troubleshooting diff --git a/docs/observability/advanced-configuration.md b/docs/observability/advanced-configuration.md new file mode 100644 index 000000000..ece6d1768 --- /dev/null +++ b/docs/observability/advanced-configuration.md @@ -0,0 +1,125 @@ +# Advanced Configuration + +This guide focuses on advanced configuration options for deployed features using Firebase Genkit +Monitoring. Detailed descriptions of each configuration option can be found in our +[JS API reference documentation](https://js.api.genkit.dev/interfaces/_genkit-ai_google-cloud.GcpTelemetryConfigOptions.html). +This documentation will describe how to fine-tune GCP configuration options for more control over +which telemetry is collected, how often, and from what environments. + +## Default Configuration + +Firebase Genkit Monitoring provides default options, out of the box, to get you up and running quickly. + + * [autoInstrumentation](https://opentelemetry.io/docs/zero-code/js/): `true` + * [autoInstrumentationConfig](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations): + + ``` + { + '@opentelemetry/instrumentation-dns': { enabled: false }, + }, + ``` + + * credentials: pulled from chosen [authentication strategy](./authentication.md) + * disableMetrics: `false` + * disableTraces: `false` + * disableLoggingInputAndOutput: `false` + * forceDevExport: `false` + * metricExportIntervalMillis: 5 minutes + * metricExportTimeoutMillis: 5 minutes + * projectId: pulled from [authentication strategy](./authentication.md) + * sampler: [AlwaysOnSampler](https://js.api.genkit.dev/interfaces/_genkit-ai_google-cloud.GcpTelemetryConfigOptions.html#sampler) + +## Export local telemetry + +To export telemetry when running locally set the `forceDevExport` option to `true`. + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({forceDevExport: true}); +``` + +To see more realtime results, also adjust the export interval and timeout. + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({ + forceDevExport: true, + metricExportIntervalMillis: 10_000, // 10 seconds + metricExportTimeoutMillis: 10_000 // 10 seconds +}); +``` + +## Adjust auto instrumentation + +Auto instrumentation provides metric, trace, and log collection for many popular frameworks and +libraries with no additional code. Read more about OTEL zero-code instrumentation [here](https://opentelemetry.io/docs/zero-code/js/) and check out +the full list of auto-instrumentation options +[here](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations). + +To modify the list of instrumentations that are eligible for auto instrumentation, update the `autoInstrumentationConfig` field: + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({ + autoInstrumentationConfig: { + '@opentelemetry/instrumentation-fs': { enabled: false }, + '@opentelemetry/instrumentation-dns': { enabled: false }, + '@opentelemetry/instrumentation-net': { enabled: false }, + } +}); +``` + +## Disabling telemetry + +Firebase Genkit Monitoring leverages a combination of logging, tracing, and metrics to capture +a holistic view of your Genkit interactions, however, you can also disable each of these elements +independently if needed. + +### Disable input and output logging + +To disable logging input and output from model interactions add the following to your configuration: + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({ + disableLoggingInputAndOutput: true +}); +``` + +With this option set, input and output attributes will be redacted +in the Firebase Genkit Monitoring trace viewer and will be missing +from Google Cloud logging. + +### Disable metrics + +To disable metrics collection, add the following to your configuration: + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({ + disableMetrics: true +}); +``` + +With this option set, you will no longer see stability metrics in the +Firebase Genkit Monitoring dashboard and will be missing from Google Cloud Metrics. + +### Disable traces + +To disable trace collection, add the following to your configuration: + +```typescript +import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; + +enableFirebaseTelemetry({ + disableTraces: true +}); +``` + +With this option set, you will no longer see traces in the Firebase Genkit Monitoring +feature page, have access to the trace viewer, or see traces present in Google Cloud Tracing. \ No newline at end of file diff --git a/docs/observability/authentication.md b/docs/observability/authentication.md new file mode 100644 index 000000000..f5ccfdf54 --- /dev/null +++ b/docs/observability/authentication.md @@ -0,0 +1,102 @@ +# Authentication and authorization + +Firebase Genkit Monitoring requires a Google Cloud project ID and application credentials. + +## User Authentication + +When running Genkit locally with the intent to export telemetry to Firebase Genkit Monitoring, +you will be authenticating as yourself. The easiest way to do this is via the gcloud CLI. This +will set up [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) for your user which will be picked up automatically by the framework. + +Follow the gcloud CLI installation instructions [here](https://cloud.google.com/sdk/docs/install#installation_instructions). + +1. Authenticate using the `gcloud` CLI: + + ```posix-terminal + gcloud auth application-default login + ``` + +2. Set your project ID + + ```posix-terminal + gcloud config set project PROJECT_ID + ``` + +## Deploy with Application Default Credentials (ADC) on Google Cloud + +If deploying your code to a Google Cloud environment (Cloud +Functions, Cloud Run, etc), the project ID and credentials will be discovered +automatically via +[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc). + +You will need to apply the following roles to the service account that is +running your code (i.e. 'attached service account') via the +[IAM Console](https://console.cloud.google.com/iam-admin/iam): + +- `roles/monitoring.metricWriter` +- `roles/cloudtrace.agent` +- `roles/logging.logWriter` + +To find your default service account: + +1. Navigate to the [service accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) in the Google Cloud Console +2. Select your project +3. Your default service account should look like *project-number*-compute@developer.gserviceaccount.com or *project-id*@appspot.gserviceaccount.com if you are using App Engine. + +## Deploying outside of Google Cloud with Application Default Credentials (ADC) + +If possible, it is still recommended to leverage the +[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc) +process to make credentials available to the plugin. + +Typically this involves generating a service account key/pair and deploying +those credentials to your production environment. + +1. Follow the instructions to set up a + [service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating). + +2. Ensure the service account has the following roles: + - `roles/monitoring.metricWriter` + - `roles/cloudtrace.agent` + - `roles/logging.logWriter` + +3. Deploy the credential file to production (**do not** check into source code) + +4. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable as the path to + the credential file. + + ``` + GOOGLE_APPLICATION_CREDENTIALS = "path/to/your/key/file" + ``` + +## Deploying outside of Google Cloud without Application Default Credentials (ADC) + +In some serverless environments, you may not be able to deploy a credential +file. + +1. Follow the instructions to set up a +[service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating). + +2. Ensure the service account has the following roles: + - `roles/monitoring.metricWriter` + - `roles/cloudtrace.agent` + - `roles/logging.logWriter` + +3. Download the credential file. + +4. Assign the contents of the credential file to the `GCLOUD_SERVICE_ACCOUNT_CREDS` environment variable as follows: + +``` +GCLOUD_SERVICE_ACCOUNT_CREDS='{ + "type": "service_account", + "project_id": "your-project-id", + "private_key_id": "your-private-key-id", + "private_key": "your-private-key", + "client_email": "your-client-email", + "client_id": "your-client-id", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": "https://accounts.google.com/o/oauth2/token", + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", + "client_x509_cert_url": "your-cert-url" +}' +``` diff --git a/docs/observability/telemetry-collection.md b/docs/observability/telemetry-collection.md new file mode 100644 index 000000000..7c3d9fede --- /dev/null +++ b/docs/observability/telemetry-collection.md @@ -0,0 +1,282 @@ +# Telemetry Collection + +Firebase Genkit Monitoring leverages Google Cloud metrics, traces, and logs to capture telemetry. This +document details which metrics, trace attributes, and logs will be collected and what you can expect for +latency, quotas, and cost. + +## Telemetry delay + +There may be a slight delay before telemetry for a particular execution of a +flow is available in Firebase or Google Cloud. This is dependent on your export +interval and by default within 5 to 10 minutes. + +## Quotas and limits + +There are several quotas that are important to keep in mind: + +- [Cloud Trace Quotas](http://cloud.google.com/trace/docs/quotas) +- [Cloud Logging Quotas](http://cloud.google.com/logging/quotas) +- [Cloud Monitoring Quotas](http://cloud.google.com/monitoring/quotas) + +## Cost + +Cloud Logging, Cloud Trace, and Cloud Monitoring have generous free-of-charge +tiers. Specific pricing can be found at the following links: + +- [Cloud Logging Pricing](http://cloud.google.com/stackdriver/pricing#google-cloud-observability-pricing) +- [Cloud Trace Pricing](https://cloud.google.com/trace#pricing) +- [Cloud Monitoring Pricing](https://cloud.google.com/stackdriver/pricing#monitoring-pricing-summary) + +## Metrics + +### Feature metrics + +Features are the top-level entry-point to your Genkit code. In most cases, this +will be a flow. Otherwise, this will be the top-most span in a trace. + +| Name | Type | Description | +| ----------------------- | --------- | ----------------------- | +| genkit/feature/requests | Counter | Number of requests | +| genkit/feature/latency | Histogram | Execution latency in ms | + +Each feature metric contains the following dimensions: + +| Name | Description | +| ------------- | -------------------------------------------------------------------------------- | +| name | The name of the feature. In most cases, this is the top-level Genkit flow | +| status | 'success' or 'failure' depending on whether or not the feature request succeeded | +| error | Only set when `status=failure`. Contains the error type that caused the failure | +| source | The Genkit source language. Eg. 'ts' | +| sourceVersion | The Genkit framework version | + + +### Action metrics + +Actions represent a generic step of execution within Genkit. Each of these steps +will have the following metrics tracked: + +| Name | Type | Description | +| ----------------------- | --------- | --------------------------------------------- | +| genkit/action/requests | Counter | Number of times this action has been executed | +| genkit/action/latency | Histogram | Execution latency in ms | + +Each action metric contains the following dimensions: + +| Name | Description | +| ------------- | ---------------------------------------------------------------------------------------------------- | +| name | The name of the action | +| featureName | The name of the parent feature being executed | +| path | The path of execution from the feature root to this action. eg. '/myFeature/parentAction/thisAction' | +| status | 'success' or 'failure' depending on whether or not the action succeeded | +| error | Only set when `status=failure`. Contains the error type that caused the failure | +| source | The Genkit source language. Eg. 'ts' | +| sourceVersion | The Genkit framework version | + +### Generate metrics + +These are special action metrics relating to actions that interact with a model. +In addition to requests and latency, input and output are also tracked, with +model specific dimensions that make debugging and configuration tuning easier. + +| Name | Type | Description | +| ------------------------------------ | --------- | ------------------------------------------ | +| genkit/ai/generate/requests | Counter | Number of times this model has been called | +| genkit/ai/generate/latency | Histogram | Execution latency in ms | +| genkit/ai/generate/input/tokens | Counter | Input tokens | +| genkit/ai/generate/output/tokens | Counter | Output tokens | +| genkit/ai/generate/input/characters | Counter | Input characters | +| genkit/ai/generate/output/characters | Counter | Output characters | +| genkit/ai/generate/input/images | Counter | Input images | +| genkit/ai/generate/output/images | Counter | Output images | +| genkit/ai/generate/input/audio | Counter | Input audio files | +| genkit/ai/generate/output/audio | Counter | Output audio files | + +Each generate metric contains the following dimensions: + +| Name | Description | +| --------------- | ---------------------------------------------------------------------------------------------------- | +| modelName | The name of the model | +| featureName | The name of the parent feature being executed | +| path | The path of execution from the feature root to this action. eg. '/myFeature/parentAction/thisAction' | +| latencyMs | The response time taken by the model | +| status | 'success' or 'failure' depending on whether or not the feature request succeeded | +| error | Only set when `status=failure`. Contains the error type that caused the failure | +| source | The Genkit source language. Eg. 'ts' | +| sourceVersion | The Genkit framework version | + +## Traces + +All Genkit actions are automatically instrumented to provide detailed traces for your AI features. Locally, traces are visible in +the Developer UI. For deployed apps enable Firebase Genkit Monitoring to get the same level of visibility. + +The following sections describe what trace attributes you can expect based on the Genkit action type for a particular span in the +trace. + +### Root Spans + +Root spans have special attributes to help disambiguate the state attributes for the whole trace versus an individual span. + +| Attribute name | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| genkit/feature | The name of the parent feature being executed | +| genkit/isRoot | Marked true if this span is the root span | +| genkit/rootState | The state of the overall execution as `success` or `error`. This does not indicate that this step failed in particular. | + +### Flow + +| Attribute name | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| genkit/input | The input to the flow. This will always be `` because of trace attribute size limits. | +| genkit/metadata/subtype | The type of Genkit action. For flows it will be `flow`. | +| genkit/name | The name of this Genkit action. In this case the name of the flow | +| genkit/output | The output generated in the flow. This will always be `` because of trace attribute size limits. | +| genkit/path | The fully qualified execution path that lead to this step in the trace, including type information. | +| genkit/state | The state of this span's execution as `success` or `error`. | +| genkit/type | The type of Genkit primitive that corresponds to this span. For flows, this will be `action`. | + +### Util + +| Attribute name | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| genkit/input | The input to the util. This will always be `` because of trace attribute size limits. | +| genkit/name | The name of this Genkit action. In this case the name of the flow | +| genkit/output | The output generated in the util. This will always be `` because of trace attribute size limits. | +| genkit/path | The fully qualified execution path that lead to this step in the trace, including type information. | +| genkit/state | The state of this span's execution as `success` or `error`. | +| genkit/type | The type of Genkit primitive that corresponds to this span. For flows, this will be `util`. | + +### Model + +| Attribute name | Description | +| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| genkit/input | The input to the model. This will always be `` because of trace attribute size limits. | +| genkit/metadata/subtype | The type of Genkit action. For moedles it will be `model`. | +| genkit/model | The name of the model. | +| genkit/name | The name of this Genkit action. In this case the name of the model. | +| genkit/output | The output generated by the model. This will always be `` because of trace attribute size limits. | +| genkit/path | The fully qualified execution path that lead to this step in the trace, including type information. | +| genkit/state | The state of this span's execution as `success` or `error`. | +| genkit/type | The type of Genkit primitive that corresponds to this span. For flows, this will be `action`. | + +### Tool + +genkit/input +genkit/metadata/subtype tool +genkit/name getWeather +genkit/output +genkit/path /{flowTouristActivitiesStreaming,t:flow}/{generate,t:util}/{generate,t:util}/{generate,t:util}/{getWeather,t:action,s:tool} +genkit/state success +genkit/type action + +## Logs + +For deployed apps with Firebase Genkit Monitoring, logs are used to capture input, output, and configuration metadata that provides +rich detail about each step in your AI feature. + +All logs will include the following shared metadata fields: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| insertId | Unique id for the log entry | +| jsonPayload | Container for variable information that is unique to each log type | +| labels | `{module: genkit}` | +| logName | `projects/weather-gen-test-next/logs/genkit_log` | +| receivedTimestamp | Time the log was received by Cloud | +| resource | Information about the source of the log including deployment information region, and projectId | +| severity | The log level written. See Cloud's [LogSeverity](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#logseverity) | +| spanId | Identifier for the span that created this log | +| timestamp | Time that the client logged a message | +| trace | Identifier for the trace of the format `projects//traces/` | +| traceSampled | Boolean representing whether the trace was sampled. Logs are not sampled. | + +Each log type will have a different json payload described below. + +### Input + +JSON payload: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| message | `[genkit] Input[, ` including `(message X of N)` for multi-part messages | +| metadata | Additional context including the input message sent to the action | + +Metadata: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| content | The input message content sent to this Genkit action | +| featureName | The name of the Genkit flow, action, tool, util, or helper. | +| messageIndex * | Index indicating the order of messages for inputs that contain multiple messages. For single messages, this will always be 0. | +| model * | Model name. | +| path | The execution path that generated this log of the format `step1 > step2 > step3` | +| partIndex * | Index indicating the order of parts within a message for multi-part messages. This is typical when combining text and images in a single input. | +| qualifiedPath | The execution path that generated this log, including type information of the format: `/{flow1,t:flow}/{generate,t:util}/{modelProvider/model,t:action,s:model` | +| totalMessages * | The total number of messages for this input. For single messages, this will always be 1. | +| totalParts * | Total number of parts for this message. For single-part messages, this will always be 1. | + +(*) Starred items are only present on Input logs for model interactions. + +### Output + +JSON payload: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| message | `[genkit] Output[, ` including `(message X of N)` for multi-part messages | +| metadata | Additional context including the input message sent to the action | + +Metadata: + +| Field name | Description | +| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| candidateIndex * (deprecated) | Index indicating the order of candidates for outputs that contain multiple candidates. For logs with single candidates, this will always be 0. | +| content | The output message generated by the Genkit action | +| featureName | The name of the Genkit flow, action, tool, util, or helper. | +| messageIndex * | Index indicating the order of messages for inputs that contain multiple messages. For single messages, this will always be 0. | +| model * | Model name. | +| path | The execution path that generated this log of the format `step1 > step2 > step3 | +| partIndex * | Index indicating the order of parts within a message for multi-part messages. This is typical when combining text and images in a single output. | +| qualifiedPath | The execution path that generated this log, including type information of the format: `/{flow1,t:flow}/{generate,t:util}/{modelProvider/model,t:action,s:model` | +| totalCandidates * (deprecated) | Total number of candidates generated as output. For single-candidate messages, this will always be 1. | +| totalParts * | Total number of parts for this message. For single-part messages, this will always be 1. | + +(*) Starred items are only present on Input logs for model interactions. + +### Config + +JSON payload: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| message | `[genkit] Config[, ` including `(message X of N)` for multi-part messages | +| metadata | Additional context including the input message sent to the action | + +Metadata: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| featureName | The name of the Genkit flow, action, tool, util, or helper. | +| model | Model name. | +| path | The execution path that generated this log of the format `step1 > step2 > step3 | +| qualifiedPath | The execution path that generated this log, including type information of the format: `/{flow1,t:flow}/{generate,t:util}/{modelProvider/model,t:action,s:model` | +| source | The Genkit library language used. This will always be set to 'ts' as it is the only supported language. | +| sourceVersion | The Genkit library version. | +| temperature | Model temperature used. | + +### Paths + +Path logs are only present when genkit is used with a + +JSON payload: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| message | `[genkit] Input[, ` including `(message X of N)` for multi-part messages | +| metadata | Additional context including the input message sent to the action | + +Metadata: + +| Field name | Description | +| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| flowName | The name of the Genkit flow, action, tool, util, or helper. | +| paths | An array containing all execution paths for the collected spans. | diff --git a/docs/plugins/firebase.md b/docs/plugins/firebase.md index 8833f8f0c..42ea91fcc 100644 --- a/docs/plugins/firebase.md +++ b/docs/plugins/firebase.md @@ -84,31 +84,9 @@ Application Default Credentials. To specify your credentials: ### Telemetry -Firebase Genkit Monitoring is powered by Google's Cloud operation suite. This -requires telemetry related API's to be enabled for your project. Please refer -to the [Google Cloud plugin](google-cloud.md#set-up-a-google-cloud-account) -documentation for more details. - -Grant the following roles to the **"Default compute service account"** within -the [Google Cloud IAM Console](https://console.cloud.google.com/iam-admin/iam): - -- **Monitoring Metric Writer** (roles/monitoring.metricWriter) -- **Cloud Trace Agent** (roles/cloudtrace.agent) -- **Logs Writer** (roles/logging.logWriter) - -To enable telemetry export call `enableFirebaseTelemetry()`: - - - -```js -import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; - -enableFirebaseTelemetry({ - forceDevExport: false, // Set this to true to export telemetry for local runs -}); -``` - -This plugin shares [configuration options](google-cloud.md#plugin-configuration) with the [Google Cloud plugin](google-cloud.md). +The Firebase plugin includes an implementation for sending telemetry data, +including metrics, traces, and logs to Firebase Genkit Monitoring. To get started, +visit [../observability/getting-started.md]. ### Cloud Firestore vector search diff --git a/docs/plugins/google-cloud.md b/docs/plugins/google-cloud.md index e422c933f..5123c6677 100644 --- a/docs/plugins/google-cloud.md +++ b/docs/plugins/google-cloud.md @@ -43,370 +43,8 @@ import { enableGoogleCloudTelemetry } from '@genkit-ai/google-cloud'; enableGoogleCloudTelemetry(); ``` -When running in production, telemetry will be exported automatically. +When running in production, telemetry will be exported automatically. See the [Advanced configuration guide](./observability/advanced-configuration.md) for configuration options. -### Authentication and authorization +Note: All configuration options work for both `enableFirebaseTelemetry` and `enableGoogleCloudTelemetry`. -The plugin requires a Google Cloud project ID and application credentials. -#### Google Cloud - -If deploying your code to a Google Cloud environment (Cloud -Functions, Cloud Run, etc), the project ID and credentials will be discovered -automatically via -[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc). - -You will need to apply the following roles to the service account that is -running your code (i.e. 'attached service account') via the -[IAM Console](https://console.cloud.google.com/iam-admin/iam): - -- `roles/monitoring.metricWriter` -- `roles/cloudtrace.agent` -- `roles/logging.logWriter` - -#### Local Development - -When doing local development, in order for your user credentials to be available -to the plugin, additional steps are required. - -1. Set the `GCLOUD_PROJECT` environment variable to your Google Cloud project. - -2. Authenticate using the `gcloud` CLI: - - ```posix-terminal - gcloud auth application-default login - ``` - -#### Production environments outside of Google Cloud - -If possible, it is still recommended to leverage the -[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc) -process to make credentials available to the plugin. - -Typically this involves generating a service account key/pair and deploying -those credentials to your production environment. - -1. Follow the instructions to set up a - [service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating). - -2. Ensure the service account has the following roles: - - `roles/monitoring.metricWriter` - - `roles/cloudtrace.agent` - - `roles/logging.logWriter` - -3. Deploy the credential file to production (**do not** check into source code) - -4. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable as the path to - the credential file. - - ``` - GOOGLE_APPLICATION_CREDENTIALS = "path/to/your/key/file" - ``` - -In some serverless environments, you may not be able to deploy a credential -file. In this case, as an alternative to steps 3 & 4 above, you can set the -`GCLOUD_SERVICE_ACCOUNT_CREDS` environment variable with the contents of the -credential file as follows: - -``` -GCLOUD_SERVICE_ACCOUNT_CREDS='{ - "type": "service_account", - "project_id": "your-project-id", - "private_key_id": "your-private-key-id", - "private_key": "your-private-key", - "client_email": "your-client-email", - "client_id": "your-client-id", - "auth_uri": "https://accounts.google.com/o/oauth2/auth", - "token_uri": "https://accounts.google.com/o/oauth2/token", - "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", - "client_x509_cert_url": "your-cert-url" -}' -``` - -## Plugin configuration - -The `enableGoogleCloudTelemetry()` function takes an optional configuration -object which configures the -[OpenTelemetry NodeSDK](https://open-telemetry.github.io/opentelemetry-js/classes/_opentelemetry_sdk_node.NodeSDK.html) -instance. - -```ts -import { AlwaysOnSampler } from '@opentelemetry/sdk-trace-base'; - -enableGoogleCloudTelemetry({ - forceDevExport: false, // Set this to true to export telemetry for local runs - sampler: new AlwaysOnSampler(), - autoInstrumentation: true, - autoInstrumentationConfig: { - '@opentelemetry/instrumentation-fs': { enabled: false }, - '@opentelemetry/instrumentation-dns': { enabled: false }, - '@opentelemetry/instrumentation-net': { enabled: false }, - }, - metricExportIntervalMillis: 5_000, -}); -``` -The configuration objects allows fine grained control over various aspects of -the telemetry export outlined below. - -#### credentials -Allows specifying credentials directly using -[JWTInput](http://cloud/nodejs/docs/reference/google-auth-library/latest/google-auth-library/jwtinput) -from the google-auth library. - -#### sampler - -For cases where exporting all traces isn't practical, OpenTelemetry allows trace -[sampling](https://opentelemetry.io/docs/languages/java/instrumentation/#sampler). - -There are four preconfigured samplers: - -- [AlwaysOnSampler](https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-sdk-trace-base/src/sampler/AlwaysOnSampler.ts) - samples all traces -- [AlwaysOffSampler](https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-sdk-trace-base/src/sampler/AlwaysOffSampler.ts) - samples no traces -- [ParentBased](https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-sdk-trace-base/src/sampler/ParentBasedSampler.ts) - samples based on parent span -- [TraceIdRatioBased](https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-sdk-trace-base/src/sampler/TraceIdRatioBasedSampler.ts) - samples a configurable percentage of traces - -#### autoInstrumentation & autoInstrumentationConfig - -Enabling -[automatic instrumentation](https://opentelemetry.io/docs/languages/js/automatic/) -allows OpenTelemetry to capture telemetry data from -[third-party libraries](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/src/utils.ts) -without the need to modify code. - -#### metricExportIntervalMillis - -This field specifies the metrics export interval in milliseconds. - -> Note: The minimum export interval for Google Cloud Monitoring is 5000ms. - -#### metricExportTimeoutMillis - -This field specifies the timeout for the metrics export in milliseconds. - -#### disableMetrics - -Provides an override that disables metrics export while still exporting traces -and logs. - -#### disableTraces - -Provides an override that disables exporting traces while still exprting metrics -and logs. - -#### disableLoggingInputAndOutput - -Provides an override that disables collecting input and output logs. - -#### forceDevExport - -This option will force Genkit to export telemetry and log data when running in -the `dev` environment (e.g. locally). - -> Note: When running locally, internal telemetry buffers may not fully flush -prior to the process exiting, resulting in an incomplete telemetry export. - -## Test your integration - -When configuring the plugin, use `forceDevExport: true` to enable telemetry -export for local runs. Navigate to the Google Cloud Logs, Metrics, or Trace -Explorer to view telemetry. - -# Google Cloud Observability suite - -Once your code (e.g. flow) is deployed, navigate to the -[Cloud Monitoring](https://console.cloud.google.com/monitoring/) dashboard and -select your project. From here, you can easily navigate between the Logs, -Metrics and Trace explorers for production monitoring. - - - -## Logs and traces - -From the left hand side menu, click 'Logs explorer' under the 'Explore' heading. - - - -Here, you will see all logs that are associated with your deployed Genkit code, -including `console.log()`. Any log which has the prefix `[genkit]` is a -Genkit-internal log that contains information that may be interesting for -debugging purposes. For example, Genkit logs in the format `Config[...]` contain -metadata such as the temperature and topK values for specific LLM inferences. -Logs in the format `Output[...]` contain LLM responses while `Input[...]` logs -contain the prompts. Cloud Logging has robust ACLs that allow fine grained -control over access to sensitive logs. - -> Note: Prompts and LLM responses are redacted from trace attributes in Cloud -Trace. For specific log lines, it is possible to navigate to their respective -traces by clicking on the extended menu - icon and -selecting "View in trace details". - - - -This will bring up a trace preview pane providing a quick glance of the details -of the trace. To get to the full details, click the "View in Trace" link at the -top right of the pane. - - - -The most prominent navigation element in Cloud Trace is the trace scatter plot. -It contains all collected traces in a given time span. - - - -Clicking on each data point will show its details below the scatter plot. - - - -The detailed view contains the flow shape, including all steps, and important -timing information. Cloud Trace has the ability to interleave all logs -associated with a given trace within this view. Select the "Show expanded" -option in the "Logs & events" drop down. - - - -The resultant view allows detailed examination of logs in the context of the -trace, including prompts and LLM responses. - - - -## Metrics - -Viewing all metrics exported by Genkit can be done by clicking on -'Metrics management' under the 'Configure' heading in the left hand side menu. - - - -The metrics management console contains a tabular view of all collected metrics, -including those that pertain to Cloud Run and its surrounding environment. -Clicking on the 'Workload' option will reveal a list that includes -Genkit-collected metrics. Any metric with the `genkit` prefix constitutes an -internal Genkit metric. - - - -Genkit collects several categories of metrics including: feature, action, and -generate. Each metric has several useful dimensions facilitating robust -filtering and grouping. - -Common dimensions include: - -- `flow_name` - the top-level name of the flow. -- `flow_path` - the span and its parent span chain up to the root span. -- `error_code` - in case of an error, the corresponding error code. -- `error_message` - in case of an error, the corresponding error message. -- `model` - the name of the model. - -### Feature metrics - -Features are the top-level entry-point to your Genkit code. In most cases, this -will be a flow. Otherwise, this will be the top-most span in a trace. - -| Name | Type | Description | -| ----------------------- | --------- | ----------------------- | -| genkit/feature/requests | Counter | Number of requests | -| genkit/feature/latency | Histogram | Execution latency in ms | - -Each feature metric contains the following dimensions: - -| Name | Description | -| ------------- | -------------------------------------------------------------------------------- | -| name | The name of the feature. In most cases, this is the top-level Genkit flow | -| status | 'success' or 'failure' depending on whether or not the feature request succeeded | -| error | Only set when `status=failure`. Contains the error type that caused the failure | -| source | The Genkit source language. Eg. 'ts' | -| sourceVersion | The Genkit framework version | - - -### Action metrics - -Actions represent a generic step of execution within Genkit. Each of these steps -will have the following metrics tracked: - -| Name | Type | Description | -| ----------------------- | --------- | --------------------------------------------- | -| genkit/action/requests | Counter | Number of times this action has been executed | -| genkit/action/latency | Histogram | Execution latency in ms | - -Each action metric contains the following dimensions: - -| Name | Description | -| ------------- | ---------------------------------------------------------------------------------------------------- | -| name | The name of the action | -| featureName | The name of the parent feature being executed | -| path | The path of execution from the feature root to this action. eg. '/myFeature/parentAction/thisAction' | -| status | 'success' or 'failure' depending on whether or not the action succeeded | -| error | Only set when `status=failure`. Contains the error type that caused the failure | -| source | The Genkit source language. Eg. 'ts' | -| sourceVersion | The Genkit framework version | - -### Generate metrics - -These are special action metrics relating to actions that interact with a model. -In addition to requests and latency, input and output are also tracked, with -model specific dimensions that make debugging and configuration tuning easier. - -| Name | Type | Description | -| ------------------------------------ | --------- | ------------------------------------------ | -| genkit/ai/generate/requests | Counter | Number of times this model has been called | -| genkit/ai/generate/latency | Histogram | Execution latency in ms | -| genkit/ai/generate/input/tokens | Counter | Input tokens | -| genkit/ai/generate/output/tokens | Counter | Output tokens | -| genkit/ai/generate/input/characters | Counter | Input characters | -| genkit/ai/generate/output/characters | Counter | Output characters | -| genkit/ai/generate/input/images | Counter | Input images | -| genkit/ai/generate/output/images | Counter | Output images | -| genkit/ai/generate/input/audio | Counter | Input audio files | -| genkit/ai/generate/output/audio | Counter | Output audio files | - -Each generate metric contains the following dimensions: - -| Name | Description | -| --------------- | ---------------------------------------------------------------------------------------------------- | -| modelName | The name of the model | -| featureName | The name of the parent feature being executed | -| path | The path of execution from the feature root to this action. eg. '/myFeature/parentAction/thisAction' | -| latencyMs | The response time taken by the model | -| status | 'success' or 'failure' depending on whether or not the feature request succeeded | -| error | Only set when `status=failure`. Contains the error type that caused the failure | -| source | The Genkit source language. Eg. 'ts' | -| sourceVersion | The Genkit framework version | - -Visualizing metrics can be done through the Metrics Explorer. Using the left -hand side menu, click 'Metrics explorer' under the 'Explore' heading. - - - -Select a metrics by clicking on the 'Select a metric' dropdown, selecting -'Generic Node', 'Genkit', and a metric. - - - -The visualization of the metric will depend on its type (counter, histogram, -etc). The Metrics Explorer provides robust aggregation and querying facilities -to help graph metrics by their various dimensions. - - - -## Telemetry delay - -There may be a slight delay before telemetry for a particular execution of a -flow is displayed in Cloud's operations suite. In most cases, this delay is -under 1 minute. - -## Quotas and limits - -There are several quotas that are important to keep in mind: - -- [Cloud Trace Quotas](http://cloud.google.com/trace/docs/quotas) -- [Cloud Logging Quotas](http://cloud.google.com/logging/quotas) -- [Cloud Monitoring Quotas](http://cloud.google.com/monitoring/quotas) - -## Cost - -Cloud Logging, Cloud Trace, and Cloud Monitoring have generous free-of-charge -tiers. Specific pricing can be found at the following links: - -- [Cloud Logging Pricing](http://cloud.google.com/stackdriver/pricing#google-cloud-observability-pricing) -- [Cloud Trace Pricing](https://cloud.google.com/trace#pricing) -- [Cloud Monitoring Pricing](https://cloud.google.com/stackdriver/pricing#monitoring-pricing-summary) \ No newline at end of file From 953e3d23e2d5bed0e22c759f8fabf9e866333e9f Mon Sep 17 00:00:00 2001 From: Cleo Schneider Date: Tue, 18 Feb 2025 20:36:30 +0000 Subject: [PATCH 2/3] docs(observability): Respond to code review comments. --- docs/_guides.yaml | 2 +- docs/observability/advanced-configuration.md | 23 +++---- docs/observability/authentication.md | 63 ++++++++++++++++---- docs/observability/telemetry-collection.md | 4 +- docs/plugins/firebase.md | 12 +++- docs/plugins/google-cloud.md | 49 ++------------- 6 files changed, 81 insertions(+), 72 deletions(-) diff --git a/docs/_guides.yaml b/docs/_guides.yaml index 179ec9a52..e7c35395c 100644 --- a/docs/_guides.yaml +++ b/docs/_guides.yaml @@ -75,7 +75,7 @@ toc: path: /docs/genkit/observability/getting-started - title: Authentication path: /docs/genkit/observability/authentication - - title: Advance Configuration + - title: Advanced Configuration path: /docs/genkit/observability/advanced-configuration - title: Telemetry Collection path: /docs/genkit/observability/telemetry-collection diff --git a/docs/observability/advanced-configuration.md b/docs/observability/advanced-configuration.md index ece6d1768..3ab5d1dc2 100644 --- a/docs/observability/advanced-configuration.md +++ b/docs/observability/advanced-configuration.md @@ -3,8 +3,8 @@ This guide focuses on advanced configuration options for deployed features using Firebase Genkit Monitoring. Detailed descriptions of each configuration option can be found in our [JS API reference documentation](https://js.api.genkit.dev/interfaces/_genkit-ai_google-cloud.GcpTelemetryConfigOptions.html). -This documentation will describe how to fine-tune GCP configuration options for more control over -which telemetry is collected, how often, and from what environments. + +This documentation will describe how to fine-tune which telemetry is collected, how often, and from what environments. ## Default Configuration @@ -39,7 +39,9 @@ import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; enableFirebaseTelemetry({forceDevExport: true}); ``` -To see more realtime results, also adjust the export interval and timeout. +During development and testing, you can decrease latency by adjusting the export interval and/or timeout. + +Note: you should not ship to production with these reduced values. ```typescript import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; @@ -53,12 +55,11 @@ enableFirebaseTelemetry({ ## Adjust auto instrumentation -Auto instrumentation provides metric, trace, and log collection for many popular frameworks and -libraries with no additional code. Read more about OTEL zero-code instrumentation [here](https://opentelemetry.io/docs/zero-code/js/) and check out -the full list of auto-instrumentation options -[here](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations). +The Firebase telemetry plugin will automatically collect traces and metrics for popular frameworks, via by OpenTelemetry [zero-code instrumentation](https://opentelemetry.io/docs/zero-code/js/). -To modify the list of instrumentations that are eligible for auto instrumentation, update the `autoInstrumentationConfig` field: +A full list of available instrumentations can be found in the [auto-instrumentations-node](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations) documentation. + +To selectively disable or enable instrumentations that are eligible for auto instrumentation, update the `autoInstrumentationConfig` field: ```typescript import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; @@ -72,7 +73,7 @@ enableFirebaseTelemetry({ }); ``` -## Disabling telemetry +## Disable telemetry Firebase Genkit Monitoring leverages a combination of logging, tracing, and metrics to capture a holistic view of your Genkit interactions, however, you can also disable each of these elements @@ -80,7 +81,9 @@ independently if needed. ### Disable input and output logging -To disable logging input and output from model interactions add the following to your configuration: +By default, the Firebase telemetry plugin will capture inputs and outputs for each Genkit feature and/or step. + +To help you control how customer data is stored, you can disable the logging of input and output by adding the following to your configuration: ```typescript import { enableFirebaseTelemetry } from '@genkit-ai/firebase'; diff --git a/docs/observability/authentication.md b/docs/observability/authentication.md index f5ccfdf54..d9cfd9f40 100644 --- a/docs/observability/authentication.md +++ b/docs/observability/authentication.md @@ -2,13 +2,30 @@ Firebase Genkit Monitoring requires a Google Cloud project ID and application credentials. +If you don't have a Google Cloud project and account, you can set one up in the [Firebase Console](https://console.firebase.google.com/) or in the [Google Cloud Console](https://cloud.google.com). All Firebase project IDs are Google Cloud project IDs. + +## Enable APIs + +Prior to adding the plugin, make sure the following APIs are enabled for +your project: + +- [Cloud Logging API](https://console.cloud.google.com/apis/library/logging.googleapis.com) +- [Cloud Trace API](https://console.cloud.google.com/apis/library/cloudtrace.googleapis.com) +- [Cloud Monitoring API](https://console.cloud.google.com/apis/library/monitoring.googleapis.com) + +These APIs should be listed in the +[API dashboard](https://console.cloud.google.com/apis/dashboard) for your +project. +Click [here](https://support.google.com/googleapi/answer/6158841) to learn more +about enabling and disabling APIs. + ## User Authentication -When running Genkit locally with the intent to export telemetry to Firebase Genkit Monitoring, -you will be authenticating as yourself. The easiest way to do this is via the gcloud CLI. This -will set up [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials) for your user which will be picked up automatically by the framework. +To export telemetry from your local development environment to Firebase Genkit Monitoring, you will need to authenticate yourself with Google Cloud. -Follow the gcloud CLI installation instructions [here](https://cloud.google.com/sdk/docs/install#installation_instructions). +The easiest way to authenticate as yourself is via the gcloud CLI, which will automatically make your credentials available to the framework via [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials). + +If you don't have the gcloud CLI installed, first follow the [installation instructions](https://cloud.google.com/sdk/docs/install#installation_instructions). 1. Authenticate using the `gcloud` CLI: @@ -22,7 +39,7 @@ Follow the gcloud CLI installation instructions [here](https://cloud.google.com/ gcloud config set project PROJECT_ID ``` -## Deploy with Application Default Credentials (ADC) on Google Cloud +## Deploy to Google Cloud If deploying your code to a Google Cloud environment (Cloud Functions, Cloud Run, etc), the project ID and credentials will be discovered @@ -37,13 +54,9 @@ running your code (i.e. 'attached service account') via the - `roles/cloudtrace.agent` - `roles/logging.logWriter` -To find your default service account: +Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below. -1. Navigate to the [service accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) in the Google Cloud Console -2. Select your project -3. Your default service account should look like *project-number*-compute@developer.gserviceaccount.com or *project-id*@appspot.gserviceaccount.com if you are using App Engine. - -## Deploying outside of Google Cloud with Application Default Credentials (ADC) +## Deploy outside of Google Cloud (with ADC) If possible, it is still recommended to leverage the [Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc) @@ -69,7 +82,9 @@ those credentials to your production environment. GOOGLE_APPLICATION_CREDENTIALS = "path/to/your/key/file" ``` -## Deploying outside of Google Cloud without Application Default Credentials (ADC) +Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below. + +## Deploy outside of Google Cloud (without ADC) In some serverless environments, you may not be able to deploy a credential file. @@ -100,3 +115,27 @@ GCLOUD_SERVICE_ACCOUNT_CREDS='{ "client_x509_cert_url": "your-cert-url" }' ``` + +Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below. + +## Find or create your service account + +To find the appropriate service account: + +1. Navigate to the [service accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) in the Google Cloud Console +2. Select your project +3. Find the appropriate service account. Common default service accounts are as follows: + + - Firebase functions & Cloud Run + + PROJECT ID-compute@developer.gserviceaccount.com + + - App Engine + + PROJECT ID@appspot.gserviceaccount.com + + - App Hosting + + firebase-app-hosting-compute@PROJECT ID.iam.gserviceaccount.com + +If you are deploying outside of the Google ecosystem or don't want to use a default service account, you can [create a service account](https://cloud.google.com/iam/docs/service-accounts-create#creating) in the Google Cloud console. diff --git a/docs/observability/telemetry-collection.md b/docs/observability/telemetry-collection.md index 7c3d9fede..3b8e50aba 100644 --- a/docs/observability/telemetry-collection.md +++ b/docs/observability/telemetry-collection.md @@ -6,9 +6,7 @@ latency, quotas, and cost. ## Telemetry delay -There may be a slight delay before telemetry for a particular execution of a -flow is available in Firebase or Google Cloud. This is dependent on your export -interval and by default within 5 to 10 minutes. +There may be a slight delay before telemetry from a given invocation is available in Firebase. This is dependent on your export interval (5 minutes by default). ## Quotas and limits diff --git a/docs/plugins/firebase.md b/docs/plugins/firebase.md index 42ea91fcc..8dd76b144 100644 --- a/docs/plugins/firebase.md +++ b/docs/plugins/firebase.md @@ -84,9 +84,15 @@ Application Default Credentials. To specify your credentials: ### Telemetry -The Firebase plugin includes an implementation for sending telemetry data, -including metrics, traces, and logs to Firebase Genkit Monitoring. To get started, -visit [../observability/getting-started.md]. +The Firebase plugin provides a telemetry implementation for sending metrics, traces, and logs to Firebase Genkit Monitoring and Cloud Observability. + +To get started, visit the [Getting started guide](../observability/getting-started.md) for installation and configuration instructions. + +See the [Authentication and authorization guide](../observability/authentication.md) to authenticate with Google Cloud. + +See the [Advanced configuration guide](../observability/advanced-configuration.md) for configuration options. + +See the [Telemetry collection](../observability/telemetry-collection.md) for details on which Genkit metrics, traces, and logs collected. ### Cloud Firestore vector search diff --git a/docs/plugins/google-cloud.md b/docs/plugins/google-cloud.md index 5123c6677..1ccdf4008 100644 --- a/docs/plugins/google-cloud.md +++ b/docs/plugins/google-cloud.md @@ -1,50 +1,13 @@ -# Google Cloud plugin +# Google Cloud plugin The Google Cloud plugin exports Firebase Genkit telemetry and logging data to the [Cloud Observability](https://cloud.google.com/products/operations) suite. -## Installation +Note: The Firebase telemetry plugin wraps the Google Cloud plugin and offers all the same functionality and configuration options. All exported telemetry data will be visible in Firebase Genkit Monitoring and Cloud Observability. -```posix-terminal -npm i --save @genkit-ai/google-cloud -``` +To get started, visit the [Getting started guide](../observability/getting-started.md) for installation and configuration instructions. -When running Genkit code locally that includes this plugin, you will also need -the [Google Cloud CLI tool](https://cloud.google.com/sdk/docs/install) -installed. - -## Set up a Google Cloud account - -This plugin requires a Google Cloud account/project. All Firebase projects -include one by default ([GCP Console](https://console.cloud.google.com)), -or you can sign up at https://cloud.google.com. - -Prior to adding the plugin, make sure that the following APIs are enabled for -your GCP project: - -- [Cloud Logging API](https://console.cloud.google.com/apis/library/logging.googleapis.com) -- [Cloud Trace API](https://console.cloud.google.com/apis/library/cloudtrace.googleapis.com) -- [Cloud Monitoring API](https://console.cloud.google.com/apis/library/monitoring.googleapis.com) - -These APIs should be listed in the -[API dashboard](https://console.cloud.google.com/apis/dashboard) for your -project. - -Click [here](https://support.google.com/googleapi/answer/6158841) to learn more -about enabling and disabling APIs. - -## Genkit configuration - -To enable Cloud Tracing, Logging, and Monitoring (metrics), simply call -`enableGoogleCloudTelemetry()`: - -```ts -import { enableGoogleCloudTelemetry } from '@genkit-ai/google-cloud'; - -enableGoogleCloudTelemetry(); -``` - -When running in production, telemetry will be exported automatically. See the [Advanced configuration guide](./observability/advanced-configuration.md) for configuration options. - -Note: All configuration options work for both `enableFirebaseTelemetry` and `enableGoogleCloudTelemetry`. +See the [Authentication and authorization guide](../observability/authentication.md) to authenticate with Google Cloud. +See the [Advanced configuration guide](../observability/advanced-configuration.md) for configuration options. +See the [Telemetry collection](../observability/telemetry-collection.md) for details on which Genkit metrics, traces, and logs collected. From b091bff594bd0e82c5baa5e5f68f9cca15ec34a8 Mon Sep 17 00:00:00 2001 From: Cleo Schneider Date: Wed, 19 Feb 2025 16:41:58 +0000 Subject: [PATCH 3/3] docs(observability): Remove google cloud docs and redirect to the Firebase plugin --- docs/_guides.yaml | 2 -- docs/observability/authentication.md | 2 +- docs/observability/telemetry-collection.md | 4 +--- docs/plugins/firebase.md | 2 +- docs/plugins/google-cloud.md | 13 ------------- js/plugins/google-cloud/README.md | 21 ++------------------- 6 files changed, 5 insertions(+), 39 deletions(-) delete mode 100644 docs/plugins/google-cloud.md diff --git a/docs/_guides.yaml b/docs/_guides.yaml index e7c35395c..a4a69e7c3 100644 --- a/docs/_guides.yaml +++ b/docs/_guides.yaml @@ -103,8 +103,6 @@ toc: path: /docs/genkit/templates/pgvector - title: Firebase path: /docs/genkit/plugins/firebase - - title: Google Cloud - path: /docs/genkit/plugins/google-cloud - break: true - title: Using Genkit with Next.js diff --git a/docs/observability/authentication.md b/docs/observability/authentication.md index d9cfd9f40..c6e49040c 100644 --- a/docs/observability/authentication.md +++ b/docs/observability/authentication.md @@ -1,6 +1,6 @@ # Authentication and authorization -Firebase Genkit Monitoring requires a Google Cloud project ID and application credentials. +The Firebase telemetry plugin requires a Google Cloud project ID and application credentials. If you don't have a Google Cloud project and account, you can set one up in the [Firebase Console](https://console.firebase.google.com/) or in the [Google Cloud Console](https://cloud.google.com). All Firebase project IDs are Google Cloud project IDs. diff --git a/docs/observability/telemetry-collection.md b/docs/observability/telemetry-collection.md index 3b8e50aba..896599334 100644 --- a/docs/observability/telemetry-collection.md +++ b/docs/observability/telemetry-collection.md @@ -1,8 +1,6 @@ # Telemetry Collection -Firebase Genkit Monitoring leverages Google Cloud metrics, traces, and logs to capture telemetry. This -document details which metrics, trace attributes, and logs will be collected and what you can expect for -latency, quotas, and cost. +The Firebase telemetry plugin exports a combination of metrics, traces, and logs to Google Cloud Observability. This document details which metrics, trace attributes, and logs will be collected and what you can expect in terms of latency, quotas, and cost. ## Telemetry delay diff --git a/docs/plugins/firebase.md b/docs/plugins/firebase.md index 8dd76b144..de7145feb 100644 --- a/docs/plugins/firebase.md +++ b/docs/plugins/firebase.md @@ -84,7 +84,7 @@ Application Default Credentials. To specify your credentials: ### Telemetry -The Firebase plugin provides a telemetry implementation for sending metrics, traces, and logs to Firebase Genkit Monitoring and Cloud Observability. +The Firebase plugin provides a telemetry implementation for sending metrics, traces, and logs to Firebase Genkit Monitoring. To get started, visit the [Getting started guide](../observability/getting-started.md) for installation and configuration instructions. diff --git a/docs/plugins/google-cloud.md b/docs/plugins/google-cloud.md deleted file mode 100644 index 1ccdf4008..000000000 --- a/docs/plugins/google-cloud.md +++ /dev/null @@ -1,13 +0,0 @@ -# Google Cloud plugin - -The Google Cloud plugin exports Firebase Genkit telemetry and logging data to the [Cloud Observability](https://cloud.google.com/products/operations) suite. - -Note: The Firebase telemetry plugin wraps the Google Cloud plugin and offers all the same functionality and configuration options. All exported telemetry data will be visible in Firebase Genkit Monitoring and Cloud Observability. - -To get started, visit the [Getting started guide](../observability/getting-started.md) for installation and configuration instructions. - -See the [Authentication and authorization guide](../observability/authentication.md) to authenticate with Google Cloud. - -See the [Advanced configuration guide](../observability/advanced-configuration.md) for configuration options. - -See the [Telemetry collection](../observability/telemetry-collection.md) for details on which Genkit metrics, traces, and logs collected. diff --git a/js/plugins/google-cloud/README.md b/js/plugins/google-cloud/README.md index e8b4c25f4..aee7992e2 100644 --- a/js/plugins/google-cloud/README.md +++ b/js/plugins/google-cloud/README.md @@ -1,25 +1,8 @@ # Google Cloud plugin for Genkit -## Installing the plugin +The Google Cloud plugin is a utility plugin to export telemetry and logs to Cloud Observability. This functionality is exposed through the Firebase plugin for customer use. -```bash -npm i --save @genkit-ai/google-cloud -``` - -## Using the plugin - -```ts -import { genkit } from 'genkit'; -import { enableGoogleCloudTelemetry } from '@genkit-ai/google-cloud'; - -enableGoogleCloudTelemetry(); - -const ai = genkit({ - plugins: [ - // ... - ], -}); -``` +Visit the [Getting started](https://firebase.google.com/docs/genkit/observability/getting-started) docs to set up Firebase Genkit Monitoring. The sources for this package are in the main [Genkit](https://github.com/firebase/genkit) repo. Please file issues and pull requests against that repo.