docs(observability): Add advanced configuration, authentication, and telemetry collection docs

docs/_guides.yaml

@@ -73,6 +73,12 @@ toc:
   - heading: Observing AI workflows
   - title: Getting Started
     path: /docs/genkit/observability/getting-started
+  - title: Authentication
+    path: /docs/genkit/observability/authentication
+  - title: Advanced Configuration
+    path: /docs/genkit/observability/advanced-configuration
+  - title: Telemetry Collection
+    path: /docs/genkit/observability/telemetry-collection
   - title: Troubleshooting
     path: /docs/genkit/observability/troubleshooting
 
@@ -97,8 +103,6 @@ toc:
     path: /docs/genkit/templates/pgvector
   - title: Firebase
     path: /docs/genkit/plugins/firebase
-  - title: Google Cloud
-    path: /docs/genkit/plugins/google-cloud
 
   - break: true
   - title: Using Genkit with Next.js

docs/observability/advanced-configuration.md

@@ -0,0 +1,128 @@
+# Advanced Configuration
+
+This guide focuses on advanced configuration options for deployed features using Firebase Genkit 
+Monitoring. Detailed descriptions of each configuration option can be found in our 
+[JS API reference documentation](https://js.api.genkit.dev/interfaces/_genkit-ai_google-cloud.GcpTelemetryConfigOptions.html).
+
+This documentation will describe how to fine-tune which telemetry is collected, how often, and from what environments.
+
+## Default Configuration
+
+Firebase Genkit Monitoring provides default options, out of the box, to get you up and running quickly.
+
+  * [autoInstrumentation](https://opentelemetry.io/docs/zero-code/js/): `true`
+  * [autoInstrumentationConfig](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations):
+
+    ```
+    {
+      '@opentelemetry/instrumentation-dns': { enabled: false },
+    },
+    ```
+
+  * credentials: pulled from chosen [authentication strategy](./authentication.md)
+  * disableMetrics: `false`
+  * disableTraces: `false`
+  * disableLoggingInputAndOutput: `false`
+  * forceDevExport: `false`
+  * metricExportIntervalMillis: 5 minutes
+  * metricExportTimeoutMillis: 5 minutes
+  * projectId: pulled from [authentication strategy](./authentication.md)
+  * sampler: [AlwaysOnSampler](https://js.api.genkit.dev/interfaces/_genkit-ai_google-cloud.GcpTelemetryConfigOptions.html#sampler)
+
+## Export local telemetry
+
+To export telemetry when running locally set the `forceDevExport` option to `true`.
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({forceDevExport: true});
+```
+
+During development and testing, you can decrease latency by adjusting the export interval and/or timeout. 
+
+Note: you should not ship to production with these reduced values.
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({
+  forceDevExport: true,
+  metricExportIntervalMillis: 10_000, // 10 seconds
+  metricExportTimeoutMillis: 10_000 // 10 seconds
+});
+```
+
+## Adjust auto instrumentation
+
+The Firebase telemetry plugin will automatically collect traces and metrics for popular frameworks, via by OpenTelemetry [zero-code instrumentation](https://opentelemetry.io/docs/zero-code/js/).
+
+A full list of available instrumentations can be found in the [auto-instrumentations-node](https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/metapackages/auto-instrumentations-node/README.md#supported-instrumentations) documentation.
+
+To selectively disable or enable instrumentations that are eligible for auto instrumentation, update the `autoInstrumentationConfig` field:
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({
+  autoInstrumentationConfig: {
+    '@opentelemetry/instrumentation-fs': { enabled: false },
+    '@opentelemetry/instrumentation-dns': { enabled: false },
+    '@opentelemetry/instrumentation-net': { enabled: false },
+  }
+});
+```
+
+## Disable telemetry
+
+Firebase Genkit Monitoring leverages a combination of logging, tracing, and metrics to capture
+a holistic view of your Genkit interactions, however, you can also disable each of these elements
+independently if needed.
+
+### Disable input and output logging
+
+By default, the Firebase telemetry plugin will capture inputs and outputs for each Genkit feature and/or step. 
+
+To help you control how customer data is stored, you can disable the logging of input and output by adding the following to your configuration:
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({
+  disableLoggingInputAndOutput: true
+});
+```
+
+With this option set, input and output attributes will be redacted 
+in the Firebase Genkit Monitoring trace viewer and will be missing 
+from Google Cloud logging.
+
+### Disable metrics
+
+To disable metrics collection, add the following to your configuration:
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({
+  disableMetrics: true
+});
+```
+
+With this option set, you will no longer see stability metrics in the 
+Firebase Genkit Monitoring dashboard and will be missing from Google Cloud Metrics.
+
+### Disable traces
+
+To disable trace collection, add the following to your configuration:
+
+```typescript
+import { enableFirebaseTelemetry } from '@genkit-ai/firebase';
+
+enableFirebaseTelemetry({
+  disableTraces: true
+});
+```
+
+With this option set, you will no longer see traces in the Firebase Genkit Monitoring
+feature page, have access to the trace viewer, or see traces present in Google Cloud Tracing.

docs/observability/authentication.md

@@ -0,0 +1,141 @@
+# Authentication and authorization
+
+The Firebase telemetry plugin requires a Google Cloud project ID and application credentials. 
+
+If you don't have a Google Cloud project and account, you can set one up in the [Firebase Console](https://console.firebase.google.com/) or in the [Google Cloud Console](https://cloud.google.com). All Firebase project IDs are Google Cloud project IDs.
+
+## Enable APIs
+
+Prior to adding the plugin, make sure the following APIs are enabled for
+your project:
+
+- [Cloud Logging API](https://console.cloud.google.com/apis/library/logging.googleapis.com)
+- [Cloud Trace API](https://console.cloud.google.com/apis/library/cloudtrace.googleapis.com)
+- [Cloud Monitoring API](https://console.cloud.google.com/apis/library/monitoring.googleapis.com)
+
+These APIs should be listed in the
+[API dashboard](https://console.cloud.google.com/apis/dashboard) for your
+project.
+Click [here](https://support.google.com/googleapi/answer/6158841) to learn more
+about enabling and disabling APIs.
+
+## User Authentication
+
+To export telemetry from your local development environment to Firebase Genkit Monitoring, you will need to authenticate yourself with Google Cloud.
+
+The easiest way to authenticate as yourself is via the gcloud CLI, which will automatically make your credentials available to the framework via [Application Default Credentials (ADC)](https://cloud.google.com/docs/authentication/application-default-credentials).
+
+If you don't have the gcloud CLI installed, first follow the [installation instructions](https://cloud.google.com/sdk/docs/install#installation_instructions).
+
+1. Authenticate using the `gcloud` CLI:
+
+   ```posix-terminal
+   gcloud auth application-default login
+   ```
+
+2. Set your project ID
+
+   ```posix-terminal
+   gcloud config set project PROJECT_ID
+   ```
+
+## Deploy to Google Cloud
+
+If deploying your code to a Google Cloud environment (Cloud
+Functions, Cloud Run, etc), the project ID and credentials will be discovered
+automatically via
+[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc).
+
+You will need to apply the following roles to the service account that is
+running your code (i.e. 'attached service account') via the
+[IAM Console](https://console.cloud.google.com/iam-admin/iam):
+
+- `roles/monitoring.metricWriter`
+- `roles/cloudtrace.agent`
+- `roles/logging.logWriter`
+
+Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below.
+
+## Deploy outside of Google Cloud (with ADC)
+
+If possible, it is still recommended to leverage the
+[Application Default Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc)
+process to make credentials available to the plugin.
+
+Typically this involves generating a service account key/pair and deploying
+those credentials to your production environment.
+
+1. Follow the instructions to set up a
+   [service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating).
+
+2. Ensure the service account has the following roles:
+   - `roles/monitoring.metricWriter`
+   - `roles/cloudtrace.agent`
+   - `roles/logging.logWriter`
+
+3. Deploy the credential file to production (**do not** check into source code)
+
+4. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable as the path to
+   the credential file.
+
+    ```
+    GOOGLE_APPLICATION_CREDENTIALS = "path/to/your/key/file"
+    ```
+
+Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below.
+
+## Deploy outside of Google Cloud (without ADC)
+
+In some serverless environments, you may not be able to deploy a credential
+file.
+
+1. Follow the instructions to set up a 
+[service account key](https://cloud.google.com/iam/docs/keys-create-delete#creating).
+
+2. Ensure the service account has the following roles:
+    - `roles/monitoring.metricWriter`
+    - `roles/cloudtrace.agent`
+    - `roles/logging.logWriter`
+
+3. Download the credential file.
+
+4. Assign the contents of the credential file to the `GCLOUD_SERVICE_ACCOUNT_CREDS` environment variable as follows:
+
+```
+GCLOUD_SERVICE_ACCOUNT_CREDS='{
+  "type": "service_account",
+  "project_id": "your-project-id",
+  "private_key_id": "your-private-key-id",
+  "private_key": "your-private-key",
+  "client_email": "your-client-email",
+  "client_id": "your-client-id",
+  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
+  "token_uri": "https://accounts.google.com/o/oauth2/token",
+  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
+  "client_x509_cert_url": "your-cert-url"
+}'
+```
+
+Not sure which service account is the right one? See the [Find or create your service account](#find-or-create-your-service-account) section below.
+
+## Find or create your service account
+
+To find the appropriate service account:
+
+1. Navigate to the [service accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) in the Google Cloud Console
+2. Select your project
+3. Find the appropriate service account. Common default service accounts are as follows:
+
+  - Firebase functions & Cloud Run
+
+    <code><var>PROJECT ID</var>[email protected]</code>
+
+  - App Engine
+
+    <code><var>PROJECT ID</var>@appspot.gserviceaccount.com</code>
+
+  - App Hosting
+
+    <code>firebase-app-hosting-compute@<var>PROJECT ID</var>.iam.gserviceaccount.com</code>
+
+If you are deploying outside of the Google ecosystem or don't want to use a default service account, you can [create a service account](https://cloud.google.com/iam/docs/service-accounts-create#creating) in the Google Cloud console.