fix(experiments): Avoid displaying results on draft experiments

[danielbachhuber]

See #27014

Changes

Prevents results from displaying on a draft experiment when you add or remove a metric.

Previously, adding or removing a metric would call updateExperiment() which progresses along to call loadMetricResults() and loadSecondaryMetricResults(), regardless of whether the experiment is started. The experiment results then erroneously displayed until you refreshed the page.

This new logic only fetches metric results when the experiment is started, and noops them otherwise.

Before

CleanShot.2025-01-08.at.13.16.21.mp4

After

CleanShot.2025-01-08.at.13.17.26.mp4

How did you test this code?

I verified experiment results no longer appear unexpectedly when adding or removing a metric on a draft experiment.

I also verified experiment results appear as expected when adding or removing a metric on a launched experiment.

[github-actions]

Size Change: 0 B

Total Size: 1.11 MB

ℹ️ View Unchanged

Filename	Size
frontend/dist/toolbar.js	1.11 MB

_{compressed-size-action}

[andehen]

Nice! 👍 This has confused me a couple of times.

[neilkakkar]

i think do the same at setExperimentStatsVersion ? Not sure if that can only happen after the experiment starts running though

[danielbachhuber]

i think do the same at setExperimentStatsVersion ? Not sure if that can only happen after the experiment starts running though

Yes, correct, and no, it can be used whenever. Good catch. setExperimentStatsVersion is only exposed for a few customers right now though, but for the sake of completeness: d1bc2a2

[neilkakkar]

do we depend on other follow up listeners running here? Or can we just not fire the actions at all?

[danielbachhuber]

do we depend on other follow up listeners running here? Or can we just not fire the actions at all?

I think this falls under "general refactor of experimentLogic".

Another oddity is that addSharedMetricToExperiment calls closePrimarySharedMetricModal, closeSecondarySharedMetricModal, and loadExperiment:

posthog/frontend/src/scenes/experiments/experimentLogic.tsx

Lines 904 to 918 in 5c0e29c

	addSharedMetricToExperiment: async ({ sharedMetricId, metadata }) => {
	const sharedMetricsIds = values.experiment.saved_metrics.map((sharedMetric) => ({
	id: sharedMetric.saved_metric,
	metadata,
	}))
	sharedMetricsIds.push({ id: sharedMetricId, metadata })
	await api.update(`api/projects/${values.currentProjectId}/experiments/${values.experimentId}`, {
	saved_metrics_ids: sharedMetricsIds,
	})
	actions.closePrimarySharedMetricModal()
	actions.closeSecondarySharedMetricModal()
	actions.loadExperiment()
	},

However, closePrimarySharedMetricModal and closeSecondarySharedMetricModal each call loadExperiment too:

posthog/frontend/src/scenes/experiments/experimentLogic.tsx

Lines 753 to 758 in 5c0e29c

	closePrimarySharedMetricModal: () => {
	actions.loadExperiment()
	},
	closeSecondarySharedMetricModal: () => {
	actions.loadExperiment()
	},

I think @jurajmajerik might have some ideas in mind, so I'd rather wait until he's back.