color issue for chart bars or chart lines

[sunilreddy0720]

We have this setting in JSON properties file at dashboard level and for some reasons it is not allowing us to save 2bit languages like Hindi, Chinese, Japanese etc.

[dosubot[bot]]

To resolve the issue of the system not allowing you to save 2-bit languages like Hindi, Chinese, Japanese, etc., when setting colors for chart bars or chart lines in a JSON properties file at the dashboard level, ensure that the JSON properties file correctly handles Unicode characters.

In JSON, represent Unicode characters using the \u escape sequence followed by the character's code point. For example, the Unicode code point for the character 'ह' (Hindi) is \u0939.

Here is an example of how you can set colors for chart bars or chart lines with labels in 2-bit languages:

{
  "label_colors": {
    "हिंदी": "#FF5733",  // Hindi
    "中文": "#33FF57",   // Chinese
    "日本語": "#3357FF"  // Japanese
  }
}

Ensure that your JSON properties file is saved with UTF-8 encoding to properly handle these characters. Additionally, make sure the label_colors property is correctly formatted and included in the json_metadata of the dashboard:

{
  "timed_refresh_immune_slices": [],
  "expanded_slices": {},
  "refresh_frequency": 0,
  "default_filters": "{}",
  "color_scheme": "supersetColors",
  "label_colors": {
    "हिंदी": "#FF5733",  // Hindi
    "中文": "#33FF57",   // Chinese
    "日本語": "#3357FF"  // Japanese
  },
  "filter_scopes": {
    "358": {
      "Country_Name": {"scope": ["ROOT_ID"], "immune": []},
      "Product_Category": {"scope": ["ROOT_ID"], "immune": []},
      "Clinical Stage": {"scope": ["ROOT_ID"], "immune": []}
    }
  }
}

This should help in resolving the issue with saving and displaying 2-bit languages in the dashboard ^[1][2].

To continue talking to Dosu, mention @dosu.

[sunilreddy0720]

Hi,
when i put this {
"label_colors": {
"हिंदी": "#FF5733", // Hindi
"中文": "#33FF57", // Chinese
"日本語": "#3357FF" // Japanese
}
} code in json and it's giving fatal error

[dosubot[bot]]

The JSON properties file is giving a fatal error because JSON does not support comments. The comments in the JSON file are causing the error.

To resolve this issue, you need to remove the comments from the JSON file. Here is the corrected JSON code:

{
  "label_colors": {
    "हिंदी": "#FF5733",
    "中文": "#33FF57",
    "日本語": "#3357FF"
  }
}

By removing the comments, the JSON file will be valid and should not cause any errors ^[1].

To continue talking to Dosu, mention @dosu.

[sunilreddy0720]

Hi,
i have removed the comments and added the code but still gives the fatal error.

[dosubot[bot]]

The fatal error might be due to improper encoding or other issues with the JSON file. Here are a few steps to ensure the JSON file is correctly formatted and encoded:

1. Ensure UTF-8 Encoding: Make sure the JSON file is saved with UTF-8 encoding. This is crucial for handling non-Latin characters.

2. Validate JSON: Use a JSON validator to ensure the JSON structure is correct. You can use online tools like JSONLint or the validate_json method provided by Apache Superset.

3. Check for Hidden Characters: Sometimes, hidden characters or incorrect line endings can cause issues. Ensure there are no hidden characters in your JSON file.

Here is the JSON code again for reference:

{
  "label_colors": {
    "हिंदी": "#FF5733",
    "中文": "#33FF57",
    "日本語": "#3357FF"
  }
}

If the issue persists, you can use the validate_json method in Apache Superset to check the correctness of the JSON configuration file:

def validate_json(value: Union[bytes, bytearray, str]) -> None:
    try:
        json.loads(value)
    except json.JSONDecodeError as ex:
        raise ValidationError("JSON not valid") from ex

This method will help identify any issues with the JSON structure ^[1].

To continue talking to Dosu, mention @dosu.

[sunilreddy0720]

Hi,
IS this achievable from code side.

[dosubot[bot]]

Yes, you can achieve this programmatically using the validate_json_metadata method in Apache Superset. Here is an example of how you can validate and apply the JSON metadata for setting colors, including labels in 2-bit languages like Hindi, Chinese, and Japanese:

1. Validate JSON Metadata: Ensure that the JSON metadata is valid using the validate_json_metadata method.
2. Handle Color Scheme: Apply the color scheme from the JSON metadata.
3. Handle Errors: Display appropriate error messages if the JSON metadata is invalid.

Here is a code snippet that demonstrates these steps:

import { validate_json_metadata } from 'superset/dashboards/schemas.py';

// validate currentJsonMetadata
let metadata;
try {
  validate_json_metadata(currentJsonMetadata);
  metadata = JSON.parse(currentJsonMetadata);
} catch (error) {
  addDangerToast(t('JSON metadata is invalid!'));
  return;
}

const copyMetadata = { ...metadata };
const colorNamespace = getColorNamespace(metadata?.color_namespace);

// color scheme in json metadata has precedence over selection
currentColorScheme = metadata?.color_scheme || colorScheme;

// remove information from user facing input
if (metadata?.shared_label_colors) {
  delete metadata.shared_label_colors;
}
if (metadata?.color_scheme_domain) {
  delete metadata.color_scheme_domain;
}

// only apply colors, the user has not saved yet
applyColors(copyMetadata, true);

currentJsonMetadata = jsonStringify(metadata);

onColorSchemeChange(currentColorScheme, {
  updateMetadata: false,
});

This code ensures that the JSON metadata is valid using the validate_json_metadata method and applies the color scheme specified in the metadata. If the JSON metadata is invalid, it displays an error message. This approach should handle labels in 2-bit languages like Hindi, Chinese, and Japanese as well ^[1][2].

To continue talking to Dosu, mention @dosu.

[sunilreddy0720]

Hi

Can we change json metadata in dashboard API, when modified its not getting save instead its giving json is invalid

[dosubot[bot]]

Yes, you can change JSON metadata in the dashboard API. The "invalid JSON" error occurs when the JSON metadata is not properly formatted. Specifically, the code checks if the JSON metadata starts with { and ends with } and attempts to parse it. If the JSON is not valid, it throws an error and displays a toast message indicating that the JSON metadata is invalid.

Here is the relevant code snippet that validates the JSON metadata:

let metadata;
try {
  if (
    !currentJsonMetadata.startsWith('{') ||
    !currentJsonMetadata.endsWith('}')
  ) {
    throw new Error();
  }
  metadata = JSON.parse(currentJsonMetadata);
} catch (error) {
  addDangerToast(t('JSON metadata is invalid!'));
  return;
}

To avoid the "invalid JSON" error, ensure that the JSON metadata you are modifying is correctly formatted as a valid JSON object. According to the Superset documentation, the json_metadata field is a string that represents a JSON object and is generated dynamically when saving or overwriting the dashboard. It is exposed for reference and for power users who may want to alter specific parameters. Make sure your JSON metadata adheres to this format and includes all necessary properties ^[1][2].

To continue talking to Dosu, mention @dosu.

[sunilreddy0720]

How to change label colors from dashboard data API ?

[dosubot[bot]]

To change label colors from the dashboard data API in Apache Superset, you can use the updateDashboardLabelsColor function. This function updates the color scheme and label colors in the dashboard metadata.

Here is the relevant code snippet:

const updateDashboardMetadata = async (id, metadata, dispatch) => {
  await SupersetClient.put({
    endpoint: `/api/v1/dashboard/${id}`,
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ json_metadata: JSON.stringify(metadata) }),
  });
  dispatch(dashboardInfoChanged({ metadata }));
};

export const updateDashboardLabelsColor = () => async (dispatch, getState) => {
  const {
    dashboardInfo: { id, metadata },
  } = getState();
  const categoricalSchemes = getCategoricalSchemeRegistry();
  const colorScheme = metadata?.color_scheme;
  const colorSchemeRegistry = categoricalSchemes.get(
    metadata?.color_scheme,
    true,
  );
  const defaultScheme = categoricalSchemes.defaultKey;
  const fallbackScheme = defaultScheme?.toString() || 'supersetColors';
  const colorSchemeDomain = metadata?.color_scheme_domain || [];

  try {
    const updatedMetadata = { ...metadata };
    let updatedScheme = metadata?.color_scheme;

    // Color scheme does not exist anymore, fallback to default
    if (colorScheme && !colorSchemeRegistry) {
      updatedScheme = fallbackScheme;
      updatedMetadata.color_scheme = updatedScheme;
      updatedMetadata.color_scheme_domain = getColorSchemeDomain(colorScheme);

      dispatch(setColorScheme(updatedScheme));
      // must re-apply colors from fresh labels color map
      applyColors(updatedMetadata, true);
    }

    // stored labels color map and applied might differ
    const isMapSynced = isLabelsColorMapSynced(metadata);
    if (!isMapSynced) {
      // re-apply a fresh labels color map
      applyColors(updatedMetadata, true);
      // pull and store the just applied labels color map
      updatedMetadata.shared_label_colors = getLabelsColorMapEntries();
    }

    await updateDashboardMetadata(id, updatedMetadata, dispatch);
  } catch (error) {
    dispatch(addDangerToast(t('There was an issue updating the label colors.')));
  }
};

This function retrieves the current dashboard metadata, checks and updates the color scheme if necessary, and applies the new label colors. Finally, it updates the dashboard metadata via the API ^[1].

Additionally, here are some utility functions from superset-frontend/src/utils/colorScheme.ts that are used in the process:

import {
  CategoricalColorNamespace,
  getCategoricalSchemeRegistry,
  getLabelsColorMap,
} from '@superset-ui/core';

export const getColorNamespace = (namespace?: string) => namespace || undefined;

export const getLabelsColorMapEntries = (): Record<string, string> => {
  const labelsColorMapInstance = getLabelsColorMap();
  const updatedLabelsColorMapEntries = Object.fromEntries(
    labelsColorMapInstance.getColorMap(),
  );
  return updatedLabelsColorMapEntries;
};

export const getColorSchemeDomain = (colorScheme: string) =>
  getCategoricalSchemeRegistry().get(colorScheme)?.colors || [];

export const isLabelsColorMapSynced = (
  metadata: Record<string, any>,
): boolean => {
  const currentLabelsColorMap = metadata?.shared_label_colors || {};
  const customLabelColors = metadata?.label_colors || {};
  const freshLabelsColorMap = getLabelsColorMap().getColorMap();
  const isSynced = Array.from(freshLabelsColorMap.entries()).every(
    ([label, color]) =>
      currentLabelsColorMap.hasOwnProperty(label) &&
      (currentLabelsColorMap[label] === color ||
        customLabelColors[label] !== undefined),
  );
  return isSynced;
};

export const resetColors = (color_namespace?: string) => {
  const labelsColorMapInstance = getLabelsColorMap();
  const categoricalNamespace = CategoricalColorNamespace.getNamespace(
    getColorNamespace(color_namespace),
  );
  categoricalNamespace.resetColors();
  labelsColorMapInstance.clear();
};

export const refreshLabelsColorMap = (
  namespace?: string,
  colorScheme?: string,
) => {
  const colorNameSpace = getColorNamespace(namespace);
  const categoricalNamespace =
    CategoricalColorNamespace.getNamespace(colorNameSpace);
  const labelsColorMapInstance = getLabelsColorMap();

  labelsColorMapInstance.updateColorMap(categoricalNamespace, colorScheme);
};

export const applyColors = (metadata: Record<string, any>, fresh = false) => {
  const colorNameSpace = getColorNamespace(metadata?.color_namespace);
  const categoricalNamespace =
    CategoricalColorNamespace.getNamespace(colorNameSpace);
  const colorScheme = metadata?.color_scheme;
  const customLabelColors = metadata?.label_colors || {};
  const labelsColorMap = metadata?.shared_label_colors || {};

  categoricalNamespace.resetColors();

  Object.keys(customLabelColors).forEach(label => {
    categoricalNamespace.setColor(label, customLabelColors[label]);
  });

  refreshLabelsColorMap(metadata?.color_namespace, colorScheme);

  const labelsColorMapEntries = fresh
    ? getLabelsColorMapEntries()
    : labelsColorMap;

  Object.keys(labelsColorMapEntries).forEach(label => {
    categoricalNamespace.setColor(label, labelsColorMapEntries[label]);
  });
};

These utility functions help manage the color schemes and label colors effectively ^[2].

Additionally, you can use Cypress tests to verify the changes in label colors. For example, the editDashboard function in the Cypress test suite can be used to open the dashboard in edit mode, and the selectColorScheme function can be used to select a color scheme. The applyChanges and saveChanges functions can be used to apply and save the changes, respectively. Here is an example of how you can use these functions in a test:

describe('Dashboard edit', () => {
  describe('Color consistency', () => {
    beforeEach(() => {
      visitResetTabbedDashboard();
    });

    after(() => {
      resetTabbedDashboard();
    });

    it('should apply same color to same labels with color scheme set', () => {
      openProperties();
      selectColorScheme('blueToGreen');
      applyChanges();
      saveChanges();

      // open nested tab
      openTab(1, 1);
      waitForChartLoad({
        name: 'Top 10 California Names Timeseries',
        viz: 'line',
      });

      // label Anthony
      cy.get(
        '[data-test-chart-name="Top 10 California Names Timeseries"] .line .nv-legend-symbol',
      )
        .first()
        .should('have.css', 'fill', 'rgb(50, 0, 167)');

      // open 2nd main tab
      openTab(0, 1);
      waitForChartLoad({ name: 'Trends', viz: 'line' });

      // label Anthony
      cy.get('[data-test-chart-name="Trends"] .line .nv-legend-symbol')
        .eq(2)
        .should('have.css', 'fill', 'rgb(50, 0, 167)');
    });
  });
});

This test ensures that the same color is applied to the same labels when a color scheme is set ^[3].

To continue talking to Dosu, mention @dosu.