Runtime tensor error when trying to convert cpu model to tflite

[itzjac]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

WSL2

MediaPipe Tasks SDK version

0.10.18

Task name (e.g. Image classification, Gesture recognition etc.)

convert model

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

Runtime Erorr for generating cpu model

Describe the expected behaviour

convert model and generate a tflite file

Standalone code/steps you may have used to try to get what you need

Using the provided LLM inference example as found in github (text-to-text)

Other info / Complete Logs

Running the conversion using the gpu backend works and load on device (is super slow). cpu backend stops the process with the runtime error

RuntimeError: INTERNAL: ; RET_CHECK failure (external/odml/odml/infra/genai/inference/utils/xnn_utils/model_ckpt_util.cc:116) tensor

I tried different ubuntu versions, both generated same runtime error for the cpu backend and worked fine with gpu backend.

[itzjac]

The wsl 2 has a default installation ubuntu 24

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04.1 LTS
Release:        24.04
Codename:       noble

[kuaashish]

HI @itzjac,

Could you please share the complete script used for model conversion? Providing the full, standalone code would be very helpful in understanding and potentially reproducing the issue.

Thank you!!

[itzjac]

Could you please share the complete script used for model conversion? Providing the full, standalone code would be very helpful in understanding and potentially reproducing the issue.

Thank you!!

Hi @kuaashish !

Thanks for the follow up.
The script is like any other reference I could find on the web. https://github.com/google-ai-edge/mediapipe

from mediapipe.tasks.python.genai import converter
import os

def gemma_convert_config(backend):
    input_ckpt = '/home/me/gemma-2b-it/'
    vocab_model_file = '/home/me/gemma-2b-it/'
    output_dir = '/home/me/gemma-2b-it/intermediate/'
    output_tflite_file = f'/home/me/gemma-2b-it-{backend}.tflite'
    return converter.ConversionConfig(input_ckpt=input_ckpt, ckpt_format='safetensors', 
model_type='GEMMA_2B', backend=backend, output_dir=output_dir, combine_file_only=False, 
vocab_model_file=vocab_model_file, output_tflite_file=output_tflite_file)


config = gemma_convert_config("cpu")
converter.convert_checkpoint(config)

By changing the cpu to gpu, i can produce the tflite and run on device, is super slow though. I expect the CPu model would run normally as that's the general recommendation, correct?

[kuaashish]

Hi @itzjac,

Could you please confirm if you still require support for this issue or if it has been resolved on your end? If not, kindly provide the latest complete error code for further assistance.

Thank you!!

[itzjac]

Hi @kuaashish,

The issue is still present, the last error is shown at the beginning of the thread, no progress whatsoever trying to fix this issue.
Can you help me to confirm if this is general/expected error ?

Regards

[kuaashish]

Hi @itzjac,

Thank you for your input. It seems there is a bug on our end. Currently, the issue #5811 also appears in Colab when converting to 'All supported' in CPU mode. We are actively working on a fix and will update you once it is resolved.