Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[centerpose_decoder_node.NitrosSubscriber]: 10.0s passed while waiting to push a message entity (eid=1614) to the receiver camera_model_input #43

Open
Yeager-101 opened this issue Jul 3, 2024 · 19 comments

Comments

@Yeager-101
Copy link

When I try to use centerpose to start the node ros2 launch isaac_ros_centerpose isaac_ros_centerpose_tensor_rt.launch.py ​​model_file_path:=/home/nvidia/Chen/centerpose/bottle_DLA34.onnx engine_file_path:=/home/nvidia/Chen/centerpose/bottle.plan, the terminal appears the following problem:

[INFO] [launch]: All log files can be found below /home/nvidia/.ros/log/2024-07-03-22-03-07-667750-ubuntu-20821
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [20842]
[component_container_mt-1] [INFO] [1720015388.060679791] [centerpose_container]: Load Library: /opt/ros/humble/lib/libtensor_rt_node.so
[component_container_mt-1] [INFO] [1720015388.134159284] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1720015388.134249330] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1720015388.139843306] [centerpose_inference]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.140262690] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1720015388.143588356] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_gxf_helpers.so
[component_container_mt-1] [INFO] [1720015388.148251405] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_sight.so
[component_container_mt-1] [INFO] [1720015388.153292015] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_atlas.so
[component_container_mt-1] 2024-07-03 22:03:08.165 WARN  gxf/std/program.cpp@532: No GXF scheduler specified.
[component_container_mt-1] [INFO] [1720015388.166755923] [centerpose_inference]: [TensorRTNode] Set input data format to: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015388.166812210] [centerpose_inference]: [TensorRTNode] Set output data format to: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1720015388.166853394] [centerpose_inference]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.170892198] [centerpose_inference]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.170967813] [centerpose_inference]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1720015388.174443748] [centerpose_inference]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_message_compositor.so
[component_container_mt-1] [INFO] [1720015388.175484433] [centerpose_inference]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1720015388.178943248] [centerpose_inference]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1720015388.180159769] [centerpose_inference]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_tensor_rt.so
[component_container_mt-1] [INFO] [1720015388.181827578] [centerpose_inference]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.183663832] [centerpose_inference]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.200557117] [centerpose_inference]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.202033313] [centerpose_inference]: [NitrosPublisherSubscriberGroup] Pinning the component "inference/rx" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015388.202088896] [centerpose_inference]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1720015388.206588652] [centerpose_inference]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_inference' in container '/centerpose_container'
[component_container_mt-1] [INFO] [1720015388.207498747] [centerpose_container]: Load Library: /opt/ros/humble/lib/libresize_node.so
[component_container_mt-1] [INFO] [1720015388.227484966] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1720015388.227572133] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container_mt-1] [INFO] [1720015388.233267674] [centerpose_encoder.resize_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.234201161] [centerpose_encoder.resize_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.245175676] [centerpose_encoder.resize_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.245321529] [centerpose_encoder.resize_node]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_tensorops.so
[component_container_mt-1] [INFO] [1720015388.250240638] [centerpose_encoder.resize_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.254562573] [centerpose_encoder.resize_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.437379002] [centerpose_encoder.resize_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.456850383] [centerpose_encoder.resize_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_encoder/resize_node' in container 'centerpose_container'
[component_container_mt-1] [INFO] [1720015388.458160374] [centerpose_container]: Load Library: /opt/ros/humble/lib/libcenterpose_decoder_node.so
[component_container_mt-1] [INFO] [1720015388.461621174] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::centerpose::CenterPoseDecoderNode>
[component_container_mt-1] [INFO] [1720015388.461678069] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::centerpose::CenterPoseDecoderNode>
[component_container_mt-1] [INFO] [1720015388.469589345] [centerpose_decoder_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.470258357] [centerpose_decoder_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.473979727] [centerpose_decoder_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.474149644] [centerpose_decoder_node]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_centerpose.so
[component_container_mt-1] [INFO] [1720015388.479528520] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_isaac_messages.so
[component_container_mt-1] [INFO] [1720015388.480724977] [centerpose_decoder_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.482627278] [centerpose_decoder_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.488595006] [centerpose_decoder_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.496111602] [centerpose_decoder_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_decoder_node' in container '/centerpose_container'
[component_container_mt-1] [INFO] [1720015388.496744966] [centerpose_inference]: Negotiating
[component_container_mt-1] [INFO] [1720015388.496809925] [centerpose_inference]: Could not negotiate
[component_container_mt-1] [INFO] [1720015388.497189470] [centerpose_container]: Load Library: /opt/ros/humble/lib/libcrop_node.so
[component_container_mt-1] [INFO] [1720015388.500545631] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::CropNode>
[component_container_mt-1] [INFO] [1720015388.500601470] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::CropNode>
[component_container_mt-1] [INFO] [1720015388.508742662] [centerpose_encoder.crop_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.510135053] [centerpose_encoder.crop_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.522545765] [centerpose_encoder.crop_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.522759425] [centerpose_encoder.crop_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.525807688] [centerpose_encoder.crop_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.569600855] [centerpose_encoder.crop_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.577394598] [centerpose_encoder.crop_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_encoder/crop_node' in container 'centerpose_container'
[component_container_mt-1] [INFO] [1720015388.578470705] [centerpose_encoder.resize_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.578667854] [centerpose_container]: Load Library: /opt/ros/humble/lib/libcenterpose_visualizer_node.so
[component_container_mt-1] [INFO] [1720015388.579378977] [centerpose_encoder.resize_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.582597732] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::centerpose::CenterPoseVisualizerNode>
[component_container_mt-1] [INFO] [1720015388.582641956] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::centerpose::CenterPoseVisualizerNode>
[component_container_mt-1] [INFO] [1720015388.583243608] [centerpose_encoder.crop_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.583648593] [centerpose_encoder.crop_node]: Could not negotiate
[component_container_mt-1] [INFO] [1720015388.583667632] [centerpose_encoder.crop_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.583676944] [centerpose_encoder.crop_node]: Could not negotiate
[component_container_mt-1] [INFO] [1720015388.589543395] [centerpose_visualizer_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.590347924] [centerpose_visualizer_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.594073198] [centerpose_visualizer_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.594286314] [centerpose_visualizer_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.595549107] [centerpose_visualizer_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.599765380] [centerpose_visualizer_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.609651212] [centerpose_visualizer_node]: [NitrosNode] Starting negotiation...
[component_container_mt-1] [INFO] [1720015388.610332991] [centerpose_decoder_node]: Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_visualizer_node' in container '/centerpose_container'
[component_container_mt-1] [INFO] [1720015388.610584090] [centerpose_inference]: Negotiating
[component_container_mt-1] [INFO] [1720015388.610644889] [centerpose_inference]: Could not negotiate
[component_container_mt-1] [INFO] [1720015388.610747543] [centerpose_container]: Load Library: /opt/ros/humble/lib/libimage_format_converter_node.so
[component_container_mt-1] [INFO] [1720015388.613200553] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[component_container_mt-1] [INFO] [1720015388.613249321] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ImageFormatConverterNode>
[component_container_mt-1] [INFO] [1720015388.621808649] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.622729816] [centerpose_encoder.image_format_converter_node]: [ImageFormatConverterNode] Set output data format to: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015388.622833366] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.633143605] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.633319826] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.634737976] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.708973966] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.723080455] [centerpose_encoder.image_format_converter_node]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015388.726260652] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_encoder/image_format_converter_node' in container 'centerpose_container'
[component_container_mt-1] [INFO] [1720015388.727091516] [centerpose_encoder.crop_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.727437910] [centerpose_encoder.resize_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.727795791] [centerpose_encoder.resize_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.728081418] [centerpose_encoder.image_format_converter_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.728104426] [centerpose_encoder.image_format_converter_node]: Could not negotiate
[component_container_mt-1] [INFO] [1720015388.729308307] [centerpose_container]: Load Library: /opt/ros/humble/lib/libimage_to_tensor_node.so
[component_container_mt-1] [ERROR] [1720015388.730105060] [centerpose_container]: Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99
[component_container_mt-1] [INFO] [1720015388.755699687] [centerpose_container]: Load Library: /opt/ros/humble/lib/libimage_tensor_normalize_node.so
[component_container_mt-1] [ERROR] [1720015388.756321019] [centerpose_container]: Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99
[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'normalize_node' of type 'nvidia::isaac_ros::dnn_inference::ImageTensorNormalizeNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99
[component_container_mt-1] [INFO] [1720015388.759578942] [centerpose_container]: Load Library: /opt/ros/humble/lib/libinterleaved_to_planar_node.so
[component_container_mt-1] [INFO] [1720015388.761357533] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::InterleavedToPlanarNode>
[component_container_mt-1] [INFO] [1720015388.761413756] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::InterleavedToPlanarNode>
[component_container_mt-1] [INFO] [1720015388.768329627] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.769192267] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.774981919] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.775127708] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.776727038] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.781818655] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.785343230] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_encoder/interleaved_to_planar_node' in container 'centerpose_container'
[component_container_mt-1] [INFO] [1720015388.788781565] [centerpose_container]: Load Library: /opt/ros/humble/lib/libreshape_node.so
[component_container_mt-1] [INFO] [1720015388.790756985] [centerpose_container]: Found class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::ReshapeNode>
[component_container_mt-1] [INFO] [1720015388.790803288] [centerpose_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::ReshapeNode>
[component_container_mt-1] [INFO] [1720015388.798695204] [centerpose_encoder.reshape_node]: [NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1720015388.799666514] [centerpose_encoder.reshape_node]: [NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1720015388.805439623] [centerpose_encoder.reshape_node]: [NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1720015388.805590468] [centerpose_encoder.reshape_node]: [NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1720015388.807184998] [centerpose_encoder.reshape_node]: [NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1720015388.812571362] [centerpose_encoder.reshape_node]: [NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1720015388.816337051] [centerpose_encoder.reshape_node]: [NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/centerpose_encoder/reshape_node' in container 'centerpose_container'
[component_container_mt-1] [INFO] [1720015388.817313257] [centerpose_encoder.interleaved_to_planar_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.886999477] [centerpose_encoder.reshape_node]: Negotiating
[component_container_mt-1] [INFO] [1720015388.887129874] [centerpose_encoder.interleaved_to_planar_node]: Negotiating
[component_container_mt-1] [INFO] [1720015389.207228065] [centerpose_inference]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.207323615] [centerpose_inference]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.207349182] [centerpose_inference]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.207360798] [centerpose_inference]: [NitrosPublisher] Use only the compatible publisher: topic_name="/tensor_sub", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.207371966] [centerpose_inference]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.207684952] [centerpose_inference]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.212375809] [centerpose_inference]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/DYQQERNXCW/DYQQERNXCW.yaml"
[component_container_mt-1] [INFO] [1720015389.212443327] [centerpose_inference]: [NitrosNode] Loading application
[component_container_mt-1] 2024-07-03 22:03:09.214 WARN  gxf/std/yaml_file_loader.cpp@1077: Using unregistered parameter 'dev_id' in component 'stream'.
[component_container_mt-1] [INFO] [1720015389.214923697] [centerpose_inference]: In TensorRTNode postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.324162365] [centerpose_inference]: Read tensor shape information from TRT Model Engine: /home/nvidia/Chen/centerpose/bottle.plan
[component_container_mt-1] [INFO] [1720015389.324370713] [centerpose_inference]: Tensors 6400 bytes, num outputs 40 x tensors per output 7 = 280 blocks
[component_container_mt-1] [INFO] [1720015389.324534454] [centerpose_inference]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.328163090] [centerpose_inference]: [NitrosNode] Node was started
[component_container_mt-1] 2024-07-03 22:03:09.352 WARN  ./gxf/extensions/tensor_rt/tensor_rt_inference.cpp@155: TRT WARNING: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[component_container_mt-1] [INFO] [1720015389.457971262] [centerpose_encoder.resize_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.458060636] [centerpose_encoder.resize_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.458075324] [centerpose_encoder.resize_node]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.458088668] [centerpose_encoder.resize_node]: [NitrosPublisher] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.458098491] [centerpose_encoder.resize_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.458105883] [centerpose_encoder.resize_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1720015389.458184698] [centerpose_encoder.resize_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.458194842] [centerpose_encoder.resize_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.458221337] [centerpose_encoder.resize_node]: [NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component "sync/image_in" (type="nvidia::gxf::DoubleBufferReceiver") from "nitros_image_bgr8" to "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.458430869] [centerpose_encoder.resize_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.467704648] [centerpose_encoder.resize_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/ZQWGZUFNZS/ZQWGZUFNZS.yaml"
[component_container_mt-1] [INFO] [1720015389.467784903] [centerpose_encoder.resize_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.472776426] [centerpose_encoder.resize_node]: [ResizeNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.472947463] [centerpose_encoder.resize_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.496880456] [centerpose_decoder_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.496963527] [centerpose_decoder_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.496978535] [centerpose_decoder_node]: [NitrosPublisher] Use the negotiated data format: "nitros_detection3_d_array"
[component_container_mt-1] [INFO] [1720015389.496991334] [centerpose_decoder_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.497000038] [centerpose_decoder_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/tensor_sub", data_format="nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.497013094] [centerpose_decoder_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.497019270] [centerpose_decoder_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.497234338] [centerpose_decoder_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.540058467] [centerpose_decoder_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/RSACMKTBUR/RSACMKTBUR.yaml"
[component_container_mt-1] [INFO] [1720015389.540146626] [centerpose_decoder_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.578414328] [centerpose_encoder.crop_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.578501943] [centerpose_encoder.crop_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.578517942] [centerpose_encoder.crop_node]: [NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.578528886] [centerpose_encoder.crop_node]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.578538518] [centerpose_encoder.crop_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/centerpose_encoder/crop/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.578548374] [centerpose_encoder.crop_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.578589269] [centerpose_encoder.crop_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.578813073] [centerpose_encoder.crop_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] libEGL warning: DRI3: failed to query the version
[component_container_mt-1] libEGL warning: DRI2: failed to authenticate
[component_container_mt-1] [INFO] [1720015389.610331941] [centerpose_visualizer_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.610427043] [centerpose_visualizer_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.610443939] [centerpose_visualizer_node]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.610455651] [centerpose_visualizer_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/centerpose/image_visualized", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1720015389.610527745] [centerpose_visualizer_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.610539585] [centerpose_visualizer_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1720015389.610550433] [centerpose_visualizer_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_detection3_d_array"
[component_container_mt-1] [INFO] [1720015389.610559553] [centerpose_visualizer_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.610566209] [centerpose_visualizer_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/camera_info", data_format="nitros_camera_info"
[component_container_mt-1] [INFO] [1720015389.610748285] [centerpose_visualizer_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.612970452] [centerpose_visualizer_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/JKMGOUOCFY/JKMGOUOCFY.yaml"
[component_container_mt-1] [INFO] [1720015389.613056402] [centerpose_visualizer_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.618013526] [centerpose_encoder.crop_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/HEXQQQFLBM/HEXQQQFLBM.yaml"
[component_container_mt-1] [INFO] [1720015389.618082677] [centerpose_encoder.crop_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.632397610] [centerpose_encoder.resize_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.640420084] [centerpose_visualizer_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.643529242] [centerpose_visualizer_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.643562266] [centerpose_encoder.crop_node]: [CropNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.643583961] [centerpose_decoder_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.647268564] [centerpose_decoder_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.647411570] [centerpose_encoder.crop_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.654886022] [centerpose_encoder.crop_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.726975494] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.727059621] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.727073637] [centerpose_encoder.image_format_converter_node]: [NitrosPublisher] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.727084804] [centerpose_encoder.image_format_converter_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/centerpose_encoder/converted/image", data_format="nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.727097284] [centerpose_encoder.image_format_converter_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_image_rgb8"
[component_container_mt-1] [INFO] [1720015389.727332128] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.739456414] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/UVJSJWKDRW/UVJSJWKDRW.yaml"
[component_container_mt-1] [INFO] [1720015389.739538108] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.741757331] [centerpose_encoder.image_format_converter_node]: [ImageFormatConverterNode] postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.741884848] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.747288780] [centerpose_encoder.image_format_converter_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.786299124] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.786379091] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.786402002] [centerpose_encoder.interleaved_to_planar_node]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.786643822] [centerpose_encoder.interleaved_to_planar_node]: [NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1720015389.786660942] [centerpose_encoder.interleaved_to_planar_node]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/centerpose_encoder/normalized_tensor", data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.786856906] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.789989840] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/OGREIAFSSX/OGREIAFSSX.yaml"
[component_container_mt-1] [INFO] [1720015389.790054414] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.792294469] [centerpose_encoder.interleaved_to_planar_node]: In InterleavedToPlanarNode postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.792470401] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.799248515] [centerpose_encoder.interleaved_to_planar_node]: [NitrosNode] Node was started
[component_container_mt-1] [INFO] [1720015389.817108182] [centerpose_encoder.reshape_node]: [NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1720015389.817183573] [centerpose_encoder.reshape_node]: [NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1720015389.817199668] [centerpose_encoder.reshape_node]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.817219284] [centerpose_encoder.reshape_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1720015389.817450320] [centerpose_encoder.reshape_node]: [NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1720015389.820658164] [centerpose_encoder.reshape_node]: [NitrosNode] Wrote the final top level YAML graph to "/tmp/isaac_ros_nitros/graphs/NIMLREPFDX/NIMLREPFDX.yaml"
[component_container_mt-1] [INFO] [1720015389.820708403] [centerpose_encoder.reshape_node]: [NitrosNode] Loading application
[component_container_mt-1] [INFO] [1720015389.823513982] [centerpose_encoder.reshape_node]: In ReshapeNode postLoadGraphCallback().
[component_container_mt-1] [INFO] [1720015389.823713723] [centerpose_encoder.reshape_node]: [NitrosNode] Initializing and running GXF graph
[component_container_mt-1] [INFO] [1720015389.837260638] [centerpose_encoder.reshape_node]: [NitrosNode] Node was started
[component_container_mt-1] [WARN] [1720015405.465157381] [centerpose_visualizer_node.NitrosSubscriber]: 10.0s passed while waiting to push a message entity (eid=1298) to the receiver video_buffer_input
[component_container_mt-1] [WARN] [1720015405.811588983] [centerpose_decoder_node.NitrosSubscriber]: 10.0s passed while waiting to push a message entity (eid=1614) to the receiver camera_model_input
[component_container_mt-1] [WARN] [1720015405.820685199] [centerpose_visualizer_node.NitrosSubscriber]: 10.0s passed while waiting to push a message entity (eid=1602) to the receiver camera_model_input
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)

I don't know why this problem occurs. There is no image output in my "/centerpose/image_visualized" topic. My "/image" and "/camera_info" topics are normal.

@huihui6666
Copy link

I encounter the similar problem, do you solve it?

@Yeager-101
Copy link
Author

我遇到了类似的问题,你解决了吗?

After I changed the system to ubuntu20.04 and JetPack5 version, everything was normal. I don’t know why it can’t run normally on ubuntu22.04 and JetPack6.

@huihui6666
Copy link

我遇到了类似的问题,你解决了吗?

After I changed the system to ubuntu20.04 and JetPack5 version, everything was normal. I don’t know why it can’t run normally on ubuntu22.04 and JetPack6.

Thanks, although it is a really strange problem

@Yeager-101
Copy link
Author

我遇到了类似的问题,你解决了吗?

After I changed the system to ubuntu20.04 and JetPack5 version, everything was normal. I don’t know why it can’t run normally on ubuntu22.04 and JetPack6.

Thanks, although it is a really strange problem

I recently rebuilt the development environment of Isaac_ros and tried the centerpose algorithm again. This time I succeeded (successful within 10 minutes). If you need it, I have the following suggestions:

  1. Development environment ubuntu22.04 JetPack6.0
  2. It is not recommended to install the package in apt form. It is recommended to compile and install it in source code form, because I found that there seems to be some problems with the opencv version when installing in apt form
  3. After the source code is compiled, modify the corresponding path in the launch file, especially the model path
  4. If you have not modified the camera subscribed in the source code, please use ros2 run topic_tools relay ...Complete the mapping of /camera/color/image_raw and /camera/color/camera_info topics. Centerpose needs /image and /camera_info. If you don't want to use relay to remap topics, you can go to src and modify the two topics in the source code (both cpp files need to be modified). After modifying them to your own corresponding topics, modify /camera and /camera_info in the launch file to the topics you need. Re-colcon build --symlink-packages ... and you're done.
  5. Finally, source the install/setup.bash of the compiled package, and then start the node and you will find that everything is normal.
    Currently, I found an additional problem in my test. When multiple objects of the same type appear in the camera's field of view at the same time, the program will directly have a similar interruption problem. This is also the problem I am currently thinking about. I hope the above suggestions will be helpful to you.

@DavidePeron19
Copy link

Did you find any solution for the multiple objects situation? I was trying to run the example given after installing and got the same problem since there are 2 shoes example

@Yeager-101
Copy link
Author

Did you find any solution for the multiple objects situation? I was trying to run the example given after installing and got the same problem since there are 2 shoes example

Unfortunately, I haven't found a solution to the problem of multiple objects so far, but I have learned about the CenterPose algorithm itself. The algorithm does not seem to support the pose estimation of multiple objects. Of course, this is just my own judgment. If you want to solve this problem fundamentally, I think you can analyze it from the source code of CenterPose. I have been busy with other projects recently, so I have not further analyzed CenterPose. Regarding pose estimation, I finally used FoundationPose in my project. This algorithm has the best performance in terms of accuracy and stability that I have tested so far. Of course, if you are willing to continue to analyze the problem of the CenterPose algorithm, I look forward to your analysis and test conclusions.

@HappySamuel
Copy link

Hi @Yeager-101

I faced the similar issue as you. You mention of rebuilding using source code instead of apt installing. Do you mean i need to git clone dnn_inference to do so? Because my main error is on ImageToTensorNode ,

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Best
Samuel

@Yeager-101
Copy link
Author

Hi @Yeager-101

I faced the similar issue as you. You mention of rebuilding using source code instead of apt installing. Do you mean i need to git clone dnn_inference to do so? Because my main error is on ImageToTensorNode ,

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Best Samuel

What I mean is that the centerpose package can be downloaded using git clone... and then manually compiled and tested, and other required packages can be directly installed using apt, if I remember correctly.

@HappySamuel
Copy link

Hi @Yeager-101
I faced the similar issue as you. You mention of rebuilding using source code instead of apt installing. Do you mean i need to git clone dnn_inference to do so? Because my main error is on ImageToTensorNode ,

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Best Samuel

What I mean is that the centerpose package can be downloaded using git clone... and then manually compiled and tested, and other required packages can be directly installed using apt, if I remember correctly.

So you mean, even though ImageToTensorNode belong to dnn_inference also doesn't mind, just need centerpose to be git cloned and compiled ? Ok, i will go try it out.

Best,
Samuel

@Yeager-101
Copy link
Author

Hi @Yeager-101
I faced the similar issue as you. You mention of rebuilding using source code instead of apt installing. Do you mean i need to git clone dnn_inference to do so? Because my main error is on ImageToTensorNode ,

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Best Samuel

What I mean is that the centerpose package can be downloaded using git clone... and then manually compiled and tested, and other required packages can be directly installed using apt, if I remember correctly.

So you mean, even though ImageToTensorNode belong to dnn_inference also doesn't mind, just need centerpose to be git cloned and compiled ? Ok, i will go try it out.

Best, Samuel

What I mean is that you can try it this way. First, use apt to install all the function packages you need and their dependent function packages (including your centerPose function package), but you need to create a workspace in one of your folders, then git clone the Centerpose function package, and then compile it. After the compilation is complete, please modify the various paths under the launch file. Finally, please note that you should source your workspace source devel/setup.bash, and then roslaunch your centerPose node. I think there should be no more strange problems.

@HappySamuel
Copy link

HappySamuel commented Dec 12, 2024

Hi @Yeager-101

I found the solution to this kind of error of missing libcvcuda.so.0. It's due to missing of installing cvcuda.

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Solution to this is shown below

cd /tmp && \
    wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.5.0-beta/nvcv-lib-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    sudo dpkg -i nvcv-lib-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.5.0-beta/nvcv-dev-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    sudo dpkg -i nvcv-dev-0.5.0_beta_DP-cuda12-aarch64-linux.deb

https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/release-3.0/docker/Dockerfile.aarch64

Mainly depend on the release-3.x that you use

Best
Samuel

@Yeager-101
Copy link
Author

Hi @Yeager-101

I found the solution to this kind of error of missing libcvcuda.so.0. It's due to missing of installing cvcuda.

[ERROR] [launch_ros.actions.load_composable_nodes]: Failed to load node 'image_to_tensor' of type 'nvidia::isaac_ros::dnn_inference::ImageToTensorNode' in container 'centerpose_container': Failed to load library: Could not load library dlopen error: libcvcuda.so.0: cannot open shared object file: No such file or directory, at ./src/shared_library.c:99

Solution to this is shown below

cd /tmp && \
    wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.5.0-beta/nvcv-lib-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    sudo dpkg -i nvcv-lib-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    wget https://github.com/CVCUDA/CV-CUDA/releases/download/v0.5.0-beta/nvcv-dev-0.5.0_beta_DP-cuda12-aarch64-linux.deb && \
    sudo dpkg -i nvcv-dev-0.5.0_beta_DP-cuda12-aarch64-linux.deb

https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/release-3.0/docker/Dockerfile.aarch64

Mainly depend on the release-3.x that you use

Best Samuel

Congratulations on finding a solution to your problem. Sometimes CUDA, TensorRT, and Cudnn environments can cause a lot of errors. Although I didn’t encounter the problem you described when testing the package, I’m glad you can discuss the problem you encountered.

@czy1998916
Copy link

Hello, I have encountered a similar problem as you and would like to ask for your advice. If possible, could you please add a vx or qq! Thank you very much!

@Yeager-101
Copy link
Author

Hello, I have encountered a similar problem as you and would like to ask for your advice. If possible, could you please add a vx or qq! Thank you very much!

You can directly describe your problem here, so that it is easier for everyone to analyze the problem you encountered and better help others solve similar problems.

@czy1998916
Copy link

Hello, I have encountered a similar problem as you and would like to ask for your advice. If possible, could you please add a vx or qq! Thank you very much!

You can directly describe your problem here, so that it is easier for everyone to analyze the problem you encountered and better help others solve similar problems.

Okay, thank you very much for your prompt response!

Firstly, let me introduce my situation. I am using Ubuntu 22.04 and Jetpack 6.1 on Orin
I used the model I trained myself, and when I started isaac_ros_centerpose _tensor_rt. launch. py, there were no errors. There were no image information for/image and/enterpose/image_isualized. After reading your previous content, I realized that I probably didn't modify the image topic, but I'm not sure how to modify it specifically. I would like to ask you how to modify it!

@czy1998916
Copy link

There are two more questions

  1. What is the frame rate of your centerpose? I used ros2 launch isaac_ros_deamples isaac_ros_deamples.launch. py launch_fragments:= realsense_mono_rect,centerpose,centerpose_visualizer model_name:=centerpose_shoe model_repository_paths:=[${ISAAC_ROS_WS}/isaac_ros_assets/models] At the command line, the detected pose of the object appears to be twice the actual pose.
  2. I noticed that you are using a foundation pose and I have deployed it. However, I found that the target detection module, RT-DETR, does not provide model training for this module. How do you use foundation pose to estimate your own objects? What is your frame rate, accuracy error, and stability? Do you have tutorial posts?
    Thank you very much for sharing! I also look forward to receiving your reply!!

@Yeager-101
Copy link
Author

There are two more questions

  1. What is the frame rate of your centerpose? I used ros2 launch isaac_ros_deamples isaac_ros_deamples.launch. py launch_fragments:= realsense_mono_rect,centerpose,centerpose_visualizer model_name:=centerpose_shoe model_repository_paths:=[${ISAAC_ROS_WS}/isaac_ros_assets/models] At the command line, the detected pose of the object appears to be twice the actual pose.
  2. I noticed that you are using a foundation pose and I have deployed it. However, I found that the target detection module, RT-DETR, does not provide model training for this module. How do you use foundation pose to estimate your own objects? What is your frame rate, accuracy error, and stability? Do you have tutorial posts?
    Thank you very much for sharing! I also look forward to receiving your reply!!

I did not evaluate the frame rate of CenterPose, but the intuitive feeling is that it is less than 10FPS. As for how to use FoundationPose for pose estimation, there seem to be many reference projects on GitHub.

@Yeager-101
Copy link
Author

Yeager-101 commented Jan 13, 2025

Hello, I have encountered a similar problem as you and would like to ask for your advice. If possible, could you please add a vx or qq! Thank you very much!

You can directly describe your problem here, so that it is easier for everyone to analyze the problem you encountered and better help others solve similar problems.

Okay, thank you very much for your prompt response!

Firstly, let me introduce my situation. I am using Ubuntu 22.04 and Jetpack 6.1 on Orin I used the model I trained myself, and when I started isaac_ros_centerpose _tensor_rt. launch. py, there were no errors. There were no image information for/image and/enterpose/image_isualized. After reading your previous content, I realized that I probably didn't modify the image topic, but I'm not sure how to modify it specifically. I would like to ask you how to modify it!

Topic mapping is done using 'ros2 run topic_tools relay <source_topic> <destination_topic>', all you need to do is 'ros2 run topic_tools relay /camera/color/image_raw /image' and 'ros2 run topic_tools relay /camera/color/camera_info /camera_info' as I mentioned in my previous answer.

@czy1998916
Copy link

你好,我也遇到跟你类似的问题,想请教一下,如果可以的话,可以加个vx或者qq吗?非常感谢!

您可以在这里直接描述您的问题,这样大家更容易分析您遇到的问题,更好地帮助其他人解决类似的问题。

好的,非常感谢您的及时回复!
首先介绍一下我的情况,我在Orin上使用Ubuntu 22.04和Jetpack 6.1,用的是自己训练的模型,启动isaac_ros_centerpose_tensor_rt.launch.py​​时没有报错,/image和/enterpose/image_isualized都没有图片信息,看了你之前的内容,意识到自己很可能没有修改image这个主题,但是具体怎么修改我又不太清楚,想请教你该怎么修改!

主题映射是使用“ros2 run topic_tools reply <source_topic> <destination_topic>”完成的,您需要做的就是“ros2 run topic_tools reply /camera/color/image_raw /image”和“ros2 run topic_tools reply /camera/color/camera_info /camera_info”,正如我在之前的回答中提到的那样。

I encountered an error after changing the topic of image information,

[component_container_mt-1] 2025-01-13 14:22:54.547 ERROR /home/rx/workspaces/isaac_ros-dev/src/isaac_ros_dnn_inference/isaac_ros_gxf_extensions/gxf_isaac_tensor_rt/gxf/extensions/tensor_rt/tensor_rt_inference.cpp@543: Failed to retrieve Tensor input
[component_container_mt-1] 2025-01-13 14:22:54.547 ERROR gxf/std/entity_executor.cpp@596: Failed to tick codelet in entity: IYDUKTLKHF_inference code: GXF_FAILURE
[component_container_mt-1] 2025-01-13 14:22:54.599 WARN gxf/std/multi_thread_scheduler.cpp@343: Error while executing entity E309 named 'IYDUKTLKHF_inference': GXF_FAILURE
[component_container_mt-1] 2025-01-13 14:22:54.661 ERROR gxf/std/entity_executor.cpp@212: Entity with eid 309 not found!

May I ask what is the reason for this? Can you help me check my isacc_ros_centerpose _tensor_rt. launch. py file to see if it has been modified correctly

SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES

Copyright (c) 2021-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

SPDX-License-Identifier: Apache-2.0

import os

from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import ComposableNodeContainer
from launch_ros.descriptions import ComposableNode

def generate_launch_description():
"""Generate launch description for Centerpose."""
tensor_names = [
'bboxes',
'scores',
'kps',
'clses',
'obj_scale',
'kps_displacement_mean',
'kps_heatmap_mean',
]
tensor_names_str = '['
for tensor_name in tensor_names:
tensor_names_str += f'"{tensor_name}"'
if tensor_name != tensor_names[-1]:
tensor_names_str += ','
tensor_names_str += ']'

launch_args = [
    DeclareLaunchArgument(
        'input_image_width',
        default_value='640',
        description='The input image width'),
    DeclareLaunchArgument(
        'input_image_height',
        default_value='480',
        description='The input image height'),
    DeclareLaunchArgument(
        'network_image_width',
        default_value='512',
        description='The input image width that the network expects'),
    DeclareLaunchArgument(
        'network_image_height',
        default_value='512',
        description='The input image height that the network expects'),
    DeclareLaunchArgument(
        'encoder_image_mean',
        default_value='[0.408, 0.447, 0.47]',
        description='The mean for image normalization'),
    DeclareLaunchArgument(
        'encoder_image_stddev',
        default_value='[0.289, 0.274, 0.278]',
        description='The standard deviation for image normalization'),
    DeclareLaunchArgument(
        'model_file_path',
        default_value='/home/rx/workspaces/isaac_ros-dev/isaac_ros_assets/models/centerpose_shoe/1/model.onnx',
        description='The absolute file path to the ONNX file'),
    DeclareLaunchArgument(
        'engine_file_path',
        default_value='/home/rx/workspaces/isaac_ros-dev/isaac_ros_assets/models/centerpose_shoe/1/model.plan',
        description='The absolute file path to the TensorRT engine file'),
    DeclareLaunchArgument(
        'input_tensor_names',
        default_value='["input"]',
        description='A list of tensor names to bound to the specified input binding names'),
    DeclareLaunchArgument(
        'input_binding_names',
        default_value='["input"]',
        description='A list of input tensor binding names (specified by model)'),
    DeclareLaunchArgument(
        'input_tensor_formats',
        default_value='["nitros_tensor_list_nchw_rgb_f32"]',
        description='The nitros format of the input tensors'),
    DeclareLaunchArgument(
        'output_tensor_names',
        default_value=tensor_names_str,
        description='A list of tensor names to bound to the specified output binding names'),
    DeclareLaunchArgument(
        'output_binding_names',
        default_value=tensor_names_str,
        description='A  list of output tensor binding names (specified by model)'),
    DeclareLaunchArgument(
        'output_tensor_formats',
        default_value='["nitros_tensor_list_nhwc_rgb_f32"]',
        description='The nitros format of the output tensors'),
    DeclareLaunchArgument(
        'tensorrt_verbose',
        default_value='False',
        description='Whether TensorRT should verbosely log or not'),
    DeclareLaunchArgument(
        'force_engine_update',
        default_value='False',
        description='Whether TensorRT should update the TensorRT engine file or not'),
    DeclareLaunchArgument(
        'output_field_size',
        default_value='[128, 128]',
        description='Represents the size of the 2D keypoint decoding from the network output'),
    DeclareLaunchArgument(
        'cuboid_scaling_factor',
        default_value='1.0',
        description='Scales the cuboid used for calculating the size of the objects detected'),
    DeclareLaunchArgument(
        'score_threshold',
        default_value='0.3',
        description='The threshold for scores values to discard.'),
    DeclareLaunchArgument(
        'object_name',
        default_value='shoe',
        description='The name of the category instance that is being detected',
    ),
    DeclareLaunchArgument(
        'show_axes',
        default_value='True',
        description='Whether to show axes or not for visualization',
    ),
    DeclareLaunchArgument(
        'bounding_box_color',
        default_value='0x000000ff',
        description='The color of the bounding box for visualization',
    ),
]

# DNN Image Encoder parameters
input_image_width = LaunchConfiguration('input_image_width')
input_image_height = LaunchConfiguration('input_image_height')
network_image_width = LaunchConfiguration('network_image_width')
network_image_height = LaunchConfiguration('network_image_height')
encoder_image_mean = LaunchConfiguration('encoder_image_mean')
encoder_image_stddev = LaunchConfiguration('encoder_image_stddev')

# Tensor RT parameters
model_file_path = LaunchConfiguration('model_file_path')
engine_file_path = LaunchConfiguration('engine_file_path')
input_tensor_names = LaunchConfiguration('input_tensor_names')
input_binding_names = LaunchConfiguration('input_binding_names')
input_tensor_formats = LaunchConfiguration('input_tensor_formats')
output_tensor_names = LaunchConfiguration('output_tensor_names')
output_binding_names = LaunchConfiguration('output_binding_names')
output_tensor_formats = LaunchConfiguration('output_tensor_formats')
tensorrt_verbose = LaunchConfiguration('tensorrt_verbose')
force_engine_update = LaunchConfiguration('force_engine_update')

# Centerpose Decoder parameters
output_field_size = LaunchConfiguration('output_field_size')
cuboid_scaling_factor = LaunchConfiguration('cuboid_scaling_factor')
score_threshold = LaunchConfiguration('score_threshold')
object_name = LaunchConfiguration('object_name')

# Centerpose visualization parameters
show_axes = LaunchConfiguration('show_axes')
bounding_box_color = LaunchConfiguration('bounding_box_color')

centerpose_inference_node = ComposableNode(
    name='centerpose_inference',
    package='isaac_ros_tensor_rt',
    plugin='nvidia::isaac_ros::dnn_inference::TensorRTNode',
    parameters=[{
        'model_file_path': model_file_path,
        'engine_file_path': engine_file_path,
        'input_tensor_names': input_tensor_names,
        'input_binding_names': input_binding_names,
        'input_tensor_formats': input_tensor_formats,
        'output_tensor_names': output_tensor_names,
        'output_binding_names': output_binding_names,
        'output_tensor_formats': output_tensor_formats,
        'verbose': tensorrt_verbose,
        'force_engine_update': force_engine_update
    }])

centerpose_decoder_node = ComposableNode(
    name='centerpose_decoder_node',
    package='isaac_ros_centerpose',
    plugin='nvidia::isaac_ros::centerpose::CenterPoseDecoderNode',
    parameters=[{
        'output_field_size': output_field_size,
        'cuboid_scaling_factor': cuboid_scaling_factor,
        'score_threshold': score_threshold,
        'object_name': object_name,
    }],
)

centerpose_visualizer_node = ComposableNode(
    name='centerpose_visualizer_node',
    package='isaac_ros_centerpose',
    plugin='nvidia::isaac_ros::centerpose::CenterPoseVisualizerNode',
    parameters=[{
        'show_axes': show_axes,
        'bounding_box_color': bounding_box_color,
    }],
)

rclcpp_container = ComposableNodeContainer(
    name='centerpose_container',
    namespace='',
    package='rclcpp_components',
    executable='component_container_mt',
    composable_node_descriptions=[
        centerpose_inference_node, centerpose_decoder_node, centerpose_visualizer_node],
    output='screen',
)

encoder_dir = get_package_share_directory('isaac_ros_dnn_image_encoder')
centerpose_encoder_launch = IncludeLaunchDescription(
    PythonLaunchDescriptionSource(
        [os.path.join(encoder_dir, 'launch', 'dnn_image_encoder.launch.py')]
    ),
    launch_arguments={
        'input_image_width': input_image_width,
        'input_image_height': input_image_height,
        'network_image_width': network_image_width,
        'network_image_height': network_image_height,
        'image_mean': encoder_image_mean,
        'image_stddev': encoder_image_stddev,
        'attach_to_shared_component_container': 'True',
        'component_container_name': 'centerpose_container',
        'dnn_image_encoder_namespace': 'centerpose_encoder',
        'image_input_topic': '/image',
        'camera_info_input_topic': '/camera_info',
        'tensor_output_topic': '/tensor_pub',
    }.items(),
)

final_launch_container = launch_args + [rclcpp_container, centerpose_encoder_launch]
return LaunchDescription(final_launch_container)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants