Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a viewport UI widget #17253

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Conversation

chompaa
Copy link
Member

@chompaa chompaa commented Jan 9, 2025

Objective

Add a viewport widget. Though I've marked this as a draft, I'm happy to receive reviews :)

Solution

  • Add a new Viewport component with field camera that points to the camera to render.
  • Add viewport_picking to pass pointer inputs from other pointers to the viewport's pointer.
    • Notably, this is somewhat functionally different from the viewport widget in the editor prototype, which just moves the pointer's location onto the render target. Viewport widgets have their own pointers.
    • Pointer inputs have a delay of one frame through the widget. This can be avoided but will incur a one frame lag on the location instead (see Discord discussion here).
    • Care is taken to handle dragging in and out of viewports.
  • Add an on_viewport_added function to spawn an ImageNode and PointerId for the viewport entity.
    • ImageNode is spawned as a child of the viewport entity.
  • Add update_viewport_render_target_size to update the viewport's render target's size if the node size changes.
  • Feature gate picking-related viewport items behind bevy_ui_picking_backend (subject to change).

Testing

I've been using an example I made to test the widget:

Code
//! A simple scene to demonstrate spawning a viewport widget. The example will demonstrate how to
//! pick "through" this widget.

use bevy::picking::pointer::PointerInteraction;
use bevy::prelude::*;

use bevy::{
    image::{TextureFormatPixelInfo, Volume},
    prelude::*,
    ui::widget::Viewport,
    window::PrimaryWindow,
};
use bevy_render::{
    camera::RenderTarget,
    render_resource::{
        Extent3d, TextureDescriptor, TextureDimension, TextureFormat, TextureUsages,
    },
};

fn main() {
    App::new()
        .add_plugins((DefaultPlugins, MeshPickingPlugin))
        .add_systems(Startup, test)
        .add_systems(Update, draw_mesh_intersections)
        .run();
}

#[derive(Component, Reflect, Debug)]
#[reflect(Component)]
struct Shape;

fn test(
    mut commands: Commands,
    window: Query<&Window, With<PrimaryWindow>>,
    mut images: ResMut<Assets<Image>>,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    // Spawn a UI camera
    commands.spawn(Camera2d::default());

    // Set up an texture for the 3D camera to render to
    let window = window.get_single().unwrap();
    let window_size = window.physical_size();
    let size = Extent3d {
        width: window_size.x,
        height: window_size.y,
        ..default()
    };
    let format = TextureFormat::Bgra8UnormSrgb;
    let image = Image {
        data: vec![0; size.volume() * format.pixel_size()],
        texture_descriptor: TextureDescriptor {
            label: None,
            size,
            dimension: TextureDimension::D2,
            format,
            mip_level_count: 1,
            sample_count: 1,
            usage: TextureUsages::TEXTURE_BINDING
                | TextureUsages::COPY_DST
                | TextureUsages::RENDER_ATTACHMENT,
            view_formats: &[],
        },
        ..default()
    };
    let image_handle = images.add(image);

    // Spawn the 3D camera
    let camera = commands
        .spawn((
            Camera3d::default(),
            Camera {
                // Render this camera before our UI camera
                order: -1,
                target: RenderTarget::Image(image_handle.clone().into()),
                ..default()
            },
        ))
        .id();

    // Spawn something for the 3D camera to look at
    commands
        .spawn((
            Mesh3d(meshes.add(Cuboid::new(5.0, 5.0, 5.0))),
            MeshMaterial3d(materials.add(Color::WHITE)),
            Transform::from_xyz(0.0, 0.0, -10.0),
            Shape,
        ))
        // We can observe pointer events on our objects as normal, the
        // `bevy::ui::widgets::viewport_picking` system will take care of ensuring our viewport
        // clicks pass through
        .observe(on_drag_cuboid);

    // Spawn our viewport widget
    commands
        .spawn((
            Node {
                position_type: PositionType::Absolute,
                top: Val::Px(50.0),
                left: Val::Px(50.0),
                width: Val::Px(200.0),
                height: Val::Px(200.0),
                border: UiRect::all(Val::Px(5.0)),
                ..default()
            },
            BorderColor(Color::WHITE),
            Viewport::new(camera),
        ))
        .observe(on_drag_viewport);
}

fn on_drag_viewport(drag: Trigger<Pointer<Drag>>, mut node_query: Query<&mut Node>) {
    if matches!(drag.button, PointerButton::Secondary) {
        let mut node = node_query.get_mut(drag.target()).unwrap();

        if let (Val::Px(top), Val::Px(left)) = (node.top, node.left) {
            node.left = Val::Px(left + drag.delta.x);
            node.top = Val::Px(top + drag.delta.y);
        };
    }
}

fn on_drag_cuboid(drag: Trigger<Pointer<Drag>>, mut transform_query: Query<&mut Transform>) {
    if matches!(drag.button, PointerButton::Primary) {
        let mut transform = transform_query.get_mut(drag.target()).unwrap();
        transform.rotate_y(drag.delta.x * 0.02);
        transform.rotate_x(drag.delta.y * 0.02);
    }
}

fn draw_mesh_intersections(
    pointers: Query<&PointerInteraction>,
    untargetable: Query<Entity, Without<Shape>>,
    mut gizmos: Gizmos,
) {
    for (point, normal) in pointers
        .iter()
        .flat_map(|interaction| interaction.iter())
        .filter_map(|(entity, hit)| {
            if !untargetable.contains(*entity) {
                hit.position.zip(hit.normal)
            } else {
                None
            }
        })
    {
        gizmos.arrow(point, point + normal.normalize() * 0.5, Color::WHITE);
    }
}

Showcase

2025-01-09.00-32-30.mp4

To do

  • Documentation
  • Despawn camera when Viewport is removed.
  • Feature flag feedback
  • Example (?)

Open Questions

  • Not sure whether the entire widget should be feature gated behind bevy_ui_picking_backend or not? I chose a partial approach since maybe someone will want to use the widget without any picking being involved.
  • Maybe the NodeImage for the viewport shouldn't be automatically added?
    • I decided it should be a child so that styling on the original entity is easier, but am open to change.

@chompaa chompaa added C-Feature A new feature, making something new possible D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes A-Picking Pointing at and selecting objects of all sorts labels Jan 9, 2025
@chompaa chompaa changed the title Add viewport widget Add a viewport UI widget Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Picking Pointing at and selecting objects of all sorts C-Feature A new feature, making something new possible D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant