Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a11y tree #198

Merged
merged 2 commits into from
Nov 22, 2024
Merged

Fix a11y tree #198

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1a39d2a
Revert "fix: rebuild a11y tree when rebuilding widget tree"
wash2 Nov 22, 2024
2b8baa6
fix: send a11y tree after actions
wash2 Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 101 additions & 83 deletions winit/src/program.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
use crate::futures::{Executor, Runtime};
use crate::graphics;
use crate::graphics::{compositor, Compositor};
use crate::platform_specific;

Check warning on line 30 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused import: `crate::platform_specific`

Check warning on line 30 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused import: `crate::platform_specific`

Check warning on line 30 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused import: `crate::platform_specific`

Check warning on line 30 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused import: `crate::platform_specific`
use crate::runtime::user_interface::{self, UserInterface};
use crate::runtime::Debug;
use crate::runtime::{self, Action, Task};
Expand Down Expand Up @@ -379,8 +379,8 @@

#[cfg(target_arch = "wasm32")]
{
use winit::platform::web::WindowExtWebSys;

Check failure on line 382 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 382 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 382 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 382 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowExtWebSys`
self.canvas = window.canvas();

Check failure on line 383 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `canvas` found for struct `std::sync::Arc<dyn winit::window::Window>` in the current scope

Check failure on line 383 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `canvas` found for struct `std::sync::Arc<dyn winit::window::Window>` in the current scope

Check failure on line 383 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `canvas` found for struct `std::sync::Arc<dyn winit::window::Window>` in the current scope

Check failure on line 383 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `canvas` found for struct `std::sync::Arc<dyn winit::window::Window>` in the current scope
}

let finish_boot = async move {
Expand Down Expand Up @@ -496,14 +496,14 @@

#[cfg(target_arch = "wasm32")]
let window_attributes = {
use winit::platform::web::WindowAttributesExtWebSys;

Check failure on line 499 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowAttributesExtWebSys`

Check failure on line 499 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowAttributesExtWebSys`

Check failure on line 499 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowAttributesExtWebSys`

Check failure on line 499 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowAttributesExtWebSys`
window_attributes
.with_canvas(self.canvas.take())

Check failure on line 501 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `with_canvas` found for struct `WindowAttributes` in the current scope

Check failure on line 501 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `with_canvas` found for struct `WindowAttributes` in the current scope

Check failure on line 501 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `with_canvas` found for struct `WindowAttributes` in the current scope

Check failure on line 501 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `with_canvas` found for struct `WindowAttributes` in the current scope
};

log::info!("Window attributes for id `{id:#?}`: {window_attributes:#?}");

let window = Arc::from(

Check failure on line 506 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

type annotations needed for `std::sync::Arc<_, _>`

Check failure on line 506 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

type annotations needed for `std::sync::Arc<_, _>`

Check failure on line 506 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

type annotations needed for `std::sync::Arc<_, _>`

Check failure on line 506 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

type annotations needed for `std::sync::Arc<_, _>`
event_loop
.create_window(window_attributes)
.expect("Create window"),
Expand All @@ -511,7 +511,7 @@

#[cfg(target_arch = "wasm32")]
{
use winit::platform::web::WindowExtWebSys;

Check failure on line 514 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 514 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 514 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowExtWebSys`

Check failure on line 514 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::WindowExtWebSys`

let canvas = window
.canvas()
Expand Down Expand Up @@ -631,8 +631,8 @@

#[cfg(target_arch = "wasm32")]
{
use winit::platform::web::EventLoopExtWebSys;

Check failure on line 634 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::EventLoopExtWebSys`

Check failure on line 634 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unresolved import `winit::platform::web::EventLoopExtWebSys`

Check failure on line 634 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::EventLoopExtWebSys`

Check failure on line 634 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unresolved import `winit::platform::web::EventLoopExtWebSys`
let _ = event_loop.spawn_app(runner);

Check failure on line 635 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `spawn_app` found for struct `EventLoop` in the current scope

Check failure on line 635 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

no method named `spawn_app` found for struct `EventLoop` in the current scope

Check failure on line 635 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `spawn_app` found for struct `EventLoop` in the current scope

Check failure on line 635 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

no method named `spawn_app` found for struct `EventLoop` in the current scope

Ok(())
}
Expand Down Expand Up @@ -694,7 +694,7 @@
boot: oneshot::Receiver<Boot<C>>,
mut event_receiver: mpsc::UnboundedReceiver<Event<P::Message>>,
mut control_sender: mpsc::UnboundedSender<Control>,
display_handle: OwnedDisplayHandle,

Check warning on line 697 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused variable: `display_handle`

Check warning on line 697 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused variable: `display_handle`

Check warning on line 697 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused variable: `display_handle`

Check warning on line 697 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused variable: `display_handle`
is_daemon: bool,
) where
P: Program + 'static,
Expand All @@ -706,7 +706,7 @@

let Boot {
mut compositor,
is_wayland,

Check warning on line 709 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused variable: `is_wayland`

Check warning on line 709 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unused variable: `is_wayland`

Check warning on line 709 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused variable: `is_wayland`

Check warning on line 709 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unused variable: `is_wayland`
} = boot.await.expect("Receive boot");

let mut platform_specific_handler =
Expand Down Expand Up @@ -802,6 +802,7 @@
let Some(event) = event else {
break;
};
let mut rebuild_a11y_tree = false;

Check warning on line 805 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

variable `rebuild_a11y_tree` is assigned to, but never used

Check warning on line 805 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

variable `rebuild_a11y_tree` is assigned to, but never used

Check warning on line 805 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

variable `rebuild_a11y_tree` is assigned to, but never used

Check warning on line 805 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

variable `rebuild_a11y_tree` is assigned to, but never used

match event {
Event::StartDnd => {
Expand Down Expand Up @@ -1097,6 +1098,7 @@
is_window_opening = false;
}
Event::UserEvent(action) => {
rebuild_a11y_tree = true;

Check warning on line 1101 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

value assigned to `rebuild_a11y_tree` is never read

Check warning on line 1101 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

value assigned to `rebuild_a11y_tree` is never read
let exited = run_action(
action,
&program,
Expand Down Expand Up @@ -1130,6 +1132,18 @@
}
}
Event::Winit(window_id, event) => {
#[cfg(feature = "a11y")]
{
if let Some((id, window)) =
window_manager.get_mut_alias(window_id)
{
if let Some(Some((_, adapter))) =
a11y_enabled.then(|| adapters.get_mut(&id))
{
adapter.process_event(window.raw.as_ref(), &event);
};
}
}
match event {
event::WindowEvent::RedrawRequested => {
let Some((id, window)) =
Expand Down Expand Up @@ -1235,7 +1249,7 @@
{
let logical_size = window.state.logical_size();
debug.layout_started();
let ui = user_interfaces
let mut ui = user_interfaces

Check warning on line 1252 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

variable does not need to be mutable

Check warning on line 1252 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

variable does not need to be mutable
.remove(&id)
.expect("Remove user interface")
.relayout(logical_size, &mut window.renderer);
Expand Down Expand Up @@ -1413,7 +1427,7 @@
}
}
}
_ => {}

Check warning on line 1430 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unreachable pattern

Check warning on line 1430 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

unreachable pattern

Check warning on line 1430 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unreachable pattern

Check warning on line 1430 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / wasm 

unreachable pattern
}
}
Event::AboutToWait => {
Expand Down Expand Up @@ -1565,17 +1579,14 @@

window.request_redraw();
}
rebuild_a11y_tree = true;

Check warning on line 1582 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

value assigned to `rebuild_a11y_tree` is never read

Check warning on line 1582 in winit/src/program.rs

View workflow job for this annotation

GitHub Actions / web 

value assigned to `rebuild_a11y_tree` is never read

user_interfaces = ManuallyDrop::new(build_user_interfaces(
&program,
&mut debug,
&mut window_manager,
cached_interfaces,
&mut clipboard,
#[cfg(feature = "a11y")]
a11y_enabled,
#[cfg(feature = "a11y")]
&mut adapters,
));

if actions > 0 {
Expand Down Expand Up @@ -1718,6 +1729,7 @@
}
#[cfg(feature = "a11y")]
Event::Accessibility(id, e) => {
rebuild_a11y_tree = true;
match e.action {
iced_accessibility::accesskit::Action::Focus => {
// TODO send a command for this
Expand Down Expand Up @@ -1749,6 +1761,89 @@
// log ignored events?
}
}
#[cfg(feature = "a11y")]
{
use iced_accessibility::{
accesskit::{NodeBuilder, NodeId, Role, Tree, TreeUpdate},
A11yId, A11yNode, A11yTree,
};
if !a11y_enabled || !rebuild_a11y_tree {
continue;
}

for id in window_manager.ids() {
let Some((a11y_id, adapter)) = adapters.get_mut(&id) else {
continue;
};
let Some(window) = window_manager.get(id) else {
continue;
};
let interface =
user_interfaces.get_mut(&id).expect("Get user interface");

// TODO cleanup duplication
let child_tree = interface.a11y_nodes(window.state.cursor());
let mut root = NodeBuilder::new(Role::Window);
root.set_name(window.state.title.to_string());
let window_tree = A11yTree::node_with_child_tree(
A11yNode::new(root, *a11y_id),
child_tree,
);
let tree = Tree::new(NodeId(*a11y_id));

let focus = Arc::new(std::sync::Mutex::new(None));
let focus_clone = focus.clone();
let operation: Box<dyn Operation<()>> =
Box::new(operation::map(
Box::new(operation::focusable::find_focused()),
move |id| {
let mut guard = focus.lock().unwrap();
_ = guard.replace(id);
},
));
let mut current_operation = Some(operation);

while let Some(mut operation) = current_operation.take() {
interface.operate(&window.renderer, operation.as_mut());

match operation.finish() {
operation::Outcome::None => {}
operation::Outcome::Some(()) => {
break;
}
operation::Outcome::Chain(next) => {
current_operation = Some(next);
}
}
}
let mut guard = focus_clone.lock().unwrap();
let focus = guard
.take()
.map(|id| A11yId::Widget(id))
.filter(|f_id| window_tree.contains(f_id));
tracing::debug!(
"tree root: {:?}\nchildren: {:?}\nfocus: {:?}\n",
window_tree
.root()
.iter()
.map(|n| (n.node(), n.id()))
.collect::<Vec<_>>(),
window_tree
.children()
.iter()
.map(|n| (n.node(), n.id()))
.collect::<Vec<_>>(),
&focus,
);
let focus =
focus.map(|id| id.into()).unwrap_or_else(|| tree.root);
adapter.update_if_active(|| TreeUpdate {
nodes: window_tree.into(),
tree: Some(tree),
focus,
});
}
}
}

let _ = ManuallyDrop::into_inner(user_interfaces);
Expand Down Expand Up @@ -2220,11 +2315,6 @@
window_manager: &mut WindowManager<P, C>,
mut cached_user_interfaces: FxHashMap<window::Id, user_interface::Cache>,
clipboard: &mut Clipboard,
#[cfg(feature = "a11y")] a11y_enabled: bool,
#[cfg(feature = "a11y")] adapters: &mut HashMap<
window::Id,
(u64, iced_accessibility::accesskit_winit::Adapter),
>,
) -> FxHashMap<window::Id, UserInterface<'a, P::Message, P::Theme, P::Renderer>>
where
C: Compositor<Renderer = P::Renderer>,
Expand All @@ -2234,7 +2324,7 @@
.drain()
.filter_map(|(id, cache)| {
let window = window_manager.get_mut(id)?;
let mut interface = build_user_interface(
let interface = build_user_interface(
program,
cache,
&mut window.renderer,
Expand All @@ -2245,78 +2335,6 @@
window.prev_dnd_destination_rectangles_count,
clipboard,
);
#[cfg(feature = "a11y")]
{
use iced_accessibility::{
accesskit::{NodeBuilder, NodeId, Role, Tree, TreeUpdate},
A11yId, A11yNode, A11yTree,
};
if let Some(Some((a11y_id, adapter))) =
a11y_enabled.then(|| adapters.get_mut(&id))
{
// TODO cleanup duplication
let child_tree =
interface.a11y_nodes(window.state.cursor());
let mut root = NodeBuilder::new(Role::Window);
root.set_name(window.state.title.to_string());
let window_tree = A11yTree::node_with_child_tree(
A11yNode::new(root, *a11y_id),
child_tree,
);
let tree = Tree::new(NodeId(*a11y_id));

let focus = Arc::new(std::sync::Mutex::new(None));
let focus_clone = focus.clone();
let operation: Box<dyn Operation<()>> =
Box::new(operation::map(
Box::new(operation::focusable::find_focused()),
move |id| {
let mut guard = focus.lock().unwrap();
_ = guard.replace(id);
},
));
let mut current_operation = Some(operation);

while let Some(mut operation) = current_operation.take() {
interface.operate(&window.renderer, operation.as_mut());

match operation.finish() {
operation::Outcome::None => {}
operation::Outcome::Some(()) => {
break;
}
operation::Outcome::Chain(next) => {
current_operation = Some(next);
}
}
}
let mut guard = focus_clone.lock().unwrap();
let focus = guard.take().map(|id| A11yId::Widget(id));
tracing::debug!(
"focus: {:?}\ntree root: {:?}\n children: {:?}",
&focus,
window_tree
.root()
.iter()
.map(|n| (n.node().role(), n.id()))
.collect::<Vec<_>>(),
window_tree
.children()
.iter()
.map(|n| (n.node().role(), n.id()))
.collect::<Vec<_>>()
);
let focus = focus
.filter(|f_id| window_tree.contains(f_id))
.map(|id| id.into())
.unwrap_or_else(|| tree.root);
adapter.update_if_active(|| TreeUpdate {
nodes: window_tree.into(),
tree: Some(tree),
focus,
});
}
}

let dnd_rectangles = interface.dnd_rectangles(
window.prev_dnd_destination_rectangles_count,
Expand Down
Loading