Revamping Bite Selection UI

[amalnanavati]

Currently, the bite selection UI shows buttons with the names of food items, and the user selects one of those buttons. This requires the perception system to be able to name the food items.

Instead, for the MVP, we'd like to switch to a system where an image of the plate is displayed, with one of the following options for UI:

1. The app shows bounding boxes around perceived food items as "buttons", and the user clicks on one of the buttons. The user's selection is then sent to the robot.
2. The user freely places a bounding box on the image, by clicking once to place the top-left corner and clicking again to place the bottom-right corner. That bounding box is then sent to the robot.
3. The user clicks a point on the image, and the perception algorithm segments the object that they clicked on. It then sends that object to the robot.

Part of this depends on the perception capabilities (e.g., can the perception system take a single point from a user click and segment the object that click was on?), part of it depends on robot capabilities (e.g., can the robot acquire a bite with an arbitrary bounding box the user places?), and part of it depends on user preferences (which of these UIs does the user prefer)? Therefore, we don't yet know which option we will go with. However, someone can still make progress on this by implementing all 3 UIs so we can later decide which works best for the technology stack and user preference. Further, once all 3 UIs are implemented, we could potentially run a small user study to identify which one(s) the user prefers.

[amalnanavati]

As mentioned by Raida, if the final bite selection UI involves users scrolling, then important buttons like "Done Eating" should appear both at the top and bottom so users don't have to scroll again.

However, ideally the UI should be designed to not require scrolling in the first place.

[Raidakarim]

Here is a slidedeck with some initial thoughts of mine. Amal @amalnanavati left feedback for improvement in comments on that.

I currently don't have the time to actively work on this issue. So, if anyone in the web app team wants to assign it to themselves, please feel free to. Otherwise, I can self-assign this issue to myself in the future when I am done with my current issues.

[amalnanavati]

I actually think we should keep this open until we implement the revamped UI into the actual app (and with the ROS dummy node(s)). The mock-ups for the user study is one crucial part of it, but I see this issue as also encompassing the final implementation.

[atharva-kashyap]

Tyler indicated that he and most users might prefer Option B. As such, we will be working on issue #44. So, I am closing this issue