Make comments on both Supports & Partially Supports required

[mgifford]

If a report says that it supports or partially supports, we should check that there is a comment to help justify that claim. It is not uncommon for ACRs to simply say "supports" and not include any comments.

This check should be simple and would do a lot to raise the bar for existing ACRs.

[dmundra]

@mgifford started looking at this and looks like the Drupal one will generate a lot of errors. We will need notes for all of them. Do we have those?

Invalid: level 'supports' for 'electronic-docs' in criteria '1.3.2' requires 'notes', level 'supports' for 'electronic-docs' in criteria '1.3.3' requires 'notes', level 'supports' for 'electronic-docs' in criteria '1.4.1' requires 'notes', level 'supports' for 'electronic-docs' in criteria '2.1.1' requires 'notes', level 'supports' for 'electronic-docs' in criteria '2.1.2' requires 'notes', level 'supports' for 'electronic-docs' in criteria '2.2.1' requires 'notes', level 'supports' for 'electronic-docs' in criteria '2.4.2' requires 'notes', level 'supports' for 'electronic-docs' in criteria '3.3.1' requires 'notes', level 'supports' for 'electronic-docs' in criteria '3.3.2' requires 'notes', level 'supports' for 'electronic-docs' in criteria '1.4.5' requires 'notes', level 'supports' for 'electronic-docs' in criteria '2.4.7' requires 'notes'

[dmundra]

@mgifford going with your theme of less restrictive now and more restrictive later. I think we should table this. The Moodle example is also throwing a lot of missing notes for Supports & Partially Supports adherence levels. What do you think?

[mgifford]

Yes.. I think this makes sense. Seems to be very common for this to be undocumented. Might be easier to introduce into the editor first.

Also, having notes with "" isn't very useful. It shoudl actually be a text description of why it suports.

[dmundra]

Yes.. I think this makes sense. Seems to be very common for this to be undocumented. Might be easier to introduce into the editor first.

Agreed. Moving this back to backlog.

Also, having notes with "" isn't very useful. It shoudl actually be a text description of why it supports.

Agreed. I did have code ready that checked for not defined and empty string.

[dmundra]

I pushed up a branch with these optional changes if desired.

[mitchellevan]

Definitely I've seen too many blank remarks in ACRs, but I'm not sure making all remarks required is the solution.

Even accessibility SMEs stumble over what to say sometimes. Suppose you have a native mobile app, and you've responsibly tested and carefully reported for 1.3.1 Info and Relationships, 4.1.2 Name, Role, Value, and all of the other most significant criteria. Now you're staring at 2.4.4 Link Purpose — is it an automatic pass for native code? Is it applicable for some apps but N/A for yours?

If a report writer leaves some remarks blank, it's not an optimal ACR. Yet as a report reader I could still deem the ACR trustworthy overall if it has a small number of blank remarks, and none of them in the heavy-hitters.

Conversely, I worry that making all remarks required would encourage guessing. This is not a multiple-choice test where guessing is encouraged; I want to know what you actually know.

If OpenACR does make all remarks required, then I would suggest prioritizing a robust set of instructions or boilerplate for remarks. Help report writers understand what kinds of remarks are expected for a variety of scenarios, including remarks for criteria that pass for various reasons.

Boilerplate could look like this. 1.3.3 Sensory Characteristics — examples of "Supports" remarks:

• The product does not provide instructions.
• Instructions in the product do not rely solely on sensory information for understanding. Example: [where do such instructions occur, and how do they meet this requirement?]

[mgifford]

Great comments @mitchellevan

We aren't requiring comments in OpenACR yet, but thinking about future use cases.

It may in fact be preferable to simply continue provide warnings like we presently do in the editor. Adding specific examples for each SC may also be beneficial. Authors do need all the help that they can get when producing these reports.

We are hoping to have a comparison tool in the future. That comparison tool could look at how many SC have Remarks & explanations for them. If authors know that the quality of the comments will be highlighted in the comparison tool, but might also be of interest.

With Drupal's image alt text, we made it required, but also that this could be over-ridden. We wanted to make it just more difficult to not fill in the alt text field, as it was to override it. Introducing friction can help shape author decision-making.