Skip to content

Commit

Permalink
Removing BOMs and extra end of line spaces.
Browse files Browse the repository at this point in the history
Normalizing end of file lines.
Adjusting execute bit on non-executable files.
  • Loading branch information
david-waltermire committed Apr 13, 2021
1 parent 6f8e04e commit f44426e
Show file tree
Hide file tree
Showing 203 changed files with 6,127 additions and 6,154 deletions.
15 changes: 13 additions & 2 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@
* text=auto

# structured formats
*.css text eol=lf
*.dtd text eol=lf
*.ent text eol=lf
*.html text eol=lf
*.js text eol=lf
*.json text eol=lf
*.sch text eol=lf
*.scss text eol=lf
*.xml text eol=lf
*.xpl text eol=lf
*.xsd text eol=lf
*.xsl text eol=lf
*.sch text eol=lf
*.xspec text eol=lf
*.yaml text eol=lf
*.yml text eol=lf

Expand All @@ -17,7 +25,10 @@
*.sh text eol=lf
/build/ci-cd/config/* text eol=lf

# programming language files
*.py text eol=lf
*.ts text eol=lf

# other binary files
*.png binary
*.jpg binary

4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ We track our current work items using GitHub [project cards](https://github.com/
## Contribution options

The OSCAL project is producing several types of deliverables, including the following:
- *Schemas* for the OSCAL component models
- *Schemas* for the OSCAL component models
- *Schematron definitions*, which are basically an extension of the XML schemas that provide more validation capabilities
- *XSL templates* for production of human-readable versions of OSCAL XML content
- *CSS*, so people who are developing catalogs and profiles using XML tools can use CSS for data entry, which offers a much more usable interface
Expand Down Expand Up @@ -69,7 +69,7 @@ The NIST OSCAL team is using the GitHub [project cards](https://github.com/usnis

### User Stories

Each development sprint consists of a set of [user stories](https://github.com/usnistgov/OSCAL/issues?q=is%3Aopen+is%3Aissue+label%3A%22User+Story%22), that represent features, actions, or enhancements that are intended to be developed during the sprint. Each user story is based on a [template](https://github.com/usnistgov/OSCAL/issues/new?template=feature_request.md&labels=enhancement%2C+User+Story) and describes the basic problem or need to be addressed, a set of detailed goals to accomplish, any dependencies that must be addressed to start or complete the user story, and the criteria for acceptance of the contribution.
Each development sprint consists of a set of [user stories](https://github.com/usnistgov/OSCAL/issues?q=is%3Aopen+is%3Aissue+label%3A%22User+Story%22), that represent features, actions, or enhancements that are intended to be developed during the sprint. Each user story is based on a [template](https://github.com/usnistgov/OSCAL/issues/new?template=feature_request.md&labels=enhancement%2C+User+Story) and describes the basic problem or need to be addressed, a set of detailed goals to accomplish, any dependencies that must be addressed to start or complete the user story, and the criteria for acceptance of the contribution.

The goals in a user story will be bulleted, indicating that each goal can be worked on in parallel, or numbered, indicating that each goal must be worked on sequentially. Each goal will be assigned to one or more individuals to accomplish.

Expand Down
1 change: 0 additions & 1 deletion build/ci-cd/config/metaschema
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,3 @@ src/metaschema/oscal_ssp_metaschema.xml|xml,json|xml,json|xml,json
src/metaschema/oscal_poam_metaschema.xml|xml,json|xml,json|xml,json
src/metaschema/oscal_assessment-plan_metaschema.xml|xml,json|xml,json|xml,json
src/metaschema/oscal_assessment-results_metaschema.xml|xml,json|xml,json|xml,json

6 changes: 3 additions & 3 deletions build/ci-cd/copy-and-convert-content.sh
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ while IFS="|" read path_glob format model converttoformats || [[ -n "$path_glob"
# echo "Format: $format"
# echo "Model: $model"
# echo "Convert to: $converttoformats"

paths+=("$path")
formats+=("$format")
models+=("$model")
Expand Down Expand Up @@ -341,7 +341,7 @@ copy_or_convert_content() {
fi
resolved_profile="${target_dir}/${source_filename%_profile.xml}-resolved-profile_catalog.xml"
# printf 'resolved profile: %s\n' "$resolved_profile"

result=$(xsl_transform "${PROFILE_RESOLVER}" "$source_file" "${resolved_profile}" 2>&1)
cmd_exitcode=$?
if [ $cmd_exitcode != 0 ]; then
Expand All @@ -351,7 +351,7 @@ copy_or_convert_content() {
echo -e "${P_ERROR}Failed to resolve profile '${P_END}${resolved_profile}${P_ERROR}'.${P_END}"
return 1;
fi

result=$(validate_xml "$CATALOG_SCHEMA" "${resolved_profile}")
cmd_exitcode=$?
if [ $cmd_exitcode != 0 ]; then
Expand Down
3 changes: 1 addition & 2 deletions build/ci-cd/generate-content-converters.sh
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ for i in ${!paths[@]}; do

if [ "$VERBOSE" == "true" ]; then
args+=("-v")
fi
fi

args+=("$metaschema")
args+=("$converter")
Expand All @@ -154,4 +154,3 @@ for i in ${!paths[@]}; do
done

exit $exitcode

4 changes: 2 additions & 2 deletions build/ci-cd/generate-schema.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ if [ -z ${OSCAL_SCRIPT_INIT+x} ]; then
fi

# A bash function that displays the command help message called when "-h" argument is provided
usage() {
usage() {
cat << EOF
Usage: $0 [options] [metaschema paths]
Expand Down Expand Up @@ -164,7 +164,7 @@ for i in ${!paths[@]}; do

if [ "$VERBOSE" == "true" ]; then
args+=("-v") # if verbose, have the generation script generate verbose messages
fi
fi

# call the schema generation script
result=$("$OSCALDIR/build/metaschema/scripts/generate-schema.sh" "${args[@]}" 2>&1)
Expand Down
1 change: 0 additions & 1 deletion build/ci-cd/generate-specification-documentation.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,3 @@ if [ $cmd_exitcode -ne 0 ]; then
else
echo -e "${P_OK}Generating specification '${P_END}${SPEC_OUTPUT}${P_OK}' was successful for '${P_END}${SPEC_SOURCE}${P_OK}.${P_END}"
fi

1 change: 0 additions & 1 deletion build/ci-cd/include/init-oscal.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,3 @@ if [ -z ${OSCAL_SCRIPT_INIT+x} ]; then

OSCAL_SCRIPT_INIT=true
fi

2 changes: 1 addition & 1 deletion build/ci-cd/python/printColors.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ class bcolors:
FAIL = '\033[91m'
ENDC = '\033[0m'
BOLD = '\033[1m'
UNDERLINE = '\033[4m'
UNDERLINE = '\033[4m'
8 changes: 4 additions & 4 deletions build/ci-cd/python/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ To run the full suite of tests for OSCAL, run the following at the command line:

```sh fullTests.sh```

This file will output to the console SUCCESS/ERROR messages with supporting logs for all relevant XML/XSD pairs defined in the validation script.
This file will output to the console SUCCESS/ERROR messages with supporting logs for all relevant XML/XSD pairs defined in the validation script.

In addition to the full test suite, individual test files can be run. Each of these tests is captured in separate .sh files.

Expand All @@ -30,14 +30,14 @@ The JSON schema validation is done via jsonschema (pip install jsonschema). Tes
NOTE: If unable to perform validation using the script, you can call jsonschema directly from the command line as follows:

```
jsonschema -i ../content/nist.gov/SP800-53/rev4/NIST_SP-800-53_rev4_LOW-baseline_profile.json ../schema/json/oscal-profile-schema.json
jsonschema -i ../content/nist.gov/SP800-53/rev4/NIST_SP-800-53_rev4_LOW-baseline_profile.json ../schema/json/oscal-profile-schema.json
```
The command line will provide more detailed error messaging for troubleshooting of problematic files.

Reading JSON files is done via simplejson (pip install simplejson). This library reads the JSON files and passes to JSON schema above.

To cover transformations, the testing relies on teh Saxon 9 HE open source library. This can be command line driven with Java to perform transformations. This library will be used to do round trip transformations to create temporary files that can be validated by additional Python scripts to provide full test coverage.

### Future State Work
### Future State Work

Once the basic testing is in place, additional data sets will be created to allow lower level testing and deeper regression testing. This work is post-poned until the standard and supporting artifacts stabilize to minimize rework.
Once the basic testing is in place, additional data sets will be created to allow lower level testing and deeper regression testing. This work is post-poned until the standard and supporting artifacts stabilize to minimize rework.
4 changes: 2 additions & 2 deletions build/ci-cd/python/timeStampValidation.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
sys.exit(0)

try:
# get the date created
# get the date created
print('Filename 1 timestamp: ' + datetime.fromtimestamp(stat1.st_birthtime).strftime('%Y-%m-%d %H:%M:%S:%f'))
print('Filename 2 timestamp: ' + datetime.fromtimestamp(stat2.st_birthtime).strftime('%Y-%m-%d %H:%M:%S:%f'))
except AttributeError:
Expand All @@ -36,4 +36,4 @@
if stat2.st_mtime >= stat1.st_mtime:
print(bcolors.OKGREEN + 'SUCCESS: Time check passed for: 1) File 1: ' + filename1 + ', 2) File 2: ' + filename2 + bcolors.ENDC)
else:
print(bcolors.FAIL + 'ERROR: Time check failed for: 1) File 1: ' + filename1 + ', 2) File 2: ' + filename2 + bcolors.ENDC)
print(bcolors.FAIL + 'ERROR: Time check failed for: 1) File 1: ' + filename1 + ', 2) File 2: ' + filename2 + bcolors.ENDC)
4 changes: 2 additions & 2 deletions build/ci-cd/python/xmlValidation.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
# error checking for schema load
try:
xmlschema = etree.XMLSchema(xmlschema_doc)
except etree.XMLSchemaParseError as e:
except etree.XMLSchemaParseError as e:
print (e)
exit(1)

Expand Down Expand Up @@ -67,4 +67,4 @@

except:
print('Unknown error, exiting.')
quit()
quit()
3 changes: 1 addition & 2 deletions build/ci-cd/setup-ubuntu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ sudo apt-get -y install libxml2-utils nodejs npm python-pip jq

sudo apt-get -y install openjdk-11-jre-headless maven

sudo npm install -g ajv-cli prettyjson
sudo npm install -g ajv-cli prettyjson

pip install --user lxml

Expand All @@ -26,4 +26,3 @@ git checkout master -- trunk/schematron/code
git clone --depth 1 https://github.com/usnistgov/oscal-tools.git "${OSCAL_TOOLS_DIR}"
cd "$OSCAL_TOOLS_DIR/json-cli"
mvn clean install

30 changes: 15 additions & 15 deletions build/ci-cd/support/XMLSchema.dtd
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@
because at the Infoset level where schemas operate,
xmlns(:prefix) is NOT an attribute! -->
<!-- The declaration of xmlns is a convenience for schema authors -->

<!-- The id attribute here and below is for use in external references
from non-schemas using simple fragment identifiers.
It is NOT used for schema-to-schema reference, internal or
Expand Down Expand Up @@ -205,14 +205,14 @@
id ID #IMPLIED
%simpleContentAttrs;>

<!-- restriction should use the simple branch from part2, not the
<!-- restriction should use the simple branch from part2, not the
one defined above; extension should have no particle -->

<!ELEMENT %extension; ((%annotation;)?, (%particleAndAttrs;))>
<!ATTLIST %extension;
base %QName; #REQUIRED
id ID #IMPLIED

%extensionAttrs;>

<!-- an element is declared by either:
Expand Down Expand Up @@ -247,17 +247,17 @@
substitutionGroup, if any, else xs:anyType, i.e. unconstrained -->
<!-- default and fixed are mutually exclusive -->

<!ELEMENT %alternative; ((%annotation;)?,
<!ELEMENT %alternative; ((%annotation;)?,
(%simpleType; | %complexType;)?) >
<!ATTLIST %alternative;
<!ATTLIST %alternative;
test CDATA #IMPLIED
type %QName; #IMPLIED
xpathDefaultNamespace CDATA #IMPLIED
id ID #IMPLIED >


<!ELEMENT %group; ((%annotation;)?,(%mgs;)?)>
<!ATTLIST %group;
<!ATTLIST %group;
name %NCName; #IMPLIED
ref %QName; #IMPLIED
minOccurs %nonNegativeInteger; #IMPLIED
Expand Down Expand Up @@ -476,20 +476,20 @@
<!NOTATION XML PUBLIC
'REC-xml-1998-0210' 'http://www.w3.org/TR/1998/REC-xml-19980210' >

<!--
In keeping with the XML Schema WG's standard versioning policy,
<!--
In keeping with the XML Schema WG's standard versioning policy,
this DTD will persist at the URI
http://www.w3.org/2012/04/XMLSchema.dtd.

At the date of issue it can also be found at the URI
http://www.w3.org/2009/XMLSchema/XMLSchema.dtd.

The schema document at that URI may however change in the future,
in order to remain compatible with the latest version of XSD
and its namespace. In other words, if XSD or the XML Schema
namespace change, the version of this document at
http://www.w3.org/2009/XMLSchema/XMLSchema.dtd will change accordingly;
the version at http://www.w3.org/2012/04/XMLSchema.dtd
The schema document at that URI may however change in the future,
in order to remain compatible with the latest version of XSD
and its namespace. In other words, if XSD or the XML Schema
namespace change, the version of this document at
http://www.w3.org/2009/XMLSchema/XMLSchema.dtd will change accordingly;
the version at http://www.w3.org/2012/04/XMLSchema.dtd
will not change.

Previous dated (and unchanging) versions of this DTD include:
Expand All @@ -501,7 +501,7 @@
http://www.w3.org/2011/07/XMLSchema.dtd
(XSD 1.1 Candidate Recommendation)

http://www.w3.org/2009/04/XMLSchema.dtd
http://www.w3.org/2009/04/XMLSchema.dtd
(XSD 1.1 Candidate Recommendation)

http://www.w3.org/2004/10/XMLSchema.dtd
Expand Down
Loading

0 comments on commit f44426e

Please sign in to comment.