forked from osmlab/atlas-checks
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adding new SignPostCheck and IntersectingBuildingsCheck. (osmlab#19)
* Adding new SignPostCheck and IntersectingBuildingsCheck. Also updating MapRoulette functions to allow the use of tags when creating challenges. Updating documents to describe how to develop and debug Atlas-Checks. * Updates to SignPostCheck * Updates to debugging and unit test documentation * Removing git merge artifacts
- Loading branch information
1 parent
0bcb65c
commit c3d65d3
Showing
21 changed files
with
1,518 additions
and
40 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # Debugging Atlas Checks | ||
|
|
||
| ### Overview | ||
|
|
||
| When developing an Atlas Checks, it is very important to be able to debug your checks to understand what they are doing or potentially where your check is not doing what you would expect it to do. The document describes how you would debug Atlas Checks using the Intellij IDE by JetBrains. There are various IDE's out there that perform similar debugging functions, namely an IDE like Eclipse, and transferring the instructions from Intellij to your IDE of choice should be fairly straight forward. This document should give you the basics for whatever IDE you wish to use. | ||
|
|
||
| ### Setup | ||
|
|
||
| The first thing required is to import the project into Intellij. Intellij supports gradle projects which is what Atlas-Checks is, so importing the project is fairly straight forward. These instructions assumed | ||
|
|
||
| 1. Clone the Atlas-Checks project into a directory of your choice | ||
| 2. Click on "File" -> "New" -> "Project from Existing Sources..." | ||
| 3. Select the folder that you original cloned your Atlas-Checks project into in Step 1. | ||
| 4. Select the "Import project from external model" and then select the "Gradle" option | ||
| 5. In the final screen check the box "Use auto-import" and make sure that "Use default gradle wrapper (recommended)" is selected. | ||
| 6. Click Finish and wait for gradle tasks to complete. | ||
|
|
||
| This will create a new Atlas-Checks project that you can start developing in Intellij. Most other full fledged IDEs should following a similar path. Smaller lightweight IDEs will simply have you point to a directory. | ||
|
|
||
| ### Creating Configuration | ||
|
|
||
| In Intellij, a configuration is used to run a project. With the setup above you will be able to build the project directly from within IDE but we need to setup a configuration to actually run and debug it. We won't be running it as mentioned in other documents using the gradle task `gradle run`, we will be running it directly against the main class. Essentially diving into what `gradle run` hides from the user. | ||
|
|
||
| 1. Click on the drop down bar in the toolbar, it should currently be empty. If you are not sure where this is you can also go to File Menu and click on "run" and then select the "run" option in the drop down. | ||
|
|
||
| - If you click on the drop down bar, it will drop down an "edit configurations" option that you must click. | ||
| - If you went through the file menu, then a small dialog will pop up, click on the "edit configurations" option. | ||
|
|
||
| 2. In the "Run/Debug Configurations" dialog box click on the + sign in the top left hand corner of the dialog. | ||
| 3. Select the "Application" option | ||
| 4. In the pane on the right that now contains the new Application update the following. | ||
|
|
||
| - Change the name to something like 'Atlas-Checks' | ||
| - Change the main class to `org.openstreetmap.atlas.checks.distributed.IntegrityCheckSparkJob` | ||
| - Optionally include VM Options: -Xms2048m -Xmx10240m -XX:MaxPermSize=4096m, this will help with larger atlas files. | ||
| - Update the working directory to be the current directory of your Atlas-Checks project. | ||
| - Set "Use of classpath module" to `atlas-checks_main` | ||
| - Include the following parameter template in "Program Arguments". Replace explanation with actual value. | ||
|
|
||
| - inputFolder=[Folder pointing to location of country folders with atlas files] | ||
| - startedFolder=[Output directory for Spark, can be any directory you create] | ||
| - output=[Output directory for results of job] | ||
| - countries=[ISO3 country code comma separated list] | ||
| - maproulette=[Maproulette configuration in format "server:host:project:api_key"] | ||
| - saveCheckOutput=false | ||
| - master=local | ||
| - configFiles=[Points to the configuration file, should be something like file:/atlas_checks_root_dir/config/configuration.json] | ||
| - sparkOptions=spark.executor.memory->4g,spark.driver.memory->4g,spark.rdd.compress->true | ||
| A couple of notes about the program arguments. | ||
|
|
||
| 1. SparkOptions should generally not be changed, but if you require more memory for either driver or workers then you can update it. For more information about Spark options see [here](http://spark.apache.org/docs/1.6.0/configuration.html). | ||
| 2. This can be used to deploy your jobs to a Spark cluster, that information you can find [here](cluster.md) | ||
| 3. The `gradle run` task hides the structure of the atlas files, however in the background the files are separated by country into ISO3 country folders. So unlike using the gradle task you need to have the actual atlas files and place them in the correct folder structure for this to work, an example folder structure would be like below: | ||
|
|
||
| ``` | ||
| - Root Folder | ||
| - ABC | ||
| - ABC_7-41-57.atlas | ||
| - XYZ | ||
| - XYZ_10-806-508.atlas | ||
| - XYZ_11-1614-1016.atlas | ||
| - XYZ_11-1614-1017.atlas | ||
| - XYZ_11-1615-1016.atlas | ||
| - XYZ_7-101-63.atlas | ||
| - XYZ_8-201-126.atlas | ||
| ``` | ||
| Where ABC and XYZ are two different countries. | ||
|
|
||
| ### Running/Debugging Atlas Checks | ||
|
|
||
| At this point running Atlas-Checks is as simple as either clicking on the play button on the taskbar or clicking on "Run" -> "Run..." in the file menu. Similarly with debugging you can either click on the little bug icon (usually next to the play button) on the taskbar or clicking "Run" -> "Debug..." in the file menu. The code will run and the code will break at any breakpoints that you place in the code. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # Writing Unit Tests for Altas Checks | ||
|
|
||
| It is important to make sure your code works as expected as well as making sure that any changes you make don't break existing changes. A good approach to developing integrity checks is using a [Test Driven Development](https://en.wikipedia.org/wiki/Test-driven_development) based approach. In a nutshell design your unit tests first and then build your check making sure that the tests that you originally built succeeds. | ||
|
|
||
| ### Example Unit Test | ||
|
|
||
| For real examples you can look a couple that have already been built for the currently implemented checks: | ||
|
|
||
| - [SinkIslandCheckTest](../src/test/java/org/openstreetmap/atlas/checks/validation/linear/edges/SinkIslandCheckTest.java) and [SinkIslandCheckTestRule](../src/test/java/org/openstreetmap/atlas/checks/validation/linear/edges/SinkIslandCheckTestRule.java) | ||
| - [AbbreviatedNameCheckTest](../src/test/java/org/openstreetmap/atlas/checks/validation/tag/AbbreviatedNameCheckTest.java) and [AbbreviatedNameCheckTestRule](../src/test/java/org/openstreetmap/atlas/checks/validation/tag/AbbreviatedNameCheckTestRule.java) | ||
|
|
||
| As you see in the above their are two files associated with the unit tests. The first file are the unit tests themselves, the second file is, for lack of a better term, test Atlas files. Below we will first describe the structure of a unit test to be used to validate your Atlas Check and then describe how the data file is built and how to generate test atlas'. | ||
|
|
||
| ### Our First Unit Test | ||
|
|
||
| ```java | ||
| public class MyUnitTest { | ||
| // For now we will assume that there is a class called MyUnitTestRule with a test atlas inside called "testAtlas" | ||
| @Rule | ||
| public MyUnitTestRule setup = new MyUnitTestRule(); | ||
|
|
||
| @Rule | ||
| public ConsumerBasedExpectedCheckVerifier verifier = new ConsumerBasedExpectedCheckVerifier(); | ||
|
|
||
| @Test | ||
| public void testMyUnitTest() | ||
| { | ||
| // MyCheck would be replaced with the name of the check that you are testing | ||
| // The configuration is optional, and only required if you have specific configuration for the check that you want to test or is required for the check itself | ||
| this.verifier.actual(this.setup.testAtlas(), | ||
| new MyCheck(ConfigurationResolver.inlineConfiguration("{\"key\":\"value\"}"))); | ||
| // There are various helper functions described below, this one simply checks if the result of the check on the test atlas produces at least 1 flag. | ||
| this.verifier.verifyNotEmpty(); | ||
|
|
||
| // Another common scenario would be to loop through the results and check something on it. | ||
| this.verifier.verify(flag -> { | ||
| // Check the flag object for some expected values | ||
| }); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| A basic Atlas-Checks unit test contains 3 things: | ||
| 1. A TestRule class that is essentially an atlas or multiple atlas to test against. | ||
| 2. A `ConsumerBasedExpectedCheckVerifier` instance that will execute the check you are testing against the test atlas. | ||
| 3. A series of unit tests using the JUnit framework. | ||
|
|
||
| ### The `ConsumerBasedExpectedCheckVerifier` | ||
|
|
||
| The ConsumerBasedExpectedCheckVerifier is a class that will run your check against a test atlas. The test atlas are often very small atlas objects that contain specific data to test against. This verifier has too main types of functions | ||
|
|
||
| 1. `actual(Atlas, BaseCheck)` - The actual method establishes what data to run against (ie. what atlas file) and what check to run against the provided test atlas. | ||
| 2. verify functions - The verify functions will be used to verify the results of the running the check over the test atlas file. Below are some useful functions | ||
|
|
||
| - `verifyNotEmpty()` - Verifies that at least 1 flag was produced by the check | ||
| - `verifyEmpty()` - Verifies that no flags were produced by the check | ||
| - `verifyExpectedSize(int)` - Verifies that a specific number of flags were produced by the check | ||
| - `verify(Consumer<List<CheckFlag>>)` - A custom verifier allowing the developer to loop through each individual flag that is produced and verify it in any way that the developer wishes. | ||
|
|
||
| ### The `TestRule` | ||
|
|
||
| As mentioned previously the TestRule is a class that is associated with the Unit test class and contains all the test data, which would essentially be an Atlas in some form or another. An Atlas can be created in 2 primary ways: | ||
|
|
||
| - Inline - An inline atlas uses tags to build it within the code. The code below will create an inline atlas that contains 3 nodes and 2 edges. You can also use the inline @tags to build an Atlas containing `lines`, `points` or `relations`. An example of this can be found in the [FloatingEdgeCheckTestRule](../src/test/java/org/openstreetmap/atlas/checks/validation/linear/edges/FloatingEdgeCheckTestRule.java) | ||
| ```java | ||
| @TestAtlas(nodes = { @Node(coordinates = @Loc(value = "37.32544,-122.033948")), | ||
| @Node(coordinates = @Loc(value = "37.33531,-122.009566")), | ||
| @Node(coordinates = @Loc(value = "37.3314171,-122.0304871")) }, edges = { | ||
| @Edge(coordinates = { @Loc(value = "37.32544,-122.033948"), @Loc(value = "37.33531,-122.009566") }, tags = { | ||
| "highway=SECONDARY" }), | ||
| @Edge(coordinates = { @Loc(value = "37.33531,-122.009566"), @Loc(value = "37.3314171,-122.0304871") }, tags = { | ||
| "highway=SECONDARY" }) }) | ||
| ``` | ||
| - Atlas Text Files - It is fairly easy to produce very small test atlas text files which that can be stored in resources much like the [poolsize.altas](../src/test/resources/org/openstreetmap/atlas/checks/validation/areas/poolsize.atlas). And generating this files are fairly straight forward, and can be done by following these steps: | ||
| 1. Generate a new Configuration in Intellij Idea IDE. (These general steps should work in any IDE, and the concept should be able to be transferred over to anything) | ||
| 2. Choose Application and input the following parameters in the dialog to the right. | ||
| - Name: AtlasTestGenerator | ||
| - Main Class: org.openstreetmap.atlas.utilities.runtime.FlexibleCommand | ||
| - Program Arguments: atlas-with-this-entity<br/> | ||
| -input=`[Input location for atlas file]`<br/> | ||
| -osmid=`[OSM ID of primary feature]`<br/> | ||
| -expand=`[Amount you want to expand around feature]`<br/> | ||
| -text-output=`[Output location for resultant atlas text file]`<br/> | ||
| - Working directory: [Root directory of Atlas-Checks project] | ||
| - Use classpath of module: atlas-checks_main | ||
| 3. Run new AtlasTestGenerator configuration in Intellij. | ||
|
|
||
| This will produce a text atlas file in the location provided. You can then move the resultant file to the resources directory, much like the [poolsize.atlas](../src/test/resources/org/openstreetmap/atlas/checks/validation/areas/poolsize.atlas), Or you could set the output location to the resources directory to begin with. It is important to note that these files should be small as they would be checked into the repository as part of the code. There are options in the Atlas-Generator to zip up the results, but generally it is preferable to be able to see the contents of the atlas file. Once you have the text atlas file you can insert it into your test rule with the following code. | ||
| ```java | ||
| @TestAtlas(loadFromTextResource = "test.atlas") | ||
| private Atlas testAtlas; | ||
|
|
||
| public Atlas getTestAtlas() | ||
| { | ||
| return this.testAtlas; | ||
| } | ||
| ``` | ||
| The first line is the @tag that will load the text resource and instantiate the Atlas object below it. Then include a basic getter for the Atlas so that your unit test can use the test atlas. After completing all this you are ready to run your first unit test. |
Oops, something went wrong.