Skip to content

Commit

Permalink
update the closure test docs
Browse files Browse the repository at this point in the history
  • Loading branch information
RoyStegeman committed May 7, 2024
1 parent 124ae08 commit eba19fc
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 21 deletions.
13 changes: 1 addition & 12 deletions doc/sphinx/source/tutorials/closuretest.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,7 @@ closuretest:
filterseed : 0 # Random seed to be used in filtering data partitions
fakedata : False
fakepdf : MMHT2014nnlo68cl
errorsize : 1.0 # uncertainties rescaling
fakenoise : False
rancutprob : 1.0 # Fraction of data to be included in the fit
rancutmethod: 0 # Method to select rancutprob data fraction
rancuttrnval: False # 0(1) to output training(valiation) chi2 in report
printpdf4gen: False # To print info on PDFs during minimization
```
Setting ``fakedata`` to ``True`` will cause closure test pseudodata to be generated
Expand All @@ -80,21 +75,15 @@ closuretest:
filterseed : 0 # Random seed to be used in filtering data partitions
fakedata : True
fakepdf : MMHT2014nnlo68cl
errorsize : 1.0 # uncertainties rescaling
fakenoise : True
rancutprob : 1.0 # Fraction of data to be included in the fit
rancutmethod: 0 # Method to select rancutprob data fraction
rancuttrnval: False # 0(1) to output training(valiation) chi2 in report
printpdf4gen: False # To print info on PDFs during minimization
```
Note that it is *critical* that two closure tests which are to be compared have
the same ``filterseed``. They should also both have been run during a time where
no major changes were made to data generation. This is because fits with
different level 1 noise produce different closure test estimators. See for
example a [report](https://vp.nnpdf.science/mbcTUd6-TQmQFvaGd37bkg==/)
comparing two level 2 closure tests with identical settings apart from
``filterseed``. Note that setting ``filterseed`` to 0 will use the default seed for
the selected ``fitting::rngalgo``.
``filterseed``.
There are still some relevant settings to the closure test. For the above example
we would choose that the t0 set was the same as the underlying law:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,16 +92,13 @@ datacuts:

############################################################
theory:
theoryid: 53 # database id
theoryid: 700 # database id

############################################################
trvlseed: 1727532335
nnseed: 1737785873
mcseed: 2123509817
genrep: true # true = generate MC replicas, false = use real data
# Deprecated options still required by closure tests
rngalg: 0
seed: 4

parameters: # This defines the parameter dictionary that is passed to the Model Trainer
nodes_per_layer: [18, 24, 36, 8]
Expand Down Expand Up @@ -168,12 +165,7 @@ closuretest:
filterseed: 3345348918 # Random seed to be used in filtering data partitions
fakedata: true # true = to use FAKEPDF to generate pseudo-data
fakepdf: 201204-mw-001_closure_fakepdf # Theory input for pseudo-data
errorsize: 1.0 # uncertainties rescaling
fakenoise: true # true = to add random fluctuations to pseudo-data
rancutprob: 1.0 # Fraction of data to be included in the fit
rancutmethod: 0 # Method to select rancutprob data fraction
rancuttrnval: false # 0(1) to output training(valiation) chi2 in report
printpdf4gen: false # To print info on PDFs during minimization

############################################################
debug: false

0 comments on commit eba19fc

Please sign in to comment.