Updated Occupancies

[GNiendorf]

Updated occupancies for all LST objects, and small updates to the notebook for printing them.

This PR Timing (0.8 GeV):

Original Timing (0.8 GeV):

This PR Timing (0.6 GeV):

Original Timing (0.6 GeV):

[GNiendorf]

/run all

[github-actions]

There was a problem while building and running with CMSSW. The logs can be found here.

[GNiendorf]

@ariostas Seems like an issue with the CMSSW tests? Something about alpaka math?

[github-actions]

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     46.0    401.4    188.0    150.9    146.0    551.4    122.8    233.0    151.3      3.7    1994.5    1397.1+/- 387.4     528.0   explicit[s=4] (target branch)
   avg     43.0    393.4    188.3    161.6    149.4    549.3    123.6    231.1    150.6      3.2    1993.6    1401.3+/- 390.3     529.1   explicit[s=4] (this PR)

[GNiendorf]

/run standalone lowpt

[ariostas]

Oh that's a package that was recently introduced by Manos. You could cherry-pick that commit or see if by early next week the CMSSW PR finally gets merged so that you can rebase

[github-actions]

The PR was built and ran successfully in standalone mode (low pT setup). Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     51.0    407.0    591.2    761.4   1052.4   1228.5    294.0   1152.9    360.3      5.0    5903.7    4624.2+/- 1634.9    1533.7   explicit[s=4] (target branch)
   avg     51.7    403.2    586.2    788.8   1215.9   1222.9    293.5   1161.9    367.7      6.4    6098.2    4823.6+/- 1758.5    1590.7   explicit[s=4] (this PR)

[slava77]

this is a pretty large drop: does this rely on the pending T3 DNN?

[GNiendorf]

I also don't think the T3 DNN will affect the T5 occupancies much. The T5 fake rate is already pretty low compared to the T3 fake rate, so a cleaner set of T3 candidates shouldn't lower the T5 occupancies much.

edit: Maybe this is from the T5 DNN? These occupancies were selected before the T5 DNN was added to the code. The location seems correct.

[GNiendorf]

Percentiles look correct. Either I made a mistake, or it was the T5 DNN that lowered these.

[slava77]

am I guessing correctly that the old derivation was based on 1000 events and the new on 500?
Considering some relatively large changes at 99.99% quantile, I suspect that this reduction in the occupancies may be chasing some tails of occasional combinatorial explosions. Are there always enough entries in the remaining 0.01% to make the decision? Something >>10 (e.g. > 30) would be good.

[GNiendorf]

Yes, in the slides Yonsi sent me it said she used a 500 event sample for the 0.8 occupancies, so I tried to match that here. I will check.

[slava77]

which kinematic regions are these 4 bins? (to follow up on the discussion during the meeting for why earlier zeroes are now not)

[GNiendorf]

[slava77]

this still requires parsing: what are the two middle elements out of 4 in the category 2?
Is Category 2 defined by radius and z range or by the disk and ring index in the endcap?

[GNiendorf]

Definition of Category 2: (module_layers >= 4) & (module_subdets == 5)
Definition of Eta ranges:
eta_numbers[module_eta < 0.75] = 0
eta_numbers[(module_eta >= 0.75) & (module_eta < 1.5)] = 1
eta_numbers[(module_eta >= 1.5) & (module_eta < 2.25)] = 2
eta_numbers[(module_eta >= 2.25) & (module_eta < 3)] = 3

[GNiendorf]

/run all

[github-actions]

There was a problem while building and running with CMSSW. The logs can be found here.

[github-actions]

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     46.6    395.5    187.1    152.4    147.6    549.4    124.5    235.1    151.6      3.6    1993.5    1397.5+/- 386.1     530.6   explicit[s=4] (target branch)
   avg     44.2    394.5    190.0    154.8    141.5    551.4    124.9    235.9    151.0      3.3    1991.6    1396.0+/- 388.1     529.6   explicit[s=4] (this PR)

[GNiendorf]

/run standalone lowpt

[github-actions]

The PR was built and ran successfully in standalone mode (low pT setup). Here are some of the comparison plots.

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     50.1    402.7    580.2    758.5   1048.2   1222.7    290.8   1153.1    359.7      5.6    5871.5    4598.7+/- 1628.6    1527.6   explicit[s=4] (target branch)
   avg     50.5    404.7    578.5    721.4   1132.0   1228.3    293.1   1159.6    363.2      6.3    5937.6    4658.8+/- 1669.7    1536.4   explicit[s=4] (this PR)

[GNiendorf]

/run standalone

[github-actions]

There was a problem while building and running in standalone mode. The logs can be found here.

[GNiendorf]

Moving the dynamic memory allocation to #148

[GNiendorf]

I think it makes sense to merge #148 first and then quickly reevaluate the occupancy thresholds. It's likely we can increase the caps now without a huge increase in memory in order to decrease truncation.