Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T3 DNN Draft PR + Multi-class Architecture #144

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

GNiendorf
Copy link
Member

@GNiendorf GNiendorf commented Jan 17, 2025

This PR introduces a T3 DNN with a multi-class classification architecture that outputs a softmax distribution for T3 candidates that are fake (0), real and prompt (1), and real and displaced (2). By placing an explicit cut on displaced t3 candidates and prompt t3 candidates, I can reject a lot of junk t3 objects at build-time without substantially decreasing the displaced track efficiency.

@GNiendorf
Copy link
Member Author

/run all

Copy link

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

Efficiency vs pT comparison Efficiency vs eta comparison
Fake rate vs pT comparison Fake rate vs eta comparison
Duplicate rate vs pT comparison Duplicate rate vs eta comparison

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     45.5    394.2    185.7    150.2    146.2    549.1    123.0    234.3    152.1      3.5    1983.9    1389.3+/- 383.1     525.2   explicit[s=4] (target branch)
   avg     45.4    396.6    186.2    183.8     33.4    545.1    120.8     83.5    148.2      3.0    1745.9    1155.4+/- 264.2     468.2   explicit[s=4] (this PR)

Copy link

The PR was built and ran successfully with CMSSW. Here are some plots.

OOTB All Tracks
Efficiency and fake rate vs pT, eta, and phi

The full set of validation and comparison plots can be found here.

@GNiendorf
Copy link
Member Author

Testing a new architecture type. Timing benefit is not as good, but displaced track efficiency loss diminishes.

/run all

Copy link

There was a problem while building and running in standalone mode. The logs can be found here.

@GNiendorf GNiendorf force-pushed the t3_dnn branch 2 times, most recently from fbb8ace to d11bf0e Compare January 23, 2025 14:19
@GNiendorf
Copy link
Member Author

/run all

Copy link

The PR was built and ran successfully in standalone mode. Here are some of the comparison plots.

Efficiency vs pT comparison Efficiency vs eta comparison
Fake rate vs pT comparison Fake rate vs eta comparison
Duplicate rate vs pT comparison Duplicate rate vs eta comparison

The full set of validation and comparison plots can be found here.

Here is a timing comparison:

   Evt    Hits       MD       LS      T3       T5       pLS       pT5      pT3      TC       Reset    Event     Short             Rate
   avg     46.4    393.2    185.6    150.9    148.0    548.1    122.8    233.1    152.5      3.3    1983.9    1389.4+/- 387.3     527.3   explicit[s=4] (target branch)
   avg     46.1    396.3    187.0    181.3     52.4    507.5    123.1    135.3    145.4      3.6    1778.0    1224.4+/- 295.9     473.6   explicit[s=4] (this PR)

@GNiendorf GNiendorf changed the title T3 DNN Draft PR Reference T3 DNN Draft PR + Multi-class Architecture Jan 23, 2025
Copy link

The PR was built and ran successfully with CMSSW. Here are some plots.

OOTB All Tracks
Efficiency and fake rate vs pT, eta, and phi

The full set of validation and comparison plots can be found here.

@GNiendorf
Copy link
Member Author

/run all

@GNiendorf
Copy link
Member Author

This PR Timing on L40
Screenshot 2025-01-23 at 5 37 49 PM

Current Timing on L40
Screenshot 2025-01-23 at 5 52 37 PM

@GNiendorf
Copy link
Member Author

GNiendorf commented Jan 23, 2025

Looks like there's some loss at high pT from updating the occupancies. @YonsiG Do you know what percentile the T3 occupancies were set at? Right now I have them all at 99.9%

Edit: Removed the occupancy update, will figure this out first in a separate PR.

@slava77
Copy link

slava77 commented Jan 30, 2025

@jchismar
you may want to test the T4 on top of this PR: perhaps the issue with fakes will be more manageable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants