g4GraphicalScan.py: produce 2d scan of models, using g4 material description #1397

danieljeans · 2025-02-06T06:19:17Z

BEGINRELEASENOTES

add python utility g4GraphicalScan.py to produce 2-d scan of a detector model
- similar to graphicalScan, but directly reads the g4 geometry (rather than the DDRec geometry used by graphicalScan)
- this can therefore cope with CAD-based elements ( see materialScan on CAD-imported geometry : strange results... #1322 )

ENDRELEASENOTES

github-actions · 2025-02-06T07:35:41Z

Test Results

16 files 16 suites 5h 52m 14s ⏱️
370 tests 370 ✅ 0 💤 0 ❌
2 887 runs 2 887 ✅ 0 💤 0 ❌

Results for commit 850fb9a.

♻️ This comment has been updated with latest results.

danieljeans · 2025-02-06T08:28:08Z

I think this tool can be useful for users: I'd be happy to have it included.

andresailer · 2025-02-06T08:57:25Z

Hi @danieljeans

Thanks for this! Can you add an example figure of the output for further advertising and showing what the tool does?

danieljeans · 2025-02-06T09:20:55Z

this produces a series of 2-d histograms in a requested slice of the detector (perpendicular to either the x, y, or z axis).

the "top" plot is of the material density (in units of 1/x0):

for each material it encounters, it also produces a histogram of its distribution: an estimate of the fraction of each bin that is occupied by the material:

danieljeans · 2025-02-06T09:27:22Z

I find it useful for pretty plots for presentations, understanding what is in a model at both large and small scales, and debugging models.

andresailer · 2025-02-06T10:30:02Z

It looks very useful!

Can you add the line to install the tool, like

DD4hep/DDG4/CMakeLists.txt

Lines 185 to 186 in 0aafe1c

    
           install(PROGRAMS python/bin/g4MaterialScan.py DESTINATION bin RENAME g4MaterialScan) 
        
           install(PROGRAMS python/bin/g4GeometryScan.py DESTINATION bin RENAME g4GeometryScan)

DDG4/python/bin/g4GraphicalScan.py

…py to CMakeLists

DDG4/python/bin/g4GraphicalScan.py

MarkusFrankATcernch · 2025-02-07T08:55:31Z

If you change it all, please also add the header with the licence. CERN's legal office likes this.....

# ==========================================================================
#  AIDA Detector description implementation
# --------------------------------------------------------------------------
# Copyright (C) Organisation europeenne pour la Recherche nucleaire (CERN)
# All rights reserved.
#
# For the licensing terms see $DD4hepINSTALL/LICENSE.
# For the list of contributors see $DD4hepINSTALL/doc/CREDITS.
#
# ==========================================================================

DDG4/python/bin/g4GraphicalScan.py

danieljeans · 2025-02-07T09:07:10Z

@andresailer thanks

aciarma · 2025-02-07T17:36:47Z

I am sorry, I am trying to use this tool but I can't manage to make it run, and also g4MaterialScan is now failing.
I install this locally on lxplus sourcing latest key4hep env and using option -DDD4HEP_USE_GEANT4=ON, and then sourcing the install/bin/thisdd4hep.sh, do I have to do something else to correctly install this?

MarkusFrankATcernch · 2025-02-07T18:03:22Z

The default executable of python changed to python3 rather than python. Do you have this on your platform ?
This was the only change in g4MaterialScan as far as I can see.
Otherwise:

python `which g4MaterialScan.py` .....

aciarma · 2025-02-07T18:33:27Z

nope, this is not working for me.

source /cvmfs/fcc.cern.ch/sw/latest/setup.sh

git clone [email protected]:AIDASoft/DD4hep.git
cd DD4hep/
mkdir build install
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install -DDD4HEP_USE_GEANT4=ON
make install -j15
source ../install/bin/thisdd4hep_only.sh
MYDDG4=../DDG4/python/bin

cd /path/to/test/folder
                                                                                                                                    
python $MYDDG4/g4MaterialScan.py -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -p "0,0,0" -d "1,0,0"                                                           
python3 $MYDDG4/g4MaterialScan.py -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -p "0,0,0" -d "1,0,0"                                                          
g4MaterialScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -p "0,0,0" -d   "1,0,0"

exit

None of the three calls seem to be working. If instead I use the version installed on afs it works (but Daniel's tool is not there yet... that's why I wanted a local install!)

danieljeans · 2025-02-10T06:15:38Z

thanks for the report.

@aciarma there seems to be a problem with this IDEA model?

$ source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh 
AlmaLinux/RockyLinux/RHEL 9 detected
Setting up the Key4hep software stack nightly build latest-opt from CVMFS
Use the following command to reproduce the current environment: 

        source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh -r 2025-02-10

$ g4MaterialScan -c $k4geo_DIR/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -p 0,0,0 -d 0,1,0

Geant4UI                          INFO  +++ UI> Install Geant4 control directory:/ddg4/UI/
Geant4Kernel                      INFO  ++ Registered global action UI of type dd4hep::sim::Geant4UIManager
ERROR: +++  CDCH                             type:SimpleDriftChamber  --> Unknown Sensitive type: SimpleDriftChamber
corrupted size vs. prev_size in fastbins
 *** Break *** abort



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

danieljeans · 2025-02-10T06:16:37Z

this error is not at all visible if you run g4GraphicalScan. ddsim just hangs, as does g4GraphicalScan.
I will improve g4GraphicalScan's robustness to bad inputs.

danieljeans · 2025-02-10T08:35:35Z

#1398 improves the robustness, and should not just hang if passed bad input.

aciarma · 2025-02-10T14:56:15Z

Hello,

this keeps not working correctly for me, both g4MaterialScan and g4GraphicalScan either crash or produce empty histograms for different geometries.

source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh


which g4MaterialScan
which g4GraphicalScan


time g4MaterialScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v03/IDEA_o1_v03.xml -p "0,0,0" -d      "1,0,0"
#1m14, crashes at DRcalo                                                                                                                                                     
time g4GraphicalScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v03/IDEA_o1_v03.xml -s "ZX" -x "-20,20" -y "0" -z "0,100" -n "3" -o "IDEAv03out.root"
#1m8, produces empty histogram                                                                                                                                               

#g4MaterialScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v02/IDEA_o1_v02.xml -p "0,0,0" -d   "1,0,0"                                                                             
#g4GraphicalScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -s "ZX" -x "-20,20" -y "0" -z "0,100" -n "3" -o "IDEAv01out.root"                                  

#g4MaterialScan -c $K4GEO/FCCee/IDEA/compact/IDEA_o1_v01/IDEA_o1_v01.xml -p "0,0,0" -d   "1,0,0"                                                                             
#g4GraphicalScan -c $K4GEO/FCCee/ALLEGRO/compact/ALLEGRO_o1_v01/ALLEGRO_o1_v01.xml -s "ZX" -x "-20,20" -y "0" -z "0,100" -n "3" -o "ALLEGROv01out.root"                      

cd /afs/cern.ch/user/a/aciarma/public/git_k4geo/aciarma_commonMDI/k4geo/
k4_local_repo
#compiled with the same environment                                                                                                                                          
cd $OLDPWD
MDI_DIR=/afs/cern.ch/user/a/aciarma/public/git_k4geo/aciarma_commonMDI/k4geo/FCCee/MDI/compact


time g4MaterialScan  -c $MDI_DIR/MDI_o1_v00/MDI_standalone_o1_v00.xml -p "0,0,900," -d "2,1,0"
#few seconds, works                                                                                                                                                          
time g4GraphicalScan -c $MDI_DIR/MDI_o1_v00/MDI_standalone_o1_v00.xml -s "ZX" -x "-20,20" -y "0" -z "0,100" -n "3" -o "MDIv00out.root"
#few seconds, fails                                                                                                                                                          
#Traceback (most recent call last):                                                                                                                                          
#  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2025-02-10/x86_64-almalinux9-gcc14.2.0-opt/dd4hep/94df2c68f5f13ac4c9d3c472d010ed2e6501f17d_develop-hywj4c/bin/g4Graphi\
calScan", line 280, in <module>                                                                                                                                              
#    index = int(line.split()[1])                                                                                                                                            
#            ^^^^^^^^^^^^^^^^^^^^                                                                                                                                            
#ValueError: invalid literal for int() with base 10: 'INFO'                                                                                                                  

time g4MaterialScan  -c $MDI_DIR/MDI_o1_v01/MDI_standalone_o1_v01.xml -p "0,0,900," -d "2,1,0"
#few seconds, works                                                                                                                                                          
time g4GraphicalScan -c $MDI_DIR/MDI_o1_v01/MDI_standalone_o1_v01.xml -s "ZX" -x "-20,20" -y "0" -z "0,100" -n "3" -o "MDIv01out.root"
#few seconds, fails                                                                                                                                                          
#Traceback (most recent call last):                                                                                                                                          
#  File "/cvmfs/sw-nightlies.hsf.org/key4hep/releases/2025-02-10/x86_64-almalinux9-gcc14.2.0-opt/dd4hep/94df2c68f5f13ac4c9d3c472d010ed2e6501f17d_develop-hywj4c/bin/g4Graphi\
calScan", line 280, in <module>                                                                                                                                              
#    index = int(line.split()[1])                                                                                                                                            
#            ^^^^^^^^^^^^^^^^^^^^                                                                                                                                            
#ValueError: invalid literal for int() with base 10: 'INFO'                                                                                                                  

exit

aciarma · 2025-02-11T16:33:37Z

update, now with nightlies 2025-02-11 works, but I get a weird behavior for Air, even for non-CAD geometries. These are plots for the v00 MDI. See how the air histogram is weirdly filled with 0.5 or 0, while where air is present should be 1. Is this due to how the scan is processed?

while this is for updated CAD geometries

danieljeans · 2025-02-12T02:14:52Z

@aciarma thanks for reporting this "weird air" behaviour: it's due to a bug.
Fixed in #1401

aciarma · 2025-02-13T10:57:17Z

hello @danieljeans , thank you very much for the reply! I am using nightlies 2025-02-13 (which should have the #1401 fix) but I still see something weird. It looks like the air from world volume is being averaged with the materials of the volumes inside it. This is clearly visible with the "beam" volume, where the material budget is 50% that of the air

(here there are no CAD volumes, all DD4hep geometries)

Air seems to be filled at 0.5 where other volumes are defined, and 1 elsewhere.

The other materials seems to contribute 50% to the X0 in these plots.

Am I misinterpreting the plots?

danieljeans · 2025-02-14T03:16:12Z

@aciarma I think this unexpected behaviour is arising because you're slicing at exactly z=0

to illustrate:

g4MaterialScan -c ~/k4geo-ciarma/FCCee/MDI/compact/MDI_o1_v01/MDI_standalone_o1_v01.xml -p 0,-20,0 -d 0,1,0
gives

 +--------------------------------------------------------------------------------------------------------------------------------------------------
 + Material scan between: x_0 = (   0.00,  -2.00,   0.00) [cm] and x_1 = (   0.00,2500.00,   0.00) [cm]  TrackID:1: 
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     \   Material           Atomic                 Radiation   Interaction               Path   Integrated  Integrated    Material
 | Num. \  Name          Number/Z   Mass/A  Density    Length       Length    Thickness   Length      X0        Lambda      Endpoint  
 | Layer \                        [g/mole]  [g/cm3]     [cm]        [cm]          [cm]      [cm]     [cm]        [cm]     (     cm,     cm,     cm)
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     1 Air                    7   14.784   0.0012  30528.8402   71282.7920   2502.000  2502.00    0.081955    0.035100  (   0.00,2500.00,   0.00)

we see only air, no beampipe, coolant, gold, vacuum, ...,

while shifting z by 1 micron

g4MaterialScan -c ~/k4geo-ciarma/FCCee/MDI/compact/MDI_o1_v01/MDI_standalone_o1_v01.xml -p 0,-20,0.001 -d 0,1,0

gives the expected structure

 +--------------------------------------------------------------------------------------------------------------------------------------------------
 + Material scan between: x_0 = (   0.00,  -2.00,   0.00) [cm] and x_1 = (   0.00,2500.00,   0.00) [cm]  TrackID:1: 
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     \   Material           Atomic                 Radiation   Interaction               Path   Integrated  Integrated    Material
 | Num. \  Name          Number/Z   Mass/A  Density    Length       Length    Thickness   Length      X0        Lambda      Endpoint  
 | Layer \                        [g/mole]  [g/cm3]     [cm]        [cm]          [cm]      [cm]     [cm]        [cm]     (     cm,     cm,     cm)
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     1 Air                    7   14.784   0.0012  30528.8402   71282.7920      0.830     0.83    0.000027    0.000012  (   0.00,  -1.17,   0.00)
 |     2 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     0.86    0.001889    0.000903  (   0.00,  -1.14,   0.00)
 |     3 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.100     0.96    0.003513    0.001997  (   0.00,  -1.03,   0.00)
 |     4 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     1.00    0.005375    0.002889  (   0.00,  -1.00,   0.00)
 |     5 Au                    79  196.967  19.3200      0.3344      10.5404      0.001     1.00    0.006870    0.002936  (   0.00,  -1.00,   0.00)
 |     6 beam                   5    9.370   0.0000 2.59816e+15  3.28469e+15      1.999     3.00    0.006870    0.002936  (   0.00,   1.00,   0.00)
 |     7 Au                    79  196.967  19.3200      0.3344      10.5404      0.001     3.00    0.008366    0.002984  (   0.00,   1.00,   0.00)
 |     8 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     3.04    0.010228    0.003876  (   0.00,   1.03,   0.00)
 |     9 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.100     3.14    0.011851    0.004969  (   0.00,   1.14,   0.00)
 |    10 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     3.17    0.013714    0.005861  (   0.00,   1.17,   0.00)
 |    11 Air                    7   14.784   0.0012  30528.8402   71282.7920   2498.830  2502.00    0.095565    0.040916  (   0.00,2500.00,   0.00)

I guess there is somehow a very thin boundary at z=0 which gives rise to unexpected behaviour.

Can you try slicing with a tiny offset in z?

danieljeans · 2025-02-14T03:54:35Z

scanning horizontally at z=0 is weird in a different way

> g4MaterialScan -c ~/k4geo-ciarma/FCCee/MDI/compact/MDI_o1_v01/MDI_standalone_o1_v01.xml -p -20,0,0 -d 1,0,0

 +--------------------------------------------------------------------------------------------------------------------------------------------------
 + Material scan between: x_0 = (  -2.00,   0.00,   0.00) [cm] and x_1 = (2500.00,   0.00,   0.00) [cm]  TrackID:1: 
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     \   Material           Atomic                 Radiation   Interaction               Path   Integrated  Integrated    Material
 | Num. \  Name          Number/Z   Mass/A  Density    Length       Length    Thickness   Length      X0        Lambda      Endpoint  
 | Layer \                        [g/mole]  [g/cm3]     [cm]        [cm]          [cm]      [cm]     [cm]        [cm]     (     cm,     cm,     cm)
 +--------------------------------------------------------------------------------------------------------------------------------------------------
 |     1 Air                    7   14.784   0.0012  30528.8402   71282.7920      0.830     0.83    0.000027    0.000012  (  -1.17,   0.00,   0.00)
 |     2 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     0.86    0.001889    0.000903  (  -1.14,   0.00,   0.00)
 |     3 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.100     0.96    0.003513    0.001997  (  -1.03,   0.00,   0.00)
 |     4 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.000     0.96    0.003513    0.001997  (  -1.03,   0.00,   0.00)
 |     5 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.000     0.96    0.003513    0.001997  (  -1.03,   0.00,   0.00)
 |     6 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     1.00    0.005375    0.002889  (  -1.00,   0.00,   0.00)
 |     7 Au                    79  196.967  19.3200      0.3344      10.5404      0.001     1.00    0.006870    0.002936  (  -1.00,   0.00,   0.00)
 |     8 beam                   5    9.370   0.0000 2.59816e+15  3.28469e+15      1.999     3.00    0.006870    0.002936  (   1.00,   0.00,   0.00)
 |     9 beam                   5    9.370   0.0000 2.59816e+15  3.28469e+15      0.000     3.00    0.006870    0.002936  (   1.00,   0.00,   0.00)
 |    10 beam                   5    9.370   0.0000 2.59816e+15  3.28469e+15      0.000     3.00    0.006870    0.002936  (   1.00,   0.00,   0.00)
 |    11 Au                    79  196.967  19.3200      0.3344      10.5404      0.001     3.00    0.008366    0.002984  (   1.00,   0.00,   0.00)
 |    12 Au                    79  196.967  19.3200      0.3344      10.5404      0.000     3.00    0.008366    0.002984  (   1.00,   0.00,   0.00)
 |    13 Au                    79  196.967  19.3200      0.3344      10.5404      0.000     3.00    0.008366    0.002984  (   1.00,   0.00,   0.00)
 |    14 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     3.04    0.010228    0.003876  (   1.03,   0.00,   0.00)
 |    15 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.100     3.14    0.011851    0.004969  (   1.14,   0.00,   0.00)
 |    16 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.000     3.14    0.011851    0.004969  (   1.14,   0.00,   0.00)
 |    17 LiquidNDecane          5   10.296   0.7300     61.5867      91.4187      0.000     3.14    0.011851    0.004969  (   1.14,   0.00,   0.00)
 |    18 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.035     3.17    0.013714    0.005861  (   1.17,   0.00,   0.00)
 |    19 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.000     3.17    0.013714    0.005861  (   1.17,   0.00,   0.00)
 |    20 AlBeMet162             7   15.841   2.1000     18.7945      39.2502      0.000     3.17    0.013714    0.005861  (   1.17,   0.00,   0.00)
 |    21 Air                    7   14.784   0.0012  30528.8402   71282.7920   2498.830  2502.00    0.095565    0.040916  (2500.00,   0.00,   0.00)

adding a tiny offset in z again fixes it.

I cannot explain why the behaviour at z=0 is different when scanning in different directions...

The plots show an average of the scans in X and Y. At z=0, the scans in y see only air, while the scans in x see somewhat more reasonable materials: hence the 50% air.

aciarma · 2025-02-14T10:08:27Z

Indeed, adding a small offset removes this behavior. I think this was also happening when using the non-CAD geometries because they are defined by symmetry from z=0. Thank you for the explanation and for the tool!

danieljeans added 2 commits February 6, 2025 14:48

add python program to produce 2-d scans of detector material

1e73a20

use AddBinContent

aab667e

danieljeans added 3 commits February 6, 2025 16:52

satisfy FLAKE8 formatting check...

937e90e

Merge branch 'master' into matScan

9662a43

one more FLAKE8 formatting check...

891c0df

andresailer reviewed Feb 6, 2025

View reviewed changes

DDG4/python/bin/g4GraphicalScan.py Outdated Show resolved Hide resolved

#!/usr/bin/env python -> #!/usr/bin/env python3 ; add g4GeometryScan.…

00c5075

…py to CMakeLists