Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full conversion #2

Open
wants to merge 46 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
cc321cd
Add files via upload
abbyxh Jul 23, 2024
2a6756d
Update index.js
abbyxh Jul 23, 2024
b6dcade
Update config_ild_trial.json
abbyxh Jul 23, 2024
1fe8fcb
Update config_ild_trial.json
abbyxh Jul 23, 2024
0215c4f
changes
abbyxh Jul 23, 2024
4fd3bc1
changes
abbyxh Jul 23, 2024
38d5fbf
cahnges
abbyxh Jul 23, 2024
1963a27
change name test
abbyxh Jul 23, 2024
8ac05be
changes
abbyxh Jul 23, 2024
31881f9
Merge branch 'main' of https://github.com/abbyxh/root2gltf
abbyxh Jul 23, 2024
9d09b97
change
abbyxh Jul 23, 2024
d56c302
TEST
abbyxh Jul 23, 2024
4c194dd
is this working
abbyxh Jul 23, 2024
75f7c3c
completed detector changes
abbyxh Jul 23, 2024
b996379
Finished
abbyxh Jul 23, 2024
0cf2042
Added xml -> gltf detector conversion python scripts so that the full…
Jul 30, 2024
8dda039
Combined the .xml -> .root converter with the .root -> .gltf converte…
Jul 30, 2024
49bdb02
Added python script to get a dictionary of the layers in a given xml …
Aug 1, 2024
f33a7aa
Added a 'maxDepth' variable that can be used if you don't want to sho…
Aug 2, 2024
78bf31f
Created a processing function to turn the original tree dictionary in…
Aug 2, 2024
c262670
Tidied up the configfile generator python script so it could be added…
Aug 5, 2024
2e71168
Finished the 'conversion_xml2gltf_autoConfig.py' script to convert th…
Aug 8, 2024
027db15
Moved python files to 'bin' and added folders for root and gltf files…
Aug 8, 2024
a5a7979
Finished adding in 'inputs' that are given to the user throughout the…
Aug 8, 2024
a963223
Added in more detectors (some xml files seem unable to work currently…
Aug 9, 2024
568ed7b
Was able to add in an option for users to define parts of the detecto…
Aug 12, 2024
4265929
Added some comments on how the python scripts work as well as includi…
Aug 12, 2024
ad6ddb8
Changed the RegEx in index.js to be a different flag 'i' so that the …
Aug 12, 2024
0ff7ecd
Have been working out a way to allow users to add their own colours a…
Aug 15, 2024
c01dce2
Managed to fix up the colour changing code so that a user can put the…
Aug 16, 2024
b960abc
Functionised the colour changing code and made it so that if colourin…
Aug 16, 2024
fdec236
Added in a way for users to add the ILD colors to the automatic confi…
Aug 19, 2024
06f886b
Fixed a bug that added the subparts of the subdetector to the previou…
Aug 21, 2024
ed814ae
Made edits to the new code and edited the readme file to setup for me…
Sep 4, 2024
e683f9c
Create root_folder.txt
abbyxh Sep 4, 2024
3bb8b87
Create gltf.txt
abbyxh Sep 4, 2024
68ad178
Add files via upload
abbyxh Sep 5, 2024
dca3085
Add files via upload
abbyxh Sep 5, 2024
97fd933
Add files via upload
abbyxh Sep 5, 2024
07b9d9b
Add files via upload
abbyxh Sep 5, 2024
391414b
Add files via upload
abbyxh Sep 5, 2024
07b2013
Add files via upload
abbyxh Sep 5, 2024
2699afa
Add files via upload
abbyxh Sep 5, 2024
fadf0d1
Add files via upload
abbyxh Sep 5, 2024
bfb540e
Delete gltf_files/gltf.txt
abbyxh Sep 5, 2024
891e550
Delete root_files/root_folder.txt
abbyxh Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions bin/full_conversion.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@

import argparse
import ROOT

def main():
parser = argparse.ArgumentParser(description='Convert detector')
parser.add_argument('-c', '--compact', help='Compact file location(s)',
required=True, type=str, nargs='+')
parser.add_argument('-o', '--out', help='Converted file path',
default='nothing', type=str)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
parser.add_argument('-o', '--out', help='Converted file path',
default='nothing', type=str)
parser.add_argument('-o', '--out', help='Converted file path',
default='', type=str)

Unlikely in this case but people might want to use whatever magical string that you put here. You can then no longer differentiate between that magic string and an actual user value. It's easier to deal with an empty string here, because an empty output file doesn't make sense.

parser.add_argument('-v', '--visibility', help='Level of layers in detector',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
parser.add_argument('-v', '--visibility', help='Level of layers in detector',
parser.add_argument('-v', '--visibility-level', help='Level of layers in detector to consider in conversion',

Just to make it easier to understand

default=9, type=int)

args = parser.parse_args()

convert(args.compact, args.out, args.visibility)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
convert(args.compact, args.out, args.visibility)
convert(args.compact, args.out, args.visibility_level)

To keep things working after the argparse changes



def convert(compact_files, out_path, visibility):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be the same function as in conversion_xml2root.py so we could also just import it from there instead of copying it over.

print('INFO: Converting following compact file(s):')
for cfile in compact_files:
print(' ' + cfile)

ROOT.gSystem.Load('libDDCore')
description = ROOT.dd4hep.Detector.getInstance()
for cfile in compact_files:
description.fromXML(cfile)

ROOT.gGeoManager.SetVisLevel(visibility)
ROOT.gGeoManager.SetVisOption(0)

if out_path == 'nothing':

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if out_path == 'nothing':
if not out_path:

to keep things working after the argparse changes


counter = 0
slash_list = []
for cfile in compact_files:
for i in cfile:
counter += 1
if i == '/':
slash_list.append(counter)

last_slash = max(slash_list)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move this into a function to declutter the conversion function. I am also not entirely sure I understand what the output will be yet if there are multiple compact files.

The end result would look something like

    # ... all the rest from above
    ROOT.gGeoManager.SetVisOption(0)

    out_path = determine_outpath(out_path, compact_files)  # to be implemented
    ROOT.gGeoManager.Export(out_path)

ROOT.gGeoManager.Export(f'{cfile[last_slash:len(cfile)-4]}.root')

else:
ROOT.gGeoManager.Export(out_path)

if __name__ == '__main__':
main()
8 changes: 8 additions & 0 deletions bin/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ if (options.outputFile) {
outFileName = options.outputFile;
}

for (var [name, entry] of Object.entries(config.subParts)) {
let temp = [];
for (let path of entry[0]) {
temp.push(new RegExp(path));
}
entry[0] = temp;
}

convertGeometry(inFilePath,
outFileName,
config.maxLevel,
Expand Down
14 changes: 14 additions & 0 deletions calling_root2gltf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import argparse
import subprocess

parser = argparse.ArgumentParser(description='Convert detector (to .gltf)')
parser.add_argument('root_file', help='Root file',
type=str)
parser.add_argument('-c', '--config_file', help='Json file of detector structure',
default='a', type=str)
parser.add_argument('-g', '--gltf_file', help='Gltf file',
default='detector.gltf', type=str)

args = parser.parse_args()

subprocess.run(["node", ".", "-c", f'{args.config_file}', "-o", f'{args.gltf_file}', f'{args.root_file}'])
1 change: 1 addition & 0 deletions config_automatic.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"childrenToHide": [], "subParts": {"BeamCal": [["BeamCal_(?!envelope)\\w+"], 0.8], "Coil": [["Coil_(?!envelope)\\w+"], 0.8], "EcalBarrel": [["EcalBarrel_(?!envelope)\\w+"], 0.8], "EcalEndcap": [["EcalEndcap_(?!envelope)\\w+"], 0.8], "EcalPlug": [["EcalPlug_(?!envelope)\\w+"], 0.8], "FTD": [["FTD_(?!envelope)\\w+", "FTDDisk_.*_negZ", "ftd_petal_negZ_.*_.*", "ftd_sensor_negZ_.*_.*_.*", "FTDDisk_.*_posZ", "ftd_petal_posZ_.*_.*", "ftd_sensor_posZ_.*_.*_.*", "FTD_support"], 0.8], "HcalBarrel": [["HcalBarrel_(?!envelope)\\w+"], 0.8], "HcalEndcap": [["HcalEndcap_(?!envelope)\\w+"], 0.8], "HcalRing": [["HcalRing_(?!envelope)\\w+"], 0.8], "LHCal": [["LHCal_(?!envelope)\\w+"], 0.8], "Lcal": [["Lcal_(?!envelope)\\w+"], 0.8], "QD0_cryostat": [["QD0_cryostat_(?!envelope)\\w+"], 0.8], "QD0_support": [["QD0_support_(?!envelope)\\w+"], 0.8], "SET": [["SET_(?!envelope)\\w+", "set_ladder_.*_.*", "set_ladder_.*_.*_.*"], 0.8], "SIT": [["SIT_(?!envelope)\\w+", "sit_ladder_.*_.*", "sit_ladder_.*_.*_.*"], 0.8], "SServices00": [["SServices00_(?!envelope)\\w+"], 0.8], "TPC": [["TPC_(?!envelope)\\w+", "tpc_endcap_bwd", "tpc_endcap_fwd", "tpc_sensGas_bwd", "tpc_row_bwd_.*", "tpc_sensGas_fwd", "tpc_row_fwd_.*"], 0.8], "Tube": [["Tube_(?!envelope)\\w+"], 0.8], "VXD": [["VXD_(?!envelope)\\w+", "BerylliumAnnulusBlock_.*_negZ_.*", "BerylliumAnnulusBlock_.*_posZ_.*", "ElectronicsEnd_.*_negZ_.*", "ElectronicsEnd_.*_posZ_.*", "SiActiveLayer_.*_negZ_.*", "SiActiveLayer_.*_posZ_.*", "VXD_support", "KaptonLines_.*_negZ", "KaptonLines_.*_posZ", "MetalLines_.*_negZ", "MetalLines_.*_posZ", "VXD_ExtKaptonCab_neg", "VXD_ExtKaptonCab_pos", "VXD_ExtMetalCab_neg", "VXD_ExtMetalCab_pos", "VXD_endplate_bwd", "VXD_endplate_fwd", "VXD_support_bwd_inner", "VXD_support_bwd_outer", "VXD_support_fwd_inner", "VXD_support_fwd_outer", "negShellCone", "posShellCone"], 0.8], "YokeBarrel": [["YokeBarrel_(?!envelope)\\w+"], 0.8], "YokeEndcap": [["YokeEndcap_(?!envelope)\\w+"], 0.8]}, "maxLevel": 3}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an auto-generated output? Very nice. If you want to make it more readable you can use the indent parameter of the json.dump function.

73 changes: 73 additions & 0 deletions configfile_generator.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#!/usr/bin/env python
# coding: utf-8

from dd4hep import Detector
import re
from argparse import ArgumentParser
import pprint
import json

parser = ArgumentParser()

parser.add_argument(
"--compactFile", help="DD4hep compact description xml", required=True
)
parser.add_argument(
"--maxDepth",
help="Maximum traversal depth of the detector tree",
default=10,
type=int,
)

args = parser.parse_args()

theDetector = Detector.getInstance()
theDetector.fromXML(args.compactFile)
start = theDetector.world()

def process_name(raw_name):
name = re.sub(r"\d+", ".*", raw_name)
return name

def tree(detElement, depth, maxDepth):
nd = {}
depth += 1
children = detElement.children()
for raw_name, child in children:
if depth > maxDepth:
tree(child, depth, maxDepth)
else:
dictionary = tree(child, depth, maxDepth)
nd.update({raw_name: dictionary})
return nd

detector_dict = tree(start, 0, args.maxDepth)
#pprint.pprint(detector_dict)

def post_processing(obj, main_parts, subParts={}, sublist= []):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this also be made to take a list of subParts to ignore? That would make it possible to define a list that can be put into childrenToHide without leading to conflicts

for k, v in obj.items():
if k in main_parts:
sublist = [f'{k}_(?!envelope)\\w+']

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the people who don't know regex by heart (like me). Could you put a comment here briefly explaining what (!?envelope) does in the generated regular expression?

outer_list = []
outer_list.append(sublist)
outer_list.append(0.8)
subParts.update({k: outer_list})
post_processing(v, main_parts, subParts, sublist)

else:
k_new = process_name(k)
x = re.search("module|stave|layer|Calorimeter", k_new)
if k_new not in sublist and x == None:
sublist.append(k_new)
post_processing(v, main_parts, subParts, sublist)
return subParts

subPart_processed = post_processing(detector_dict, list(detector_dict.keys()))

final_dict = {"childrenToHide": [],
"subParts": subPart_processed,
"maxLevel": 3}

pprint.pprint(final_dict)
with open("config_automatic.json", "w") as outfile:
json.dump(final_dict, outfile)
28 changes: 28 additions & 0 deletions configs/config_ild_trial2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"childrenToHide": [
"SServices",
"QDX_cryo",
"QDX_support"
],
"subParts": {
"EcalBarrel" : [["EcalBarrel_module\\w+"], 0.8],
"Coil" : [["coil_vol_0"], 0.8],
"Lcal" : [["Lcal_module_0", "Lcal_module_1"], 0.8],
"EcalEndcap" : [["EcalEndcapQuadrant_."], 0.8],
"EcalPlug" : [["ECRing_0", "ECRing_1"], 0.8],
"BeamCal" : [["Beam_Cal_module_0", "Beam_Cal_module_1"], 0.8],
"FTD" : [["FTDAirDiskLogicalPZ_._.", "FTDAirDiskLogicalNZ_._."], 0.8],
"HcalEndcap" : [["HcalEndcap_EndcapModule._.", "Hcal_endcap_FEE_."], 0.8],
"HcalRing" : [["HcalEndCapRingLogical_0", "HcalEndCapRingLogical_1"], 0.8],
"LHCal" : [["LHCal_module_0", "LHCal_module_1"], 0.8],
"HcalBarrel": [["HcalBarrel_module_.", "HcalBarrel_Module_lateral_plate_."], 0.8],
"SIT": [["SIT_LadderLogical\\w+"], 0.8],
"VXD": [["SiActiveLayer\\w+"], 0.8],
"SET": [["SET_LadderLogical\\w+"], 0.8],
"TPC": [["TPC\\w+"], 0.8],
"Tube": [["tube\\w+"], 0.8],
"YokeBarrel": [["YokeBarrel\\w+"], 0.8],
"YokeEndcap": [["YokeEndcap\\w+"], 0.8]
},
"maxLevel": 3
}
81 changes: 81 additions & 0 deletions conversion_xml2gltf_autoConfig.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@

import argparse
import ROOT
import subprocess

def main():
parser = argparse.ArgumentParser(description='Convert detector')
parser.add_argument('-cm', '--compact_file', help='Compact file location(s)',
required=True, type=str, nargs='+')
parser.add_argument('-cn', '--config_file', help='Json file of detector structure',
default='nothing', type=str)
parser.add_argument('-ro', '--out_root', help='Converted root file path',
default='nothing', type=str)
parser.add_argument('-ri', '--in_root', help='Input root file path (if a root file has already been obtained)',
default='nothing', type=str)
parser.add_argument('-g', '--out_gltf', help='Converted gltf file path',
default='nothing', type=str)
parser.add_argument('-d', '--depth', help='Level of layers in detector',
default=9, type=int)

args = parser.parse_args()

#if args.in_root != 'nothing' and args.config_file != 'nothing':
# gltf_convert(args.config_file, args.out_gltf, args.in_root)
#else:
print(args.compact_file)
root_path = root_convert(args.compact_file, args.out_root, args.depth)

if args.config_file == 'nothing':
subprocess.run(["python", "configfile_generator.py", "--compactFile", f'{args.compact_file}'])
else:
config_file = args.config_file

gltf_convert(config_file, args.out_gltf, root_path)


def root_convert(compact_files, out_path, visibility):
print('INFO: Converting following compact file(s):')
for cfile in compact_files:
print(' ' + cfile)

ROOT.gSystem.Load('libDDCore')
description = ROOT.dd4hep.Detector.getInstance()
for cfile in compact_files:
description.fromXML(cfile)

ROOT.gGeoManager.SetVisLevel(visibility)
ROOT.gGeoManager.SetVisOption(0)

if out_path == 'nothing':
counter = 0
slash_list = []
for cfile in compact_files:
for i in cfile:
counter += 1
if i == '/':
slash_list.append(counter)

last_slash = max(slash_list)
root_path = f'{cfile[last_slash:len(cfile)-4]}.root'

else:
root_path = out_path

ROOT.gGeoManager.Export(root_path)

return root_path

def gltf_convert(config, gltf, root):

if gltf == 'nothing':
gltf_path = f'{root[:len(root)-5]}.gltf'

else:
gltf_path = gltf

subprocess.run(["node", ".", "-c", f'{config}', "-o", f'{gltf_path}', f'{root}'])


if __name__ == '__main__':
main()
67 changes: 67 additions & 0 deletions conversion_xml2root.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@

import argparse
import ROOT
import subprocess

def main():
parser = argparse.ArgumentParser(description='Convert detector')
parser.add_argument('-cm', '--compact', help='Compact file location(s)',
required=True, type=str, nargs='+')
parser.add_argument('-r', '--out_root', help='Converted root file path',
default='nothing', type=str)
parser.add_argument('-v', '--visibility', help='Level of layers in detector',
default=9, type=int)
parser.add_argument('-g', '--out_gltf', help='Converted gltf file path',
default='nothing', type=str)
parser.add_argument('-cn', '--config_file', help='Json file of detector structure',
default='a', type=str)
args = parser.parse_args()

root_path = root_convert(args.compact, args.out_root, args.visibility)
gltf_convert(args.config_file, args.out_gltf, root_path)

def root_convert(compact_files, out_path, visibility):
print('INFO: Converting following compact file(s):')
for cfile in compact_files:
print(' ' + cfile)

ROOT.gSystem.Load('libDDCore')
description = ROOT.dd4hep.Detector.getInstance()
for cfile in compact_files:
description.fromXML(cfile)

ROOT.gGeoManager.SetVisLevel(visibility)
ROOT.gGeoManager.SetVisOption(0)

if out_path == 'nothing':
counter = 0
slash_list = []
for cfile in compact_files:
for i in cfile:
counter += 1
if i == '/':
slash_list.append(counter)

last_slash = max(slash_list)
root_path = f'{cfile[last_slash:len(cfile)-4]}.root'

else:
root_path = out_path

ROOT.gGeoManager.Export(root_path)

return root_path


def gltf_convert(config, gltf, root):

if gltf == 'nothing':
gltf_path = f'{root[:len(root)-5]}.gltf'

else:
gltf_path = gltf

subprocess.run(["node", ".", "-c", f'{config}', "-o", f'{gltf_path}', f'{root}'])

if __name__ == '__main__':
main()
Loading