ROM hyper reduction for density evaluation

[dreamer2368]

testROMRhoOperator tests hyper-reduction for electron density evaluation.

• computeRhoOnSamplePts evaluates electron density at the sampled grid points, given density matrix, orbital POD basis and ROM orbital coefficients.
  • This is equivalent to Rho::computeRho without rescaleTotalCharge.
• Since rescaleTotalCharge requires every grid point information, it cannot be executed with hyper-reduction. It can be still performed after the electron density is projected onto ROM space via hyperreduction.
• For code verification, exact equivalence between ROM and FOM can be achieved only when we have access to density before rescaling. This would require additional restart file save/load routines, which wouldn't be used at all in production runs. For now, the code is tested with adjusted error tolerance.

[dreamer2368]

@jeanlucf22 , this PR is still work-in-progress, but I'm wondering if I can get checked with you on one part. Can you take a look at line 517-550 of src/rom_workflows.cc, the function computeRhoOnSamplePts?

In essence, it computes the density rho only on sampled grid points, using libROM Matrix and Vector.

• It is mainly based on computeRhoSubdomainUsingBlas3
• For now we assume Mesh::subdivx() to be 1. This is the number of grids?
• Also we assume density matrix is not distributed, so only 1 local matrix.

I'm wondering these assumptions are okay. These assumptions are also checked within the routines, so users won't misuse it unknowingly. Comparing with the slide, and also with computeRhoSubdomain, computeRhoSubdomainUsingBlas3 didn't seem to have a scalar factor of 2. The newly implemented function computeRhoOnSamplePts also does not include the factor of 2. I'm wondering if this is correct, or if I missed anything.

[jeanlucf22]

"spin" can be ignored for now. when spin is on, it is like having two replicated copy of everything, on two different sets of processors, and in this case rho is the sum of these two parts -> gather op

[jeanlucf22]

this is to make sure the charge is numerically exactly the physical charge. Sometimes, a slight rescaling is needed for that.

[dreamer2368]

We'll have to discuss this next week, but hyperreduction is not applicable for rescaleTotalCharge. We can still perform rescaling after (unscaled) density is projected onto ROM basis.
I mentioned a caveat for code verification above in the description.

[jeanlucf22]

@jeanlucf22 , this PR is still work-in-progress, but I'm wondering if I can get checked with you on one part. Can you take a look at line 517-550 of src/rom_workflows.cc, the function computeRhoOnSamplePts?

In essence, it computes the density rho only on sampled grid points, using libROM Matrix and Vector.

• It is mainly based on computeRhoSubdomainUsingBlas3
• For now we assume Mesh::subdivx() to be 1. This is the number of grids?
• Also we assume density matrix is not distributed, so only 1 local matrix.

I'm wondering these assumptions are okay. These assumptions are also checked within the routines, so users won't misuse it unknowingly. Comparing with the slide, and also with computeRhoSubdomain, computeRhoSubdomainUsingBlas3 didn't seem to have a scalar factor of 2. The newly implemented function computeRhoOnSamplePts also does not include the factor of 2. I'm wondering if this is correct, or if I missed anything.

Indeed subdivx can be 1. It is an implementation optimization, sometimes using subdivx>1 to divide a subdomain associated with 1 MPI tasks into several sub-subdomains.
No factor 2 should be correct.

[dreamer2368]

Indeed subdivx can be 1. It is an implementation optimization, sometimes using subdivx>1 to divide a subdomain associated with 1 MPI tasks into several sub-subdomains. No factor 2 should be correct.

For now, computeRhoOnSamplePts keeps consistency with computeRhoSubdomainUsingBlas3 by not having the factor 2. This is the used routine for carbyne example. @jeanlucf22 , it'd be great if you can point me to where the factor 2 is multiplied outside of computeRhoSubdomainUsingBlas3, or whether computeRhoSubdomainUsingBlas3 needs to be fixed with factor 2.

[jeanlucf22]

Indeed subdivx can be 1. It is an implementation optimization, sometimes using subdivx>1 to divide a subdomain associated with 1 MPI tasks into several sub-subdomains. No factor 2 should be correct.

For now, computeRhoOnSamplePts keeps consistency with computeRhoSubdomainUsingBlas3 by not having the factor 2. This is the used routine for carbyne example. @jeanlucf22 , it'd be great if you can point me to where the factor 2 is multiplied outside of computeRhoSubdomainUsingBlas3, or whether computeRhoSubdomainUsingBlas3 needs to be fixed with factor 2.

The factor 2 should be in localX

[jeanlucf22]

@dreamer2368 Can you squash your commits so that at least the git commit comments are meaningful? Thanks

[dreamer2368]

@jeanlucf22 , if you're okay, I can do the squash and merge and edit the commit message there, since the squashing is the code practice here anyway.

[jeanlucf22]

@jeanlucf22 , if you're okay, I can do the squash and merge and edit the commit message there, since the squashing is the code practice here anyway.

Yes, please go ahead