CASSCF calculation on large system

[similt]

Dear all,
I had submitted a few CASSCF(4o,4e) jobs for geometry optimization and nonadiabatic coupling for a system with 560 electrons. But all of them crashed. From the output file, "Storage requirement is 121.566 GB".
Is it possible to complete these jobs using bagel?
(Single point CASSCF(4o,4e) calculations were successfully completed.)
I am using CRAY machine with 128GB RAM in single node and tried to use up to 20 nodes.
Any help is greatly appreciated.

[Lenara98]

I have the same problem, but my molecular system is not as big as yours. Have you found a solution to this problem?

[shiozaki]

If you can post a minimal input and output here I can take a look when I have time. Likely some issues with the input.

[Lenara98]

Molecule has 36 atoms
input:
{
"title" : "hf"
},
{
"title" : "casscf",
"nstate" : 1,
"charge" : 2,
"nact" : 4,
"nspin" : 0,
"maxiter":500,
"nclosed": 89,
"active" : [90, 91, 92, 93],
"davidson_subspace" : 3
},
{
"title" : "optimize",
"opttype" : "energy",
"target" : 0,
"numerical":false,
"hess_approx":true,
"adaptive":false,
"method" : [ {
"title" : "caspt2",
"smith" : {
"method" : "caspt2",
"ms" : "true",
"xms" : "true",
"sssr" : "true",
"shift" : 0.2,
"maxstep":0.1,
"maxiter":500,
"frozen" : true,
"internal" : false,
"davidson_subspace" : 3,
"maxgrad":0.0001
},
"nstate" : 1,
"nact" : 4,
"nspin" : 0,
"nclosed" : 89,
"maxiter":500,
"charge":2,
"maxiter_micro":500,
"thresh":5e-7
} ]
},
{
"title" : "hessian",
"target" : 0,
"method" : [ {
"title" : "caspt2",
"smith" : {
"method" : "caspt2",
"ms" : "true",
"xms" : "true",
"sssr" : "true",
"shift" : 0.2,
"davidson_subspace" : 3,
"frozen" : true
},
"nstate" : 1,
"nact" : 4,
"nspin" : 0,
"nclosed" : 89,
"maxiter":500,
"charge":2,
"maxiter_micro":500,
"thresh":5e-7
} ]
},
{
"title" : "print",
"file" : "orbitals.molden",
"orbitals" : true
},
{
"title" : "print",
"file" : "freq.molden",
"vibration" : true
}
]}
otput ended with:
=== DF-CASPT2Grad calculation ===

* freezing 28 orbitals

* Approximate memory requirement for SMITH calculation per MPI process:
  o Storage requirement for T-amplitude, lambda, and residual is 19.44 GB
  o Storage requirement for MO integrals is 0.99 GB
  o Storage requirement for SMITH-computed gradient tensors is 0.00 GB
   - MO integral evaluation                   68.08
* Calculating and saving rdm4

- linear dependency detected:    3 /   32    min eigenvalue:     3.8139e-15    max eigenvalue:     1.4535e-13
- linear dependency detected:   44 /   64    min eigenvalue:    -3.2645e-15    max eigenvalue:     5.2220e-15
- linear dependency detected:   44 /   64    min eigenvalue:    -6.6461e-15    max eigenvalue:     6.2272e-15
   - RDM + denominator evaluation              0.03

* Zeroth order energy : state  0       -2.2042181472

* CASPT2 iteration is performed using redundant basis

  ---- iteration ----

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 18075 RUNNING AT node280
= EXIT CODE: 7
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES

[shiozaki]

I would need an entire output and input.

ps: if you can give me an input that crashes on my laptop the chance of me being able to trouble shoot increases.

[Lenara98]

4OarS.log

[shiozaki]

One more question. Are you using a latest release or the master branch?

[Lenara98]

Generated by GNU Autoconf 2.69 for BAGEL 1.2.0.

[xubwa]

I tried running this calculation on a 28 cores node/128G of memory and the calculation seems to be fine. I did both 1core/mpi process (mpirun -n 28) and 6core/mpi process(mpirun -n 4), the memory consumption per mpi process is
o Storage requirement for T-amplitude, lambda, and residual is 0.93 GB
o Storage requirement for MO integrals is 0.14 GB
o Storage requirement for SMITH-computed gradient tensors is 0.00 GB
and
o Storage requirement for T-amplitude, lambda, and residual is 6.48 GB
o Storage requirement for MO integrals is 0.99 GB
o Storage requirement for SMITH-computed gradient tensors is 0.00 GB
It's not a gigantic calculation at all. The storage requirement that you @Lenara98 reported seems a bit suspicious to me, maybe there's some problem with your compilation. I'm also curious how much memory do you have per node?
For your question @similt, if you use more nodes, the memory requirement per node will be reduced. I would be willing to take a look if you can upload your input.

[Lenara98]

The system has 16 Gb per node. I suppose that the problem with my computing system. I'll try to set correct parameters (nodes, ppn). Thanks for your help in solving problem.

[similt]

@xubwa
Here is my input:

{ "bagel" : [
{
"title" : "molecule",
"basis" : "cc-pvdz",
"df_basis" : "cc-pvdz-jkfit",
"angstrom" : "true",
"geometry" : [
{ "atom" : "C", "xyz" : [ -0.791431, 3.172867, 0.728499] },
{ "atom" : "C", "xyz" : [ 0.188472, 3.951044, 1.416907] },
{ "atom" : "C", "xyz" : [ 1.009243, 4.830764, 0.714381] },
{ "atom" : "C", "xyz" : [ 1.946892, 5.702281, 1.391868] },
{ "atom" : "C", "xyz" : [ 2.772090, 6.516424, 0.715404] },
{ "atom" : "C", "xyz" : [ 0.546932, 3.738494, 2.864750] },
{ "atom" : "C", "xyz" : [ 0.158482, 4.603473, 3.881233] },
{ "atom" : "C", "xyz" : [ 0.616830, 4.425231, 5.174998] },
{ "atom" : "C", "xyz" : [ 1.479324, 3.381430, 5.473645] },
{ "atom" : "C", "xyz" : [ 1.882266, 2.521059, 4.466910] },
{ "atom" : "C", "xyz" : [ 1.422557, 2.702860, 3.171469] },
{ "atom" : "H", "xyz" : [ 1.971590, 5.704531, 2.460710] },
{ "atom" : "H", "xyz" : [ 3.444649, 7.165753, 1.246754] },
{ "atom" : "H", "xyz" : [ -0.506693, 5.417345, 3.659154] },
{ "atom" : "H", "xyz" : [ 0.300443, 5.101255, 5.949146] },
{ "atom" : "H", "xyz" : [ 1.837403, 3.246308, 6.478755] },
{ "atom" : "H", "xyz" : [ 2.560757, 1.714703, 4.682697] },
{ "atom" : "H", "xyz" : [ 1.756193, 2.043351, 2.390278] },
{ "atom" : "C", "xyz" : [ -1.769356, 2.394043, 1.416916] },
{ "atom" : "C", "xyz" : [ -2.600334, 1.524184, 0.714157] },
{ "atom" : "C", "xyz" : [ -3.543334, 0.657238, 1.391267] },
{ "atom" : "C", "xyz" : [ -4.368666, -0.158142, 0.715430] },
{ "atom" : "C", "xyz" : [ -2.107021, 2.585325, 2.872597] },
{ "atom" : "C", "xyz" : [ -1.711099, 1.696122, 3.865438] },
{ "atom" : "C", "xyz" : [ -2.148037, 1.850854, 5.169336] },
{ "atom" : "C", "xyz" : [ -2.997791, 2.894965, 5.503019] },
{ "atom" : "C", "xyz" : [ -3.407588, 3.779472, 4.520865] },
{ "atom" : "C", "xyz" : [ -2.968096, 3.620881, 3.215423] },
{ "atom" : "H", "xyz" : [ -3.574410, 0.661108, 2.459934] },
{ "atom" : "H", "xyz" : [ -5.043632, -0.804844, 1.246568] },
{ "atom" : "H", "xyz" : [ -1.058451, 0.880456, 3.615921] },
{ "atom" : "H", "xyz" : [ -1.823347, 1.157150, 5.924268] },
{ "atom" : "H", "xyz" : [ -3.338878, 3.012806, 6.516077] },
{ "atom" : "H", "xyz" : [ -4.075265, 4.586816, 4.764615] },
{ "atom" : "H", "xyz" : [ -3.302582, 4.302235, 2.453498] },
{ "atom" : "C", "xyz" : [ -0.791431, 3.172867, -0.728498] },
{ "atom" : "C", "xyz" : [ -1.769355, 2.394043, -1.416916] },
{ "atom" : "C", "xyz" : [ -2.600334, 1.524184, -0.714157] },
{ "atom" : "C", "xyz" : [ -3.543334, 0.657237, -1.391264] },
{ "atom" : "C", "xyz" : [ -4.368666, -0.158142, -0.715426] },
{ "atom" : "C", "xyz" : [ -2.107015, 2.585329, -2.872598] },
{ "atom" : "C", "xyz" : [ -1.711091, 1.696127, -3.865439] },
{ "atom" : "C", "xyz" : [ -2.148028, 1.850859, -5.169337] },
{ "atom" : "C", "xyz" : [ -2.997783, 2.894970, -5.503020] },
{ "atom" : "C", "xyz" : [ -3.407581, 3.779476, -4.520867] },
{ "atom" : "C", "xyz" : [ -2.968090, 3.620885, -3.215424] },
{ "atom" : "H", "xyz" : [ -3.574411, 0.661105, -2.459931] },
{ "atom" : "H", "xyz" : [ -5.043632, -0.804846, -1.246564] },
{ "atom" : "H", "xyz" : [ -1.058443, 0.880461, -3.615922] },
{ "atom" : "H", "xyz" : [ -1.823338, 1.157155, -5.924269] },
{ "atom" : "H", "xyz" : [ -3.338869, 3.012811, -6.516079] },
{ "atom" : "H", "xyz" : [ -4.075258, 4.586819, -4.764617] },
{ "atom" : "H", "xyz" : [ -3.302577, 4.302238, -2.453499] },
{ "atom" : "C", "xyz" : [ 0.188471, 3.951044, -1.416907] },
{ "atom" : "C", "xyz" : [ 1.009242, 4.830763, -0.714380] },
{ "atom" : "C", "xyz" : [ 1.946892, 5.702283, -1.391865] },
{ "atom" : "C", "xyz" : [ 2.772090, 6.516425, -0.715400] },
{ "atom" : "C", "xyz" : [ 0.546926, 3.738490, -2.864750] },
{ "atom" : "C", "xyz" : [ 0.158474, 4.603468, -3.881233] },
{ "atom" : "C", "xyz" : [ 0.616822, 4.425226, -5.174999] },
{ "atom" : "C", "xyz" : [ 1.479316, 3.381426, -5.473646] },
{ "atom" : "C", "xyz" : [ 1.882259, 2.521056, -4.466912] },
{ "atom" : "C", "xyz" : [ 1.422551, 2.702856, -3.171470] },
{ "atom" : "H", "xyz" : [ 1.971591, 5.704534, -2.460707] },
{ "atom" : "H", "xyz" : [ 3.444649, 7.165754, -1.246749] },
{ "atom" : "H", "xyz" : [ -0.506701, 5.417340, -3.659155] },
{ "atom" : "H", "xyz" : [ 0.300435, 5.101250, -5.949147] },
{ "atom" : "H", "xyz" : [ 1.837395, 3.246303, -6.478757] },
{ "atom" : "H", "xyz" : [ 2.560751, 1.714700, -4.682698] },
{ "atom" : "H", "xyz" : [ 1.756187, 2.043348, -2.390279] },
{ "atom" : "C", "xyz" : [ 0.791431, -3.172867, 0.728498] },
{ "atom" : "C", "xyz" : [ 1.769355, -2.394043, 1.416916] },
{ "atom" : "C", "xyz" : [ 2.600333, -1.524183, 0.714157] },
{ "atom" : "C", "xyz" : [ 3.543334, -0.657236, 1.391264] },
{ "atom" : "C", "xyz" : [ 4.368666, 0.158143, 0.715426] },
{ "atom" : "C", "xyz" : [ 2.107012, -2.585330, 2.872598] },
{ "atom" : "C", "xyz" : [ 1.711088, -1.696128, 3.865439] },
{ "atom" : "C", "xyz" : [ 2.148025, -1.850860, 5.169337] },
{ "atom" : "C", "xyz" : [ 2.997780, -2.894970, 5.503021] },
{ "atom" : "C", "xyz" : [ 3.407579, -3.779476, 4.520867] },
{ "atom" : "C", "xyz" : [ 2.968088, -3.620885, 3.215425] },
{ "atom" : "H", "xyz" : [ 3.574410, -0.661104, 2.459931] },
{ "atom" : "H", "xyz" : [ 5.043632, 0.804846, 1.246563] },
{ "atom" : "H", "xyz" : [ 1.058439, -0.880463, 3.615922] },
{ "atom" : "H", "xyz" : [ 1.823334, -1.157156, 5.924270] },
{ "atom" : "H", "xyz" : [ 3.338866, -3.012812, 6.516080] },
{ "atom" : "H", "xyz" : [ 4.075257, -4.586819, 4.764618] },
{ "atom" : "H", "xyz" : [ 3.302575, -4.302238, 2.453500] },
{ "atom" : "C", "xyz" : [ -0.188470, -3.951044, 1.416907] },
{ "atom" : "C", "xyz" : [ -1.009242, -4.830764, 0.714381] },
{ "atom" : "C", "xyz" : [ -1.946891, -5.702283, 1.391865] },
{ "atom" : "C", "xyz" : [ -2.772089, -6.516426, 0.715400] },
{ "atom" : "C", "xyz" : [ -0.546923, -3.738489, 2.864751] },
{ "atom" : "C", "xyz" : [ -0.158471, -4.603466, 3.881234] },
{ "atom" : "C", "xyz" : [ -0.616819, -4.425224, 5.175000] },
{ "atom" : "C", "xyz" : [ -1.479313, -3.381424, 5.473647] },
{ "atom" : "C", "xyz" : [ -1.882257, -2.521055, 4.466912] },
{ "atom" : "C", "xyz" : [ -1.422549, -2.702855, 3.171471] },
{ "atom" : "H", "xyz" : [ -1.971591, -5.704535, 2.460707] },
{ "atom" : "H", "xyz" : [ -3.444648, -7.165755, 1.246749] },
{ "atom" : "H", "xyz" : [ 0.506705, -5.417337, 3.659155] },
{ "atom" : "H", "xyz" : [ -0.300431, -5.101248, 5.949148] },
{ "atom" : "H", "xyz" : [ -1.837391, -3.246302, 6.478758] },
{ "atom" : "H", "xyz" : [ -2.560749, -1.714699, 4.682699] },
{ "atom" : "H", "xyz" : [ -1.756186, -2.043347, 2.390280] },
{ "atom" : "C", "xyz" : [ 0.791431, -3.172867, -0.728499] },
{ "atom" : "C", "xyz" : [ -0.188471, -3.951044, -1.416907] },
{ "atom" : "C", "xyz" : [ -1.009242, -4.830764, -0.714381] },
{ "atom" : "C", "xyz" : [ -1.946891, -5.702282, -1.391868] },
{ "atom" : "C", "xyz" : [ -2.772089, -6.516425, -0.715404] },
{ "atom" : "C", "xyz" : [ -0.546930, -3.738493, -2.864750] },
{ "atom" : "C", "xyz" : [ -0.158479, -4.603472, -3.881233] },
{ "atom" : "C", "xyz" : [ -0.616828, -4.425230, -5.174999] },
{ "atom" : "C", "xyz" : [ -1.479322, -3.381429, -5.473645] },
{ "atom" : "C", "xyz" : [ -1.882264, -2.521059, -4.466911] },
{ "atom" : "C", "xyz" : [ -1.422555, -2.702859, -3.171470] },
{ "atom" : "H", "xyz" : [ -1.971590, -5.704532, -2.460710] },
{ "atom" : "H", "xyz" : [ -3.444648, -7.165753, -1.246754] },
{ "atom" : "H", "xyz" : [ 0.506695, -5.417343, -3.659154] },
{ "atom" : "H", "xyz" : [ -0.300441, -5.101254, -5.949147] },
{ "atom" : "H", "xyz" : [ -1.837401, -3.246307, -6.478756] },
{ "atom" : "H", "xyz" : [ -2.560756, -1.714703, -4.682697] },
{ "atom" : "H", "xyz" : [ -1.756191, -2.043351, -2.390279] },
{ "atom" : "C", "xyz" : [ 1.769356, -2.394043, -1.416916] },
{ "atom" : "C", "xyz" : [ 2.600334, -1.524183, -0.714157] },
{ "atom" : "C", "xyz" : [ 3.543333, -0.657238, -1.391267] },
{ "atom" : "C", "xyz" : [ 4.368666, 0.158142, -0.715430] },
{ "atom" : "C", "xyz" : [ 2.107019, -2.585326, -2.872597] },
{ "atom" : "C", "xyz" : [ 1.711097, -1.696123, -3.865438] },
{ "atom" : "C", "xyz" : [ 2.148034, -1.850856, -5.169336] },
{ "atom" : "C", "xyz" : [ 2.997788, -2.894966, -5.503020] },
{ "atom" : "C", "xyz" : [ 3.407585, -3.779473, -4.520866] },
{ "atom" : "C", "xyz" : [ 2.968094, -3.620882, -3.215424] },
{ "atom" : "H", "xyz" : [ 3.574409, -0.661108, -2.459934] },
{ "atom" : "H", "xyz" : [ 5.043631, 0.804845, -1.246569] },
{ "atom" : "H", "xyz" : [ 1.058448, -0.880458, -3.615921] },
{ "atom" : "H", "xyz" : [ 1.823344, -1.157152, -5.924269] },
{ "atom" : "H", "xyz" : [ 3.338875, -3.012808, -6.516078] },
{ "atom" : "H", "xyz" : [ 4.075263, -4.586817, -4.764616] },
{ "atom" : "H", "xyz" : [ 3.302580, -4.302235, -2.453498] }
]
},

{
"title" : "nacme",
"target" : 3,
"target2" : 2,
"method" : [ {
"title" : "casscf",
"nstate" : 5,
"nact" : 4,
"nclosed" : 278,
"active" : [279, 280, 281, 282]
} ]
}
]}