Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory errors on snap-02 #759

Closed
tomhughes opened this issue Oct 16, 2022 · 12 comments
Closed

Memory errors on snap-02 #759

tomhughes opened this issue Oct 16, 2022 · 12 comments
Labels
hardware:ram-failure location:ucl The data centre sponsored by UCL
Milestone
2023 Mar UCL Visit

Comments

@tomhughes
Copy link
Member

It seems snap-02 has reported a lot of (corrected) memory errors today:

Memory controller events:
1 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470094872512, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16f2d col:0x3f8
2 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473919553472, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d7b col:0x3f8
3 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473836715968, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d37 col:0x3f8
4 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473551503296, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c1f col:0x3f8
5 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472963514304, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169f4 col:0x3f8
6 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468919943104, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16acb col:0x3f8
7 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474524319680, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fbe col:0x3f8
8 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469960130496, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16eab col:0x3f8
9 2022-10-16 06:37:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469389705152, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c8b col:0x3f8
10 2022-10-16 06:37:57 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474388529088, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16f38 col:0x3f8
11 2022-10-16 06:38:31 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468786249664, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16a4d col:0x3f8
12 2022-10-16 06:38:38 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468383072192, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168cb col:0x3f8
13 2022-10-16 06:38:50 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473652166592, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c7f col:0x3f8
14 2022-10-16 06:38:54 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473199181760, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ad7 col:0x3f8
15 2022-10-16 06:39:11 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473600524224, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c52 col:0x3f8
16 2022-10-16 06:39:27 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472846598080, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1697e col:0x3f8
17 2022-10-16 06:39:55 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468634730432, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169c3 col:0x3f8
18 2022-10-16 06:46:57 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469708210112, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16dc2 col:0x3f8
19 2022-10-16 06:47:23 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468969750464, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b01 col:0x3f8
20 2022-10-16 06:47:38 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473197608896, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ad1 col:0x3f8
21 2022-10-16 06:48:42 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470060531648, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16f0a col:0x3f8
22 2022-10-16 06:53:07 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473852706752, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d3c col:0x3f8
23 2022-10-16 06:54:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468467220416, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16924 col:0x3f8
24 2022-10-16 06:55:24 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473047662528, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16a3d col:0x3f8
25 2022-10-16 06:55:49 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468652556224, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169cf col:0x3f8
26 2022-10-16 06:56:49 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470212313024, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fa5 col:0x3f8
27 2022-10-16 06:58:59 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469238448064, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c02 col:0x3f8
28 2022-10-16 07:00:24 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473585057728, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c3f col:0x3f8
29 2022-10-16 07:00:32 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469155610560, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16bae col:0x3f8
30 2022-10-16 07:01:51 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473650855872, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c7a col:0x3f8
31 2022-10-16 07:04:31 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470295150528, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fe9 col:0x3f8
32 2022-10-16 07:04:55 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469843476416, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e46 col:0x3f8
33 2022-10-16 07:05:44 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469792096192, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e0a col:0x3f8
34 2022-10-16 07:11:36 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473987710912, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16dbf col:0x3f8
35 2022-10-16 07:13:20 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469070675904, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b62 col:0x3f8
36 2022-10-16 07:13:25 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469674917824, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16da3 col:0x3f8
37 2022-10-16 07:15:41 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473517424576, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16bfd col:0x3f8
38 2022-10-16 07:20:25 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469658140608, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d8b col:0x3f8
39 2022-10-16 07:23:49 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469960916928, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16eae col:0x3f8
40 2022-10-16 07:25:26 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469339897792, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c65 col:0x3f8
41 2022-10-16 07:29:13 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469557477312, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d2b col:0x3f8
42 2022-10-16 07:30:10 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472510005184, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1683a col:0x3f8
43 2022-10-16 07:31:07 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469573468096, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d40 col:0x3f8
44 2022-10-16 07:34:25 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469692481472, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16dae col:0x3f8
45 2022-10-16 07:37:27 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469088763840, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b6f col:0x3f8
46 2022-10-16 07:37:56 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472677777344, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168da col:0x3f8
47 2022-10-16 07:37:57 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468365508544, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168c0 col:0x3f8
48 2022-10-16 07:40:09 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469339635648, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c64 col:0x3f8
49 2022-10-16 07:40:53 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474423656384, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16f5e col:0x3f8
50 2022-10-16 07:44:08 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472846073792, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1697c col:0x3f8
51 2022-10-16 07:45:44 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469925789632, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e88 col:0x3f8
52 2022-10-16 07:50:43 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473432227776, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16bb0 col:0x3f8
53 2022-10-16 07:51:48 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468501036992, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16945 col:0x3f8
54 2022-10-16 07:53:44 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473685721024, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c9f col:0x3f8
55 2022-10-16 07:56:44 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469910060992, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e84 col:0x3f8
56 2022-10-16 08:00:10 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469859729344, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e4c col:0x3f8
57 2022-10-16 08:08:52 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468315176896, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16888 col:0x3f8
58 2022-10-16 08:09:32 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473164316608, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ab2 col:0x3f8
59 2022-10-16 08:11:51 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468886912960, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16aad col:0x3f8
60 2022-10-16 08:20:22 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468298399680, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16880 col:0x3f8
61 2022-10-16 08:24:04 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474053246912, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16df9 col:0x3f8
62 2022-10-16 08:26:15 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474171736000, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e75 col:0x3f8
63 2022-10-16 08:27:26 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472610144192, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16898 col:0x3f8
64 2022-10-16 08:27:36 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473601310656, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c55 col:0x3f8
65 2022-10-16 08:31:01 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474541359040, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fd7 col:0x3f8
66 2022-10-16 08:35:33 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469406220224, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ca2 col:0x3f8
67 2022-10-16 08:43:35 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470244556736, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fc0 col:0x3f8
68 2022-10-16 08:44:28 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469674131392, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16da0 col:0x3f8
69 2022-10-16 08:44:30 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468936458176, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ae2 col:0x3f8
70 2022-10-16 08:46:28 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469943091136, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ea2 col:0x3f8
71 2022-10-16 08:52:35 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469791571904, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e08 col:0x3f8
72 2022-10-16 08:54:38 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469810184128, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e27 col:0x3f8
73 2022-10-16 09:00:11 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468819279808, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16a6b col:0x3f8
74 2022-10-16 09:05:19 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470212050880, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16fa4 col:0x3f8
75 2022-10-16 09:06:22 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472979505088, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169f9 col:0x3f8
76 2022-10-16 09:07:58 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472662048704, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168d6 col:0x3f8
77 2022-10-16 09:15:26 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472494014400, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16835 col:0x3f8
78 2022-10-16 09:21:07 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474087849920, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e1d col:0x3f8
79 2022-10-16 09:28:14 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468466696128, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16922 col:0x3f8
80 2022-10-16 09:29:17 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469138309056, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ba4 col:0x3f8
81 2022-10-16 09:31:45 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472577900480, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1687d col:0x3f8
82 2022-10-16 09:32:03 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473668419520, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c95 col:0x3f8
83 2022-10-16 09:33:25 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469036597184, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b40 col:0x3f8
84 2022-10-16 09:49:57 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472695865280, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168f7 col:0x3f8
85 2022-10-16 09:50:34 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 474103316416, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16e30 col:0x3f8
86 2022-10-16 09:55:15 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 470009675712, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ee0 col:0x3f8
87 2022-10-16 09:55:56 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472845549504, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1697a col:0x3f8
88 2022-10-16 09:59:26 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473264455616, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b10 col:0x3f8
89 2022-10-16 09:59:47 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469357199296, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16c6f col:0x3f8
90 2022-10-16 10:01:48 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468299186112, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16883 col:0x3f8
91 2022-10-16 10:05:23 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 469977432000, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ec5 col:0x3f8
92 2022-10-16 10:52:59 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468248068032, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16848 col:0x3f8
93 2022-10-16 10:56:21 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468919418816, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16ac9 col:0x3f8
94 2022-10-16 11:14:08 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473331826624, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16b51 col:0x3f8
95 2022-10-16 11:25:06 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 473818628032, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16d1a col:0x3f8
96 2022-10-16 11:26:39 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468333789120, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x168a7 col:0x3f8
97 2022-10-16 11:26:52 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468566835136, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16980 col:0x3f8
98 2022-10-16 11:31:25 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468249378752, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x1684d col:0x3f8
99 2022-10-16 11:35:16 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468684275648, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169e8 col:0x3f8
100 2022-10-16 11:49:30 +0000 1 Corrected error(s): memory read error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 472913444800, grain 5, syndrome 0  err_code:0x0101:0x0091 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x169bd col:0x3f8
101 2022-10-16 11:49:30 +0000 181 Corrected error(s): memory scrubbing error at CPU_SrcID#1_MC#1_Chan#1_DIMM#0 location: 3:1:0:-1, addr 468568670144, grain 5, syndrome 0  OVERFLOW err_code:0x0008:0x00c1 socket:1 imc:1 rank:0 bg:1 ba:3 row:0x16987 col:0x3f8
@tomhughes tomhughes added location:ucl The data centre sponsored by UCL hardware:ram-failure labels Oct 16, 2022
@pnorman
Copy link
Collaborator

pnorman commented Oct 17, 2022

This was purchased from Sentral in 2020-03-12, right before COVID. Invoice SI/5862.

3 year return to base warranty. I've contacted Sentral to ask about fixing the problems.

@Firefishy
Copy link
Member

Sentral has closed. They have passed us onto Supermicro directly. I am following up with Supermicro.

@Firefishy
Copy link
Member

We have been passed to Supermicro. They may honour the warranty. I am completed their RMA form.

@Firefishy Firefishy self-assigned this Nov 3, 2022
@Firefishy
Copy link
Member

Faulty DIMM serial: F0E343C7 / S802C252001F0E343C7

@pnorman
Copy link
Collaborator

pnorman commented Nov 18, 2022

RMA in progress

@Firefishy
Copy link
Member

Ordered replacement RAM module for bargainhardware.co.uk. Should arrive Wednesday.

On trip to UCL / Slough I will replace faulty DIMM and collect module for RMA.

@Firefishy Firefishy added this to the 2022 Dec UCL Visit milestone Dec 12, 2022
@Firefishy
Copy link
Member

Purchased replacement has arrived.

@Firefishy
Copy link
Member

Firefishy commented Jul 8, 2023 •
edited
Loading

Site visit blocked by me wanting to complete #813 before I visit as it allows me to retire more machines on a single visit.

@Firefishy
Copy link
Member

Site visit is still planned. AFAIK the machine hasn't had any further memory errors.

@Firefishy
Copy link
Member

Firefishy commented Apr 23, 2024 •
edited by grischard
Loading

Blocked until our access to UCL Slough is restored. (#1060)

@Firefishy Firefishy removed the blocked label May 28, 2024
@Firefishy
Copy link
Member

Faulty DIMM is in P2-DIMME1

@Firefishy
Copy link
Member

DIMM replaced. It was the most outer facing DIMM.

@Firefishy Firefishy removed their assignment Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
hardware:ram-failure location:ucl The data centre sponsored by UCL
Projects
None yet
Milestone
2023 Mar UCL Visit
Development

No branches or pull requests
3 participants
@tomhughes @Firefishy @pnorman