Basic MeerKAT reduction of the DEEP2 field
Please ensure to first read Putting-together-a-basic-reduction-recipe and Available-parameters-for-a-execution-task-(cab)
The dataset and interactive CASA 4.7 tutorial can be downloaded from DEEP2 100MHz 4 hour 16 antenna tutorial. The goal of this tutorial is to translate the interactive tutorial into a Stimela recipe. We may not use the exact same parameters and steps set out in the accompanying document of the original.
In this tutorial we will create a basic reduction recipe for the a MEERKAT array release 1.5 (16 antenna) observation of the so-called DEEP2 field (Mauch et al.). This includes the core antennae and outer-most antennae, with which we can form baselines up to ~7600m. To keep the size down I already averaged the data down conservatively both in time and in frequency and selected the first 4 hours of the observation data between 1 to 1.1 GHz.
In this tutorial we will iteratively develop our reduction recipe. Therefore we will extensively make use of the steps list parameter in the Recipe.run method at the bottom of the standard recipe.
Lets start by inspecting the data. We will use the listobs task to inspect what data is available for our reduction. In its most basic form the task takes only the Measurement Set name ('vis' parameter). We begin by defining a most basic recipe. Please make the necessary adjustments to run with uDocker or Singularity if you so desire.
1 import stimela
2
3 INPUT="input"
4 MSDIR="msdir"
5 OUTPUT="output"
6
7 recipe = stimela.Recipe("MEERKAT AR1.5 DEEP2 reduction",
8 ms_dir=MSDIR,
9 JOB_TYPE="docker")
10
11 # basic data inspection
12 recipe.add("cab/casa_listobs", "get_obsinfo", {
13 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms"
14 }, input=INPUT, output=OUTPUT, label="view obs info")
15
16 STEPS = [
17 "view obs info"
18 ]
19 recipe.run(STEPS)
Running this recipe should yield output along the following lines
CASA 5.4.1-31 -- Common Astronomy Software Applications
Creating a new telemetry file
Telemetry initialized. Telemetry will send anonymized usage statistics to NRAO.
You can disable telemetry by setting environment variable with
export CASA_ENABLE_TELEMETRY=false
or by adding the following line in your ~/.casarc file:
EnableTelemetry: False
2019-08-19 08:59:47 INFO ::casa CASA Version 5.4.1-31
--> CrashReporter initialized.
2019-08-19 08:59:51 INFO listobs::::
2019-08-19 08:59:51 INFO listobs::::+ ##########################################
2019-08-19 08:59:51 INFO listobs::::+ ##### Begin Task: listobs #####
2019-08-19 08:59:51 INFO listobs:::: listobs(vis="/home/bhugo/msdir/1491291289.1ghz.1.1ghz.4hrs.ms",selectdata=True,spw="",field="",antenna="",
2019-08-19 08:59:51 INFO listobs::::+ uvrange="",timerange="",correlation="",scan="",intent="",
2019-08-19 08:59:51 INFO listobs::::+ feed="",array="",observation="",verbose=True,listfile="",
2019-08-19 08:59:51 INFO listobs::::+ listunfl=False,cachesize=50,overwrite=False)
2019-08-19 08:59:51 INFO listobs::ms::summary ================================================================================
2019-08-19 08:59:51 INFO listobs::ms::summary+ MeasurementSet Name: /home/bhugo/msdir/1491291289.1ghz.1.1ghz.4hrs.ms MS Version 2
2019-08-19 08:59:51 INFO listobs::ms::summary+ ================================================================================
2019-08-19 08:59:51 INFO listobs::ms::summary+ Observer: Tom Project: 20170404-0004
2019-08-19 08:59:51 INFO listobs::ms::summary+ Observation: MeerKAT
2019-08-19 08:59:52 INFO listobs::MSMetaData::_computeScanAndSubScanProperties Computing scan and subscan properties...
2019-08-19 08:59:52 INFO listobs::ms::summary Data records: 190920 Total elapsed time = 14070 seconds
2019-08-19 08:59:52 INFO listobs::ms::summary+ Observed from 04-Apr-2017/07:40:34.0 to 04-Apr-2017/11:35:04.0 (UTC)
2019-08-19 08:59:52 INFO listobs::ms::summary
2019-08-19 08:59:52 INFO listobs::ms::summary+ ObservationID = 0 ArrayID = 0
2019-08-19 08:59:52 INFO listobs::ms::summary+ Date Timerange (UTC) Scan FldId FieldName nRows SpwIds Average Interval(s) ScanIntent
2019-08-19 08:59:52 INFO listobs::ms::summary+ 04-Apr-2017/07:40:34.0 - 07:45:29.9 2 0 0408-65 4440 [0] [7.92] [CALIBRATE_AMPLI,CALIBRATE_BANDPASS,CALIBRATE_FLUX,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 07:45:49.9 - 07:47:45.8 3 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 07:48:01.8 - 07:52:57.7 4 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 07:53:13.7 - 07:55:09.6 5 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 07:55:25.6 - 08:00:25.5 6 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:00:41.5 - 08:02:37.4 7 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:02:53.4 - 08:07:49.3 8 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:08:05.3 - 08:10:01.2 9 1 0252-712 1800 [0] [7.72] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:10:17.2 - 08:15:13.1 10 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:15:29.1 - 08:17:29.1 11 1 0252-712 1800 [0] [7.96] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:17:45.0 - 08:22:40.9 12 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:22:56.9 - 08:24:52.9 13 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:25:08.9 - 08:30:08.7 14 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:30:24.7 - 08:32:20.7 15 1 0252-712 1800 [0] [7.69] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:32:36.7 - 08:37:32.5 16 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:37:52.5 - 08:39:48.5 17 1 0252-712 1800 [0] [7.69] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:40:04.5 - 08:45:00.4 18 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:45:20.3 - 08:47:16.3 19 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:47:32.3 - 08:52:28.2 20 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:52:48.2 - 08:54:44.1 21 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 08:55:00.1 - 09:00:00.0 22 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:00:16.0 - 09:02:11.9 23 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:02:31.9 - 09:07:27.8 24 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:07:43.8 - 09:09:43.7 25 1 0252-712 1800 [0] [7.96] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:09:59.7 - 09:14:55.6 26 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:15:15.6 - 09:17:11.5 27 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:17:31.5 - 09:22:27.4 28 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:22:47.4 - 09:24:43.3 29 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:25:03.3 - 09:29:59.2 30 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:30:19.2 - 09:32:15.2 31 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:32:35.1 - 09:37:31.0 32 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:43:38.9 - 09:48:38.7 34 0 0408-65 4560 [0] [7.82] [CALIBRATE_AMPLI,CALIBRATE_BANDPASS,CALIBRATE_FLUX,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:48:54.7 - 09:50:50.7 35 1 0252-712 1800 [0] [7.72] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:51:10.7 - 09:56:06.5 36 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:56:26.5 - 09:58:22.5 37 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 09:58:42.5 - 10:03:38.4 38 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:03:58.3 - 10:05:54.3 39 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:06:14.3 - 10:11:10.2 40 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:11:30.2 - 10:13:26.1 41 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:13:46.1 - 10:18:42.0 42 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:19:02.0 - 10:21:01.9 43 1 0252-712 1800 [0] [7.96] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:21:21.9 - 10:26:17.8 44 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:26:37.8 - 10:28:33.7 45 1 0252-712 1800 [0] [7.73] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:28:53.7 - 10:33:49.6 46 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:34:09.6 - 10:36:05.5 47 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:36:25.5 - 10:41:25.4 48 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:41:45.4 - 10:43:41.3 49 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:44:01.3 - 10:49:01.2 50 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:49:21.2 - 10:51:17.1 51 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:51:37.1 - 10:56:33.0 52 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:56:53.0 - 10:58:53.0 53 1 0252-712 1800 [0] [7.97] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10:59:12.9 - 11:04:08.8 54 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:04:28.8 - 11:06:24.8 55 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:06:44.8 - 11:11:44.6 56 2 DEEP_2 4560 [0] [7.89] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:12:04.6 - 11:14:00.6 57 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:14:20.6 - 11:19:16.4 58 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:19:36.4 - 11:21:32.4 59 1 0252-712 1800 [0] [7.7] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:21:56.4 - 11:26:52.2 60 2 DEEP_2 4440 [0] [7.99] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:27:12.2 - 11:29:08.2 61 1 0252-712 1800 [0] [7.71] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:29:28.2 - 11:34:24.1 62 2 DEEP_2 4440 [0] [8] [TARGET]
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11:34:48.0 - 11:35:04.0 63 1 0252-712 240 [0] [7.95] [CALIBRATE_AMPLI,CALIBRATE_PHASE]
2019-08-19 08:59:52 INFO listobs::ms::summary (nRows = Total number of rows per scan)
2019-08-19 08:59:52 INFO listobs::ms::summary Fields: 3
2019-08-19 08:59:52 INFO listobs::ms::summary+ ID Code Name RA Decl Epoch SrcId nRows
2019-08-19 08:59:52 INFO listobs::ms::summary+ 0 T 0408-65 04:08:20.380000 -65.45.09.10000 J2000 0 9000
2019-08-19 08:59:52 INFO listobs::ms::summary+ 1 T 0252-712 02:52:46.150000 -71.04.35.30000 J2000 1 52440
2019-08-19 08:59:52 INFO listobs::ms::summary+ 2 T DEEP_2 04:13:26.400000 -80.00.00.00000 J2000 2 129480
2019-08-19 08:59:52 INFO listobs::ms::summary Spectral Windows: (1 unique spectral windows and 1 unique polarization setups)
2019-08-19 08:59:52 INFO listobs::ms::summary+ SpwID Name #Chans Frame Ch0(MHz) ChanWid(kHz) TotBW(kHz) CtrFreq(MHz) Corrs
2019-08-19 08:59:52 INFO listobs::ms::summary+ 0 none 119 TOPO 1000.513 835.938 99476.6 1049.8330 XX YY
2019-08-19 08:59:52 INFO listobs::ms::summary Sources: 3
2019-08-19 08:59:52 INFO listobs::ms::summary+ ID Name SpwId
2019-08-19 08:59:52 INFO listobs::ms::summary+ 0 0408-65 0
2019-08-19 08:59:52 INFO listobs::ms::summary+ 1 0252-712 0
2019-08-19 08:59:52 INFO listobs::ms::summary+ 2 DEEP_2 0
2019-08-19 08:59:52 INFO listobs::ms::summary+ NB: No rest frequency information found in SOURCE table.
2019-08-19 08:59:52 INFO listobs::ms::summary+ NB: No systemic velocity information found in SOURCE table.
2019-08-19 08:59:52 INFO listobs::ms::summary Antennas: 16:
2019-08-19 08:59:52 INFO listobs::ms::summary+ ID Name Station Diam. Long. Lat. Offset from array center (m) ITRF Geocentric coordinates (m)
2019-08-19 08:59:52 INFO listobs::ms::summary+ East North Elevation x y z
2019-08-19 08:59:52 INFO listobs::ms::summary+ 0 m000 m000 13.5 m +021.26.37.7 -30.32.39.3 -9.3315 30.5123 -13.7488 5109224.244081 2006790.329682 -3239100.601853
2019-08-19 08:59:52 INFO listobs::ms::summary+ 1 m002 m002 13.5 m +021.26.36.8 -30.32.39.8 -33.1815 13.5672 -13.7541 5109224.943711 2006764.980703 -3239115.192851
2019-08-19 08:59:52 INFO listobs::ms::summary+ 2 m005 m005 13.5 m +021.26.34.2 -30.32.41.7 -103.1671 -45.3159 -13.6884 5109222.728014 2006688.920477 -3239165.938258
2019-08-19 08:59:52 INFO listobs::ms::summary+ 3 m011 m011 13.5 m +021.26.41.2 -30.32.43.9 82.9436 -114.2856 -12.6212 5109122.918981 2006849.667876 -3239225.879103
2019-08-19 08:59:52 INFO listobs::ms::summary+ 4 m012 m012 13.5 m +021.26.43.3 -30.32.44.5 138.9495 -130.4748 -12.3820 5109094.976657 2006898.862921 -3239239.943208
2019-08-19 08:59:52 INFO listobs::ms::summary+ 5 m018 m018 13.5 m +021.26.42.0 -30.32.40.5 104.6648 -8.0697 -12.9552 5109164.953297 2006889.515614 -3239134.233344
2019-08-19 08:59:52 INFO listobs::ms::summary+ 6 m020 m020 13.5 m +021.26.41.6 -30.32.42.2 95.9433 -61.8458 -12.7533 5109142.866214 2006871.469582 -3239180.649435
2019-08-19 08:59:52 INFO listobs::ms::summary+ 7 m042 m042 13.5 m +021.26.24.4 -30.32.47.4 -362.7948 -222.5028 -14.5892 5109233.093012 2006414.063100 -3239318.077690
2019-08-19 08:59:52 INFO listobs::ms::summary+ 8 m049 m049 13.5 m +021.24.22.5 -30.32.18.4 -3606.7979 674.9449 -16.6018 5110840.935829 2003560.083385 -3238544.127600
2019-08-19 08:59:52 INFO listobs::ms::summary+ 9 m057 m057 13.5 m +021.26.49.1 -30.30.47.0 293.5452 3497.7956 -21.9060 5110746.279858 2007713.657982 -3236109.834690
2019-08-19 08:59:52 INFO listobs::ms::summary+ 10 m058 m058 13.5 m +021.28.23.4 -30.31.05.6 2803.9945 2925.0388 -16.3818 5109561.316727 2009946.092403 -3236606.062809
2019-08-19 08:59:52 INFO listobs::ms::summary+ 11 m059 m059 13.5 m +021.28.56.5 -30.32.08.0 3685.0434 997.2513 -6.7247 5108335.204937 2010410.628369 -3238271.543744
2019-08-19 08:59:52 INFO listobs::ms::summary+ 12 m060 m060 13.5 m +021.28.46.5 -30.33.32.1 3419.1035 -1602.2355 -2.4315 5107206.614034 2009680.714437 -3240512.429428
2019-08-19 08:59:52 INFO listobs::ms::summary+ 13 m061 m061 13.5 m +021.26.37.4 -30.33.47.8 -17.5027 -2086.0199 -6.9077 5108231.263887 2006391.531815 -3240926.733858
2019-08-19 08:59:52 INFO listobs::ms::summary+ 14 m062 m062 13.5 m +021.25.43.9 -30.33.53.6 -1442.0284 -2265.8985 -6.7546 5108666.756192 2005032.436433 -3241081.698019
2019-08-19 08:59:52 INFO listobs::ms::summary+ 15 m063 m063 13.5 m +021.24.29.5 -30.33.32.1 -3421.1962 -1602.0938 -9.4852 5109701.398386 2003312.684080 -3240508.721138
2019-08-19 08:59:52 INFO listobs:::: ##### End Task: listobs #####
2019-08-19 08:59:52 INFO listobs::::+ ##########################################
2019-08-19 10:59:55,190 - STIMELA - INFO - Container [get_obsinfo-140643909791904156620517608] has executed successfully
INFO:STIMELA:Container [get_obsinfo-140643909791904156620517608] has executed successfully
2019-08-19 10:59:55,191 - STIMELA - INFO - Runtime was 0:00:17.078296.
INFO:STIMELA:Runtime was 0:00:17.078296.
2019-08-19 10:59:55,223 - STIMELA - INFO - Container get_obsinfo-140643909791904156620517608 has been stopped.
INFO:STIMELA:Container get_obsinfo-140643909791904156620517608 has been stopped.
Running: docker rm get_obsinfo-140643909791904156620517608
get_obsinfo-140643909791904156620517608
2019-08-19 10:59:56,257 - STIMELA - INFO - Saving pipeline information in .last_meerkat_ar1.5_deep2_reduction.json
INFO:STIMELA:Saving pipeline information in .last_meerkat_ar1.5_deep2_reduction.json
2019-08-19 10:59:56,258 - STIMELA - INFO - Recipe executed successfully
INFO:STIMELA:Recipe executed successfully
Take a moment to peruse this a bit. You will see that there are 3 fields in this observation: 0408-65, 0252-712 and a custom field labelled “DEEP_2”. The logger reports their positions and the scans in which the telescope was tracking them. Note that 0408-65 was marked as a bandpass (primary) calibrator by the observer, 0252-712 was marked as a amplitude and phase (secondary) calibrator and DEEP_2 is the target field. We will get back to how to use these in the calibration section. The logger also displays which antennae, along with the bandwidth of the observation.
To get a better understanding of the layout of the array and the associated uv tracks the 120 possible baselines (excluding autocorrelations) traced out in the measurement coordinate frame during synthesis we take a look at the two tasks: “plotants” and “plotuv”. Keep the “vis” parameter as is but set the “field” parameter to “DEEP_2”. Notice that we comment out the listobs steps which already ran. We will continue to do this throughout this tutorial.
...
15 recipe.add("cab/casa_plotants", "plot_ants", {
16 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
17 "figfile": "deep2_antennas.png"
18 }, input=INPUT, output=OUTPUT, label="plot subarray ants")
19 recipe.add("cab/casa47_plotuv", "plot_uv", {
20 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
21 "field": "DEEP_2",
22 "figfile": "deep2_uvcoverage.png"
23 }, input=INPUT, output=OUTPUT, label="plot subarray uv coverage")
24
25 STEPS = [
26 # "view obs info",
27 "plot subarray ants",
28 "plot subarray uv coverage"
29 ]
Working with the measurement set format typically involves in place calibration and flagging procedures. Hence the FLAG column is likely the easiest column to mess up in an interactive run. To ensure that we can always go back to raw data we will create a flag set called "legacy". The backup step needs to be run once and then commented out of the steps in subsequent runs. You can always restore the flags to their pristine version by running restore original flags.
...
24 recipe.add("cab/casa_flagmanager", "backup_original_flags", {
25 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
26 "mode": "save",
27 "versionname": "legacy",
28 }, input=INPUT, output=OUTPUT, label="backup original flags")
29 recipe.add("cab/casa_flagmanager", "restore_original_flags", {
30 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
31 "mode": "restore",
32 "versionname": "legacy",
33 }, input=INPUT, output=OUTPUT, label="restore original flags")
34
35 STEPS = [
36 # "view obs info",
37 # "plot subarray ants",
38 # "plot subarray uv coverage",
39 "backup original flags",
40 "restore original flags",
41 ]
You should be met with a log along the lines of the following:
2019-08-19 10:35:27 INFO ::casa CASA Version 5.4.1-31
--> CrashReporter initialized.
2019-08-19 10:35:28 INFO flagmanager::::
2019-08-19 10:35:28 INFO flagmanager::::+ ##########################################
2019-08-19 10:35:28 INFO flagmanager::::+ ##### Begin Task: flagmanager #####
2019-08-19 10:35:28 INFO flagmanager:::: flagmanager(vis="/home/bhugo/msdir/1491291289.1ghz.1.1ghz.4hrs.ms",mode="restore",versionname="legacy",oldname="",comment="",
2019-08-19 10:35:28 INFO flagmanager::::+ merge="replace")
2019-08-19 10:35:28 INFO flagmanager::AgentFlagger::open Table type is Measurement Set
2019-08-19 10:35:28 INFO flagmanager:::: Restore flagversions legacy
2019-08-19 10:35:28 INFO flagmanager:::: ##### End Task: flagmanager #####
2019-08-19 10:35:28 INFO flagmanager::::+ ##########################################
2019-08-19 12:35:32,128 - STIMELA - INFO - Container [restore_original_flags-140086412236528156621090591] has executed successfully
INFO:STIMELA:Container [restore_original_flags-140086412236528156621090591] has executed successfully
2019-08-19 12:35:32,129 - STIMELA - INFO - Runtime was 0:00:07.063757.
INFO:STIMELA:Runtime was 0:00:07.063757.
2019-08-19 12:35:32,162 - STIMELA - INFO - Container restore_original_flags-140086412236528156621090591 has been stopped.
INFO:STIMELA:Container restore_original_flags-140086412236528156621090591 has been stopped.
Running: docker rm restore_original_flags-140086412236528156621090591
restore_original_flags-140086412236528156621090591
2019-08-19 12:35:33,195 - STIMELA - INFO - Saving pipeline information in .last_meerkat_ar1.5_deep2_reduction.json
INFO:STIMELA:Saving pipeline information in .last_meerkat_ar1.5_deep2_reduction.json
2019-08-19 12:35:33,196 - STIMELA - INFO - Recipe executed successfully
INFO:STIMELA:Recipe executed successfully
The auto correlation products are not needed for either basic calibration or imaging so lets go ahead and flag them out. Set autocorr to “True” and set field to “” as follows:
...
34 recipe.add("cab/casa_flagdata", "flag_autos", {
35 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
36 "mode": "manual",
37 "autocorr": True,
38 }, input=INPUT, output=OUTPUT, label="flag autocorrelation products")
39
40 STEPS = [
41 # "view obs info",
42 # "plot subarray ants",
43 # "plot subarray uv coverage",
44 #####"backup original flags",
45 #"restore original flags",
46 "flag autocorrelation products",
47 ]
The log should look something like the following:
2019-08-19 10:40:57 INFO FlagMSHandler:: Chunk = 57 [progress: 94%], Observation = 0, Array = 0, Scan = 59, Field = 1 (0252-712), Spw = 0, Channels = 119, Corrs = [ XX YY ], Total
2019-08-19 10:40:57 INFO FlagMSHandler:: ------------------------------------------------------------------------------------
2019-08-19 10:40:57 INFO FlagMSHandler:: Chunk = 58 [progress: 96%], Observation = 0, Array = 0, Scan = 60, Field = 2 (DEEP_2), Spw = 0, Channels = 119, Corrs = [ XX YY ], Total R
2019-08-19 10:40:57 INFO FlagMSHandler:: ------------------------------------------------------------------------------------
2019-08-19 10:40:57 INFO FlagMSHandler:: Chunk = 59 [progress: 97%], Observation = 0, Array = 0, Scan = 61, Field = 1 (0252-712), Spw = 0, Channels = 119, Corrs = [ XX YY ], Total
2019-08-19 10:40:57 INFO FlagMSHandler:: ------------------------------------------------------------------------------------
2019-08-19 10:40:57 INFO FlagMSHandler:: Chunk = 60 [progress: 99%], Observation = 0, Array = 0, Scan = 62, Field = 2 (DEEP_2), Spw = 0, Channels = 119, Corrs = [ XX YY ], Total R
2019-08-19 10:40:57 INFO FlagMSHandler:: ------------------------------------------------------------------------------------
2019-08-19 10:40:57 INFO FlagMSHandler:: Chunk = 61 [progress: 100%], Observation = 0, Array = 0, Scan = 63, Field = 1 (0252-712), Spw = 0, Channels = 119, Corrs = [ XX YY ], Tota
2019-08-19 10:40:57 INFO FlagMSHandler::nextChunk ====================================================================================
2019-08-19 10:40:57 INFO AgentFlagger::run => Writing flags to the MS
2019-08-19 10:40:57 INFO flagdata:::: CASA Version 5.4.1-31
2019-08-19 10:40:57 INFO flagdata::::
2019-08-19 10:41:02 INFO flagdata:::: ##### End Task: flagdata #####
2019-08-19 10:41:02 INFO flagdata::::+ ##########################################
This dataset is quite small so lets eyeball the data using the visualization task “plotms” to see if we can spot problems. Lets plot the raw amplitudes of the visibilities as a function of frequency and colourise by field to get a first glance of any issues:
...
39 recipe.add("cab/casa_plotms", "plot_ampfreq_prior", {
40 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
41 "plotfile": "amp_freq_prior_rf_flag",
42 "exprange": "all",
43 "expformat": "png",
44 "showgui": False,
45 "xaxis": "frequency",
46 "yaxis": "amplitude",
47 "ydatacolumn": "data",
48 "selectdata": True,
49 "correlation": "XX,YY",
50 "iteraxis": "corr",
51 "coloraxis": "field",
52 "showmajorgrid": True,
53 "showminorgrid": False,
54 "overwrite": True,
55 }, input=INPUT, output=OUTPUT, label="plot raw ampfreq before flagging")
56
57 STEPS = [
58 # "view obs info",
59 # "plot subarray ants",
60 # "plot subarray uv coverage",
61 #####"backup original flags",
62 # "restore original flags",
63 # "flag autocorrelation products",
64 "plot raw ampfreq before flagging"
65 ]
The primary calibrator is the one in black, the secondary calibrator is the one in magenta and the target field is in orange – it may be different on your viewer, but the amplitudes should make it easy to identify them.
There is clearly some problems here. The spikes around 1.09 GHz are clearly outliers which we should remove before attempting calibration and imaging. We can investigate this a bit further by swapping plotting amplitude vs. time and uv-dist. We expect the the latter to be flat across all uv distances for a calibrator source (not necessarily for a target field). Take a moment to contemplate why this is the case. We see that the frequency selection contains RFI that is particularly visible on the shortest baselines:
...
56 recipe.add("cab/casa_plotms", "plot_ampuvdist_prior", {
57 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
58 "plotfile": "amp_uvdist_prior_rf_flag",
59 "exprange": "all",
60 "expformat": "png",
61 "showgui": False,
62 "xaxis": "uvdist",
63 "yaxis": "amplitude",
64 "ydatacolumn": "data",
65 "selectdata": True,
66 "correlation": "XX,YY",
67 "iteraxis": "corr",
68 "coloraxis": "field",
69 "showmajorgrid": True,
70 "showminorgrid": False,
71 "overwrite": True,
72 }, input=INPUT, output=OUTPUT, label="plot raw ampuvdist before flagging")
73
74 STEPS = [
75 # "view obs info",
76 # "plot subarray ants",
77 # "plot subarray uv coverage",
78 #####"backup original flags",
79 # "restore original flags",
80 # "flag autocorrelation products",
81 # "plot raw ampfreq before flagging",
82 "plot raw ampuvdist before flagging",
83 ]
This is normal for a RF source out in the sidelobes of the instrument. Just as with any other radio source, the interfering source is subject to the fringe rate of the observing interferometer; a long baseline interferometer
washes out most of the interference, whereas the interference remains largely visible on short spacings. We
will use the SARAO autoflagger, Tricolour to flag out this RF interference. We will flag the calibrators based on their total quadrature power because we know apriori that the field is dominated by a source at phase centre. The target will be flagged based on total polarized power since we do not know if there are strong on-axis sources that may appear to resemble RFI fringes. As this bandwidth is already reasonably clean by any measure let us set higher sigma thresholds for the autoflagger. Copy the following to a stragegy file called high_tol.yaml in your pipeline input folder:
# List of strategies to apply in order
strategies:
# only enable me if you really want to start from scratch
# -
# name: reset_flags:
# task: unflag
-
name: nan_dropouts_flag
task: flag_nans_zeros
-
name: background_static_mask
task: apply_static_mask
kwargs:
accumulation_mode: "or"
uvrange: ""
-
name: background_flags
task: sum_threshold
kwargs:
outlier_nsigma: 15
windows_time: [1, 2, 4, 8]
windows_freq: [1, 2, 4, 8]
background_reject: 2.0
background_iterations: 5
spike_width_time: 12.5
spike_width_freq: 10.0
time_extend: 3
freq_extend: 3
freq_chunks: 10
average_freq: 1
flag_all_time_frac: 0.6
flag_all_freq_frac: 0.8
rho: 1.3
num_major_iterations: 4
-
name: residual_flag_initial
task: uvcontsub_flagger
kwargs:
major_cycles: 10
or_original_from_cycle: 1
taylor_degrees: 25
sigma: 15.0
# flags are discarded at this point since we or from cycle 1
# reflag nans and zeros
-
name: nan_dropouts_reflag
task: flag_nans_zeros
-
name: flag_autos
task: flag_autos
-
name: combine_with_input_flags
task: combine_with_input_flags
Then add the following flagging and plotting tasks to the recipe to get an idea of the posterior RF properties of the three fields:
...
73 recipe.add('cab/tricolour', "autoflag_rfi",
74 {
75 "ms" : "1491291289.1ghz.1.1ghz.4hrs.ms",
76 "data-column" : "DATA",
77 "window-backend" : 'numpy',
78 "flagging-strategy" : "total_power",
79 "field-names" : "0408-65,0252-712",
80 "config" : "high_tol.yaml"
81 },
82 input=INPUT, output=OUTPUT, label="autoflag rfi calibrators")
83 recipe.add('cab/tricolour', "autoflag_rfi",
84 {
85 "ms" : "1491291289.1ghz.1.1ghz.4hrs.ms",
86 "data-column" : "DATA",
87 "window-backend" : 'numpy',
88 "flagging-strategy" : "polarisation",
89 "field-names" : "DEEP_2"
90 },
91 input=INPUT, output=OUTPUT, label="autoflag rfi target")
92 recipe.add("cab/casa_plotms", "plot_ampfreq_post", {
93 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
94 "plotfile": "amp_freq_post_rf_flag",
95 "exprange": "all",
96 "expformat": "png",
97 "showgui": False,
98 "xaxis": "frequency",
99 "yaxis": "amplitude",
100 "ydatacolumn": "data",
101 "selectdata": True,
102 "correlation": "XX,YY",
103 "iteraxis": "corr",
104 "coloraxis": "field",
105 "showmajorgrid": True,
106 "showminorgrid": False,
107 "overwrite": True,
108 }, input=INPUT, output=OUTPUT, label="plot raw ampfreq after flagging")
109 recipe.add("cab/casa_plotms", "plot_ampuvdist_post", {
110 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
111 "plotfile": "amp_uvdist_post_rf_flag",
112 "exprange": "all",
113 "expformat": "png",
114 "showgui": False,
115 "xaxis": "uvdist",
116 "yaxis": "amplitude",
117 "ydatacolumn": "data",
118 "selectdata": True,
119 "correlation": "XX,YY",
120 "iteraxis": "corr",
121 "coloraxis": "field",
122 "showmajorgrid": True,
123 "showminorgrid": False,
124 "overwrite": True,
125 }, input=INPUT, output=OUTPUT, label="plot raw ampuvdist after flagging")
126
127
128 STEPS = [
129 # "view obs info",
130 # "plot subarray ants",
131 # "plot subarray uv coverage",
132 #####"backup original flags",
133 # "restore original flags",
134 # "flag autocorrelation products",
135 # "plot raw ampfreq before flagging",
136 # "plot raw ampuvdist before flagging",
137 "autoflag rfi calibrators",
138 "autoflag rfi target",
139 "plot raw ampfreq after flagging",
140 "plot raw ampuvdist after flagging",
141 ]
This is not perfect, but a significant portion of interference on the secondary calibrator has been flagged and there should be enough data remaining to calibrate this dataset. Next we will move on to transfer calibration of our dataset.
Generally speaking, when we do observations for the purposes of imaging we observe two calibrators: a very bright (preferably unresolved) primary calibrator used to correct the frequency response of the interferometer and a secondary (or “gain”) calibrator to very roughly track atmospheric changes with time. This is called first generation calibration. It is important to realise that an interferometer cannot measure the average sky temperature since the autocorrelations should be excluded from imaging. Take a moment to convince yourself why?
The bright primary calibrator thus serves a dual purpose of serving as a reference for the desired frequency response of the system, as well as amplitude of the frequency response. During the observation the primary calibrator is observed only a few times, while the (preferably closer) secondary calibrator is observed throughout the observation. The telescope will thus, typically alternate between the target field and the secondary calibrator. One can see this from visualising the observation:
The star on the plot to the left is the south celestial pole. The tracks indicate the starting and ending positions of the 3 sources, including the sun as reference for the duration of the observation on the day in question. The plot on the bottom shows that the observation starts with a scan over the bright primary calibrator (0408-65) and then alternates between the secondary calibrator (0252-712) and the target (DEEP_2).
For simplicity lets consider that the measurements taken by the telescope only vary with time and frequency, that these variations are separable in time and frequency, and that the variations are largely independent of where the telescope is pointing. The two calibrators are located right at the phase-tracking and pointing centre of the telescope. This is done so that we can assume that the phase response of the telescope is centred around 0 and the amplitude is centred around some large value, which is well known from the literature. The calibrator may have a small amplitude slope in frequency, but over small observation bands this is not too great a concern. If we therefore plot amplitude vs. phase of the two calibrators we expect, roughly, a ball-like structure. This is not what we see in the raw data.
The following steps in the recipe produced these plots (apart from the azimuth elevation plot):
...
126 recipe.add("cab/casa_plotms", "plot_ampphase_prior", {
127 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
128 "plotfile": "amp_phase_post_rf_flag",
129 "exprange": "all",
130 "expformat": "png",
131 "showgui": False,
132 "xaxis": "phase",
133 "yaxis": "amplitude",
134 "ydatacolumn": "data",
135 "selectdata": True,
136 "field": "0408-65,0252-712",
137 "correlation": "XX,YY",
138 "iteraxis": "field",
139 "coloraxis": "scan",
140 "showmajorgrid": True,
141 "showminorgrid": False,
142 "overwrite": True,
143 }, input=INPUT, output=OUTPUT, label="plot raw ampphase after flagging")
144 recipe.add("cab/casa_plotms", "plot_amptime_post", {
145 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
146 "plotfile": "amp_time_post_rf_flag",
147 "expformat": "png",
148 "showgui": False,
149 "xaxis": "time",
150 "yaxis": "amplitude",
151 "ydatacolumn": "data",
152 "selectdata": True,
153 "correlation": "XX,YY",
154 "coloraxis": "field",
155 "showmajorgrid": True,
156 "showminorgrid": False,
157 "overwrite": True,
158 }, input=INPUT, output=OUTPUT, label="plot raw amptime after flagging")
159
160
161 STEPS = [
162 # "view obs info",
163 # "plot subarray ants",
164 # "plot subarray uv coverage",
165 #####"backup original flags",
166 # "restore original flags",
167 # "flag autocorrelation products",
168 # "plot raw ampfreq before flagging",
169 # "plot raw ampuvdist before flagging",
170 # "autoflag rfi calibrators",
171 # "autoflag rfi target",
172 # "plot raw ampfreq after flagging",
173 # "plot raw ampuvdist after flagging",
174 "plot raw ampphase after flagging",
175 "plot raw amptime after flagging",
176 ]
With this dataset the correlator engineers did a fantastic job setting delays by hand. That said there are residual delays (frequency slopes) that we will solve for before doing a bandpass and gain calibration. To do this we use the “gaincal” task in “K-Jones” mode. Throughout we will be using tasks from CASA 4.7, because of a bug in CASA 5. We will start by clearing calibration and adding a model data column, initialized to unity. This is generally a good enough first order assumption for delay (and phase) calibration. However, if you do not know your gain calibrator field that well you should thoroughly inspect any delay and phase solutions before applying them.
...
159 recipe.add("cab/casa_clearcal", "clear_calibration",{
160 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
161 "addmodel": True,
162 }, input=INPUT, output=OUTPUT, label="clear calibration")
163 recipe.add("cab/casa47_gaincal", "calibrate_delays", {
164 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
165 "caltable": '1491291289.K0',
166 "field": '0408-65,0252-712',
167 "solint": "32s",
168 "refant": 'm002',
169 "gaintype": 'K'
170 }, input=INPUT, output=OUTPUT, label="calibrate delays")
171 recipe.add("cab/msutils", "plot_delays", {
172 "command": "plot_gains",
173 "ctable": "1491291289.K0:output",
174 "tabtype": "delay",
175 "plot_file": "delays.png",
176 "subplot_scale": 4,
177 "plot_dpi": 180,
178 }, input=INPUT, output=OUTPUT, label="plot delays")
179
180
181 STEPS = [
182 # "view obs info",
183 # "plot subarray ants",
184 # "plot subarray uv coverage",
185 #####"backup original flags",
186 # "restore original flags",
187 # "flag autocorrelation products",
188 # "plot raw ampfreq before flagging",
189 # "plot raw ampuvdist before flagging",
190 # "autoflag rfi calibrators",
191 # "autoflag rfi target",
192 # "plot raw ampfreq after flagging",
193 # "plot raw ampuvdist after flagging",
194 # "plot raw ampphase after flagging",
195 # "plot raw amptime after flagging",
196 "clear calibration",
197 "calibrate delays",
198 "plot delays",
199 ]
This looks very good. Delays should not be more than a few tens of nanoseconds over the entire observation. With the solutions in hand lets continue on to bandpass calibration.
A radio telescope, whether it be a large stand-alone antenna or an array of antennae, can be thought of as a thermometer without any scales. We always need a trustworthy standard candle to calibrate the flux scale of the measuring system. For this exercise we will assume that the bandpass calibrator 0408-65 can be described by a log polynomial expansion around a frequency of 1.28GHz. The flux at this frequency is ~17.109 Jy and the 4 coefficients for the expansion are given as a=-1.1108895185342547, b=-0.31867594399324456, c=0.08610034561730988, d=2.602474680067308e-06. This source is not in the standard catalogue of CASA calibrators, so it has to be specified manually as follows. Please be aware that we use this model for demonstration purposes only. It may vary over time and may only be accurate to double digit percentages. Consult facilities like the CSIRO ATCA / NRAO long-term calibrator observation databases before reducing your data. Furthermore you must check whether your model is unpolarized prior to calibration. With a linear feed system a very accurate model of the fractional polarization is typically needed if a source is to be used as a flux calibrator.
...
179 recipe.add("cab/casa_setjy", "set_flux_model", {
180 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
181 "field": "0408-65",
182 "scalebychan": True,
183 "usescratch": True,
184 "standard": "manual",
185 "fluxdensity": [17.109, 0, 0, 0],
186 "reffreq": '1.28GHz',
187 "spix": [-1.1108895185342547, -0.31867594399324456,
188 0.08610034561730988, 2.602474680067308e-06],
189 }, input=INPUT, output=OUTPUT, label="set flux model")
190
191
192 STEPS = [
193 # "view obs info",
194 # "plot subarray ants",
195 # "plot subarray uv coverage",
196 #####"backup original flags",
197 # "restore original flags",
198 # "flag autocorrelation products",
199 # "plot raw ampfreq before flagging",
200 # "plot raw ampuvdist before flagging",
201 # "autoflag rfi calibrators",
202 # "autoflag rfi target",
203 # "plot raw ampfreq after flagging",
204 # "plot raw ampuvdist after flagging",
205 # "plot raw ampphase after flagging",
206 # "plot raw amptime after flagging",
207 # "clear calibration",
208 # "calibrate delays",
209 # "plot delays",
210 "set flux model"
211 ]
The logger should tell you it has successfully predicted the model visibilities and stored it in the header of the Measurement Set. Now that CASA knows what the model for the source looks like we can calibrate the bandpass. This is done using the task “bandpass”. When solving for the frequency-dependent gains CASA averages data along the time axis and solves for a solution per channel. Understandably, we need a very bright source to be able to find solutions per channel, so this is why measurements are averaged along the time axis. We apply the previous delay solutions when solving for the frequency-dependant solutions:
...
190 recipe.add("cab/casa47_bandpass", "calibrate_bandpass", {
191 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
192 "caltable": '1491291289.B0',
193 "field": '0408-65',
194 "solint": "inf",
195 "combine": "scan",
196 "refant": 'm002',
197 "gaintable": ["1491291289.K0:output"],
198 "interp": ["linear"]
199 }, input=INPUT, output=OUTPUT, label="calibrate bandpass")
200 recipe.add("cab/msutils", "plot_bandpass", {
201 "command": "plot_gains",
202 "ctable": "1491291289.B0:output",
203 "tabtype": "bandpass",
204 "plot_file": "bandpass.png",
205 "subplot_scale": 4,
206 "plot_dpi": 180,
207 }, input=INPUT, output=OUTPUT, label="plot bandpass")
208
209
210
211 STEPS = [
212 # "view obs info",
213 # "plot subarray ants",
214 # "plot subarray uv coverage",
215 #####"backup original flags",
216 # "restore original flags",
217 # "flag autocorrelation products",
218 # "plot raw ampfreq before flagging",
219 # "plot raw ampuvdist before flagging",
220 # "autoflag rfi calibrators",
221 # "autoflag rfi target",
222 # "plot raw ampfreq after flagging",
223 # "plot raw ampuvdist after flagging",
224 # "plot raw ampphase after flagging",
225 # "plot raw amptime after flagging",
226 # "clear calibration",
227 # "calibrate delays",
228 # "plot delays",
229 # "set flux model",
230 "calibrate bandpass",
231 "plot bandpass",
232 ]
The bandpass gains may not necessarily be close to unity, but they should be bundled together. If the solutions are not smooth or is significantly higher or lower than the rest it could indicate a problem with the antenna or systematics, in which case it may be necessary to flag all baselines associated with the antenna. The gains around 1.09 GHz still look a bit dodgy and may come back to haunt us later on.
With solutions to correct the frequency response of the interferometer in hand we now solve for time-variable gains using the calibrator closer to the target field. This will give us a first order approximation to systemic and atmospheric variation with time. Since we are aiming to make a map averaging all 4 hours together it is advantageous to remove time-dependant effects. We again use the “gaincal” task, only this time set to “G-Jones” mode. In contrast to solving for the bandpass, CASA now "averages" all channels together, as well as averaging in time. We ask it to compute solutions every 24 seconds and to stop at the edge of each scan. This is necessary because the amplitudes may not be the same between scans of the calibrator due to systemic or atmospheric conditions
...
208 recipe.add("cab/casa47_gaincal", "calibrate_gains", {
209 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
210 "caltable": '1491291289.G0',
211 "field": '0408-65,0252-712',
212 "solint": "24s",
213 "refant": 'm002',
214 "gaintype": 'G',
215 "calmode": 'ap',
216 "gaintable": ["1491291289.K0:output", "1491291289.B0:output"],
217 "interp": ["linear", "linear"]
218 }, input=INPUT, output=OUTPUT, label="calibrate gains")
219 recipe.add("cab/casa_fluxscale", "fluxscale_gains", {
220 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
221 "caltable": '1491291289.G0:output',
222 "fluxtable": "1491291289.F0",
223 "reference": ['0408-65'],
224 "transfer": ['0252-712'],
225 }, input=INPUT, output=OUTPUT, label="fluxscale gains")
226
227 recipe.add("cab/msutils", "plot_gains", {
228 "command": "plot_gains",
229 "ctable": "1491291289.F0:output",
230 "tabtype": "gain",
231 "plot_file": "gains.png",
232 "subplot_scale": 4,
233 "plot_dpi": 180,
234 }, input=INPUT, output=OUTPUT, label="plot gains")
235
236
237 STEPS = [
238 # "view obs info",
239 # "plot subarray ants",
240 # "plot subarray uv coverage",
241 #####"backup original flags",
242 # "restore original flags",
243 # "flag autocorrelation products",
244 # "plot raw ampfreq before flagging",
245 # "plot raw ampuvdist before flagging",
246 # "autoflag rfi calibrators",
247 # "autoflag rfi target",
248 # "plot raw ampfreq after flagging",
249 # "plot raw ampuvdist after flagging",
250 # "plot raw ampphase after flagging",
251 # "plot raw amptime after flagging",
252 # "clear calibration",
253 # "calibrate delays",
254 # "plot delays",
255 # "set flux model",
256 # "calibrate bandpass",
257 # "plot bandpass",
258 "calibrate gains",
259 "fluxscale gains",
260 "plot gains"
261 ]
The last thing is to scale the solutions. Since we're still using a model for the bandpass calibrator to calibrate the secondary it is necessary to correct for the difference in amplitude of the bandpass and secondary calibrator. This assumes that the sources have roughly the same spectral slopes. To achieve this we run the task fluxscale and output a new table.
The gains appear to be relatively smooth. Things to watch out for are significant outliers (+/- 0.5 in amplitude relative to the other gains) or steep overall slopes in the solutions. The latter could indicate antenna pointing problems for instance. These are all acceptable:
Thus far our calibration has worked out as planned - the data is flagged to a reasonable extent and all of the calibration solutions are relatively smooth in time and frequency. Before cracking the bubbly and calling it a day we should check the quality of our calibration by inspecting the corrected data. Go ahead and apply the calibration solutions and by doing so, transfer the solutions onto the target field:
...
236 recipe.add("cab/casa47_applycal", "apply_calibration", {
237 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
238 "field": "0252-712,0408-65,DEEP_2",
239 "gaintable": ['1491291289.K0:output', '1491291289.B0:output',
240 '1491291289.F0:output'],
241 "interp": ['linear', 'nearest', 'linear'],
242 }, input=INPUT, output=OUTPUT, label="apply calibration")
243 recipe.add('cab/tricolour', "autoflag_rfi",
244 {
245 "ms" : "1491291289.1ghz.1.1ghz.4hrs.ms",
246 "data-column" : "CORRECTED_DATA",
247 "window-backend" : 'numpy',
248 "flagging-strategy" : "total_power",
249 "field-names" : "0408-65,0252-712",
250 "config" : "high_tol.yaml"
251 },
252 input=INPUT, output=OUTPUT, label="autoflag rfi calibrators round 2")
253 recipe.add('cab/tricolour', "autoflag_rfi",
254 {
255 "ms" : "1491291289.1ghz.1.1ghz.4hrs.ms",
256 "data-column" : "CORRECTED_DATA",
257 "window-backend" : 'numpy',
258 "flagging-strategy" : "polarisation",
259 "field-names" : "DEEP_2"
260 },
261 input=INPUT, output=OUTPUT, label="autoflag rfi target round 2")
262
263 recipe.add("cab/casa_plotms", "plot_ampphase_post", {
264 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
265 "plotfile": "amp_phase_post_cal",
266 "exprange": "all",
267 "expformat": "png",
268 "showgui": False,
269 "xaxis": "phase",
270 "yaxis": "amplitude",
271 "xdatacolumn": "corrected",
272 "ydatacolumn": "corrected",
273 "selectdata": True,
274 "field": "0408-65,0252-712",
275 "correlation": "XX,YY",
276 "iteraxis": "field",
277 "coloraxis": "scan",
278 "showmajorgrid": True,
279 "showminorgrid": False,
280 "overwrite": True,
281 }, input=INPUT, output=OUTPUT, label="plot corrected ampphase after calibration")
282 recipe.add("cab/casa_plotms", "plot_reim_prior", {
283 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
284 "plotfile": "re_im_pre_cal",
285 "exprange": "all",
286 "expformat": "png",
287 "showgui": False,
288 "xaxis": "real",
289 "yaxis": "imaginary",
290 "xdatacolumn": "data",
291 "ydatacolumn": "data",
292 "selectdata": True,
293 "field": "0408-65,0252-712",
294 "correlation": "XX,YY",
295 "iteraxis": "field",
296 "coloraxis": "scan",
297 "showmajorgrid": True,
298 "showminorgrid": False,
299 "overwrite": True,
300 }, input=INPUT, output=OUTPUT, label="plot corrected re/im before calibration")
301
302 recipe.add("cab/casa_plotms", "plot_reim_post", {
303 "vis": "1491291289.1ghz.1.1ghz.4hrs.ms",
304 "plotfile": "re_im_post_cal",
305 "exprange": "all",
306 "expformat": "png",
307 "showgui": False,
308 "xaxis": "real",
309 "yaxis": "imaginary",
310 "xdatacolumn": "corrected",
311 "ydatacolumn": "corrected",
312 "selectdata": True,
313 "field": "0408-65,0252-712",
314 "correlation": "XX,YY",
315 "iteraxis": "field",
316 "coloraxis": "scan",
317 "showmajorgrid": True,
318 "showminorgrid": False,
319 "overwrite": True,
320 }, input=INPUT, output=OUTPUT, label="plot corrected re/im after calibration")
321
322
323 STEPS = [
324 # "view obs info",
325 # "plot subarray ants",
326 # "plot subarray uv coverage",
327 #####"backup original flags",
328 # "restore original flags",
329 # "flag autocorrelation products",
330 # "plot raw ampfreq before flagging",
331 # "plot raw ampuvdist before flagging",
332 # "autoflag rfi calibrators",
333 # "autoflag rfi target",
334 # "plot raw ampfreq after flagging",
335 # "plot raw ampuvdist after flagging",
336 # "plot raw ampphase after flagging",
337 # "plot raw amptime after flagging",
338 # "clear calibration",
339 # "calibrate delays",
340 # "plot delays",
341 # "set flux model",
342 # "calibrate bandpass",
343 # "plot bandpass",
344 # "calibrate gains",
345 # "fluxscale gains",
346 # "plot gains",
347 "apply calibration",
348 "autoflag rfi calibrators",
349 "autoflag rfi target",
350 "plot corrected ampphase after calibration",
351 "plot corrected re/im before calibration",
352 "plot corrected re/im after calibration"
353 ]
Inspecting the amplitude / phase and real / imaginary plots gives us an estimation of the residual calibration error. The large spread in phase prior to calibration is substantially reduced and 0 centred. The amplitude of the secondary has been scaled by the fluxscale operation we ran earlier. This is a reasonable first estimation of the flux of the secondary calibrator.
A good calibrator source is unresolved source with some flux c. Its visibilities should therefore be centred around c + 0j. As expected the real/imaginary plots goes from being a torus for the uncalibrated products to a ball:
For imaging we will invert the measurements taken of the target field using the Fast Fourier Transform. There exists an approximate Fourier relation between the measurements taken by an interferometer and the brightness distribution being observed. This yields an image of the brightness distribution convolved with the response of the interferometer. We will use the WSClean task to image and deconvolve the DEEP_2 field.
MeerKAT has 8km baselines. One of the most important relations in imaging is the critical sampling criterion (see chapter 5 of the Rhodes Radio Astronomy Techniques and Technologies eBook)
For MeerKAT we can use values around 1.653arcsec. This will place a few pixels across the PSF response of the interferometer for any frequency in L-Band. At 1GHz the first null of the antenna response is just under 2 degrees. This will set the limit of our field of view (nothing prevents you from imaging out into the first sidelobe and beyond, but it becomes more expensive computationally and memory-wise to do so – in this regime directional dependent calibration artefacts starts dominating the image as well).
The array is currently dominated by short baselines we should down weigh the core of the telescope so that the spatial frequencies enjoy “more uniform” sampling. For this we use uniform (visibility weighted by the neighbourhood sample count) weighting. This is the default for this imaging software. Lets deconvolve the image deeply by setting the number of clean components to 20000.
We will be making a widefield image because the measurements will not all lie on the same plane. Take note that this is true for any array with non-East-West baselines, even if the array layout itself is perfectly flat! Take care to peruse these parameters, since the imaging job will take slightly longer to run.
...
321
322 recipe.add("cab/wsclean", "image_1gc_target", {
323 "msname": "1491291289.1ghz.1.1ghz.4hrs.ms",
324 "datacolumn": "CORRECTED_DATA",
325 "join-channels": True,
326 "channels-out": 7,
327 "size": [4096, 4096],
328 "scale": "1.653asec",
329 "mgain": 0.8,
330 "gain": 0.1,
331 "niter": 20000,
332 "name": "DEEP2_1GC",
333 "field": 2,
334 }, input=INPUT, output=OUTPUT, label="image target after transfer calibration")
335
336 STEPS = [
337 # "view obs info",
338 # "plot subarray ants",
339 # "plot subarray uv coverage",
340 #####"backup original flags",
341 # "restore original flags",
342 # "flag autocorrelation products",
343 # "plot raw ampfreq before flagging",
344 # "plot raw ampuvdist before flagging",
345 # "autoflag rfi calibrators",
346 # "autoflag rfi target",
347 # "plot raw ampfreq after flagging",
348 # "plot raw ampuvdist after flagging",
349 # "plot raw ampphase after flagging",
350 # "plot raw amptime after flagging",
351 # "clear calibration",
352 # "calibrate delays",
353 # "plot delays",
354 # "set flux model",
355 # "calibrate bandpass",
356 # "plot bandpass",
357 # "calibrate gains",
358 # "fluxscale gains",
359 # "plot gains",
360 # "apply calibration",
361 # "autoflag rfi calibrators",
362 # "autoflag rfi target",
363 # "plot corrected ampphase after calibration",
364 # "plot corrected re/im before calibration",
365 # "plot corrected re/im after calibration",
366 "image target after transfer calibration"
367 ]
This concludes this tutorial on MeerKAT cross-calibration. However, this is not the end - as you can see there are still significant artefacts remaining in this image. We typically need to iteratively improve the calibration by constructing a model of the field and calibrating telescope phase (primarily) in a process known as self-calibration (or "2GC calibration").