Setting up the calibration run

ehthops pipeline is driven by a series of shell scripts that execute the various stages of the calibration process step-by-step. Sample driver scripts for running the entire pipeline are provided in the scripts/ directory of the EHT-HOPS repository.

Note

This tutorial is intended only for those users who want to submit a job to run the entire calibration pipeline. Individual stages/steps of ehthops can also be run manually by executing only the relevant lines from the driver script found in scripts/ehthops_pipeline.sh.

Setting up the repository

Clone the public git repository for the EHT-HOPS pipeline. All the calibrated output files will be created within this directory:

cd /home/user/calibration
git clone https://github.com/sao-eht/ehthops.git

The code repository consists of four directories named hops-bx where x stands for the EHT “zoom” frequency band. Conventionally, in order of increasing frequency, the bands are named 1, 2, 3, and 4.

Note

The 2017 campaign has only two bands named lo and hi. We adopt the convention that b3 -> lo and b4 -> hi.

Additionally, the repository contains the following directories:

  • scripts: Contains the driver scripts for running the calibration pipeline and sample configuration files.

  • share: Marimo noteboks that are run after each iteration of fourfit to generate summary plots of the results of the current stage.

  • meta: Contains metadata files for the campaigns, which are used to generate control files.

Note

The meta directory is organized by campaign and frequency band, e.g., meta/eht2017/230GHz for the 230 GHz 2017 campaign. Note that the antab directory is not bundled with the repository and must be created and populated manually with the appropriate ANTAB files corresponding to the campaign, usually obtained from the metadata package generated by the EHTC.

The main pipeline script scripts/ehthops_pipeline.sh takes the filename of a configuration file as argument and proceeds to run the calibration. A sample configuration file scripts/settings.config explains all the keywords that can be set to control the calibration process. The user can copy and modify this file as needed. ehthops_pipeline.sh must be run from within the hops-bx directories.

For this tutorial, we will use the “lo” band, so we will copy the driver script to hops-b3:

cd ehthops/ehthops/hops-b3/
cp ../scripts/ehthops_pipeline.sh .
cp ../scripts/settings.config .

Note

The script scripts/cleanup.sh helps to quickly delete all output files created during calibration, leaving the hops-bx directories in a clean state. Copy this to the hops-b3 directory and run source cleanup.sh to clean up the directory when necessary.

Updating the configuration file for calibration

The sample configuration file ehthops/settings.config contains the following keywords:

  • SET_SRCDIR: Base directory containing the Mk4 data to be processed.

  • SET_CORRDAT: List of paths (or simple directory names such as correlator data tags/releases) relative to SRCDIR separated by ‘:’.

  • SET_METADIR: Directory where campaign metadata are to be found. Normally found under ehthops/ehthops/meta.

  • SET_EHTIMPATH: Path to eht-imaging source code.

  • stages: Stages to run (i.e. directory names found under hops-bx) as a space-separated string.

  • SET_OBSYEAR: 4-letter code representing year of observation.

  • SET_FILTERSTRING: Optional substring used to filter which data directories are linked from the archive. Ignored if empty.

  • SET_MIXEDPOL: Boolean value. Set this to true to request mixedpol calibration. This will assume that all ALMA data are in linear polarization basis while the rest are in circular polarization basis.

  • SET_HAXP: Boolean value. Set this to true to indicate that ALMA linear polarization data are present in *-haxp directories and must replace circularly polarized ALMA data originally linked from *-hops directories. Setting this to true will automatically set MIXEDPOL=true.

  • SET_CAMPAIGN: An EAT-recognizable code; currently EHT2017, EHT2018, EHT2021, EHT2022 are supported.

  • SET_INPUTDIR: Input directory for post-processing stages. This is different for different stages and must be re-set for each post-processing stage. More details could be found in the sample settings.config file in the repository.

For this tutorial, we will assign the following values to the keywords:

SET_SRCDIR="/home/user/calibration/data/extracted"
SET_CORRDAT="2016.1.01154.V"
SET_METADIR="/home/user/calibration/ehthops/meta/eht2017/230GHz"
SET_EHTIMPATH="/home/user/software/eht-imaging"
stages="0.bootstrap 1.+flags+wins 2.+pcal 3.+adhoc 4.+delays 5.+close 6.uvfits"
SET_YEAR="2017"
SET_FILTERSTRING=""
SET_MIXEDPOL=true
SET_HAXP=true
SET_CAMPAIGN="EHT2017"
SET_INPUTDIR=""

More information on how to determine the values of the command-line options can be found here.

Submitting the calibration job to SLURM

A sample SLURM script can be found at scripts/ehthops_slurm.job. You should already have set up the Python environment and installed the necessary dependencies for running the pipeline. Copy this SLURM job file to ehthops/hops-b3 and submit to SLURM with sbatch:

cp ../scripts/ehthops_slurm.job .

# Take some time to update the resources being requested and the job parameters.

# Submit to SLURM
sbatch ehthops_slurm.job