.. _DDECal.json#/: DDECal ====== Calibrate direction dependent gains. .. _DDECal.json#/inputs: inputs ****** .. _DDECal.json#/inputs/type: type ++++ :type: ``string`` :doc: Case-insensitive step type; must be 'ddecal' `.` .. _DDECal.json#/inputs/antennaconstraint: antennaconstraint +++++++++++++++++ :type: ``list`` :default: ``[]`` :doc: A list of lists specifying groups of antennas that are to be constrained to have the same solution. Example: "[ [CS002HBA0,CS002HBA1],[CS003HBA0,CS003HBA1] ]" will keep the solutions of CS002HBA0 and 1 the same, and the same for CS003 `.` .. _DDECal.json#/inputs/applycal.*: applycal.* ++++++++++++++ :doc: ApplyCal sub-step, same as in Predict step. One can pass an h5parm with as many directions as set in "directions" and each direction model is corrupted accordingly `.` .. _DDECal.json#/inputs/approxchunksize: approxchunksize +++++++++++++++ :type: ``int`` :default: ``0`` :doc: Size of fitted chunksize during approximation stage in nr of channels. With approxchunksize=1 the constraint is disabled during the approx stage (so channels are solved for independently). Once converged, the solutions are constrained and more iterations are performed until that has converged too. The default is approxchunksize=0, which calculates the chunksize from the bandwidth (resulting in 10 chunks per octave of bandwidth) `.` .. _DDECal.json#/inputs/approximatetec: approximatetec ++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Uses an approximation stage in which the phases are constrained with the piece-wise fitter, to solve local minima problems. Only effective when ``mode=tec`` or ``mode=tecandphase`` `.` .. _DDECal.json#/inputs/approxtolerance: approxtolerance +++++++++++++++ :type: ``double`` :default: ``tolerance*10`` :doc: Tolerance at which the approximating first stage is considered to be converged and the second full-constraining stage is started. The second stage convergences when the tolerance set by the 'tolerance' keyword is reached. Setting approxtolerance to lower values will cause more approximating iterations. Since tolerance is by default 1e-5, approxtolerance is by default 1e-4 `.` .. _DDECal.json#/inputs/coreconstraint: coreconstraint ++++++++++++++ :type: ``double`` :default: ``0`` :doc: Distance in meters. When unequal to 0, all stations within the given distance from the reference station (0) will be constraint to have the same solution `.` .. _DDECal.json#/inputs/detectstalling: detectstalling ++++++++++++++ :type: ``bool`` :doc: Stop iterating when no improvement is measured anymore (after a minimum of 30 iterations) `.` :default: ``True`` .. _DDECal.json#/inputs/directions: directions ++++++++++ :type: ``list`` :default: ``[]`` :doc: List of directions to calibrate on. Every element of this list should b a list of facets. Default: every facet is a direction `.` .. _DDECal.json#/inputs/flagdivergedonly: flagdivergedonly ++++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Flag only the unconverged solutions for which divergence was detected. At the moment, this option is effective only for rotation+diagonal solves, where divergence is detected when the amplitudes of any station are found to be more than a factor of 5 from the mean amplitude over all stations. If divergence for any one station is detected, all stations are flagged for that solution interval. Only effective when ``flagunconverged=true`` and ``mode=rotation+diagonal`` `.` .. _DDECal.json#/inputs/flagunconverged: flagunconverged +++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Flag unconverged solutions (i.e., those from solves that did not converge within maxiter iterations) `.` .. _DDECal.json#/inputs/grouppredict: grouppredict ++++++++++++ :type: ``bool`` :default: ``False`` :doc: Alternative predict step for BDA data, which groups the input data into similarly averaged baselines and uses a regular Predict step for each group. Due to the overhead of having different groups, the normal predict step processes BDA data faster. This step is therefore mainly used for benchmarking `.` .. _DDECal.json#/inputs/h5parm: h5parm ++++++ :type: ``string`` :doc: Filename of output H5Parm (to be read by e.g. losoto). If empty, defaults to `instrument.h5` within the measurement set `.` .. _DDECal.json#/inputs/idg.buffersize: idg.buffersize ++++++++++++++++++ :type: ``int`` :default: ``Based on memory`` :doc: Set the amount of timesteps that are to be used for each IDG buffer `.` .. _DDECal.json#/inputs/idg.images: idg.images ++++++++++++++ :type: ``list`` :doc: Filename of ``.fits`` model images, one per frequency term. The terms are defined as for a polynomial source spectra (not logarithmic), e.g. see `this WSClean page `__. The frequency in the metadata of the fits files is used as nu0 in the polynomial evaluation `.` :default: ``[]`` .. _DDECal.json#/inputs/idg.regions: idg.regions +++++++++++++++ :type: ``string`` :default: ``""`` :doc: DS9 regions file describing the facets for IDG prediction `.` .. _DDECal.json#/inputs/keep_host_buffers: keep_host_buffers +++++++++++++++++ :type: ``bool`` :default: ``False`` :doc: ``Setting this to true will cause the host buffers to be kept during the execution if used in combination with the GPU solver. This can cause an higher consumption of memory but saves execution time.`` .. _DDECal.json#/inputs/keepmodel: keepmodel +++++++++ :type: ``bool`` :default: ``False`` :doc: If true, pass the uncorrected model data to the next step. DDECal stores the model data using its step name plus the (first) direction name. For example, if "steps=[ddecal]" and "ddecal.directions=[[center,dec_off],[ra_off]]", the model data names are "ddecal.center" and "ddecal.ra_off". If false, delete all model data, including the model data from "reusemodel" `.` .. _DDECal.json#/inputs/llssolver: llssolver +++++++++ :type: ``string`` :default: ``qr`` :doc: Select which linear least-squares solver to use for the direction-solving algorithm. When ``solveralgorithm`` is not ``directionsolve``, this parameter has no effect. Supported least-squares solvers are ``qr``, ``svd`` and ``normalequations`` `.` .. _DDECal.json#/inputs/maxapproxiter: maxapproxiter +++++++++++++ :type: ``int`` :default: ``maxiter/2`` :doc: Maximum number of iterations during approximating stage `.` .. _DDECal.json#/inputs/maxiter: maxiter +++++++ :type: ``int`` :default: ``50`` :doc: Maximum number of iterations `.` .. _DDECal.json#/inputs/minvisratio: minvisratio +++++++++++ :type: ``double`` :default: ``0`` :doc: Minimum number of visibilities within a solution interval, e.g. 0.6 for at least 60% unflagged vis. Intervals with fewer vis will be flagged `.` .. _DDECal.json#/inputs/mode: mode ++++ :type: ``string`` :default: ``diagonal`` :doc: Solve type. Options are scalar, scalarphase, scalaramplitude, diagonal, diagonalphase, diagonalamplitude, fulljones, tec, tecandphase, rotation and rotation+diagonal `.` .. _DDECal.json#/inputs/modeldatacolumns: modeldatacolumns ++++++++++++++++ :type: ``list`` :default: ``[]`` :doc: Use model data from the measurement set. All directions from this parameter will be pointing to the measurement set metadata `.` .. _DDECal.json#/inputs/modelnextsteps: modelnextsteps ++++++++++++++ :type: ``list`` :default: ``[]`` :doc: ``Defines the steps that will be applied to the model in every direction before using the output for calibration.`` .. _DDECal.json#/inputs/modelnextsteps.: modelnextsteps ++++++++++++++ :type: ``string`` :doc: Allows steps to be defined for a specific direction. These directions, (given in the `` placeholder) can be specified by their model column name, the name of a DS9 region, or the first patch name in the direction if using a sky model. For example: `modelnextsteps.MODEL_DATA_2=[applyextrabeam], applyextrabeam.type=applybeam` `.` .. _DDECal.json#/inputs/nchan: nchan +++++ :type: ``int`` :doc: Number of channels in each channel block, for which the solution is assumed to be constant. The default is 1, meaning one solution per channel (or in the case of constraints, fitting the constraint over all channels individually). 0 means one solution for the whole channel range. If the total number of channels is not divisable by nchan, some channelblocks will become slightly larger. When using BDA, the value of nchan should be bigger than or equal to the maximum number of averaged frequency channels `.` :default: ``1`` .. _DDECal.json#/inputs/onlypredict: onlypredict +++++++++++ :type: ``bool`` :default: ``False`` :doc: Instead of solving, output the predicted visibilities instead. This is useful for testing, although when doing faceted prediction with IDG, it might be fast for certain cases `.` .. _DDECal.json#/inputs/phasereference: phasereference ++++++++++++++ :type: ``bool`` :default: ``True`` :doc: Use phase referencing. This setting is only effective when ``mode=tec`` or ``mode=tecandphase`` `.` .. _DDECal.json#/inputs/propagateconvergedonly: propagateconvergedonly ++++++++++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Propagate solutions of the previous time slot only if the solve converged. Only effective when propagatesolutions=true `.` .. _DDECal.json#/inputs/propagatesolutions: propagatesolutions ++++++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Initialize solver with the solutions of the previous time slot `.` .. _DDECal.json#/inputs/reusemodel: reusemodel ++++++++++ :type: ``list`` :default: ``[]`` :doc: Reuse model data from previous (ddecal) step(s). The list should contain the model data names, as set by a previous step, e.g., "[ddecal1.MODEL_DATA, ddecal1.center]" `.` .. _DDECal.json#/inputs/rotationreference: rotationreference +++++++++++++++++ :type: ``bool`` :default: ``False`` :doc: Use rotation referencing with the first antenna as the reference antenna. When enabled, the first antenna gets a rotation angle of 0. The rotation angle of the other antennas will be relative to the first antenna. If the first antenna has no valid data, the second antenna will be the reference antenna, and so on. This setting is only effective when ``mode=rotation+diagonal`` `.` .. _DDECal.json#/inputs/savefacets: savefacets ++++++++++ :type: ``bool`` :default: ``False`` :doc: Write out each facet as a fits file (named facet.fits). Only useful using idg `.` .. _DDECal.json#/inputs/smoothnessconstraint: smoothnessconstraint ++++++++++++++++++++ :type: ``double`` :default: ``0.0`` :doc: Kernel size in Hz. When unequal to 0, will constrain the solutions to be smooth over frequency by convolving the solutions with a kernel of the given size (bandwidth). The default kernel is a Gaussian kernel, and the kernel size parameter is the 3 sigma point from the centre where the kernel is cut off. For example, with a kernel size of 2 MHz and with 1 MHz channels, the kernel will cover 2 channels on both side from the centre, and therefore 5 channels in total `.` .. _DDECal.json#/inputs/smoothnessrefdistance: smoothnessrefdistance +++++++++++++++++++++ :type: ``double`` :default: ``0.0`` :doc: An optional reference distance (in meters) for the smoothness constraint. When unequal to 0, the size of the smoothing kernel will vary, such that antennas (stations for LOFAR) closer to antenna 0 will be smoothed stronger, i.e. with a larger kernel size. `.` .. _DDECal.json#/inputs/smoothnessreffrequency: smoothnessreffrequency ++++++++++++++++++++++ :type: ``double`` :default: ``0.0`` :doc: An optional reference frequency (in Hz) for the smoothness constraint. When unequal to 0, the size of the smoothing kernel will vary over frequency by a factor of "frequency / smoothnessreffrequency", i.e., the kernel will be smaller for lower frequencies. This compensates for the fact that the ionosphere causes larger fluctuations at lower frequencies `.` .. _DDECal.json#/inputs/solint: solint ++++++ :type: ``int`` :doc: Solution interval in timesteps. If a value of 0 is specified, the solver will use a single solution interval that covers the entire observation. When using BDA, the solint should be bigger than or equal to the maximum averaged time interval `.` :default: ``1`` .. _DDECal.json#/inputs/solutions_per_direction: solutions_per_direction +++++++++++++++++++++++ :type: ``list`` :default: ``[1``, ``1``, ``...``, ``1]`` :doc: List of number of subsolutions per solution interval per direction. Defaults to a single solution per solution interval per direction. Entries in provided list should be integer divisors of solint. If not all directions are specified, missing directions are solved with one solution per solution interval. Direction-dependent intervals are only supported when solveralgorithm is set to directioniterative `.` .. _DDECal.json#/inputs/solveralgorithm: solveralgorithm +++++++++++++++ :type: ``string`` :doc: Select the algorithm used for the solving. Options are ``directionsolve``, ``directioniterative``, ``lbfgs``, and ``hybrid``. For 50 directions, the direction-iterating algorithm can be an order of magnitude faster than the direction-solving algorithm. The downside might be that it is less stable and/or less accurate, which is still to be tested. All algorithms support all solver modes (scalar, diagonal, full Jones, etc). The use of direction-dependent solution intervals (see ``solutions_per_direction``) is only supported for the directioniterative method. The hybrid algorithm is the best of both. It runs 1/6th of the maximum iterations with the direction-solving algorithm, and then switches to the direction-iterating `.` :default: ``directionsolve`` .. _DDECal.json#/inputs/solverlbfgs.dof: solverlbfgs.dof +++++++++++++++++++ :type: ``double`` :default: ``200.0`` :doc: The degrees of freedom of the robust noise model `.` .. _DDECal.json#/inputs/solverlbfgs.history: solverlbfgs.history +++++++++++++++++++++++ :type: ``int`` :doc: LBFGS solver history size. This is the memory (as a multiple of the number of parameters) used to store information pertaining to previous iterations `.` :default: ``10`` .. _DDECal.json#/inputs/solverlbfgs.iter: solverlbfgs.iter ++++++++++++++++++++ :type: ``int`` :default: ``4`` :doc: Iterations per each minibatch of data. This is the inner iterations and ``maxiter`` is the outer iterations. Therefore, the total number iterations is the product of these two values `.` .. _DDECal.json#/inputs/solverlbfgs.minibatches: solverlbfgs.minibatches +++++++++++++++++++++++++++ :type: ``int`` :default: ``1`` :doc: Divide the data into minibatches equal to this number at each iteration. If this value is greater than one, less data is used at each iteration (without losing the final solution accuracy) making the iterations faster `.` .. _DDECal.json#/inputs/sourcedb: sourcedb ++++++++ :type: ``string`` :doc: Sourcedb (created with `makesourcedb`) with the sky model to calibrate on `.` When the path ends with ``.skymodel`` or ``.txt`` DP3 expects a skymodel file as used by makesourcedb. This makes it possible to directly use a skymodel file without using makesourcedb to convert the file. .. _DDECal.json#/inputs/statfilename: statfilename ++++++++++++ :type: ``string`` :doc: File to write the step-sizes to. Form of the file is: " ", and all solution intervals are concatenated. File is not written when this parameter is empty `.` .. _DDECal.json#/inputs/stepsigma: stepsigma +++++++++ :type: ``double`` :default: ``0.1`` :doc: Threshold for stalling detection, stop iterating when the running mean of the step sizes is less than stepsigma times their running standard deviation, i.e., the step size is just noise `.` .. _DDECal.json#/inputs/stepsize: stepsize ++++++++ :type: ``double`` :default: ``0.2`` :doc: stepsize between iterations `.` .. _DDECal.json#/inputs/storebuffer: storebuffer +++++++++++ :type: ``bool`` :doc: ``Setting this to true will store the solution of DDECal into the buffer``, ``allowing the usage of this solution in a later step. For example``, ``a pipeline with DDECal -> OneApplyCal would be able to apply solutions to the data without requiring an intermediate format to be stored to disk. Note that it currently only works for single-direction solutions.`` :default: ``False`` .. _DDECal.json#/inputs/subtract: subtract ++++++++ :type: ``bool`` :default: ``False`` :doc: Subtracts the corrected model from the data. **NOTE** This may not work when you apply a uv-cut `.` .. _DDECal.json#/inputs/tolerance: tolerance +++++++++ :type: ``double`` :default: ``1e-5`` :doc: Controls the accuracy to be reached: when the normalized solutions move less than this value, the solutions are considered to be converged and the algorithm finishes. Lower values will cause more iterations to be performed `.` .. _DDECal.json#/inputs/usebeammodel: usebeammodel ++++++++++++ :type: ``bool`` :default: ``False`` :doc: use the beam model. All beam-related options of the Predict step are also valid `.` .. _DDECal.json#/inputs/usegpu: usegpu ++++++ :type: ``bool`` :default: ``False`` :doc: Use GPU solver. This is an experimental feature only available for the iterative diagonal solver and requires DP3 to be built with BUILD_WITH_CUDA=1 `.` .. _DDECal.json#/inputs/uvlambdamin: uvlambdamin +++++++++++ :type: ``double`` :default: ``0`` :doc: Ignore baselines / channels with UV < uvlambdamin wavelengths. **Note**: also all other variants of uv flagging described in :ref:`UVWFlagger` (uvmmin, uvmrange, uvlambdarange, etc) are supported (New in 3.1) `.`