Pointing workflow

docs/site_pipeline.rst

@@ -377,8 +377,251 @@ The output database ``wafer_info.sqlite`` and HDF5 file
 ``wafer_info.h5`` are written to the ``output_dir``, which is created
 if it does not exist.
 
-make_read_det_match
+
+get_brightsrc_pointing_part1  
+----------------------------
+
+The two-part ``get_brightsrc_pointing_part{}`` script set will solve for the xieta
+coordinates of detectors that observe a bright source during an observation.
+
+It is a two part process that requires a map step and then a TOD step.
+The scripts require the settings and preprocessing config files described below. 
+
+For job submission and parallelization, see example NERSC slurm submission config at the end of this section.
+
+The code will process all wafers unless otherwise specified. 
+It is recommended to run with ``parallel_job: True`` in the config files if analyzing 
+multiple wafers at once. 
+Otherwise, specify a wafer slot or restrict detectors in command line args to debug.
+
+Command Line arguments:
+.. argparse::
+   :module: sotodlib.site_pipeline.get_brightsrc_pointing_part1
+   :func: get_parser
+
+
+There options to include min_ctime and max_ctime arguments, which will process all observations
+in the time frame, is not recommended unless severely restricting the detectors for debugging.
+
+
+Generated results
 ```````````````````
+The Step 1 map-based analysis scripts will generate the following outputs in the specified directory:
+
+ 1. Single detector maps in ``/results/single_det_maps/<obs_id>_<ws#>.hdf``. 
+
+  * All single maps are packaged in a single hdf file, with detector readout_id as the keys in the h5py file. 
+
+ 2. Fitted xi-eta focal plane position results saved as ResultSet in ``/path/to/results/map_based_results``
+ as specified in the Step 1 config file. Script will append 'force_zero_roll' onto the specified results_dir
+ if True in config file. Load ResultSet with keyword 'focal_plane'
+
+  * Contents: ``ResultSet<[dets:readout_id, xi, eta, gamma, R2], N rows>``
+
+
+The Step 2 TOD-based analysis scripts will use the map-based results as a starting point
+ and then generate the finalized outputs in the specified directory:
+
+ 1.  Fitted xi-eta focal plane position results saved as ResultSet in ``/path/to/results/tod_based_results``
+ as specified in config file for Step-2. Script will append 'force_zero_roll' onto the specified results_dir
+ if True in config file. Load ResultSet with keyword 'focal_plane'.
+ The median boresight values from small time range the source was visible to each detector is included.
+
+  * Contents: ``ResultSet<[dets:readout_id, xi, eta, gamma, xi_err, eta_err, R2, redchi2, az, el, roll], N rows>``
+
+Configuration Files
+```````````````````
+The configuration files to be input as ``configs`` in the command line should 
+have the following arguments as well as any preprocessing steps wished to be taken.
+Only processing steps that are agnostic of det-match can be used to do
+initial analyses without formalized metadata.
+
+The parameters in these examples could be used for SAT mid-freq moon observations.
+
+Step 1 Config:
+
+.. code-block:: yaml
+
+  context_file: /path/to/context.yaml
+  query_tags: ['moon=1'] #(alternatively specify --sso_name in kwargs)
+
+  optics_config_fn: '/global/cfs/cdirs/sobs/users/elleshaw/process_brightsrc/ufm_to_fp.yaml'
+  single_det_maps_dir: /path/to/results/single_det_maps
+  results_dir: /path/to/results/map_based_results
+
+  parallel_job: True  #For job submission. Parallel across wafers.
+  wafer_mask_det: 8.  # (degrees) mask around detector to cut TOD when source too far away. 
+  res_deg: 0.3
+  xieta_bs_offset: [0., 0.]  #Good to input xieta offset in radians. (!!! for satp2)
+  save_normal_roll: False  #false for SAT, true for LAT
+  save_force_zero_roll: True  #true for SAT, false for LAT
+
+  hit_circle_r_deg: 7.  # radial mask to decide which UFMs are hit by source and should be analyzed.
+  hit_time_threshold: 600  #seconds, if hit_time not met then UFM does not get analyzed.
+
+  process_pipe:
+    - name: 'detrend'
+      process:
+        count: 2000
+        method: 'linear'
+    - name: 'apodize'
+      process:
+        apodize_samps: 2000
+    - name: 'fourier_filter'
+      process:
+        signal_name: "signal"
+        wrap_name: null
+        filt_function: "low_pass_sine2"
+        trim_samps: null
+        filter_params:
+          cutoff: 1.9
+          width: 0.2
+    - name: 'fourier_filter'
+      process:
+        signal_name: "signal"
+        wrap_name: null
+        filt_function: "high_pass_sine2"
+        trim_samps: 2000
+        filter_params:
+          cutoff: 0.05
+          width: 0.1
+
+Part 2 is the TOD-based step. Its config file should look like the following.
+The parameters in these examples are used for SAT mid-freq moon observations.
+
+.. code-block:: yaml
+
+  context_file: /path/to/context.yaml
+  query_tags: ['moon=1'] #(alternatively specify --sso_name in kwargs)
+
+  optics_config_fn: '/global/cfs/cdirs/sobs/users/elleshaw/process_brightsrc/ufm_to_fp.yaml'
+  fp_hdf_dir: /path/to/results/map_based_results from step 1 config file. 
+      # If force_zero_roll is was True, then append _force_zero_roll to the end.
+      # Just make sure it matches where the results from Step 1 are.
+  result_dir: /path/to/resuls/tod_based_results #Where you want the final Step2 results to show up.
+
+  parallel_job: True
+  force_zero_roll: True  #Results will show up roatated in the xi-eta results as they are on the sky.
+  ds_factor: 40
+  mask_deg: 2.5  # (degrees) size for circular mask around SSO (helps exclude focal plane reflections too)
+  fit_func_name: 'gaussian2d_nonlin'
+  max_non_linear_order: 3  #Suggested to use 1 for jupiter or sso's 
+                           #that do not saturate.
+  fwhm_init_deg: 0.5  # (degrees) Lower for SATp2
+  error_estimation_method: 'force_one_redchi2'
+  flag_name_rms_calc: 'around_source'
+  flag_rms_calc_exclusive: False
+
+  process_pipe:
+    - name: 'detrend'
+      process:
+        count: 2000
+        method: 'linear'
+    - name: 'fourier_filter'
+      process:
+        signal_name: 'signal'
+        filt_function: 'iir_filter'
+        trim_samps: null
+        filter_params:
+          invert: True
+    - name: 'apodize'
+      process:
+        apodize_samps: 2000
+    - name: 'fourier_filter'
+      process:
+        signal_name: "signal"
+        wrap_name: null
+        filt_function: "low_pass_sine2"
+        trim_samps: null
+        filter_params:
+          cutoff: 1.9
+          width: 0.2
+    - name: 'source_flags'
+      source_flags_name: 'source_wide'
+      save: True
+      calc:
+        mask:
+          shape: circle
+          xyr: [0., 0., 5.0]
+        merge: True
+        max_pix: 10000000000
+    - name: 'source_flags'
+      source_flags_name: 'source_narrow'
+      save: True
+      calc:
+        mask:
+          shape: circle
+          xyr: [0., 0., 3.0]
+        merge: True
+        max_pix: 10000000000
+    - name: 'combine_flags'
+      process:
+        flag_labels: ['source_wide.moon', 'source_narrow.moon']
+        method: 'except' 
+        total_flags_label: 'around_source'
+    - name: 'flag_turnarounds'
+      process:
+        truncate: True
+    - name: 'sub_polyf'
+      process: 
+        method: 'legendre'
+        degree: 2
+        mask: 'around_source'
+        exclusive: False
+
+Example NERSC slurm job submission config file
+``````````````````````````````````````````````
+
+.. code-block:: yaml
+  #!/bin/bash -l
+
+  #SBATCH --qos=shared
+  #SBATCH --constraint=cpu
+  #SBATCH --nodes=1
+  #SBATCH --ntasks=1
+
+  #SBATCH --cpus-per-task=14
+  #SBATCH --time=00:30:00
+  #SBATCH --mem=220G`` #(may need regular queue & up to 400 Gb for long obs)
+
+  export OMP_NUM_THREADS=1
+  set -e
+
+  tele=$1
+  obs=$2
+  map=$3
+  basis=$4
+  source="moon_from_moon"
+
+  ymldir="/path/to/processing_settings_config_folder"
+  yfile="${ymldir}/preprocess_config_moon_${basis}_based_${tele}.yaml"
+
+  if (($map)); then
+    echo submitted map job;
+    srun -n 1 -N 1 -c 14 python3 
+       /path/to/sotodlib/site_pipeline/get_brightsrc_pointing_step1.py $yfile
+       --obs_id=${2} --sso_name="moon";
+  else
+    echo submitted tod job;
+    srun -n 1 -N 1 -c 14 python3 
+        /path/to/sotodlib/site_pipeline/get_brightsrc_pointing_step2.py $yfile 
+        --obs_id=${2} --sso_name="moon";
+  fi
+
+
+Submit the job submission file with the following commands:
+
+1. For Step 1 map-based
+
+ * ``sbatch submit_moon_job_script.sh <platform> <obs_id> 1 map``
+
+2. For Step 2 TOD based
+
+ * ``sbatch submit_moon_job_script.sh <platform> <obs_id> 0 tod``
+
+
+make_read_det_match
+-------------------
 This script generates the readout ID to detector ID mapping required to
 translate between the detector hardware information (ex: pixel position) and the
 readout IDs of the resonators used to index the SMuRF data. The script uses the 
@@ -447,6 +690,16 @@ entries mater.
           det_info: true
           multi: true
 
+
+get_brightsrc_pointing_part2
+----------------------------
+See Part 1 for description
+
+.. argparse::
+   :module: sotodlib.site_pipeline.get_brightsrc_pointing_part2
+   :func: get_parser
+
+
 update_det_match
 ------------------
 
@@ -692,6 +945,10 @@ The ``focal_plane_full`` dataset contains nine columns:
 - ``eta_m``: The measured eta in radians
 - ``gamma_m``: The measured gamma in radians.
 - ``weights``: The average weights of the measurements for this det.
+- ``r2``: The fit weight passed in from the get_brightsrc_pointing dataset
+- ``az``: The median Az value in radians from source-detector crossing
+- ``el``: The median El value in radians from source-detector crossing
+- ``roll``: The median Roll value in radians from source-detector crossing
 - ``n_point``: The number of pointing fits used for the det.
 - ``n_gamma``: The number of gamma fits used for this det.
 
@@ -732,7 +989,7 @@ always be ``(1, 1, 1)`` and ``shear`` will be ``0``.
 ``finalize_focal_plane`` will also output a ``ManifestDb`` as a file called ``db.sqlite``
 in the output directory.
 By default this will be indexed by ``stream_id`` and ``obs:timestamp`` and will point to the ``focal_plane`` dataset.
-If you are running in ``per_obs`` mode then it wirbe indexed by ``obs_id`` and will point
+If you are running in ``per_obs`` mode then it will be indexed by ``obs_id`` and will point
 to results associated with data observation.
 Be warned that in this case there will only be entries for observations with pointing fits,
 so design your context accordingly.