Additional Data#

Some additional data can be calculated (model slew times, etc). Some additional data can be gathered by querying multiple sources.

rubin_nights.augment_visits.augment_visits(visits, instrument='lsstcam', skip_rs_columns=False, predicted_zeropoint_offsets=None, cols_from='visit')[source]#

Add additional columns to the dataframe resulting from querying the visit1 + visit1_quicklook tables. Calls add_rubin_scheduler_cols and add_rubin_sim_cols, after adding some basic additional information.

Parameters:

  • visits (DataFrame) – The visit information from cdb_{instrument}.visit1 and cdb_{instrument}.visit1_quicklook (if available).

  • instrument (str, default: 'lsstcam') – The instrument for the visits. Used to calculate the approximate rotTelPos value.

  • skip_rs_columns (bool, default: False) – Skip calculation of any columns that require rubin_scheduler or rubin_sim, even if those packages are installed. This will only sort the visits in time, calculate visit_gap, and calculate the boresight coordinates in ecliptic and galactic coordinates.

  • predicted_zeropoint_offsets (dict | None, default: None) – Offsets to add to the predicted zeropoint values. If None, will pick appropriate defaults based on instrument.

  • cols_from (str, default: 'visit') – A string to indicate whether the visits come from the ‘visit’ oriented tables or the ‘ccd’ oriented tables, as the column names vary slightly.

Returns:

visits – The visit information, with additional columns added for predicted zeropoint values, sky background in magnitudes, an estimated m5 depth (from zeropoint + sky), as well as an approximate rotTelPos (likely off by ~1 deg). Some columns may be reformatted for dtypes.

Return type:

pd.DataFrame

Notes

In addition to the columns added by add_rubin_scheduler_cols and add_rubin_sim_cols, this will add the visit_gap values as well as translations of the coordinates into galactic and ecliptic coordinates. Some columns which can occasionally be Object due to None values are also converted explicitly back to floats.

rubin_nights.augment_visits.exclude_visits(visits, bad_visit_ids)[source]#

Remove the visits_ids in bad_visit_ids from visits.

Parameters:

  • visits (pd.DataFrame) – A dataframe containing visit information, with visit_id values.

  • bad_visit_ids (list [ str ]) – The list of bad visit_ids to remove. This could be generated from rubin_nights.consdb.fetch_excluded_visits or rubin_nights.targets_and_visits.flag_potential_bad_visits or any other list of unwanted visit_ids.

Returns:

good_visits – The visits dataframe but with bad_visit_ids removed.

Return type:

pd.DataFrame

rubin_nights.augment_visits.fetch_excluded_visits(instrument='lsstcam')[source]#

Retrieve excluded visit list from the instrument-appropriate BAD_VISITS URI at github @ lsst-dm/excluded_visits.

The bad visit list at this repo is very incomplete. See also targets_and_visits.flag_potential_bad_visits.

Parameters:

instrument (str, default: 'lsstcam') – Which bad.ecsv file to retrieve. The options are lsstcam or lsstcomcam.

Returns:

bad_visit_ids – The bad visit_ids from the github repo bad.ecsv file.

Return type:

list [ str ]

rubin_nights.rubin_scheduler_addons.add_model_slew_times(visits, efd_client, model_settle=1, dome_crawl=False, slew_while_changing_filter=False, ideal_tma=40)[source]#

“Add model (applied tma limits plus FBS-default tma limits) calculated slewtimes to visits dataframe, in slew_model and slew_model_ideal.

This is only applicable to SimonyiTel at present!

Parameters:

  • visits (DataFrame) – The visit information. Expected to contain columns of s_ra, s_dec, sky_rotation, obs_start_mjd and band for slewtime calculation.

  • efd_client (InfluxQueryClient) – Used to query the EFD for the applied TMA limits at the time of the visits.

  • model_settle (float, default: 1) – The amount of settle time to add to the model_slew. This should make the model_slew time match the TMAevent time. Might vary over time.

  • dome_crawl (bool, default: False) – Enable dome crawl when calculating slew times, if True.

  • slew_while_changing_filter (bool, default: False) – Slew and change filter at the same time, or if False - sequentially.

  • ideal_tma (float, default: 40) – Model TMA movement value to use for the ideal model, in percent.

Returns:

visits_with_slews, slews – Same visit information, with additional columns slew_model and slew_model_ideal.

Return type:

pd.DataFrame, pd.DataFrame

Notes

Since the slew should be calculated from the previous location on the sky, subsets of visits that do not include the starting position may have inaccurate first slew estimates. Slews are the model slewtime to the visit (and compare against visit_gap for the same visit).

rubin_nights.rubin_scheduler_addons.add_rubin_scheduler_cols(visits, cols_from='visit1_quicklook')[source]#

Add columns that require rubin_scheduler (including Almanac) parallactic angle and rotator angle, LST, and moon information.

Parameters:

  • visits (DataFrame) – The visit information from cdb_{instrument}.visit1 and cdb_{instrument}.visit1_quicklook (if available).

  • cols_from (str, default: 'visit1_quicklook') – Use columns expected from the visit1_quicklook table or from the ccdvisit1_quicklook table. The difference is whether _median is at the end of the column name.

Returns:

visits – The visit information, with additional columns added for predicted zeropoint values, sky background in magnitudes, an estimated m5 depth (from zeropoint + sky).

Return type:

pd.DataFrame

Notes

Columns calculated and added: lst, HA (via astropy) moon_alt, moon_az, moon_RA, moon_dec, moon_distance, moon_illum (via the rubin_scheduler almanac) fwhm_eff, fwhm_geom, fwhm_500_zenith (via something close to the rubin_scheduler SeeingModel (but lambda^-0.2) approx_pa, approx_rotTelPos (via rubin_scheduler approx values)

rubin_nights.rubin_sim_addons.add_rubin_sim_cols(visits, instrument='lsstcam', predicted_zeropoint_offsets=None, cols_from='visit1_quicklook')[source]#

Add columns that require rubin_sim: predicted zeropoint and converted skybackground (mag/sq arcsec).

Parameters:

  • visits (DataFrame) – The visit information from cdb_{instrument}.visit1 and cdb_{instrument}.visit1_quicklook (if available).

  • instrument (str, default: 'lsstcam') – The instrument for the visits. Used to select the appropriate zeropoint offsets, if not provided.

  • predicted_zeropoint_offsets (dict | None, default: None) – Offsets to add to the predicted zeropoint values. If None, will pick appropriate defaults based on instrument.

  • cols_from (str, default: 'visit1_quicklook') – Use columns expected from the visit1_quicklook table or from the ccdvisit1_quicklook table. The difference is whether _median is at the end of the column name.

Returns:

visits – The visit information, with additional columns added for predicted zeropoint values, sky background in magnitudes, an estimated m5 depth (from zeropoint + sky).

Return type:

pd.DataFrame

Notes

Columns added are: zero_point_1s (zero_point[_median] scaled to 1s) zero_point_1s_pred (predicted from rubin_sim.predicted_zeropoint) clouds (the difference of the above values) sky_bg_mag (sky_bg[_median] scaled to mag/arcsecond^2) cat_m5 (calculated m5 from zeropoint/sky/readnoise values)

rubin_nights.rubin_sim_addons.consdb_to_opsim(consdb_visits)[source]#

Minimal conversion from consdb columns to opsim columns.

Parameters:

consdb_visits (DataFrame) – Dataframe of visit + quicklook information from the ConsDB.

Returns:

Dataframe of visit information reformatted for opsim. This is primarily renaming columns. The night is also added using the first night of the SV survey as night=0.

Return type:

opsim_visits

rubin_nights.observatory_status.get_dome_open_close(t_start, t_end, efd_client, with_sunset_sunrise=True)[source]#

Dataframe containing the open and close times for Simonyi dome, from positionCommanded and positionActual shutter values in lsst.sal.MTDome.apertureShutter.

Parameters:

  • t_start (Time) – Time of the start of the events.

  • t_end (Time) – Time of the end of the events.

  • efd_client (InfluxQueryClient) – Sync EFD client.

  • with_sunrise_sunset – If True (default), add -12 degree sunset and sunrise columns.

Returns:

dome_open_close – Dataframe containing pairs of open/close datetimes + elapsed time for each dome-open period in each day_obs. Note that the dome open/close times as well as sunset/sunrise are in UTC, including utc timescale.

Return type:

pd.DataFrame

Notes

This primarily returns dome open + close pairs. However, a dome open event without a later close will be returned as simply a dome open.

Parameters:

with_sunset_sunrise (bool, default: True)

rubin_nights.observatory_status.mtm1m3_slewflag_times(t_start, t_end, efd_client)[source]#

Dataframe containing slew times calculated from the mtm1m3 clear/set SlewFlags, and linked to groupId using nextVisit.

Parameters:

  • t_start (Time) – Time of the start of the events.

  • t_end (Time) – Time of the end of the events.

  • efd_client (InfluxQueryClient) – Sync EFD client.

Returns:

mt_slews – Dataframe containing groupId, scriptSalIndex, and mt_slew_time.

Return type:

pd.DataFrame

rubin_nights.targets_and_visits.targets_and_visits(t_start, t_end, endpoints, queue_index=1)[source]#

Dataframe showing linked Targets, Observations, NextVisits and Visits.

Parameters:

  • t_start (Time) – Time of the start of the events.

  • t_end (Time) – Time of the end of the events.

  • endpoints (dict) – Endpoints is a dictionary of client connections to the EFD and the ConsDb, such as returned by rubin_nights.connections.get_clients.

  • queueIndex – The SalIndex to query for Targets, corresponding to the Scheduler queue. Default of 1 corresponds to the Simonyi queue. Using queueIndex = 2 will trigger a request for latiss visits.

  • queue_index (int, default: 1)

Return type:

tuple[DataFrame, list[str], DataFrame, DataFrame, DataFrame]

Returns:

  • targets_and_visits (pd.DataFrame) – A Dataframe of Target, Observation, NextVisit and Visits.

  • cols (list [str]) – The short-list of columns for display in the table.

  • target_and_observations (pd.DataFrame) – A Dataframe of Targets joined to Observations.

  • nextvisit_and_visits (pd.DataFrame) – A Dataframe of nextVisits joined to Visits.

  • visits (pd.DataFrame) – A dataframe of all of the visits during the time period.