{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "### A demonstration of sims_movingObjects ###\n", "\n", "This notebook is also available in the [sims_movingObjects](https://github.com/lsst/sims_movingObjects) github repository, in the examples directory:\n", "https://github.com/lsst/sims_movingObjects/blob/master/examples/Demo%20sims_movingObjects.ipynb" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import lsst.sims.movingObjects as mo\n", "import lsst.sims.maf.db as db\n", "from lsst.sims.maf.batches import getColMap" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For most typical use of sims_movingObjects, you just need to use the script 'makeLSSTobs.py' :\n", "\n", "```\n", "(lsst-scipipe) [ewok:~] lynnej% makeLSSTobs.py --help\n", "\n", "usage: makeLSSTobs.py [-h] [--opsimDb OPSIMDB] [--orbitFile ORBITFILE]\n", " [--outDir OUTDIR] [--obsFile OBSFILE]\n", " [--sqlConstraint SQLCONSTRAINT]\n", " [--obsMetadata OBSMETADATA] [--footprint FOOTPRINT]\n", " [--rFov RFOV] [--xTol XTOL] [--yTol YTOL]\n", " [--roughTol ROUGHTOL] [--obsType OBSTYPE]\n", " [--obsCode OBSCODE] [--tStep TSTEP] [--ephMode EPHMODE]\n", " [--prelimEphMode PRELIMEPHMODE] [--ephType EPHTYPE]\n", "\n", "Generate moving object detections.\n", "\n", "optional arguments:\n", " -h, --help show this help message and exit\n", " --opsimDb OPSIMDB Opsim output db file (example: kraken_2026.db).\n", " Default None.\n", " --orbitFile ORBITFILE\n", " File containing the moving object orbits. See\n", " https://github.com/lsst/oorb/blob/lsst-\n", " dev/python/README.rst for additional documentation on\n", " the orbit file format. Default None.\n", " --outDir OUTDIR Output directory for moving object detections. Default\n", " '.'\n", " --obsFile OBSFILE Output file name for moving object observations.\n", " Default will build outDir/opsimRun_orbitFile_obs.txt.\n", " --sqlConstraint SQLCONSTRAINT\n", " SQL constraint to use to select data from opsimDb.\n", " Default no constraint.\n", " --obsMetadata OBSMETADATA\n", " Additional metadata to write into output file. The\n", " default metadata will combine the opsimDb name, the\n", " sqlconstraint, and the name of the orbit file;\n", " obsMetadata is an optional addition.\n", " --footprint FOOTPRINT\n", " Type of footprint to use to identify observations of\n", " each object. Options are 'circle', 'rectangle', or\n", " 'camera' (apply camera footprint). Default is 'circle'\n", " (which will then have a 1.75 deg radius).\n", " --rFov RFOV If using a circular footprint, this is the radius of\n", " the FOV (in degrees). Default 1.75 degrees.\n", " --xTol XTOL If using a rectangular footprint, this is the\n", " tolerance in the RA direction (in degrees). Default is\n", " 5 degrees.\n", " --yTol YTOL If using a rectangular footprint, this is the\n", " tolerance in the Dec direction (in degrees). Default\n", " is 3 degrees.\n", " --roughTol ROUGHTOL If using direct/exact ephemeris generation, this is\n", " the tolerance for the preliminary matches between\n", " ephemerides and pointings (in degrees). Default 20\n", " degrees.\n", " --obsType OBSTYPE Method for generating observations: 'direct' or\n", " 'linear'. Linear will use linear interpolation between\n", " a grid of ephemeris points. Direct will first generate\n", " rough ephemerides, look for observations within\n", " roughTol of these points, and then generate exact\n", " ephemerides at those times. Default 'direct'.\n", " --obsCode OBSCODE Observatory code for generating observations. Default\n", " is I11 (Cerro Pachon).\n", " --tStep TSTEP Timestep between ephemeris generation for either the\n", " first (rough) stage of direct ephemeris generation or\n", " the grid for linear interpolation ephemerides. Default\n", " 1 day.\n", " --ephMode EPHMODE 2body or nbody mode for ephemeris generation. Default\n", " is nbody.\n", " --prelimEphMode PRELIMEPHMODE\n", " Use either 2body or nbody for preliminary ephemeris\n", " generation in the rough stage for DirectObs. Default\n", " 2body.\n", " --ephType EPHTYPE Generate either 'basic' or 'full' ephemerides from\n", " OOrb. See https://github.com/lsst/oorb/blob/lsst-\n", " dev/python/README.rst for detailsof the contents of\n", " 'full' or 'basic' ephemerides. Default basic.\n", "```\n", "\n", "This runs the steps below, as a whole, to produce an output file with the observations.\n", "\n", "\n", "But let's break down a little bit of what's happening behind the scenes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Reading the orbit file." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# First the orbit file - this is read into an Orbits object.\n", "orbitDir = '.'\n", "orbitfile = 'neos_100.s3m'\n", "\n", "orbits = mo.Orbits()\n", "orbits.readOrbits(os.path.join(orbitDir, orbitfile))" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "COM ['objId', 'q', 'e', 'inc', 'Omega', 'argPeri', 'tPeri', 'epoch', 'H', 'g', 'sed_filename']\n", "KEP ['objId', 'a', 'e', 'inc', 'Omega', 'argPeri', 'meanAnomaly', 'epoch', 'H', 'g', 'sed_filename']\n", "CART ['objId', 'x', 'y', 'z', 'xdot', 'ydot', 'zdot', 'epoch', 'H', 'g', 'sed_filename']\n" ] } ], "source": [ "# The actual orbits are checked to see if they contain required columns: \n", "for orbit_type in orbits.dataCols:\n", " print(orbit_type, orbits.dataCols[orbit_type])" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on method assignSed in module lsst.sims.movingObjects.orbits:\n", "\n", "assignSed(orbits, randomSeed=None) method of lsst.sims.movingObjects.orbits.Orbits instance\n", " Assign either a C or S type SED, depending on the semi-major axis of the object.\n", " P(C type) = 0 (a<2); 0.5*a - 1 (2 4),\n", " based on figure 23 from Ivezic et al 2001 (AJ, 122, 2749).\n", " \n", " Parameters\n", " ----------\n", " orbits : pandas.DataFrame, pandas.Series or numpy.ndarray\n", " Array-like object containing orbital parameter information.\n", " \n", " Returns\n", " -------\n", " numpy.ndarray\n", " Array containing the SED type for each object in 'orbits'.\n", "\n" ] } ], "source": [ "# and if 'standard but not quite the same' orbit column names are used, \n", "# the column names are swapped to these standard names. \n", "\n", "# Note that columns 'objId' = anything (including strings), and columns H / g / sed_filename are not required.\n", "# If H is not specified, the value of 20 will be used\n", "# If g is not specified, the value of 0.15 will be used\n", "# If sed_filename is not specified (it should match a filename in $SIMS_MOVINGOBJECTS_DIR/data), one will be assigned.\n", "\n", "help(orbits.assignSed)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objIdotypeqeincOmegaargPeritPeriHgepochmodelsed_filename
0S1000000S3M3.018220.0520822.56035211.00286335.4213457801.6449314.200.1557231.0V150728C.dat
1S1000001S3M2.109740.075184.91571209.40298322.6644756722.7877920.570.1557231.0V150728S.dat
2S1000002S3M2.805230.077771.24945112.52284139.8685856406.4430414.650.1557231.0V150728S.dat
3S1000003S3M2.109170.132191.46615266.54621232.2441256980.1167619.580.1557231.0V150728S.dat
4S1000004S3M2.176760.1994912.92422162.14580192.2231256808.9025210.560.1557231.0V150728C.dat
\n", "
" ], "text/plain": [ " objId otype q e inc Omega argPeri \\\n", "0 S1000000 S3M 3.01822 0.05208 22.56035 211.00286 335.42134 \n", "1 S1000001 S3M 2.10974 0.07518 4.91571 209.40298 322.66447 \n", "2 S1000002 S3M 2.80523 0.07777 1.24945 112.52284 139.86858 \n", "3 S1000003 S3M 2.10917 0.13219 1.46615 266.54621 232.24412 \n", "4 S1000004 S3M 2.17676 0.19949 12.92422 162.14580 192.22312 \n", "\n", " tPeri H g epoch model sed_filename \n", "0 57801.64493 14.20 0.15 57231.0 V150728 C.dat \n", "1 56722.78779 20.57 0.15 57231.0 V150728 S.dat \n", "2 56406.44304 14.65 0.15 57231.0 V150728 S.dat \n", "3 56980.11676 19.58 0.15 57231.0 V150728 S.dat \n", "4 56808.90252 10.56 0.15 57231.0 V150728 C.dat " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# The final orbit data is kept in a class attribute: orbits: \n", "# note that additional data in the orbit file (such as otype) is retained.\n", "orbits.orbits[0:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Connecting to (and reading from) the opsim data file." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# Look at the MAF documentation for more information on the lsst.sims.maf.db.OpsimDatabase class and methods.\n", "opsimDir = '/Users/lynnej/opsim/db'\n", "opsimrun = 'baseline2018b'\n", "opsdb = db.OpsimDatabase(os.path.join(opsimDir, opsimrun + '.db'))\n", "colmap = getColMap(opsdb)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['observationStartMJD', 'night', 'fieldRA', 'fieldDec', 'rotSkyPos', 'filter', 'visitExposureTime', 'seeingFwhmEff', 'seeingFwhmGeom', 'fiveSigmaDepth', 'solarElong']\n" ] } ], "source": [ "# Basically, we will just read all of these requested columns from the database: (for now we'll stick to night < 100)\n", "reqcols = [colmap['mjd'], colmap['night'], colmap['ra'], colmap['dec'],\n", " 'rotSkyPos', colmap['filter'], colmap['exptime'], colmap['seeingEff'],\n", " colmap['seeingGeom'], colmap['fiveSigmaDepth'], 'solarElong']\n", "degreesIn = colmap['raDecDeg']\n", "print(reqcols)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "simdata = opsdb.fetchMetricData(reqcols, sqlconstraint='night < 100')" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
observationStartMJDnightfieldRAfieldDecrotSkyPosfiltervisitExposureTimeseeingFwhmEffseeingFwhmGeomfiveSigmaDepthsolarElong
059853.0167941305.088793-24.889283180.000000z30.00.6920290.62084823.348066113.652908
159853.0180211304.77069435.966846180.786087z30.01.1821041.02369022.308947113.957998
259853.0201851305.482200-62.802603181.134222z30.00.7663030.68190123.17882799.684211
\n", "
" ], "text/plain": [ " observationStartMJD night fieldRA fieldDec rotSkyPos filter \\\n", "0 59853.016794 1 305.088793 -24.889283 180.000000 z \n", "1 59853.018021 1 304.770694 35.966846 180.786087 z \n", "2 59853.020185 1 305.482200 -62.802603 181.134222 z \n", "\n", " visitExposureTime seeingFwhmEff seeingFwhmGeom fiveSigmaDepth \\\n", "0 30.0 0.692029 0.620848 23.348066 \n", "1 30.0 1.182104 1.023690 22.308947 \n", "2 30.0 0.766303 0.681901 23.178827 \n", "\n", " solarElong \n", "0 113.652908 \n", "1 113.957998 \n", "2 99.684211 " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Simdata is a numpy recarray - let's just use pandas to make it easier to read here.\n", "pd.DataFrame(simdata[0:3])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Generating observations. \n", "\n", "There is more than one way to actually generate observations, but let's look at the 'direct' approach. This is implemented in the class \"DirectObs\", which provides higher level methods that basically do the following: \n", "\n", "* generate (rough) ephemerides between the start and end of the survey data, on stepsizes of tStep (typically, 1 day)\n", "* identify pointings which come within roughTol of these rough ephemerides\n", "* generate exact ephemerides at the times of those pointings, and test if they fall within 'footprint'\n", "* 'footprint' can be circle, rectangle or camera -- 'circle' just checks if an object falls within rFov of the center of the pointing, 'rectangle' checks if RA x cos(dec) is within xTol and Dec within yTol of the pointing center, and camera applies the actual LSST camera footprint to check if an object falls on active silicon.\n", "* for the pointings where the object was observed, then calculate trailing losses and write the output to disk (including a color term, appropriate for the filter of each observation).\n", "\n", "(Note: there's plenty of room to improve on the procedure above, which could both speed things up and improve accuracy .. if this is something you're interested in contributing to, please consider writing a new Obs class, preferably inheriting from the BaseObs class).\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Using the \"DirectObs\" class.\n", "rFov = 1.75 # in degrees\n", "obscode = 'I11'\n", "footprint = 'circle' # circle, rectangle or camera\n", "ephMode = 'nbody' # nbody or 2body\n", "ephType = 'basic' # basic (subset of ephemeride values - no helio, topo, etc.) or full (all that oorb calculates)\n", "tStep = 1.0\n", "roughTol = 10.0\n", "obsFile = '_'.join([opsimrun, orbitfile.replace('.des', '').replace('.s3m', ''), 'obs.txt'])\n", "obs = mo.DirectObs(footprint, rFov=rFov, tstep=tStep, roughTol=roughTol, \n", " obsCode=obscode, obsTimeScale='TAI', ephMode=ephMode, ephType=ephType,\n", " obsTimeCol=colmap['mjd'], obsRA=colmap['ra'], obsDec=colmap['dec'], \n", " seeingCol=colmap['seeingGeom'], visitExpTimeCol=colmap['exptime'],\n", " outfileName=obsFile, obsMetadata='%s test (night<100)' % (opsimrun))" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'C.dat': {'g': 0.28041793225569478,\n", " 'i': -0.2926196728132453,\n", " 'r': -0.177293845688272,\n", " 'u': 1.5442111906705982,\n", " 'y': -0.3028771664514025,\n", " 'z': -0.29808899160965296},\n", " 'S.dat': {'g': 0.36850606460945201,\n", " 'i': -0.45720772665728049,\n", " 'r': -0.26270911487248227,\n", " 'u': 1.8626393418302385,\n", " 'y': -0.4073469917610737,\n", " 'z': -0.39986031775197262}}" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Read the filter curves (to calculate the colors for each object)\n", "filterlist = np.unique(simdata[colmap['filter']])\n", "obs.readFilters(filterlist=filterlist)\n", "# Calculate all colors ahead of time - this is just to create a dictionary early.\n", "sednames = np.unique(orbits.orbits['sed_filename'])\n", "for sedname in sednames:\n", " obs.calcColors(sedname)\n", "# Want to see the lsst colors? (these are all x-V band colors)\n", "obs.colors" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Will write output to baseline2018b_neos_100_obs.txt\n", "Generating preliminary ephemerides on a grid of 1.000000 day timesteps.\n" ] } ], "source": [ "# Generate the actual observations and write to output file.\n", "print('Will write output to %s' % obs.outfileName)\n", "obs.run(orbits, simdata)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# baseline2018b test (night<100)\r\n", "# baseline2018b_neos_100_obs.txt\r\n", "# ephemeris generation via PyOrbEphemerides\r\n", "# planetary ephemeris file /Users/lynnej/lsstRepos/oorb/data/de430.dat \r\n", "# obscode I11\r\n", "# direct obs header metadata\r\n", "# observation generation via DirectObs\r\n", "# ephMode nbody prelimEphMode 2body\r\n", "# rough tolerance for preliminary match 10.000000\r\n", "# time step for preliminary match 1.000000\r\n", "# pointing footprint circle\r\n", "# rfov 1.750000\r\n", "# obsRA fieldRA obsDec fieldDec obsRotSkyPos rotSkyPos obsDeg True\r\n", "# obsMJD observationStartMJD obsTimeScale TAI seeing seeingFwhmGeom expTime visitExposureTime\r\n", "objId time ra dec dradt ddecdt phase solarelon helio_dist geo_dist magV trueAnomaly velocity observationStartMJD night fieldRA fieldDec rotSkyPos filter visitExposureTime seeingFwhmEff seeingFwhmGeom fiveSigmaDepth solarElong magFilter dmagColor dmagTrail dmagDetect \r\n", "S1000001 59854.0760648 335.661572918 -4.66872150058 -0.127477289493 -0.0991730463403 12.9028682217 146.83925488 2.45213370361 1.55215427431 21.6707534809 -99.0 12.9032493442 59854.0760648 2 335.151362 -5.689996 228.530394116 y 30.0 0.829128431145 0.733543570401 22.0670183628 145.792553804 21.2634064892 -0.407346991761 1.46493584452 2.8526839965 \r\n" ] } ], "source": [ "# This is what the output file looks like: \n", "# it contains some metadata about how it was generated, and then the observations.\n", "!head -16 $obsFile" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Using the \"DirectObs\" class.\n", "rFov = 1.75 # in degrees\n", "obscode = 'I11'\n", "footprint = 'circle' # circle, rectangle or camera\n", "ephMode = 'nbody' # nbody or 2body\n", "ephType = 'full' # basic (subset of ephemeride values - no helio, topo, etc.) or full (all that oorb calculates)\n", "tStep = 1.0\n", "roughTol = 10.0\n", "obsFile = '_'.join([opsimrun, orbitfile.replace('.des', '').replace('.s3m', ''), 'full_obs.txt'])\n", "obs = mo.DirectObs(footprint, rFov=rFov, tstep=tStep, roughTol=roughTol, \n", " obsCode=obscode, obsTimeScale='TAI', ephMode=ephMode, ephType=ephType,\n", " obsTimeCol=colmap['mjd'], obsRA=colmap['ra'], obsDec=colmap['dec'], \n", " seeingCol=colmap['seeingGeom'], visitExpTimeCol=colmap['exptime'],\n", " outfileName=obsFile, obsMetadata='%s test (night<100)' % (opsimrun))" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# Read the filter curves (to calculate the colors for each object)\n", "filterlist = np.unique(simdata[colmap['filter']])\n", "obs.readFilters(filterlist=filterlist)\n", "# Calculate all colors ahead of time - this is just to create a dictionary early.\n", "sednames = np.unique(orbits.orbits['sed_filename'])\n", "for sedname in sednames:\n", " obs.calcColors(sedname)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Will write output to baseline2018b_neos_100_full_obs.txt\n", "Generating preliminary ephemerides on a grid of 1.000000 day timesteps.\n" ] } ], "source": [ "# Generate the actual observations and write to output file.\n", "print('Will write output to %s' % obs.outfileName)\n", "obs.run(orbits, simdata)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# baseline2018b test (night<100)\r\n", "# baseline2018b_neos_100_full_obs.txt\r\n", "# ephemeris generation via PyOrbEphemerides\r\n", "# planetary ephemeris file /Users/lynnej/lsstRepos/oorb/data/de430.dat \r\n", "# obscode I11\r\n", "# direct obs header metadata\r\n", "# observation generation via DirectObs\r\n", "# ephMode nbody prelimEphMode 2body\r\n", "# rough tolerance for preliminary match 10.000000\r\n", "# time step for preliminary match 1.000000\r\n", "# pointing footprint circle\r\n", "# rfov 1.750000\r\n", "# obsRA fieldRA obsDec fieldDec obsRotSkyPos rotSkyPos obsDeg True\r\n", "# obsMJD observationStartMJD obsTimeScale TAI seeing seeingFwhmGeom expTime visitExposureTime\r\n", "objId time ra dec dradt ddecdt phase solarelon helio_dist geo_dist magV pa topo_lon topo_lat opp_topo_lon opp_topo_lat helio_lon helio_lat opp_helio_lon opp_helio_lat topo_obj_alt topo_solar_alt topo_lunar_alt lunar_phase lunar_dist helio_x helio_y helio_z helio_dx helio_dy helio_dz obs_helio_x obs_helio_y obs_helio_z trueAnom velocity observationStartMJD night fieldRA fieldDec rotSkyPos filter visitExposureTime seeingFwhmEff seeingFwhmGeom fiveSigmaDepth solarElong magFilter dmagColor dmagTrail dmagDetect \r\n", "S1000001 59854.0760648 335.661572918 -4.66872150058 -0.127477289493 -0.0991730463403 12.9028682217 146.840912582 2.45213370361 1.55215427431 21.6707534809 232.118283673 335.741135515 5.08952875392 -32.810127053 5.09083443774 348.541969683 3.21849979771 -20.0092928851 3.21980548153 63.1861602278 -38.4665724493 36.1010079827 0.38739595815 70.6520986767 2.3994731612 -0.486348214868 0.137672302874 0.00218415563495 0.0103172709944 -0.000681661036841 0.989955765886 0.148855498807 -2.28131980833e-05 -99.0 12.9032493442 59854.0760648 2 335.151362 -5.689996 228.530394116 y 30.0 0.829128431145 0.733543570401 22.0670183628 145.792553804 21.2634064892 -0.407346991761 1.46493584452 2.8526839965 \r\n" ] } ], "source": [ "# This is what the output file looks like:\n", "!head -16 $obsFile" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objIdtimeradecdradtddecdtphasesolarelonhelio_distgeo_dist...filtervisitExposureTimeseeingFwhmEffseeingFwhmGeomfiveSigmaDepthsolarElongmagFilterdmagColordmagTraildmagDetect
0S100000159854.076065335.661573-4.668722-0.127477-0.09917312.902868146.8409132.4521341.552154...y30.00.8291280.73354422.067018145.79255421.263406-0.4073471.4649362.852684
1S100000159854.083738335.660591-4.669482-0.127479-0.09912312.905885146.8322612.4521341.552209...y30.00.7779210.69145122.123200147.79275921.263484-0.4073471.4961392.914742
2S100000159856.017419335.442640-4.856711-0.113656-0.09487213.637453144.6978152.4522291.566566...z30.01.0151720.88647122.712273143.89914721.290969-0.3998601.3947072.712270
3S100000159856.038565335.440224-4.858716-0.113971-0.09473513.645485144.6742722.4522301.566726...z30.00.7866760.69864823.004635143.87836221.291190-0.3998601.5200392.962138
4S100000159861.079375334.989973-5.303635-0.079843-0.08181315.435813139.2046842.4524131.607632...z30.00.6997990.62723422.641844138.94587821.347170-0.3998601.6422813.202670
\n", "

5 rows × 51 columns

\n", "
" ], "text/plain": [ " objId time ra dec dradt ddecdt \\\n", "0 S1000001 59854.076065 335.661573 -4.668722 -0.127477 -0.099173 \n", "1 S1000001 59854.083738 335.660591 -4.669482 -0.127479 -0.099123 \n", "2 S1000001 59856.017419 335.442640 -4.856711 -0.113656 -0.094872 \n", "3 S1000001 59856.038565 335.440224 -4.858716 -0.113971 -0.094735 \n", "4 S1000001 59861.079375 334.989973 -5.303635 -0.079843 -0.081813 \n", "\n", " phase solarelon helio_dist geo_dist ... filter \\\n", "0 12.902868 146.840913 2.452134 1.552154 ... y \n", "1 12.905885 146.832261 2.452134 1.552209 ... y \n", "2 13.637453 144.697815 2.452229 1.566566 ... z \n", "3 13.645485 144.674272 2.452230 1.566726 ... z \n", "4 15.435813 139.204684 2.452413 1.607632 ... z \n", "\n", " visitExposureTime seeingFwhmEff seeingFwhmGeom fiveSigmaDepth \\\n", "0 30.0 0.829128 0.733544 22.067018 \n", "1 30.0 0.777921 0.691451 22.123200 \n", "2 30.0 1.015172 0.886471 22.712273 \n", "3 30.0 0.786676 0.698648 23.004635 \n", "4 30.0 0.699799 0.627234 22.641844 \n", "\n", " solarElong magFilter dmagColor dmagTrail dmagDetect \n", "0 145.792554 21.263406 -0.407347 1.464936 2.852684 \n", "1 147.792759 21.263484 -0.407347 1.496139 2.914742 \n", "2 143.899147 21.290969 -0.399860 1.394707 2.712270 \n", "3 143.878362 21.291190 -0.399860 1.520039 2.962138 \n", "4 138.945878 21.347170 -0.399860 1.642281 3.202670 \n", "\n", "[5 rows x 51 columns]" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Or nicely read into a pandas dataframe..\n", "d = pd.read_csv(obsFile, delim_whitespace=True, comment='#')\n", "d[0:5]" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objIdmagVdmagColormagFilterfilterseeingFwhmGeomdradtddecdtdmagDetectappMagfiveSigmaDepth
23S100000122.114624-0.39986021.714764z0.8853940.108423-0.0029843.21510024.92986422.400191
24S100000122.114850-0.39986021.714990z0.9364050.108590-0.0028603.15709124.87208122.395591
25S100000122.142264-0.39986021.742403z0.7141360.1201330.0024693.44932525.19172822.631871
26S100000122.142476-0.39986021.742615z0.6285920.1203050.0025833.58005025.32266622.909896
27S100000412.801211-0.30287712.498334y0.7876450.339752-0.0974343.35201415.85034821.768465
28S10000059.375908-0.3028779.073030y0.7174910.159049-0.2509013.66120612.73423621.991805
29S10000059.375825-0.3028779.072947y0.6374360.158952-0.2507973.78099212.85393921.993364
30S10000059.2622231.54421110.806434u0.6268730.110650-0.2353043.77945514.58588923.613301
31S10000059.2479781.54421110.792189u0.5602440.104324-0.2328913.88949614.68168523.786732
32S10000059.176317-0.3028778.873440y0.7189280.071651-0.2182663.62019112.49363122.121915
\n", "
" ], "text/plain": [ " objId magV dmagColor magFilter filter seeingFwhmGeom \\\n", "23 S1000001 22.114624 -0.399860 21.714764 z 0.885394 \n", "24 S1000001 22.114850 -0.399860 21.714990 z 0.936405 \n", "25 S1000001 22.142264 -0.399860 21.742403 z 0.714136 \n", "26 S1000001 22.142476 -0.399860 21.742615 z 0.628592 \n", "27 S1000004 12.801211 -0.302877 12.498334 y 0.787645 \n", "28 S1000005 9.375908 -0.302877 9.073030 y 0.717491 \n", "29 S1000005 9.375825 -0.302877 9.072947 y 0.637436 \n", "30 S1000005 9.262223 1.544211 10.806434 u 0.626873 \n", "31 S1000005 9.247978 1.544211 10.792189 u 0.560244 \n", "32 S1000005 9.176317 -0.302877 8.873440 y 0.718928 \n", "\n", " dradt ddecdt dmagDetect appMag fiveSigmaDepth \n", "23 0.108423 -0.002984 3.215100 24.929864 22.400191 \n", "24 0.108590 -0.002860 3.157091 24.872081 22.395591 \n", "25 0.120133 0.002469 3.449325 25.191728 22.631871 \n", "26 0.120305 0.002583 3.580050 25.322666 22.909896 \n", "27 0.339752 -0.097434 3.352014 15.850348 21.768465 \n", "28 0.159049 -0.250901 3.661206 12.734236 21.991805 \n", "29 0.158952 -0.250797 3.780992 12.853939 21.993364 \n", "30 0.110650 -0.235304 3.779455 14.585889 23.613301 \n", "31 0.104324 -0.232891 3.889496 14.681685 23.786732 \n", "32 0.071651 -0.218266 3.620191 12.493631 22.121915 " ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Note that the apparent magnitude in any given visit will be generated by MAF, or can be calculated as:\n", "d['appMag'] = d['magV'] + d['dmagColor'] + d['dmagDetect']\n", "cols = ['objId', 'magV', 'dmagColor', 'magFilter', 'filter', \n", " 'seeingFwhmGeom', 'dradt', 'ddecdt', 'dmagDetect', \n", " 'appMag', 'fiveSigmaDepth']\n", "tmp = d[cols][23:33]\n", "tmp" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
objIdotypeqeincOmegaargPeritPeriHgepochmodelsed_filename
1S1000001S3M2.109740.075184.91571209.40298322.6644756722.7877920.570.1557231.0V150728S.dat
4S1000004S3M2.176760.1994912.92422162.14580192.2231256808.9025210.560.1557231.0V150728C.dat
5S1000005S3M1.982620.1632423.42404210.02222326.1851457395.295928.170.1557231.0V150728C.dat
\n", "
" ], "text/plain": [ " objId otype q e inc Omega argPeri \\\n", "1 S1000001 S3M 2.10974 0.07518 4.91571 209.40298 322.66447 \n", "4 S1000004 S3M 2.17676 0.19949 12.92422 162.14580 192.22312 \n", "5 S1000005 S3M 1.98262 0.16324 23.42404 210.02222 326.18514 \n", "\n", " tPeri H g epoch model sed_filename \n", "1 56722.78779 20.57 0.15 57231.0 V150728 S.dat \n", "4 56808.90252 10.56 0.15 57231.0 V150728 C.dat \n", "5 57395.29592 8.17 0.15 57231.0 V150728 C.dat " ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "orbits.orbits.query('objId in @tmp.objId')" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" } }, "nbformat": 4, "nbformat_minor": 2 }