U e{jO@sdZddlZddlZddlZddlmZddlZddlm Z ddl m Z ddl m Z mZd dd Zd!d d Zd d Zd"ddZd#ddZddZd$ddZd%ddZGdddejZGddde jZGddde jZGddde jZdS)&z This package contains functions for reading and writing QDP tables that are not meant to be used directly, but instead are available as readers/writers in `astropy.table`. See :ref:`astropy:table_io` for more details. N)Iterable)Table)AstropyUserWarning)basiccorec Csd}d}|}|dkrd}d|d}d|d|d|d }d |d |d |d }t|}|}|sndS||} | dkrtd|| D]<\} } | dkrq| dkrdt| j|dS| SqdS)aInterpret a QDP file line Parameters ---------- line : str a single line of the file Returns ------- type : str Line type: "comment", "command", or "data" Examples -------- >>> _line_type("READ SERR 3") 'command' >>> _line_type(" \n !some gibberish") 'comment' >>> _line_type(" ") 'comment' >>> _line_type(" 21345.45") 'data,1' >>> _line_type(" 21345.45 1.53e-3 1e-3 .04 NO nan") 'data,6' >>> _line_type(" 21345.45,1.53e-3,1e-3,.04,NO,nan", delimiter=',') 'data,6' >>> _line_type(" 21345.45 ! a comment to disturb") 'data,1' >>> _line_type("NO NO NO NO NO") 'new' >>> _line_type("NO,NO,NO,NO,NO", delimiter=',') 'new' >>> _line_type("N O N NOON OON O") Traceback (most recent call last): ... ValueError: Unrecognized QDP line... >>> _line_type(" some non-comment gibberish") Traceback (most recent call last): ... ValueError: Unrecognized QDP line... z'[+-]?(\d+(\.\d*)?|\.\d+)([eE][+-]?\d+)?zREAD [TS]ERR(\s+[0-9]+)+Nz\s+zNO(zNO)+(z|NO|[-+]?nan)(z|NO|[-+]?nan))*)z^\s*((?Pz )|(?Pz )|(?Pz)?\s*(\!(?P.*))?\s*$commentzUnrecognized QDP line: datadata,)sep) recompilestripmatch ValueError groupdictitemslensplit) line delimiterZ _decimal_reZ _command_rer Z_new_reZ_data_reZ_type_reZ _line_type_rertype_valrS/var/www/html/bwave/python/venv/lib/python3.8/site-packages/astropy/io/ascii/qdp.py _line_types**   rcs`fdd|D}d}|D]<}|drt|dd}|dkrF|}q||krtdq||fS)a:Read through the list of QDP file lines and label each line by type Parameters ---------- lines : list List containing one file line in each entry Returns ------- contents : list List containing the type for each line (see `line_type_and_data`) ncol : int The number of columns in the data lines. Must be the same throughout the file Examples -------- >>> line0 = "! A comment" >>> line1 = "543 12 456.0" >>> lines = [line0, line1] >>> types, ncol = _get_type_from_list_of_lines(lines) >>> types[0] 'comment' >>> types[1] 'data,3' >>> ncol 3 >>> lines.append("23") >>> _get_type_from_list_of_lines(lines) Traceback (most recent call last): ... ValueError: Inconsistent number of columns csg|]}t|dqS)r)r.0rrrr {sz0_get_type_from_list_of_lines..Nr zInconsistent number of columns) startswithintr)linesrtypesZ current_ncolrncolrrr_get_type_from_list_of_linesXs#  r'c Csbd|kr|d}nJt|trFt|}dd|D}W5QRXnt|trV|}ntd|S)N cSsg|] }|qSr)rrrrrr sz(_get_lines_from_file..zinvalid value of qdb_file)r isinstancestropen readlinesrr)qdp_filer$fobjrrr_get_lines_from_files    r/c CsFddt|D}|dkr$g}}n"t|}|dg}|dg}|dk rzt|t|d}|t||krztdd}t|D]}|d |} ||d krqd | } |dk r|| d } | ||<| |kr| d ||d <|d 7}q| |kr| d ||d <| d||d<|d7}qqtdd|DrBt|S)aGive list of column names from the READ SERR and TERR commands Parameters ---------- err_specs : dict ``{'serr': [n0, n1, ...], 'terr': [n2, n3, ...]}`` Error specifications for symmetric and two-sided errors ncols : int Number of data columns Other Parameters ---------------- names : list of str Name of data columns (defaults to ['col1', 'col2', ...]), _not_ including error columns. Returns ------- colnames : list List containing the column names. Error columns will have the name of the main column plus ``_err`` for symmetric errors, and ``_perr`` and ``_nerr`` for positive and negative errors respectively Examples -------- >>> col_in = ['MJD', 'Rate'] >>> cols = _interpret_err_lines(None, 2, names=col_in) >>> cols[0] 'MJD' >>> err_specs = {'terr': [1], 'serr': [2]} >>> ncols = 5 >>> cols = _interpret_err_lines(err_specs, ncols, names=col_in) >>> cols[0] 'MJD' >>> cols[2] 'MJD_nerr' >>> cols[4] 'Rate_err' >>> _interpret_err_lines(err_specs, 6, names=col_in) Traceback (most recent call last): ... ValueError: Inconsistent number of input colnames cSsg|]}dqSr)rirrrr sz(_interpret_err_lines..Nserrterrz%Inconsistent number of input colnamesrrr1col_err_perr_nerrcSsg|] }|dkqSr0rrcrrrr s) rangecopydeepcopypoprrnpanyAssertionError) err_specsZncolsnamescolnames serr_cols terr_colsZall_error_colsshiftr2Zcol_numZ colname_rootrrr_interpret_err_liness<-          rIc CsVt|}t||d\}}g}i}d}d} d} d} d} t||D]\} }| d} |dkrl| | d7} q>|dkr| dkr| } d} |ikrtdt| | d7} q>|d r|ikr| dkr|  dD]@}| }t |d krqd d |d dD||d <q|dkr.t |||d}| dkr|dkr>| dk r>t|| d}|  d|jd<|  d|jd<d} | |d} q>q>| dk rRt|| d}|  d|jd<|  d|jd<| ||S)aGet all tables from a QDP file Parameters ---------- qdp_file : str Input QDP file name Other Parameters ---------------- input_colnames : list of str Name of data columns (defaults to ['col1', 'col2', ...]), _not_ including error columns. delimiter : str Delimiter for the values in the table. Returns ------- list of `~astropy.table.Table` List containing all the tables present inside the QDP file rNr1!r r(commandz9This file contains multiple command blocks. Please verifyr cSsg|] }t|qSr)r#r:rrrr /sz-_get_tables_from_qdp_file..r5r)rDNOnew)rDrowsinitial_commentscomments)r/r'ziprlstripwarningswarnrr"rrlowerrIappendr@mamaskedr#rfloatrmeta)r-input_colnamesrr$contentsr&Z table_listrCrEZ comment_textrPZ command_linesZ current_rowsrdatatypeZclinerKvaluesvZ new_tablerrr_get_tables_from_qdp_files~     $          racCsd}g}g}t|D]\}}|dr>||||d7}q|dr|||t||dksx||ddstd|d7}q|dr||ddstdq||fS) a,Get which column names are error columns Examples -------- >>> colnames = ['a', 'a_err', 'b', 'b_perr', 'b_nerr'] >>> serr, terr = _understand_err_col(colnames) >>> np.allclose(serr, [1]) True >>> np.allclose(terr, [2]) True >>> serr, terr = _understand_err_col(['a', 'a_nerr']) Traceback (most recent call last): ... ValueError: Missing positive error... >>> serr, terr = _understand_err_col(['a', 'a_perr']) Traceback (most recent call last): ... ValueError: Missing negative error... rr7rr8r9zMissing negative errorr5zMissing positive error) enumerateendswithrWrr)rErHr3r4r2r6rrr_understand_err_col\s   "  rdcCs.|dkrtdtd}t|||d}||S)aRead a table from a QDP file Parameters ---------- qdp_file : str Input QDP file name Other Parameters ---------------- names : list of str Name of data columns (defaults to ['col1', 'col2', ...]), _not_ including error columns. table_id : int, default 0 Number of the table to be read from the QDP file. This is useful when multiple tables present in the file. By default, the first is read. delimiter : str Any delimiter accepted by the `sep` argument of str.split() Returns ------- tables : list of `~astropy.table.Table` List containing all the tables present inside the QDP file Nz9table_id not specified. Reading the first available tabler)r\r)rTrUrra)r-rDtable_idrZtablesrrr_read_table_qdpsrfc Csddl}|}d|jkr^|jdgkr^|jdD]*}|}|dsPd|}t||dq2|dkrvt|j\}}n|dg}|dg}|gkrd d d |D}td ||d|gkrd d d |D}td ||dd|jkr@|jdgkr@|jdD].}|}|ds0d|}t||dq|j} tdd | |d|D]N} g} | D],} t j | st | } nd} | | qltd | |dq`|}||dk rt|d}t||dW5QRX|dS)aWrite a table to a QDP file Parameters ---------- table : :class:`~astropy.table.Table` Input table to be written filename : str Output QDP file name Other Parameters ---------------- err_specs : dict Dictionary of the format {'serr': [1], 'terr': [2, 3]}, specifying which columns have symmetric and two-sided errors (see QDP format specification) rNrPrJ)filer3r4 cSsg|] }t|qSrr*rrrrrr sz$_write_table_qdp..z READ SERR cSsg|] }t|qSrrirjrrrr sz READ TERR rQrMwr()ioStringIOr[rr"printrdrEr?joinr@rXZ is_maskedr*rWgetvaluecloser+r)tablefilenamerCrlr.rrFrGZ col_stringrErowr_rrepZ full_stringrrr_write_table_qdpsP       rvc@seZdZdZdZdS) QDPSplitterz' Split on space for QDP tables rhN)__name__ __module__ __qualname____doc__rrrrrrwsrwc@seZdZdZeZdZdZdS) QDPHeaderzJ Header that uses the :class:`astropy.io.ascii.basic.QDPSplitter` rJN)rxryrzr{rwsplitter_classr write_commentrrrrr|sr|c@s(eZdZdZeZejdfgZdZ dZ dS)QDPDatazH Data that uses the :class:`astropy.io.ascii.basic.CsvSplitter` rMrJN) rxryrzr{rwr}rrYZ fill_valuesr r~rrrrrs  rcsJeZdZdZdZdZdZdZeZ e Z d fdd Z d d Z d d ZZS)QDPaQuick and Dandy Plot table. Example:: ! Initial comment line 1 ! Initial comment line 2 READ TERR 1 READ SERR 3 ! Table 0 comment !a a(pos) a(neg) b be c d 53000.5 0.25 -0.5 1 1.5 3.5 2 54000.5 1.25 -1.5 2 2.5 4.5 3 NO NO NO NO NO ! Table 1 comment !a a(pos) a(neg) b be c d 54000.5 2.25 -2.5 NO 3.5 5.5 5 55000.5 3.25 -3.5 4 4.5 6.5 nan The input table above contains some initial comments, the error commands, then two tables. This file format can contain multiple tables, separated by a line full of ``NO``s. Comments are exclamation marks, and missing values are single ``NO`` entries. The delimiter is usually whitespace, more rarely a comma. The QDP format differentiates between data and error columns. The table above has commands:: READ TERR 1 READ SERR 3 which mean that after data column 1 there will be two error columns containing its positive and engative error bars, then data column 2 without error bars, then column 3, then a column with the symmetric error of column 3, then the remaining data columns. As explained below, table headers are highly inconsistent. Possible comments containing column names will be ignored and columns will be called ``col1``, ``col2``, etc. unless the user specifies their names with the ``names=`` keyword argument, When passing column names, pass **only the names of the data columns, not the error columns.** Error information will be encoded in the names of the table columns. (e.g. ``a_perr`` and ``a_nerr`` for the positive and negative error of column ``a``, ``b_err`` the symmetric error of column ``b``.) When writing tables to this format, users can pass an ``err_specs`` keyword passing a dictionary ``{'serr': [3], 'terr': [1, 2]}``, meaning that data columns 1 and two will have two additional columns each with their positive and negative errors, and data column 3 will have an additional column with a symmetric error (just like the ``READ SERR`` and ``READ TERR`` commands above) Headers are just comments, and tables distributed by various missions can differ greatly in their use of conventions. For example, light curves distributed by the Swift-Gehrels mission have an extra space in one header entry that makes the number of labels inconsistent with the number of cols. For this reason, we ignore the comments that might encode the column names and leave the name specification to the user. Example:: > Extra space > | > v >! MJD Err (pos) Err(neg) Rate Error >53000.123456 2.378e-05 -2.378472e-05 NO 0.212439 These readers and writer classes will strive to understand which of the comments belong to all the tables, and which ones to each single table. General comments will be stored in the ``initial_comments`` meta of each table. The comments of each table will be stored in the ``comments`` meta. Example:: t = Table.read(example_qdp, format='ascii.qdp', table_id=1, names=['a', 'b', 'c', 'd']) reads the second table (``table_id=1``) in file ``example.qdp`` containing the table above. There are four column names but seven data columns, why? Because the ``READ SERR`` and ``READ TERR`` commands say that there are three error columns. ``t.meta['initial_comments']`` will contain the initial two comment lines in the file, while ``t.meta['comments']`` will contain ``Table 1 comment`` The table can be written to another file, preserving the same information, as:: t.write(test_file, err_specs={'terr': [1], 'serr': [3]}) Note how the ``terr`` and ``serr`` commands are passed to the writer. ZqdpTz.qdpzQuick and Dandy PlotterNcs&t||_||_||_||_dS)N)super__init__rerDrCr)selfrerDrCr  __class__rrrrs  z QDP.__init__cCs*|jj|dd|_t|j|j|j|jdS)Nr()newline)rerDr)Zinputter get_linesr$rfrerDr)rrrrrrreadyszQDP.readcCs||t||jd}|S)N)rC)Z_check_multidim_tablervrC)rrrr$rrrwrites z QDP.write)NNNN)rxryrzr{Z _format_nameZ_io_registry_can_writeZ_io_registry_suffixZ _descriptionr|Z header_classrZ data_classrrr __classcell__rrrrrs[ r)N)N)N)NN)NNN)NN)r{r=r rTcollections.abcrnumpyr@Z astropy.tablerZastropy.utils.exceptionsrr1rrrr'r/rIrardrfrvZDefaultSplitterrwZCommentedHeaderHeaderr|Z BasicDatarZBasicrrrrrs(    D 0 Y m( ( E