diff --git a/docs/index.rst b/docs/index.rst index 14448315..2a0be46d 100755 --- a/docs/index.rst +++ b/docs/index.rst @@ -23,7 +23,7 @@ Intro Files ***** .. toctree:: - :caption: Reading Files + :caption: Reading :maxdepth: 2 src/Formats/timeseries.rst @@ -35,17 +35,26 @@ Files Web *** .. toctree:: - :caption: Downloading Data + :caption: Downloading :maxdepth: 2 src/Web/webclients.rst src/Web/seedlink.rst +******* +Writing +******* +.. toctree:: + :caption: Writing + :maxdepth: 2 + + src/writing.rst + ********** Processing ********** .. toctree:: - :caption: Data Processing + :caption: Processing :maxdepth: 2 src/Processing/processing.rst diff --git a/docs/src/Appendices/examples.rst b/docs/src/Appendices/examples.rst index 4351fbbe..1149d803 100644 --- a/docs/src/Appendices/examples.rst +++ b/docs/src/Appendices/examples.rst @@ -100,14 +100,14 @@ SeedLink sessions :: S = SeisData() - seedlink!(S, "SL.conf", mode="DATA", r=10, w=true) + seedlink!(S, "DATA", "SL.conf", r=10, w=true) 2. An unattended SeedLink download in TIME mode. Get the next two minutes of data from stations GPW, MBW, SHUK in the UW network. Put the Julia REPL to sleep while the request fills. If the connection is still open, close it (SeedLink's time bounds arent precise in TIME mode, so this may or may not be necessary). Pause briefly so that the last data packets are written. Synchronize results and write data in native SeisIO file format. :: sta = "UW.GPW,UW.MBW,UW.SHUK" s0 = now() - S = seedlink(sta, mode="TIME", s=s0, t=120, r=10) + S = seedlink("TIME", sta, s=s0, t=120, r=10) sleep(180) isopen(S.c[1]) && close(S.c[1]) sleep(20) @@ -119,7 +119,7 @@ SeedLink sessions :: sta = "UW.GPW, UW.MBW, UW.SHUK" - S1 = seedlink(sta, mode="TIME", s=0, t=120) + S1 = seedlink("TIME", sta, s=0, t=120) 4. A SeedLink session in DATA mode with multiple servers, including a config file. Data are parsed roughly every 10 seconds. A total of 5 minutes of data are requested. :: @@ -132,14 +132,14 @@ SeedLink sessions (d0,d1) = parsetimewin(st,en) S = SeisData() - seedlink!(S, sta, mode="TIME", r=10.0, s=d0, t=d1) + seedlink!(S, "DATA", sta, r=10.0) println(stdout, "...first link initialized...") # Seedlink with a config file config_file = "seedlink.conf" - seedlink!(S, config_file, r=10.0, mode="TIME", s=d0, t=d1) + seedlink!(S, "DATA", config_file, r=10.0) println(stdout, "...second link initialized...") # Seedlink with a config string - seedlink!(S, "CC.VALT..???, UW.ELK..EHZ", mode="TIME", r=10.0, s=d0, t=d1) + seedlink!(S, "DATA", "CC.VALT..???, UW.ELK..EHZ", r=10.0) println(stdout, "...third link initialized...") diff --git a/docs/src/Appendices/keywords.rst b/docs/src/Appendices/keywords.rst index 5fb2788a..ef974bbf 100644 --- a/docs/src/Appendices/keywords.rst +++ b/docs/src/Appendices/keywords.rst @@ -54,26 +54,3 @@ default for nev to 2. .. [#] Not used with IRISWS. .. [#] **-v=0** = quiet; 1 = verbose, 2 = debug; 3 = verbose debug .. [#] If **-w=true**, a file name is automatically generated from the request parameters, in addition to parsing data to a SeisData structure. Files are created from the raw download even if data processing fails, in contrast to get_data(... wsac=true). - -.. _slkw: - -***************** -SeedLink Keywords -***************** -Change these with SeisIO.KW.SL.[key] = value, e.g., SeisIO.KW.SL.refresh = 30. - -.. csv-table:: - :header: kw, def, type, meaning - :delim: ; - :widths: 8, 8, 8, 24 - - gap; 3600; R; a stream with no data in >gap seconds is considered offline - kai; 600; R; keepalive interval (s) - mode; \"DATA\"; I; \"TIME\", \"DATA\", or \"FETCH\" - port; 18000; I; port number - refresh; 20; R; base refresh interval (s) [#]_ - x\_on\_err; true; Bool; exit on error? - -.. rubric:: Table Footnotes - -.. [#] This value is modified slightly by each SeedLink session to minimize the risk of congestion diff --git a/docs/src/Formats/hdf5.rst b/docs/src/Formats/hdf5.rst index cf2b32e7..a741c513 100644 --- a/docs/src/Formats/hdf5.rst +++ b/docs/src/Formats/hdf5.rst @@ -33,60 +33,7 @@ Supported Keywords :sup:`(a)` A question mark ('?') is a wildcard for a single character (exactly one); an asterisk ('*') is a wildcard for zero or more characters. -.. function:: write_hdf5(fname, S::Union{GphysData, SeisEvent}) - -Write data from **S** to file **fname** in a seismic HDF5 format. The default -file format is ASDF. - -With ASDF files, if typeof(S) == SeisEvent, **S.hdr** and **S.source** are -written (appended) to the "QuakeML/" element. - -****************** -Supported Keywords -****************** -.. csv-table:: - :header: KW, Type, Default, Meaning - :delim: | - :widths: 1, 1, 1, 4 - - add | Bool | false | Add new traces to file as needed? - chans | ChanSpec | 1:S.n | Channels to write to file - len | Period | Day(1) | Length of new traces added to file - ovr | Bool | false | Overwrite data in existing traces? - v | Int64 | 0 | verbosity - -Write Method: Add (**add=true**) -================================ -Initialize new traces (filled with NaNs) of length = **len** as needed, and -overwrite with data in appropriate places. - -ASDF behavior -------------- -For the ASDF file format, **add=true** follows these steps in this order: -1. Determine times of all data in **S[chans]** and all traces in "Waveforms/". -2. For all data in **S[chans]** that cannot be written to an existing trace, a new -trace of length = **len** sampled at **S.fs[i]** is initialized (filled with NaNs). -3. If a segment in **S[chans]** overlaps a trace in "Waveforms/" (including newly- -created traces): -+ Merge the header data in **S[chans]** into the relevant station XML. -+ Overwrite the relevant segment(s) of the trace. - - -Unless **len** exactly matches the time boundaries of each segment in **S**, -new traces will contain more data than **S**, with the extra samples initialized -to NaNs. Presumably these will be replaced with real data in subsequent -overwrites. - - -Write Method: Overwrite (**ovr = true**) -======================================== -If **ovr=true** is specified, but **add=false**, **write_hdf5** *only* overwrites -*existing* data in **hdf_out**. -* No new trace data objects are created in **hdf_out**. -* No new file is created. If **hdf_out** doesn't exist, nothing happens. -* If no traces in **hdf_out** overlap segments in **S**, **hdf_out** isn't modified. -* In ASDF format, station XML is merged in channels that are partly overwritten. - +Writing to HDF5 volumes is supported through *write_hdf5*, described in :ref:`Writing to File`. .. function:: scan_hdf5(fname::String) .. function:: scan_hdf5(fname::String, level="trace") diff --git a/docs/src/Formats/metadata.rst b/docs/src/Formats/metadata.rst index 052f22b2..02a99df2 100644 --- a/docs/src/Formats/metadata.rst +++ b/docs/src/Formats/metadata.rst @@ -46,10 +46,3 @@ Supported Keywords **Note**: `mmap=true` improves read speed for ASCII formats but requires caution. Julia language handling of SIGBUS/SIGSEGV and associated risks is unknown and undocumented. - -**************** -Writing Metadata -**************** -.. function:: writesacpz(S::GphysData, pzfile::String) - -Write fields from SeisIO struct `S` to sacpz file `pzfile`. diff --git a/docs/src/Formats/timeseries.rst b/docs/src/Formats/timeseries.rst index bc287644..c2fb1528 100644 --- a/docs/src/Formats/timeseries.rst +++ b/docs/src/Formats/timeseries.rst @@ -174,14 +174,3 @@ Print headers from SEG Y file to stdout. Specify ``passcal=true`` for PASSCAL SE .. function:: uwdf(dfname) Parse UW event data file ``dfname`` into a new SeisEvent structure. - -.. function:: writesac(S[, xy=true]) - -Write SAC data to SAC files with auto-generated names. Specify xy=true to write -time-stamped x-y data; this will flag the file as generic x-y data in the SAC -interpreter. - -.. function:: wseis(fname, S) -.. function:: wseis(fname, S, T, U...) - -Write SeisIO data to fname. Multiple objects can be written at once. diff --git a/docs/src/Formats/xml.rst b/docs/src/Formats/xml.rst index 742e3e3f..b4e5a4dc 100644 --- a/docs/src/Formats/xml.rst +++ b/docs/src/Formats/xml.rst @@ -32,13 +32,6 @@ object. | simple instrument response with poles, zeros, sensitivity (**:a0**), and | sensitivity frequency (**:f0**). Very few use cases require more detail. -.. function:: write_sxml(fname::String, S::GphysData[, chans=Cha]) - -Write station XML from the fields of **S** to file **fname**. - -Use keyword **chans=Cha** to restrict station XML write to **Cha**. This -keyword can accept an Integer, UnitRange, or Array{Int64,1} argument. - ******* QuakeML ******* @@ -72,26 +65,3 @@ each per event: Non-essential QuakeML data are saved to `misc` in each SeisHdr or SeisSrc object as appropriate. - -.. function:: write_qml(fname, SHDR::Array{SeisHdr,1}, SSRC::Array{SeisSrc,1}; v::Int64=0) -.. function:: write_qml(fname, SHDR::SeisHdr, SSRC::SeisSrc; v::Int64=0) - :noindex: - -.. function:: write_qml(fname, SHDR::SeisHdr; v::Int64=0) -.. function:: write_qml(fname, SHDR::Array{SeisHdr,1}; v::Int64=0) - :noindex: - -Write QML to **fname** from **SHDR**. - -If **fname** exists, and is QuakeML, SeisIO appends the existing XML. If the -file is NOT QuakeML, an error is thrown; the file isn't overwritten. - -.. function:: write_qml(fname, SHDR::SeisHdr, SSRC::SeisSrc; v::Int64=0) - :noindex: -.. function:: write_qml(fname, SHDR::Array{SeisHdr,1}, SSRC::Array{SeisSrc,1}; v::Int64=0) - :noindex: - -Write QML to **fname** from **SHDR** and **SSRC**. - -**Warning**: to write data from a SeisSrc object R in SSRC, it must be true -that R.eid == H.id for some H in SHDR. diff --git a/docs/src/Processing/processing.rst b/docs/src/Processing/processing.rst index 292f06af..137d66d7 100644 --- a/docs/src/Processing/processing.rst +++ b/docs/src/Processing/processing.rst @@ -304,14 +304,14 @@ Keywords :delim: | :widths: 1, 2, 1, 4 - chans | [] | :sup:`(a)` | channel numbers to filter - fl | 1.0 | Float64 | lower corner frequency [Hz] \ :sup:`(b)` - fh | 15.0 | Float64 | upper corner frequency [Hz] \ :sup:`(b)` - np | 4 | Int64 | number of poles - rp | 10 | Int64 | pass-band ripple (dB) - rs | 30 | Int64 | stop-band ripple (dB) - rt | \"Bandpass\" | String | response type (type of filter) - dm | \"Butterworth\" | String | design mode (name of filter) + chans | [] | :sup:`(a)` | channel numbers to filter + fl | 1.0 | Float64 | lower corner frequency [Hz] \ :sup:`(b)` + fh | 15.0 | Float64 | upper corner frequency [Hz] \ :sup:`(b)` + np | 4 | Int64 | number of poles + rp | 10 | Int64 | pass-band ripple (dB) + rs | 30 | Int64 | stop-band ripple (dB) + rt | \"Bandpass\" | String | response type (type of filter) + dm | \"Butterworth\" | String | design mode (name of filter) | :sup:`(a)` Allowed types are Integer, UnitRange, and Array{Int64, 1}. | :sup:`(b)` By convention, the lower corner frequency (fl) is used in a diff --git a/docs/src/Submodules/quake.rst b/docs/src/Submodules/quake.rst index 0a064988..452ec8bc 100644 --- a/docs/src/Submodules/quake.rst +++ b/docs/src/Submodules/quake.rst @@ -94,15 +94,14 @@ Returns a SeisEvent. Phase Onset Query ***************** -.. function:: get_pha(:math:`\Delta`::Float64, z::Float64) +.. function:: get_pha!(S::Data[, keywords]) -Command-line interface to IRIS online implementation of the TauP travel time -calculator [1-2]. Returns a matrix of strings. Specify :math:`\Delta` in decimal degrees -and z in km with + = down. +Keywords: -| Shared keywords keywords: pha, to, v -| Other keywords: -| ``-model``: velocity model (defaults to "iasp91") +* pha: comma-separated String of phases ("P, S, SP") +* model: velocity model ("iasp91") +* to: timeout in seconds +* v: verbosity **References** diff --git a/docs/src/Web/seedlink.rst b/docs/src/Web/seedlink.rst index fb3b463f..0beb4747 100644 --- a/docs/src/Web/seedlink.rst +++ b/docs/src/Web/seedlink.rst @@ -9,12 +9,15 @@ TCP/IP-based data transmission protocol that allows near-real-time access to data from thousands of geophysical monitoring instruments. See :ref:`data keywords list ` and :ref:`channel id syntax ` for options. -.. function:: seedlink!(S, chans, KWs) -.. function:: seedlink!(S, chans, patts, KWs) -.. function:: S = seedlink(chans, KWs) +.. function:: seedlink!(S, mode, chans, KWs) +.. function:: seedlink!(S, mode, chans, patts, KWs) +.. function:: S = seedlink(mode, chans, KWs) Initiate a SeedLink session in DATA mode to feed data from channels ``chans`` with selection patterns ``patts`` to SeisData structure ``S``. A handle to a TCP connection is appended to ``S.c``. Data are periodically parsed until the connection is closed. One SeisData object can support multiple connections, provided that each connection's streams feed unique channels. +| **mode** +| SeedLink mode ("DATA", "FETCH", or "TIME"; case-sensitive). +| | **chans** | Channel specification can use any of the following options: | @@ -31,11 +34,29 @@ Pass keywords with `name=value` pairs. :ref:`Standard Keywords` ***************************** -fmt, opts, q, si, to, v, w, y +s, t, v, w + +.. _slkw: + +SeedLink Keywords +***************** +Change these with SeisIO.KW.SL.[key] = value, e.g., SeisIO.KW.SL.refresh = 30. + +.. csv-table:: + :header: kw, def, type, meaning + :delim: ; + :widths: 8, 8, 8, 24 + + u; "rtserve.iris.washington.edu"; S; base SeedLink service URL, no "http://" + gap; 3600; R; a stream with no data in >gap seconds is considered offline + kai; 600; R; keepalive interval (s) + port; 18000; I; port number + refresh; 20; R; base refresh interval (s) [#]_ + x\_on\_err; true; Bool; exit on error? + +.. rubric:: Table Footnotes -:ref:`SeedLink-Specific Keywords` -*************************************** -gap, kai, mode, port, refresh, safety, x\_on\_err +.. [#] This value is a base value; a small amount is added to this number by each new SeedLink session to minimize the risk of congestion Other Keywords ************** diff --git a/docs/src/working_with_data.rst b/docs/src/working_with_data.rst index ad9ab673..0cce231b 100644 --- a/docs/src/working_with_data.rst +++ b/docs/src/working_with_data.rst @@ -161,14 +161,14 @@ Checking Your Work ******************* If you need to check what's been done to a channel, or the sources present in the channel data, two commands are helpful: -.. function:: processing_log(S::SeisData) -.. function:: processing_log(S::SeisData, i::Int) -.. function:: processing_log(C::SeisChannel) +.. function:: show_processing(S::SeisData) +.. function:: show_processing(S::SeisData, i::Int) +.. function:: show_processing(C::SeisChannel) Tabulate and print all processing steps in `:notes` to stdout in human-readable format. -.. function:: source_log(S::SeisData) -.. function:: source_log(S::SeisData, i::Int) -.. function:: source_log(C::SeisChannel) +.. function:: show_src(S::SeisData) +.. function:: show_src(S::SeisData, i::Int) +.. function:: show_src(C::SeisChannel) Tabulate and print all data sources in `:notes` to stdout. diff --git a/docs/src/writing.rst b/docs/src/writing.rst new file mode 100644 index 00000000..b75cf389 --- /dev/null +++ b/docs/src/writing.rst @@ -0,0 +1,149 @@ +.. _write: + +############### +Writing to File +############### +The table below sumamrizes the current write options for SeisIO. Each function is described in detail in this chapter. + +.. csv-table:: + :header: Structure/Description, Output Format, Function + :delim: | + :widths: 2, 2, 1 + + GphysChannel | ASDF | write_hdf5 + GphysChannel | SAC timeseries | writesac + GphysChannel channel metadata | StationXML | write_sxml + GphysChannel instrument response | SAC polezero | writesacpz + GphysData | ASDF | write_hdf5 + GphysData | SAC timeseries | writesac + GphysData channel metadata | StationXML | write_sxml + GphysData instrument response | SAC polezero | writesacpz + SeisEvent | ASDF | write_hdf5 + SeisEvent header and source info | ASDF QuakeML | asdf_wqml + SeisEvent header and source info | QuakeML | write_qml + SeisEvent trace data only | SAC timeseries | writesac + Array{SeisEvent, 1} | ASDF QuakeML | asdf_wqml + Array{SeisHdr, 1} | QuakeML | write_qml + Array{SeisHdr, 1}, Array{SeisSrc, 1} | ASDF QuakeML | asdf_wqml + Array{SeisHdr, 1}, Array{SeisSrc, 1} | QuakeML | write_qml + SeisHdr | QuakeML | write_qml + SeisHdr, SeisSrc | ASDF QuakeML | asdf_wqml + SeisHdr, SeisSrc | QuakeML | wqml + any SeisIO structure | SeisIO file | wseis + primitive data type or array | ASDF AuxiliaryData | asdf_waux + +Methods for SeisEvent, SeisHdr, or SeisSrc structures require loading the Quake submodule with *using SeisIO.Quake*. + + +*************** +Write Functions +*************** + +HDF5/ASDF +========= +.. function:: write_hdf5(fname, S) + +Write data from **S** to file **fname** in a seismic HDF5 format. The default +file format is ASDF. + +With ASDF files, if typeof(S) == SeisEvent, **S.hdr** and **S.source** are +written (appended) to the "QuakeML " element. + +Supported Keywords +****************** +.. csv-table:: + :header: KW, Type, Default, Meaning + :delim: | + :widths: 1, 1, 1, 4 + + add | Bool | false | Add new traces to file as needed? + chans | ChanSpec | 1:S.n | Channels to write to file + len | Period | Day(1) | Length of new traces added to file + ovr | Bool | false | Overwrite data in existing traces? + v | Integer | 0 | verbosity + +Write Method: Add (**add=true**) +-------------------------------- +Initializes new traces (filled with NaNs) of length = **len** as needed, and +overwrite with data in appropriate places. + +**add=true** follows these steps in this order: +1. Determine times of all data in **S[chans]** and all traces in "Waveforms/". +2. For all data in **S[chans]** that cannot be written to an existing trace, a new trace of length = **len** sampled at **S.fs[i]** is initialized (filled with NaNs). +3. If a segment in **S[chans]** overlaps a trace in "Waveforms/" (including newly- created traces): ++ Merge the header data in **S[chans]** into the relevant station XML. ++ Overwrite the relevant segment(s) of the trace. + +Unless **len** exactly matches the time boundaries of each segment in **S**, +new traces will contain more data than **S**, with the extra samples initialized +to NaNs. Presumably these will be replaced with real data in subsequent +overwrites. + +Write Method: Overwrite (**ovr = true**) +---------------------------------------- +If **ovr=true** is specified, but **add=false**, **write_hdf5** *only* overwrites *existing* data in **hdf_out**. +* No new trace data objects are created in **hdf_out**. +* No new file is created. If **hdf_out** doesn't exist, nothing happens. +* If no traces in **hdf_out** overlap segments in **S**, **hdf_out** isn't modified. +* Station XML is merged in channels that are partly overwritten. + +.. function:: asdf_wqml(fname, H, R[, keywords]) +.. function:: asdf_wqml(fname, EV[, KWs]) + +Write to ASDF "QuakeML " group in file *fname*. In the above function calls, **H** can be a SeisHdr or Array{SeisHdr, 1}; **R** can be a SeisSource or Array{SeisSource, 1}; **EV** can be a SeisEvent or Array{SeisEvent, 1}. + +Supported Keywords +****************** +.. csv-table:: + :header: KW, Type, Default, Meaning + :delim: | + :widths: 1, 1, 1, 4 + + ovr | Bool | false | Overwrite data in existing traces? + v | Integer | 0 | verbosity + + +.. function:: asdf_waux(fname, path, X) + +Write *X* to AuxiliaryData/path in file *fname*. If an object already exists at +AuxiliaryData/path, it will be deleted and overwritten with *X*. + +XML Metadata +============ + +.. function:: write_sxml(fname, S[, chans=CC]) + +Write station XML from the fields of **S** to file **fname**. Specify channel numbers to write in a GphysData object with *chans=CC*. + +Use keyword **chans=Cha** to restrict station XML write to **Cha**. This +keyword can accept an Integer, UnitRange, or Array{Int64,1} argument. + +.. function:: write_qml(fname, H, R[, v=V]) +.. function:: write_qml(fname, H, R[, v=V]) + :noindex: + +.. function:: write_qml(fname, H[, v=V]) +.. function:: write_qml(fname, H[, v=V]) + :noindex: + +Write QML to **fname** from SeisHdr (or Array{SeisHdr, 1})**H**, and (optionally) SeisSrc (or Array{SeisSrc, 1})**R** + +If **fname** exists, and is QuakeML, SeisIO appends the existing XML. If the +file is NOT QuakeML, an error is thrown; the file isn't overwritten. + + +Other Formats +============= + +.. function:: writesac(S[, chans=CC, fname=FF, v=V]) + +Write SAC data to SAC files with auto-generated names. With any GphysChannel subtype, specifying *fname=FF* sets the filename to FF. Specify channel numbers to write in a GphysData object with *chans=CC*. + +.. function:: writesacpz(pzf, S[, chans=CC]) + +Write fields from SeisIO structure *S* to SACPZ file *pzf*. Specify which channels to write in a GphysDaya structure with *chans=CC*. + +.. function:: wseis(fname, S) +.. function:: wseis(fname, S, T, U...) + +Write SeisIO data to file *fname*. Multiple objects can be written at once.