Gaps over the data
tonino0013 opened this issue · comments
@ThomasLecocq, nice work!!, however if we had data with gaps due to the transmission the script will stop at each gap, can we full with zeros if we find those gaps.
Could you provide a little example?
I don't know if this is the same problem that @tonino0013 was discussing above, but this station:
network = "AM"
station = "RB30C"
location = "00"
channel = "EHZ"
dataset = "Falmouth"
time_zone = "UTC/GMT +1 hour"
sitedesc = "Falmouth Office (Cornwall, UK)"
Has data missing on the following days:
Can't load Falmouth_2020-01-25_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-01-26_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-08_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-09_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-10_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-11_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-12_AM.RB30C.00.EHZ.mseed
Can't load Falmouth_2020-02-13_AM.RB30C.00.EHZ.mseed
By default, the code crashes. It is possible to catch the error using try: except: else: so the program does not crash, but then I will need to add the same thing in step 4 and the question of interpolation, raised by @tonino0013 then also arises.
Hey both of you. Please provide the "step" at which it fails , and also copy/paste the full traceback message!
Thanks !
Step 3
Fetching Falmouth_2020-01-25_AM.RB30C.00.EHZ.mseed: 0%| | 0/182 [00:00<?, ?it/s]
---------------------------------------------------------------------------
FDSNNoDataException Traceback (most recent call last)
<ipython-input-3-080e32aa0d67> in <module>
14 pbar.set_description("Fetching %s" % fn)
15 st = c.get_waveforms(network, station, location, channel,
---> 16 UTCDateTime(day)-1801, UTCDateTime(day)+86400+1801, attach_response=True)
17 st.write(fn)
18 resp = c.get_stations(UTCDateTime(day), network=network, station=station, location=location,
~\AppData\Roaming\Python\Python37\site-packages\obspy\clients\fdsn\client.py in get_waveforms(self, network, station, location, channel, starttime, endtime, quality, minimumlength, longestonly, filename, attach_response, **kwargs)
842 # Gzip not worth it for MiniSEED and most likely disabled for this
843 # route in any case.
--> 844 data_stream = self._download(url, use_gzip=False)
845 data_stream.seek(0, 0)
846 if filename:
~\AppData\Roaming\Python\Python37\site-packages\obspy\clients\fdsn\client.py in _download(self, url, return_string, data, use_gzip)
1395 debug=self.debug, return_string=return_string, data=data,
1396 timeout=self.timeout, use_gzip=use_gzip)
-> 1397 raise_on_error(code, data)
1398 return data
1399
~\AppData\Roaming\Python\Python37\site-packages\obspy\clients\fdsn\client.py in raise_on_error(code, data)
1719 if code == 204:
1720 raise FDSNNoDataException("No data available for request.",
-> 1721 server_info)
1722 elif code == 400:
1723 msg = ("Bad request. If you think your request was valid "
FDSNNoDataException: No data available for request.
Detailed response of server:
@ThomasLecocq , this is the example
Traceback (most recent call last):
File "ruidos.py", line 41, in <module>
UTCDateTime(day)-1801, UTCDateTime(day)+86400+1801, attach_response=True)
File "/home/tonino/.local/lib/python3.5/site-packages/obspy/clients/fdsn/client.py", line 844, in get_waveforms
data_stream = self._download(url, use_gzip=False)
File "/home/tonino/.local/lib/python3.5/site-packages/obspy/clients/fdsn/client.py", line 1397, in _download
raise_on_error(code, data)
File "/home/tonino/.local/lib/python3.5/site-packages/obspy/clients/fdsn/client.py", line 1721, in raise_on_error
server_info)
obspy.clients.fdsn.header.FDSNNoDataException: No data available for request.
Detailed response of server:
The step is: STEP3
Well it is a bit hard to make the difference between an incorrect request and a data gap... maybe with fdsnws/availability ?
@ThomasLecocq, in my case it is because of fdsnws availability, due to snow time here sometimes the VSAT did not send data so no data availability, I will try to merge different CSV outfiles. Thanks
fixed with 03ce568