How to fix the issue of missing data?

Usage Questions
1

Hi everyone,
I need to calculate the amplitude of the waveform record for multiple short intervals for a single station. However, some time for few intervals data is missing or there is a data/gap. Eventually, code stop working. May someone suggests how I can modify my script so that it work more efficiently.

client = Client("IRIS")
T_P=ce["P_Time"]
ev_dis=ce["dist(km)"]
ev_m=ce["Mag"]
ev_dep=ce["depth"]
station_latitude=45.90
station_longitude=-130
pgv_v=[]
for a in range (0,150):
    time=T_P[a]
    time=UTCDateTime(time)
    dis=ev_dis[a]
    M=ev_m[a]
    depth=ev_dep[a]
    starttime=time-timedelta(minutes=5)
    endtime= time+timedelta(minutes=60)
    net = "IRIS"
    sta = "Nil"
    loc = "*"
    chan = "HHZ"
    st = client.get_waveforms(net, sta, loc, chan, starttime, endtime, attach_response= True);
    st_rem=st.copy()
    pre_filt = (0.001, 0.005, 45.0, 50.0)
    st_tr=st_rem.remove_response(output = 'VEL',plot=False, pre_filt=pre_filt, zero_mean=True, taper=True);
    st_dm=st_tr.detrend('demean')
    st_tr=st_rem.detrend('linear')
    st_filt=st_tr.filter('bandpass', freqmin =0.00000001, freqmax=0.1)
    data=st_filt[0].data
    pgv=max(data)
    pgv_v.append(float(pgv))
    st_filt.plot(equal_scale=False, automerge=False);
    print(time, depth, dis, M)
    
pgv = pd.DataFrame(pgv_v)
print(pgv)
2

Depends on how you want to handle missing data.

A starting point (untested):

try:
    pgv=np.nanmax(data)
except:
    print('no data for')
else:
    pgv_v.append(pgv)
    st_filt.plot(equal_scale=False, automerge=False);
print(time, depth, dis, M)

This looks like a lowpass:

st_filt=st_tr.filter('bandpass', freqmin =0.00000001, freqmax=0.1)

Better explicitly use a lowpass filter.

1 Like
3

Hi, thanks for your suggestion, I attempted with your suggestion but still there is some error

client = Client("IRIS")
T_S=CE["S_arrival"]
T_E=CE["S_end"]
ev_dis=CE["dist(km)"]
ev_m=CE["Mag"]
ev_dep=CE["depth"]
station_latitude=45.90
station_longitude=-130
pgv_v=[]
for a in range (34, 38):
    time=T_S[a]
    starttime=UTCDateTime(time)
    time_e=T_E[a]
    dis=ev_dis[a]
    M=ev_m[a]
    depth=ev_dep[a]
    endtime= UTCDateTime(time_e)
    net = "OO"
    sta = "AXCC1"
    loc = "*"
    chan = "HHZ"
    st = client.get_waveforms(net, sta, loc, chan, starttime, endtime, attach_response= True);
    st_rem=st.copy()
    pre_filt = (0.001, 0.005, 45.0, 50.0)
    st_tr=st_rem.remove_response(output = 'VEL',plot=False, pre_filt=pre_filt, zero_mean=True, taper=True);
    st_dm=st_tr.detrend('demean')
    st_tr=st_rem.detrend('linear')
    st_filt=st_tr.filter('lowpass', freq=0.5, zerophase=True)
    #st_filt=st_tr.filter('bandpass', freqmin =0.001, freqmax=10)
    data=st_filt[0].data
    pgv=max(data)
    except:
    print('no data for')
    else:
    pgv_v.append(float(pgv))
    st_filt.plot(equal_scale=False, automerge=False);
    print(time, depth, dis, M)
ev_1_50 = pd.DataFrame(pgv_v)
print(ev_1_50)