ppsd.plot() stagnant

Users Mailing List Archive
1

Hi All,
I am trying to save my ppsd.plot() by creating a handle as follows;

highfreq = 125
lowfreq = 0.01

ppsdfig = sta_ppsd.plot(period_lim=(1/highfreq, 1/lowfreq), show_mean=True,
xaxis_frequency=True, cmap=pqlx, show=False)

My PPSD has 18,380 time segments. The above step, which I have isolated, appears to take an enormous amount of time. In several attempts I ran it for 5 - 6 hours and yet it didn’t finish. Isn’t this unusual? Is there a way to speed up this process? If I disrupt the kernel, I get a figure in the Markdown, which means that the figure is being created in the above step.

The reason I need this step is because I create custom figures in the next ones as follows;

ppsdfig.set_size_inches(6,6)
ppsdfig.savefig(‘tt.png’, bbox_inches=‘tight’, dpi=300)

The whole process works fine with a smaller number of time segments (e.g. for a day, 47)

Any help is much appreciated!

Thanks!

Januka

Januka Attanayake

Research Fellow | Earthquake Seismology
Homepage: https://sites.google.com/site/janukaattanayake

School of Earth Sciences | McCoy Bldg. 200
University of Melbourne | Parkville 3010 VIC
Australia

2

Please run you program outside of Jupyter and check if the problem
persists. I've never seen problems with PPSD plots and 18k segments is
nothing unusual at all.

best,
Tobias

3

Thanks Tobias. If by that you meant running that piece of the code as a python script rather than a Jupyter nb, I tried the following as a python script and it crashes when attempting to plot the ppsd (the last line). Note that I saved the ppsd as a .npz within my Jupyter nb previously and I am recalling it within this python script. I do this after activating obspy within conda in the terminal.

import obspy
from obspy import read
from obspy.io.xseed import Parser
from obspy.signal import PPSD
from obspy.imaging.cm import pqlx
from obspy import read_inventory
import matplotlib.pyplot as plt
import os, shutil, glob

read the saved binary file and plot

sta_ppsd = PPSD.load_npz(my_ppsd.npz)

ppsdfig = sta_ppsd.plot(period_lim=(0.01, 100), show_mean=True,
xaxis_frequency=True, cmap=pqlx, show=False)
ppsdfig.set_size_inches(width,height)
ppsdfig.savefig(outfig1, bbox_inches=‘tight’, dpi=resdpi)

Januka Attanayake

Research Fellow | Earthquake Seismology
Homepage: https://sites.google.com/site/janukaattanayake

School of Earth Sciences | McCoy Bldg. 200
University of Melbourne | Parkville 3010 VIC
Australia

Please run you program outside of Jupyter and check if the problem
persists. I’ve never seen problems with PPSD plots and 18k segments is
nothing unusual at all.

best,
Tobias

4

Hi Januka,

can you upload your npz file somewhere? Then I could test your problem.

best,
Tobias

5

Hi Januka,

after looking at the data you sent me, it looks like you have some
problem in your processing. The PPSD's storage for what data was added
to the PPSD object is blown up like crazy, maybe due to a malformed loop
when fetching data to add it to the PPSD or actually it looks like you
have crazily fragmented data in some parts (or both)? Your data holds
18k processed psd segments which is what you would expect for roughly a
year of processed data. However it also holds 1.5 Mio objects for
individual Traces added to the PPSD object.

data = np.load("DDNE_ppsd.npz")
data['_times_data'].shape

-> _times_data shape: (1515898, 2)

This is also the problem why your plot takes forever, the coverage plot
on the bottom tries to plot 1.5 Mio rectangles for your time
information. The plot finishes OK (in a few minutes) when you leave out
the bottom part of the plot with "show_coverage=False" (the loading of
the PPSD still takes a long time due to the bloated npz object).

best,
Tobias

import obspy
from obspy import read
from obspy.io.xseed import Parser
from obspy.signal import PPSD
from obspy.imaging.cm import pqlx
from obspy import read_inventory
import matplotlib.pyplot as plt
import os, shutil, glob

# read the saved binary file and plot
sta_ppsd = PPSD.load_npz("DDNE_ppsd.npz")

ppsdfig = sta_ppsd.plot(period_lim=(0.01, 100), show_mean=True,
                        xaxis_frequency=True, cmap=pqlx, show=False,
show_coverage=False)
# ppsdfig.set_size_inches(width,height)
ppsdfig.savefig("/tmp/ppsdplot.png", bbox_inches='tight', dpi=100)

6

Thanks very much for diagnosing this Tobias. Our recorders save data as one-minute files which we merge to make day-long time series. Perhaps this might explain your observation. I will look into fixing this. Do you have any recommendations?

Thanks again!

Januka

Januka Attanayake

Research Fellow | Earthquake Seismology
Homepage: https://sites.google.com/site/janukaattanayake

School of Earth Sciences | McCoy Bldg. 200
University of Melbourne | Parkville 3010 VIC
Australia

Hi Januka,

after looking at the data you sent me, it looks like you have some
problem in your processing. The PPSD’s storage for what data was added
to the PPSD object is blown up like crazy, maybe due to a malformed loop
when fetching data to add it to the PPSD or actually it looks like you
have crazily fragmented data in some parts (or both)? Your data holds
18k processed psd segments which is what you would expect for roughly a
year of processed data. However it also holds 1.5 Mio objects for
individual Traces added to the PPSD object.

data = np.load(“DDNE_ppsd.npz”)
data[’_times_data’].shape

-> _times_data shape: (1515898, 2)

This is also the problem why your plot takes forever, the coverage plot
on the bottom tries to plot 1.5 Mio rectangles for your time
information. The plot finishes OK (in a few minutes) when you leave out
the bottom part of the plot with “show_coverage=False” (the loading of
the PPSD still takes a long time due to the bloated npz object).

best,
Tobias

import obspy
from obspy import read
from obspy.io.xseed import Parser
from obspy.signal import PPSD
from obspy.imaging.cm import pqlx
from obspy import read_inventory
import matplotlib.pyplot as plt
import os, shutil, glob

read the saved binary file and plot

sta_ppsd = PPSD.load_npz(“DDNE_ppsd.npz”)

ppsdfig = sta_ppsd.plot(period_lim=(0.01, 100), show_mean=True,
xaxis_frequency=True, cmap=pqlx, show=False,
show_coverage=False)

ppsdfig.set_size_inches(width,height)

ppsdfig.savefig("/tmp/ppsdplot.png", bbox_inches=‘tight’, dpi=100)