File size increasing on removing the response from mseed files

Usage Questions
1

Hello everyone

I am facing a situation where downloaded mseed files are increasing in size and number after they are read, response removed and written again. I understand the reason of increase in number but the big increase in size is not making sense to me. To get around this issue I am thinking of the following

  1. Use mass downloader with response removal. More specifically I require the download_helpers of mass downloader to check if any files are missing. If there is a way to do this please let me know.

  2. Directly address the issue of increase in size while writing the mseed files.

EDIT: I could not attach the codes as new users are not allowed.

2

Hi. Uncorrected miniSEED files are usually stored as integers, which compress very well in the original files. Perhaps correcting them will change the values to floats, which most certainly compress poorly or not at all. Do you get a warning from ObsPy about the dtype when you write the files?

3

Hi Jonathan

Thank you for your reply. Yes I did get an warning from Obspy while the files are being written.

C:\Users\NiyoS.conda\envs\obspy\lib\site-packages\obspy\io\mseed\core.py:790: UserWarning: The encoding specified in trace.stats.mseed.encoding does not match the dtype of the data.
A suitable encoding will be chosen.
warnings.warn(msg, UserWarning)

4

Ah, it sounds like this could be the source of the file size problem. There aren’t many options for compressing floats, but you may want search this forum or the ObsPy GitHub issues for this topic.

5

So the solution given by Lion Krischer here works: miniseed filesize - #3 by LionKrischer. The file size still increases but not to that extent before. Overall it boils down to changing the precision of the floating point numbers in the dtype before writing the file.

Thanks Jonathan.

1 Like