Miniseed header byte-order flag -- incompatibility with GIPPTOOLS software

Hi Christof and Lion,

Thanks for your responses! Sorry it took a while to get back to you.

I have uploaded the two miniseed files I used as examples to this folder - please let me know if you can’t access it!

One of the peculiarities of the issue I’m having is that the large miniseed file (spanning 2007-2017) downloaded from IRIS itself cannot be read by mseedinfo or mseedcut - it returns the same error. If I try to read it with mseedhdr (from PASSCAL) it gives the error “Non-Default Blocksize detected 1, 4096” and fails to read the file properly. The only think I can read it with successfully is obspy.

When I have read it with obspy, sliced it, then saved it as daily mSEED files, some of the resulting day-long mSEED can be read by GIPPTOOLS, some cannot. The two example mSEED files are consecutive days of data, but 2015-113 does work, while 2015-114 does not. All days following 2015-114 then don’t work for the Z component.

On further investigation, this corresponds to a change in the encoding method of the mSEED downloaded from IRIS - from STEIM1 to STEIM2. I think this is perhaps why the large mSEED files can’t be read by other programs - they contain data with mixed encodings. This change happens partway through the day on day 2015-113 for the Z component - the file I have uploaded only includes the first part of that day’s data so as to not mix encodings. For the E and N components the same is the case - all data from 2007 to around this date in 2015 is encoded as STEIM1 and the day-long mSEED files produced by slicing work. After the change to STEIM2 they don’t. However this is not generally a problem - we have other data in STEIM2 format that works fine with the GIPPTOOLS programs.

I wondered if the download may have been the issue, so re-downloaded a portion of the encoding change (so the large mSEED file did not have mixed encoding). This large mSEED file could be read successfully by all programs, but when sliced with obspy and written to day files the same issues returned - this indicates to me that the download was not part of the problem, but something about how this mSEED is written by obspy, and this clashing with the GIPPTOOLS programs. All the day files output by obspy can be read by mseedhdr, from the PASSCAL software suite, so perhaps this is an issue specific to the GIPPTOOLS programs. Either way you two seem best placed to answer this query - thanks for looking at it!

Cheers,

Tom

Heyhey,

From ObsPy’s perspective nothing seems wrong with the files so its kind of hard to tell what is happening here.

ObsPy can read any sequence of records even if they change record length, encoding, … throughout. Be aware that reading and writing is not fully lossless in ObsPy as our internal waveform representation does not work on the record level so there is some information we can never preserve. If you chunk the data I would do it manually and just cut along the record boundaries. ObsPy’s mass downloader () does this internally so there is already some functionality for that available:

The function does a bit more that you need but the main logic should be easy to grasp.

@Christof: I attached a small file that has a blockette 100. Maybe that is the culprit? If you discover anything wrong with the MiniSEED files we produce please let us know!

Cheers!

Lion

single_record_with_blockette_100.mseed (4 KB)