bad nanometrics miniseed

Users Mailing List Archive
1

Hello,

I am using obspy to read miniseed files produced by a Nanometrics system. Many of the files seem to read OK, but several produce messages such as:

Warning: msr_unpack_steim1(do_domp__SHN_D): number of samples indicated in header (3609) does not equal data (3346)
do_domp__SHN_D: Warning: Data integrity check for Steim-1 failed, last_data=-1843867963, xn=-65

I have read in the trac page http://obspy.org/ticket/222 that a similar error message was produced was reading data from EarthData data loggers, but am unsure if I’m encountering the same problem (which was related to byte order). In particular, it seems odd that some files are OK while others are not.

I am wondering if anyone has any ideas for diagnosing this problem, or at least is there a way to test which data are bad and which are not? As it is I loop through many files and see a lot of these messages produced, but it’s hard to tell just which files are causing the problem.

Many thanks,

  • Phil

Phil Cummins
Prof. Natural Hazards
Research School of Earth Sciences
Australian National University
Bldg 61 Mills Road Acton ACT
0200 Australia

2

Hi Phil,

its pretty much exactly as Tobi said. My best idea would also be to just print the
filename prior to reading each file with ObsPy to identify the problematic files.

It seems very odd to me that, assuming all files have been written with the same
system, some work fine and some do not. Maybe some files have been written
with different settings then other files?
ObsPy stores some attributes of read MiniSEED files in trace.stats.mseed so that
can be checked quite easily.

As far as I can tell all files are still read and you end up with a Stream object for
each file. Is that correct? If the Stream objects are plotted and the data looks fine
then a byteorder problem can most likely be discarded. Do the other stats
attributes of the broken files make sense (e.g. time/date/samling rate/...)? If the
data as well as the header attributes are ok, than my best guess would simply be
that some files are broken.

But that is hard to tell without an actual file so posting one would indeed help a lot
or even just one broken record of a file.

Best wishes,

Lion

3

Hi Lion,

Many thanks for your reply. Attached is a file that produces the warning messages when I read it. Yes, it seems odd that only some files would be bad. I don't know the explanation for that. They are dumps from the Nanometrics ring buffer, and as far as I know the same settings were used.

dompu (876 KB)

4

Hi @all,

if it's the same issue I faced some months ago (ticket #222), then
reading these files won't be possible using obspy / libmseed.

Marcus

5

Hi Lion,

It turned out that the numbers of bad files were not as many as I had thought. I've just gotten rid of them manually. I'm sorry to have bothered you about looking at my bad miniseed files.
I'm very happy that obspy can process the remaining data.

Thanks in any case for your support!

- Phil

6

Hi Phil,

good to hear that just some files are broken. The warning is echoed for every record it cannot
read therefore it looks like a lot of broken files.

I still had a look at your file and the problem is that it contains no Blockette 1000 for any record
which is necessary according to the SEED manual. So its actually not a valid MiniSEED file.

The attached script fixes the file and should work for any file that has the same problem.

Best wishes,

Lion Krischer

fixfile.py (1.08 KB)

7

Hi Lion,

Thanks so much for this. You script works great and lets me fill the gaps in my dataset. Fantastic!
Cheers,

- Phil