Error while convert data from suds to mseed

Usage Questions
1

hello everyone.
I have old seismograms stored in suds format. I can read data with python and then, after set the first date and other parameters like
New_tr = obspy.Trace(data = np.array(values), header = {
“network”: Net,
“station”: Station,
“location”: “”,
“channel”: BandCode+IntCode+Component,
“starttime” : datetime(Year, Month, Day, Hour, Minutes, Seconds),
“sampling_rate”: Rate,
“npts”: len(values)})
I can write data with command
st.write(fileout, format=‘MSEED’, reclen=512), data is write in SDS structure.
There is the possibility that where founded data in same days and after have prepared data as specified up, I add read data stored as
existingt = obspy.read(fileout),
appendig traces like
for trace in st:
existingt.append(trace)
sorting by date
existingt.sort([‘starttime’])
and then rewrited
existingt.write(fileout, format=‘MSEED’, reclen=512)
For some file I encountered error like

File “/home/user/Binawin_8.1.1.py/suds.py”, line 1395, in converti
existingt.write(fileout, format=‘MSEED’, reclen=512)
File “/home/user/anaconda3/lib/python3.11/site-packages/obspy/core/stream.py”, line 1458, in write
write_format(self, filename, **kwargs)
File “/home/user/anaconda3/lib/python3.11/site-packages/obspy/io/mseed/core.py”, line 843, in _write_mseed
mst = MST(trace, data, dataquality=trace_attr[‘dataquality’])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/user/anaconda3/lib/python3.11/site-packages/obspy/io/mseed/core.py”, line 969, in init
sampletype = SAMPLETYPE[data.dtype.type]
~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: <class ‘numpy.int64’>

Why this happen?
What I miss?

Regards and cheers,
Sergio

2

Not sure why you end up with int64, in any case mseed can not store int64 data. There should be a meaningful error getting shown that int64 doesn’t work though (if no specific mseed encoding is chosen and a proper mseed encoding is autoselected). We have that since obspy 1.2.0, what version are you? Since I don’t see you specifying an encoding manually.

Also please consider looking at the “welcome, read this first” post for how to format posts, especially code blocks and blockquotes.

3

I have try to change the reclen = 1024 and then the error does not appear

4

Hello megies,
I have obspy 1.4.6.
Sorry for haven’t coded the code passing to you.
Now I code it for reading well.
Those are parameters to set the trace

New_tr = obspy.Trace(data = np.array(values), header = {
“network”: Net,
“station”: Station,
“location”: “”,
“channel”: BandCode+IntCode+Component,
“starttime” : datetime(Year, Month, Day, Hour, Minutes, Seconds),
“sampling_rate”: Rate,
“npts”: len(values)})

and this is the code where I write to files.

existingt = obspy.read(fileout),
appendig traces like
for trace in st:
existingt.append(trace)
sorting by date
existingt.sort([‘starttime’])

and then rewrite wit

existingt.write(fileout, format='MSEED', reclen=512)

As you can see I have not declared the format archive, but I can tell you that the data are all integer. Old recordings.

Sergio

5

I do not know why, but not I have not errors… Thanks to all

6

The record length has nothing to do with the problem you described above.

You live in the future! :astonished:
Current latest stable obspy is 1.4.1

Like I said before, your problem was just that data somehow ended up in int64 format which is quite unusual and miniseed cannot encode that format. We have had for a long time code that checks if int64 data can be downcast to int32 safely and then write it as MiniSEED but somehow that did not happen in your case.

Well, good if it’s fixed on your side now, anyway. :+1:

7

Sory for my mistake.
Versione 1.4.1

8

Change of the program,
I whant to convert older Suds data into mseed and I read and convert windows number in linux number.
After that I fill data with this code:

        if  Component == 'V':
          Component = 'Z'
        Rate = descriptTrace.Rate
        if Rate >=1000 and Rate < 5000:
          BandCode = 'F'
        elif Rate >=250 and Rate < 1000:
          BandCode = 'C'
        elif Rate >=80 and Rate < 250:
          BandCode = 'H'
        elif Rate >=10 and Rate < 80:
          BandCode = 'S'
        elif Rate >1:
          BandCode = 'L'
        elif Rate >.1:
          BandCode = 'V'
        elif Rate >.01:
          BandCode = 'U'
        IntCode = 'L' 
        New_tr = obspy.Trace(data = np.array(values), header = {
          "network": Net,
          "station": Station,
          "location": "",
          "channel": BandCode+IntCode+Component,
          "starttime" : datetime(Anno, Mese, Giorno, Ore, Minuti, Secondi),
          "sampling_rate": Rate,
          "npts": len(values)})
        st = obspy.Stream(traces=[New_tr])

where “values” are sequential data.

and then I create path and file name and I try to save it with this command

        DOY = datetime_obj.strftime('%j').zfill(3)
        fileoutname = Archive + slash + str(Anno) + slash + str(Net) + slash + Station
        if not os.path.isdir(fileoutname + slash + BandCode + IntCode + Component + '.D'):
          os.makedirs(fileoutname + slash + BandCode + IntCode + Component + '.D')
        fileout = fileoutname + slash + BandCode + IntCode + Component + '.D' + slash + Net + '.' + Station + '.' + BandCode + IntCode + Component + '.D.' + DOY
        if not os.path.isfile(fileout):
          st.write(fileout, format = 'MSEED', reclen = 512)

Else, if file exists I do this other command

        else:
          existingt = obspy.read(fileout)
          for trace in st:
            if packet_exists(existingt,trace) == False:
              existingt.append(trace)
              with open(logFile,'at') as log:
                for esistente in existingt:
                  log.write(str(fileout))
                  log.write(str(esistente))
                log.write(str(trace))
          existingt.sort(['starttime'])

When I read in the log file I receive again the error of the class of numpy

home/User/var/lib/archive/1991/OB/SERR/HLZ.D/OB.SERR.HLZ.D.292OB.SERR..HLZ | 1991-10-19T23:46:26.000000Z - 1991-10-19T23:47:11.992000Z | 125.0 Hz, 5750 samples/home/User/var/lib/archive/1991/OB/SERR/HLZ.D/OB.SERR.HLZ.D.292OB.SERR..HLZ | 1991-10-19T22:32:18.000000Z - 1991-10-19T22:33:55.992000Z | 125.0 Hz, 12250 samplesOB.SERR..HLZ | 1991-10-19T22:32:18.000000Z - 1991-10-19T22:33:55.992000Z | 125.0 Hz, 12250 samples-----------------------------------------------------------------------------------------
tipo dei dati specificati in descriptTrace.datatype = i
/home/User/var/lib/archive/1991/OB/SERR/HLZ.D/OB.SERR.HLZ.D.292
Traceback (most recent call last):
  File "/home/User/Binawin_8.1.1.py/suds.py", line 1411, in converti
    existingt.write(fileout, format = 'MSEED', reclen = 512)
  File "/home/User/anaconda3/lib/python3.11/site-packages/obspy/core/stream.py", line 1458, in write
    write_format(self, filename, **kwargs)
  File "/home/User/anaconda3/lib/python3.11/site-packages/obspy/io/mseed/core.py", line 843, in _write_mseed
    mst = MST(trace, data, dataquality=trace_attr['dataquality'])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/User/anaconda3/lib/python3.11/site-packages/obspy/io/mseed/core.py", line 969, in __init__
    sampletype = SAMPLETYPE[data.dtype.type]
                 ~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: <class 'numpy.int64'>
OB.SERR..HLZ | 1991-10-19T22:32:18.000000Z - 1991-10-19T22:33:55.992000Z | 125.0 Hz, 12250 samples

Where I make mistake(s)?

Greetings again and thanks for your patience
Sergio

9

Please, can anybody help me?

10

Like I said, int64 does not work with MiniSEED, you need to downcast to int32. Like I said, we have code that tries to automatically downcast if safely possible, I’m not sure why that piece of our code is not run, it should only get skipped if in your traces specific MSEED header values are set, but according to your code that isn’t the case.
Are you like multiplying your raw data by some scaling factor? I don’t see why your “values” are int64, but my best guess is that numpy automatically scaled it up for you after multiplying by some large integer number (some scaling factor?) maybe…?

You could do something like that yourself, before your “write” operation. Obviously, you wanna check if the downcast is safe like we do in that code piece, or you end up with corrupt data.

11

something like

for tr in st:
    if abs(tr.max()) <= np.iinfo(np.int32).max:
        raise Exception("cant downcast safely")
    tr.data = tr.data.astype(np.int32, casting='safe', copy=True)