I tried reading in chunks but the order and length of traces is not same as it was originally. Is there any way I can load the mseed file as a whole at once with the order and length of traces preserved. The memory of my system is 64gb so I think there won’t be any issue with memory usage.
Ordering of the blocks in the mseed file just depends on how the acquisition software received and wrote it. It doesn’t have any meaning. To be honest, I would just clean up the data, split it into multiple files, one per channel and per day. Makes more sense to me than working insanely hard to work around all kinds of issues with a gigantic single file.
The following is absolutely not the fastest or most efficient way to do it, but takes 2 mins to implement.
import io
from obspy import read
record_size = 512 # or 1024, 2048 or whatever it is
with open("my_insanely_big_mseed_file", "rb") as fh:
while True:
data = fh.read(record_size)
if not data:
break
bio = io.BytesIO(data)
tr = read(bio, format='MSEED', headonly=True)[0]
# generate SDS filename, one file per channel per day
t = tr.stats.starttime
out_filename = f'{tr.id}.D.{t.year}.{t.julday}'
with open(out_filename, "ab") as out: # "a" for append to file if it exists
out.write(data)