I am trying to mass download some data of INGV, but I hit some problems on the way. I use the Continuous request as described here:
. The waveforms start do download, then at some point i start to get msg as: If i retry from different IP it works again for some time, and then I get blocked… What to do? are INGV blocking the mass downloading? Thanks, Blaž
hmm - good question. I’ve downloaded a fair amount of data from the INGV and have not noticed that behavior - maybe they changed their policy. The mass downloader does send a lot of requests (it has to to work at all) but it tries to batch the request in a reasonable manner to be somewhat nice to data centers so I don’t think it does anything out of line. By default it launches up to 3 threads per data center to download in parallel (this seems to be a number the data centers are okay with) - you can try modifying this with
Can you contact the INGV and ask how to best proceed? Otherwise I can also do it.
Thanks, I will contact the INGV and see what can be done. Anyway, if I get some useful info, I’ll write it here.