Renaming, converting, clipping: script based raster time series processing

When working with remotely sensed time series data (e.g. MODIS, AVHRR, GIMMS, etc.), bulk processing can save a lot of time. Using a terminal in a Linux environment, simple scripts can process thousends of files in short time. Here are some basic hints on how to start, gdal has to be installed. For Windows users or users who prefer an interface, QGIS provides a graphical interface for most of the gdal modules. Also renaming can be done more user friendly.

Renaming:

Original data-names can be very long annoying. So first we rename all files to numbers from 1 to XX (like 1, 2, 3…). To do so, navigate to the folder with the files, and these lines will rename all files originally starting with MOD (like MOD13Q1203837383922837284.hdf):

for fname in MOD* 
do
mv $fname `printf $x`
x=$(($x+1))
done

Then the following lines will create a script that renames all files numbered from 1 to 60. the final files will include year and date. So we then have data named MODIS_2001_01 until MODIS_2005_12. This can be adopted to every name and numbering. The script is then executed.

for ((i=1;$i<=60;i++)) do echo "mv $i MODIS_$(( ($i-1) / 12 + 2001 ))_$(( ($i-1) % 12 + 1 )).tif" ; done >rename.sh
sed -i 's/_1/_01/g' rename.sh
sed -i 's/_2/_02/g' rename.sh
sed -i 's/_3/_03/g' rename.sh
sed -i 's/_4/_04/g' rename.sh
sed -i 's/_5/_05/g' rename.sh
sed -i 's/_6/_06/g' rename.sh
sed -i 's/_7/_07/g' rename.sh
sed -i 's/_8/_08/g' rename.sh
sed -i 's/_9/_09/g' rename.sh
chmod 0755 rename.sh
./rename.sh

Convert:

This works fine to convert all kind of formats, also very useful to change HDF files (like MODIS, VGT…) to GeoTiff. This example converts GeoTiffs to ENVI format.

# define in and out folder and output format

in="/home/martin/SPOT_VGT/NDV/"
out="/home/martin/SPOT_VGT/NDV/converted/"
format="ENVI"

# convert all Tiffs to ENVI format, use float32 as output here

cd $in
for i in *.tif; do gdal_translate -of $format -ot Float32 $i $out`basename $i`; done

Clip:

# define in and out folder

in="/home/martin/SPOT_VGT/NDV/"
out="/home/martin/SPOT_VGT/NDV/clipped/"

# define the window which should be clipped

range="-17.6 16.25 -2.72 13.16"

# do the clipping for all Tiffs

cd $in 
for i in *.tif; do gdal_translate -projwin $range $i $out`basename $i`; done

For more information, see http://www.gdal.org/

Advertisements