Pytroll/SatPy

Summary

This page contains some scripts for satpy, based on the work of Ernst Lobsiger.

See for a few messages in the MSG-1 mailing list and links to starter kits: here

A short overview on installing satpy: Installation Satpy


Some major changes have been done. For the earlier version satpy sets see here. For earlier version

Content

  • GEO scripts
  • LEO scripts
  • Creating movie
  • Creating 'live' movie
  • Selector (gui)
  • Using raspberry pi

    GEO scripts

    These are still stand-alone scripts, i.e., don't need other .py scripts.

    The adapted geo-scripts are very similar to the original ones, but they contain options for composite, area etc.

  • Note 1: These scripts are valid for both Linux and WIndows. (Un)comment the right opsys in the scripts.
  • Note 2: In these files, adapt toodrp and datdrp to your installation!

    Usage

     ./<script>.py [-h] [-t T] [-sat SAT] [-src SRC] [-dst DST] [-o O] [-area AREA] [-composite COMPOSITE] [-v]
    
    
    options:
      -h, --help            show this help message and exit
      -t T                  time: YYYYmmDDHHMM
      -sat SAT              RSS|IODC|FULL|1|2|3|4]
      -nat                  use native files
      -src SRC              source dir., MUST be absolute path!
      -dst DST              destination dir., MUST be absolute path!
      -o O                  output file
      -area AREA            area, 'list' gives a list, 'list=': items containing <str>
      -composite COMPOSITE  composite, 'list' gives a list
      -v [V]                view
    
    Note: option '-composite list' needs also satellite data so you need to add -t option!

    zip filecontainscomment
    satpy_geo_21112022.zipMSGx.py, HIMAx.py, GOESx.py, MTGx.pysupport swath, YYYY/mm/DD dir. structure

    Note MTGx.py: Test version. For date/time option -t: use for minutes 00, 10, 20, .. 50


    LEO scripts

    These scripts use functions from common scripts. There is a separate file where you must define your personal settings.

    Updated 14-11-2022: satpy_functions_14112022.zip, satpy_settings.py
    zip filecontainscomment
    satpy_leo_02122022.zipMetop.py, AquaTerra.py, FY3D.py, NOAA.py, Sentinel3.pysupport swath, YYYY/mm/DD dir. structure
    satpy_functions_02122022.zipsatpy_areas.py satpy_common.pyfunctions needed by the leo scripts
    satpy_settings.pySettings needed for LEO scriptsalso contains satpy dir. structure (Linux/Windows switch auto)

    Usage

     ./<script>.py [-h] [-t T] [-sat SAT] [-src SRC] [-dst DST] [-sp SP] [-fl FL]  [-o O] [-area AREA] [-composite COMPOSITE] [-tle TLE] [-histtle] [-v] [-dry]
    
    
    options:
      -h, --help            show this help message and exit
      -t T                  time: YYYYmmDD
      -sat SAT              A[D|N]] | B[D|N] | C[D|N] | X[D|N]] :X=B+C, D=Day, N=Night
      -src SRC              source dir., MUST be absolute path!
      -dst DST              destination dir.
      -sp [SP]              stacked pass
      -fl FL                file with list of files
      -o O                  output file
      -area AREA            area, 'list' gives a list
      -composite COMPOSITE  composite, 'list' gives a list
      -tle TLE              tle file, must contain tle elements of used satellite
      -histtle [HISTTLE]    get historical tle from spacetrack
      -v [V]                start viewer; -v : specify viewer
      -dry [DRY]            dry run; -dry f: show used files only
    
    for Metop:
      -sat SAT              A[D|N]] | B[D|N] | C[D|N]] | X[D|N] : X=B+C, D=Day, N=Night
    for NOAA:
      -sat SAT              N20[D|N]] | SUO[D|N] : D=Day, N=Night; SUO=Suomi, N20=NOAA20
    for Aqua/Terra:
      -sat SAT              A[D|N]] | T[D|N]: A=Aqua, T=Terra, D=Day, N=Night
    for Sentinel:
      -sat SAT              A[D|N]] | B[D|N] | X[D|N] : X=A+B, D=Day, N=Night
    
    Note that these scripts all support day/night and stacked, in all combinations.

    Usage of TLE's

    There are several ways to get/use tle files, needed for polar satellites to select the right files. Note that these tle's are not used for gridding etc.; satpy uses position info present in the data files. So accuracy (i.e., age tle's) is not that important.

    Parameters used to determine needed files are center position of needed area (lat,lon) and a figure to determine how big the area is: ran. This info is extracted from the areas.yaml files.
    Note:
    For ran, a calculation is done using parameters lower_left_xy and upper_right_xy. This calculation may need some fine tuning, especially for area's near the poles! To get access to space-track you need to create an account at
    this site. You need to create a config file:
    with following content:
    [configuration]
    username = <username>
    password = <password>
    
    Put this file in directory toodrp + '/EMCtools/pppconfig'
    Note that space-track is only used if tle with the right day isn't present in your tle file.


    Preparations LEO

    In satpy_settings.py: In other scripts Note: Paired satellites: Choose for satellite X instead of B or C resp. A or B

    Examples

    python3 Metop.py -t 20220324 -dry                           dry run, show settings (in this case: the defaults)
    python3 Metop.py -t 20220324 -dry f                         dry run, show also files selected
    python3 Metop.py -t 20220324 -sat BD                        use Metop-B, day pass
    python3 Metop.py -t 20220324 -sat BD -area westminster      same, choose area
    python3 Metop.py -t 20220324 -sat BD -composite overview    same, choose composite
     
    python3 Metop.py -t 20220324 -sat XN                        use combined Metop-B and C, night pass
    python3 Metop.py -t 20220324 -sat BD -sp                    use Metop-B multiple passes
    
    python3 Metop.py -t 20220324 -sat BD -src C:\aaa\bbb        use source files in this location
    python3 Metop.py -t 20220324 -sat BD -dst C:                location of output file
    python3 Metop.py -t 20220324 -sat BD -o anc.jpg             name output file
    python3 Metop.py -t 20220324 -sat BD -tle C:\weathet.txt    TLE location and file
    python3 Metop.py -t 20150324 -sat BD -histtle               get/use tle from spacetrack
    
    python3 Metop.py -sat BD -fl filelist.txt                   Use files in filelist instead of collected items by date (no -t option)
    

    Creating a movie from multiple jpegs

    Here are some experimental scripts to create a movie. Functions are: Needs: conda install -c fastai opencv-python-headless (use pip if you don't use conda)
  • movie_funcs.py
  • example_movie.py

    Note: You can specify codec, default is DIVX. For some reason WebM (VP09) doesn't work.


    Creating and extending a movie 'live'

  • live_movie_msg.py

    Running this script will create a movie and update it while MSG data is coming in.
    Uses MSGx.py and movie_funcs.py
    Needs: conda install watchdog (use pip if you don't use conda)

    At the top of this file, specify:

    Tested under Linux, and partly under Windows (watchdog and movie generation)

    Selector for satpy

    NOTE: needs to be tested on Windows!

    Updated 14-11-2022

    This selector is a simple gui using above scripts (both GEO and LEO) You need also: You must add locations of your raw files into satpy_settings.py. Selecting one of those file locations will be done using an argument at start, e.g.:
    python3 satpy_selector.py msg
    
    Selection may also be done in the gui.

    This gives the gui: (geo and leo)

    Now:

    Best is to start this in a command shell so you can see (error-)messages generated by the activated scripts.

    satpy_selector.py contains 2 lists of satellites including the command to execute them:

    Don't change these lists. Note variable exe_cmd. You can choose here 2 methods to execute the scripts:

    satpy_selector.py uses lists defined in satpy_settings.py:

    Note: For directories, for windows, use '/', not '\'.


    Location of files

    Location satpy_selector.py: same as for Metop.py etc.

    Problems with tkinter

    If your OS is Ubuntu then try this:
  • sudo apt-get install python3-tk
    See also https://realpython.com/python-gui-tkinter

    Using raspberry Pi

    Some scripts are tested on a raspberry pi, pi-4 with 8 GByte RAM. Using a Pi only makes sense if you can also get the data to the Pi.
    With Eumetcast this isn't possible (at least, directly) because you would need Tellique software which isn't available for ARM based PC's.
    Using Eumetsat Data Store, so data from internet, this is possible; also see
    Python eds scripts for downloading these files.
    What works: What does not work: About Metop files (AVHR*):
    Another difficulty with these big Metop files is that the selection of files as it is now implemented doesn't work because the range of each file is so big. Needs to be adapted (in case these Metop files need to be processed on a PC, as said, a Pi is just not powerfull enough).