Hide navigation sidebar
Hide table of contents sidebar
Back to top
Toggle table of contents sidebar

# sceewlog

Logs EEW magnitude messages via file, email, ActiveMQ and FCM.

## Description

Part of the EEW package (used by VS and FinDer).

sceewlog logs the VS and FinDer magnitude messages received from scvsmag and scfinder (configurable), broadcasts these alerts, and triggers scripts, generates and disseminates reports.

sceewlog is released under the GNU Affero General Public License (Free Software Foundation, version 3 or later). It requires the Python 3 and the package dateutil [2] to be installed. sceewlog replaces scvslog that was originally part of the SeisComP implementation of the Virtual Seismologist [1] (VS) Earthquake Early Warning algorithm.

Once an event has timed out, the modules generates report files. These report files are saved to disk and can also be sent via email.

It also implements ActiveMQ [3] and Firebase Cloud Messaging [8] interfaces which can send alert messages in real-time. It supports regionalized filter profiles (based on polygons and EQ parameter threshold values). Currently, messages can be sent in four different formats (sc3ml, qml1.2-rt, shakealertml the UserDisplay format, CAP1.2). Optionally, a comment with ID EEW storing the EEW update number, can be added to each magnitude that passes all configured filters and scoring.

The SED-ETHZ team provides a client that can display qml1.2-rt messages sent via ActiveMQ, the Earthquake Early Warning Display (EEWD) [4] an OpenSource user interface developed within the European REAKT project and based on early versions of the ShakeAlert UserDisplay [5].

To receive alerts with the EEWD set the format to qml1.2-rt. There are currently no clients which can digest sc3ml and the shakealertml is not supported. Using pipelines alerts can be sent out in more than one format.

The real-time ActiveMQ interface requires the Python packages stompy [6] and lxml [7] to be installed.

It is beyond the scope of this documentation to explain the complete setup of an ActiveMQ broker. However, since sceewlog uses the STOMP protocol to send messages to the broker it is essential to add the following line to configuration of the ActiveMQ broker.

    <connector>
    <serverTransport uri="stomp://your-server-name:your-port"/>
    </connector>

Please refer to ActiveMQ [3] for setting up an ActiveMQ broker.

## Reports

Below is an example of the first few lines of a report file:

                                                                       |#St.   |
    Tdiff |Type|Mag.|Lat.  |Lon.   |Depth |origin time (UTC)      |Lik.|Or.|Ma.|Str.|Len. |Author   |Creation t.            |Tdiff(current o.)
    ------------------------------------------------------------------------------------------------------------------------------------------
      5.24| MVS|2.40| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  2|    |     |scvsmag2@|2020-06-23T06:25:45.99Z|  7.44
      6.24| MVS|3.69| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  4|    |     |scvsmag2@|2020-06-23T06:25:46.99Z|  8.45
      6.79| MVS|3.71| 46.05|   6.89| 20.53|2020-06-23T06:25:38.55Z|0.40|  4|  3|    |     |scvsmag@s|2020-06-23T06:25:47.54Z|  8.99
      7.24| MVS|3.65| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag2@|2020-06-23T06:25:48.00Z|  9.67
      7.79| MVS|3.53| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag@s|2020-06-23T06:25:48.54Z| 10.21
      8.24| MVS|3.61| 46.05|   6.89| 22.30|2020-06-23T06:25:38.33Z|0.99|  6|  5|    |     |scvsmag2@|2020-06-23T06:25:48.99Z| 10.66
      8.62| Mfd|4.00| 46.04|   6.88|  5.00|2020-06-23T06:25:41.93Z|0.88|  0|   |  80| 0.28|scfdalpin|2020-06-23T06:25:49.37Z|  7.44
      8.62| Mfd|3.90| 46.04|   6.88| 12.00|2020-06-23T06:25:40.29Z|0.85|  0|   | 140| 0.38|scfdforel|2020-06-23T06:25:49.37Z|  9.07

Creation time is the time the VS magnitude message was generated, tdiff is the time difference between creation time and last origin time in seconds, lik. is the likelihood that this event is a real event (see documentation of scvsmag), #St.(Or.) is the number of stations that contributed to the origin and #St.(Ma.) the number of envelope streams that contributed to the magnitude. Str. and Len. are the strike and length of the fault line provided by scfinder.

## Regionalized Filters

To filter alerts to be sent out through ActiveMQ or FCM, it is necessary to set profiles on RegFilters section. Since this is using regions defined as closed polygons, then the first step is to provide a BNA file that contains the polygons. If the user does not provide a BNA file, then the other profile parameters will be evaluated globally.

    RegFilters.bnaFile = /opt/seiscomp/share/sceewlog/closedpolygons.bna

Then profile names have to be set. Two profile examples are provided below.

    RegFilters.profiles = global, America

The global profile is not configured with polygon since this spans on the entire world. The America profile uses the “America” closed polygon defined in `activeMQ.bnaFile`.

    RegFilters.profile.global.bnaPolygonName = none
    RegFilters.profile.America.bnaPolygonName = America

The magnitude and likelihood threshold values might be:

    RegFilters.profile.global.magThresh = 6.0
    RegFilters.profile.global.likelihoodThresh = 0.5
    RegFilters.profile.America.magThresh = 5.0
    RegFilters.profile.America.likelihoodThresh = 0.3

There might also be a depth filter for each profile. The following parameters might be used to configure the global profile with shallow events, and the America profile with events from 0 to 100 km deep.

    RegFilters.profile.global.minDepth = 0
    RegFilters.profile.global.maxDepth = 33
    RegFilters.profile.America.minDepth = 0
    RegFilters.profile.America.maxDepth = 100

Finally, to avoid sending alerts for events outside of the network of interest for EEW applications, a `maxTime` can be set. The `maxTime` is the maximum delay in seconds between the magnitude creation time since the origin time. For the examples, on the global profile this parameter might be “-1” in order to skip this filter, whereas it could be set to 60 seconds for America. However, each of the Virtual Seismologist and Finite-Fault Rupture Detector algorithms have their own default thresholds superseding `maxTime` defined in sceewlog.

    RegFilters.profile.global.maxTime = -1
    RegFilters.profile.America.maxTime = 60

## Magnitude Association

The magnitude association is similar than the scevent preferred-origin selection. The magnitude association evaluation is activated with the next configuration key-value pair:

    magAssociation.activate = true

The following priorities are available:

1. typeThresh

2. likelihood

3. Authors

4. StationMagNumber

and they can be listed on the next key-value:

    magAssociation.priority = magThresh,likelihood,authors,stationMagNumber

The priority decreases in the order of the parameters.

The magThresh prority is a list of minimal magnitude to be allowed for each type of magnitude:

    magAssociation.typeThresh = Mfd:6.0,MVS:3.5,Mlv:2.5

For one specific update of an event, its magnitude type-value must be equal or higher than the listed ones to continue the evaluation. Otherwise, the evaluation ends and the evalution does not continue.

The authors can be also used and their priority depends on the position on which they are listed. For example:

    #if magAssociation.priority contains authors then
    #the next parameter must contain valid magnitude authors' names
    magAssociation.authors = scvsmag@@@hostname@, \
    scvsmag0@@@hostname@, \
    scfd85sym@@@hostname@, \
    scfd20asym@@@hostname@, \
    scfdcrust@@@hostname@

In this list of authors the highest value is for scvsmag and for this particular example this author has a value of 5. The author value reduces after each comma separator. For the same example scvsmag0 is 4, scfd85sym is 3, and so. The author is evaluated for every event update. If the author for a particular update has a higher o equal priority than an already evaluated and sent alert, then the evaluation continues, otherwise, whether the author is listed or not, the evaluation ends.

If likelihood is listed on priorities then this is evaluated against with a previous update that has been sent out as alert. Only when there is a first event update the likelihood is not evaluated. In case that an update has been sent out as an alert, if the current likelihood is higher or equal than the previous one, then it goes to the next priority evaluation, otherwise, the update is discarted.

For the stationMagNumber evaluation is necessary to have a similar list of Magtype and min number of stations used to quantify the magnitude value. For example:

    magAssociation.stationMagNumber = Mfd:7,MVS:4

In this example for the mag type Mfd is necessary to at least have 7 stations to pass this evaluation, otherwise, the evaluation ends. For MVS must be at least 4 stations that have been used to qunatify the magnitude, otherwise, the valution ends.

## Firebase Cloud Messaging

EEW messages are controlled by the parameters Magnitude Association and Regionalized Profiles. Notifications are sent using the HTTP v1 [9] when topic notification option is enabled and it requires the following python libraries:

    google
    requests
    google-auth-oauthlib
    firebase-admin

Further information on the Firebase Cloud Messaging interface can be found in Firebase Cloud Messaging [8]. This interface is activated in the module configuration file with the option:

    FCM.activate = true

When this is enable then two options are possible:

1. For sending push notification to a topic:

    FCM.topicnotification = true

In this case the configuration file defined in FCM.dataFile must be provided and containing the firebase configuration. Additionally, the python libraries mentioned above must be installed. The configuration file includes the path to a service JSON (see the Firebase project console Service JSON File [10]), the broadcasted topic (notification topic), the project ID string (see Project ID [11]), and it controls (enable or disable) notification to Android, iOS and a legacy notification format. Below is an example how this file, referenced in the FCM.dataFile, looks like:

    [TOPICS]
    topic=eqAlerts
    [SERVICEFILE]
    servicefile= /opt/fcm/credentials/projectServiceFile.json
    [PROJECTID]
    projectid = myappid
    [ENABLED_OS]
    android = true
    ios = true
    [SUPPORT_OLD_FORMAT]
    oldformat = true

2. For creating an eew message dictionary:

    FCM.eewmessagecomment = true

In this case this message contains the EQ origin information. It will be added as a comment type: eewmessage and text will be the message dictionary. In addition to this, the comment will be sent to the messaging system so that any other module can listen to this comment.

## Headline Change for CAP1.2 XML alerts

The converted CAP1.2 xml alert messages contain a headline. The default headline is:

    @AGENCY@ Magnitude X.X Date and Time (UTC): YYYY-MM-dd HH:mm:s.sssZ.

An alternative headline format might be preferred. The following alternative format can be selected:

    @AGENCY@/Earthquake Magnitude X.X, XX km NNW of SOMECITY, SOMECOUNTRY

The aternative format supports both spanish and english languages. The spanish version is:

    @AGENCY@/Sismo Magnitud X.X, XX km al SSO de SOMECITY, SOMECOUNTRY

The alternative format can be enable as follows:

    CAPheadlineChange = true

The alternative format requires to specify the language and the corresponding file listing the world cities `CitiesFile`. The language can be selected as follows:

    #Uncomment the next line to select English
    AlertTextLanguage = en-US
    #Uncomment the next line to select Spanish
    #AlertTextLanguage = es-US

The file listing the world cities `CitiesFile` must have the following format:

    city,country,lon,lat
    Tokyo,Japan,139.6922,35.6897
    Jakarta,Indonesia,106.8451,-6.2146
    Delhi,India,77.23,28.66
    Mumbai,India,72.8333,18.9667
    Manila,Philippines,120.9833,14.6
    Shanghai,China,121.4667,31.1667
    Sao Paulo,Brazil,-46.6339,-23.5504

Both an english and a spanish version are provided in “@DATADIR@/sceewlog/world_cities_english.csv” and “@DATADIR@/sceewlog/world_cities_spanish.csv”.

## References

[1]

http://www.seismo.ethz.ch/en/knowledge/earthquake-data-and-analysis-tools/EEW/Virtual-Seismologist/index.html

[2]

https://pypi.python.org/pypi/python-dateutil/

[3] (1,2)

http://activemq.apache.org/

[4]

http://www.seismo.ethz.ch/en/knowledge/earthquake-data-and-analysis-tools/EEW/earthquake-early-warning-display-eewd/

[5]

http://www.eew.caltech.edu/research/userdisplay.html

[6]

https://pypi.python.org/pypi/stompy/

[7]

http://lxml.de/

[8] (1,2)

https://firebase.google.com/docs/cloud-messaging

[9]

https://firebase.google.com/docs/cloud-messaging/migrate-v1?hl=es-419

[10]

https://console.firebase.google.com/u/0/project/_/settings/serviceaccounts/adminsdk

[11]

https://console.firebase.google.com/u/0/project/_/settings/general/

## Configuration

`etc/defaults/global.cfg`
`etc/defaults/sceewlog.cfg`
`etc/global.cfg`
`etc/sceewlog.cfg`
`~/.seiscomp/global.cfg`
`~/.seiscomp/sceewlog.cfg`

sceewlog inherits global options.

magTypes

  Type: list:string

  The list of magnitude types handled by sceewlog Default is `MVS, Mfd`.

generateReportTimeout

  Type: int

  Timeout after which a report is generated. The timer is started on magnitude reception and restarted whenerver a new magnitude is received for the same event. Default is `5`.

CitiesFile

  Type: string

  CSV file that contains the location of cities. It is mandatory when changeHeadline or FCM is true. Default is `@ROOTDIR@/share/sceewlog/world_cities_english.csv`.

EEW.comment

  Type: boolean

  Save a comment for each EEW alert sent out through ActiveMQ or FCM. The comment is part of the Magnitude object and stored on the Comment table of the SC-DB. Default is `false`.

EEW.script

  Type: string

  Executes an external script for each EEW alert that are sent out. The script must be defined with full file path, and it must be executable. It receives all EEW parameters in arguments as follows: magID=.. type=M.. author=.. magnitude=.. lat=.. lon=.. depth=.. nstorg=.. nstmag=.. ot=.. When the module is started, the script is tested with argument test=1. An example of a script preparing a data file for posting on social media is provided with $SEISCOMP_ROOT/bin/eew2file. An example of a script that can be adjusted to run Shakemap from EEW solutions is provided with $SEISCOMP_ROOT/bin/eew2shakemap.

CAPheadlineChange

  Type: boolean

  Change the headline in the cap1.2 XML alert. Default is `false`.

AlertTextLanguage

  Type: string

  Language for the string text on headline (cap1.2 XML) or FCM near location. Default is `en-US`.

email.activate

  Type: boolean

  Send email notifications. Default is `true`.

email.smtpserver

  Type: string

  URL of the smtp server to send the report files to.

email.port

  Type: int

  Port where the SMTP server accepts connections. Default is `25`.

email.usetls

  Type: boolean

  Whether to use TLS when connecting to the smtp server. Default is `false`.

email.usessl

  Type: boolean

  Whether to use SSL when connecting to the smtp server. Note, only ‘email.usetls’ or ‘user.ssl’ can be true. Default is `false`.

email.authenticate

  Type: boolean

  Whether the smtp server requires authentication (username + password). Default is `false`.

email.credentials

  Type: file

  If the smtp server requires authentication you have to specify a file that contains username and password in the format:

  username=your-username

  password=your-password

  Make sure that you set the file permission as restrictive as possible.

email.senderaddress

  Type: string

  Email address that will appear as sender in the report email.

email.subject

  Type: string

  Any string that should be prepended to the email’s subject string.

email.host

  Type: string

  Host as it is supposed to appear in the email’s subject string.

email.recipients

  Type: list:string

  A list of email addresses that receive the report emails.

email.magThresh

  Type: double

  Only send a notification email if the magnitude threshold is exceeded. Default is `0.0`.

email.sendForAlertOnly

  Type: boolean

  Only send a notification email if an alert was sent (activeMQ, FCM, or script). Default is `false`.

report.activate

  Type: boolean

  Save reports to disk. Default is `true`.

report.eventbuffer

  Type: int

  Unit: s

  Time in seconds that events and the related objects are buffered. Default is `600`.

report.directory

  Type: dir

  Directory to save reports to. Default is `~/.seiscomp/log/EEW_reports`.

ActiveMQ.activate

  Type: boolean

  Send Alerts through ActiveMQ msg broker. Default is `true`.

ActiveMQ.hostname

  Type: string

  Server name that runs the ActiveMQ broker. Default is `localhost`.

ActiveMQ.port

  Type: int

  Port on which ActiveMQ broker accepts stomp messages. Default is `61618`.

ActiveMQ.username

  Type: string

  Username with which messages are sent. Default is `your-ActiveMQ-username`.

ActiveMQ.password

  Type: string

  Password with which messages are sent. Default is `your-ActiveMQ-password`.

ActiveMQ.topic

  Type: string

  ActiveMQ broker topic to send alerts to. Default is `/topic/your-ActiveMQ-topic`.

ActiveMQ.hbtopic

  Type: string

  ActiveMQ broker topic to send heart beats to. Default is `/topic/your-ActiveMQ-topic`.

ActiveMQ.messageFormat

  Type: string

  ActiveMQ message format; can be either ‘cap1.2’, ‘qml1.2-rt’, ‘shakealert’, or ‘sc3ml’. Default is `qml1.2-rt`.

RegFilters.profiles

  Type: string

  Profile names for regionalized alert filter. Use comma for more than one profile. Default is `global`.

RegFilters.bnaFile

  Type: string

  Path of BNA file that contains closed polygons which is used for regionalized filters. Default is `none`.

Note

RegFilters.profile.* Profiles containing the profile-specific alert filtering parameters.

Note

RegFilters.profile.$name.* $name is a placeholder for the name to be used and needs to be added to `RegFilters.profiles` to become active.

    RegFilters.profiles = a,b
    RegFilters.profile.a.value1 = ...
    RegFilters.profile.b.value1 = ...
    # c is not active because it has not been added
    # to the list of RegFilters.profiles
    RegFilters.profile.c.value1 = ...

RegFilters.profile.$name.magThresh

  Type: double

  Magnitude threshold; can be any float value. Default is `0.0`.

RegFilters.profile.$name.likelihoodThresh

  Type: double

  Likelihood threshold; can be between 0.0 to 1.0. Default is `0.0`.

RegFilters.profile.$name.minDepth

  Type: double

  Minimum depth value; can be any float value but less than maxDepth. Default is `0.0`.

RegFilters.profile.$name.maxDepth

  Type: double

  Maximum depth value; can be any float value but greater than minDepth. Default is `800.0`.

RegFilters.profile.$name.bnaPolygonName

  Type: string

  The polygon name must be one of the closed polygons that contains the ActiveMQ.bnaFile. If this is set to none, then the origin location will not be checked against any polygon. Default is `none`.

RegFilters.profile.$name.maxTime

  Type: double

  Time difference between origin time and magnitude creation time (in seconds). If this parameter is setup to -1, the time difference is not checked. Default is `-1`.

magAssociation.activate

  Type: boolean

  Magnitude association and scoring. The score for each EQ update is the product of magnitude, likelihood, author and num. of arrivals. Default is `false`.

magAssociation.priority

  Type: string

  The priorities can be set as: magThresh, likelihood or authors. It can be all together or just one or two. Listed comma separated. Default is `magThresh,likelihood`.

magAssociation.typeThresh

  Type: string

  Set the minimum magnitude threshold values for each magnitude type configured in magTypes. Default is `Mfd:6,MVS:3.5`.

magAssociation.authors

  Type: string

  Set the magnitude authors in a list comma separated. This should be generally the app name followed by the hostname. Default is `scvsmag@@@hostname@,scfinder@@@hostname@`.

magAssociation.stationMagNumber

  Type: string

  Set the minimum number of stations on which magnitude was quantified for every magnitude type. Default is `Mfd:3,MVS:1`.

FCM.activate

  Type: boolean

  Enable or disable FCM for using the eews2fcm interface. Default is `false`.

FCM.topicnofication

  Type: boolean

  Enable or disable FCM for topic notifications. It this case it is mandatory to provide the dataFile. Default is `false`.

FCM.eewmessagecomment

  Type: boolean

  Enable or disable this option to create a message dictionary that will be notify to the messaging system and persist it in the DB. Default is `false`.

FCM.dataFile

  Type: string

  The datafile must contain the authorization key and topic to send out a message preformated on the eews2fcm interface. Default is `/home/sysop/seiscomp/share/sceewlog/.fcmdatafile`.

## Command Line

### Generic

### Verbosity

### Messaging

### Reports

--savedir path

  Directory to save reports to.

### Mode

--playback

  Disable database connection.