Change Log

« Back to User Guide

NOTE: change logs for versions after 1.11.0 are now maintained as github release notes: https://github.com/usgs/pdl/releases

Version 1.11.0 2017-01-11

Update authoritative region for "hv" contributor.

Version 1.10.1 2016-04-07

Add "pdl-client-version" property to products sent using ProductBuilder (which includes EIDSInputWedge) or CLIProductBuilder
Update moment tensor indexer module to use derived-magnitude-type property.
Add support for java's "simple" and "xml" log formatters (as an alternative to the default "pdl" formatter).

Version 1.10.0 2016-02-11

Improved support for internal-TYPE, TYPE-scenario, and internal-TYPE-scenario product variants.
Updated how ShakeMap properties are handled when received as both product properties and specified in XML.
Implemented customized algorithm for computing preferred weight on moment tensor products.

Version 1.9.7 2015-04-30

Fix broken link in 1.9.6 change log.
Update QuakemlProductCreator handling of focalMechanism element: add support for catalog:datatype; delete product when evaluation status is rejected; require nodal-planes for focal-mechanism; require tensor for moment-tensor.

Version 1.9.6 2015-04-29

Add support for separate keychain configuration file, see ProductStorage configuration.
Update init script to wait up to 15 seconds for process to stop.
Update MessageUtils to latest version, supports MWW/MWP mapping to CUBE magnitude type "O".

Version 1.9.5 2015-04-06

Update EIDSInputWedge to delete phase-data and origin product when a quakeml delete message is processed.
Update which event codes are considered to be active (some were hidden by deleted origins) when evaluating event associations.
Add support for persistent trump to Indexer.
Disable product tracker by default in embedded configuration.
Fix broken link to contents.xsd in user guide.

Version 1.9.4 2015-03-18

Preserve sender precision in arguments passed to listeners.
Added --test argument to EIDSInputWedge to preview generated products.
Preserve sender formatting (exponential notation) of moment tensor and axis values
Add configurable size limit to socket product receiver
Add "No space left on device" exception to troubleshooting page
Disable product tracker by default.
Fix logic error in polling time calculation for EIDSInputWedge
Improve indexer exception handling (when event unable to update, don't mark product as processed).
Archive products and events inside transactions
Fix quakeml based product properties to use quakeml string with spaces, instead of java enumeration with underscores
Update current/superseded product definition, current now includes most recent non-delete version of product which may define event.
Eliminate repeated selection of preferred origin product
Added associateUsingCurrentProducts option to indexer, improves indexing when event has hundreds of old versions of products
Better log messages when there are errors parsing CUBE messages.
Update tectonic summary indexer module to prefer reviewed summaries
Remove NE authoritative polygon from regions.xml
Update documentation of associate and link products
Log warning when tracker is enabled.
Enable indexer listener options ignoreArchive and autoArchive by default
Add optional "roundrobin" listener notifier.

Version 1.9.3 2014-09-19

Fix bug using EQMessageProductCreator parser with EIDSInputWedge

Version 1.9.2 2014-09-17

QuakemlProductCreator
- Make moment-tensor and focal-mechanism depth property kilometers instead of meters.
- Send impact-text product for "felt report" event description.
Update indexer handling of event splits to only consider latest version of products.
Use standard list of mime types for file extensions "atom", "css", "gif", "gz", "html", "jpg", "js", "json", "kml", "kmz", "pdf", "png", "ps", "txt", "xml", and "zip".
NotificationReceiver
- Update to use "storageAge", and deprecated "storageage".
- Changed storageAge default from 15 minutes to 1 hour
Add batch loading support to ProductResender

Version 1.9.1 2014-06-04

Fix bug introduced in 1.9.0 for selection of preferred origin product.

Version 1.9.0 2014-05-27

Updated indexer handling of preferred event products and properties:
- Minimum properties to create event are id (eventsource, eventsourcecode), location (latitude, longitude), and time (eventtime).
- Preferred origin-like product is used for all properties including lat/lng/time/mag/depth.
More explicit exception message when origin uncertainty preferred description is "horizontal uncertainty" without a horizontal uncertainty value.
FileProductStorage accesses product contents more reliably, especially when storage directory is moved and reconfigured.
Set origin product "title" property when Quakeml "earthquake name" event description is included.
Added pollCarefully option to EIDSInputWedge, ensures files are at least "interval" milliseconds old before processing; for producers who must write files directly into poll directory (instead of moving into place).
Log java environment properties at startup, at CONFIG log level.

Version 1.8.9 2014-01-29

Added "enablePdlProtocol" option (default true) to SocketProductSender, to explicitly check if hub has product before sending. Other exceptions that occur are now better reported.

Version 1.8.8 2013-11-25

Added writeTimeout configuration for SocketProductSender.
Updated to new MessageUtils.jar with fix for null pointer exception when converting quakeml to cube, and quakeml doesn't include originUncertainty.

Version 1.8.7 2013-11-22

Added readTimeout configuration for SocketProductSender and SocketProductReceiver.
Updated indexer handling of deleted products to
- Prefer non-deleted products when determining preferred event id.
- Ignore deleted products when evaluating possible event merges. This makes the indexer automatically merge an event that had split due to multiple event ids from the same source, by deleting the products with offending event ids.
- Handle null pointer exception when deleting a trump product that references an already deleted product.
Changed EIDSClient restart to call underlying QWConnectionMgr.reinitConnection(), instead of creating new client.
Updated to new MessageUtils.jar with support for internal and scenario Quakeml products.

Version 1.8.6 2013-11-18

Added includePaths, excludePaths configuration properties to DefaultNotificationListener.
Added legacyStorages configuration property to FileProductStorage, to support migration between incompatible product storage configurations.
Added autoArchive configuration property to ExternalIndexerListener
Fixed configuration bugs for
- AbstractListener retryDelay property
- EIDSInputWedge sendOriginWhenPhasesExist, sendMechanismWhenPhasesExist, createInternalProducts, and createScenarioProducts properties
- HeartbeatListener heartbeatFile and storageTimeout properties
Added --disableDeflate argument to CLIProductBuilder (ProductClient --send option). Updated command line arguments documentation.
Added --disableDeflate and --binaryFormat arguments to ProductResender.
Moved auto-detect of format and compression to IOUtil, and updated EIDSNotificationReceiver to use same logic as SocketProductReceiver.

Version 1.8.5 2013-11-15

Updated SocketProductSender
- Added enableDeflate and deflateLevel options.
- Documented binaryFormat option.
- Added utility for testing deflate options, described in Sending > Compression
Updated SocketProductReceiver to auto-detect format and compression, so clients can choose own compression and format settings
Set sun.awt.X11.timeoutTask.XToolkit logging level to INFO

Version 1.8.4 2013-11-05

Updated FileContent to escape base directory paths using java.util.regex.Pattern#escape . Directories with special regex characters (+,*,?,etc...) were not matching correctly.

Version 1.8.3 2013-11-05

Changed java.io.File#getAbsolute(Path|File) to getCanonical(Path|File), to fix bug in sending products using a path containing "..".
Added embedded PDL client example, gov.usgs.earthquake.distribution.EmbeddedPDLClient
Fixed configuration bug in ContentListener, for outputFormat and tempDirectory. Added description to configuration.html.
Updated install.html description of crontab PATH configuration.
Changed ShakeMap Product source description from PN to UW.

Version 1.8.2 2013-09-26

Updated MessageUtils library, with safer null handling for quakeml and eqxml to cube conversions.

Version 1.8.1 2013-09-17

EIDSInputWedge improvements
- Added "--internal" and "--scenario" options to EIDSInputWedge
- Preserve original message content when input is EQXML, or Quakeml (without custom parser argument).
- Added "--validate" option
- Added "--sendMechanismWhenPhasesExist" argument; changed default behavior to not send mechanism products when phase-data exists, to be consistent with --sendOriginWhenPhasesExist.
- Added config file options for validate, sendOriginWhenPhasesExist, sendMechanismWhenPhasesExist.
- Throw explicit ConfigurationException when parser is incorrectly configured.
QuakemlProductCreator improvements
- Send "broadband-depth" type product, instead of "moment-tensor", when moment tensor's derived magnitude type is "Mwd".
- Updated how lightweight origin, focal-mechanism, moment-tensor, and broadband-depth products are created.
Added example external listener scripts to distribution, and added ExternalIndexerListener support to python example.
Added ContentListener, that can be configured to listen for specific content paths in products; like "quakeml.xml".
Added "depth-type" property to origin and phase-data products.
Updated log message when product download from hub fails.
Updated log message when indexer has multiple candidate events, which may require manual review of associations.

Version 1.8.0 2013-07-30

added "retryDelay" configuration setting for listeners, used in combination with "maxTries"
added methods for programmatic configuration of listener properties "includeInternals", "includeScenarios", "includeTests", and "includeDevelopments"
fixed association issue near date line.
added "connect timeout" warning when EIDSInputWedge is unable to connect to a server.
fixed indexer archive policy bug that orphaned superseded and deleted products.
fixed indexer delete processing bug that loaded all events into memory when delete did not include event id.

Version 1.7.9 2013-03-12

Added utility that can resend existing products.
Updated QWFileOutClient.jar dependency from version 1.5 to 1.7
Updated MessageUtils.jar dependency with latest updates to quakeml 1.2
Updated conversion from Quakeml to Product:
- Set "review-status" property for focal-mechanism and moment-tensor products
- Set "eventParametersPublicID" property for all products
Updated EIDSInputWedge command line usage to include "--sendOriginWhenPhasesExist" argument.

Version 1.7.8 2013-02-25

Revert to previous version of SQLite library.

Version 1.7.7 2013-02-12

Fix for bug in ProductArchivePolicy that caused new versions of products to disassociate from their event, when an old version of the product was archived.
Updated api default for onlySuperseded in ProductArchivePolicy to match configuration default.
Changes to QWEmbeddedClient to suppress startup and shutdown messages, and attempt to reduce memory footprint.

Version 1.7.6 2012-12-11

Throw configuration exception when a sender is not configured correctly.
Using github eqmessageutils for quakeml parsing ( http://github.com/usgs/eqmessageutils ).
Updates for Quakeml to Product conversion:
- Fixed check and removal of phase data from "phase-data" xml when creating lightweight "origin". Also removes arrival, amplitude, stationMagnitude, stationMagnitudeContribution, in addition to picks.
- <focalMechanism> element processing:
  - "derived-depth" property as kilometers (instead of meters)
  - "beachball-source" property read from focalMechanism/creationInfo/agencyID, if it exists.
  - When <focalMechanism> element exists: "moment-tensor" product created when <tensor> element exists; otherwise, "focal-mechanism" product created when <nodalPlanes> element exists; otherwise, no product created.
Updated ShakeMapIndexerModule to read maximum mmi value from info.xml. Updated onEventUpdate trigger to prefer ShakeMap maxmmi property over LossPAGER maxmmi property.
Indexer Archive Policy bug fixes:
- ProductArchivePolicy now extends ArchivePolicy, to support archiving products based on event attributes.
- ArchivePolicy corrected to also remove productSummary from index.
- ProductArchivePolicy now updates any associated events (and removes event if last associated product).
ArchivePolicy now supports ISO8601 or millisecond epoch timestamp for minEventTime,maxEventTime,minProductTime,maxProductTime configuration properties.
Fixed bug in SearchResponseParser when searching Indexer programatically.
Fixed bug in EIDSInputWedge, error directory wasn't being created.

Version 1.7.5 2012-05-31

Fixed memory leaks in JDBCConnection, JDBCNotificationIndex, and JDBCProductIndex. Dynamic Statements and ResultSets are now closed immediately after use.

Version 1.7.4 2012-05-18

Fixed bug in CLIProductBuilder which prevented command line product senders from sending links or properties with values containing equals (=)
New command executor for handling timeouts for ShakemapIndexerWedge
Updated NotificationIndex to reconnect to database when connection is interrupted.
Updated MessageUtils with latest Quakeml changes.
Added documentation of EQXML,Quakeml to Product conversions.
Added --attach argument for EIDSInputWedge, when running in push mode
Updated handling of Quakeml CreationInfo for determining product update time
Fixed NullPointerException processing EVENT_DELETED indexer events by DefaultIndexerListener (and ExternalIndexerListener).
Updated JDBCProductIndex to not clear event properties from the event table when an event is deleted.

Version 1.7.3 2012-03-26

Better archiving handling in ExternalIndexerListener.
Removed notification queues, supported by alternate listener index feature.

Version 1.7.2 2012-03-23

Corrected an indexer bug in choosing the preferred magnitude for an event.
Changed event handling so if an event ever had an origin, and all origins are deleted, it is considered deleted. This change does not affect events with no origin product.

Indexer Users: SQL Script to fix existing events in SQLite or Mysql Index
For SQLite, be sure the ProductClient is not running.

Senders: Make sure senders and tracker url are properly configured in the config file or on the command line.

Version 1.7.1 2012-03-08

Added listener index option to more robustly process products.
Added indexer "trump" support.
Updated userguide based on user feedback.
Minor changes to QuakeML to Product converter.
EIDSNotificationSender now uses "listenerIndex" instead of "index" as its configuration property.

Version 1.7 2012-01-25

Fixed indexer handling of "associate" admin product, to force 2 events to associate.
Fixed shakemap eventsource and eventsourcecode handling: honor sender eventsource and eventsourcecode properties when they do not match grid.xml.
Fixed handling of inline content when using Directory product format.
Fixed argument parsing bug that ignored contentType for inline content.
Added ExternalIndexerListener property processOnlyWhenEventChanged
Incorporated updated JavaUtils, with URL timeouts of 15seconds for connect and read.
Added EIDSInputWedge for polling a directory for CUBE/EQXML/Quakeml
Added EIDSOutputWedge for outputting CUBE/EQXML/Quakeml to a directory
Incorporated updated MessageUtils, with quakeml support.
Added Quakeml to Product converter.
Updated EQXML to Product converter.
Added IndexerListener "processOnlyWhenEventChanged" option, to only trigger processing when an event's magnitude, latitude, longitude, depth, or time change.
Minor refactoring of IndexerListener and NotificationListener.
Added Bootstrap class to handle configuration and startup.

Version 1.6.5 - 2011-09-07

Added JMX MBean for monitoring queuing.

Version 1.6.4 - 2011-09-02

Removed synchronization around key pieces of ExecutorTask. This dead-locked leading to notification queuing.
Make connectTimeout and readTimeout configurable for receivers downloading products.
Fix for contentTypes bug in 1.6.3

Version 1.6.3 - 2011-07-26

Fixed bug in JDBCProductIndex search logic: when searching for minLongitude and maxLongitude, and minLongitude<maxLongitude, only minLongitude was included in the search.
Found bug where content types are not preserved, leading to potential problems with signatures. For now, all based contents mime types are labeled text/plain.

Version 1.6.2 - 2011-07-25

Updated ExternalIndexerListener to support same properties as DefaultNotificationListener. Added support for processPreferredOnly (which was previously ignored).
Fixed configuration for EIDSNotificationSender, RelayProductSender so DefaultNotificationListener properties are properly configured.
Added indexes to sqlite NotificationIndex.
Added contents.xml for EQXML derived products.
Removed sending of addon type products, addons are not products.

Version 1.6.1 - 2011-06-30

Fixed product tracker logging of listener exceptions
Added indexer ProductArchivePolicy, modified archive policy handling.
Updated indexer to use product source for authoritativeness (was event source).
Configuration options to include scenario, test, development, or internal products (now excluded by default).

Version 1.6 - 2011-06-16

Improved logging to include component name.
Fix for bug receiving product from multiple sources.
Fix for bug with listener includeTypes, excludeTypes, includeSources, and excludeSources
Changed indexer logic for detecting already processed products
Added indexes to SQLite ProductIndex

Version 1.5.2 - 2011-02-15

Better exception handling during startup and shutdown.

Version 1.5.1 - 2011-02-10

Improvements to indexer merge and split methods.
Better logging of sender exceptions.
New partial failure exit status (4), when at least one sender succeeds.
Oracle schema for JDBCProductIndex.

Version 1.5 - r???? - 2011-01-28

Updates to indexer
- New association algorithm. eventids are used before location in space and time.
- "associate" and "disassociate" type products to override automatic associations.
- Indexer events now have one or more associated changes.
- Added re-association step, to check for splits and merges. New EVENT_SPLIT and EVENT_MERGE change types.
- Indexer search interface, using XML via socket, or Command Line.
- PHP index search application.
Re-added cryptography libraries used for keypairs and signatures.
Minor changes to EQXML product generator.
- Addon and Comment products no longer include raw XML (magnitude and origin still do).
- raw XML now stored in "eqxml.xml" content, instead of empty path "".

Version 1.4.2 - r8665 - 2010-12-29

Escaping of properties that include XML unfriendly characters (", ', <, >, whitespace).

Version 1.4.1 - r8627 - 2010-12-20

Added ZabbixAgent for zabbix monitoring support.
Minor changes to ensure streams are closed asap.
Improvements in indexer stability. (still not production ready).

Version 1.4 - r7878 - 2010-10-28

Minimum version of Java changed to 1.6.
Split CLIProductBuilder --eventid argument into --eventsource and --eventsourcecode. While eventsource and eventsourcecode combine to form an eventid, they must be tracked separately so it is possible to tell when the same source has sent distinct events.
Initial implementation of Indexer. Not recommended for production yet.
- Core indexer framework.
- Basic product summaries for all products.
- ShakeMap product summaries.
- Adds products to index, and associates when possible.

Version 1.3 - r7736 - 2010-10-13

Updated bundled QWFileOutClient.jar to fix EIDS memory leak.
Updated EIDSClient to automatically reconnect once per day.
Updated DefaultNotificationReceiver to reject already expired notifications
Fix for FileContent when running on windows.

Version 1.2 - r6692 - 2010-08-27

Updated bundled usgs_utils.jar to fix XML datetime handling. All xml datetimes are now normalized to UTC.
New logging handler, one log file per day.
Added logdirectory configuration parameter, replaces logfile configuration parameter.