A Product is a group of files or content. Each product is uniquely identified by source, type, code, and update time. These key attributes are described as follows:
Products with the same source, type, and code, but different update times are considered different versions of the same product. Later updates to the same product replace previous versions, where older versions are retained in the database.
A product status may have different meanings depending on product type. "DELETE" always has the same meaning: that a product has been cancelled. Deleting a product is the same as sending a product, except the status is now "DELETE" and contents are optional. This version of the product is distributed the same way as any other update.
Properties are optional name/value pairs that describe the product. They are used to quickly identify characteristics of a product without needing to parse its content. Examples include PAGER "alertlevel", and origin "magnitude-type".
Links are optional and are a URI with a relation that describes how the resource is related to the product. Because the link is a URI, it can be either a URN or URL. One example would be including a "background" related link for a product. Another example would be identifying a related product that was used as an input.
A Notification is a small message with Product's source, type, code, update time, URL for download, and URL expiration date.
Product Distribution provides a standard solution for data distribution.
Data producers and consumers are loosely coupled. Producers broadcast a product, via redundant hubs, and all clients are notified it is available. Only clients that are interested download the full content for processing. This simplifies both sending and receiving data because all producers and consumers only need to communicate with the hubs. Because producers and consumers initiate connections to the hubs, most users do not need to reconfigure firewalls.
Producers use a private key to sign products when they are sent. Hubs verify this signature using a public key before distributing products to ensure only authorized producers are able to send products. Clients may optionally verify these signatures as well.
Clients can be configured to trigger processing when new products arrive, either using command line arguments or a Java API, eliminating polling latency. In many cases this has reduced processing time from minutes to seconds.
A NotificationReceiver receives Notifications for products and keeps a NotificationIndex of all received notifications. Whenever a notification is received, it is queued for processing by each NotificationListener. Each listener has a separate queue, and uses a separate thread to process queued notifications one at a time. This prevents a slow listener from blocking a fast listener.
When a listener processes a notification, it may request the product from the receiver. Any number of listeners may request the same product, but the receiver will only download once. Once the receiver has retrieved the product, the listener processes its contents.
Periodically, the receiver will remove expired notifications from its index, and downloaded products that are no longer needed. By default a receiver only keeps downloaded products for 15 minutes, just long enough for listeners to process, to minimize disk usage.
ExternalNotificationListeners process products by executing an external command line process for each received product. The ExternalNotificationListener does not associate related products together, for that functionality use the Indexer.
See Receiving Products for more information
The Indexer associates related products into events. The indexer uses authoritative regions to choose preferred products when multiple sources contribute the same type of product for the same event. Every time the Indexer changes the index, it notifies IndexerListeners about the change.
IndexerListeners are similar to NotificationListeners but listen to Indexer changes (which are triggered by products) instead of listening to notifications for products. Because the indexer keeps track of previously processed products, duplicate and non-preferred information can be filtered. Additionally, updates that do not change an event's location or magnitude can also be filtered.
See Indexer for more information
A SocketProductSender sends a product directly to a ProductHub, which stores the product and sends a Notification to all connected NotificationReceivers. Product hubs register a public key for each sender, and verify product signatures to ensure only known senders may distribute products.
See Sending Products for more information
Custom Notification and Indexer Listeners can be integrated with PDL using the Java API. Other languages need to use the ExternalNotificationListener and ExternalIndexerListener.
NotificationListeners must implement the gov.usgs.earthquake.distribution.NotificationListener interface. Most users can extend the gov.usgs.earthquake.distribution.DefaultNotificationListener class.
IndexerListeners must implement the gov.usgs.earthquake.indexer.IndexerListener interface. Most users can extend the gov.usgs.earthquake.indexer.DefaultIndexerListener class.
See the Java API Documentation for more information.
You can find your version of java by running
java -version java version "10.0.2" 2018-07-17 Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13) Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)
Versions and Release Notes can be found on GitHub: https://github.com/usgs/pdl/releases
Download and unzip ProductClient.zip This includes shell scripts for unix and a batch file for windows. https://usgs.github.io/pdl/ProductClient.zip
You can upgrade to the latest version by replacing ProductClient.jar and restarting the process. https://usgs.github.io/pdl/ProductClient.jar