ASEG to NetCDF

This example demonstrates the workflow for creating a GS file from the ASEG file format, as well as how to add multiple associated datasets to the Survey (e.g., Tabular and Raster groups). Specifically, this AEM survey contains the following datasets:

Raw AEM data, from the Tempest system
Inverted resistivity models
An interpolated map of total magnetic intensity

Dataset Reference: Minsley, B.J., James, S.R., Bedrosian, P.A., Pace, M.D., Hoogenboom, B.E., and Burton, B.L., 2021, Airborne electromagnetic, magnetic, and radiometric survey of the Mississippi Alluvial Plain, November 2019 - March 2020: U.S. Geological Survey data release, https://doi.org/10.5066/P9E44CTQ.

import matplotlib.pyplot as plt
from os.path import join
import gspy

Convert the ASEG data to NetCDF

Initialize the Survey

# Path to example files
data_path = "..//data_files//tempest_aseg"

# Survey Metadata file
metadata = join(data_path, "data//Tempest_survey_md.yml")

# Establish survey instance
survey = gspy.Survey.from_dict(metadata)

Raw Data -

data_container = survey.gs.add_container('data', **dict(content = "raw and processed data"))

# Import raw AEM data from ASEG-format.
# Define input data file and associated metadata file
d_data = join(data_path, 'data//Tempest.dat')
d_supp = join(data_path, 'data//Tempest_data_md.yml')

# Add the raw AEM data as a tabular dataset
rd = data_container.gs.add(key='raw_data', data_filename=d_data, metadata_file=d_supp)

Inverted Models

model_container = survey.gs.add_container('models', **dict(content = "inverted models"))

# Import inverted AEM models from ASEG-format.
# Define input data file and associated metadata file
m_data = join(data_path, 'model//Tempest_model.dat')
m_supp = join(data_path, 'model//Tempest_model_md.yml')

# Read model data and format as Tabular class object
mod = model_container.gs.add(key='inverted_models', data_filename=m_data, metadata_file=m_supp, derived_from=rd)

Magnetic Intensity Map

data_derived = data_container.gs.add_container('derived_maps', **dict(content = "derived maps"))

# Import the magnetic data from TIF-format.
# Define input metadata file (which contains the TIF filenames linked with desired variable names)
r_supp = join(data_path, 'data//Tempest_raster_md.yml')

# Read data and format as Raster class object
data_derived.gs.add(key='maps', metadata_file = r_supp)

# Save NetCDF file
d_out = join(data_path, 'Tempest.nc')
survey.gs.to_netcdf(d_out)

uuid
title
institution
source
history
references
comment
content
conventions
created_by
type
_FillValue
contractor_project_number
contractor
client
survey_type
survey_area_name
state
country
acquisition_start
acquisition_end
dataset_created
coordinates
_FillValue
time
area
current
frequency
electromagnetic_moment
magnetometer_b_field
electromagnetic_b_field
coordinates
_FillValue
traverse_line_spacing
traverse_line_direction
nominal_terrain_clearance
final_line_kilometers
TRAVERSE LINE NUMBERS
REPEAT LINE NUMBERS
PRE ZERO LINE NUMBERS
POST ZERO LINE NUMBERS
coordinates
_FillValue
aircraft
aircraft_registration
magnetometer
magnetometer_installation
electromagnetic_system
electromagnetic_installation
electromagnetic_coil_orientations
spectrometer_system
spectrometer_installation
radar_altimeter_system
radar_altimeter_sample_rate
laser_altimeter_system
laser_altimeter_sample_rate
navigation_system
navigation_sample_rate
acquisition_system
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
content
type
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
content
type
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
uuid
content
comment
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
positive
datum
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
em_system_components
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
em_system_components
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
em_system_components
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
em_system_components
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
format
axis
positive
datum
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
bounds
content
type
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
uuid
content
comment
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
axis
positive
datum
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
format
valid_range
grid_mapping
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
bounds
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
bounds
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
format
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
format
axis
positive
datum
valid_range
grid_mapping
type
mode
method
submethod
instrument
uuid
name
structure
data_normalized
output_data_type
reference_frame
output_sample_frequency
digitization_frequency
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
grid_mapping
valid_range
grid_mapping
grid_mapping
_FillValue
valid_range
grid_mapping
_FillValue
valid_range
grid_mapping
valid_range
grid_mapping
valid_range
grid_mapping
valid_range
grid_mapping
grid_mapping
valid_range
grid_mapping
_FillValue
valid_range
grid_mapping
_FillValue
valid_range
grid_mapping
grid_mapping
grid_mapping
grid_mapping
grid_mapping
grid_mapping
valid_range
grid_mapping
valid_range
grid_mapping
valid_range
grid_mapping
standard_name
long_name
units
null_value
valid_range
grid_mapping
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
label
prefix
long_name
null_value
units
standard_name
valid_range
grid_mapping
standard_name
long_name
units
null_value
valid_range
grid_mapping
standard_name
long_name
units
null_value
valid_range
grid_mapping
type
mode
method
submethod
instrument
name
sample_frequency
resolution
type
mode
method
submethod
instrument
name
sample_frequency
uuid
comment
content
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
standard_parallel
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
authority
wkid
GeoTransform
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
bounds
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
bounds

Read back in the NetCDF file

new_survey = gspy.open_datatree(d_out)['survey']

# Once the survey is read in, we can access variables like a standard xarray dataset.
print(new_survey['data/derived_maps/maps'].magnetic_tmi)
print(new_survey['data/derived_maps/maps']['magnetic_tmi'])

<xarray.DataArray 'magnetic_tmi' (y: 1212, x: 599)> Size: 6MB
[725988 values with dtype=float64]
Coordinates:
  * y            (y) float64 10kB 1.607e+06 1.606e+06 ... 8.808e+05 8.802e+05
  * x            (x) float64 5kB 2.928e+05 2.934e+05 ... 6.51e+05 6.516e+05
    spatial_ref  float64 8B ...
Attributes:
    standard_name:  total_magnetic_intensity
    long_name:      Total magnetic intensity, diurnally corrected and filtered
    units:          nT
    null_value:     1.70141e+38
    valid_range:    [-17504.6640625   11490.32324219]
    grid_mapping:   spatial_ref
<xarray.DataArray 'magnetic_tmi' (y: 1212, x: 599)> Size: 6MB
[725988 values with dtype=float64]
Coordinates:
  * y            (y) float64 10kB 1.607e+06 1.606e+06 ... 8.808e+05 8.802e+05
  * x            (x) float64 5kB 2.928e+05 2.934e+05 ... 6.51e+05 6.516e+05
    spatial_ref  float64 8B ...
Attributes:
    standard_name:  total_magnetic_intensity
    long_name:      Total magnetic intensity, diurnally corrected and filtered
    units:          nT
    null_value:     1.70141e+38
    valid_range:    [-17504.6640625   11490.32324219]
    grid_mapping:   spatial_ref

Plotting

# Make a scatter plot of a specific tabular variable, using GSPy's plotter
plt.figure()
new_survey['data']['raw_data'].gs.scatter(x='x', hue='tx_height', cmap='jet')

# Make a 2-D map plot of a specific raster variable, using Xarrays's plotter
plt.figure()
new_survey['data/derived_maps/maps']['magnetic_tmi'].plot(cmap='jet', robust=True)
# plt.show()

<matplotlib.collections.QuadMesh object at 0x15980c6b0>

Total running time of the script: (0 minutes 1.286 seconds)

Gallery generated by Sphinx-Gallery