Magnetic Survey

These magnetic data channels were pulled from the Wisconsin Skytem example in this repository

Dataset Reference: Minsley, B.J, Bloss, B.R., Hart, D.J., Fitzpatrick, W., Muldoon, M.A., Stewart, E.K., Hunt, R.J., James, S.R., Foks, N.L., and Komiskey, M.J., 2022, Airborne electromagnetic and magnetic survey data, northeast Wisconsin (ver. 1.1, June 2022): U.S. Geological Survey data release, https://doi.org/10.5066/P93SY9LI.

import matplotlib.pyplot as plt
from os.path import join
import numpy as np
import gspy
from gspy import Survey
import xarray as xr
from pprint import pprint

Convert the magnetic csv data to NetCDF

Initialize the Survey

# Path to example files
data_path = '..//data_files//magnetics'

# Survey metadata file
metadata = join(data_path, "WI_Magnetics_survey_md.yml")

# Establish the Survey
survey = Survey.from_dict(metadata)

data_container = survey.gs.add_container('data', **dict(content = "raw and gridded data",
                                                        comment = "This is a test"))

1 - Raw Data - Import raw mag data from CSV-format. Define input data file and associated metadata file

d_data1 = join(data_path, 'WI_Magnetics.csv')
d_supp1 = join(data_path, 'WI_Magnetics_raw_data_md.yml')

# Add the raw AEM data as a tabular dataset
data_container.gs.add(key='raw_data', data_filename=d_data1, metadata_file=d_supp1)
<xarray.DataTree 'raw_data'>
Group: /survey/data/raw_data
│   Dimensions:      (index: 6785)
│   Coordinates:
│     * index        (index) int32 27kB 0 1 2 3 4 5 ... 6780 6781 6782 6783 6784
│       spatial_ref  float64 8B 0.0
│       x            (index) float64 54kB 7.243e+05 7.239e+05 ... 6.604e+05
│       y            (index) float64 54kB 4.916e+05 4.917e+05 ... 3.866e+05
│       z            (index) float64 54kB 176.8 217.3 231.6 ... 273.2 276.4 288.7
│   Data variables: (12/20)
│       fid          (index) float64 54kB 2.349e+05 2.349e+05 ... 1.868e+06
│       line         (index) int64 54kB 100101 100101 100101 ... 115201 115201
│       e_nad83      (index) float64 54kB 7.243e+05 7.239e+05 ... 6.604e+05
│       n_nad83      (index) float64 54kB 4.916e+05 4.917e+05 ... 3.866e+05
│       lon          (index) float64 54kB -87.41 -87.42 -87.42 ... -88.25 -88.25
│       lat          (index) float64 54kB 44.87 44.87 44.87 ... 43.94 43.94 43.94
│       ...           ...
│       mag_raw      (index) float64 54kB 5.481e+04 5.481e+04 ... 5.413e+04
│       tmi          (index) float64 54kB 5.482e+04 5.482e+04 ... 5.413e+04
│       rmf          (index) float64 54kB 210.6 210.6 197.3 ... -122.7 -124.5 -120.3
│       igrf         (index) float64 54kB 5.461e+04 5.461e+04 ... 5.425e+04
│       inc          (index) float64 54kB 71.13 71.13 71.13 ... 70.48 70.48 70.48
│       dec          (index) float64 54kB -4.796 -4.792 -4.787 ... -3.975 -3.972
│   Attributes:
│       uuid:        3a98c551-ee48-4202-af0e-b86756105ef8
│       content:     raw data
│       comment:     Contains mag data
│       type:        data
│       structure:   tabular
│       mode:        airborne
│       method:      magnetic
│       submethod:   total field
│       instrument:  cesium vapour
│       property:    magnetic susceptibility
└── Group: /survey/data/raw_data/magnetic_system
        Dimensions:      (dim_0: 1)
        Dimensions without coordinates: dim_0
        Data variables:
            sample_rate  (dim_0) float64 8B 0.1
            resolution   (dim_0) float64 8B 0.002
        Attributes:
            type:        system
            mode:        airborne
            method:      magnetic
            submethod:   total field
            instrument:  cesium vapour
            uuid:        f176f713-3978-4882-9f71-705bce9d189b
            name:        magnetic_system


1 - Gridded Data - Import a tif of gridded mag data.

d_supp1 = join(data_path, 'WI_Magnetics_grids_md.yml')

# Add the raw AEM data as a tabular dataset
data_container.gs.add(key='grids', metadata_file=d_supp1)
<xarray.DataTree 'grids'>
Group: /survey/data/grids
    Dimensions:       (x: 799, nv: 2, y: 1155)
    Coordinates:
      * x             (x) float64 6kB 6.551e+05 6.552e+05 ... 7.348e+05 7.349e+05
      * nv            (nv) int64 16B 0 1
      * y             (y) float64 9kB 4.953e+05 4.952e+05 ... 3.8e+05 3.799e+05
        spatial_ref   float64 8B 0.0
    Data variables:
        x_bnds        (x, nv) float64 13kB 6.55e+05 6.551e+05 ... 7.349e+05
        y_bnds        (y, nv) float64 18kB 4.954e+05 4.953e+05 ... 3.8e+05 3.799e+05
        magnetic_tmi  (y, x) float64 7MB nan nan nan nan nan ... nan nan nan nan nan
        magnetic_rmf  (y, x) float64 7MB nan nan nan nan nan ... nan nan nan nan nan
    Attributes:
        uuid:        bdfd33eb-6ead-4e97-aece-f38227ffa937
        content:     gridded magnetic maps
        comment:     This dataset includes AEM-derived estimates of the elevation...
        type:        data
        structure:   raster
        mode:        airborne
        method:      magnetic
        submethod:   total field
        instrument:  cesium vapour
        property:    magnetic


Save to NetCDF file

d_out = join(data_path, 'magnetics.nc')
survey.gs.to_netcdf(d_out)
uuid
title
institution
source
history
references
comment
summary
content
created_by
conventions
type
_FillValue
contractor_project_number
contractor
client
survey_type
survey_area_name
state
country
acquisition_start
acquisition_end
survey_attributes_units
coordinates
_FillValue
traverse_line_spacing
traverse_line_direction
tie_line_spacing
tie_line_direction
nominal_terrain_clearance
final_line_kilometers
traverse_line_numbers
repeat_line_numbers
pre_zero_line_numbers
post_zero_line_numbers
coordinates
_FillValue
aircraft
magnetometer
magnetometer_installation
radar_altimeter_system
radar_altimeter_sample_rate
laser_altimeter_system
laser_altimeter_sample_rate
navigation_system
navigation_sample_rate
acquisition_system
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
scale_factor_at_central_meridian
authority
wkid
content
comment
type
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
scale_factor_at_central_meridian
authority
wkid
uuid
content
comment
type
structure
mode
method
submethod
instrument
property
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
axis
positive
datum
valid_range
grid_mapping
coordinates
standard_name
long_name
units
null_value
grid_mapping
coordinates
standard_name
long_name
units
null_value
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
scale_factor_at_central_meridian
authority
wkid
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
axis
positive
datum
valid_range
grid_mapping
uuid
content
comment
type
structure
mode
method
submethod
instrument
property
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
standard_name
long_name
units
null_value
valid_range
grid_mapping
coordinates
_FillValue
crs_wkt
semi_major_axis
semi_minor_axis
inverse_flattening
reference_ellipsoid_name
longitude_of_prime_meridian
prime_meridian_name
geographic_crs_name
horizontal_datum_name
projected_crs_name
grid_mapping_name
latitude_of_projection_origin
longitude_of_central_meridian
false_easting
false_northing
scale_factor_at_central_meridian
authority
wkid
GeoTransform
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
bounds
standard_name
long_name
units
null_value
valid_range
grid_mapping
_FillValue
standard_name
long_name
units
null_value
axis
valid_range
grid_mapping
bounds
type
mode
method
submethod
instrument
uuid
name
_FillValue
grid_mapping
_FillValue
grid_mapping

Reading back in

new_survey = gspy.open_datatree(d_out)['survey']

print(new_survey)
<xarray.DataTree 'survey'>
Group: /survey
│   Dimensions:                 ()
│   Coordinates:
│       spatial_ref             float64 8B ...
│   Data variables:
│       survey_information      float64 8B ...
│       flightline_information  float64 8B ...
│       survey_equipment        float64 8B ...
│   Attributes:
│       uuid:         a60011af-6361-461f-88a3-0747712a3427
│       title:        Magnetic data from SkyTEM Airborne Electromagnetic (AEM) Su...
│       institution:  USGS Geology, Geophysics, and Geochemistry Science Center
│       source:       SkyTEM raw data, USGS processed data
│       history:      (1) Data acquisition 01/2021 - 02/2021 by SkyTEM Canada Inc...
│       references:   Minsley, Burke J., B.R. Bloss, D.J. Hart, W. Fitzpatrick, M...
│       comment:      This dataset includes minimally processed (raw) AEM and raw...
│       summary:      Magnetic survey data were collected during January and Febr...
│       content:      Data
│       created_by:   gspy==2.0.0
│       conventions:  CF-1.8, GS-2.0
│       type:         survey
└── Group: /survey/data
    │   Dimensions:      ()
    │   Data variables:
    │       spatial_ref  float64 8B ...
    │   Attributes:
    │       content:  raw and gridded data
    │       comment:  This is a test
    │       type:     container
    ├── Group: /survey/data/raw_data
    │   │   Dimensions:      (index: 6785)
    │   │   Coordinates:
    │   │     * index        (index) int32 27kB 0 1 2 3 4 5 ... 6780 6781 6782 6783 6784
    │   │       spatial_ref  float64 8B ...
    │   │       x            (index) float64 54kB ...
    │   │       y            (index) float64 54kB ...
    │   │       z            (index) float64 54kB ...
    │   │   Data variables: (12/20)
    │   │       fid          (index) float64 54kB ...
    │   │       line         (index) int64 54kB ...
    │   │       e_nad83      (index) float64 54kB ...
    │   │       n_nad83      (index) float64 54kB ...
    │   │       lon          (index) float64 54kB ...
    │   │       lat          (index) float64 54kB ...
    │   │       ...           ...
    │   │       mag_raw      (index) float64 54kB ...
    │   │       tmi          (index) float64 54kB ...
    │   │       rmf          (index) float64 54kB ...
    │   │       igrf         (index) float64 54kB ...
    │   │       inc          (index) float64 54kB ...
    │   │       dec          (index) float64 54kB ...
    │   │   Attributes:
    │   │       uuid:        3a98c551-ee48-4202-af0e-b86756105ef8
    │   │       content:     raw data
    │   │       comment:     Contains mag data
    │   │       type:        data
    │   │       structure:   tabular
    │   │       mode:        airborne
    │   │       method:      magnetic
    │   │       submethod:   total field
    │   │       instrument:  cesium vapour
    │   │       property:    magnetic susceptibility
    │   └── Group: /survey/data/raw_data/magnetic_system
    │           Dimensions:      (dim_0: 1)
    │           Dimensions without coordinates: dim_0
    │           Data variables:
    │               sample_rate  (dim_0) float64 8B ...
    │               resolution   (dim_0) float64 8B ...
    │           Attributes:
    │               type:        system
    │               mode:        airborne
    │               method:      magnetic
    │               submethod:   total field
    │               instrument:  cesium vapour
    │               uuid:        f176f713-3978-4882-9f71-705bce9d189b
    │               name:        magnetic_system
    └── Group: /survey/data/grids
            Dimensions:       (x: 799, nv: 2, y: 1155)
            Coordinates:
              * x             (x) float64 6kB 6.551e+05 6.552e+05 ... 7.348e+05 7.349e+05
              * nv            (nv) int64 16B 0 1
              * y             (y) float64 9kB 4.953e+05 4.952e+05 ... 3.8e+05 3.799e+05
                spatial_ref   float64 8B ...
            Data variables:
                x_bnds        (x, nv) float64 13kB ...
                y_bnds        (y, nv) float64 18kB ...
                magnetic_tmi  (y, x) float64 7MB ...
                magnetic_rmf  (y, x) float64 7MB ...
            Attributes:
                uuid:        bdfd33eb-6ead-4e97-aece-f38227ffa937
                content:     gridded magnetic maps
                comment:     This dataset includes AEM-derived estimates of the elevation...
                type:        data
                structure:   raster
                mode:        airborne
                method:      magnetic
                submethod:   total field
                instrument:  cesium vapour
                property:    magnetic

Plotting

plt.figure()
new_survey['data/raw_data']['height'].plot()
plt.tight_layout()

pd = new_survey['data/raw_data']['tmi']
plt.figure()
pd.plot()
plt.tight_layout()

m = new_survey['data/grids/magnetic_tmi']
plt.figure()
m.plot(cmap='jet')
plt.tight_layout()

plt.show()
  • spatial_ref = 0.0
  • spatial_ref = 0.0
  • spatial_ref = 0.0

Total running time of the script: (0 minutes 0.455 seconds)

Gallery generated by Sphinx-Gallery