` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_Creating_GS_Files_plot_csv_skytem.py: Multi-dataset Survey -------------------- This example demonstrates the typical workflow for creating a GS file for an AEM survey in its entirety, i.e., the NetCDF file contains all related datasets together, e.g., raw data, processed data, inverted models, and derivative products. Specifically, this survey contains: 1. Minimally processed (raw) AEM data and raw/processed magnetic data provided by SkyTEM 2. Fully processed AEM data used as input to inversion 3. Laterally constrained inverted resistivity models 4. Point-data estimates of bedrock depth derived from the AEM models 5. Interpolated magnetic and bedrock depth grids Note: To make the size of this example more managable, some of the input datasets have been downsampled relative to the source files in the data release referenced below. Dataset Reference: Minsley, B.J, Bloss, B.R., Hart, D.J., Fitzpatrick, W., Muldoon, M.A., Stewart, E.K., Hunt, R.J., James, S.R., Foks, N.L., and Komiskey, M.J., 2022, Airborne electromagnetic and magnetic survey data, northeast Wisconsin (ver. 1.1, June 2022): U.S. Geological Survey data release, https://doi.org/10.5066/P93SY9LI. .. GENERATED FROM PYTHON SOURCE LINES 20-29 .. code-block:: Python import matplotlib.pyplot as plt from os.path import join import numpy as np import gspy from gspy import Survey import xarray as xr from pprint import pprint .. GENERATED FROM PYTHON SOURCE LINES 30-32 Convert the Skytem csv data to NetCDF +++++++++++++++++++++++++++++++++++++ .. GENERATED FROM PYTHON SOURCE LINES 34-35 Initialize the Survey .. GENERATED FROM PYTHON SOURCE LINES 35-48 .. code-block:: Python # Path to example files data_path = '..//..//..//..//example_material//example_1' # Survey metadata file metadata = join(data_path, "data//WI_SkyTEM_survey_md.yml") # Establish the Survey survey = Survey.from_dict(metadata) data_container = survey.gs.add_container('data', **dict(content = "raw and processed data", comment = "This is a test")) .. GENERATED FROM PYTHON SOURCE LINES 49-52 1 - Raw Data - Import raw AEM data from CSV-format. Define input data file and associated metadata file .. GENERATED FROM PYTHON SOURCE LINES 52-58 .. code-block:: Python d_data1 = join(data_path, 'data//WI_SkyTEM_2021_ContractorData.csv') d_supp1 = join(data_path, 'data//WI_SkyTEM_raw_data_md.yml') # Add the raw AEM data as a tabular dataset data_container.gs.add(key='raw_data', data_filename=d_data1, metadata_file=d_supp1, system=survey.nominal_system) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 59-62 2 - Processed Data - Import processed AEM data from CSV-format. Define input data file and associated metadata file .. GENERATED FROM PYTHON SOURCE LINES 62-71 .. code-block:: Python d_data2 = join(data_path, 'data//WI_SkyTEM_2021_ProcessedData.csv') d_supp2 = join(data_path, 'data//WI_SkyTEM_processed_data_md.yml') system = {"skytem_system" : survey["nominal_system"].isel(lm_gate_times=np.s_[1:], hm_gate_times=np.s_[10:]), "magnetic_system" : survey["magnetic_system"]} # Add the processed AEM data as a tabular dataset pd = data_container.gs.add(key='processed_data', data_filename=d_data2, metadata_file=d_supp2, system=system) .. GENERATED FROM PYTHON SOURCE LINES 72-73 3 - Inverted Models - .. GENERATED FROM PYTHON SOURCE LINES 73-89 .. code-block:: Python # Create a new container for models model_container = survey.gs.add_container('models', **dict(content = "Inverted models", comment = "This is a test")) # pprint(survey.gs.get_all_attr('standard_name')) print(survey.gs.tree) # Import inverted AEM models from CSV-format. # Define input data file and associated metadata file m_data3 = join(data_path, 'model//WI_SkyTEM_2021_InvertedModels.csv') m_supp3 = join(data_path, 'model//WI_SkyTEM_inverted_models_md.yml') # Add the inverted AEM models as a tabular dataset model_container.gs.add(key='inverted_models', data_filename=m_data3, metadata_file=m_supp3) .. rst-class:: sphx-glr-script-out .. code-block:: none /survey /survey/nominal_system /survey/magnetic_system /survey/data /survey/models /survey/data/raw_data /survey/data/processed_data /survey/data/raw_data/nominal_system /survey/data/processed_data/skytem_system /survey/data/processed_data/magnetic_system .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 90-93 4 - Bedrock Picks - Import AEM-based estimated of depth to bedrock from CSV-format. Define input data file and associated metadata file .. GENERATED FROM PYTHON SOURCE LINES 93-99 .. code-block:: Python d_data4 = join(data_path, 'data//topDolomite_Blocky_LidarDEM.csv') d_supp4 = join(data_path, 'data//WI_SkyTEM_bedrock_picks_md.yml') # Add the AEM-based estimated of depth to bedrock as a tabular dataset data_container.gs.add(key='depth_to_bedrock', data_filename=d_data4, metadata_file=d_supp4) .. raw:: html

.. GENERATED FROM PYTHON SOURCE LINES 100-101 5 - Derivative Maps - .. GENERATED FROM PYTHON SOURCE LINES 101-114 .. code-block:: Python # We can add arbitrarily named containers to the survey derived_products = survey.gs.add_container('derived_products', **dict(content = "products derived from other data and models")) # Import interpolated bedrock and magnetic maps from TIF-format. # Define input metadata file (which contains the TIF filenames linked to variable names) m_supp5 = join(data_path, 'data//WI_SkyTEM_mag_bedrock_grids_md.yml') # Add the interpolated maps as a raster dataset derived_products.gs.add(key='maps', metadata_file=m_supp5) print(survey.gs.tree) .. rst-class:: sphx-glr-script-out .. code-block:: none /survey /survey/nominal_system /survey/magnetic_system /survey/data /survey/models /survey/derived_products /survey/data/raw_data /survey/data/processed_data /survey/data/depth_to_bedrock /survey/models/inverted_models /survey/derived_products/maps /survey/data/raw_data/nominal_system /survey/data/processed_data/skytem_system /survey/data/processed_data/magnetic_system .. GENERATED FROM PYTHON SOURCE LINES 115-116 Save to NetCDF file .. GENERATED FROM PYTHON SOURCE LINES 116-119 .. code-block:: Python d_out = join(data_path, 'model//WISkyTEM.nc') survey.gs.to_netcdf(d_out) .. GENERATED FROM PYTHON SOURCE LINES 120-121 The gspy goal is to have the complete survey in a single file. However, we can also save containers or datasets separately. .. GENERATED FROM PYTHON SOURCE LINES 121-124 .. code-block:: Python data_container.gs.to_netcdf('test_datacontainer.nc') .. GENERATED FROM PYTHON SOURCE LINES 125-126 Reading back in .. GENERATED FROM PYTHON SOURCE LINES 126-130 .. code-block:: Python new_survey = gspy.open_datatree(d_out)['survey'] print(new_survey) .. rst-class:: sphx-glr-script-out .. code-block:: none Group: /survey │ Dimensions: () │ Coordinates: │ spatial_ref float64 8B ... │ Data variables: │ survey_information float64 8B ... │ flightline_information float64 8B ... │ survey_equipment float64 8B ... │ Attributes: │ title: SkyTEM Airborne Electromagnetic (AEM) Survey, Northeast Wis... │ institution: USGS Geology, Geophysics, and Geochemistry Science Center │ source: SkyTEM raw data, USGS processed data and inverted resistivi... │ history: (1) Data acquisition 01/2021 - 02/2021 by SkyTEM Canada Inc... │ references: Minsley, Burke J., B.R. Bloss, D.J. Hart, W. Fitzpatrick, M... │ comment: This dataset includes minimally processed (raw) AEM and raw... │ summary: Airborne electromagnetic (AEM) and magnetic survey data wer... │ content: survey information (group /survey), raw data (group /survey... │ created_by: gspy==2.0.0 │ conventions: CF-1.8, GS-2.0 │ type: survey ├── Group: /survey/data │ │ Dimensions: () │ │ Data variables: │ │ spatial_ref float64 8B ... │ │ Attributes: │ │ content: raw and processed data │ │ comment: This is a test │ │ type: container │ ├── Group: /survey/data/raw_data │ │ │ Dimensions: (index: 6785, hm_gate_times: 32, lm_gate_times: 28) │ │ │ Coordinates: │ │ │ * index (index) int32 27kB 0 1 2 3 4 5 ... 6780 6781 6782 6783 6784 │ │ │ * hm_gate_times (hm_gate_times) float64 256B 2.886e-05 ... 0.003544 │ │ │ * lm_gate_times (lm_gate_times) float64 224B -1.135e-06 ... 0.001394 │ │ │ spatial_ref float64 8B ... │ │ │ x (index) float64 54kB ... │ │ │ y (index) float64 54kB ... │ │ │ z (index) float64 54kB ... │ │ │ t (index) float64 54kB ... │ │ │ Data variables: (12/34) │ │ │ _60hz_intensity (index) float64 54kB ... │ │ │ alt (index) float64 54kB ... │ │ │ anglex (index) float64 54kB ... │ │ │ angley (index) float64 54kB ... │ │ │ base_mag (index) float64 54kB ... │ │ │ curr_hm (index) float64 54kB ... │ │ │ ... ... │ │ │ mag_raw (index) float64 54kB ... │ │ │ n_nad83 (index) float64 54kB ... │ │ │ n_wgs84 (index) float64 54kB ... │ │ │ rmf (index) float64 54kB ... │ │ │ time (index) ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_csv_skytem.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_csv_skytem.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_