Querying Metadata clients

This example shows how to search and retrieve metadata using Fido. Fido supports searching metadata from services like HEKClient, HECClient, and JSOCClient.

In this example we will make one search for records from the JSOC and the HEK, and then download the corresponding file from the JSOC.

import os

from sunpy.net import Fido
from sunpy.net import attrs as a

We will query the HEK for all flares with a peak flux greater than 1000. We will also search JSOC for a ‘hmi.m_45s’ series.

timerange = a.Time('2010/8/1 03:40', '2010/8/1 3:40:10')

# Exporting data from the JSOC requires registering your email first.
# Please replace this with your email address once you have registered
# like so: jsoc_email = "your_email@example.com"
# See `this page <http://jsoc.stanford.edu/ajax/register_email.html>`__ for more details.

jsoc_email = os.environ["JSOC_EMAIL"]

results = Fido.search(timerange,
                      a.hek.FL & (a.hek.FL.PeakFlux > 1000) |
                      a.jsoc.Series('hmi.m_45s') & a.jsoc.Notify(jsoc_email))

results is a UnifiedResponse object that contains records returned from querying various clients by “Fido.search”.

print(results)

Results from 2 Providers:

2 Results from the HEKClient:

                                                                                                                     gs_thumburl                                                                                                                       ...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100731_205448_25495028/www/EDS_FlareDetective-TriggerModule_20100801T033001-20100801T035225_AIA_171_S21W87_ssw_cutout_20100801_033013_AIA_171_S21W87_20100801_033012_context_0180.gif ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100801_234037_25860951/www/EDS_FlareDetective-TriggerModule_20100801T033008-20100801T035232_AIA_193_S21W87_ssw_cutout_20100801_033020_AIA_193_S21W87_20100801_033019_context_0180.gif ...

1 Results from the JSOCClient:
Source: http://jsoc.stanford.edu

         T_REC          TELESCOP  INSTRUME  WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099

Now we will download the searched records. Since the HEK client don’t provide files, Fido.fetch will ignore it and only download files from JSOC.

files = Fido.fetch(results)
print(files)

INFO: 1 URLs found for download. Full request totaling 14MB [sunpy.net.jsoc.jsoc]
['/home/docs/sunpy/data/hmi.m_45s.20100801_034030_TAI.2.magnetogram.fits']

Now we will extract individual responses from Fido results. We can index these results using the client’s name (which is case-insensitive).

hek_results, jsoc_results = results['hek'], results['jsoc']

The results from a metadata search could have up to 100 columns. As a result, you can use use show() to specify the column names you want to display.

hek_table = hek_results.show('event_peaktime', 'obs_instrument', 'fl_peakflux')
print(hek_table)

     event_peaktime     obs_instrument fl_peakflux
----------------------- -------------- -----------
2010-08-01 03:40:37.000            AIA     1027.64
2010-08-01 03:40:44.000            AIA     1441.78

The results from JSOC have a default set of columns to show and are ['T_REC', 'TELESCOP', 'INSTRUME', 'WAVELNTH', 'CAR_ROT']. To display all of the columns, we can use show() without passings any arguments.

print(jsoc_results)

jsoc_table = jsoc_results.show()
print(jsoc_table)

         T_REC          TELESCOP  INSTRUME  WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099
        DATE                DATE__OBS        ... CALVER64
-------------------- ----------------------- ... --------
2012-09-05T07:57:40Z 2010-08-01T03:39:41.00Z ...       16