Querying Metadata clients#

This example shows how to search and retrieve metadata using Fido. Fido supports searching metadata from services like HEKClient, HECClient, and JSOCClient.

In this example we will make one search for records from the JSOC and the HEK, and then download the corresponding file from the JSOC.

import os

from sunpy.net import Fido
from sunpy.net import attrs as a

We will query the HEK for all flares with a peak flux greater than 1000. We will also search JSOC for a ‘hmi.m_45s’ series.

timerange = a.Time('2010/8/1 03:40', '2010/8/1 3:40:10')

# Exporting data from the JSOC requires registering your email first.
# Please replace this with your email address once you have registered
# like so: jsoc_email = "your_email@example.com"
# See `this page <http://jsoc.stanford.edu/ajax/register_email.html>`__ for more details.

jsoc_email = os.environ["JSOC_EMAIL"]

results = Fido.search(timerange,
                      a.hek.FL & (a.hek.FL.PeakFlux > 1000) |
                      a.jsoc.Series('hmi.m_45s') & a.jsoc.Notify(jsoc_email))

results is a UnifiedResponse object that contains records returned from querying various clients by “Fido.search”.

Results from 2 Providers:

2 Results from the HEKClient:

                                                                                                                     gs_thumburl                                                                                                                       ...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100731_205448_25495028/www/EDS_FlareDetective-TriggerModule_20100801T033001-20100801T035225_AIA_171_S21W87_ssw_cutout_20100801_033013_AIA_171_S21W87_20100801_033012_context_0180.gif ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100801_234037_25860951/www/EDS_FlareDetective-TriggerModule_20100801T033008-20100801T035232_AIA_193_S21W87_ssw_cutout_20100801_033020_AIA_193_S21W87_20100801_033019_context_0180.gif ...

1 Results from the JSOCClient:
Source: http://jsoc.stanford.edu

----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099

Now we will download the searched records. Since the HEK client don’t provide files, Fido.fetch will ignore it and only download files from JSOC.

files = Fido.fetch(results)
Export request pending. [id=JSOC_20240511_001863, status=2]
Waiting for 0 seconds...
1 URLs found for download. Full request totalling 14MB

Now we will extract individual responses from Fido results. We can index these results using the client’s name (which is case-insensitive).

hek_results, jsoc_results = results['hek'], results['jsoc']

The results from a metadata search could have up to 100 columns. As a result, you can use use show() to specify the column names you want to display.

hek_table = hek_results.show('event_peaktime', 'obs_instrument', 'fl_peakflux')
     event_peaktime     obs_instrument fl_peakflux
----------------------- -------------- -----------
2010-08-01 03:40:37.000            AIA     1027.64
2010-08-01 03:40:44.000            AIA     1441.78

The results from JSOC have a default set of columns to show and are ['T_REC', 'TELESCOP', 'INSTRUME', 'WAVELNTH', 'CAR_ROT']. To display all of the columns, we can use show() without passings any arguments.


jsoc_table = jsoc_results.show()
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099
        DATE                DATE__OBS        ... CALVER64
-------------------- ----------------------- ... --------
2012-09-05T07:57:40Z 2010-08-01T03:39:41.00Z ...       16

Total running time of the script: (0 minutes 11.674 seconds)

Gallery generated by Sphinx-Gallery