Querying Metadata clients using Fido

This example shows how to search and retrieve metadata using Fido. Fido supports searching metadata from services like HEKClient, HECClient, and JSOCClient.

In this example we will make one search for records from the JSOC and the HEK, and then download the corresponding file from the JSOC.

import os

from sunpy.net import Fido
from sunpy.net import attrs as a

We will query the HEK for all flares with a peak flux greater than 1000. We will also search JSOC for a ‘hmi.m_45s’ series.

timerange = a.Time('2010/8/1 03:40', '2010/8/1 3:40:10')

# Note that JSOC needs an email address to export the files, if you want to run
# this, you must supply your own email here.
jsoc_email = os.environ["JSOC_EMAIL"]

results = Fido.search(timerange,
                      a.hek.FL & (a.hek.FL.PeakFlux > 1000) |
                      a.jsoc.Series('hmi.m_45s') & a.jsoc.Notify(jsoc_email))

results is a UnifiedResponse object that contains records returned from querying various clients by “Fido.search”.

print(results)

Out:

Results from 2 Providers:

2 Results from the HEKClient:
                                                                                                                     gs_thumburl                                                                                                                       ...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100731_205448_25495028/www/EDS_FlareDetective-TriggerModule_20100801T033001-20100801T035225_AIA_171_S21W87_ssw_cutout_20100801_033013_AIA_171_S21W87_20100801_033012_context_0180.gif ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100801_234037_25860951/www/EDS_FlareDetective-TriggerModule_20100801T033008-20100801T035232_AIA_193_S21W87_ssw_cutout_20100801_033020_AIA_193_S21W87_20100801_033019_context_0180.gif ...

1 Results from the JSOCClient:
Source: http://jsoc.stanford.edu

         T_REC          TELESCOP  INSTRUME  WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099

Now we will download the searched records. Since the HEK client don’t provide files, Fido.fetch will ignore it and only download files from JSOC.

files = Fido.fetch(results)
print(files)

Out:

Export request pending. [id=JSOC_20211202_1081, status=2]
Waiting for 0 seconds...
1 URLs found for download. Full request totalling 14MB
['/home/docs/sunpy/data/hmi.m_45s.20100801_034030_TAI.2.magnetogram.fits']

Now we will extract individual responses from Fido results. We can index these results using the client’s name (which is case-insensitive).

hek_results, jsoc_results = results['hek'], results['jsoc']

The results from a metadata search could have up to 100 columns. As a result, you can use use show() to specify the column names you want to display.

hek_table = hek_results.show('event_peaktime', 'obs_instrument', 'fl_peakflux')
print(hek_table)

Out:

   event_peaktime   obs_instrument fl_peakflux
------------------- -------------- -----------
2010-08-01T03:40:37            AIA     1027.64
2010-08-01T03:40:44            AIA     1441.78

The results from JSOC have a default set of columns to show and are ['T_REC', 'TELESCOP', 'INSTRUME', 'WAVELNTH', 'CAR_ROT']. To display all of the columns, we can use show() without passings any arguments.

print(jsoc_results)

jsoc_table = jsoc_results.show()
print(jsoc_table)

Out:

         T_REC          TELESCOP  INSTRUME  WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI  SDO/HMI HMI_FRONT2   6173.0    2099
        DATE                DATE__OBS        ... CALVER64
-------------------- ----------------------- ... --------
2012-09-05T07:57:40Z 2010-08-01T03:39:41.00Z ...       16

Total running time of the script: ( 0 minutes 12.514 seconds)

Gallery generated by Sphinx-Gallery