sunpy core Documentation#
sunpy is a community-developed, free and open-source solar data analysis environment for Python.
It includes an interface for searching and downloading data from multiple data providers, data containers for image and time series data, commonly used solar coordinate frames and associated transformations, as well as other functionality needed for solar data analysis.
The sunpy tutorial#
Welcome to the introductory tutorial for the sunpy core package.
sunpy is a community-developed, free and open-source solar data analysis environment.
It is meant to provide the core functionality and tools to analyze solar data with Python.
Who this is for
This tutorial assumes you know how to run code in Python, but doesn’t make any assumptions beyond that.
sunpy depends heavily on other packages in the scientific Python ecosystem, including Astropy, NumPy, and Matplotlib.
This tutorial explains the basics of these packages needed to use sunpy, but is not a comprehensive tutorial for these packages and references their own introductory tutorials where relevant.
How to use this tutorial
This tutorial is designed to be read from start to finish. You can either read it without running any code, or copy and paste the code snippets as you read. Each chapter of the tutorial provides a self-contained set of codes.
Later chapters build on the concepts introduced in earlier chapters. The one exception is the two chapters that explore Maps and TimeSeries - these are independent, but depend on all earlier chapters.
Installation#
This is the first chapter in the sunpy tutorial, and by the end of it you should have a working installation of Python and sunpy.
For further information and alternative methods for installing sunpy beyond the recommended approach outlined below, refer to Advanced Installation.
Installing Python#
There are many ways to install Python, but even if you have Python installed somewhere on your computer we recommend following these instructions anyway.
That’s because we will create a new Python environment.
As well as containing a Python installation, this environment provides an isolated place to install Python packages (like sunpy) without affecting any other current Python installation.
If you already have Python and conda working you can skip the next section.
However, if you are using Anaconda, we recommend you still install miniforge as described below.
Installing miniforge#
If you don’t already have a Python installation then we recommend installing Python with miniforge.
This will install conda and automatically configure the default channel (a channel is a remote software repository) to be conda-forge, which is where sunpy is available.
First, download the installer for your system and architecture from the links below:
Then select your platform to install miniforge:
Linux & Mac Run the script downloaded above, with
bash <filename>. The following should work:
bash Miniforge3-$(uname)-$(uname -m).sh
Once the installer has completed, close and reopen your terminal.
Double click the executable file downloaded from the links above.
Once the installer has completed you should have a new “miniforge Prompt” entry in your start menu.
In a new terminal (miniforge Prompt on Windows) run conda list to test that the install has worked.
Installing sunpy#
To install sunpy, start by launching a terminal (under a UNIX-like system) or the miniforge Prompt (under Windows).
Now we will create and activate a new virtual environment to install sunpy into:
$ conda create --name sunpy
# Only run the following two lines
# if you have NOT installed miniforge or added conda-forge to your channels
# Do not run these lines if you are using Anaconda
$ conda config --add channels conda-forge
$ conda config --set channel_priority strict
$ conda activate sunpy
In this case the environment is named ‘sunpy’. Feel free to change this to a different environment name.
The benefit of using a virtual environment is that it allows you to install packages without affecting any other Python installation on your system. This also means you can work on multiple projects (research or coding) with different package requirements without them interfering with each other.
Now we have a fresh environment we can install sunpy:
$ conda install sunpy
This will install sunpy and all of its dependencies.
If you want to install another package later, you can run conda install <package_name>.
Now we’ve got a working installation of sunpy, in the next few chapters we’ll look at some of the basic data structures sunpy uses for representing times, coordinates, and data with physical units.
Units#
In this section of the tutorial you will learn about representing physical units in sunpy.
All functions in sunpy that accept or return numbers associated with physical quantities do so using astropy.units.Quantity objects.
These objects represent a number (or an array of numbers) and a unit.
This means sunpy is always explicit about the units associated with a value.
Quantities and units are powerful tools for keeping track of variables with a physical meaning and make it straightforward to convert the same physical quantity into different units.
By the end of this section of the tutorial, you will learn how to create a Quantity, perform basic arithmetic with a Quantity, convert a Quantity to different units, and write a function that ensures the inputs have the correct units.
Adding Units to Data#
To use units we must first import them from Astropy.
It is standard practice to import the units module as u:
>>> import astropy.units as u
We can create a Quantity by multiplying a number by a unit:
>>> length = 10 * u.meter
>>> length
<Quantity 10. m>
A Quantity can be decomposed into its unit and numerical value using the .unit and .value attributes:
>>> length.value
10.0
>>> length.unit
Unit("m")
Arithmetic With Units#
Quantity objects propagate units through arithmetic operations:
>>> distance_start = 10 * u.mm
>>> distance_end = 23 * u.km
>>> displacement = distance_end - distance_start
>>> displacement
<Quantity 22.99999 km>
>>> time = 15 * u.minute
>>> speed = displacement / time
>>> speed
<Quantity 1.53333267 km / min>
However, operations with incompatible units raise an error:
>>> displacement + time
Traceback (most recent call last):
...
astropy.units.core.UnitConversionError: Can only apply 'add' function to quantities with compatible dimensions
Converting Units#
Quantity objects can also be converted to other units or unit systems:
>>> length.to(u.km)
<Quantity 0.01 km>
>>> length.cgs
<Quantity 1000. cm>
Unit Equivalencies#
It is commonplace to convert between units which are only compatible under certain assumptions. For example, in spectroscopy, spectral energy and wavelength are equivalent given the relation \(E=hc/\lambda\). If we try to convert a wavelength to energy using what we learned in the previous section, we get an exception because length and energy are, in general, not compatible units:
>>> length.to(u.keV)
Traceback (most recent call last):
...
astropy.units.core.UnitConversionError: 'm' (length) and 'keV' (energy/torque/work) are not convertible
However, we can perform this conversion using the spectral equivalency:
>>> length.to(u.keV, equivalencies=u.spectral())
<Quantity 1.23984198e-10 keV>
An equivalency common in solar physics is conversion of angular distances in the plane of the sky to physical distances on the Sun.
To perform this conversion, sunpy provides solar_angle_equivalency, which requires specifying the location at which that angular distance was measured:
>>> from sunpy.coordinates import get_earth
>>> from sunpy.coordinates.utils import solar_angle_equivalency
>>> length.to(u.arcsec, equivalencies=solar_angle_equivalency(get_earth("2013-10-28")))
INFO: Apparent body location accounts for 495.82 seconds of light travel time [sunpy.coordinates.ephemeris]
<Quantity 1.38763748e-05 arcsec>
Note that in the above example we made use of sunpy.coordinates.get_earth.
We will talk more about coordinates in the Coordinates section of this tutorial.
For now, it is just important to know that this function returns the location of the Earth on 2013 October 28.
Dropping Units#
Not every package in the scientific Python ecosystem understands units.
As such, it is sometimes necessary to drop the units before passing Quantity to such functions.
As shown above, you can retrieve the just the numerical value of a Quantity:
>>> length.to_value()
10.0
>>> length.to_value(u.km)
0.01
Quantities as function arguments#
When calling a function that relies on inputs corresponding to physical quantities, there is often an implicit assumption that these input arguments are expressed in the expected units of that function. For instance, if we define a function to calculate speed as above, the inputs should correspond to a distance and a time:
>>> def speed(length, time):
... return length / time
However, this assumes that the two arguments passed in have units consistent with distance and time without checking.
The quantity_input decorator, combined with function annotations, enforces compatible units on the function inputs:
>>> @u.quantity_input
... def speed(length: u.m, time: u.s):
... return length / time
Now when this function is called, if the inputs are not convertible to the units specified, an error will be raised stating that the units are incorrect or missing:
>>> speed(1*u.m, 10*u.m)
Traceback (most recent call last):
...
astropy.units.core.UnitsError: Argument 'time' to function 'speed' must be in units convertible to 's'.
>>> speed(1*u.m, 10)
...
Traceback (most recent call last):
...
TypeError: Argument 'time' to function 'speed' has no 'unit' attribute. ... pass in an astropy Quantity instead.
The units of the inputs need only be compatible with those in the function definition.
For example, passing in a time in minutes still works even though we specified time: u.s:
>>> speed(1*u.m, 1*u.minute)
<Quantity 1. m / min>
Note that the units of the output are dependent on the units of the inputs. To ensure consistent units on the output of our function, we add an additional function annotation to force the output to always be converted to m/s before returning an answer:
>>> @u.quantity_input
... def speed(length: u.m, time: u.s) -> u.m/u.s:
... return length / time
>>> speed(1*u.m, 1*u.minute)
<Quantity 0.01666667 m / s>
Times#
In this section of the tutorial, you will learn how times are represented and manipulated in sunpy.
sunpy makes extensive use of the astropy.time module for this task.
By the end of this section of the tutorial, you will learn how to parse times from different formats as well as construct and inspect time ranges.
Parsing Times#
Solar data is associated with a number of different time formats.
To handle all these formats, sunpy has sunpy.time.parse_time() that accepts a variety of inputs, and returns a consistent Time object.
You might have come across another way of storing time that’s built into Python itself, datetime.datetime.
datetime does not provide support for common time formats used in solar physics or leap seconds, hence the use of astropy.time.Time throughout sunpy.
Here’s a few examples of using parse_time to create Time objects:
>>> from sunpy.time import parse_time
>>> parse_time('2007-05-04T21:08:12')
<Time object: scale='utc' format='isot' value=2007-05-04T21:08:12.000>
>>> parse_time(894316092.00000000, format='utime')
<Time object: scale='utc' format='utime' value=894316092.0>
See the documentation for sunpy.time.parse_time for a full list of allowed arguments.
Time Ranges#
Another standard task in data analysis is dealing with pairs of times or time ranges.
To deal with time ranges sunpy provides the sunpy.time.TimeRange object.
A sunpy.time.TimeRange object can be created by providing a start time and an end time:
>>> from sunpy.time import TimeRange
>>> time_range = TimeRange('2010/03/04 00:10', '2010/03/04 00:20')
TimeRange makes use of sunpy.time.parse_time() so it can accept a wide variety of time formats.
Alternatively, you can specify a start time and a duration:
>>> import astropy.units as u
>>> time_range = TimeRange('2010/03/04 00:10', 400 * u.second)
The time range objects provides a number of useful functions. For example, you can easily get the time at the center of your interval or the length of the interval:
>>> time_range.center
<Time object: scale='utc' format='isot' value=2010-03-04T00:13:20.000>
>>> time_range.seconds
<Quantity 400. s>
A time range can also be easily split into sub-intervals of equal length, for example to split a TimeRange object into two new TimeRange objects:
>>> time_range.split(2)
[ <sunpy.time.timerange.TimeRange object ...>
Start: 2010-03-04 00:10:00
End: 2010-03-04 00:13:20
Center:2010-03-04 00:11:40
Duration:0.002314814814814825 days or
0.0555555555555558 hours or
3.333333333333348 minutes or
200.00000000000088 seconds
, <sunpy.time.timerange.TimeRange object ...>
Start: 2010-03-04 00:13:20
End: 2010-03-04 00:16:40
Center:2010-03-04 00:15:00
Duration:0.002314814814814825 days or
0.0555555555555558 hours or
3.333333333333348 minutes or
200.00000000000088 seconds
]
Check out the code reference for the sunpy.time.TimeRange object for more information.
Coordinates#
This section of the guide introduces how coordinates are represented in sunpy.
sunpy makes use of the astropy.coordinates module for this task.
In much the same way as units are used for representing physical quantities, sunpy uses astropy.coordinates to represent points in physical space.
This applies to both points in 3D space and projected coordinates in images.
The astropy coordinates module is primarily used through the SkyCoord class, which also makes use of the astropy units system:
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
To enable the use of the solar physics specific frames defined in sunpy we also need to import them:
>>> from sunpy.coordinates import frames
A SkyCoord object to represent a point on the Sun can then be created:
>>> coord = SkyCoord(70*u.deg, -30*u.deg, obstime="2017-08-01",
... frame=frames.HeliographicStonyhurst)
>>> coord
<SkyCoord (HeliographicStonyhurst: obstime=2017-08-01T00:00:00.000, rsun=695700.0 km): (lon, lat) in deg
(70., -30.)>
This SkyCoord object can then be transformed to any other coordinate frame defined either in Astropy or sunpy (see Supported Coordinate Systems for a list of sunpy frames), for example to transform from the original Stonyhurst frame to a Helioprojective frame:
>>> coord.transform_to(frames.Helioprojective(observer="earth"))
<SkyCoord (Helioprojective: obstime=2017-08-01T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (Tx, Ty, distance) in (arcsec, arcsec, km)
(769.96270814, -498.89715922, 1.51668773e+08)>
It is also possible to convert three dimensional positions to astrophysical frames defined in Astropy, for example ICRS.
>>> coord.transform_to('icrs')
<SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, km)
(49.84856512, 0.05394699, 1417743.94689472)>
SkyCoord and all coordinate frames support array coordinates.
These work the same as single-value coordinates, but they store multiple coordinates in a single object.
When you’re going to apply the same operation to many different coordinates, this is a better choice than a list of SkyCoord objects, because it will be much faster than applying the operation to each SkyCoord in a for loop:
>>> coord = SkyCoord([-500, 400]*u.arcsec, [100, 200]*u.arcsec, frame=frames.Helioprojective)
>>> coord
<SkyCoord (Helioprojective: obstime=None, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec
[(-500., 100.), ( 400., 200.)]>
>>> coord[0]
<SkyCoord (Helioprojective: obstime=None, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec
(-500., 100.)>
Observer Location#
Both Helioprojective and Heliocentric frames are defined based on the position of the observer.
Therefore to transform either of these frames to a different frame the location of the observer must be known.
The observer can be specified for a coordinate object using the observer argument to SkyCoord.
For sunpy to calculate the location of Earth or another solar-system body, it must know the time associated with the coordinate; this is specified with the obstime argument.
Using the observer location it is possible to convert a coordinate as seen by one observer to a coordinate seen by another:
>>> hpc = SkyCoord(0*u.arcsec, 0*u.arcsec, observer="earth",
... obstime="2017-07-26",
... frame=frames.Helioprojective)
>>> hpc.transform_to(frames.Helioprojective(observer="venus",
... obstime="2017-07-26"))
<SkyCoord (Helioprojective: obstime=2017-07-26T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'venus'>): (Tx, Ty, distance) in (arcsec, arcsec, AU)
(-1285.47497992, 106.20918654, 0.72405937)>
Using Coordinates with Maps#
sunpy Map uses coordinates to specify locations on the image, and to plot overlays on plots of maps.
When a Map is created, a coordinate frame is constructed from the header information.
This can be accessed using .coordinate_frame:
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
>>> import sunpy.map
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> amap = sunpy.map.Map(AIA_171_IMAGE)
>>> amap.coordinate_frame
<Helioprojective Frame (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>)>
This can be used when creating a SkyCoord object to set the coordinate system to that image:
>>> coord = SkyCoord(100 * u.arcsec, 10*u.arcsec, frame=amap.coordinate_frame)
>>> coord
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(100., 10.)>
The SkyCoord object can be converted to a pair of pixels using GenericMap.wcs.world_to_pixel:
>>> pixels = amap.wcs.world_to_pixel(coord)
>>> pixels
(array(551.7680511), array(515.18266871))
This SkyCoord object could also be used to plot a point on top of the map:
>>> import matplotlib.pyplot as plt
>>> fig = plt.figure()
>>> ax = plt.subplot(projection=amap)
>>> amap.plot()
<matplotlib.image.AxesImage object at ...>
>>> ax.plot_coord(coord, 'o')
[<matplotlib.lines.Line2D object at ...]
(Source code, png, hires.png, pdf)
For more information on coordinates see Coordinates.
Acquiring Data#
This section of the tutorial introduces ways to obtain different kind of solar data from different places.
The main tutorial below focuses on Fido, which is a generic search interface that sunpy provides.
There are some data-provider specific tutorials that you might want to visit after going through this tutorial:
Finding and Downloading Data from JSOC#
Joint Science Operations Center (JSOC) contains data products from the Solar Dynamics Observatory, as well as certain other missions and instruments. These data are available from the JSOC database, which can be directly accessed by the online JSOC interface.
sunpy’s JSOC Client provides an easier interface to query for JSOC data and make export requests. It uses drms module as its backend.
There are two ways of downloading JSOC data.
One way is using sunpy’s unified search interface, known as Fido.
Fido supplies a single, easy and consistent way to to obtain most forms of solar physics data.
An alternative way to fetch data from JSOC is by using the underlying JSOC Client.
This option can be preferred when you need to separate the staging and downloading steps, which is not supported by Fido.
The JSOC stages data before you can download it, so a JSOC query is a three stage process.
First you query the JSOC for records and a table of these records is returned.
Then you can request these records to be staged for download and then you can download them.
Fido combines the last two stages into a single call to fetch.
Setup#
sunpy’s Fido module is in sunpy.net.
It can be imported as follows:
>>> from sunpy.net import Fido, attrs as a
The JSOC client handles the particulars of how the data from the data provider is downloaded to your computer.
Warning
You must have an email address registered with JSOC before you are allowed to make a request. See this to register your email address.
Querying the JSOC#
To search for data in JSOC, your query needs at minimum, a “Series” name and a “PrimeKey”.
>>> print(a.jsoc.Series)
sunpy.net.jsoc.attrs.Series
The JSOC Series to Download.
Attribute Name Client Full Name Description
---------------------------------- ------ ---------------------------------- --------------------------------------------------------------------------------
aia_flatfield JSOC aia.flatfield AIA flatfield
aia_lev1 JSOC aia.lev1 AIA Level 1
aia_lev1_euv_12s JSOC aia.lev1_euv_12s AIA Level 1, 12 second cadence
aia_lev1_uv_24s JSOC aia.lev1_uv_24s AIA Level 1, 24 second cadence
aia_lev1_vis_1h JSOC aia.lev1_vis_1h AIA Level 1, 3600 second cadence
aia_master_pointing3h JSOC aia.master_pointing3h Master Pointing Parameters
aia_response JSOC aia.response AIA instrument response table
aia_temperature_summary_300s JSOC aia.temperature_summary_300s Temperature Statistics from AIA Housekeeping - Thermal Packet
hmi_b_135s JSOC hmi.b_135s Full-disk Milne-Eddington inversion with the azimuth disambiguation informati...
...
Different PrimeKeys are supported by different Series, and you can find out the PrimeKeys supported in any Series by:
>>> import drms
>>> client = drms.Client()
>>> print(client.pkeys('hmi.m_720s'))
['T_REC', 'CAMERA']
The most common PrimeKey, that is supported by every Series is Time, that is denoted by T_REC or T_OBS.
Hence, Time can always be passed as an attribute while building a query.
Wavelength is another pre-defined attribute which is a PrimeKey.
Other PrimeKeys which need to be passed should be manually passed in PrimeKey.
This will be explained later in this tutorial.
Constructing a Basic Query#
Let’s start with a very simple query. We could ask for all “hmi.v_45s” series data between January 1st 2014 from 00:00 to 01:00:
>>> res = Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s'))
This returns an UnifiedResponse object containing information on the available online files which fit the criteria specified by the attrs objects in the above call.
It does not download the files.
To see a summary of results of our query, simply type the name of the variable set to the Fido search, in this case, res:
>>> res
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2014.01.01_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:01:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:02:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:04:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:05:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:07:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
... ... ... ... ...
2014.01.01_00:54:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:54:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:55:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:56:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:58:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
Length = 81 rows
Now, let’s break down the arguments of Fido.search to understand better what we’ve done.
The first argument a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00') sets the start and end times for the query (any date/time format understood by sunpy’s parse_time can be used to specify dates and time).
The Time attribute takes UTC time, as default.
If you need to pass a Time in some other time scale, such as TAI, pass an astropy.time.Time object:
>>> import astropy.time
Then, the Time attribute can be passed as:
>>> a.Time(astropy.time.Time('2014-01-01T00:00:00', scale='tai'), astropy.time.Time('2014-01-01T01:00:00', scale='tai'))
<sunpy.net.attrs.Time(2014-01-01 00:00:00.000, 2014-01-01 01:00:00.000)>
The second argument:
>>> a.jsoc.Series('hmi.v_45s')
<sunpy.net.jsoc.attrs.Series(hmi.v_45s: Dopplergrams with a cadence of 45 seconds) object ...>
sets the series we are looking for.
So what is going on here?
The notion is that a JSOC query has a set of attribute objects, imported as a.jsoc, that are specified to construct the query.
a.jsoc.Series() is compulsory to be provided in each of the jsoc queries.
Apart from this, at least one PrimeKey must be passed (generally a.Time()).
Querying with other PrimeKeys#
Other than Time, one other PrimeKey is supported with in-built attribute.
In case of AIA series, a.Wavelength() can be passed as a PrimeKey:
>>> import astropy.units as u
>>> res = Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('aia.lev1_euv_12s'),
... a.Wavelength(304*u.AA))
Note that, only Time and Wavelength are in-built attributes here. If you need to pass any other PrimeKey, it should be passed like this:
>>> a.jsoc.PrimeKey('HARPNUM', '4864')
<sunpy.net.jsoc.attrs.PrimeKey object at ...>
('HARPNUM', '4864')
If 2 or more PrimeKeys need to be passed together:
>>> a.jsoc.PrimeKey('HARPNUM', '4864') & a.jsoc.PrimeKey('CAMERA', '2')
<AttrAnd([<sunpy.net.jsoc.attrs.PrimeKey object at ...>
('HARPNUM', '4864'), <sunpy.net.jsoc.attrs.PrimeKey object at ...>
('CAMERA', '2')])>
Also, note that the pre-defined PrimeKeys, Time and Wavelength can also be passed as above, but you need to specify the exact keyword for it:
>>> a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'), a.jsoc.PrimeKey('WAVELNTH', '161')
(<sunpy.net.attrs.Time(2014-01-01 00:00:00.000, 2014-01-01 01:00:00.000)>, <sunpy.net.jsoc.attrs.PrimeKey object at ...>
('WAVELNTH', '161'))
If the correct keyword is not specified, or the passed PrimeKey is not supported by the given series, a meaningful error will be thrown, which will give you the PrimeKeys supported by that series. Hence, by looking at the error, one can easily retry building the query with correct PrimeKeys.
Another important thing to note is that, Wavelength when passed through in-built attribute, should be passed as an astropy quantity.
Specifying spectral units in arguments is necessary or an error will be raised.
For more information on units, see units.
But, when the same is passed through PrimeKey attribute, it should be passed as a string.
All other PrimeKey values passed through PrimeKey attribute, must be passed as a string.
Manually specifying keyword data to fetch#
Upon doing Fido.search() as described above, only a limited set of keywords are returned in the response object.
These default keywords are 'DATE', 'TELESCOP', 'INSTRUME', 'T_OBS' and 'WAVELNTH'.
If you want to get a manual set of keywords in the response object, you can pass the set of keywords using show() method.
>>> res = Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s'))
>>> res.show('TELESCOP', 'INSTRUME', 'T_OBS')
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
TELESCOP INSTRUME T_OBS
-------- ---------- -----------------------
SDO/HMI HMI_FRONT2 2014.01.01_00:00:37_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:01:22_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:02:07_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:02:52_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:03:37_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:04:22_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:05:07_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:05:52_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:06:37_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:07:22_TAI
... ... ...
SDO/HMI HMI_FRONT2 2014.01.01_00:53:52_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:54:37_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:55:22_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:56:07_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:56:52_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:57:37_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:58:22_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:59:07_TAI
SDO/HMI HMI_FRONT2 2014.01.01_00:59:52_TAI
SDO/HMI HMI_FRONT2 2014.01.01_01:00:37_TAI
Length = 81 rows
Passing an incorrect keyword won’t throw an error, but the corresponding column in the table will not be displayed.
To display all of the columns, we can use show() without passing any arguments:
>>> res.show()
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
DATE DATE__OBS DATE-OBS ... CODEVER3 CALVER64
-------------------- ----------------------- ----------------------- ... ------------------------------------------------------ --------
2014-01-05T17:46:02Z 2013-12-31T23:59:39.20Z 2013-12-31T23:59:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:47:10Z 2014-01-01T00:00:24.20Z 2014-01-01T00:00:24.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:48:18Z 2014-01-01T00:01:09.20Z 2014-01-01T00:01:09.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:49:25Z 2014-01-01T00:01:54.20Z 2014-01-01T00:01:54.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:50:34Z 2014-01-01T00:02:39.20Z 2014-01-01T00:02:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:51:42Z 2014-01-01T00:03:24.20Z 2014-01-01T00:03:24.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:52:50Z 2014-01-01T00:04:09.20Z 2014-01-01T00:04:09.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:53:59Z 2014-01-01T00:04:54.20Z 2014-01-01T00:04:54.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:55:08Z 2014-01-01T00:05:39.20Z 2014-01-01T00:05:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:56:16Z 2014-01-01T00:06:24.20Z 2014-01-01T00:06:24.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
... ... ... ... ... ...
2014-01-05T17:35:43Z 2014-01-01T00:52:54.20Z 2014-01-01T00:52:54.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:36:54Z 2014-01-01T00:53:39.20Z 2014-01-01T00:53:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:38:01Z 2014-01-01T00:54:24.20Z 2014-01-01T00:54:24.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:39:09Z 2014-01-01T00:55:09.20Z 2014-01-01T00:55:09.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:40:17Z 2014-01-01T00:55:54.20Z 2014-01-01T00:55:54.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:41:25Z 2014-01-01T00:56:39.20Z 2014-01-01T00:56:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:42:33Z 2014-01-01T00:57:24.20Z 2014-01-01T00:57:24.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:43:41Z 2014-01-01T00:58:09.20Z 2014-01-01T00:58:09.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:44:52Z 2014-01-01T00:58:54.20Z 2014-01-01T00:58:54.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
2014-01-05T17:46:03Z 2014-01-01T00:59:39.20Z 2014-01-01T00:59:39.20Z ... $Id: polcal.c,v 1.5 2013/12/22 22:54:08 couvidat Exp $ 4370
Length = 81 rows
Using Segments#
In some cases, more than 1 file are present for the same set of query. These data are distinguished by what are called Segments. It is necessary to specify the Segment which you need to download. Providing a segment won’t have any affect on the response object returned, but this will be required later, while making an export request.
A list of supported segments of a series, say hmi.sharp_720s can be obtained by:
>>> client = drms.Client()
>>> si = client.info('hmi.sharp_720s')
>>> print(si.segments.index.values)
['magnetogram' 'bitmap' 'Dopplergram' 'continuum' 'inclination' 'azimuth'
'field' 'vlos_mag' 'dop_width' 'eta_0' 'damping' 'src_continuum'
'src_grad' 'alpha_mag' 'chisq' 'conv_flag' 'info_map' 'confid_map'
'inclination_err' 'azimuth_err' 'field_err' 'vlos_err' 'alpha_err'
'field_inclination_err' 'field_az_err' 'inclin_azimuth_err'
'field_alpha_err' 'inclination_alpha_err' 'azimuth_alpha_err' 'disambig'
'conf_disambig']
Also, if you provide an incorrect segment name, it will throw a meaningful error, specifying which segment values are supported by the given series:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.sharp_720s'),
... a.jsoc.Segment('image'))
Traceback (most recent call last):
...
ValueError: Unexpected Segments were passed. The series hmi.sharp_720s contains the following Segments ['magnetogram', 'bitmap', 'Dopplergram', 'continuum', 'inclination', 'azimuth', 'field', 'vlos_mag', 'dop_width', 'eta_0', 'damping', 'src_continuum', 'src_grad', 'alpha_mag', 'chisq', 'conv_flag', 'info_map', 'confid_map', 'inclination_err', 'azimuth_err', 'field_err', 'vlos_err', 'alpha_err', 'field_inclination_err', 'field_az_err', 'inclin_azimuth_err', 'field_alpha_err', 'inclination_alpha_err', 'azimuth_alpha_err', 'disambig', 'conf_disambig']
To get files for more than 1 segment at the same time, chain a.jsoc.Segment() using AND operator:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.sharp_720s'),
... a.jsoc.Segment('continuum') & a.jsoc.Segment('magnetogram'))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
61 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- --------- -------- -------
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
... ... ... ... ...
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
Length = 61 rows
Using Keywords#
In some cases, you might want to filter out files based on key metadata, also called keywords.
A list of supported keywords of a series, say hmi.sharp_720s can be obtained by:
>>> client = drms.Client()
>>> keywords = client.keys('hmi.sharp_720s')
>>> print(keywords)
['cparms_sg000', 'magnetogram_bzero', 'magnetogram_bscale', 'cparms_sg001', 'bitmap_bzero', 'bitmap_bscale', 'cparms_sg002', 'Dopplergram_bzero', 'Dopplergram_bscale', 'cparms_sg003', 'continuum_bzero', 'continuum_bscale', 'cparms_sg004', 'inclination_bzero', 'inclination_bscale', 'cparms_sg005', 'azimuth_bzero', 'azimuth_bscale', 'cparms_sg006', 'field_bzero', 'field_bscale', 'cparms_sg007', ... 'ERRJHT', 'ERRVF']
Each keyword needs to be compared to a value, e.g., a.jsoc.Keyword("bitmap_bzero") == 0 or a.jsoc.Keyword("bitmap_bzero") > 1.
An example of this is:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.sharp_720s'),a.jsoc.Keyword('bitmap_bzero') == 0)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
61 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- --------- -------- -------
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
... ... ... ... ...
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
Length = 61 rows
You can pass multiple keywords and they will be chained together inside the query:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'), a.jsoc.Series('hmi.sharp_720s'),
... a.jsoc.Keyword('bitmap_bzero') == 0, a.jsoc.Keyword('continuum_bscale') > 0)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
61 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- --------- -------- -------
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
... ... ... ... ...
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:12:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:36:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_00:48:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_SIDE1 6173.0 2145
Length = 61 rows
If you provide a keyword without a comparison it will raise an error:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.sharp_720s'),
... a.jsoc.Keyword('bitmap_bzero'))
Traceback (most recent call last):
...
ValueError: Keyword 'bitmap_bzero' needs to have a comparison to a value.
If you provide an incorrect keyword name it will also raise a error:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.sharp_720s'),
... a.jsoc.Keyword('bac') == 0)
Traceback (most recent call last):
...
ValueError: Keyword: 'bac' is not supported by series: hmi.sharp_720s
Using Sample#
In case you need to query for data, at some interval of time, say every 10 min, you can pass it using Sample.
In other words, if you need to query for “hmi.v_45s” series data between January 1st 2014 from 00:00 to 01:00 at 10 minute intervals, you can do:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s'), a.Sample(10*u.min))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
7 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2014.01.01_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:10:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:20:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:30:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:39:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:49:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
Note that the argument passed in a.Sample() must be an Astropy quantity, convertible into seconds.
Constructing complex queries#
Complex queries can be built using “OR” operators. Let’s look for 2 different series data at the same time:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s') | a.jsoc.Series('aia.lev1_euv_12s'))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2014.01.01_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:01:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:02:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:04:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:05:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:07:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
... ... ... ... ...
2014.01.01_00:54:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:54:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:55:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:56:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:58:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
Length = 81 rows
2107 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
-------------------- -------- -------- -------- -------
2014-01-01T00:00:01Z SDO/AIA AIA_4 94 2145
2014-01-01T00:00:01Z SDO/AIA AIA_1 131 2145
2014-01-01T00:00:01Z SDO/AIA AIA_3 171 2145
2014-01-01T00:00:01Z SDO/AIA AIA_2 193 2145
2014-01-01T00:00:01Z SDO/AIA AIA_2 211 2145
2014-01-01T00:00:01Z SDO/AIA AIA_4 304 2145
2014-01-01T00:00:01Z SDO/AIA AIA_1 335 2145
2014-01-01T00:00:13Z SDO/AIA AIA_4 94 2145
2014-01-01T00:00:13Z SDO/AIA AIA_1 131 2145
2014-01-01T00:00:13Z SDO/AIA AIA_3 171 2145
... ... ... ... ...
2014-01-01T00:59:49Z SDO/AIA AIA_2 211 2145
2014-01-01T00:59:49Z SDO/AIA AIA_4 304 2145
2014-01-01T00:59:49Z SDO/AIA AIA_1 335 2145
2014-01-01T01:00:01Z SDO/AIA AIA_4 94 2145
2014-01-01T01:00:01Z SDO/AIA AIA_1 131 2145
2014-01-01T01:00:01Z SDO/AIA AIA_3 171 2145
2014-01-01T01:00:01Z SDO/AIA AIA_2 193 2145
2014-01-01T01:00:01Z SDO/AIA AIA_2 211 2145
2014-01-01T01:00:01Z SDO/AIA AIA_4 304 2145
2014-01-01T01:00:01Z SDO/AIA AIA_1 335 2145
Length = 2107 rows
The two series names are joined together by the operator |.
This is the “OR” operator.
Think of the above query as setting a set of conditions which get passed to the JSOC.
Let’s say you want all the “hmi.v_45s” data from two separate days:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00') |
... a.Time('2014-01-02T00:00:00', '2014-01-02T01:00:00'),
... a.jsoc.Series('hmi.v_45s'))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2014.01.01_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:01:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:02:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:04:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:05:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:06:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:07:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
... ... ... ... ...
2014.01.01_00:54:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:54:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:55:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:56:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:58:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
Length = 81 rows
81 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2014.01.02_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:01:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:02:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:03:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:03:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:04:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:05:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:06:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:06:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:07:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
... ... ... ... ...
2014.01.02_00:54:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:54:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:55:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:56:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:57:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:57:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:58:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_01:00:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.02_01:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
Length = 81 rows
Each of the arguments in this query style can be thought of as setting conditions that the returned records must satisfy.
It should be noted that AND operator is supported by some of the attributes only.
The attributes which support “&” are PrimeKey and Segment.
Using “&” with any other attributes will throw an error.
Downloading data#
To download the files located by search,
you can download them by fetch:
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s') | a.jsoc.Series('aia.lev1_euv_12s'),
... a.jsoc.Notify('solar@example.com')
>>> downloaded_files = Fido.fetch(res)
To export a request for download, you must have used the sunpy.net.jsoc.attrs.Notify attribute at search time to specify your email address.
Note
Only complete searches can be downloaded from JSOC This means that no slicing operations performed on the results object will affect the number of files downloaded.
Using JSOCClient for complex usage#
Fido interface uses JSOCClient in its backend, and combines the last 2 stages the JSOC process into one.
You can directly use the JSOC client to make queries, instead of the Fido client.
This will allow you to separate the 3 stages of the JSOC process, and perform it individually, hence allowing a greater control over the whole process.
Setup#
sunpy’s JSOC module is in net.
It can be imported as follows:
>>> from sunpy.net import jsoc
>>> client = jsoc.JSOCClient()
This creates your client object.
Making a query#
Querying JSOC using the JSOC client is very similar to what we were doing with Fido.
As above, we have to make sure we have an email address registered with JSOC before you are allowed to make a request.
See this to register your email address.
We can add an email address to the search query with the sunpy.net.jsoc.attrs.Notify attribute.
Please note you can search without this but right now, you can not add the email address after the search:
>>> res = client.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
... a.jsoc.Series('hmi.v_45s'),
... a.jsoc.Notify('sunpy@sunpy.org'))
Apart from the function name, everything is the same.
You need to pass the same values in the search as you did in search.
Complex queries can be built in a similar way, and all other things are the same.
Staging the request#
JSOC is a 3-stage process, and after getting the query results, we need to stage a request for the data to be downloaded. Only then, can we download them. The download request can be staged like this:
>>> requests = client.request_data(res)
>>> print(requests)
<ExportRequest id="JSOC_20170713_1461", status=0>
The function request_data stages the request.
It returns a drms.client.ExportRequest object, which has many attributes.
The most important ones are id and status.
Only when the status is 0, we can move to the third step, i.e., downloading the data.
If you are making more than 1 query at a time, it will return a list of ExportRequest objects.
Hence, access the list elements accordingly.
You can get the id and status of the request (if it is not a list) by:
>>> requests.id
JSOC_20170713_1461
>>> requests.status
0
Downloading data#
Once the status code is 0 you can download the data using the get_request method:
>>> res = client.get_request(requests)
This returns a Results instance which can be used to watch the progress of the download:
>>> res.wait(progress=True)
Searching the Heliophysics Event Knowledgebase#
The Heliophysics Event Knowledgebase (HEK) is a repository of feature and event information about the Sun.
Entries are generated both by automated algorithms and human observers.
sunpy accesses this information through the hek module, which was developed through support from the European Space Agency Summer of Code in Space (ESA-SOCIS) 2011.
A simple query#
To search the HEK, you need a start time, an end time, and an event type. Times are specified as strings or Python datetime objects. Event types are specified as upper case, two letter strings, and are identical to the two letter abbreviations found at the HEK website, http://www.lmsal.com/hek/VOEvent_Spec.html.
>>> from sunpy.net import attrs as a
>>> from sunpy.net import Fido
>>> tstart = '2011/08/09 07:23:56'
>>> tend = '2011/08/09 12:40:29'
>>> event_type = 'FL'
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type))
tstart and tend defines the start and end times of the query, and event_type specifies the event type which in this example we are searching for flares defined as FL.
Fido.search goes out to the web, contacts the HEK, and queries it for the information you have requested.
Event data for ALL flares available in the HEK within the time range 2011/08/09 07:23:56 UT - 2011/08/09 12:40:20 UT will be returned, regardless of which feature recognition method used to detect the flare.
Let’s break down the arguments of Fido.search.
The first argument:
>>> a.Time(tstart,tend)
sets the start and end times for the query.
The second argument:
>>> a.hek.EventType(event_type)
sets the type of event to look for.
Since we have defined event_type = 'FL', this sets the query to look for flares.
We could have also set the flare event type using the syntax:
>>> a.hek.FL
There is more information on the attributes below.
The result#
So, how many flare detections did the query turn up?
The result object returned by Fido.search is a UnifiedResponse object which contains all the results from any clients used in the search.
The first thing we need to do is access the results from the HEK client, the only ones for the query we gave:
>>> len(result['hek'])
19
This object is an astropy.table.Table object with the columns which correspond to the parameters listed at http://www.lmsal.com/hek/VOEvent_Spec.html.
You can inspect all results very simply:
>>> result['hek']
Remember, the HEK query we made returns all the flares in the time-range stored in the HEK, regardless of the feature recognition method. The HEK parameter which stores the the feature recognition method is called “frm_name”. We can select just this column:
>>> result["hek"]["frm_name"]
<QueryResponseColumn name='frm_name' dtype='str32' length=19>
asainz
asainz
asainz
asainz
asainz
asainz
asainz
SSW Latest Events
SWPC
Flare Detective - Trigger Module
Flare Detective - Trigger Module
SWPC
SSW Latest Events
Flare Detective - Trigger Module
Flare Detective - Trigger Module
Flare Detective - Trigger Module
Flare Detective - Trigger Module
Flare Detective - Trigger Module
Flare Detective - Trigger Module
It is likely each flare on the Sun was actually detected multiple times by many different methods.
More complex queries#
There are two key features you need to know in order to make more complex queries.
Firstly, the attribute module - attrs.hek - describes all the parameters stored by the HEK as listed in http://www.lmsal.com/hek/VOEvent_Spec.html, and the HEK client makes these parameters searchable.
To explain this, let’s have a closer look at attrs.hek.
By using the help command; scroll down to section DATA you will see:
>>> help(a.hek)
Help on module sunpy.net.hek.attrs in sunpy.net.hek:
NAME
sunpy.net.hek.attrs
DESCRIPTION
Attributes that can be used to construct HEK queries. They are different to
the VSO ones in that a lot of them are wrappers that conveniently expose
the comparisons by overloading Python operators. So, e.g., you are able
to say AR & AR.NumSpots < 5 to find all active regions with less than 5 spots.
As with the VSO query, you can use the fundamental logic operators AND and OR
to construct queries of almost arbitrary complexity. Note that complex queries
result in multiple requests to the server which might make them less efficient.
CLASSES
...
You’ll see that one of the attributes is a flare object:
FL = <sunpy.net.hek.attrs.FL object>
We can replace a.hek.EventType('FL') with a.hek.FL - they do the same thing, setting the query to look for flare events.
Both methods of setting the event type are provided as a convenience
Let’s look further at the FRM attribute:
>>> help(a.hek.FRM)
Help on FRM in module sunpy.net.hek.attrs object:
class FRM(builtins.object)
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables
|
| __weakref__
| list of weak references to the object
|
| ----------------------------------------------------------------------
| Data and other attributes defined here:
|
| Contact = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| HumanFlag = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| Identifier = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| Institute = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| Name = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| ParamSet = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| SpecificID = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| URL = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
|
| VersionNumber = <sunpy.net.hek.attrs._StringParamAttrWrapper object>
Let’s say I am only interested in those flares identified by the SSW Latest Events tool. One can retrieve those entries only from the HEK with the following command:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), a.hek.FRM.Name == 'SSW Latest Events')
>>> len(result[0])
2
We can also retrieve all the entries in the time range which were not made by SSW Latest Events with the following command:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), a.hek.FRM.Name != 'SSW Latest Events')
>>> len(result[0])
19
We are using Python’s comparison operators to filter the returns from Fido. Other comparisons are possible. For example, let’s say I want all the flares that have a peak flux of over 4000.0:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), a.hek.FL.PeakFlux > 4000.0)
>>> len(result[0])
1
Multiple comparisons can be included. For example, let’s say I want all the flares with a peak flux above 1000 AND west of 800 arcseconds from disk center of the Sun:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), a.hek.Event.Coord1 > 800, a.hek.FL.PeakFlux > 1000)
>>> len(result[0])
7
Multiple comparison operators can be used to filter the results back from the HEK.
The second important feature is that the comparisons we’ve made above can be combined using Python’s logical operators. This makes complex queries easy to create. However, some caution is advisable. Let’s say we want all the flares west of 50 arcseconds OR have a peak flux over 1000:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), (a.hek.Event.Coord1 > 50) or (a.hek.FL.PeakFlux > 1000))
and as a check:
>>> result["hek"]["fl_peakflux"]
<QueryResponseColumn name='fl_peakflux' dtype='object' length=17>
None
None
None
None
None
None
None
2326.86
1698.83
None
None
2360.49
3242.64
1375.93
6275.98
923.984
1019.83
>>> result["hek"]["event_coord1"]
<QueryResponseColumn name='event_coord1' dtype='float64' length=17>
51.0
51.0
51.0
924.0
924.0
924.0
69.0
883.2
883.2
69.0
69.0
883.2
883.2
883.2
883.2
883.2
883.2
Note that some of the fluxes are returned as “None”. This is because some feature recognition methods for flares do not report the peak flux. However, because the location of “event_coord1” is greater than 50, the entry from the HEK for that flare detection is returned.
Let’s say we want all the flares west of 50 arcseconds AND have a peak flux over 1000:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type), (a.hek.Event.Coord1 > 50) and (a.hek.FL.PeakFlux > 1000))
>>> result["hek"]["fl_peakflux"]
<QueryResponseColumn name='fl_peakflux' dtype='float64' length=7>
2326.86
1698.83
2360.49
3242.64
1375.93
6275.98
1019.83
>>> result["hek"]["event_coord1"]
<QueryResponseColumn name='event_coord1' dtype='float64' length=7>
883.2
883.2
883.2
883.2
883.2
883.2
883.2
In this case none of the peak fluxes are returned with the value None.
Since we are using an and logical operator we need a result from the (a.hek.FL.PeakFlux > 1000) filter.
Flares that have None for a peak flux cannot provide this, and so are excluded.
The None type in this context effectively means “Don’t know”; in such cases the client returns only those results from the HEK that definitely satisfy the criteria passed to it.
Getting data for your event#
The sunpy.net.hek2vso module allows you to take an HEK event and acquire VSO records specific to that event.
>>> from sunpy.net import hek2vso
>>> h2v = hek2vso.H2VClient()
There are several ways to use this capability. For example, you can pass in a list of HEK results and get out the corresponding VSO records. Here are the VSO records returned via the tenth result from the HEK query in Section 2 above:
>>> result = Fido.search(a.Time(tstart,tend), a.hek.EventType(event_type))
>>> vso_records = h2v.translate_and_query(result[0][10])
>>> len(vso_records[0])
31
result[0][10] is the HEK entry generated by the “Flare Detective” automated flare detection algorithm running on the AIA 193 angstrom waveband.
The VSO records are for full disk AIA 193 angstrom images between the start and end times of this event.
The translate_and_query function uses exactly that information supplied by the HEK in order to find the relevant data for that event.
Note that the VSO does not generate records for all solar data, so it is possible that an HEK entry corresponds to data that is not accessible via the VSO.
You can also go one step further back, passing in a list of HEK attribute objects to define your search, the results of which are then used to generate their corresponding VSO records:
>>> vso_query = h2v.full_query((a.Time('2011/08/09 07:00:00', '2011/08/09 07:15:00'), a.hek.EventType('FL')))
The full capabilities of the HEK query module can be used in this function (see above).
Finally, for greater flexibility, it is possible to pass in a list of HEK results and create the corresponding VSO query attributes.
>>> vso_query = hek2vso.translate_results_to_query(result[0][10])
>>> vso_query[0]
[<sunpy.net.attrs.Time(2011-08-09 07:22:44.000, 2011-08-09 07:28:56.000)>, <sunpy.net.attrs.Source(SDO: The Solar Dynamics Observatory.) object at ...>, <sunpy.net.attrs.Instrument(AIA: Atmospheric Imaging Assembly) object at ...>, <sunpy.net.attrs.Wavelength(193.0, 193.0, 'Angstrom')>]
This function allows users finer-grained control of VSO queries generated from HEK results.
This guide outlines how to search for and download data using the Fido interface for search and download.
Fido is a unified interface for searching and fetching solar physics data irrespective of the underlying client or web service through which the data is obtained.
It therefore supplies a single, easy, and consistent way to obtain most forms of solar physics data.
The Fido object is in sunpy.net.
All the examples in this guide use Fido, so lets start by importing it:
>>> from sunpy.net import Fido, attrs as a
Fido supports a number of different remote data sources. To see a list the Fido object can be printed:
>>> print(Fido)
sunpy.net.Fido
Fido is a unified data search and retrieval tool.
It provides simultaneous access to a variety of online data sources, some
cover multiple instruments and data products like the Virtual Solar
Observatory and some are specific to a single source.
For details of using `~sunpy.net.Fido` see :ref:`sunpy-tutorial-acquiring-data-index`.
Client Description
----------------- -------------------------------------------------------------------------------------------------------
CDAWEBClient Provides access to query and download from the Coordinated Data Analysis Web (CDAWeb).
EVEClient Provides access to Level 0CS Extreme ultraviolet Variability Experiment (EVE) data.
GBMClient Provides access to data from the Gamma-Ray Burst Monitor (GBM) instrument on board the Fermi satellite.
XRSClient Provides access to several GOES XRS files archive.
SUVIClient Provides access to data from the GOES Solar Ultraviolet Imager (SUVI).
GONGClient Provides access to the Magnetogram products of NSO-GONG synoptic Maps.
LYRAClient Provides access to the LYRA/Proba2 data archive.
NOAAIndicesClient Provides access to the NOAA solar cycle indices.
NOAAPredictClient Provides access to the NOAA SWPC predicted sunspot Number and 10.7 cm radio flux values.
SRSClient Provides access to the NOAA SWPC solar region summary data.
NoRHClient Provides access to the Nobeyama RadioHeliograph (NoRH) averaged correlation time series data.
RHESSIClient Provides access to the RHESSI observing summary time series data.
HEKClient Provides access to the Heliophysics Event Knowledgebase (HEK).
HECClient Provides access to the HELIO webservices.
JSOCClient Provides access to the JSOC Data Export service.
VSOClient Provides access to query and download from Virtual Solar Observatory (VSO).
Searching for Data#
To search for data with Fido, you need to specify attributes to search with.
Examples of generic search attributes that work across many different data sources are:
Some other attributes are client specific, and are found under client specific submodules, e.g. attrs.vso and attrs.jsoc.
The full list of attributes can be found in the attrs submodule reference.
Some search attributes need one or more values specifying, for example Time needs at least a start and an end date to specify a time range:
>>> a.Time('2012/3/4', '2012/3/6')
<sunpy.net.attrs.Time(2012-03-04 00:00:00.000, 2012-03-06 00:00:00.000)>
For attributes that can take a range of different values, printing the attribute lists the values sunpy knows about. These values are updated with every release of sunpy, so may not be always up to date! As an example:
>>> print(a.Instrument)
sunpy.net.attrs.Instrument
Specifies the Instrument name for the search.
Attribute Name Client Full Name Description
--------------------------- ----------- ------------------------ --------------------------------------------------------------------------------
aia VSO AIA Atmospheric Imaging Assembly
bcs VSO BCS Bragg Crystal Spectrometer
be_continuum VSO BE-Continuum INAF-OACT Barra Equatoriale Continuum Instrument
be_halpha VSO BE-Halpha INAF-OACT Barra Equatoriale Hα Instrument
bigbear VSO Big Bear Big Bear Solar Observatory, California TON and GONG+ sites
caii VSO CAII Kanzelhöhe Ca II k Instrument
cds VSO CDS Coronal Diagnostic Spectrometer
celias VSO CELIAS Charge, Element, and Isotope Analysis System
...
This is a full list of known values, a description, and which clients support those values (if you want to search using a specific data source).
Printing attributes like this is supported for most attributes, including client specific ones.
These attributes also support tab-completion to auto to auto-fill the attribute name, for example typing a.jsoc.aia_f<TAB> in a jupyter notebook will show you the available attributes that start with “aia_f”.
To search for data use the Fido.search method:
>>> result = Fido.search(a.Time('2012/3/4', '2012/3/6'), a.Instrument.lyra, a.Level.two)
this returns an UnifiedResponse object containing all the search results that match the search attributes.
This does not download the files; we’ll learn how to do that later in Downloading data.
To see a summary of the results print the result variable that came back from the previous search:
>>> print(result)
Results from 1 Provider:
3 Results from the LYRAClient:
Source: http://proba2.oma.be/lyra/data/bsd
Start Time End Time Instrument ... Provider Level
----------------------- ----------------------- ---------- ... -------- -----
2012-03-04 00:00:00.000 2012-03-04 23:59:59.999 LYRA ... ESA 2
2012-03-05 00:00:00.000 2012-03-05 23:59:59.999 LYRA ... ESA 2
2012-03-06 00:00:00.000 2012-03-06 23:59:59.999 LYRA ... ESA 2
Queries can be made more flexible or specific by adding more attrs objects to the Fido search.
As an example, specific passbands can be searched for by supplying a Quantity to the a.Wavelength attribute:
>>> import astropy.units as u
>>> Fido.search(a.Time('2012/3/4', '2012/3/4'), a.Instrument.aia,
... a.Wavelength(171*u.angstrom))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2012-03-04 00:00:00.000 2012-03-04 00:00:01.000 SDO ... FULLDISK 64.64844
Data of a given cadence can also be specified using the a.Sample attribute:
>>> Fido.search(a.Time('2012/3/4', '2012/3/6'), a.Instrument.aia,
... a.Wavelength(171*u.angstrom), a.Sample(120*u.minute))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
25 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 1.695 Gbyte
Start Time End Time Source Instrument Wavelength Provider Physobs Wavetype Extent Width Extent Length Extent Type Size
Angstrom Mibyte
----------------------- ----------------------- ------ ---------- -------------- -------- --------- -------- ------------ ------------- ----------- --------
2012-03-04 00:00:00.000 2012-03-04 00:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 02:00:00.000 2012-03-04 02:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 04:00:00.000 2012-03-04 04:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 06:00:00.000 2012-03-04 06:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 08:00:00.000 2012-03-04 08:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 10:00:00.000 2012-03-04 10:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 12:00:00.000 2012-03-04 12:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 14:00:00.000 2012-03-04 14:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-04 16:00:00.000 2012-03-04 16:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
... ... ... ... ... ... ... ... ... ... ... ...
2012-03-05 06:00:00.000 2012-03-05 06:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 08:00:00.000 2012-03-05 08:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 10:00:00.000 2012-03-05 10:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 12:00:00.000 2012-03-05 12:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 14:00:00.000 2012-03-05 14:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 16:00:00.000 2012-03-05 16:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 18:00:00.000 2012-03-05 18:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 20:00:00.000 2012-03-05 20:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-05 22:00:00.000 2012-03-05 22:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2012-03-06 00:00:00.000 2012-03-06 00:00:01.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
Length = 25 rows
To search for data from multiple instruments, wavelengths, times etc., use the pipe | operator which joins queries using a logical “OR” operator.
In this example we’ll search for LYRA or RHESSI data in a given time range:
>>> Fido.search(a.Time('2012/3/4', '2012/3/4 02:00'),
... a.Instrument.lyra | a.Instrument.rhessi)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 3 Providers:
2 Results from the LYRAClient:
Source: http://proba2.oma.be/lyra/data/bsd
Start Time End Time Instrument Physobs Source Provider Level
----------------------- ----------------------- ---------- ---------- ------ -------- -----
2012-03-04 00:00:00.000 2012-03-04 23:59:59.999 LYRA irradiance PROBA2 ESA 2
2012-03-04 00:00:00.000 2012-03-04 23:59:59.999 LYRA irradiance PROBA2 ESA 3
1 Results from the RHESSIClient:
Source: https://hesperia.gsfc.nasa.gov/hessidata
Start Time End Time Instrument Physobs Source Provider
----------------------- ----------------------- ---------- ------------------ ------ --------
2012-03-04 00:00:00.000 2012-03-04 23:59:59.999 RHESSI summary_lightcurve RHESSI NASA
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time End Time Source Instrument Wavelength Provider Physobs Extent Type Size
keV Mibyte
----------------------- ----------------------- ------ ---------- -------------- -------- --------- ----------- --------
2012-03-03 22:57:40.000 2012-03-04 00:33:20.000 RHESSI RHESSI 3.0 .. 17000.0 LSSP intensity PARTIAL_SUN -0.00098
2012-03-04 00:33:20.000 2012-03-04 01:45:40.000 RHESSI RHESSI 3.0 .. 17000.0 LSSP intensity PARTIAL_SUN -0.00098
2012-03-04 01:45:40.000 2012-03-04 02:09:00.000 RHESSI RHESSI 3.0 .. 17000.0 LSSP intensity PARTIAL_SUN -0.00098
Working with Search Results#
Fido.search can make multiple queries to multiple clients in one search.
This means that the results of a call to search can contain many sets of records, called responses, from many clients.
The results of a search are represented in a UnifiedResponse object, which provides access to all the response tables and allows some operations to be performed on all the results at once.
UnifiedResponse acts both like a two dimensional array, where the first dimension is the response index and the second index is the row index, and a dictionary where you can index the responses by the name of the client.
For example, the following code returns a response containing LYRA data from the LYRAClient, and EVE data from the VSOClient:
>>> results = Fido.search(a.Time("2012/1/1", "2012/1/2"), a.Level.two,
... a.Instrument.lyra | a.Instrument.eve)
>>> results
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
2 Results from the LYRAClient:
Source: http://proba2.oma.be/lyra/data/bsd
Start Time End Time Instrument Physobs Source Provider Level
----------------------- ----------------------- ---------- ---------- ------ -------- -----
2012-01-01 00:00:00.000 2012-01-01 23:59:59.999 LYRA irradiance PROBA2 ESA 2
2012-01-02 00:00:00.000 2012-01-02 23:59:59.999 LYRA irradiance PROBA2 ESA 2
50 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time End Time Source Instrument Wavelength Provider Physobs Extent Type Size
Angstrom Mibyte
----------------------- ----------------------- ------ ---------- -------------- -------- ---------- ----------- --------
2012-01-01 00:00:00.000 2012-01-01 01:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 00:00:00.000 2012-01-01 01:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 01:00:00.000 2012-01-01 02:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 01:00:00.000 2012-01-01 02:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 02:00:00.000 2012-01-01 03:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 02:00:00.000 2012-01-01 03:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 03:00:00.000 2012-01-01 04:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 03:00:00.000 2012-01-01 04:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 04:00:00.000 2012-01-01 05:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
... ... ... ... ... ... ... ... ...
2012-01-01 20:00:00.000 2012-01-01 21:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 20:00:00.000 2012-01-01 21:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 21:00:00.000 2012-01-01 22:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 21:00:00.000 2012-01-01 22:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 22:00:00.000 2012-01-01 23:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 22:00:00.000 2012-01-01 23:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-01 23:00:00.000 2012-01-02 00:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-01 23:00:00.000 2012-01-02 00:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
2012-01-02 00:00:00.000 2012-01-02 01:00:00.000 SDO EVE 93.0 .. 1033.0 LASP irradiance FULLDISK -0.00098
2012-01-02 00:00:00.000 2012-01-02 01:00:00.000 SDO EVE 60.0 .. 1060.0 LASP irradiance FULLDISK -0.00098
Length = 50 rows
If you then wanted to inspect just the LYRA data for the whole time range specified in the search, you would index this response to see just the results returned by the LYRAClient:
>>> results[0, :]
<sunpy.net.dataretriever.client.QueryResponse object at ...>
Start Time End Time Instrument ... Provider Level
----------------------- ----------------------- ---------- ... -------- -----
2012-01-01 00:00:00.000 2012-01-01 23:59:59.999 LYRA ... ESA 2
2012-01-02 00:00:00.000 2012-01-02 23:59:59.999 LYRA ... ESA 2
Or, equivalently:
>>> results["lyra"]
<sunpy.net.dataretriever.client.QueryResponse object at ...>
Start Time End Time Instrument ... Provider Level
----------------------- ----------------------- ---------- ... -------- -----
2012-01-01 00:00:00.000 2012-01-01 23:59:59.999 LYRA ... ESA 2
2012-01-02 00:00:00.000 2012-01-02 23:59:59.999 LYRA ... ESA 2
Normal slicing operations work as with any other Python sequence, e.g. results[1, ::10] to access every tenth file in the result returned by the second client.
Note that the first (response) index is still necessary even if results are only found for a single client.
So in this case the first result would be results[0, 0] rather than results[0] (the latter would return all results from the first - and only - client and is therefore the same as results).
As we have seen above the UnifiedResponse object contains many response tables which make up the search results.
Each of the responses are QueryResponseTable objects, which are astropy.table objects meaning that you can interact with them and filter them like any other tabular data.
This can be used to interact with results which are metadata only, i.e. searches from the HEK, or it can be used to reduce the number of files downloaded by Fido.fetch.
For example if we did a query for some AIA and HMI data:
>>> results = Fido.search(a.Time("2020/01/01", "2020/01/01 00:01"), a.Instrument.aia | a.Instrument.hmi)
>>> results
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
41 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 2.779 Gbyte
Start Time End Time Source Instrument Wavelength Provider Physobs Wavetype Extent Width Extent Length Extent Type Size
Angstrom Mibyte
----------------------- ----------------------- ------ ---------- ---------------- -------- --------- -------- ------------ ------------- ----------- --------
2020-01-01 00:00:00.000 2020-01-01 00:00:01.000 SDO AIA 335.0 .. 335.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:04.000 2020-01-01 00:00:05.000 SDO AIA 193.0 .. 193.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:05.000 2020-01-01 00:00:06.000 SDO AIA 304.0 .. 304.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:05.000 2020-01-01 00:00:06.000 SDO AIA 4500.0 .. 4500.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:06.000 2020-01-01 00:00:07.000 SDO AIA 131.0 .. 131.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:09.000 2020-01-01 00:00:10.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:09.000 2020-01-01 00:00:10.000 SDO AIA 211.0 .. 211.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:11.000 2020-01-01 00:00:12.000 SDO AIA 94.0 .. 94.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:12.000 2020-01-01 00:00:13.000 SDO AIA 335.0 .. 335.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
... ... ... ... ... ... ... ... ... ... ... ...
2020-01-01 00:00:47.000 2020-01-01 00:00:48.000 SDO AIA 94.0 .. 94.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:48.000 2020-01-01 00:00:49.000 SDO AIA 335.0 .. 335.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:52.000 2020-01-01 00:00:53.000 SDO AIA 1700.0 .. 1700.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:52.000 2020-01-01 00:00:53.000 SDO AIA 193.0 .. 193.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:53.000 2020-01-01 00:00:54.000 SDO AIA 304.0 .. 304.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:54.000 2020-01-01 00:00:55.000 SDO AIA 131.0 .. 131.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:57.000 2020-01-01 00:00:58.000 SDO AIA 171.0 .. 171.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:57.000 2020-01-01 00:00:58.000 SDO AIA 211.0 .. 211.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:00:59.000 2020-01-01 00:01:00.000 SDO AIA 94.0 .. 94.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
2020-01-01 00:01:00.000 2020-01-01 00:01:01.000 SDO AIA 335.0 .. 335.0 JSOC intensity NARROW 4096 4096 FULLDISK 64.64844
Length = 41 rows
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time End Time Source Instrument Wavelength Provider Physobs Wavetype Extent Width Extent Length Extent Type Size
Angstrom Mibyte
----------------------- ----------------------- ------ ---------- ---------------- -------- ------------------ -------- ------------ ------------- ----------- --------
2020-01-01 00:00:22.000 2020-01-01 00:00:23.000 SDO HMI 6173.0 .. 6174.0 JSOC intensity NARROW 4096 4096 FULLDISK -0.00098
2020-01-01 00:00:22.000 2020-01-01 00:00:23.000 SDO HMI 6173.0 .. 6174.0 JSOC LOS_magnetic_field NARROW 4096 4096 FULLDISK -0.00098
2020-01-01 00:00:22.000 2020-01-01 00:00:23.000 SDO HMI 6173.0 .. 6174.0 JSOC LOS_velocity NARROW 4096 4096 FULLDISK -0.00098
The VSO client returns a lot of information about the records, so the first thing we can do is show only the columns we are interested in.
We can inspect all the available column names in all the responses with the all_colnames property:
>>> results.all_colnames
['End Time', 'Extent Length', 'Extent Type', 'Extent Width', 'Instrument', 'Physobs', 'Provider', 'Size', 'Source', 'Start Time', 'Wavelength', 'Wavetype', 'fileid']
And then we can pick which ones to see with the show() method:
>>> results.show("Start Time", "Instrument", "Physobs", "Wavelength")
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
41 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time Instrument Physobs Wavelength
Angstrom
----------------------- ---------- --------- ----------------
2020-01-01 00:00:00.000 AIA intensity 335.0 .. 335.0
2020-01-01 00:00:04.000 AIA intensity 193.0 .. 193.0
2020-01-01 00:00:05.000 AIA intensity 304.0 .. 304.0
2020-01-01 00:00:05.000 AIA intensity 4500.0 .. 4500.0
2020-01-01 00:00:06.000 AIA intensity 131.0 .. 131.0
2020-01-01 00:00:09.000 AIA intensity 171.0 .. 171.0
2020-01-01 00:00:09.000 AIA intensity 211.0 .. 211.0
2020-01-01 00:00:11.000 AIA intensity 94.0 .. 94.0
2020-01-01 00:00:12.000 AIA intensity 335.0 .. 335.0
... ... ... ...
2020-01-01 00:00:47.000 AIA intensity 94.0 .. 94.0
2020-01-01 00:00:48.000 AIA intensity 335.0 .. 335.0
2020-01-01 00:00:52.000 AIA intensity 1700.0 .. 1700.0
2020-01-01 00:00:52.000 AIA intensity 193.0 .. 193.0
2020-01-01 00:00:53.000 AIA intensity 304.0 .. 304.0
2020-01-01 00:00:54.000 AIA intensity 131.0 .. 131.0
2020-01-01 00:00:57.000 AIA intensity 171.0 .. 171.0
2020-01-01 00:00:57.000 AIA intensity 211.0 .. 211.0
2020-01-01 00:00:59.000 AIA intensity 94.0 .. 94.0
2020-01-01 00:01:00.000 AIA intensity 335.0 .. 335.0
Length = 41 rows
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time Instrument Physobs Wavelength
Angstrom
----------------------- ---------- ------------------ ----------------
2020-01-01 00:00:22.000 HMI intensity 6173.0 .. 6174.0
2020-01-01 00:00:22.000 HMI LOS_magnetic_field 6173.0 .. 6174.0
2020-01-01 00:00:22.000 HMI LOS_velocity 6173.0 .. 6174.0
To give an example of filtering post-search, let’s only return the rows in the table which are line-of-sight magnetograms from HMI or the 94Å passband from AIA.
You can also always do this filtering with the a.Physobs and a.Wavelength attrs in the search command.
First we split the results in to a table for AIA and a table for HMI:
>>> aia, hmi = results
We can use boolean indexing to match the value of the "Physobs" column:
>>> hmi_los = hmi[hmi["Physobs"] == "LOS_magnetic_field"]
>>> hmi_los.show("Start Time", "Instrument", "Wavelength", "Physobs")
<sunpy.net.vso.table_response.VSOQueryResponseTable object at ...>
Start Time Instrument Wavelength Physobs
Angstrom
----------------------- ---------- ---------------- ------------------
2020-01-01 00:00:22.000 HMI 6173.0 .. 6174.0 LOS_magnetic_field
To match the "Wavelength" column we need to account for the fact that VSO results return a wavelength range of [min, max] so we match the min:
>>> aia_94 = aia[aia["Wavelength"][:, 0] == 94 * u.AA]
>>> aia_94.show("Start Time", "Instrument", "Wavelength", "Physobs")
<sunpy.net.vso.table_response.VSOQueryResponseTable object at ...>
Start Time Instrument Wavelength Physobs
Angstrom
----------------------- ---------- ------------ ---------
2020-01-01 00:00:11.000 AIA 94.0 .. 94.0 intensity
2020-01-01 00:00:23.000 AIA 94.0 .. 94.0 intensity
2020-01-01 00:00:35.000 AIA 94.0 .. 94.0 intensity
2020-01-01 00:00:47.000 AIA 94.0 .. 94.0 intensity
2020-01-01 00:00:59.000 AIA 94.0 .. 94.0 intensity
Warning
While you can reduce the number of columns and rows in the results, the fetch() method that downloads data may need certain columns to be present to successfully download the files.
It is therefore highly recommended that if you are planning on downloading data you do not slice out columns, but instead use .show() to only display the ones you are interested in.
Downloading data#
Once you have located your files via a Fido.search, you can download them via Fido.fetch.
Here we’ll just download the first file in the result:
>>> downloaded_files = Fido.fetch(results[0, 0])
>>> downloaded_files
<parfive.results.Results object at ...>
['.../aia.lev1.335A_2020_01_01T00_00_00.64Z.image_lev1.fits']
This downloads the files to the location set in the sunpy config file.
It also returns a parfive.Results object downloaded_files, which contains local file paths to all the downloaded data.
You can also explicitly specify the path to which you want the data downloaded:
>>> downloaded_files = Fido.fetch(results, path='/ThisIs/MyPath/to/Data/{file}')
This downloads the query results into the directory /ThisIs/MyPath/to/Data, naming each downloaded file with the filename {file} obtained from the client.
You can also use other properties of the returned query to define the path where the data is saved.
For example, to save the data to a subdirectory named after the instrument, use:
>>> downloaded_files = Fido.fetch(results, path='./{instrument}/{file}')
You can see the list of options that can be specified in path for all the files to be downloaded with results.path_format_keys:
>>> sorted(results.path_format_keys())
['end_time', 'extent_length', 'extent_type', 'extent_width', 'fileid', 'instrument', 'physobs', 'provider', 'size', 'source', 'start_time', 'wavelength', 'wavetype']
Retrying Downloads#
If any files failed to download, the progress bar will show an incomplete number of files (i.e. 100/150) and the parfive.Results object will contain a list of the URLs that failed to transfer and the error associated with them.
This can be accessed with the .errors attribute or by printing the Results object:
>>> print(downloaded_files.errors)
The transfer can be retried by passing the parfive.Results object back to Fido.fetch:
>>> downloaded_files = Fido.fetch(downloaded_files)
doing this will append any newly downloaded file names to the list and replace the .errors list with any errors that occurred during the second attempt.
Maps#
In this section of the tutorial, you will learn about the Map object.
Map objects hold two-dimensional data along with the accompanying metadata.
They can be used with any two-dimensional data array with two spatial axes and FITS-compliant metadata.
As you will see in this tutorial, the primary advantage of using a Map object over a bare NumPy array is the ability to perform coordinate aware operations on the underlying array, such as rotating the Map to remove the roll angle of an instrument or cropping a Map to a specific field of view.
Additionally, because Map is capable of ingesting data from many different sources, it provides a common interface for any two-dimensional data product.
By the end of this tutorial, you will learn how to create a Map, access the underlying data and metadata, convert between physical coordinates and pixel coordinates of a Map, and the basics of visualizing a Map.
Additionally, you will learn how to combine multiple Maps into a MapSequence or a CompositeMap.
Note
In this section and in Timeseries, we will use the sample data included with sunpy.
These data are primarily useful for demonstration purposes or simple debugging.
These files have names like sunpy.data.sample.AIA_171_IMAGE and sunpy.data.sample.RHESSI_IMAGE and are automatically downloaded to your computer as you need them.
Once downloaded, these sample data files will be paths to their location on your computer.
Creating Maps#
To create a Map from a sample Atmospheric Imaging Assembly (AIA) image,
>>> import sunpy.map
>>> import sunpy.data.sample
>>> sunpy.data.sample.AIA_171_IMAGE
PosixPath('.../AIA20110607_063302_0171_lowres.fits')
>>> my_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
In many cases sunpy automatically detects the type of file as well as the the instrument associated with it.
In this case, we have a FITS file from the AIA instrument onboard the Solar Dynamics Observatory (SDO).
To make sure this has all worked correctly, we can take a quick look at my_map,
>>> my_map
<sunpy.map.sources.sdo.AIAMap object at ...>
SunPy Map
---------
Observatory: SDO
Instrument: AIA 3
Detector: AIA
Measurement: 171.0 Angstrom
Wavelength: 171.0 Angstrom
Observation Date: 2011-06-07 06:33:02
Exposure Time: 0.234256 s
Dimension: [1024. 1024.] pix
Coordinate System: helioprojective
Scale: [2.402792 2.402792] arcsec / pix
Reference Pixel: [511.5 511.5] pix
Reference Coord: [3.22309951 1.38578135] arcsec
array([[ -95.92475 , 7.076416 , -1.9656711, ..., -127.96519 ,
-127.96519 , -127.96519 ],
[ -96.97533 , -5.1167884, 0. , ..., -98.924576 ,
-104.04137 , -127.919716 ],
[ -93.99607 , 1.0189276, -4.0757103, ..., -5.094638 ,
-37.95505 , -127.87541 ],
...,
[-128.01454 , -128.01454 , -128.01454 , ..., -128.01454 ,
-128.01454 , -128.01454 ],
[-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
-127.899666 , -127.899666 ],
[-128.03072 , -128.03072 , -128.03072 , ..., -128.03072 ,
-128.03072 , -128.03072 ]], dtype=float32)
This should show a representation of the data as well as some of its associated attributes.
If you are in a Jupyter Notebook, this will show a rich HTML view of the table along with several quick-look plots.
Otherwise, you can use the quicklook() method to see these quick-look plots.
>>> my_map.quicklook()
<sunpy.map.sources.sdo.AIAMap object at 0x7f6f1b742560>
|
Image colormap uses histogram equalization
Click on the image to toggle between units 
|
||||||||||||||||||||||||
|
Inspecting Map Metadata#
The metadata for a Map is exposed via attributes on the Map.
These attributes can be accessed by typing my_map.<attribute-name>.
For example, to access the date of the observation,
>>> my_map.date
<Time object: scale='utc' format='isot' value=2011-06-07T06:33:02.770>
Notice that this is an Time object which we discussed in the previous Times section of the tutorial.
Similarly, we can access the exposure time of the image,
>>> my_map.exposure_time
<Quantity 0.234256 s>
Notice that this returns an Quantity object which we discussed in the previous Units section of the tutorial.
The full list of attributes can be found in the reference documentation for GenericMap.
These metadata attributes are all derived from the underlying FITS metadata, but are represented as rich Python objects, rather than simple strings or numbers.
Map Data#
The data in a Map is stored as a numpy.ndarray object and is accessible through the data attribute:
>>> my_map.data
array([[ -95.92475 , 7.076416 , -1.9656711, ..., -127.96519 ,
-127.96519 , -127.96519 ],
[ -96.97533 , -5.1167884, 0. , ..., -98.924576 ,
-104.04137 , -127.919716 ],
[ -93.99607 , 1.0189276, -4.0757103, ..., -5.094638 ,
-37.95505 , -127.87541 ],
...,
[-128.01454 , -128.01454 , -128.01454 , ..., -128.01454 ,
-128.01454 , -128.01454 ],
[-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
-127.899666 , -127.899666 ],
[-128.03072 , -128.03072 , -128.03072 , ..., -128.03072 ,
-128.03072 , -128.03072 ]], dtype=float32)
This array can then be indexed like any other NumPy array. For example, to get the 0th element in the array:
>>> my_map.data[0, 0]
-95.92475
The first index corresponds to the y direction and the second to the x direction in the two-dimensional pixel coordinate system. For more information about indexing, please refer to the numpy documentation.
Data attributes like dimensionality and type are also accessible as attributes on my_map:
>>> my_map.dimensions
PixelPair(x=<Quantity 1024. pix>, y=<Quantity 1024. pix>)
>>> my_map.dtype
dtype('float32')
Additionally, there are several methods that provide basic summary statistics of the data:
>>> my_map.min()
-129.78036
>>> my_map.max()
192130.17
>>> my_map.mean()
427.02252
Coordinates, and the World Coordinate System#
In Coordinates, you learned how to define coordinates with SkyCoord using different solar coordinate frames.
The coordinate frame of a Map is provided as an attribute,
>>> my_map.coordinate_frame
<Helioprojective Frame (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>)>
This tells us that the coordinate system of the image is Helioprojective (HPC) and that it is defined by an observer at a particular location. This observer coordinate is also provided as an attribute,
>>> my_map.observer_coordinate
<SkyCoord (HeliographicStonyhurst: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>
This tells us the location of the spacecraft, in this case SDO, when it recorded this particular observation, as derived from the FITS metadata.
Map has several additional coordinate-related attributes that provide the coordinates of the center and corners of the Map,
>>> my_map.center
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(3.22309951, 1.38578135)>
>>> my_map.bottom_left_coord
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(-1228.76466158, -1224.62447509)>
>>> my_map.top_right_coord
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(1235.21095899, 1227.39598836)>
But what if we wanted to know what pixel these physical coordinates correspond to?
Each Map has an associated World Coordinate System, or WCS, which is derived from the underlying metadata and expressed as an astropy.wcs.WCS object.
The WCS is accessible as an attribute:
>>> my_map.wcs
WCS Keywords
Number of WCS axes: 2
CTYPE : 'HPLN-TAN' 'HPLT-TAN'
CRVAL : 0.00089530541880571 0.00038493926472939
CRPIX : 512.5 512.5
PC1_1 PC1_2 : 0.99999706448085 0.0024230207763071
PC2_1 PC2_2 : -0.0024230207763071 0.99999706448085
CDELT : 0.00066744222222222 0.00066744222222222
NAXIS : 1024 1024
WCS is a fairly complex topic, but all we need to know for now is that the WCS provides the transformation between the pixel coordinates of the image and physical or “world” coordinates.
In particular, we will only focus on two methods: world_to_pixel and pixel_to_world.
First, let’s find the pixel location corresponding to the center of the Map,
>>> center_pixel = my_map.wcs.world_to_pixel(my_map.center)
>>> center_pixel
(array(511.5), array(511.5))
Notice that these coordinates are not necessarily integers. The corresponding pixel-to-world transformation should then give us back our center coordinate from above,
>>> my_map.wcs.pixel_to_world(center_pixel[0], center_pixel[1])
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(3.22309951, 1.38578135)>
As another example, if we transform the center of the lower-left pixel to a world coordinate, it should correspond to bottom left coordinate from above,
>>> my_map.wcs.pixel_to_world(0, 0)
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>): (Tx, Ty) in arcsec
(-1228.76466158, -1224.62447509)>
These two methods are extremely useful when trying to understand which pixels correspond to which physical coordinates or when trying to locate the same physical location in images taken by separate spacecraft.
Visualizing Maps#
In the Creating Maps section, you learned how to generate a quicklook summary of a Map.
However, the Map object also has a plot() method that allows for more fine-grained control over how the Map is visualized and is especially useful for generating publication-quality plots.
In this section of the tutorial, you will learn how to build up an increasingly detailed visualization of a Map, including adjusting the colormap and normalization and and overlaying coordinates and contours.
Basic Plotting#
First, let’s create a basic plot of our Map, including a colorbar,
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax)
plt.colorbar()
plt.show()
(Source code, png, hires.png, pdf)
Note
We imported matplotlib.pyplot in order to create the figure and the axis we plotted on our map onto.
Under the hood, sunpy uses matplotlib to visualize the image meaning that plots built with sunpy can be further customized using matplotlib.
However, for the purposes of this tutorial, you do not need to be familiar with Matplotlib.
For a series of detailed examples showing how to customize your Map plots, see the Plotting section of the Example Gallery as well as the documentation for astropy.visualization.wcsaxes.
Note that the title and colormap have been set by sunpy based on the observing instrument and wavelength. Furthermore, the tick and axes labels have been automatically set based on the coordinate system of the Map.
Looking at the plot above, you likely notice that the resulting image is a bit dim.
To fix this, we can use the clip_interval keyword to automatically adjust the colorbar limits to clip out the dimmest 1% and the brightest 0.5% of pixels.
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1, 99.5)*u.percent)
plt.colorbar()
plt.show()
(Source code, png, hires.png, pdf)
Changing the Colormap and Normalization#
Historically, particular colormaps are assigned to images based on what instrument they are from and what wavelength is being observed. By default, sunpy will select the colormap based on the available metadata. This default colormap is available as an attribute,
>>> my_map.cmap.name
'sdoaia171'
When visualizing a Map, you can change the colormap using the cmap keyword argument.
For example, you can use the ‘inferno’ colormap from matplotlib:
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, cmap='inferno', clip_interval=(1,99.5)*u.percent)
plt.colorbar()
plt.show()
(Source code, png, hires.png, pdf)
Note
sunpy provides specific colormaps for many different instruments.
For a list of all colormaps provided by sunpy, see the documentation for sunpy.visualization.colormaps.
The normalization, or the mapping between the data values and the colors in our colormap, is also determined based on the underlying metadata.
Notice that in the plots we’ve made so far, the ticks on our colorbar are not linearly spaced.
Just like in the case of the colormap, we can use a normalization other than the default by passing a keyword argument to the plot() method.
For example, we can use a logarithmic normalization instead:
import matplotlib.colors
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(norm=matplotlib.colors.LogNorm())
plt.colorbar()
plt.show()
(Source code, png, hires.png, pdf)
Note
You can also view or make changes to the default settings through the sunpy.map.GenericMap.plot_settings dictionary.
See Editing the colormap and normalization of a Map for an example of of how to change the default plot settings.
Overlaying Contours and Coordinates#
When plotting images, we often want to highlight certain features or overlay certain data points. There are several methods attached to Map that make this task easy. For example, we can draw contours around the brightest 0.5% percent of pixels in the image:
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1,99.5)*u.percent)
my_map.draw_contours([2, 5, 10, 50, 90] * u.percent, axes=ax)
plt.show()
(Source code, png, hires.png, pdf)
Additionally, the solar limb, as determined by the location of the observing instrument at the time of the observation, can be easily overlaid on an image:
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1,99.5)*u.percent)
my_map.draw_limb(axes=ax, color='C0')
plt.show()
(Source code, png, hires.png, pdf)
We can also overlay a box denoting a particular a region of interest as expressed in world coordinates using the the coordinate frame of our image:
roi_bottom_left = SkyCoord(Tx=-300*u.arcsec, Ty=-100*u.arcsec, frame=my_map.coordinate_frame)
roi_top_right = SkyCoord(Tx=200*u.arcsec, Ty=400*u.arcsec, frame=my_map.coordinate_frame)
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1,99.5)*u.percent)
my_map.draw_quadrangle(roi_bottom_left, top_right=roi_top_right, axes=ax, color='C0')
plt.show()
(Source code, png, hires.png, pdf)
Because our visualization knows about the coordinate system of our Map, it can transform any coordinate to the coordinate frame of our Map and then use the underlying WCS that we discussed in the Coordinates, and the World Coordinate System section to translate this to a pixel position.
This makes it simple to plot any coordinate on top of our Map using the plot_coord() method.
The following example shows how to plot some points on our Map, including the center coordinate of our Map:
coords = SkyCoord(Tx=[100,1000] * u.arcsec, Ty=[100,1000] * u.arcsec, frame=my_map.coordinate_frame)
fig = plt.figure()
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1,99.5)*u.percent)
ax.plot_coord(coords, 'o')
ax.plot_coord(my_map.center, 'X')
plt.show()
(Source code, png, hires.png, pdf)
Note
Map visualizations can be heavily customized using both matplotlib and astropy.visualization.wcsaxes.
See the Plotting section of the Example Gallery for more detailed examples of how to customize Map visualizations.
Cropping Maps and Combining Pixels#
In analyzing images of the Sun, we often want to choose a smaller portion of the full disk to look at more closely. Let’s use the region of interest we defined above to crop out that portion of our image:
my_submap = my_map.submap(roi_bottom_left, top_right=roi_top_right)
fig = plt.figure()
ax = fig.add_subplot(projection=my_submap)
my_submap.plot(axes=ax)
plt.show()
(Source code, png, hires.png, pdf)
Additionally, we also may want to combine multiple pixels into a single pixel (a “superpixel”) to, for example, increase our signal-to-noise ratio.
We can accomplish this with the superpixel method by specifying how many pixels, in each dimension, we want our new superpixels to contain.
For example, we can combine 4 pixels in each dimension such that our new superpixels contain 16 original pixels:
my_super_submap = my_submap.superpixel((5,5)*u.pixel)
fig = plt.figure()
ax = fig.add_subplot(projection=my_super_submap)
my_super_submap.plot(axes=ax)
plt.show()
(Source code, png, hires.png, pdf)
Note
Map provides additional methods for manipulating the underlying image data.
See the reference documentation for GenericMap for a complete list of available methods as well as the Map section of the Example Gallery for more detailed examples.
Map Sequences#
While GenericMap can only contain a two-dimensional array and metadata corresponding to a single observation, a MapSequence is comprised of an ordered list of maps.
By default, the Maps are ordered by their observation date, from earliest to latest date.
A MapSequence can be created by supplying multiple existing maps:
>>> another_map = sunpy.map.Map(sunpy.data.sample.EIT_195_IMAGE)
>>> map_seq = sunpy.map.Map([my_map, another_map], sequence=True)
A map sequence can be indexed in the same manner as a list. For example, the following returns the same information as in Creating Maps:
>>> map_seq.maps[0]
<sunpy.map.sources.sdo.AIAMap object at ...>
SunPy Map
---------
Observatory: SDO
Instrument: AIA 3
Detector: AIA
Measurement: 171.0 Angstrom
Wavelength: 171.0 Angstrom
Observation Date: 2011-06-07 06:33:02
Exposure Time: 0.234256 s
Dimension: [1024. 1024.] pix
Coordinate System: helioprojective
Scale: [2.402792 2.402792] arcsec / pix
Reference Pixel: [511.5 511.5] pix
Reference Coord: [3.22309951 1.38578135] arcsec
array([[ -95.92475 , 7.076416 , -1.9656711, ..., -127.96519 ,
-127.96519 , -127.96519 ],
[ -96.97533 , -5.1167884, 0. , ..., -98.924576 ,
-104.04137 , -127.919716 ],
[ -93.99607 , 1.0189276, -4.0757103, ..., -5.094638 ,
-37.95505 , -127.87541 ],
...,
[-128.01454 , -128.01454 , -128.01454 , ..., -128.01454 ,
-128.01454 , -128.01454 ],
[-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
-127.899666 , -127.899666 ],
[-128.03072 , -128.03072 , -128.03072 , ..., -128.03072 ,
-128.03072 , -128.03072 ]], dtype=float32)
MapSequences can hold maps that have different shapes.
To test if all the maps in a MapSequence have the same shape:
>>> map_seq.all_maps_same_shape()
True
It is often useful to return the image data in a MapSequence as a single three dimensional NumPy ndarray:
>>> map_seq_array = map_seq.as_array()
Since all of the maps in our sequence of the same shape, the first two dimensions of our combined array will be the same as the component maps while the last dimension will correspond to the number of maps in the map sequence. We can confirm this by looking at the shape of the above array.
>>> map_seq_array.shape
(1024, 1024, 2)
Warning
MapSequence does not automatically perform any coalignment between the maps comprising a sequence.
For information on coaligning images and compensating for solar rotation, see this section of the Example Gallery as well as the sunkit_image.coalignment module.
Timeseries#
In this section of the tutorial, you will learn about the TimeSeries object.
TimeSeries objects hold time-dependent data with their accompanying metadata.
They can be used with multiple one-dimensional arrays which are all associated with a common time axis.
Much like the Map object, the TimeSeries object can handle generic data, but also provides instrument specific data loading and plotting capabilities.
Importantly, TimeSeries allows you to select specific time ranges or combine multiple TimeSeries in a metadata-aware way.
By the end of this tutorial, you will learn how to create a TimeSeries, extract the data and metadata, and easily visualize the TimeSeries. Additionally, you will learn how to truncate a TimeSeries to a specific window in time as well as combine multiple TimeSeries objects.
Note
In this section and in Maps, we will use the sample data included with sunpy.
These data are primarily useful for demonstration purposes or simple debugging.
These files have names like sunpy.data.sample.EVE_TIMESERIES and sunpy.data.sample.GOES_XRS_TIMESERIES and are automatically downloaded to your computer as you need them.
Once downloaded, these sample data files will be paths to their location on your computer.
Creating a TimeSeries#
To create a TimeSeries from some sample GOES XRS data:
>>> import sunpy.timeseries
>>> import sunpy.data.sample
>>> sunpy.data.sample.GOES_XRS_TIMESERIES
PosixPath('.../go1520110607.fits')
>>> my_timeseries = sunpy.timeseries.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES)
In many cases, sunpy will automatically detect the type of the file as well as the instrument associated with it. In this case, we have a FITS file containing an X-ray light curve as observed by the the XRS instrument on the GOES satellite.
Note
Time series data are stored in a variety of file types (e.g. FITS, csv, CDF), and so it is not always possible to detect the source. sunpy ships with a number of known instrumental sources, and can also load CDF files that conform to the Space Physics Guidelines for CDF.
To make sure this has all worked correctly, we can take a quick look at my_timeseries,
>>> my_timeseries
<sunpy.timeseries.sources.goes.XRSTimeSeries object at ...>
SunPy TimeSeries
----------------
Observatory: GOES-15
Instrument: <a href=https://www.swpc.noaa.gov/products/goes-x-ray-flux target="_blank">X-ray Detector</a>
Channel(s): xrsa<br>xrsb
Start Date: 2011-06-07 00:00:00
End Date: 2011-06-07 23:59:58
Center Date: 2011-06-07 11:59:58
Resolution: 2.048 s
Samples per Channel: 42177
Data Range(s): xrsa 3.64E-06<br>xrsb 2.54E-05
Units: W / m2
xrsa xrsb
2011-06-06 23:59:59.961999893 1.000000e-09 1.887100e-07
2011-06-07 00:00:02.008999944 1.000000e-09 1.834600e-07
2011-06-07 00:00:04.058999896 1.000000e-09 1.860900e-07
2011-06-07 00:00:06.104999900 1.000000e-09 1.808400e-07
2011-06-07 00:00:08.151999950 1.000000e-09 1.860900e-07
... ... ...
2011-06-07 23:59:49.441999912 1.000000e-09 1.624800e-07
2011-06-07 23:59:51.488999844 1.000000e-09 1.624800e-07
2011-06-07 23:59:53.538999915 1.000000e-09 1.598500e-07
2011-06-07 23:59:55.584999919 1.000000e-09 1.624800e-07
2011-06-07 23:59:57.631999850 1.000000e-09 1.598500e-07
[42177 rows x 2 columns]
This should show a table of information taken from the metadata and a preview of your data.
If you are in a Jupyter Notebook, this will show a rich HTML version that includes plots of the data.
Otherwise, you can use the quicklook() method to see this quick-look plot,
>>> my_timeseries.quicklook()
<sunpy.timeseries.sources.goes.XRSTimeSeries object at 0x7f6f165880d0>
|
|
||||||||||||||||||||
|
TimeSeries Data#
We can easily check which columns are contained in the TimeSeries,
>>> my_timeseries.columns
['xrsa', 'xrsb']
“xrsa” denotes the short wavelength channel of the XRS data which contains emission between 0.5 and 4 Angstrom.
To pull out the just the data corresponding to this column, we can use the quantity() method:
>>> my_timeseries.quantity('xrsa')
<Quantity [1.e-09, 1.e-09, 1.e-09, ..., 1.e-09, 1.e-09, 1.e-09] W / m2>
Notice that this is a Quantity object which we discussed in Units.
Additionally, the timestamp associated with each point and the time range of the observation are accessible as attributes,
>>> my_timeseries.time
<Time object: scale='utc' format='iso' value=['2011-06-06 23:59:59.962' '2011-06-07 00:00:02.009'
'2011-06-07 00:00:04.059' ... '2011-06-07 23:59:53.539'
'2011-06-07 23:59:55.585' '2011-06-07 23:59:57.632']>
>>> my_timeseries.time_range
<sunpy.time.timerange.TimeRange object at ...>
Start: 2011-06-06 23:59:59
End: 2011-06-07 23:59:57
Center:2011-06-07 11:59:58
Duration:0.9999730324069096 days or
23.99935277776583 hours or
1439.9611666659498 minutes or
86397.66999995698 seconds
Notice that these return a astropy.time.Time and sunpy.time.TimeRange, both of which we covered in Times.
Inspecting TimeSeries Metadata#
A TimeSeries object also includes metadata associated with that observation. Some of this metadata is exposed via attributes on the TimeSeries. For example, to find out which observatory observed this data,
>>> my_timeseries.observatory
'GOES-15'
Additionally, to find out which instrument this timeseries data came from,
>>> my_timeseries.source
'xrs'
All of the metadata can also be accessed using the meta attribute,
>>> my_timeseries.meta
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2011-06-06T23:59:59.961999 | xrsa | simple: True |
| to | xrsb | bitpix: 8 |
|2011-06-07T23:59:57.631999 | | naxis: 0 |
| | | extend: True |
| | | date: 26/06/2012 |
| | | numext: 3 |
| | | telescop: GOES 15 |
| | | instrume: X-ray Detector |
| | | object: Sun |
| | | origin: SDAC/GSFC |
| | | ... |
|-------------------------------------------------------------------------------------------------|
Warning
A word of caution: many data sources provide little to no meta data so this variable might be empty. See A Detailed Look at the TimeSeries Metadata for a more detailed explanation of how metadata on TimeSeries objects is handled.
Visualizing TimeSeries#
The sunpy TimeSeries object has its own built-in plot methods so that it is easy to quickly view your time series. To create a plot,
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
my_timeseries.plot(axes=ax)
plt.show()
(Source code, png, hires.png, pdf)
Note
For additional examples of building more complex visualization with TimeSeries, see the examples in Time Series.
Adding Columns#
TimeSeries provides the add_column method which will either add a new column or update a current column if the colname is already present.
This can take numpy array or preferably an Astropy Quantity value.
For example:
>>> values = my_timeseries.quantity('xrsa') * 2
>>> my_timeseries = my_timeseries.add_column('xrsa*2', values)
>>> my_timeseries.columns
['xrsa', 'xrsb', 'xrsa*2']
Adding a column is not done in place, but instead returns a new TimeSeries with the new column added.
Note that the values will be converted into the column units if an Astropy Quantity is given.
Caution should be taken when adding a new column because this column won’t have any associated MetaData entry.
Truncating a TimeSeries#
It is often useful to truncate an existing TimeSeries object to retain a specific time range.
This is easily achieved by using the truncate method.
For example, to trim our GOES data into a period of interest use:
>>> from sunpy.time import TimeRange
>>> tr = TimeRange('2012-06-01 05:00', '2012-06-01 06:30')
>>> my_timeseries_trunc = my_timeseries.truncate(tr)
This takes a number of different arguments, such as the start and end dates (as datetime or string objects) or a TimeRange as used above.
Note that the truncated TimeSeries will have a truncated TimeSeriesMetaData object, which may include dropping metadata entries for data totally cut out from the TimeSeries.
If you want to truncate using slice-like values you can, for example taking every 2nd value from 0 to 10000 can be done using:
>>> my_timeseries_trunc = my_timeseries.truncate(0, 100000, 2)
Concatenating TimeSeries#
It’s common to want to combine a number of TimeSeries together into a single TimeSeries.
In the simplest scenario this is to combine data from a single source over several time ranges, for example if you wanted to combine the daily GOES data to get a week or more of constant data in one TimeSeries.
This can be performed using the TimeSeries factory with the concatenate=True keyword argument:
>>> concatenated_timeseries = sunpy.timeseries.TimeSeries(filepath1, filepath2, source='XRS', concatenate=True)
Note, you can list any number of files, or a folder or use a glob to select the input files to be concatenated.
It is possible to concatenate two TimeSeries after creating them using the concatenate method.
For example:
>>> concatenated_timeseries = goes_timeseries_1.concatenate(goes_timeseries_2)
This will result in a TimeSeries identical to if you had created them in one step.
A limitation of the TimeSeries class is that it is not always possible to determine the source observatory or instrument of a given file.
Thus some sources need to be explicitly stated (using the keyword argument) and so, it is not possible to concatenate files from multiple sources.
To do this, you can still use the concatenate method, which will create a new TimeSeries with all the rows and columns of the source and concatenated TimeSeries in one:
>>> eve_ts = sunpy.timeseries.TimeSeries(sunpy.data.sample.EVE_TIMESERIES, source='eve')
>>> concatenated_timeseries = my_timeseries.concatenate(eve_ts)
Note that the more complex TimeSeriesMetaData object now has 2 entries and shows details on both:
>>> concatenated_timeseries.meta
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2011-06-06T23:59:59.961999 | xrsa | simple: True |
| to | xrsb | bitpix: 8 |
|2011-06-07T23:59:57.631999 | | naxis: 0 |
| | | extend: True |
| | | date: 26/06/2012 |
| | | numext: 3 |
| | | telescop: GOES 15 |
| | | instrume: X-ray Detector |
| | | object: Sun |
| | | origin: SDAC/GSFC |
| | | ... |
|-------------------------------------------------------------------------------------------------|
|2011-06-07T00:00:00.000000 | XRS-B proxy | data_list: 20110607_EVE_L0CS_DIODES_1m.txt |
| to | XRS-A proxy | created: Tue Jun 7 23:59:10 2011 UTC |
|2011-06-07T23:59:00.000000 | SEM proxy | origin: SDO/EVE Science Processing and Operations |
| | 0.1-7ESPquad | units: W/m^2 for irradiance, dark is counts/(0.25s|
| | 17.1ESP | source: SDO-EVE ESP and MEGS-P instruments, http:/|
| | 25.7ESP | product: Level 0CS, 1-minute averaged SDO-EVE Sola|
| | 30.4ESP | version: 2.1, code updated 2011-May-12 |
| | 36.6ESP | missing data: -1.00e+00 |
| | darkESP | hhmm: hour and minute in UT |
| | 121.6MEGS-P | xrs-b proxy: a model of the expected XRS-B 0.1-0.8|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
Example Gallery#
The gallery contains examples of how to use sunpy. Each example is a short and self contained how-to guide for performing a specific task.
Sample data#
Some of these examples require the SunPy sample data, which are downloaded as needed via the module sunpy.data.sample.
If you want to download all of the sample data files in advance, call sunpy.data.sample.download_all().
Acquiring Data#
Examples of downloading solar data located on remote servers
Map#
Examples using Map with solar data
Segmenting a Map based on transformation of coordinates
Combining, Co-aligning, and Reprojecting Images#
Examples of combining, aligning, and reprojecting sunpy maps
Reprojecting to a Map Projection with a Custom Origin
Time Series#
Examples using TimeSeries
Creating a TimeSeries from GOES-XRS near real time data
Retrieving and analyzing GOES X-Ray Sensor (XRS) data
Smoothing of timeSeries data using convolution filters
Coordinates, times, and units#
Examples of working with coordinate information, times, and scientific units
Converting between Helioprojective and AltAz Coordinate
Create a Helioprojective Map from observations in the RA-DEC coordinate system
Identifying stars in a STEREO/SECCHI COR2 coronagraph image
Obtaining a spacecraft trajectory from JPL Horizons
Setting the correct position for SOHO in a LASCO C3 Map
Plotting#
Examples of visualizing supported data types
Overplotting SRS active region locations on a magnetograms
Differential Rotation of the Sun#
Examples of accounting for differential rotation (i.e., the latitude-dependent rotation rate of the Sun) in the coordinates framework
Saving and Loading Data#
Examples of saving and loading data
Computer Vision Techniques#
Examples of using computer vision techniques to analyze solar data
Showcase#
Examples that use an advanced combination of capabilities in sunpy
Understanding Observer Orientation Relative to a Simulated Volume with astropy.coordinates
Acquiring Data#
Examples of downloading solar data located on remote servers
Note
Go to the end to download the full example code.
Getting data from CDAWeb#
How to download data from the Coordinated Data Analysis Web (CDAWeb).
CDAWeb stores data from from current and past space physics missions, and is full of heliospheric insitu datasets.
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.timeseries import TimeSeries
sunpy.net.Fido is the primary interface to search for and download data and
will automatically search CDAWeb when the cdaweb.Dataset attribute is provided to
the search. To lookup the different dataset IDs available, you can use the
form at https://cdaweb.gsfc.nasa.gov/index.html/
trange = a.Time('2021/07/01', '2021/07/08')
dataset = a.cdaweb.Dataset('SOLO_L2_MAG-RTN-NORMAL-1-MINUTE')
result = Fido.search(trange, dataset)
Let’s inspect the results. We can see that there’s seven files, one for each day within the query.
print(result)
Results from 1 Provider:
7 Results from the CDAWEBClient:
Source: https://cdaweb.gsfc.nasa.gov/index.html
Dataset Start time End time
------------------------------- ----------------------- -----------------------
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-01 00:00:29.000 2021-07-01 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-02 00:00:29.000 2021-07-02 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-03 00:00:29.000 2021-07-03 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-04 00:00:29.000 2021-07-04 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-05 00:00:29.000 2021-07-05 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-06 00:00:29.000 2021-07-06 23:59:30.000
SOLO_L2_MAG-RTN-NORMAL-1-MINUTE 2021-07-07 00:00:29.000 2021-07-07 23:59:30.000
Let’s download the first two files
downloaded_files = Fido.fetch(result[0, 0:2])
print(downloaded_files)
['/home/docs/sunpy/data/solo_l2_mag-rtn-normal-1-minute_20210701_v01.cdf', '/home/docs/sunpy/data/solo_l2_mag-rtn-normal-1-minute_20210702_v01.cdf']
Finally we can load and take a look at the data using
TimeSeries This requires an installation of the cdflib
Python library to read the CDF file.
solo_mag = TimeSeries(downloaded_files, concatenate=True)
print(solo_mag.columns)
solo_mag.peek(columns=['B_RTN_0', 'B_RTN_1', 'B_RTN_2'])
['B_RTN_0', 'B_RTN_1', 'B_RTN_2', 'QUALITY_BITMASK', 'QUALITY_FLAG', 'VECTOR_RANGE', 'VECTOR_TIME_RESOLUTION']
Total running time of the script: (0 minutes 1.068 seconds)
Note
Go to the end to download the full example code.
Querying Metadata clients#
This example shows how to search and retrieve metadata using Fido.
Fido supports searching metadata from services like HEKClient,
HECClient, and JSOCClient.
In this example we will make one search for records from the JSOC and the HEK, and then download the corresponding file from the JSOC.
import os
from sunpy.net import Fido
from sunpy.net import attrs as a
We will query the HEK for all flares with a peak flux greater than 1000. We will also search JSOC for a ‘hmi.m_45s’ series.
timerange = a.Time('2010/8/1 03:40', '2010/8/1 3:40:10')
# Exporting data from the JSOC requires registering your email first.
# Please replace this with your email address once you have registered
# like so: jsoc_email = "your_email@example.com"
# See `this page <http://jsoc.stanford.edu/ajax/register_email.html>`__ for more details.
jsoc_email = os.environ["JSOC_EMAIL"]
results = Fido.search(timerange,
a.hek.FL & (a.hek.FL.PeakFlux > 1000) |
a.jsoc.Series('hmi.m_45s') & a.jsoc.Notify(jsoc_email))
results is a UnifiedResponse object that
contains records returned from querying various clients by “Fido.search”.
print(results)
Results from 2 Providers:
2 Results from the HEKClient:
gs_thumburl ...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100731_205448_25495028/www/EDS_FlareDetective-TriggerModule_20100801T033001-20100801T035225_AIA_171_S21W87_ssw_cutout_20100801_033013_AIA_171_S21W87_20100801_033012_context_0180.gif ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100801_234037_25860951/www/EDS_FlareDetective-TriggerModule_20100801T033008-20100801T035232_AIA_193_S21W87_ssw_cutout_20100801_033020_AIA_193_S21W87_20100801_033019_context_0180.gif ...
1 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI SDO/HMI HMI_FRONT2 6173.0 2099
Now we will download the searched records. Since the HEK
client don’t provide files, Fido.fetch will
ignore it and only download files from JSOC.
files = Fido.fetch(results)
print(files)
INFO: 1 URLs found for download. Full request totaling 14MB [sunpy.net.jsoc.jsoc]
['/home/docs/sunpy/data/hmi.m_45s.20100801_034030_TAI.2.magnetogram.fits']
Now we will extract individual responses from Fido results. We can index these results using the client’s name (which is case-insensitive).
hek_results, jsoc_results = results['hek'], results['jsoc']
The results from a metadata search could have up to 100 columns.
As a result, you can use use show() to specify the column names you want to display.
hek_table = hek_results.show('event_peaktime', 'obs_instrument', 'fl_peakflux')
print(hek_table)
event_peaktime obs_instrument fl_peakflux
----------------------- -------------- -----------
2010-08-01 03:40:37.000 AIA 1027.64
2010-08-01 03:40:44.000 AIA 1441.78
The results from JSOC have a default set of columns to show and are
['T_REC', 'TELESCOP', 'INSTRUME', 'WAVELNTH', 'CAR_ROT'].
To display all of the columns, we can use show() without passings any arguments.
print(jsoc_results)
jsoc_table = jsoc_results.show()
print(jsoc_table)
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI SDO/HMI HMI_FRONT2 6173.0 2099
DATE DATE__OBS ... CALVER64
-------------------- ----------------------- ... --------
2012-09-05T07:57:40Z 2010-08-01T03:39:41.00Z ... 16
Total running time of the script: (0 minutes 12.164 seconds)
Note
Go to the end to download the full example code.
Querying and loading SHARP data#
In this example we will demonstrate how to acquire Spaceweather HMI Active Region Patch (SHARP) data and load it into a sunpy.map.Map.
import os
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.map
from sunpy.net import Fido
from sunpy.net import attrs as a
To search for SHARP data, we will need to query the JSOC.
We will use Fido
and make use of the search attributes in sunpy.net.jsoc that allow us to query the JSOC.
Exporting data from the JSOC requires registering your email first. Please replace this with your email address once you have registered like so: jsoc_email = “your_email@example.com” See this page for more details.
jsoc_email = os.environ["JSOC_EMAIL"]
result = Fido.search(a.Time("2011-03-09 23:20:00", "2011-03-09 23:30:00"),
a.Sample(1*u.hour),
a.jsoc.Series("hmi.sharp_cea_720s"),
a.jsoc.PrimeKey("HARPNUM", 401),
a.jsoc.Notify(jsoc_email),
a.jsoc.Segment("Bp"))
print(result)
Results from 1 Provider:
1 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- --------- -------- -------
2011.03.09_23:24:00_TAI SDO/HMI HMI_SIDE1 6173.0 2107
Next, we can download the file.
file = Fido.fetch(result)
INFO: 1 URLs found for download. Full request totaling 1MB [sunpy.net.jsoc.jsoc]
Now that we have the file, we can construct a Map and plot it.
sharp_map = sunpy.map.Map(file)
fig = plt.figure()
ax = fig.add_subplot(projection=sharp_map)
sharp_map.plot(axes=ax, vmin=-1500, vmax=1500)
plt.show()
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:736: SunpyMetadataWarning: Could not parse unit string "Mx/cm^2" as a valid FITS unit.
See https://docs.sunpy.org/en/stable/how_to/fix_map_metadata.html for how to fix metadata before loading it with sunpy.map.Map.
See https://fits.gsfc.nasa.gov/fits_standard.html for the FITS unit standards.
warn_metadata(f'Could not parse unit string "{unit_str}" as a valid FITS unit.\n'
Total running time of the script: (0 minutes 12.944 seconds)
Note
Go to the end to download the full example code.
Querying the GOES flare event list#
How to retrieve the GOES flare event list, sort and save the results.
from sunpy.net import Fido
from sunpy.net import attrs as a
We use Fido to to query the HEK catalogue. We define our event type
as a flare (“FL”). We also set up the start and end times over which
we will search for flare events. We want the list of events
that were detected by the GOES X-ray Sensor (XRS) instrument between
tstart and tend and GOES class > M1.0.
event_type = "FL"
tstart = "2013/10/28"
tend = "2013/10/29"
result = Fido.search(a.Time(tstart, tend),
a.hek.EventType(event_type),
a.hek.FL.GOESCls > "M1.0",
a.hek.OBS.Observatory == "GOES")
The result is returned as a UnifiedResponse,
from which we can see a table from one provider is found and returned.
# Here we only show two columns due there being over 100 columns returned normally.
print(result.show("hpc_bbox", "refs"))
# It"s also possible to access the HEK results from the
# `~sunpy.net.fido_factory.UnifiedResponse` by name.
hek_results = result["hek"]
Results from 1 Provider:
7 Results from the HEKClient:
hpc_bbox ...
---------------------------------------------------------------------------------------------------- ...
POLYGON((16.92036 -63.6504,16.92036 -63.6504,16.92036 -63.6504,16.92036 -63.6504,16.92036 -63.6504)) ...
POLYGON((16.9209 -63.4614,16.9209 -63.4614,16.9209 -63.4614,16.9209 -63.4614,16.9209 -63.4614)) ...
POLYGON((16.92216 -62.9952,16.92216 -62.9952,16.92216 -62.9952,16.92216 -62.9952,16.92216 -62.9952)) ...
POLYGON((0.000084 -961.62,-0.000084 -961.62,0 962.358,0 962.358,0.000084 -961.62)) ...
POLYGON((0.000084 -961.632,-0.000084 -961.632,0 962.37,0 962.37,0.000084 -961.632)) ...
POLYGON((0.000084 -961.638,-0.000084 -961.638,0 962.376,0 962.376,0.000084 -961.638)) ...
POLYGON((0.000084 -961.734,-0.000084 -961.734,0 962.466,0 962.466,0.000084 -961.734)) ...
We can also print the key names that correspond to the HEK parameters returned by the query.
# We only print every 10th key to avoid the output being too long.
print(hek_results.colnames[::10])
['gs_thumburl', 'ar_polarity', 'skel_chaincode', 'obs_dataprepurl', 'eventtype', 'frm_identifier', 'event_maskurl', 'event_probability', 'intenstotal', 'ar_noaanum', 'hgs_y', 'hpc_radius', 'hgc_boundcc', 'bound_ccnsteps', 'obs_lastprocessingdate']
The results returned contain a lot of information and we may only want to keep some main results such as start time, end time, peak time, GOES-class, and active region number. This can be done as so:
filtered_results = hek_results["event_starttime", "event_peaktime",
"event_endtime", "fl_goescls", "ar_noaanum"]
hek_result is already sorted by date, if one wants to sort
by magnitude, one can do the following:
# Sorting is done by using the flare class from "fl_goescls"
# By converting the flare class to a number using ord()
# and adding the flare strength, we can sort by value
by_magnitude = sorted(filtered_results, key=lambda x: ord(x['fl_goescls'][0]) + float(x['fl_goescls'][1:]), reverse=True)
for flare in by_magnitude:
print(f"Class {flare['fl_goescls']} occurred on {flare['event_starttime']}")
Class X1.0 occurred on 2013-10-28 01:41:00.000
Class M5.1 occurred on 2013-10-28 04:32:00.000
Class M4.4 occurred on 2013-10-28 15:07:00.000
Class M2.8 occurred on 2013-10-28 14:00:00.000
Class M2.7 occurred on 2013-10-28 14:46:00.000
Class M1.5 occurred on 2013-10-28 20:48:00.000
Class M1.4 occurred on 2013-10-28 11:32:00.000
These results can then be saved to a CSV file, or any other file
format that Table supports.
filtered_results.write("october_M1_flares.csv", format="csv")
Total running time of the script: (0 minutes 0.691 seconds)
Note
Go to the end to download the full example code.
Requesting cutouts of AIA images from the JSOC#
This example shows how to request a cutout of a series of AIA images from the JSOC.
import os
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import Time
from astropy.visualization import ImageNormalize, SqrtStretch
import sunpy.coordinates # NOQA
import sunpy.map
from sunpy.net import Fido
from sunpy.net import attrs as a
As this is an example, we have already worked out where we need to crop for the active region we want to showcase.
start_time = Time('2012-09-24T14:56:03', scale='utc', format='isot')
bottom_left = SkyCoord(-500*u.arcsec, -275*u.arcsec, obstime=start_time, observer="earth", frame="helioprojective")
top_right = SkyCoord(150*u.arcsec, 375*u.arcsec, obstime=start_time, observer="earth", frame="helioprojective")
Now construct the cutout from the coordinates above
above using the Cutout attribute.
cutout = a.jsoc.Cutout(bottom_left, top_right=top_right, tracking=True)
Exporting data from the JSOC requires registering your email first. Please replace this with your email address once you have registered like so: jsoc_email = “your_email@example.com” See this page for more details.
jsoc_email = os.environ["JSOC_EMAIL"]
Now we are ready to construct the query. Note that all of this is the same for a full-frame image except for the cutout component. We will download images from a 12 hour interval centered on the time of the above cutout. We request one image every 2 hours.
query = Fido.search(
a.Time(start_time - 6*u.h, start_time + 6*u.h),
a.Wavelength(171*u.angstrom),
a.Sample(2*u.h),
a.jsoc.Series.aia_lev1_euv_12s,
a.jsoc.Notify(jsoc_email),
a.jsoc.Segment.image,
cutout,
)
print(query)
Results from 1 Provider:
7 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
-------------------- -------- -------- -------- -------
2012-09-24T08:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T10:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T12:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T14:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T16:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T18:56:01Z SDO/AIA AIA_3 171 2128
2012-09-24T20:56:01Z SDO/AIA AIA_3 171 2128
Submit the export request and download the data.
files = Fido.fetch(query)
files.sort()
INFO: 7 URLs found for download. Full request totaling 7MB [sunpy.net.jsoc.jsoc]
Now that we’ve downloaded the files, we can create
a MapSequence from them and animate
them.
sequence = sunpy.map.Map(files, sequence=True)
fig = plt.figure()
ax = fig.add_subplot(projection=sequence.maps[0])
ani = sequence.plot(axes=ax, norm=ImageNormalize(vmin=0, vmax=5e3, stretch=SqrtStretch()))
plt.show()
Total running time of the script: (0 minutes 19.868 seconds)
Map#
Examples using Map with solar data
Segmenting a Map based on transformation of coordinates
Note
Go to the end to download the full example code.
Comparing Map Rotation Functions#
This example will compare between the current library implementations for sunpy.map.GenericMap.rotate.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
Rotating a map in sunpy has a choice between three libraries: scipy (the default),
scikit-image and opencv. Furthermore, one can also create a custom rotation
function and register it for use with rotate(),
see Adding a new rotation method.
Defining an appropriate metric to compare different algorithms is challenging. This example will just compare the raw value differences.
Using an HMI sample data, we will do a rotation to align the image to the north. By default, the order of rotation is 3.
hmi_map = sunpy.map.Map(sunpy.data.sample.HMI_LOS_IMAGE)
scipy_map = hmi_map.rotate(method='scipy')
skimage_map = hmi_map.rotate(method='scikit-image')
cv2_map = hmi_map.rotate(method='opencv')
Now for a visual comparison, the raw differences, that should highlight the differences. Note that only two comparisons are shown. Note the scale here is ± 10.
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
img1 = ax1.imshow(scipy_map.data - skimage_map.data, cmap='RdBu_r', vmin=-10, vmax=10)
ax1.set_title("HMI Difference: scipy vs scikit-image")
fig.colorbar(img1, ax=ax1)
img2 = ax2.imshow(scipy_map.data - cv2_map.data, cmap='RdBu_r', vmin=-10, vmax=10)
ax2.set_title("HMI Difference: scipy vs opencv")
fig.colorbar(img2, ax=ax2)
plt.show()
We can repeat this but for AIA data, using a 171 sample image. We will rotate it by the large amount of 30 degrees.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
scipy_map = aia_map.rotate(30*u.deg, method='scipy')
skimage_map = aia_map.rotate(30*u.deg, method='scikit-image')
cv2_map = aia_map.rotate(30*u.deg, method='opencv')
Now for a visual comparison, the raw differences, that should highlight the differences. Note that only two comparisons are shown. Note the scale here is ± 75.
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
img1 = ax1.imshow(scipy_map.data - skimage_map.data, cmap='RdBu_r', vmin=-75, vmax=75)
ax1.set_title("AIA Difference: scipy vs scikit-image")
fig.colorbar(img1, ax=ax1)
img2 = ax2.imshow(scipy_map.data - cv2_map.data, cmap='RdBu_r', vmin=-75, vmax=75)
ax2.set_title("AIA Difference: scipy vs opencv2")
fig.colorbar(img2, ax=ax2)
plt.show()
Total running time of the script: (0 minutes 2.683 seconds)
Note
Go to the end to download the full example code.
Creating a Composite map#
How to create a composite map and use it to overplot two maps to compare features.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
We start with the sample data. HMI shows the line-of-sight magnetic field at the photosphere while AIA 171 images show the resulting magnetic fields filled with hot plasma above, in the corona. We want to see what coronal features overlap with regions of strong line-of-sight magnetic fields.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
hmi_map = sunpy.map.Map(sunpy.data.sample.HMI_LOS_IMAGE)
bottom_left = [0, 0] * u.arcsec
top_right = [800, 800] * u.arcsec
aia_smap = aia_map.submap(SkyCoord(*bottom_left, frame=aia_map.coordinate_frame),
top_right=SkyCoord(*top_right, frame=aia_map.coordinate_frame))
hmi_smap = hmi_map.submap(SkyCoord(*bottom_left, frame=hmi_map.coordinate_frame),
top_right=SkyCoord(*top_right, frame=hmi_map.coordinate_frame))
Let’s create a CompositeMap which includes both maps.
comp_map = sunpy.map.Map(aia_smap, hmi_smap, composite=True)
# Let's set the contours of the HMI map, the second image in our composite map
# (therefore the index is 1), from a few hundred to a thousand Gauss which
# is the typical field associated with umbral regions of active regions.
levels = [-1000, -500, -250, 250, 500, 1000] * u.G
comp_map.set_levels(index=1, levels=levels)
Now let us look at the result. Notice that we can see the coronal structures present on the AIA image and how they correspond to the line of sight magnetic field.
fig = plt.figure()
ax = fig.add_subplot(projection=comp_map.get_map(0))
comp_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.704 seconds)
Note
Go to the end to download the full example code.
Creating a mask for LASCO C2 data#
In this example, we will manually create a mask to block the occulter in an unprocessed LASCO C2 coronagraph image.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
from sunpy.map import Map
from sunpy.map.maputils import all_coordinates_from_map
from sunpy.net import Fido
from sunpy.net import attrs as a
First, download some unprocessed LASCO C2 data with Fido.
result = Fido.search(a.Time('2011/06/07 06:30', '2011/06/07 06:36'),
a.Instrument.lasco,
a.Detector.c2)
lasco_file = Fido.fetch(result)
lasco_map = Map(lasco_file)
The LASCO C2 coronagraph has a field of view extending from 2-6 solar radii. So, our mask will have two parts: an inner component which masks the occulter and an outer component which masks data outside the field of view.
We will follow a process similar to Finding and masking bright pixels to express the coordinates relative to the occulter center.
pixel_coords = all_coordinates_from_map(lasco_map)
solar_center = SkyCoord(0*u.deg, 0*u.deg, frame=lasco_map.coordinate_frame)
pixel_radii = np.sqrt((pixel_coords.Tx-solar_center.Tx)**2 +
(pixel_coords.Ty-solar_center.Ty)**2)
# Note that the inner mask extends just beyond 2 solar radii to mask the
# Fresnel diffraction caused by the occulter edge.
mask_inner = pixel_radii < lasco_map.rsun_obs*2.4
mask_outer = pixel_radii > lasco_map.rsun_obs*6
final_mask = mask_inner + mask_outer
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:631: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obs_coord = self.observer_coordinate
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
To apply the final mask, we must create a new map.
masked_lasco = Map(lasco_map.data, lasco_map.meta, mask=final_mask)
# Before plotting the map, we need to create a new colormap to ensure we mask
# the bad values correctly.
occult_colormap = lasco_map.cmap.copy()
occult_colormap.set_bad('black')
fig = plt.figure()
ax1 = fig.add_subplot(1, 2, 1, projection=lasco_map)
ax2 = fig.add_subplot(1, 2, 2, projection=masked_lasco)
lasco_map.plot(clip_interval=(2, 98)*u.percent, cmap=occult_colormap, axes=ax1)
lasco_map.draw_limb()
ax1.set_title("Level 1 LASCO C2")
masked_lasco.plot(clip_interval=(2, 98)*u.percent, cmap=occult_colormap, axes=ax2)
masked_lasco.draw_limb()
ax2.set_title("Masked LASCO C2")
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:631: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obs_coord = self.observer_coordinate
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Total running time of the script: (0 minutes 8.393 seconds)
Note
Go to the end to download the full example code.
Cropping a Map#
How to crop a map by using submap.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
We start with the sample data
swap_map = sunpy.map.Map(sunpy.data.sample.SWAP_LEVEL1_IMAGE)
To crop the data you create a submap, specifying the top right and bottom left as SkyCoord objects.
top_right = SkyCoord(0 * u.arcsec, -200 * u.arcsec, frame=swap_map.coordinate_frame)
bottom_left = SkyCoord(-900 * u.arcsec, -900 * u.arcsec, frame=swap_map.coordinate_frame)
swap_submap = swap_map.submap(bottom_left, top_right=top_right)
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=swap_submap)
image = swap_submap.plot(axes=ax)
swap_submap.draw_limb(axes=ax)
swap_submap.draw_grid(axes=ax)
# Make some room and put the title at the top of the figure
ax.set_position([0.1, 0.1, 0.8, 0.7])
ax.set_title(ax.get_title(), pad=45)
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Total running time of the script: (0 minutes 0.939 seconds)
Note
Go to the end to download the full example code.
Finding bright regions with ndimage#
How you can to find the brightest regions in an AIA image and count the approximate number of regions of interest using ndimage.
import matplotlib.pyplot as plt
from scipy import ndimage
import sunpy.map
from sunpy.data.sample import AIA_193_IMAGE
We start with the sample data.
aiamap_mask = sunpy.map.Map(AIA_193_IMAGE)
aiamap = sunpy.map.Map(AIA_193_IMAGE)
First we make a mask, which tells us which regions are bright. We choose the criterion that the data should be at least 10% of the maximum value. Pixels with intensity values greater than this are included in the mask, while all other pixels are excluded.
mask = aiamap.data < aiamap.max() * 0.10
Mask is a bool array. It can be used to modify the original map object
without modifying the data. Once this mask attribute is set, we can plot the
image again.
aiamap_mask.mask = mask
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap_mask)
aiamap_mask.plot(axes=ax)
plt.colorbar()
plt.show()
Only the brightest pixels remain in the image. However, these areas are artificially broken up into small regions. We can solve this by applying some smoothing to the image data. Here we apply a 2D Gaussian smoothing function to the data.
data2 = ndimage.gaussian_filter(aiamap.data * ~mask, 14)
The issue with the filtering is that it create pixels where the values are small (<100), so when we go on later to label this array, we get one large region which encompasses the entire array. If you want to see, just remove this line.
data2[data2 < 100] = 0
Now we will make a second sunpy map with this smoothed data.
aiamap2 = sunpy.map.Map(data2, aiamap.meta)
The function scipy.ndimage.label counts the number of contiguous regions
in an image.
labels, n = ndimage.label(aiamap2.data)
Finally, we plot the smoothed bright image data, along with the estimate of the number of distinct regions. We can see that approximately 6 distinct hot regions are present above the 10% of the maximum level.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
ax.contour(labels)
plt.figtext(0.3, 0.2, f'Number of regions = {n}', color='white')
plt.show()
Total running time of the script: (0 minutes 1.028 seconds)
Note
Go to the end to download the full example code.
Finding contours of a map#
This example shows how to find and plot contours on a map.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_193_IMAGE
Start by loading the sample data.
aiamap = sunpy.map.Map(AIA_193_IMAGE)
In finding a set of contours, we have to provide the level to contour in the
same units as the map data. To find out the units we can inspect
sunpy.map.GenericMap.unit.
print(aiamap.unit)
ct
We can see that the units of this map are ct, or counts. We can now
chose a contour level, and use the contour()
method to extract the contours.
contours = aiamap.contour(50000 * u.ct)
Finally, we can plot the map, and add each of the contours in turn.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
for contour in contours:
ax.plot_coord(contour)
plt.show()
Total running time of the script: (0 minutes 0.972 seconds)
Note
Go to the end to download the full example code.
Finding the brightest pixel#
How to find and overplot the location of the brightest pixel.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data.
aia = sunpy.map.Map(AIA_171_IMAGE)
To find the brightest pixel, we find the maximum in the AIA image data then transform that pixel coordinate to a map coordinate.
pixel_pos = np.argwhere(aia.data == aia.data.max()) * u.pixel
hpc_max = aia.wcs.pixel_to_world(pixel_pos[:, 1], pixel_pos[:, 0])
Let’s now plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=aia)
aia.plot(axes=ax)
ax.plot_coord(hpc_max, 'wx', fillstyle='none', markersize=10)
plt.show()
Total running time of the script: (0 minutes 0.336 seconds)
Note
Go to the end to download the full example code.
Generating a map from data array#
A simple demonstration of creating a map from a numpy array of data.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import frames
Let’s create some data.
data = np.arange(0, 100).reshape(10, 10)
Next we need to create the metadata. This is made easier using the
make_fitswcs_header() function which will
create a header object for you. First define the reference coordinate
which requires a time and an observer location.
coord = SkyCoord(0*u.arcsec, 0*u.arcsec, obstime='2013-10-28 08:24',
observer='earth', frame=frames.Helioprojective)
Let’s pass that into the helper function along with some parameters. The reference pixel is the pixel is the one at the reference coordinate. The scale sets the size of the pixels. You can also to set a number of other metadata as well such as the instrument name and wavelength.
header = sunpy.map.make_fitswcs_header(data, coord,
reference_pixel=[0, 0]*u.pixel,
scale=[2, 2]*u.arcsec/u.pixel,
telescope='Fake Telescope', instrument='UV detector',
wavelength=1000*u.angstrom)
Let’s now create our map.
manual_map = sunpy.map.Map(data, header)
Let’s plot the result.
fig = plt.figure()
ax = fig.add_subplot(projection=manual_map)
manual_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.223 seconds)
Note
Go to the end to download the full example code.
Histograming map data#
How to inspect the histogram of the data of a map.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data and create a cutout.
aia = sunpy.map.Map(AIA_171_IMAGE)
bottom_left = SkyCoord(-300 * u.arcsec, 0 * u.arcsec, frame=aia.coordinate_frame)
top_right = SkyCoord(100 * u.arcsec, 400 * u.arcsec, frame=aia.coordinate_frame)
aia_smap = aia.submap(bottom_left, top_right=top_right)
aia_smap.plot()
<matplotlib.image.AxesImage object at 0x7f6f50413130>
The image of a GenericMap is always available in the data attribute.
Map also provides shortcuts to the image minimum and maximum values.
Let’s create a histogram of the data in this submap.
num_bins = 50
bins = np.linspace(aia_smap.min(), aia_smap.max(), num_bins)
hist, bin_edges = np.histogram(aia_smap.data, bins=bins)
Let’s plot the histogram as well as some standard values such as mean upper, and lower value and the one-sigma range.
fig, ax = plt.subplots()
# Note that we have to use ``.ravel()`` here to avoid matplotlib interpreting each
# row in the array as a different dataset to histogram.
ax.hist(aia_smap.data.ravel(), bins=bins, label='Histogram', histtype='step')
ax.set_xlabel('Intensity')
ax.axvline(aia_smap.min(), label=f'Data min={aia_smap.min():.2f}', color='black')
ax.axvline(aia_smap.max(), label=f'Data max={aia_smap.max():.2f}', color='black')
ax.axvline(aia_smap.data.mean(),
label=f'mean={aia_smap.data.mean():.2f}', color='green')
one_sigma = np.array([aia_smap.data.mean() - aia_smap.data.std(),
aia_smap.data.mean() + aia_smap.data.std()])
ax.axvspan(one_sigma[0], one_sigma[1], alpha=0.3, color='green',
label=f'mean +/- std = [{one_sigma[0]:.2f}, {one_sigma[1]:.2f}]')
ax.axvline(one_sigma[0], color='green')
ax.axvline(one_sigma[1], color='red')
ax.set_yscale('log')
ax.legend(loc=9)
<matplotlib.legend.Legend object at 0x7f6f4dfaa770>
Finally let’s overplot the one-sigma contours.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_smap)
aia_smap.plot(axes=ax)
levels = one_sigma / aia_smap.max() * u.percent * 100
aia_smap.draw_contours(axes=ax, levels=levels, colors=['blue'])
plt.show()
Total running time of the script: (0 minutes 0.833 seconds)
Note
Go to the end to download the full example code.
Map metadata modification#
How to query map metadata for changes.
sunpy has a series of map sources that can fix common issues with the original metadata stored in FITS files. A copy of the original (unaltered) metadata is stored, so any changes that sunpy (or the user) subsequently makes to the metadata can be easily queried.
In the example below, we load a HMI sample image, and query the metadata for any added, removed, or modified items.
import astropy.units as u
import sunpy.map
from sunpy.data.sample import HMI_LOS_IMAGE
Start by creating a map.
hmimap = sunpy.map.Map(HMI_LOS_IMAGE)
Now query the .meta attribute for any changes. We can see that nothing
has been added or removed, but the ‘bunit’ key has been updated from “Gauss”
to “G” to make it FITS standard compliant.
print("Added items:", hmimap.meta.added_items)
print("Removed items:", hmimap.meta.removed_items)
print("Modified items:", hmimap.meta.modified_items)
Added items: {}
Removed items: {}
Modified items: {}
If we modify the map in a way that updates the metadata, these properties allow use to easily see what has changed. As an example, lets rotate the map by 90 degrees, and see what has been updated.
hmimap = hmimap.rotate(90 * u.deg)
print("Added items:", hmimap.meta.added_items)
print("Removed items:", hmimap.meta.removed_items)
print("Modified items:", hmimap.meta.modified_items)
Added items: {'pc2_2': 0.0012234755004124243, 'pc2_1': 0.9999992515535698, 'pc1_1': 0.0012234755004124243, 'pc1_2': -0.9999992515535698}
Removed items: {'crota2': 179.9299}
Modified items: {'crval1': (original=-4.234319825764032, current=-4.234319825656712)}
Total running time of the script: (0 minutes 0.352 seconds)
Note
Go to the end to download the full example code.
Masking HMI based on the intensity of AIA#
In this example we will demonstrate how to mask out regions within a HMI image based on the intensity values of AIA.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.colors import Normalize
from skimage.measure import label, regionprops
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE, HMI_LOS_IMAGE
We will use an AIA 171 image from the sample data and crop it to capture a region of interest.
aia = sunpy.map.Map(AIA_171_IMAGE)
aia = aia.submap(
bottom_left=SkyCoord(-250, 200, unit=u.arcsec, frame=aia.coordinate_frame),
width=500 * u.arcsec,
height=400 * u.arcsec,
)
Next we create the HMI map and crop it to the same field of view as the AIA image.
hmi = sunpy.map.Map(HMI_LOS_IMAGE)
hmi = hmi.submap(aia.bottom_left_coord, top_right=aia.top_right_coord)
We then call the reproject_to() to reproject the AIA Map
to have exactly the same grid as the HMI Map.
We choose to reproject the AIA data to the HMI grid, rather than the reverse,
to avoid interpolating the LOS HMI magnetic field data.
This is because the range of the HMI data includes both positive and negative values and interpolation can destroy small scale variations in the LOS magnetic field which may be important in some scientific contexts.
aia = aia.reproject_to(hmi.wcs)
aia.nickname = 'AIA'
Now we will identify separate regions below a threshold in the AIA Map.
In this case, we want the darker patches that have pixel values below 200.
Then, using skimage, we can label()
and calculate the properties of each region using regionprops().
segmented = aia.data < 200
labeled = label(segmented)
regions = regionprops(labeled, hmi.data)
# We want the largest region, so we will sort by descending order in size.
regions = sorted(regions, key=lambda r: r.area, reverse=True)
Now to plot and label the first 7 regions seen in AIA with the region “0” being the largest.
fig = plt.figure()
ax = fig.add_subplot(projection=aia)
aia.plot(axes=ax)
aia.draw_contours(axes=ax, levels=200, colors="r")
for i in range(7):
plt.text(*np.flip(regions[i].centroid), str(i), color="w", ha="center", va="center")
Now let’s plot those same regions on the HMI Map.
fig = plt.figure()
ax = fig.add_subplot(projection=hmi)
im = hmi.plot(axes=ax, cmap="hmimag", norm=Normalize(-1500, 1500))
aia.draw_contours(axes=ax, levels=200, colors="r")
fig.colorbar(im)
<matplotlib.colorbar.Colorbar object at 0x7f6f50410e20>
Now we have the regions, we need to create a new HMI map that masks out everything but the largest region.
To do so, we need to create the mask from the bounding box returned by skimage.
Note that we can do this from the thresholded region only because our AIA and HMI images are on the same pixel grid after reprojecting the AIA image.
bbox = regions[0].bbox
mask = np.ones_like(hmi.data, dtype=bool)
mask[bbox[0]: bbox[2], bbox[1]: bbox[3]] = ~regions[0].image
hmi_masked = sunpy.map.Map((hmi.data, hmi.meta), mask=mask)
We can then plot the largest HMI region.
fig = plt.figure()
ax = fig.add_subplot(projection=hmi_masked)
im = hmi_masked.plot(axes=ax, cmap="hmimag", norm=Normalize(-1500, 1500))
fig.colorbar(im)
<matplotlib.colorbar.Colorbar object at 0x7f6f4e1fb910>
Finally, we can plot the distribution of HMI LOS magnetic field for only the unmasked values in the largest region shown above.
fig = plt.figure()
ax = fig.add_subplot()
ax.hist(hmi_masked.data[~hmi_masked.mask], bins='auto', histtype='step')
ax.set_ylabel('Number of Pixels')
ax.set_xlabel(f'LOS Magnetic Field [{hmi.unit:latex_inline}]')
plt.show()
Total running time of the script: (0 minutes 1.403 seconds)
Note
Go to the end to download the full example code.
Overplotting HMI Contours on an AIA Image#
This example shows how to use wcsaxes to overplot
unaligned HMI magnetic field strength contours on an AIA map.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_193_IMAGE, HMI_LOS_IMAGE
First let’s load two of the sample files into two Map objects.
aia, hmi = sunpy.map.Map(AIA_193_IMAGE, HMI_LOS_IMAGE)
To make the plot neater, we start by submapping the same region. We define the region in HGS coordinates and then apply the same submap to both the HMI and AIA maps.
bottom_left = SkyCoord(30 * u.deg, -40 * u.deg, frame='heliographic_stonyhurst')
top_right = SkyCoord(70 * u.deg, 0 * u.deg, frame='heliographic_stonyhurst')
sub_aia = aia.submap(bottom_left, top_right=top_right)
sub_hmi = hmi.submap(bottom_left, top_right=top_right)
To highlight the fact that the AIA and HMI images are not aligned, let us quickly view the two maps side-by-side.
fig = plt.figure(figsize=(11, 5))
ax1 = fig.add_subplot(121, projection=sub_aia)
sub_aia.plot(axes=ax1, clip_interval=(1, 99.99)*u.percent)
ax2 = fig.add_subplot(122, projection=sub_hmi)
sub_hmi.plot(axes=ax2)
<matplotlib.image.AxesImage object at 0x7f6f42a30790>
In the next plot we will start by plotting the same aia submap, and draw a heliographic grid on top.
fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(projection=sub_aia)
sub_aia.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
sub_aia.draw_grid(axes=ax)
ax.set_title("AIA 193 with HMI magnetic field strength contours", y=1.1)
Text(0.5, 1.1, 'AIA 193 with HMI magnetic field strength contours')
Now we want to draw the contours, to enhance the appearance of the plot we explicitly list the levels, but then make them symmetric around 0
levels = [50, 100, 150, 300, 500, 1000] * u.Gauss
matplotlib requires the levels to be sorted, so we order them from lowest to highest by reversing the array.
levels = np.concatenate((-1 * levels[::-1], levels))
Before we add the contours to the axis we store the existing bounds of the as overplotting the contours will sometimes change the bounds, we re-apply them to the axis after the contours have been added.
bounds = ax.axis()
We use the map method draw_contours to simplify this process,
but this is a wrapper around contour. We set the
colormap, line width and transparency of the lines to improve the final
appearance.
cset = sub_hmi.draw_contours(levels, axes=ax, cmap='seismic', alpha=0.5)
ax.axis(bounds)
(-0.5, 219.5, -0.5, 254.5)
Finally, add a colorbar. We add an extra tick to the colorbar at the 0 point to make it clearer that it is symmetric, and tweak the size and location to fit with the axis better.
plt.colorbar(cset,
label=f"Magnetic Field Strength [{sub_hmi.unit}]",
ticks=list(levels.value) + [0],
shrink=0.8,
pad=0.17)
plt.show()
Total running time of the script: (0 minutes 1.483 seconds)
Note
Go to the end to download the full example code.
Plotting a Map without any Axes#
This examples shows you how to plot a Map without any annotations at all, i.e., to save as an image.
import matplotlib.pyplot as plt
import numpy as np
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
Create a sunpy Map from the sample data.
smap = sunpy.map.Map(AIA_171_IMAGE)
Plot the Map without a frame. We can setup a frameless figure and an axes which spans the whole canvas.
figure = plt.figure(frameon=False)
ax = plt.axes([0, 0, 1, 1])
# Disable the axis
ax.set_axis_off()
# Plot the map.
# Since we are not interested in the exact map coordinates,
# we can simply use :meth:`~matplotlib.Axes.imshow`.
norm = smap.plot_settings['norm']
norm.vmin, norm.vmax = np.percentile(smap.data, [1, 99.9])
ax.imshow(smap.data,
norm=norm,
cmap=smap.plot_settings['cmap'],
origin="lower")
<matplotlib.image.AxesImage object at 0x7f6f4dd4ff70>
At this point you could save the figure with savefig()
or show it:
plt.show()
Total running time of the script: (0 minutes 0.229 seconds)
Note
Go to the end to download the full example code.
Plotting a difference image#
This example shows how to compute and plot a difference image.
import matplotlib.colors as colors
import matplotlib.pyplot as plt
from astropy.visualization import ImageNormalize, SqrtStretch
import sunpy.data.sample
import sunpy.map
When analyzing solar imaging data it is often useful to look at the difference from one time step to the next (running difference) or the difference from the start of the sequence (base difference). In this example, we’ll use a sequence of AIA 193 cutout images taken during a flare.
First, load a series of images into a MapSequence.
m_seq = sunpy.map.Map([
sunpy.data.sample.AIA_193_CUTOUT01_IMAGE,
sunpy.data.sample.AIA_193_CUTOUT02_IMAGE,
sunpy.data.sample.AIA_193_CUTOUT03_IMAGE,
sunpy.data.sample.AIA_193_CUTOUT04_IMAGE,
sunpy.data.sample.AIA_193_CUTOUT05_IMAGE,
], sequence=True)
Let’s take a look at each image in the sequence. Note that these images are sampled at a 6.4 minute cadence, much lower than the actual 12 s AIA cadence. We adjust the plot setting to ensure the colorbar is the same at each time step.
fig = plt.figure()
ax = fig.add_subplot(projection=m_seq.maps[0])
ani = m_seq.plot(axes=ax, norm=ImageNormalize(vmin=0, vmax=5e3, stretch=SqrtStretch()))
plt.show()
And now we can take the actual difference. We will compute the running difference and the base difference for each image in the sequence. For the case of the first entry in the running difference, we just subtract it from itself.
But we have to decide what to do with the metadata. For example, what time does this difference image correspond to? The time of the first or second image? The mean time? You’ll have to decide what makes most sense for your application. Here, by subtracting the data of the first (and previous) data array from each map in the sequence, the resulting difference maps have the same metadata as each corresponding map in the sequence.
Note that, because arithmetic operations between GenericMap
objects are not supported, we subtract just the array data (with units
attached) of the second map from the first map. The quantity attribute
returns the image data as an Quantity, where the resulting
units are those returned by the unit attribute of the map.
m_seq_base = sunpy.map.Map([m - m_seq[0].quantity for m in m_seq[1:]], sequence=True)
m_seq_running = sunpy.map.Map(
[m - prev_m.quantity for m, prev_m in zip(m_seq[1:], m_seq[:-1])],
sequence=True
)
Finally, let’s plot the difference maps. We’ll apply a colormap and re-normalize the intensity so that it shows up well. First, we show the base difference map.
fig = plt.figure()
ax = fig.add_subplot(projection=m_seq_base.maps[0])
ani = m_seq_base.plot(axes=ax, title='Base Difference', norm=colors.Normalize(vmin=-200, vmax=200), cmap='Greys_r')
plt.colorbar(extend='both', label=m_seq_base[0].unit.to_string())
plt.show()
Then, we show the running difference map.
fig = plt.figure()
ax = fig.add_subplot(projection=m_seq_running.maps[0])
ani = m_seq_running.plot(axes=ax, title='Running Difference', norm=colors.Normalize(vmin=-200, vmax=200), cmap='Greys_r')
plt.colorbar(extend='both', label=m_seq_running[0].unit.to_string())
plt.show()
Total running time of the script: (0 minutes 9.056 seconds)
Note
Go to the end to download the full example code.
Resampling Maps#
How to resample a map using the resample method, which implements interpolation, or using superpixels, which combines pixels.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We start with the sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
To reduce the angular resolution of the map, you can use the
resample() method, specifying the new dimensions
in pixels. By default, this method uses linear interpolation but this can be
changed with the method argument (‘nearest’, ‘linear’ or ‘spline’).
new_dimensions = [40, 40] * u.pixel
aia_resampled_map = aia_map.resample(new_dimensions)
Let’s plot the result.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_resampled_map)
aia_resampled_map.plot(axes=ax)
plt.show()
Another way to reduce the angular resolution of the map is by using the
superpixel() method, which combines pixels.
The superpixel dimensions do not need to be square, and the intensity of
each superpixel defaults to the sum of the constituent pixels. For example,
you can reduce the AIA map resolution by a factor of 16 by specifying 16x16
superpixels.
superpixel_size = [16, 16] * u.pixel
aia_superpixel_map = aia_map.superpixel(superpixel_size)
Let’s plot the result.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_superpixel_map)
aia_superpixel_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.521 seconds)
Note
Go to the end to download the full example code.
Rotating a Map#
How to rotate a map.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We start with the sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
GenericMap provides the rotate()
method which accepts an angle. This returns a rotated map and does not
modify the original map. The data array size is expanded so that none of the
original data is lost due to cropping.
aia_rotated = aia_map.rotate(angle=30 * u.deg)
Let’s now plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_rotated)
aia_rotated.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
aia_rotated.draw_limb(axes=ax)
aia_rotated.draw_grid(axes=ax)
plt.show()
Total running time of the script: (0 minutes 1.129 seconds)
Note
Go to the end to download the full example code.
Segmenting a Map based on transformation of coordinates#
This example demonstrates extracting a region of a particular map based on world coordinates in different systems.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import NorthOffsetFrame
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data.
smap = sunpy.map.Map(AIA_171_IMAGE)
A utility function gives us access to the helioprojective coordinate of each pixel in this map. From this we then transform the coordinates to the required frame. For this example we are going to extract a region based on the heliographic Stonyhurst coordinates, so we transform to that frame.
all_hpc = sunpy.map.all_coordinates_from_map(smap)
all_hgs = all_hpc.transform_to("heliographic_stonyhurst")
Let’s then segment the data based on coordinates and create a boolean mask
where True indicates invalid or deselected data.
Numpy’s masked arrays allow for a combination of standard numpy array and
a mask array. When an element of the mask is True, the corresponding element
of the associated array is said to be masked (invalid).
For more information about numpy’s masked arrays see numpy.ma.
We now mask out all values not in our coordinate range or where the
coordinates are NaN (because they could not be transformed to the
surface of the Sun).
segment_mask = np.logical_or(all_hgs.lon >= 35 * u.deg, all_hgs.lon <= -35 * u.deg)
segment_mask |= np.isnan(all_hgs.lon)
To plot the segment separately, we create a new map with the segment as the mask.
new_frame_map = sunpy.map.Map(smap.data, smap.meta, mask=segment_mask)
fig = plt.figure()
ax = fig.add_subplot(projection=new_frame_map)
new_frame_map.plot(axes=ax)
new_frame_map.draw_grid(axes=ax, color='red')
plt.show()
We can perform various mathematical operations on the extracted segment such as averaging the pixel values or finding the sum of the segment.
masked_data = np.ma.array(new_frame_map.data, mask=new_frame_map.mask)
print(f"Original Map : mean = {smap.data.mean()}, sum = {smap.data.sum()}")
print(f"Segment : mean = {masked_data.mean()}, sum = {masked_data.sum()}")
Original Map : mean = 427.02252197265625, sum = 447765568.0
Segment : mean = 556.6856616925717, sum = 156049568.0
Using sunpy.coordinates.NorthOffsetFrame#
Let us offset the north pole and create the frame.
north = SkyCoord(20 * u.deg, 20 * u.deg, frame="heliographic_stonyhurst")
offset_frame = NorthOffsetFrame(north=north)
We then transform coordinates to the offsetted frame and segment the data based on conditions.
all_hpc = sunpy.map.all_coordinates_from_map(smap)
offsetted_coords = all_hpc.transform_to(offset_frame)
segment_mask = np.logical_or(offsetted_coords.lon >= 30 * u.deg,
offsetted_coords.lon <= -20 * u.deg)
Masking out the NaN values of offsetted_coords.lon, we get:
segment_mask |= np.isnan(offsetted_coords.lon)
Let’s plot the offsetted segment separately.
offsetted_map = sunpy.map.Map(smap.data, smap.meta, mask=segment_mask)
fig = plt.figure()
ax = fig.add_subplot(projection=smap)
offsetted_map.plot(axes=ax)
overlay = ax.get_coords_overlay(offset_frame)
overlay[0].set_ticks(spacing=30. * u.deg)
overlay.grid(ls='--', color='blue')
offsetted_map.draw_grid(axes=ax, color='red')
plt.show()
We can also find the maximum, minimum or average pixel values of the segment and compare it with the original map.
offset_masked_data = np.ma.array(offsetted_map.data, mask=offsetted_map.mask)
print(f"Original Map : mean = {smap.data.mean()}, "
f"maximum value = {smap.data.max()}, "
f"minimum value = {smap.data.min()}")
print(f"Offset segment : mean = {offset_masked_data.mean()}, "
f"maximum value = {offset_masked_data.max()}, "
f"minimum value = {offset_masked_data.min()}")
Original Map : mean = 427.02252197265625, maximum value = 192130.171875, minimum value = -129.78036499023438
Offset segment : mean = 637.0416340830179, maximum value = 9466.236328125, minimum value = 91.12757110595703
Total running time of the script: (0 minutes 2.566 seconds)
Combining, Co-aligning, and Reprojecting Images#
Examples of combining, aligning, and reprojecting sunpy maps
Reprojecting to a Map Projection with a Custom Origin
Note
Go to the end to download the full example code.
Aligning AIA and HMI Data with Reproject#
This example shows how to reproject one image to be aligned with another image.
Here we use the reproject package to transform one image to the reference frame of another image. The reference frames are defined by the respective World Coordinate System (WCS) information. This is a very generic way of aligning data, and can be very accurate.
You will need reproject v0.6 or higher installed.
See Auto-Aligning AIA and HMI Data During Plotting for an alternate approach to image alignment, where the images are aligned as they are plotted, with no modified map created.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We use the AIA image and HMI image from the sample data. For the HMI map, we use the special HMI color map, which expects the plotted range to be -1500 to 1500.
map_aia = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
map_hmi = sunpy.map.Map(sunpy.data.sample.HMI_LOS_IMAGE)
map_hmi.plot_settings['cmap'] = "hmimag"
map_hmi.plot_settings['norm'] = plt.Normalize(-1500, 1500)
Plot both images side by side.
fig = plt.figure(figsize=(12, 5))
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
ax2 = fig.add_subplot(122, projection=map_hmi)
map_hmi.plot(axes=ax2)
<matplotlib.image.AxesImage object at 0x7f6f427bc580>
We can now reproject the HMI image to the WCS of the AIA image. We are using
the fast reproject_interp, however the slower but most accurate
reproject_exact would also work well here. The
reproject_exact function only works when reprojecting between
two WCSes with the same observer, which makes it well suited to aligning
data.
out_hmi = map_hmi.reproject_to(map_aia.wcs)
Plot the images side by side.
Note that off-disk HMI data are not retained by default because an
additional assumption is required to define the location of the HMI
emission in 3D space. We can use
assume_spherical_screen() to
retain the off-disk HMI data. See
Reprojecting Using a Spherical Screen
for more reference.
fig = plt.figure(figsize=(12, 5))
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
ax2 = fig.add_subplot(122, projection=out_hmi)
out_hmi.plot(axes=ax2, title='Reprojected HMI image')
<matplotlib.image.AxesImage object at 0x7f6f427bdb40>
As both of these images are now on the same pixel grid we can directly plot them over one another, by setting the transparency of the HMI plot.
fig = plt.figure()
ax1 = fig.add_subplot(projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
out_hmi.plot(axes=ax1, alpha=0.5)
plt.title('HMI overlaid on AIA')
plt.show()
Total running time of the script: (0 minutes 3.165 seconds)
Note
Go to the end to download the full example code.
Auto-Aligning AIA and HMI Data During Plotting#
This example shows how to auto-align two images with different reference frames during plotting.
Here we use the optional keyword autoalign when calling Map’s
plot() method. The reference frames are defined by
the respective World Coordinate System (WCS) information.
See Aligning AIA and HMI Data with Reproject for an alternate approach to image alignment, where one of the maps is modified prior to plotting, and thus is available for purposes other than plotting.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We use the AIA image and HMI image from the sample data. For the HMI map, we use the special HMI color map, which expects the plotted range to be -1500 to 1500.
map_aia = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
map_hmi = sunpy.map.Map(sunpy.data.sample.HMI_LOS_IMAGE)
map_hmi.plot_settings['cmap'] = "hmimag"
map_hmi.plot_settings['norm'] = plt.Normalize(-1500, 1500)
Plot both images side by side. Note that the HMI image is oriented “upside down” relative to the AIA image.
fig = plt.figure(figsize=(12, 5))
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
ax2 = fig.add_subplot(122, projection=map_hmi)
map_hmi.plot(axes=ax2)
<matplotlib.image.AxesImage object at 0x7f6f42a3c0d0>
Setting autoalign=True allows plotting the HMI image onto axes
defined by the AIA reference frame. In contrast to the above code
block, we intentionally set the projection for the axes to be
the AIA map instead of the HMI map. We also need to manually set
the plot limits because Matplotlib gets confused by the off-disk
parts of the image. The HMI image now has the same
orientation as the AIA image.
Note that off-disk HMI data are not retained by default because an
additional assumption is required to define the location of the HMI
emission in 3D space. We can use
assume_spherical_screen() to
retain the off-disk HMI data. See
Reprojecting Using a Spherical Screen
for more reference.
fig = plt.figure(figsize=(12, 5))
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
ax2 = fig.add_subplot(122, projection=map_aia)
map_hmi.plot(axes=ax2, autoalign=True, title='HMI image in AIA reference frame')
ax2.axis(ax1.axis())
(-0.5, 1023.5, -0.5, 1023.5)
We can directly plot them over one another, by setting the transparency of the HMI plot.
fig = plt.figure()
ax1 = fig.add_subplot(projection=map_aia)
map_aia.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
map_hmi.plot(axes=ax1, autoalign=True, alpha=0.5)
ax1.set_title('HMI overlaid on AIA')
plt.show()
Total running time of the script: (0 minutes 5.642 seconds)
Note
Go to the end to download the full example code.
Creating Carrington Maps#
In this example we use the reproject generate a map in heliographic Carrington coordinates from a full-disk AIA image.
You will need reproject v0.6 or higher installed.
import matplotlib.pyplot as plt
import sunpy.data.sample
import sunpy.map
from sunpy.map.header_helper import make_heliographic_header
We will start with using sunpy’s sample data for this example.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_193_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax)
<matplotlib.image.AxesImage object at 0x7f6f42925ba0>
Reproject works by transforming an input image to a desired World Coordinate
System (WCS) projection. Here we use sunpy.map.header_helper.make_heliographic_header()
to create a FITS WCS header based on a heliographic Carrington reference
coordinate.
shape = (720, 1440)
carr_header = make_heliographic_header(aia_map.date, aia_map.observer_coordinate, shape, frame='carrington')
With the new header, re-project the data into the new coordinate system.
The reproject_to() defaults to using
the fast reproject.reproject_interp() algorithm, but a different
algorithm can be specified (e.g., reproject.reproject_adaptive()).
outmap = aia_map.reproject_to(carr_header)
Plot the result.
fig = plt.figure()
ax = fig.add_subplot(projection=outmap)
outmap.plot(axes=ax)
outmap.draw_limb(color='blue')
plt.show()
Total running time of the script: (0 minutes 1.882 seconds)
Note
Go to the end to download the full example code.
Creating a Full Sun Map with AIA and EUVI#
With SDO/AIA and STEREO/A and STEREO/B, it is possible (for specific dates) to combine combine three EUV images from these satellites to produce a nearly full latitude / longitude map of the Sun.
You will need reproject v0.6 or higher installed.
import matplotlib.pyplot as plt
import numpy as np
from reproject import reproject_interp
from reproject.mosaicking import reproject_and_coadd
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.wcs import WCS
import sunpy.map
import sunpy.sun
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.data.sample import AIA_193_JUN2012, STEREO_A_195_JUN2012, STEREO_B_195_JUN2012
First create a sunpy map for each of the files.
maps = sunpy.map.Map(sorted([AIA_193_JUN2012, STEREO_A_195_JUN2012, STEREO_B_195_JUN2012]))
To reduce memory consumption we also downsample these maps before continuing, you can disable this.
maps = [m.resample((1024, 1024)*u.pix) for m in maps]
When combining these images all three need to assume the same radius of the Sun for the data. The AIA images specify a slightly different value than the IAU 2015 constant. To avoid coordinate transformation issues we reset this here.
maps[0].meta['rsun_ref'] = sunpy.sun.constants.radius.to_value(u.m)
Next we will plot the locations of the three spacecraft with respect to the Sun so we can easily see the relative separations.
earth = get_body_heliographic_stonyhurst('earth', maps[0].date)
fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(projection='polar')
circle = plt.Circle((0.0, 0.0), (10*u.Rsun).to_value(u.AU),
transform=ax.transProjectionAffine + ax.transAxes, color="yellow",
alpha=1, label="Sun")
ax.add_artist(circle)
ax.text(earth.lon.to_value("rad")+0.05, earth.radius.to_value(u.AU), "Earth")
for this_satellite, this_coord in [(m.observatory, m.observer_coordinate) for m in maps]:
ax.plot(this_coord.lon.to('rad'), this_coord.radius.to(u.AU), 'o', label=this_satellite)
ax.set_theta_zero_location("S")
ax.set_rlim(0, 1.3)
ax.legend()
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
The next step is to calculate the output coordinate system for the combined
map. We select a heliographic Stonyhurst frame, and a Plate Carree (CAR)
projection, and generate a header using sunpy.map.header_helper.make_fitswcs_header and
then construct a World Coordinate System (WCS) object for that header.
shape_out = (180, 360) # This is set deliberately low to reduce memory consumption
header = sunpy.map.make_fitswcs_header(shape_out,
SkyCoord(0, 0, unit=u.deg,
frame="heliographic_stonyhurst",
obstime=maps[0].date),
scale=[360 / shape_out[1],
180 / shape_out[0]] * u.deg / u.pix,
wavelength=int(maps[0].meta['wavelnth']) * u.AA,
projection_code="CAR")
out_wcs = WCS(header)
Next we call the reproject.mosaicking.reproject_and_coadd function, which
takes a list of maps, and the desired output WCS and array shape.
array, footprint = reproject_and_coadd(maps, out_wcs, shape_out,
reproject_function=reproject_interp)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
To display the output we construct a new map using the new array and our generated header. We also borrow the plot settings from the AIA map.
outmap = sunpy.map.Map((array, header))
outmap.plot_settings = maps[0].plot_settings
fig = plt.figure()
ax = fig.add_subplot(projection=outmap)
outmap.plot(axes=ax)
plt.show()
Improving the Output#
As you can see this leaves a little to be desired. To reduce the obvious warping towards the points which are close to the limb in the input images, we can define a set of weights to use when co-adding the output arrays. To reduce this warping we want to calculate an set of weights which highly weigh points close to the centre of the disk in the input image.
We can achieve this by using sunpy’s coordinate framework. First we calculate all the world coordinates for all the pixels in all three input maps.
coordinates = tuple(map(sunpy.map.all_coordinates_from_map, maps))
To get a weighting which is high close to disk centre and low towards the limb, we can use the Z coordinate in the heliocentric frame. This coordinate is the distance of the sphere from the centre of the Sun towards the observer.
weights = [coord.transform_to("heliocentric").z.value for coord in coordinates]
These weights are good, but they are better if the ramp down is a little smoother, and more biased to the centre. Also we can scale them to the range 0-1, and set any off disk (NaN) regions to 0.
weights = [(w / np.nanmax(w)) ** 3 for w in weights]
for w in weights:
w[np.isnan(w)] = 0
fig, ax = plt.subplots()
im = ax.imshow(weights[0])
fig.colorbar(im)
plt.show()
Now we can rerun the reprojection. This time we also set
match_background=True which scales the images by a single scaling
factor so they are of similar brightness. We also set
background_reference=0 which uses the AIA map as the reference for
the background scaling.
Here we are using the fastest but least accurate method of reprojection,
reproject.reproject_interp, a more accurate but slower method is
reproject.reproject_adaptive.
array, _ = reproject_and_coadd(maps, out_wcs, shape_out,
input_weights=weights,
reproject_function=reproject_interp,
match_background=True,
background_reference=0)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/coordinates/frames.py:589: SunpyUserWarning: The conversion of these 2D helioprojective coordinates to 3D is all NaNs because off-disk coordinates need an additional assumption to be mapped to calculate distance from the observer. Consider using the context manager `Helioprojective.assume_spherical_screen()`.
warn_user("The conversion of these 2D helioprojective coordinates to 3D is all NaNs "
Once again we create a new map, and this time we customise the plot a little.
outmap = sunpy.map.Map((array, header))
outmap.plot_settings = maps[0].plot_settings
outmap.nickname = 'AIA + EUVI/A + EUVI/B'
fig = plt.figure(figsize=(10, 5))
ax = fig.add_subplot(projection=outmap)
im = outmap.plot(axes=ax, vmin=400)
lon, lat = ax.coords
lon.set_coord_type("longitude")
lon.coord_wrap = 180 * u.deg
lon.set_format_unit(u.deg)
lat.set_coord_type("latitude")
lat.set_format_unit(u.deg)
lon.set_axislabel('Heliographic Longitude', minpad=0.8)
lat.set_axislabel('Heliographic Latitude', minpad=0.9)
lon.set_ticks(spacing=25*u.deg, color='k')
lat.set_ticks(spacing=15*u.deg, color='k')
plt.colorbar(im, ax=ax)
# Reset the view to pixel centers
_ = ax.axis((0, shape_out[1], 0, shape_out[0]))
plt.show()
Total running time of the script: (0 minutes 4.175 seconds)
Note
Go to the end to download the full example code.
Reprojecting Images to Different Observers#
This example demonstrates how you can reproject images to the view from different observers. We use data from these two instruments:
AIA on SDO, which is in orbit around Earth
EUVI on STEREO A, which is in orbit around the Sun away from the Earth
You will need reproject v0.6 or higher installed.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.data.sample import AIA_193_JUN2012, STEREO_A_195_JUN2012
In this example we are going to make a lot of side by side figures, so let’s change the default figure size.
plt.rcParams['figure.figsize'] = (16, 8)
Create a map for each image, after making sure to sort by the appropriate name attribute (i.e., “AIA” and “EUVI”) so that the order is reliable.
map_list = sunpy.map.Map([AIA_193_JUN2012, STEREO_A_195_JUN2012])
map_list.sort(key=lambda m: m.detector)
map_aia, map_euvi = map_list
# We downsample these maps to reduce memory consumption, but you can
# comment this out.
out_shape = (512, 512)
map_aia = map_aia.resample(out_shape * u.pix)
map_euvi = map_euvi.resample(out_shape * u.pix)
Plot the two maps, with the solar limb as seen by each observatory overlaid on both plots.
fig = plt.figure()
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1)
map_aia.draw_limb(axes=ax1, color='white')
map_euvi.draw_limb(axes=ax1, color='red')
ax2 = fig.add_subplot(122, projection=map_euvi)
map_euvi.plot(axes=ax2)
limb_aia = map_aia.draw_limb(axes=ax2, color='white')
limb_euvi = map_euvi.draw_limb(axes=ax2, color='red')
plt.legend([limb_aia[0], limb_euvi[0]],
['Limb as seen by AIA', 'Limb as seen by EUVI A'])
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
<matplotlib.legend.Legend object at 0x7f6f5058d960>
Data providers can set the radius at which emission in the map is assumed to have come from. Most maps use a default value for photospheric radius (including EUVI maps), but some maps (including AIA maps) are set to a slightly different value. A mismatch in solar radius means a reprojection will not work correctly on pixels near the limb. This can be prevented by modifying the values for rsun on one map to match the other.
map_euvi.meta['rsun_ref'] = map_aia.meta['rsun_ref']
We can reproject the EUVI map to the AIA observer wcs using
reproject_to(). This method defaults to using
the fast reproject.reproject_interp() algorithm, but a different
algorithm can be specified (e.g., reproject.reproject_adaptive()).
outmap = map_euvi.reproject_to(map_aia.wcs)
We can now plot the STEREO/EUVI image as seen from the position of SDO, next to the AIA image.
fig = plt.figure()
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1)
ax2 = fig.add_subplot(122, projection=outmap)
outmap.plot(axes=ax2, title='EUVI image as seen from SDO')
map_euvi.draw_limb(color='blue')
# Set the HPC grid color to black as the background is white
ax2.coords[0].grid_lines_kwargs['edgecolor'] = 'k'
ax2.coords[1].grid_lines_kwargs['edgecolor'] = 'k'
AIA as Seen from Mars#
The new observer coordinate doesn’t have to be associated with an existing Map. sunpy provides a function which can get the location coordinate for any known body. In this example, we use Mars.
mars = get_body_heliographic_stonyhurst('mars', map_aia.date)
Without a target Map wcs, we can generate our own for an arbitrary observer.
First, we need an appropriate reference coordinate. This will be similar to
the one contained in map_aia, except with the observer placed at Mars.
mars_ref_coord = SkyCoord(0*u.arcsec, 0*u.arcsec,
obstime=map_aia.reference_coordinate.obstime,
observer=mars,
rsun=map_aia.reference_coordinate.rsun,
frame="helioprojective")
We now need to construct our output WCS; we build a custom header using
sunpy.map.header_helper.make_fitswcs_header() using the map_aia
properties and our new, mars-based reference coordinate.
mars_header = sunpy.map.make_fitswcs_header(
out_shape,
mars_ref_coord,
scale=u.Quantity(map_aia.scale),
instrument="AIA",
wavelength=map_aia.wavelength
)
We generate the output map and plot it next to the original image.
outmap = map_aia.reproject_to(mars_header)
fig = plt.figure()
ax1 = fig.add_subplot(121, projection=map_aia)
map_aia.plot(axes=ax1)
map_aia.draw_grid(color='w')
ax2 = fig.add_subplot(122, projection=outmap)
outmap.plot(axes=ax2, title='AIA observation as seen from Mars')
map_aia.draw_grid(color='w')
map_aia.draw_limb(color='blue')
plt.show()
Total running time of the script: (0 minutes 3.556 seconds)
Note
Go to the end to download the full example code.
Reprojecting Using a Spherical Screen#
This example demonstrates how you can reproject an image as if it lies on the inside of a spherical screen and the observer is not at the center of the sphere. This functionality is primarily for visualization purposes, since features in the image are unlikely to actually lie on this spherical screen.
You will need reproject v0.6 or higher installed.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import Helioprojective
from sunpy.data.sample import AIA_171_IMAGE
We will use one of the AIA images from the sample data. We fix the range of values for the Map’s normalizer.
aia_map = sunpy.map.Map(AIA_171_IMAGE)
aia_map.plot_settings['norm'].vmin = 0
aia_map.plot_settings['norm'].vmax = 10000
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax)
plt.show()
Let’s define a new observer that is well separated from Earth.
new_observer = SkyCoord(70*u.deg, 20*u.deg, 1*u.AU, obstime=aia_map.date,
frame='heliographic_stonyhurst')
Create a WCS header for this new observer using helioprojective coordinates.
out_shape = aia_map.data.shape
out_ref_coord = SkyCoord(0*u.arcsec, 0*u.arcsec, obstime=new_observer.obstime,
frame='helioprojective', observer=new_observer,
rsun=aia_map.coordinate_frame.rsun)
out_header = sunpy.map.make_fitswcs_header(
out_shape,
out_ref_coord,
scale=u.Quantity(aia_map.scale),
instrument=aia_map.instrument,
wavelength=aia_map.wavelength
)
If you reproject the AIA Map to the perspective of the new observer, the default assumption is that the image lies on the surface of the Sun. However, the parts of the image beyond the solar disk cannot be mapped to the surface of the Sun, and thus do not show up in the output.
outmap_default = aia_map.reproject_to(out_header)
fig = plt.figure()
ax = fig.add_subplot(projection=outmap_default)
outmap_default.plot(axes=ax)
plt.show()
You can use the different assumption that the image lies on the surface of a spherical screen centered at AIA, with a radius equal to the Sun-AIA distance. The curvature of the spherical screen is not obvious in this plot due to the relatively small field of view of AIA (compared to, say, a coronagraph).
with Helioprojective.assume_spherical_screen(aia_map.observer_coordinate):
outmap_screen_all = aia_map.reproject_to(out_header)
fig = plt.figure()
ax = fig.add_subplot(projection=outmap_screen_all)
outmap_screen_all.plot(axes=ax)
plt.show()
Finally, you can specify that the spherical-screen assumption should be used for only off-disk parts of the image, and continue to map on-disk parts of the image to the surface of the Sun.
with Helioprojective.assume_spherical_screen(aia_map.observer_coordinate,
only_off_disk=True):
outmap_screen_off_disk = aia_map.reproject_to(out_header)
fig = plt.figure()
ax = fig.add_subplot(projection=outmap_screen_off_disk)
outmap_screen_off_disk.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 5.761 seconds)
Note
Go to the end to download the full example code.
Reprojecting to a Map Projection with a Custom Origin#
In this example, we show how to reproject a map to a map projection with a custom origin. Here, we choose the target map projection to be the azimuthal equidistant projection, also known as the Postel projection, which has useful properties relative to a specified origin of the projection. If a different map projection is desired, modifying this example is straightforward.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We will use one of the AIA images from the sample data. We fix the range of values for the Map’s normalizer for a prettier image.
aia_map = sunpy.map.Map(AIA_171_IMAGE)
aia_map.plot_settings['norm'].vmin = 0
aia_map.plot_settings['norm'].vmax = 10000
Next, we create a SkyCoord to define the custom origin
of the map projection. Here, we are going to center the projection at the
helioprojective coordinates of a particular active region. We want our map
projection to be in heliographic Stonyhurst coordinates, so we transform the
origin coordinate accordingly.
origin_hpc = SkyCoord(735*u.arcsec, -340*u.arcsec, frame=aia_map.coordinate_frame)
origin = origin_hpc.heliographic_stonyhurst
We then create a FITS-WCS header that includes our custom origin coordinate.
The azimuthal equidistant projection is specified by the code "ARC".
See Supported projections for the projection codes for
other projections.
out_shape = (750, 750)
out_header = sunpy.map.make_fitswcs_header(
out_shape,
origin,
scale=[0.4, 0.4]*u.deg/u.pix,
projection_code="ARC"
)
We reproject the map to our FITS-WCS header and copy over the plot settings.
out_map = aia_map.reproject_to(out_header)
out_map.plot_settings = aia_map.plot_settings
Finally, we plot both the original and reprojected maps side by side.
fig = plt.figure(figsize=(8, 4))
Plot the original AIA map, with the active region circled in red and the heliographic grid and solar limb in blue.
ax = fig.add_subplot(1, 2, 1, projection=aia_map)
aia_map.plot(axes=ax)
aia_map.draw_grid(axes=ax, color='blue')
aia_map.draw_limb(axes=ax, color='blue')
ax.plot_coord(origin, 'o', color='red', fillstyle='none', markersize=20)
[<matplotlib.lines.Line2D object at 0x7f6f5049ae60>]
Plot the reprojected AIA map, again with the active region circled in red and the heliographic grid and solar limb in blue.
ax = fig.add_subplot(1, 2, 2, projection=out_map)
out_map.plot(axes=ax)
out_map.draw_grid(axes=ax, color='blue')
out_map.draw_limb(axes=ax, color='blue')
ax.plot_coord(origin, 'o', color='red', fillstyle='none', markersize=20)
ax.set_title('Postel projection centered at ROI', y=-0.1)
plt.show()
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:2165: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
observer=self.observer_coordinate,
Total running time of the script: (0 minutes 1.624 seconds)
Note
Go to the end to download the full example code.
Rotating HMI maps so they’re not ‘upside-down’#
This example shows how to rotate a HMI magnetogram, so when you plot it it appears with solar North pointing up.
import matplotlib.pyplot as plt
import sunpy.map
from sunpy.data.sample import HMI_LOS_IMAGE
We will use the sunpy HMI sample image, load it into a map and plot it.
We see that solar North is pointed down instead of up in this image, which is
indicated by the coordinates (that range from positive to negative, rather
than negative to positive).
hmi_map = sunpy.map.Map(HMI_LOS_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=hmi_map)
hmi_map.plot(axes=ax)
plt.show()
Now rotate the image such that solar North is pointed up.
We have to do this because the HMI instrument is mounted upside-down
relative to the AIA instrument on the SDO satellite, which means most
of the images are taken with solar North pointed down.
The roll angle of the instrument is reported in the FITS header
keyword CROTA2 (see Figure 17 of
Couvidat et al. (2016),
which states that “the nominal CROTA2 for HMI is ≈179.93”).
The order keyword, below, specifies the type of interpolation; in this case, 3 refers to bi-cubic.
hmi_rotated = hmi_map.rotate(order=3)
fig = plt.figure()
ax = fig.add_subplot(projection=hmi_rotated)
hmi_rotated.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.825 seconds)
Time Series#
Examples using TimeSeries
Creating a TimeSeries from GOES-XRS near real time data
Retrieving and analyzing GOES X-Ray Sensor (XRS) data
Smoothing of timeSeries data using convolution filters
Note
Go to the end to download the full example code.
Creating a TimeSeries from GOES-XRS near real time data#
This example will demonstrate how to download and load GOES XRS Near Real Time (NRT) data into a sunpy.timeseries.TimeSeries.
import matplotlib.pyplot as plt
import pandas as pd
from astropy import units as u
from sunpy import timeseries as ts
from sunpy.time import parse_time
We will start by getting reading the GOES-XRS JSON file using pandas.read_json.
This allows us to download the file and load it straight into a pandas.DataFrame.
This file updates every minute and contains only the last 7 days worth of data.
goes_json_data = pd.read_json("https://services.swpc.noaa.gov/json/goes/primary/xrays-7-day.json")
XRS collects data in two energy channels, “0.05-0.4nm” and “0.1-0.8nm”. We separate these “short” and “long” wavelength readings into two arrays.
# This will get us the short wavelength data.
goes_short = goes_json_data[goes_json_data["energy"] == "0.05-0.4nm"]
# This will get us the long wavelength data.
goes_long = goes_json_data[goes_json_data["energy"] == "0.1-0.8nm"]
sunpy.timeseries.TimeSeries requires a datetime index which we can get
directly and transform into astropy.time.Time.
time_array = parse_time(goes_short["time_tag"])
sunpy.timeseries.TimeSeries requires that there are units for data variables.
To do this, we will create a dictionary that will map the names of the two columns,
“xrsa” and “xrsb” (the channel names for GOES XRS), to their corresponding
physical flux units, u.W/u.m**2.
units = dict([("xrsa", u.W/u.m**2), ("xrsb", u.W/u.m**2)])
We need to create a metadata dictionary for the data.
Typically, sunpy.timeseries.TimeSeries reads the metadata directly from the file.
However, here we need to define our own metadata and we will keep it fairly simple.
meta = dict({"instrument": "GOES X-ray sensor", "measurements": "primary", "type": "quicklook"})
The final pre-step is create a new pandas.DataFrame which we can pass to
sunpy.timeseries.TimeSeries as the data input.
goes_data = pd.DataFrame({"xrsa": goes_short["flux"].values, "xrsb": goes_long["flux"].values}, index=time_array.datetime)
Now we will create a sunpy.timeseries.TimeSeries by passing in the data,
the metadata and the units.
goes_ts = ts.TimeSeries(goes_data, meta, units, source="xrs")
Finally, we can plot the timeseries.
fig, ax = plt.subplots()
goes_ts.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 1.182 seconds)
Note
Go to the end to download the full example code.
Find Peaks in sunpy TimeSeries#
This example illustrates how to find minimum or maximum peaks in a TimeSeries. Note: Peak finding is a complex problem that has many potential solutions and this example is just one method of many.
import matplotlib.pyplot as plt
import numpy as np
from sunpy.data.sample import GOES_XRS_TIMESERIES
from sunpy.timeseries import TimeSeries
We will now create a TimeSeries object from an observational data source, Also, we will truncate it to do analysis on a smaller time duration of 10 years.
goes_lc = TimeSeries(GOES_XRS_TIMESERIES)
my_timeseries = goes_lc.truncate('2011/06/07 06:10', '2011/06/07 09:00')
fig, ax = plt.subplots()
my_timeseries.plot(axes=ax)
<Axes: ylabel='Watts m$^{-2}$'>
To find extrema in any TimeSeries, we first define a function findpeaks that takes in input an iterable data series and a DELTA value. The DELTA value controls how much difference between values in the TimeSeries defines an extremum point. Inside the function, we iterate over the data values of TimeSeries and consider a point to be a local maxima if it has the maximal value, and was preceded (to the left) by a value lower by DELTA. Similar logic applies to find a local minima.
def findpeaks(series, DELTA):
"""
Finds extrema in a pandas series data.
Parameters
----------
series : `pandas.Series`
The data series from which we need to find extrema.
DELTA : `float`
The minimum difference between data values that defines a peak.
Returns
-------
minpeaks, maxpeaks : `list`
Lists consisting of pos, val pairs for both local minima points and
local maxima points.
"""
# Set initial values
mn, mx = np.Inf, -np.Inf
minpeaks = []
maxpeaks = []
lookformax = True
start = True
# Iterate over items in series
for time_pos, value in series.items():
if value > mx:
mx = value
mxpos = time_pos
if value < mn:
mn = value
mnpos = time_pos
if lookformax:
if value < mx-DELTA:
# a local maxima
maxpeaks.append((mxpos, mx))
mn = value
mnpos = time_pos
lookformax = False
elif start:
# a local minima at beginning
minpeaks.append((mnpos, mn))
mx = value
mxpos = time_pos
start = False
else:
if value > mn+DELTA:
# a local minima
minpeaks.append((mnpos, mn))
mx = value
mxpos = time_pos
lookformax = True
# check for extrema at end
if value > mn+DELTA:
maxpeaks.append((mxpos, mx))
elif value < mx-DELTA:
minpeaks.append((mnpos, mn))
return minpeaks, maxpeaks
Now we take the column ‘sunspot SWO’ of this TimeSeries and try to find it’s extrema using the function findpeaks. We take the value of DELTA to be approximately the length of smallest peak that we wish to detect.
series = my_timeseries.to_dataframe()['xrsa']
minpeaks, maxpeaks = findpeaks(series, DELTA=1e-7)
# Plotting the figure and extremum points
fig, ax = plt.subplots()
ax.set_xlabel('Time')
ax.set_ylabel("Flux (Wm$^{-2}$)")
ax.set_title('Peaks in TimeSeries')
series.plot(ax=ax)
ax.scatter(*zip(*minpeaks), color='red', label='min')
ax.scatter(*zip(*maxpeaks), color='green', label='max')
ax.legend()
ax.grid(True)
plt.show()
Total running time of the script: (0 minutes 0.774 seconds)
Note
Go to the end to download the full example code.
Flare times on a GOES XRS plot#
How to plot flare times as provided by the HEK on a GOES XRS plot.
import matplotlib.pyplot as plt
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import parse_time
from sunpy.timeseries import TimeSeries
Let’s grab GOES XRS data for a particular time of interest and the HEK flare data for this time from the NOAA Space Weather Prediction Center (SWPC). Here we are searching for data from the GOES-15 satellite and for the 1-min average time-sampled data.
tr = a.Time('2011-06-07 04:00', '2011-06-07 12:00')
results = Fido.search(tr, a.Instrument.xrs & a.goes.SatelliteNumber(15) & a.Resolution("avg1m") | a.hek.FL & (a.hek.FRM.Name == 'SWPC'))
Then download the XRS data and load it into a TimeSeries.
files = Fido.fetch(results)
goes = TimeSeries(files)
Next let’s retrieve HEKTable from the Fido result
and then load the first row from HEK results into flares_hek.
hek_results = results['hek']
flares_hek = hek_results[0]
Lets plot everything together.
fig, ax = plt.subplots()
goes.plot(axes=ax)
ax.axvline(parse_time(flares_hek['event_peaktime']).datetime)
ax.axvspan(
parse_time(flares_hek['event_starttime']).datetime,
parse_time(flares_hek['event_endtime']).datetime,
alpha=0.2, label=flares_hek['fl_goescls']
)
ax.legend(loc=2)
ax.set_yscale('log')
ax.set_xlim(tr.start.to_datetime(), tr.end.to_datetime())
plt.show()
Total running time of the script: (0 minutes 2.607 seconds)
Note
Go to the end to download the full example code.
Making a power spectrum from a TimeSeries#
How to estimate the power spectrum of a TimeSeries.
import matplotlib.pyplot as plt
from scipy import signal
import astropy.units as u
import sunpy.timeseries
from sunpy.data.sample import RHESSI_TIMESERIES
Let’s first load a RHESSI TimeSeries from sunpy’s sample data. This data contains 9 columns, which are evenly sampled with a time step of 4 seconds.
ts = sunpy.timeseries.TimeSeries(RHESSI_TIMESERIES)
We now use SciPy’s periodogram to estimate the
power spectra of the first column of the Timeseries. The first column contains
X-Ray emissions in the range of 3-6 keV. An alternative version is Astropy’s
LombScargle periodogram.
x_ray = ts.columns[0]
# The suitable value for fs would be 0.25 Hz as the time step is 4 s.
freq, spectra = signal.periodogram(ts.quantity(x_ray), fs=0.25)
Let’s plot the results.
fig, ax = plt.subplots()
ax.semilogy(freq, spectra)
ax.set_title(f'Power Spectrum of {x_ray}')
ax.set_ylabel(f'Power Spectral Density [{ts.units[x_ray] ** 2 / u.Hz:LaTeX}]')
ax.set_xlabel('Frequency [Hz]')
plt.show()
Total running time of the script: (0 minutes 0.365 seconds)
Note
Go to the end to download the full example code.
Retrieving and analyzing GOES X-Ray Sensor (XRS) data#
The X-ray Sensor (XRS) on board the GOES series of satellites have provided soft X-ray measurements in two broadband energy ranges 0.5-4 and 1-8 angstrom since 1975. The GOES 16 and 17 satellites are the latest in line. The flux levels in the GOES 1-8 angstrom channel are used to report flares and determine their size (i.e. their GOES class).
In this example we are going to look at how you can query and
retrieve the GOES XRS data using Fido and load it
into a TimeSeries.
Some things to note: NOAA have recently re-processed the GOES 13, 14 and 15 XRS science quality data, such that the SWPC scaling factor has been removed. This means that the fluxes will have a different values, and so will flare peak fluxes from previous 13, 14 and 15 XRS data. See here for more details. The sunpy GOES XRS client for Fido now provides this new re-processed data. We now also provide the data for GOES 16 and 17.
Another thing to note is that the GOES XRS client Fido now
returns all available GOES data for the specific timerange queried. For
example, there are times when GOES 13, 14 and 15 overlap and such data is
available from each satellite. Similarly there are times when GOES 16 and 17 overlap.
import matplotlib.pyplot as plt
import numpy as np
from astropy.visualization import time_support
from sunpy import timeseries as ts
from sunpy.net import Fido
from sunpy.net import attrs as a
Lets first define our start and end times and query using the
Fido.
tstart = "2015-06-21 01:00"
tend = "2015-06-21 23:00"
result = Fido.search(a.Time(tstart, tend), a.Instrument("XRS"))
print(result)
Results from 1 Provider:
6 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA flx1s
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA avg1m
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA flx1s
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA avg1m
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA flx1s
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA avg1m
As we can see this now returns six results, two files for each GOES
13, one for GOES 14 and one for GOES 15, which can be identified
by the SatelliteNumber column.
The GOES data provided can either be high-cadence (1s/2s/3s based on GOES satellite) or averaged over 1 minute.
This can be noted in the Resolution column, where the avg1m and flx1s attributes are the 1 minute average and the high-cadence data, respectively.
However, we probably will only want one of these files for our analysis, so we can query by the sunpy.net.attrs:
sunpy.net.dataretriever.attrs.goes.SatelliteNumber to specify what GOES satellite number we want
to use, and sunpy.net.attrs.Resolution for the resolution. Here we will use the high-cadence observations.
result_goes15 = Fido.search(a.Time(tstart, tend), a.Instrument("XRS"), a.goes.SatelliteNumber(15), a.Resolution("flx1s"))
print(result_goes15)
Results from 1 Provider:
1 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2015-06-21 00:00:00.000 2015-06-21 23:59:59.999 ... NOAA flx1s
Now we can see that this returns just one file for the GOES 15 data.
Lets now download this data using fetch.
file_goes15 = Fido.fetch(result_goes15)
Lets now load this data into a TimeSeries,
and inspect the data using peek().
goes_15 = ts.TimeSeries(file_goes15)
goes_15.peek()
The resulting TimeSeries can be filtered by GOES quality flags. For more information
refer to the GOES Data Guide.
df = goes_15.to_dataframe()
df = df[(df["xrsa_quality"] == 0) & (df["xrsb_quality"] == 0)]
goes_15 = ts.TimeSeries(df, goes_15.meta, goes_15.units)
We can also pull out the individual GOES channels and plot. The 0.5-4 angstrom channel is known as the “xrsa” channel and the 1-8 angstrom channel is known as the “xrsb” channel.
fig, ax = plt.subplots()
goes_15.plot(axes=ax, columns=["xrsb"])
plt.show()
We can also truncate the data for the time of the large flare, and analyze the different channels. For example, we can plot the derivative which is useful in terms of the Neupert effect when analyzing flares.
goes_flare = goes_15.truncate("2015-06-21 09:35", "2015-06-21 10:30")
time_support()
fig, ax = plt.subplots()
ax.plot(goes_flare.time, np.gradient(goes_flare.quantity("xrsb")))
ax.set_ylabel("Flux (Wm$^{-2}$$s^{-1}$)")
fig.autofmt_xdate()
plt.show()
GOES 16 and 17 data#
Since March 2020, data prior to GOES 15 (incl) is no longer supported by NOAA and GOES 16 and 17 data is now provided. See here for more details. GOES 16 and 17 are part of the GOES-R series and provide XRS data at a better time resolution (1s). sunpy now supports this data also. GOES 16 has been taking observations from 2017, and GOES 17 since 2018, both of which are now and its now available through sunpy.net.Fido.
Lets query for some data over two days.
results = Fido.search(a.Time("2020-11-20 00:00", "2020-11-21 23:00"), a.Instrument("XRS"), a.Resolution("flx1s"))
print(results)
Results from 1 Provider:
4 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2020-11-20 00:00:00.000 2020-11-20 23:59:59.999 ... NOAA flx1s
2020-11-21 00:00:00.000 2020-11-21 23:59:59.999 ... NOAA flx1s
2020-11-20 00:00:00.000 2020-11-20 23:59:59.999 ... NOAA flx1s
2020-11-21 00:00:00.000 2020-11-21 23:59:59.999 ... NOAA flx1s
We can see that we are provided with 4 results, two files for GOES 16 and two for GOES 17. Again we can make the query only specifying one GOES satellite number.
results_16 = Fido.search(a.Time("2020-11-20 00:00", "2020-11-21 23:00"), a.Instrument("XRS"),
a.goes.SatelliteNumber(16))
print(results_16)
Results from 1 Provider:
4 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2020-11-20 00:00:00.000 2020-11-20 23:59:59.999 ... NOAA flx1s
2020-11-21 00:00:00.000 2020-11-21 23:59:59.999 ... NOAA flx1s
2020-11-20 00:00:00.000 2020-11-20 23:59:59.999 ... NOAA avg1m
2020-11-21 00:00:00.000 2020-11-21 23:59:59.999 ... NOAA avg1m
Lets now download this data and load into a
TimeSeries.
files = Fido.fetch(results_16)
# We use the `concatenate=True` keyword argument in TimeSeries, as
# we have two files and want to create one timeseries from them.
goes_16 = ts.TimeSeries(files, concatenate=True)
goes_16.peek()
Total running time of the script: (0 minutes 17.191 seconds)
Note
Go to the end to download the full example code.
Smoothing of timeSeries data using convolution filters#
How to smooth a TimeSeries using a convolution filter
kernel from convolution and convolve
function.
import matplotlib.pyplot as plt
from astropy.convolution import Box1DKernel, convolve
from sunpy.data.sample import GOES_XRS_TIMESERIES
from sunpy.timeseries import TimeSeries
Let’s first create a TimeSeries from sample data.
goes_lc = TimeSeries(GOES_XRS_TIMESERIES).truncate('2011/06/07 06:10', '2011/06/07 07:00')
Now we will extract data values from the TimeSeries and apply a BoxCar filter to get smooth data. Boxcar smoothing is equivalent to taking our signal and using it to make a new signal where each element is the average of w adjacent elements. Here we will use astropy’s convolve function with a “boxcar” kernel of width w = 10.
goes_lc = goes_lc.add_column(
'xrsa_smoothed',
convolve(goes_lc.quantity('xrsa'), kernel=Box1DKernel(50))
)
Plotting original and smoothed timeseries.
fig, ax = plt.subplots()
ax.set_xlabel('Time')
ax.set_ylabel("Flux (Wm$^{-2}$")
ax.set_title('Smoothing of Time Series')
ax.plot(goes_lc.quantity('xrsa'), label='original')
ax.plot(goes_lc.quantity('xrsa_smoothed'), label='smoothed')
ax.legend()
plt.show()
Total running time of the script: (0 minutes 0.506 seconds)
Note
Go to the end to download the full example code.
The TimeSeriesMetaData class#
This provides an overview of the basic functionality of the TimeSeriesMetaData class.
import astropy.units as u
import sunpy.data.sample
import sunpy.timeseries
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import TimeRange, parse_time
Search for timeseries data.
goes_res = Fido.search(a.Time("2010-11-03", "2010-11-03 23:59:59"), a.Instrument.xrs,
a.goes.SatelliteNumber(15))
norh_res = Fido.search(a.Time("2010-11-03", "2010-11-03 23:59:59"), a.Instrument.norh,
a.Wavelength(17 * u.GHz))
Download the data and load it.
goes_files = Fido.fetch(goes_res)
norh_files = Fido.fetch(norh_res)
goes_ts = sunpy.timeseries.TimeSeries(goes_files, source='XRS', concatenate=True)
norh_ts = sunpy.timeseries.TimeSeries(norh_files, source='NoRH', concatenate=True)
Combining the two series.
large_ts = goes_ts.concatenate(norh_ts)
The metadata can be easily viewed:
print(large_ts.meta)
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2010-11-02T22:44:51.801000 | Correlation Coe | simple: True |
| to | | bitpix: 16 |
|2010-11-03T06:29:58.801000 | | naxis: 1 |
| | | naxis1: 27908 |
| | | bscale: 3.053e-07 |
| | | bzero: 0.0 |
| | | bunit: CORRELATION COEFF. |
| | | blank: -32768 |
| | | crval1: 22:44:51.801 |
| | | startfrm: 1 |
| | | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T00:00:00.000000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
| to | xrsb | title: GOES 1-15 L2 XRS 1-minute Irradiance averag|
|2010-11-03T23:59:00.000000 | xrsa_quality | summary: The X-ray flux product consists of reproc|
| | xrsb_quality | keywords: b'' |
| | | keywords_vocabulary: SPASE: Space Physics Archive |
| | | naming_authority: gov.nesdis.noaa |
| | | history: See algorithm information. |
| | | source: GOES XRS high-resolution irradiances |
| | | processing_level: Level 2 |
| | | processing_level_description: Derived products |
| | | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T00:00:00.851000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
| to | xrsb | title: GOES 1-15 L2 XRS high-resolution Irradiance|
|2010-11-03T23:59:58.881000 | xrsa_quality | id: b' ' |
| | xrsb_quality | summary: The X-ray flux product consists of reproc|
| | | keywords: NumericalData.MeasurementType.Irradiance|
| | | keywords_vocabulary: SPASE: Space Physics Archive |
| | | naming_authority: gov.nesdis.noaa |
| | | history: See algorithm information. |
| | | source: GOES XRS counts |
| | | processing_level: Level 2 |
| | | ... |
|-------------------------------------------------------------------------------------------------|
You can reduce the depth of the view:
print(large_ts.meta.to_string(2))
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2010-11-02T22:44:51.801000 | Correlation Coe | simple: True |
|2010-11-03T06:29:58.801000 | | bitpix: 16 |
| | | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T00:00:00.000000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:00.000000 | xrsb | title: GOES 1-15 L2 XRS 1-minute Irradiance averag|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T00:00:00.851000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:58.881000 | xrsb | title: GOES 1-15 L2 XRS high-resolution Irradiance|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
The TimeSeriesMetaData class stores all the individual file metadata MetaDict objects as 3-tuple entries in it’s internal list with the TimeRange, list of column names and metadictionary. This list is stored in order of ascending TR.start. Access of the the list is done using the metadata property:
print(large_ts.meta.metadata)
[( <sunpy.time.timerange.TimeRange object at 0x7f6f42457be0>
Start: 2010-11-02 22:44:51
End: 2010-11-03 06:29:58
Center:2010-11-03 02:37:25
Duration:0.3229976851851851 days or
7.751944444444443 hours or
465.11666666666656 minutes or
27906.999999999996 seconds
, ['Correlation Coefficient'], MetaDict([('simple': 'True')
('bitpix': '16')
('naxis': '1')
('naxis1': '27908')
('bscale': '3.053e-07')
('bzero': '0.0')
('bunit': 'CORRELATION COEFF.')
('blank': '-32768')
('crval1': '22:44:51.801')
('startfrm': '1')
('crpix1': '1.0')
('ctype1': 'TIME(SECOND)')
('cdelt1': '1.0')
('fund-amp': '11227')
('object': 'SUN')
('image1': 'R+L')
('obs-freq': '17GHZ')
('telescop': 'RADIOHELIOGRAPH')
('date-obs': '2010-11-02')
('jstdate': '2010-11-03')
('jsttime': '07:44:51.801')
('origin': 'NOBEYAMA RADIO OBS')
('date': '****-**-**')
('pversion': '6.00')
('comment': 'BUNIT IS THE MEAN OF CORRELATION COEFFICIENTS OF THE ANTENNA BASELINE
LENGTH ABOVE 100*D*COS(THETA).
THETA: ANGLE BETWEEN THE ANTENNA BASELINE & THE DIRECTION OF THE SUN.
D : FUNDAMENTAL BASELINE LENGTH OF 1.528 METERS
SFACT: FACTOR INTRODUCED TO REDUCE THE DISCRETIZATION ERROR')
('history': '')
('keycomments': '{'SIMPLE': 'BASIC FITS TAPE FORM', 'BITPIX': "2-BYTE TWO'S COMPLEMENT INTEGER", 'NAXIS': '1-DIMENSIONAL TEMPORAL PROF', 'NAXIS1': '# PIXEL/ROW', 'BSCALE': 'SFACT/32768, REAL = TAPE * BSCALE + BZERO', 'BZERO': 'NO BITS ADDED', 'BUNIT': 'CROSS CORRELATION COEFFICIENT', 'BLANK': 'VALUE FOR VALUE-UNDEFINED PIXEL', 'CRVAL1': 'REF POINT VALUE IN HH:MM:SS (UT)', 'STARTFRM': 'REF FRAME NUMBER', 'CRPIX1': 'REF POINT PIXEL LOCATION', 'CTYPE1': 'TYPE OF PHYSICAL COORD. ON AXIS1', 'CDELT1': 'PIXEL SIZE ON AXIS1 IN SECOND', 'FUND-AMP': 'COR_AMP. OF FUND. ANTENNA SPACING AT STARTFRAM', 'OBJECT': 'OBJECT NAME', 'OBS-FREQ': 'INTENSITY AT 17GHZ', 'TELESCOP': 'NOBEYAMA RADIO HELIOGRAPH', 'DATE-OBS': 'DATE OF DATA ACQUISITION YYYY-MM-DD (UT)', 'JSTDATE': 'DATE OF DATA ACQUISITION YYYY-MM-DD (JST)', 'JSTTIME': 'TIME OF DATA ACQUISITION HH/MM/SS (JST)', 'ORIGIN': 'TAPE WRITE INSTITUTION', 'DATE': 'DATE WHEN DATA FILE WRITTEN YYYY-MM-DD (JST)', 'PVERSION': 'CODED BY N.SHINOHARA 14.MAR.95'}')])), ( <sunpy.time.timerange.TimeRange object at 0x7f6f42455570>
Start: 2010-11-03 00:00:00
End: 2010-11-03 23:59:00
Center:2010-11-03 11:59:30
Duration:0.9993055555555556 days or
23.983333333333334 hours or
1439.0 minutes or
86340.0 seconds
, ['xrsa', 'xrsb', 'xrsa_quality', 'xrsb_quality'], MetaDict([('conventions': 'ACDD-1.3, Spase v2.2.6')
('title': 'GOES 1-15 L2 XRS 1-minute Irradiance averages')
('summary': 'The X-ray flux product consists of reprocessed science-quality high-resolution X-ray solar measurements. The GOES X-Ray Sensor (XRS) measures in a short wavelength channel (XRS-A) with a nominal bandpass of 0.05 to 0.4 nm and in a longer wavelength channel (XRS-B) with a bandpass of 0.1 to 0.8 nm.
For the given bandpass, the XRS response is based on a flat spectrum beginning with GOES-3. A true solar spectrum is assumed for GOES 1-2.
GOES 13-15 measurements are made at a 2-s cadence.
GOES 1-12 measured made at a 3-s cadence.
The data products for both are 1-minute averages of these high-resolution data.
For GOES 3-12, the XRS-A response was originally determined assuming a bandpass of 0.05 to 0.3 nm.')
('keywords': 'b''')
('keywords_vocabulary': 'SPASE: Space Physics Archive Search and Extract Data Model version 2.2.6, GCMD: NASA Global Change Master Directory (GCMD) Earth Science Keywords version 8.5')
('naming_authority': 'gov.nesdis.noaa')
('history': 'See algorithm information.')
('source': 'GOES XRS high-resolution irradiances')
('processing_level': 'Level 2')
('processing_level_description': 'Derived products')
('license': 'These data may be redistributed and used without restriction. ')
('acknowledgment': 'TBD')
('metadata_link': 'b''')
('creator_name': 'Erika Zetterlund, Janet Machol')
('creator_type': 'person')
('creator_institution': 'DOC/NOAA/NCEI/CCOG/STP')
('creator_email': 'goesr.exis@noaa.gov')
('creator_url': 'https://www.ncei.noaa.gov/')
('institution': 'DOC/NOAA/NESDIS> U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services')
('publisher_name': 'National Centers for Environmental Information')
('publisher_type': 'institution')
('publisher_institution': 'DOC/NOAA/NESDIS/NCEI')
('publisher_email': 'goesr.exis@noaa.gov')
('publisher_url': 'https://www.ncei.noaa.gov/')
('references': 'README FOR SCIENCE QUALITY GOES 13-15 XRS DATA')
('instrument': 'GOES 1-15 X-ray Irradiance Sensor (XRS)')
('program': 'Geostationary Operational Environmental Satellite (GOES)')
('project': 'Geostationary Operational Environmental Satellite (GOES) Solar Terrestrial Physics at NCEI')
('time_coverage_resolution': 'PT1M')
('processing_parameters_file': 'b' '')
('algorithm_parameters': 'None')
('id': 'sci_xrsf-l2-avg1m_g15_d20101103_v1-0-0.nc')
('date_created': '2020-05-15T19:06:17.835Z')
('instrument_id': 'b' '')
('orbital_slot': 'b' '')
('platform': 'g15')
('l1b_system_environment': 'b' '')
('l1b_production_site': 'b' '')
('time_coverage_start': '2010-11-03T00:00:00.000Z')
('time_coverage_end': '2010-11-04T00:00:00.000Z')
('algorithm': 'cli-gxrs_avg')
('algorithm_version': '[1 0]')
('algorithm_date': '2019-12-16')
('input_system_environments': 'sci')
('input_files_first': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')
('input_files_last': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')
('input_files_total': '1')])), ( <sunpy.time.timerange.TimeRange object at 0x7f6f42454670>
Start: 2010-11-03 00:00:00
End: 2010-11-03 23:59:58
Center:2010-11-03 11:59:59
Duration:0.999977199074074 days or
23.999452777777776 hours or
1439.9671666666666 minutes or
86398.03 seconds
, ['xrsa', 'xrsb', 'xrsa_quality', 'xrsb_quality'], MetaDict([('conventions': 'ACDD-1.3, Spase v2.2.6')
('title': 'GOES 1-15 L2 XRS high-resolution Irradiances')
('id': 'b' '')
('summary': 'The X-ray flux product consists of reprocessed science-quality high-resolution X-ray solar measurements. The GOES X-Ray Sensor (XRS) measures in a short wavelength channel (XRS-A) with a nominal bandpass of 0.05 to 0.4 nm and in a longer wavelength channel (XRS-B) with a bandpass of 0.1 to 0.8 nm.
For the given bandpass, the XRS response is based on a flat spectrum beginning with GOES-3. A true solar spectrum is assumed for GOES 1-2.
GOES 13-15 measurements are at 2-s cadence.
GOES 1-12 measured at a 3-s cadence.
For GOES 3-12, the XRS-A response was originally determined assuming a bandpass of 0.05 to 0.3 nm.')
('keywords': 'NumericalData.MeasurementType.Irradiance')
('keywords_vocabulary': 'SPASE: Space Physics Archive Search and Extract Data Model version 2.2.6, GCMD: NASA Global Change Master Directory (GCMD) Earth Science Keywords version 8.5')
('naming_authority': 'gov.nesdis.noaa')
('history': 'See algorithm information.')
('source': 'GOES XRS counts')
('processing_level': 'Level 2')
('processing_level_description': 'Derived products')
('license': 'These data may be redistributed and used without restriction. ')
('acknowledgment': 'TBD')
('metadata_link': 'b''')
('date_created': 'b' '')
('creator_name': 'Erika Zetterlund, Janet Machol')
('creator_type': 'person')
('creator_institution': 'DOC/NOAA/NCEI/CCOG/STP')
('creator_email': 'goesr.exis@noaa.gov')
('creator_url': 'https://www.ncei.noaa.gov/')
('institution': 'DOC/NOAA/NESDIS> U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services')
('publisher_name': 'National Centers for Environmental Information')
('publisher_type': 'institution')
('publisher_institution': 'DOC/NOAA/NESDIS/NCEI')
('publisher_email': 'goesr.exis@noaa.gov')
('publisher_url': 'https://www.ncei.noaa.gov/')
('references': 'README FOR SCIENCE QUALITY GOES 13-15 XRS DATA')
('instrument': 'GOES 1-15 X-ray Irradiance Sensor (XRS)')
('instrument_id': 'b' '')
('orbital_slot': 'b' '')
('program': 'Geostationary Operational Environmental Satellite (GOES)')
('project': 'Geostationary Operational Environmental Satellite (GOES) Solar Terrestrial Physics at NCEI')
('platform': 'b' '')
('l1b_system_environment': 'b' '')
('l1b_production_site': 'b' '')
('time_coverage_start': 'b' '')
('time_coverage_end': 'b' '')
('time_coverage_resolution': 'PT2S')
('processing_parameters_file': 'b' '')
('algorithm': 'b''')
('algorithm_version': 'b' '')
('algorithm_date': 'b' '')
('algorithm_parameters': 'None')
('input_system_environments': 'b' '')
('input_files_first': 'b' '')
('input_files_last': 'b' '')
('input_files_total': 'b' '')
('filename_id': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')]))]
The TimeSeriesMetaData class has a number of other properties, including the timerange property that returns a TimeRange for the entire metadata:
print(large_ts.meta.time_range)
<sunpy.time.timerange.TimeRange object at 0x7f6f42454640>
Start: 2010-11-02 22:44:51
End: 2010-11-03 23:59:58
Center:2010-11-03 11:22:25
Duration:1.0521652777777777 days or
25.251966666666664 hours or
1515.118 minutes or
90907.07999999999 seconds
Further properties can be used to get lists of details, e.g., List of the time ranges
print(large_ts.meta.timeranges)
[ <sunpy.time.timerange.TimeRange object at 0x7f6f42457be0>
Start: 2010-11-02 22:44:51
End: 2010-11-03 06:29:58
Center:2010-11-03 02:37:25
Duration:0.3229976851851851 days or
7.751944444444443 hours or
465.11666666666656 minutes or
27906.999999999996 seconds
, <sunpy.time.timerange.TimeRange object at 0x7f6f42455570>
Start: 2010-11-03 00:00:00
End: 2010-11-03 23:59:00
Center:2010-11-03 11:59:30
Duration:0.9993055555555556 days or
23.983333333333334 hours or
1439.0 minutes or
86340.0 seconds
, <sunpy.time.timerange.TimeRange object at 0x7f6f42454670>
Start: 2010-11-03 00:00:00
End: 2010-11-03 23:59:58
Center:2010-11-03 11:59:59
Duration:0.999977199074074 days or
23.999452777777776 hours or
1439.9671666666666 minutes or
86398.03 seconds
]
List of the column names
print(large_ts.meta.columns)
['Correlation Coefficient', 'xrsa', 'xrsa_quality', 'xrsb', 'xrsb_quality']
List of the meta dictionaries
print(large_ts.meta.metas)
[MetaDict([('simple': 'True')
('bitpix': '16')
('naxis': '1')
('naxis1': '27908')
('bscale': '3.053e-07')
('bzero': '0.0')
('bunit': 'CORRELATION COEFF.')
('blank': '-32768')
('crval1': '22:44:51.801')
('startfrm': '1')
('crpix1': '1.0')
('ctype1': 'TIME(SECOND)')
('cdelt1': '1.0')
('fund-amp': '11227')
('object': 'SUN')
('image1': 'R+L')
('obs-freq': '17GHZ')
('telescop': 'RADIOHELIOGRAPH')
('date-obs': '2010-11-02')
('jstdate': '2010-11-03')
('jsttime': '07:44:51.801')
('origin': 'NOBEYAMA RADIO OBS')
('date': '****-**-**')
('pversion': '6.00')
('comment': 'BUNIT IS THE MEAN OF CORRELATION COEFFICIENTS OF THE ANTENNA BASELINE
LENGTH ABOVE 100*D*COS(THETA).
THETA: ANGLE BETWEEN THE ANTENNA BASELINE & THE DIRECTION OF THE SUN.
D : FUNDAMENTAL BASELINE LENGTH OF 1.528 METERS
SFACT: FACTOR INTRODUCED TO REDUCE THE DISCRETIZATION ERROR')
('history': '')
('keycomments': '{'SIMPLE': 'BASIC FITS TAPE FORM', 'BITPIX': "2-BYTE TWO'S COMPLEMENT INTEGER", 'NAXIS': '1-DIMENSIONAL TEMPORAL PROF', 'NAXIS1': '# PIXEL/ROW', 'BSCALE': 'SFACT/32768, REAL = TAPE * BSCALE + BZERO', 'BZERO': 'NO BITS ADDED', 'BUNIT': 'CROSS CORRELATION COEFFICIENT', 'BLANK': 'VALUE FOR VALUE-UNDEFINED PIXEL', 'CRVAL1': 'REF POINT VALUE IN HH:MM:SS (UT)', 'STARTFRM': 'REF FRAME NUMBER', 'CRPIX1': 'REF POINT PIXEL LOCATION', 'CTYPE1': 'TYPE OF PHYSICAL COORD. ON AXIS1', 'CDELT1': 'PIXEL SIZE ON AXIS1 IN SECOND', 'FUND-AMP': 'COR_AMP. OF FUND. ANTENNA SPACING AT STARTFRAM', 'OBJECT': 'OBJECT NAME', 'OBS-FREQ': 'INTENSITY AT 17GHZ', 'TELESCOP': 'NOBEYAMA RADIO HELIOGRAPH', 'DATE-OBS': 'DATE OF DATA ACQUISITION YYYY-MM-DD (UT)', 'JSTDATE': 'DATE OF DATA ACQUISITION YYYY-MM-DD (JST)', 'JSTTIME': 'TIME OF DATA ACQUISITION HH/MM/SS (JST)', 'ORIGIN': 'TAPE WRITE INSTITUTION', 'DATE': 'DATE WHEN DATA FILE WRITTEN YYYY-MM-DD (JST)', 'PVERSION': 'CODED BY N.SHINOHARA 14.MAR.95'}')]), MetaDict([('conventions': 'ACDD-1.3, Spase v2.2.6')
('title': 'GOES 1-15 L2 XRS 1-minute Irradiance averages')
('summary': 'The X-ray flux product consists of reprocessed science-quality high-resolution X-ray solar measurements. The GOES X-Ray Sensor (XRS) measures in a short wavelength channel (XRS-A) with a nominal bandpass of 0.05 to 0.4 nm and in a longer wavelength channel (XRS-B) with a bandpass of 0.1 to 0.8 nm.
For the given bandpass, the XRS response is based on a flat spectrum beginning with GOES-3. A true solar spectrum is assumed for GOES 1-2.
GOES 13-15 measurements are made at a 2-s cadence.
GOES 1-12 measured made at a 3-s cadence.
The data products for both are 1-minute averages of these high-resolution data.
For GOES 3-12, the XRS-A response was originally determined assuming a bandpass of 0.05 to 0.3 nm.')
('keywords': 'b''')
('keywords_vocabulary': 'SPASE: Space Physics Archive Search and Extract Data Model version 2.2.6, GCMD: NASA Global Change Master Directory (GCMD) Earth Science Keywords version 8.5')
('naming_authority': 'gov.nesdis.noaa')
('history': 'See algorithm information.')
('source': 'GOES XRS high-resolution irradiances')
('processing_level': 'Level 2')
('processing_level_description': 'Derived products')
('license': 'These data may be redistributed and used without restriction. ')
('acknowledgment': 'TBD')
('metadata_link': 'b''')
('creator_name': 'Erika Zetterlund, Janet Machol')
('creator_type': 'person')
('creator_institution': 'DOC/NOAA/NCEI/CCOG/STP')
('creator_email': 'goesr.exis@noaa.gov')
('creator_url': 'https://www.ncei.noaa.gov/')
('institution': 'DOC/NOAA/NESDIS> U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services')
('publisher_name': 'National Centers for Environmental Information')
('publisher_type': 'institution')
('publisher_institution': 'DOC/NOAA/NESDIS/NCEI')
('publisher_email': 'goesr.exis@noaa.gov')
('publisher_url': 'https://www.ncei.noaa.gov/')
('references': 'README FOR SCIENCE QUALITY GOES 13-15 XRS DATA')
('instrument': 'GOES 1-15 X-ray Irradiance Sensor (XRS)')
('program': 'Geostationary Operational Environmental Satellite (GOES)')
('project': 'Geostationary Operational Environmental Satellite (GOES) Solar Terrestrial Physics at NCEI')
('time_coverage_resolution': 'PT1M')
('processing_parameters_file': 'b' '')
('algorithm_parameters': 'None')
('id': 'sci_xrsf-l2-avg1m_g15_d20101103_v1-0-0.nc')
('date_created': '2020-05-15T19:06:17.835Z')
('instrument_id': 'b' '')
('orbital_slot': 'b' '')
('platform': 'g15')
('l1b_system_environment': 'b' '')
('l1b_production_site': 'b' '')
('time_coverage_start': '2010-11-03T00:00:00.000Z')
('time_coverage_end': '2010-11-04T00:00:00.000Z')
('algorithm': 'cli-gxrs_avg')
('algorithm_version': '[1 0]')
('algorithm_date': '2019-12-16')
('input_system_environments': 'sci')
('input_files_first': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')
('input_files_last': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')
('input_files_total': '1')]), MetaDict([('conventions': 'ACDD-1.3, Spase v2.2.6')
('title': 'GOES 1-15 L2 XRS high-resolution Irradiances')
('id': 'b' '')
('summary': 'The X-ray flux product consists of reprocessed science-quality high-resolution X-ray solar measurements. The GOES X-Ray Sensor (XRS) measures in a short wavelength channel (XRS-A) with a nominal bandpass of 0.05 to 0.4 nm and in a longer wavelength channel (XRS-B) with a bandpass of 0.1 to 0.8 nm.
For the given bandpass, the XRS response is based on a flat spectrum beginning with GOES-3. A true solar spectrum is assumed for GOES 1-2.
GOES 13-15 measurements are at 2-s cadence.
GOES 1-12 measured at a 3-s cadence.
For GOES 3-12, the XRS-A response was originally determined assuming a bandpass of 0.05 to 0.3 nm.')
('keywords': 'NumericalData.MeasurementType.Irradiance')
('keywords_vocabulary': 'SPASE: Space Physics Archive Search and Extract Data Model version 2.2.6, GCMD: NASA Global Change Master Directory (GCMD) Earth Science Keywords version 8.5')
('naming_authority': 'gov.nesdis.noaa')
('history': 'See algorithm information.')
('source': 'GOES XRS counts')
('processing_level': 'Level 2')
('processing_level_description': 'Derived products')
('license': 'These data may be redistributed and used without restriction. ')
('acknowledgment': 'TBD')
('metadata_link': 'b''')
('date_created': 'b' '')
('creator_name': 'Erika Zetterlund, Janet Machol')
('creator_type': 'person')
('creator_institution': 'DOC/NOAA/NCEI/CCOG/STP')
('creator_email': 'goesr.exis@noaa.gov')
('creator_url': 'https://www.ncei.noaa.gov/')
('institution': 'DOC/NOAA/NESDIS> U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Environmental Satellite, Data, and Information Services')
('publisher_name': 'National Centers for Environmental Information')
('publisher_type': 'institution')
('publisher_institution': 'DOC/NOAA/NESDIS/NCEI')
('publisher_email': 'goesr.exis@noaa.gov')
('publisher_url': 'https://www.ncei.noaa.gov/')
('references': 'README FOR SCIENCE QUALITY GOES 13-15 XRS DATA')
('instrument': 'GOES 1-15 X-ray Irradiance Sensor (XRS)')
('instrument_id': 'b' '')
('orbital_slot': 'b' '')
('program': 'Geostationary Operational Environmental Satellite (GOES)')
('project': 'Geostationary Operational Environmental Satellite (GOES) Solar Terrestrial Physics at NCEI')
('platform': 'b' '')
('l1b_system_environment': 'b' '')
('l1b_production_site': 'b' '')
('time_coverage_start': 'b' '')
('time_coverage_end': 'b' '')
('time_coverage_resolution': 'PT2S')
('processing_parameters_file': 'b' '')
('algorithm': 'b''')
('algorithm_version': 'b' '')
('algorithm_date': 'b' '')
('algorithm_parameters': 'None')
('input_system_environments': 'b' '')
('input_files_first': 'b' '')
('input_files_last': 'b' '')
('input_files_total': 'b' '')
('filename_id': 'sci_gxrs-l2-irrad_g15_d20101103_v0-0-0.nc')])]
When you truncate the TimeSeries, the metadata is truncated too:
large_trunc_ts = large_ts.truncate(TimeRange('2010-11-03 13:59:57.468999',
'2010-11-04 13:59:56.091999'))
print(large_trunc_ts.meta.to_string(2))
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:00.000000 | xrsb | title: GOES 1-15 L2 XRS 1-minute Irradiance averag|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:58.881000 | xrsb | title: GOES 1-15 L2 XRS high-resolution Irradiance|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
Finding metadata can be achieved using the find method and applying filters for time and/or colname. This returns another TimeSeriesMetaData object:
large_trunc_ts.meta.find(time=parse_time('2010-11-04 09:01:16'))
large_trunc_ts.meta.find(time='2010-11-04 09:01:16', colname='xrsb')
# You can get the time of a row a from the TimeSeries object's times:
large_trunc_ts.meta.find(time=large_trunc_ts.time[10])
# There is also a get method:
large_trunc_ts.meta.get('telescop')
# Again, filters can be used:
large_trunc_ts.meta.get('telescop', time='2010-11-04 09:01:16', colname='xrsb')
# And if we just want the values, the values method returns just a list:
large_trunc_ts.meta.get('telescop').values()
[]
You can update values similar to dictionaries, though all of the contained MetaDict objects will be updated that match your filter criteria:
large_trunc_ts.meta.update({'new_key_1': 'added to all.'})
large_trunc_ts.meta.update({'new_key_2': 'added to some.'}, colname='xrsa')
print(large_trunc_ts.meta.to_string(2))
# but you can't overwrite previous entries without setting the overwrite kwarg,
# this is to protect the integrity of the metadata:
large_trunc_ts.meta.update({'new_key_1': 'changed'}, overwrite=True)
print(large_trunc_ts.meta.to_string(2))
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:00.000000 | xrsb | title: GOES 1-15 L2 XRS 1-minute Irradiance averag|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:58.881000 | xrsb | title: GOES 1-15 L2 XRS high-resolution Irradiance|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:00.000000 | xrsb | title: GOES 1-15 L2 XRS 1-minute Irradiance averag|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
|2010-11-03T13:59:59.421000 | xrsa | conventions: ACDD-1.3, Spase v2.2.6 |
|2010-11-03T23:59:58.881000 | xrsb | title: GOES 1-15 L2 XRS high-resolution Irradiance|
| | ... | ... |
|-------------------------------------------------------------------------------------------------|
Total running time of the script: (0 minutes 10.909 seconds)
Note
Go to the end to download the full example code.
Using TimeSeries#
This example is intended to demonstrate the current state of TimeSeries.
import datetime
from collections import OrderedDict
import matplotlib.pyplot as plt
import numpy as np
from pandas import DataFrame
import astropy.units as u
from astropy.time import Time, TimeDelta
import sunpy.data.sample
import sunpy.timeseries
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import TimeRange, parse_time
from sunpy.util.metadata import MetaDict
We can create a range of supported timeseries:
There is a source keyword that allows one to specify the source of the data
It should be auto-detected in most cases.
ts_eve = sunpy.timeseries.TimeSeries(sunpy.data.sample.EVE_TIMESERIES, source='EVE')
ts_goes = sunpy.timeseries.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES, source='XRS')
ts_lyra = sunpy.timeseries.TimeSeries(sunpy.data.sample.LYRA_LEVEL3_TIMESERIES, source='LYRA')
ts_norh = sunpy.timeseries.TimeSeries(sunpy.data.sample.NORH_TIMESERIES, source='NoRH')
ts_rhessi = sunpy.timeseries.TimeSeries(sunpy.data.sample.RHESSI_TIMESERIES, source='RHESSI')
ts_gbm = sunpy.timeseries.TimeSeries(sunpy.data.sample.GBM_TIMESERIES, source='GBMSummary')
You can create a list of timeseries when using multiple files.
First, we shall download these files using Fido.
goes = Fido.search(a.Time("2012/06/01", "2012/06/04"), a.Instrument.xrs)
goes_files = Fido.fetch(goes)
# Using these new files you get a list of timeseries
list_of_goes_ts = sunpy.timeseries.TimeSeries(goes_files, source='XRS')
You can concatenate them together using concatenate
or when creating the TimeSeries
combined_goes_ts = sunpy.timeseries.TimeSeries(goes_files, source='XRS', concatenate=True)
# Manually
combined_goes_ts = list_of_goes_ts[0].concatenate(list_of_goes_ts[1])
fig, ax = plt.subplots()
combined_goes_ts.plot(axes=ax)
plt.show()
The TimeSeries object has 3 primary components:
.data : The internal data representation. If you want the underlying data,
use to_dataframe()
.meta : Stores the metadata that is able to be parsed from the data files
.units : Stores the units for each column, with keys that match the name of each column.
# This will give you a useful dataframe to manipulate the data with.
lyra_data = ts_lyra.to_dataframe()
lyra_data
This will give you the metadata
ts_lyra.meta
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2011-06-07T00:00:00.010000 | CHANNEL1 | simple: True |
| to | CHANNEL2 | bitpix: 8 |
|2011-06-07T23:59:00.010000 | CHANNEL3 | naxis: 0 |
| | CHANNEL4 | extend: True |
| | | origin: ROB |
| | | telescop: PROBA2 |
| | | instrume: LYRA |
| | | object: EUV solar irrad |
| | | obs_mode: standard |
| | | date: 2015-12-16 |
| | | ... |
|-------------------------------------------------------------------------------------------------|
This will give you the units
ts_lyra.units
OrderedDict([('CHANNEL1', Unit("W / m2")), ('CHANNEL2', Unit("W / m2")), ('CHANNEL3', Unit("W / m2")), ('CHANNEL4', Unit("W / m2"))])
There are a couple of other useful properties:
# The time range of the data, the name of the data columns
ts_lyra.time_range, ts_lyra.columns
( <sunpy.time.timerange.TimeRange object at 0x7f6f4df871c0>
Start: 2011-06-07 00:00:00
End: 2011-06-07 23:59:00
Center:2011-06-07 11:59:30
Duration:0.9993055555555554 days or
23.98333333333333 hours or
1438.9999999999998 minutes or
86339.99999999999 seconds
, ['CHANNEL1', 'CHANNEL2', 'CHANNEL3', 'CHANNEL4'])
Further data is available from within the metadata, you can filter out for a
key using the TimeSeriesMetaData.get() method.
combined_goes_ts.meta.get("publisher_name")
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2012-06-01T00:00:00.111000 | xrsa | publisher_name: National Centers for Environmental|
| to | xrsb | |
|2012-06-01T23:59:58.375000 | xrsa_quality | |
| | xrsb_quality | |
|-------------------------------------------------------------------------------------------------|
|2012-06-02T00:00:00.421000 | xrsa | publisher_name: National Centers for Environmental|
| to | xrsb | |
|2012-06-02T23:59:58.685000 | xrsa_quality | |
| | xrsb_quality | |
|-------------------------------------------------------------------------------------------------|
You can access a specific value within the TimeSeries data
using all the normal pandas methods.
For example, to get the row with the index of “2015-01-01 00:02:00.008000”
Pandas will actually parse a string to a datetime automatically if it can:
lyra_data.loc['2011-06-07 00:02:00.010']
# If this fails, you will need to use parse_time to convert the string to a datetime
lyra_data.loc[parse_time('2011-06-07 00:02:00.010').datetime]
# Pandas includes methods to find the indexes of the max/min values in a dataframe:
ts_lyra.to_dataframe()['CHANNEL1'].idxmax(), ts_lyra.to_dataframe()['CHANNEL1'].idxmin()
(Timestamp('2011-06-07 03:09:00.010000'), Timestamp('2011-06-07 09:09:00.010000'))
An individual column can be extracted
ts_eve.extract('CMLon')
<sunpy.timeseries.timeseriesbase.GenericTimeSeries object at 0x7f6f504100d0>
|
|
||||||||||||||||||||
|
Changing the units for a column simply requires changing the value
ts_eve.units['a'] = u.m
Quantities can be extracted from a column using sunpy.timeseries.GenericTimeSeries.quantity()
colname = 'CMLat'
quantity = ts_eve.quantity(colname)
quantity
<Quantity [-3.7, -3.9, -4.1, ..., -4.2, -3.9, -3.7] deg>
You can add or overwrite a column using sunpy.timeseries.GenericTimeSeries.add_column().
This method only accepts an Quantity and will convert to the intended units
if necessary.
new_quantity = quantity.value * 0.01 * ts_eve.units[colname]
new_eve_ts = ts_eve.add_column(colname, new_quantity, overwrite=True)
You can truncate using the sunpy.timeseries.GenericTimeSeries.truncate() method.
ts_goes_trunc = ts_goes.truncate(0, 100000, 2)
# Or using a `TimeRange`
ts_goes_trunc = ts_goes.truncate(TimeRange('2011-06-07 05:00', '2011-06-07 06:30'))
# Or using strings
ts_goes_trunc = ts_goes.truncate('2011-06-07 05:00', '2011-06-07 06:30')
fig, ax = plt.subplots()
ts_goes_trunc.plot(axes=ax)
plt.show()
For now you can only resample by using pandas.
Changing values within the dataframe directly will often affect the units
involved, but these won’t be picked up by TimeSeries.
Take care when doing this to ensure dimensional consistency.
df_downsampled = ts_goes_trunc.to_dataframe().resample('10T').mean()
ts_downsampled = sunpy.timeseries.TimeSeries(df_downsampled,
ts_goes_trunc.meta,
ts_goes_trunc.units)
fig, ax = plt.subplots()
ts_downsampled.plot(axes=ax)
plt.show()
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/checkouts/latest/examples/time_series/timeseries_example.py:149: FutureWarning: 'T' is deprecated and will be removed in a future version, please use 'min' instead.
df_downsampled = ts_goes_trunc.to_dataframe().resample('10T').mean()
The data from the TimeSeries can be retrieved in a number of formats
# pandas DataFrame
ts_goes.to_dataframe()
# astropy Table
ts_goes.to_table()
# numpy array
ts_goes.to_array()
array([[1.0000e-09, 1.8871e-07],
[1.0000e-09, 1.8346e-07],
[1.0000e-09, 1.8609e-07],
...,
[1.0000e-09, 1.5985e-07],
[1.0000e-09, 1.6248e-07],
[1.0000e-09, 1.5985e-07]], dtype=float32)
Creating a TimeSeries from scratch can be done several ways.
Input data can be in the form of a pandas.DataFrame (preferred),
an astropy.table.Table or a numpy.array.
base = datetime.datetime.today()
dates = Time(base) - TimeDelta(np.arange(24 * 60)*u.minute)
intensity = np.sin(np.arange(0, 12 * np.pi, ((12 * np.pi) / (24 * 60))))
# Create the data DataFrame, header MetaDict and units OrderedDict
data = DataFrame(intensity, index=dates, columns=['intensity'])
meta = MetaDict({'key': 'value'})
units = OrderedDict([('intensity', u.W / u.m**2)])
# Create the TimeSeries
ts_custom = sunpy.timeseries.TimeSeries(data, meta, units)
fig, ax = plt.subplots()
ts_custom.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 16.202 seconds)
Coordinates, times, and units#
Examples of working with coordinate information, times, and scientific units
Converting between Helioprojective and AltAz Coordinate
Create a Helioprojective Map from observations in the RA-DEC coordinate system
Identifying stars in a STEREO/SECCHI COR2 coronagraph image
Obtaining a spacecraft trajectory from JPL Horizons
Setting the correct position for SOHO in a LASCO C3 Map
Note
Go to the end to download the full example code.
AIA to STEREO coordinate conversion#
How to convert a point of a source on an AIA image to a position on a STEREO image.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.coordinates
import sunpy.map
from sunpy.data.sample import AIA_193_JUN2012, STEREO_A_195_JUN2012
from sunpy.sun import constants
Create a dictionary with the two maps, cropped down to full disk.
maps = {m.detector: m.submap(SkyCoord([-1100, 1100]*u.arcsec,
[-1100, 1100]*u.arcsec,
frame=m.coordinate_frame))
for m in sunpy.map.Map([AIA_193_JUN2012, STEREO_A_195_JUN2012])}
maps['AIA'].plot_settings['vmin'] = 0 # set the minimum plotted pixel value
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
We will be transforming coordinates where the formation height of 304 A emission makes a difference, so we set the reference solar radius to 4 Mm above the solar surface (see Alissandrakis 2019).
for m in maps.values():
m.meta['rsun_ref'] = (constants.radius + 4*u.Mm).to_value('m')
Plot both maps.
fig = plt.figure(figsize=(10, 4))
for i, m in enumerate(maps.values()):
ax = fig.add_subplot(1, 2, i+1, projection=m)
m.plot(axes=ax)
We are now going to pick out a region around the south west corner:
aia_bottom_left = SkyCoord(700 * u.arcsec,
100 * u.arcsec,
frame=maps['AIA'].coordinate_frame)
aia_top_right = SkyCoord(850 * u.arcsec,
350 * u.arcsec,
frame=maps['AIA'].coordinate_frame)
Plot a rectangle around the region we want to crop.
fig = plt.figure()
ax = fig.add_subplot(projection=maps['AIA'])
maps['AIA'].plot(axes=ax)
maps['AIA'].draw_quadrangle(aia_bottom_left, top_right=aia_top_right)
<astropy.visualization.wcsaxes.patches.Quadrangle object at 0x7f6f425c5bd0>
Create a submap of this area and draw an X at a specific feature of interest.
subaia = maps['AIA'].submap(aia_bottom_left, top_right=aia_top_right)
fig = plt.figure()
ax = fig.add_subplot(projection=subaia)
subaia.plot(axes=ax)
feature_aia = SkyCoord(800 * u.arcsec,
300 * u.arcsec,
frame=maps['AIA'].coordinate_frame)
ax.plot_coord(feature_aia, 'bx', fillstyle='none', markersize=20)
[<matplotlib.lines.Line2D object at 0x7f6f41cb5b40>]
We can transform the coordinate of the feature to see its representation as seen by STEREO EUVI. The original coordinate did not contain a distance from the observer, so it is converted to a 3D coordinate assuming that it has a radius from Sun center equal to the reference solar radius of the coordinate frame, which we set earlier to be the formation height of 304 A emission.
print(feature_aia.transform_to(maps['EUVI'].coordinate_frame))
<SkyCoord (Helioprojective: obstime=2012-06-01T00:05:30.831, rsun=699700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2012-06-01T00:05:30.831, rsun=699700.0 km): (lon, lat, radius) in (deg, deg, m)
(116.29405786, 6.81772843, 1.43457186e+11)>): (Tx, Ty, distance) in (arcsec, arcsec, m)
(-779.65052512, 243.90553172, 1.43047687e+11)>
Now we can plot this box on both the AIA and EUVI images. Note that using
draw_quadrangle() means that the plotted
rectangle will be automatically warped appropriately to account for the
different coordinate frames.
fig = plt.figure(figsize=(10, 4))
ax1 = fig.add_subplot(121, projection=maps['AIA'])
maps['AIA'].plot(axes=ax1)
maps['AIA'].draw_quadrangle(aia_bottom_left, top_right=aia_top_right, axes=ax1)
ax2 = fig.add_subplot(122, projection=maps['EUVI'])
maps['EUVI'].plot(axes=ax2)
maps['AIA'].draw_quadrangle(aia_bottom_left, top_right=aia_top_right, axes=ax2)
<astropy.visualization.wcsaxes.patches.Quadrangle object at 0x7f6f42480be0>
We can now zoom in on the region in the EUVI image, and we also draw an X
at the feature marked earlier. We do not need to explicitly transform the
feature coordinate to the matching coordinate frame; that is performed
automatically by plot_coord().
fig = plt.figure(figsize=(15, 5))
ax1 = fig.add_subplot(121, projection=subaia)
subaia.plot(axes=ax1)
ax1.plot_coord(feature_aia, 'bx', fillstyle='none', markersize=20)
subeuvi = maps['EUVI'].submap(aia_bottom_left, top_right=aia_top_right)
ax2 = fig.add_subplot(122, projection=subeuvi)
subeuvi.plot(axes=ax2)
maps['AIA'].draw_quadrangle(aia_bottom_left, top_right=aia_top_right, axes=ax2)
ax2.plot_coord(feature_aia, 'bx', fillstyle='none', markersize=20)
plt.show()
Total running time of the script: (0 minutes 2.463 seconds)
Note
Go to the end to download the full example code.
Converting between Helioprojective and AltAz Coordinate#
How to find the Sun in the sky as viewed from a particular location.
import astropy.units as u
from astropy.coordinates import AltAz, EarthLocation, SkyCoord
from astropy.time import Time
from sunpy.coordinates import frames, sun
We use SkyCoord to define the center of the Sun.
obstime = "2013-09-21 16:00:00"
c = SkyCoord(0 * u.arcsec, 0 * u.arcsec, obstime=obstime,
observer="earth", frame=frames.Helioprojective)
Now we establish our location on the Earth, in this case let’s consider a high altitude balloon launched from Fort Sumner, NM.
Fort_Sumner = EarthLocation(lat=34.4900*u.deg, lon=-104.221800*u.deg, height=40*u.km)
Now lets convert this to a local measurement of Altitude and Azimuth.
frame_altaz = AltAz(obstime=Time(obstime), location=Fort_Sumner)
sun_altaz = c.transform_to(frame_altaz)
print(f'Altitude is {sun_altaz.T.alt} and Azimuth is {sun_altaz.T.az}')
Altitude is 37.782959993653094 deg and Azimuth is 121.34217350367328 deg
Next let’s check this calculation by converting it back to helioprojective. We should get our original input which was the center of the Sun. To go from Altitude/Azimuth to Helioprojective, you will need the distance to the Sun. solar distance. Define distance with sunpy’s almanac.
distance = sun.earth_distance(obstime)
b = SkyCoord(az=sun_altaz.T.az, alt=sun_altaz.T.alt, distance=distance, frame=frame_altaz)
sun_helio = b.transform_to(frames.Helioprojective(observer="earth"))
print(f'The helioprojective point is {sun_helio.T.Tx}, {sun_helio.T.Ty}')
The helioprojective point is 0.3736719261032472 arcsec, 0.13601753457547278 arcsec
Total running time of the script: (0 minutes 9.010 seconds)
Note
Go to the end to download the full example code.
Coordinates computations using SPICE kernels#
How to use SPICE kernels provided by space missions to perform coordinates computations.
The SPICE observation geometry information system is being increasingly used by space missions to describe the locations of spacecraft and the time-varying orientations of reference frames. Here are two examples of mission SPICE kernels:
The sunpy.coordinates.spice module enables the use of the
SkyCoord API to perform SPICE computations such as the
location of bodies or the transformation of a vector from one coordinate frame
to another coordinate frame. Although SPICE kernels can define coordinate
frames that are very similar to the frames that sunpy.coordinates already
provides, there will very likely be slight differences. Using
sunpy.coordinates.spice will ensure that the definitions are exactly what the
mission specifies and that the results are identical to other implementations
of SPICE (e.g., CSPICE or Icy).
Note
This example requires the optional dependency spiceypy to be
installed.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.dates import DateFormatter
import astropy.units as u
from astropy.coordinates import SkyCoord
from sunpy.coordinates import spice
from sunpy.data import cache
from sunpy.time import parse_time
Download a small subset (~30 MB) of the Solar Orbiter SPICE kernel set that corresponds to about a day of the mission.
obstime = parse_time('2022-10-12') + np.arange(720) * u.min
kernel_urls = [
"ck/solo_ANC_soc-sc-fof-ck_20180930-21000101_V03.bc",
"ck/solo_ANC_soc-stix-ck_20180930-21000101_V03.bc",
"ck/solo_ANC_soc-flown-att_20221011T142135-20221012T141817_V01.bc",
"fk/solo_ANC_soc-sc-fk_V09.tf",
"fk/solo_ANC_soc-sci-fk_V08.tf",
"ik/solo_ANC_soc-stix-ik_V02.ti",
"lsk/naif0012.tls",
"pck/pck00010.tpc",
"sclk/solo_ANC_soc-sclk_20231015_V01.tsc",
"spk/de421.bsp",
"spk/solo_ANC_soc-orbit-stp_20200210-20301120_280_V1_00288_V01.bsp",
]
kernel_urls = [f"http://spiftp.esac.esa.int/data/SPICE/SOLAR-ORBITER/kernels/{url}"
for url in kernel_urls]
kernel_files = [cache.download(url) for url in kernel_urls]
Initialize sunpy.coordinates.spice with these kernels, which will create
classes for SkyCoord to use.
spice.initialize(kernel_files)
INFO: Installing SOLO_IAU_SUN_2003 PCK frame (-144994) as 'spice_SOLO_IAU_SUN_2003' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with SUN (10) origin [sunpy.coordinates.spice]
INFO: Installing SOLO_IAU_SUN_2009 PCK frame (-144993) as 'spice_SOLO_IAU_SUN_2009' [sunpy.coordinates.spice]
INFO: Installing SOLO_STIX_ILS CK frame (-144851) as 'spice_SOLO_STIX_ILS' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with SOLAR-ORBITER (-144) origin [sunpy.coordinates.spice]
INFO: Installing SOLO_SPICE_LW_ILS CK frame (-144821) as 'spice_SOLO_SPICE_LW_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_SPICE_SW_ILS CK frame (-144811) as 'spice_SOLO_SPICE_SW_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_SOLOHI_ILS CK frame (-144701) as 'spice_SOLO_SOLOHI_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_PHI_HRT_ILS CK frame (-144521) as 'spice_SOLO_PHI_HRT_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_PHI_FDT_ILS CK frame (-144511) as 'spice_SOLO_PHI_FDT_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_M0_TEL CK frame (-144431) as 'spice_SOLO_METIS_M0_TEL' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_IEO-M0 CK frame (-144430) as 'spice_SOLO_METIS_IEOnM0' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_VIS_ILS CK frame (-144421) as 'spice_SOLO_METIS_VIS_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_EUV_ILS CK frame (-144411) as 'spice_SOLO_METIS_EUV_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_HRI_EUV_ILS CK frame (-144231) as 'spice_SOLO_EUI_HRI_EUV_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_HRI_LYA_ILS CK frame (-144221) as 'spice_SOLO_EUI_HRI_LYA_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_FSI_ILS CK frame (-144211) as 'spice_SOLO_EUI_FSI_ILS' [sunpy.coordinates.spice]
INFO: Installing SOLO_INS_BOOM_OB CK frame (-144042) as 'spice_SOLO_INS_BOOM_OB' [sunpy.coordinates.spice]
INFO: Installing SOLO_INS_BOOM_IB CK frame (-144041) as 'spice_SOLO_INS_BOOM_IB' [sunpy.coordinates.spice]
INFO: Installing SOLO_MGA_EL CK frame (-144031) as 'spice_SOLO_MGA_EL' [sunpy.coordinates.spice]
INFO: Installing SOLO_SA-Y CK frame (-144017) as 'spice_SOLO_SAnY' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with SOLO_SA-Y (-144017) origin [sunpy.coordinates.spice]
INFO: Installing SOLO_SA+Y CK frame (-144015) as 'spice_SOLO_SApY' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with SOLO_SA+Y (-144015) origin [sunpy.coordinates.spice]
INFO: Installing SOLO_HGA_AZ CK frame (-144012) as 'spice_SOLO_HGA_AZ' [sunpy.coordinates.spice]
INFO: Installing SOLO_HGA_EL CK frame (-144011) as 'spice_SOLO_HGA_EL' [sunpy.coordinates.spice]
INFO: Installing SOLO_FOF CK frame (-144001) as 'spice_SOLO_FOF' [sunpy.coordinates.spice]
INFO: Installing SOLO_SRF CK frame (-144000) as 'spice_SOLO_SRF' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_EAS2-SCI TK frame (-144877) as 'spice_SOLO_SWA_EAS2nSCI' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_EAS1-SCI TK frame (-144876) as 'spice_SOLO_SWA_EAS1nSCI' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_EAS2 TK frame (-144875) as 'spice_SOLO_SWA_EAS2' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_EAS1 TK frame (-144874) as 'spice_SOLO_SWA_EAS1' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_EAS TK frame (-144873) as 'spice_SOLO_SWA_EAS' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_PAS TK frame (-144872) as 'spice_SOLO_SWA_PAS' [sunpy.coordinates.spice]
INFO: Installing SOLO_SWA_HIS TK frame (-144871) as 'spice_SOLO_SWA_HIS' [sunpy.coordinates.spice]
INFO: Installing SOLO_STIX_OPT TK frame (-144852) as 'spice_SOLO_STIX_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_SPICE_LW_OPT TK frame (-144822) as 'spice_SOLO_SPICE_LW_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_SPICE_SW_OPT TK frame (-144812) as 'spice_SOLO_SPICE_SW_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_SOLOHI_OPT TK frame (-144702) as 'spice_SOLO_SOLOHI_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_RPW_SCM TK frame (-144640) as 'spice_SOLO_RPW_SCM' [sunpy.coordinates.spice]
INFO: Installing SOLO_RPW_ANT_3 TK frame (-144630) as 'spice_SOLO_RPW_ANT_3' [sunpy.coordinates.spice]
INFO: Installing SOLO_RPW_ANT_2 TK frame (-144620) as 'spice_SOLO_RPW_ANT_2' [sunpy.coordinates.spice]
INFO: Installing SOLO_RPW_ANT_1 TK frame (-144610) as 'spice_SOLO_RPW_ANT_1' [sunpy.coordinates.spice]
INFO: Installing SOLO_PHI_HRT_OPT TK frame (-144522) as 'spice_SOLO_PHI_HRT_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_PHI_FDT_OPT TK frame (-144512) as 'spice_SOLO_PHI_FDT_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_VIS_OPT TK frame (-144422) as 'spice_SOLO_METIS_VIS_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_METIS_EUV_OPT TK frame (-144412) as 'spice_SOLO_METIS_EUV_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_MAG_OBS TK frame (-144302) as 'spice_SOLO_MAG_OBS' [sunpy.coordinates.spice]
INFO: Installing SOLO_MAG_IBS TK frame (-144301) as 'spice_SOLO_MAG_IBS' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_HRI_EUV_OPT TK frame (-144232) as 'spice_SOLO_EUI_HRI_EUV_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_HRI_LYA_OPT TK frame (-144222) as 'spice_SOLO_EUI_HRI_LYA_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_EUI_FSI_OPT TK frame (-144212) as 'spice_SOLO_EUI_FSI_OPT' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_EPT-HET_PY TK frame (-144122) as 'spice_SOLO_EPD_EPTnHET_PY' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_EPT-HET_MY TK frame (-144121) as 'spice_SOLO_EPD_EPTnHET_MY' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_SIS_SW TK frame (-144112) as 'spice_SOLO_EPD_SIS_SW' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_SIS_ASW TK frame (-144111) as 'spice_SOLO_EPD_SIS_ASW' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_SIS TK frame (-144110) as 'spice_SOLO_EPD_SIS' [sunpy.coordinates.spice]
INFO: Installing SOLO_EPD_STEP TK frame (-144100) as 'spice_SOLO_EPD_STEP' [sunpy.coordinates.spice]
INFO: Installing SOLO_STR-2 TK frame (-144052) as 'spice_SOLO_STRn2' [sunpy.coordinates.spice]
INFO: Installing SOLO_STR-1 TK frame (-144051) as 'spice_SOLO_STRn1' [sunpy.coordinates.spice]
INFO: Installing SOLO_MGA_MRF TK frame (-144032) as 'spice_SOLO_MGA_MRF' [sunpy.coordinates.spice]
INFO: Installing SOLO_MGA_URF TK frame (-144030) as 'spice_SOLO_MGA_URF' [sunpy.coordinates.spice]
INFO: Installing SOLO_LGA_MZ TK frame (-144021) as 'spice_SOLO_LGA_MZ' [sunpy.coordinates.spice]
INFO: Installing SOLO_LGA_PZ TK frame (-144020) as 'spice_SOLO_LGA_PZ' [sunpy.coordinates.spice]
INFO: Installing SOLO_SA-Y_ZERO TK frame (-144016) as 'spice_SOLO_SAnY_ZERO' [sunpy.coordinates.spice]
INFO: Installing SOLO_SA+Y_ZERO TK frame (-144014) as 'spice_SOLO_SApY_ZERO' [sunpy.coordinates.spice]
INFO: Installing SOLO_HGA_MRF TK frame (-144013) as 'spice_SOLO_HGA_MRF' [sunpy.coordinates.spice]
INFO: Installing SOLO_HGA_URF TK frame (-144010) as 'spice_SOLO_HGA_URF' [sunpy.coordinates.spice]
INFO: Installing SUN_ARIES_ECL TK frame (1000010000) as 'spice_SUN_ARIES_ECL' [sunpy.coordinates.spice]
INFO: Installing SOLO_VSO dynamic frame (-144999) as 'spice_SOLO_VSO' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with VENUS (299) origin [sunpy.coordinates.spice]
INFO: Installing EARTH_MECL_MEQX_J2000 dynamic frame (-144998) as 'spice_EARTH_MECL_MEQX_J2000' [sunpy.coordinates.spice]
INFO: Creating ICRF frame with EARTH (399) origin [sunpy.coordinates.spice]
INFO: Installing SOLO_HEE dynamic frame (-144997) as 'spice_SOLO_HEE' [sunpy.coordinates.spice]
INFO: Installing SOLO_GSE dynamic frame (-144996) as 'spice_SOLO_GSE' [sunpy.coordinates.spice]
INFO: Installing SOLO_GAE dynamic frame (-144995) as 'spice_SOLO_GAE' [sunpy.coordinates.spice]
INFO: Installing SOLO_SOLAR_MHP dynamic frame (-144992) as 'spice_SOLO_SOLAR_MHP' [sunpy.coordinates.spice]
INFO: Installing SOLO_SUN_RTN dynamic frame (-144991) as 'spice_SOLO_SUN_RTN' [sunpy.coordinates.spice]
INFO: Installing SOLO_HOR dynamic frame (-144985) as 'spice_SOLO_HOR' [sunpy.coordinates.spice]
INFO: Installing SOLO_GEORTN dynamic frame (-144984) as 'spice_SOLO_GEORTN' [sunpy.coordinates.spice]
INFO: Installing SOLO_HEEQ dynamic frame (-144983) as 'spice_SOLO_HEEQ' [sunpy.coordinates.spice]
INFO: Installing SOLO_HEE_NASA dynamic frame (-144982) as 'spice_SOLO_HEE_NASA' [sunpy.coordinates.spice]
INFO: Installing SOLO_HCI dynamic frame (-144981) as 'spice_SOLO_HCI' [sunpy.coordinates.spice]
INFO: Installing SOLO_ECLIPDATE dynamic frame (-144980) as 'spice_SOLO_ECLIPDATE' [sunpy.coordinates.spice]
INFO: Installing SOLO_GSM dynamic frame (-144962) as 'spice_SOLO_GSM' [sunpy.coordinates.spice]
INFO: Installing EARTH_MECL_MEQX dynamic frame (300399000) as 'spice_EARTH_MECL_MEQX' [sunpy.coordinates.spice]
INFO: Installing EARTH_SUN_ECL dynamic frame (300399005) as 'spice_EARTH_SUN_ECL' [sunpy.coordinates.spice]
INFO: Installing SUN_EARTH_CEQU dynamic frame (1000010001) as 'spice_SUN_EARTH_CEQU' [sunpy.coordinates.spice]
INFO: Installing SUN_EARTH_ECL dynamic frame (1000010002) as 'spice_SUN_EARTH_ECL' [sunpy.coordinates.spice]
INFO: Installing SUN_INERTIAL dynamic frame (1000010004) as 'spice_SUN_INERTIAL' [sunpy.coordinates.spice]
The above call automatically installs all SPICE frames defined in the kernels, but you may also want to use one of the built-in SPICE frames (e.g., inertial frames or body-fixed frames). Here, we manually install the ‘IAU_SUN’ built-in SPICE frame for potential later use.
spice.install_frame('IAU_SUN')
INFO: Installing IAU_SUN PCK frame (10010) as 'spice_IAU_SUN' [sunpy.coordinates.spice]
We can request the location of the spacecraft in any SPICE frame. Here, we request it in ‘SOLO_HEEQ’, which is Stonyhurst heliographic coordinates.
spacecraft = spice.get_body('Solar Orbiter', obstime, spice_frame='SOLO_HEEQ')
print(spacecraft[:4])
<SkyCoord (spice_SOLO_HEEQ: obstime=['2022-10-12T00:00:00.000' '2022-10-12T00:01:00.000'
'2022-10-12T00:02:00.000' '2022-10-12T00:03:00.000']): (lon, lat, distance) in (deg, deg, km)
[(-119.01442862, -3.70146925, 43862837.40904176),
(-119.00981917, -3.7008108 , 43862685.53850186),
(-119.0052097 , -3.70015231, 43862533.79895207),
(-119.00060019, -3.69949378, 43862382.19039429)]>
Plot the radial distance from the Sun over the time range.
fig = plt.figure()
ax = fig.add_subplot()
ax.plot(obstime.datetime64, spacecraft.distance.to('AU'))
ax.xaxis.set_major_formatter(DateFormatter('%H:%M'))
ax.set_xlabel('2022 October 12 (UTC)')
ax.set_ylabel('Radial distance (AU)')
ax.set_title('Solar Orbiter distance from Sun center')
Text(0.5, 1.0, 'Solar Orbiter distance from Sun center')
We can then transform the coordinate to a different SPICE frame. When
specifying the frame for SkyCoord, SPICE frame names
should be prepended with 'spice_'. Here, we transform it to ‘SOLO_GAE’.
spacecraft_gae = spacecraft.transform_to("spice_SOLO_GAE")
print(spacecraft_gae[:4])
<SkyCoord (spice_SOLO_GAE: obstime=['2022-10-12T00:00:00.000' '2022-10-12T00:01:00.000'
'2022-10-12T00:02:00.000' '2022-10-12T00:03:00.000']): (lon, lat, distance) in (deg, deg, km)
[(-149.06635461, -1.03769419, 1.74975667e+08),
(-149.06492563, -1.03771064, 1.74972891e+08),
(-149.06349666, -1.03772709, 1.74970115e+08),
(-149.06206771, -1.03774353, 1.74967339e+08)]>
Plot the radial distance from the Earth over the time range.
fig = plt.figure()
ax = fig.add_subplot()
ax.plot(obstime.datetime64, spacecraft_gae.distance.to('AU'))
ax.xaxis.set_major_formatter(DateFormatter('%H:%M'))
ax.set_xlabel('2022 October 12 (UTC)')
ax.set_ylabel('Radial distance (AU)')
ax.set_title('Solar Orbiter distance from Earth center')
Text(0.5, 1.0, 'Solar Orbiter distance from Earth center')
We can also leverage the Solar Orbiter SPICE kernels to look at instrument pointing. Let’s define a coordinate that points directly along the line of sight of the STIX instrument. For the ‘SOLO_STIX_ILS’ frame, 0 degrees longitude is in the anti-Sun direction, while 180 degrees longitude is in the Sun direction.
stix_ils = SkyCoord(np.repeat(0*u.deg, len(obstime)),
np.repeat(0*u.deg, len(obstime)),
frame='spice_SOLO_STIX_ILS', obstime=obstime)
print(stix_ils[:4])
<SkyCoord (spice_SOLO_STIX_ILS: obstime=['2022-10-12T00:00:00.000' '2022-10-12T00:01:00.000'
'2022-10-12T00:02:00.000' '2022-10-12T00:03:00.000']): (lon, lat) in deg
[(0., 0.), (0., 0.), (0., 0.), (0., 0.)]>
We can transform that line of sight to the SPICE frame ‘SOLO_SUN_RTN’, which is similar to helioprojective coordinates as observed from Solar Orbiter, except that the disk center is at 180 degrees longitude instead of 0 degrees longitude. Given how the line-of-sight coordinate is defined above, the latitude and longitude values of the resulting coordinate are the pitch and yaw offsets from disk center, respectively.
stix_ils_rtn = stix_ils.transform_to("spice_SOLO_SUN_RTN")
print(stix_ils_rtn[:4])
<SkyCoord (spice_SOLO_SUN_RTN: obstime=['2022-10-12T00:00:00.000' '2022-10-12T00:01:00.000'
'2022-10-12T00:02:00.000' '2022-10-12T00:03:00.000']): (lon, lat) in deg
[( 4.39956418e-05, -9.64403588e-06),
(-6.14438994e-06, 3.36046063e-05),
( 7.84301580e-05, 5.45084475e-05),
( 1.38833163e-04, -2.82398730e-05)]>
Plot the pitch/yaw offsets over the time range.
fig, ax = plt.subplots(2, 1)
ax[0].plot(obstime.datetime64, stix_ils_rtn.lat.to('arcsec'))
ax[0].xaxis.set_major_formatter(DateFormatter('%H:%M'))
ax[0].set_xlabel('2022 October 12 (UTC)')
ax[0].set_ylabel('Pitch offset (arcsec)')
ax[1].plot(obstime.datetime64, stix_ils_rtn.lon.to('arcsec'))
ax[1].xaxis.set_major_formatter(DateFormatter('%H:%M'))
ax[1].set_xlabel('2022 October 12 (UTC)')
ax[1].set_ylabel('Yaw offset (arcsec)')
ax[0].set_title('Pointing offset of STIX from disk center')
plt.show()
Finally, we can query the instrument field of view (FOV) via SPICE, which will be in the ‘SOLO_STIX_ILS’ frame. This call returns the corners of the rectangular FOV of the STIX instrument, and you can see they are centered around 180 degrees longitude, which is the direction of the Sun in this frame.
stix_fov = spice.get_fov('SOLO_STIX', obstime[0])
print(stix_fov)
<SkyCoord (spice_SOLO_STIX_ILS: obstime=2022-10-12T00:00:00.000): (lon, lat) in deg
[( 179., 0.99984773), ( 179., -0.99984773), (-179., -0.99984773),
(-179., 0.99984773)]>
More usefully, every coordinate in a SPICE frame has a
to_helioprojective()
method that converts the coordinate to Helioprojective
with the observer at the center of te SPICE frame. For the
‘SOLO_STIX_ILS’ frame, the center is Solar Orbiter, which is exactly what we
want.
print(stix_fov.to_helioprojective())
<SkyCoord (Helioprojective: obstime=2022-10-12T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2022-10-12T00:00:00.000, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, km)
(-119.01442821, -3.70146956, 43862838.51397399)>): (Tx, Ty) in arcsec
[( 3127.44364431, 4016.75973995), ( 4017.02831673, -3126.97425142),
(-3127.76039243, -4016.69031087), (-4017.34510585, 3127.04367531)]>
Total running time of the script: (0 minutes 12.090 seconds)
Note
Go to the end to download the full example code.
Create a Helioprojective Map from observations in the RA-DEC coordinate system#
How to create a Map in Helioprojective Coordinate Frame from radio observations
in GCRS (RA-DEC).
In this example a LOFAR FITS file (created with LOFAR’s Default Pre-Processing Pipeline (DPPP) and
WSClean Imager) is read in,
the WCS header information is then used to make a new header with the information in Helioprojective,
and a Map is made.
The LOFAR example file has a WCS in celestial coordinates i.e. Right Ascension and
Declination (RA-DEC). For this example, we are assuming that the definition of LOFAR’s
coordinate system for this observation is exactly the same as Astropy’s ~astropy.coordinates.GCRS.
For many solar studies we may want to plot this data in some Sun-centered coordinate frame,
such as Helioprojective. In this example we read the data and
header information from the LOFAR FITS file and then create a new header with updated WCS
information to create a Map with a HPC coordinate frame. We will make use of the
astropy.coordinates and sunpy.coordinates submodules together with
make_fitswcs_header() to create a new header and generate a
Map.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import EarthLocation, SkyCoord
from astropy.io import fits
from astropy.time import Time
import sunpy.data.sample
import sunpy.map
from sunpy.coordinates import frames, sun
We will first begin be reading in the header and data from the FITS file.
hdu = fits.open(sunpy.data.sample.LOFAR_IMAGE)
header = hdu[0].header
The data in this file is in a datacube structure to hold difference frequencies and polarizations. We are only interested in the image at one frequency (the only data in the file) so we index the data array to be the 2D data of interest.
data = hdu[0].data[0, 0, :, :]
We can inspect the header, for example, we can print the coordinate system and type projection, which here is RA-DEC.
print(header['ctype1'], header['ctype2'])
RA---SIN DEC--SIN
Lets pull out the observation time and wavelength from the header, we will use these to create our new header.
obstime = Time(header['date-obs'])
frequency = header['crval3']*u.Hz
To create a new Map header we need convert the reference coordinate
in RA-DEC (that is in the header) to Helioprojective. To do this we will first create
an astropy.coordinates.SkyCoord of the reference coordinate from the header information.
We will need the location of the observer (i.e. where the observation was taken).
We first establish the location on Earth from which the observation takes place, in this
case LOFAR observations are taken from Exloo in the Netherlands, which we define in lat and lon.
We can convert this to a SkyCoord in GCRSat the observation time.
lofar_loc = EarthLocation(lat=52.905329712*u.deg, lon=6.867996528*u.deg)
lofar_gcrs = SkyCoord(lofar_loc.get_gcrs(obstime))
We can then define the reference coordinate in terms of RA-DEC from the header information.
Here we are using the obsgeoloc keyword argument to take into account that the observer is not
at the center of the Earth (i.e. the GCRS origin). The distance here is the Sun-observer distance.
reference_coord = SkyCoord(header['crval1']*u.Unit(header['cunit1']),
header['crval2']*u.Unit(header['cunit2']),
frame='gcrs',
obstime=obstime,
obsgeoloc=lofar_gcrs.cartesian,
obsgeovel=lofar_gcrs.velocity.to_cartesian(),
distance=lofar_gcrs.hcrs.distance)
Now we can convert the reference_coord to the HPC coordinate frame.
reference_coord_arcsec = reference_coord.transform_to(frames.Helioprojective(observer=lofar_gcrs))
Now we need to get the other parameters from the header that will be used to create the new header - here we can get the cdelt1 and cdelt2 which are the spatial scales of the data axes.
cdelt1 = (np.abs(header['cdelt1'])*u.deg).to(u.arcsec)
cdelt2 = (np.abs(header['cdelt2'])*u.deg).to(u.arcsec)
Finally, we need to specify the orientation of the HPC coordinate grid because
GCRS north is not in the same direction as HPC north. For convenience, we use
P() to calculate this relative rotation angle,
although due to subtleties in definitions, the returned value is inaccurate by
2 arcmin, equivalent to a worst-case shift of 0.6 arcsec for HPC coordinates
on the disk. The image will need to be rotated by this angle so that solar north
is pointing up.
P1 = sun.P(obstime)
Now we can use this information to create a new header using the helper
function make_fitswcs_header(). This will
create a MetaDict which will contain all the necessary WCS information to
create a Map. We provide a reference coordinate (in HPC),
the spatial scale of the observation (i.e., cdelt1 and cdelt2),
and the rotation angle (P1). Note that here, 1 is subtracted from the
crpix1 and crpix2 values, this is because the reference_pixel
keyword in :func:~sunpy.map.header_helper.make_fitswcs_header` is zero
indexed rather than the fits convention of 1 indexed.
new_header = sunpy.map.make_fitswcs_header(data, reference_coord_arcsec,
reference_pixel=u.Quantity([header['crpix1']-1,
header['crpix2']-1]*u.pixel),
scale=u.Quantity([cdelt1, cdelt2]*u.arcsec/u.pix),
rotation_angle=-P1,
wavelength=frequency.to(u.MHz).round(2),
observatory='LOFAR')
Let’s inspect the new header.
print(new_header)
('wcsaxes': '2')
('crpix1': '401.0')
('crpix2': '401.0')
('cdelt1': '20.000000000000018')
('cdelt2': '20.000000000000018')
('cunit1': 'arcsec')
('cunit2': 'arcsec')
('ctype1': 'HPLN-TAN')
('ctype2': 'HPLT-TAN')
('crval1': '2968.682862035918')
('crval2': '209.9412294598465')
('lonpole': '180.0')
('latpole': '0.0')
('mjdref': '0.0')
('date-obs': '2019-04-09T13:11:36.200')
('rsun_ref': '695700000.0')
('dsun_obs': '149822938834.19')
('hgln_obs': '-0.0013230706129548')
('hglt_obs': '-6.0483966759632')
('obsrvtry': 'LOFAR')
('wavelnth': '70.31')
('waveunit': 'MHz')
('naxis': '2')
('naxis1': '800')
('naxis2': '800')
('pc1_1': '0.8969123480105654')
('pc1_2': '-0.44220836715984285')
('pc2_1': '0.44220836715984285')
('pc2_2': '0.8969123480105654')
('rsun_obs': '957.7901922846717')
Lets create a Map.
lofar_map = sunpy.map.Map(data, new_header)
We can now plot this map and inspect it.
fig = plt.figure()
ax = fig.add_subplot(projection=lofar_map)
lofar_map.plot(axes=ax, cmap='viridis')
lofar_map.draw_limb(axes=ax)
plt.show()
We can now rotate the image so that solar north is pointing up and create a submap in the field of view of interest.
lofar_map_rotate = lofar_map.rotate()
bl = SkyCoord(-1500*u.arcsec, -1500*u.arcsec, frame=lofar_map_rotate.coordinate_frame)
tr = SkyCoord(1500*u.arcsec, 1500*u.arcsec, frame=lofar_map_rotate.coordinate_frame)
lofar_submap = lofar_map_rotate.submap(bl, top_right=tr)
Now lets plot this map, and overplot some contours.
fig = plt.figure()
ax = fig.add_subplot(projection=lofar_submap)
lofar_submap.plot(axes=ax, cmap='viridis')
lofar_submap.draw_limb(axes=ax)
lofar_submap.draw_grid(axes=ax)
lofar_submap.draw_contours(np.arange(30, 100, 5)*u.percent, axes=ax)
plt.show()
Total running time of the script: (0 minutes 1.245 seconds)
Note
Go to the end to download the full example code.
Drawing the AIA limb on a STEREO EUVI image#
In this example we use a STEREO-B and an SDO image to demonstrate how to overplot the limb as seen by AIA on an EUVI-B image. Then we overplot the AIA coordinate grid on the STEREO image.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.coordinates.wcs_utils
import sunpy.map
from sunpy.data.sample import AIA_193_JUN2012, STEREO_B_195_JUN2012
Let’s create a dictionary with the two maps, which we crop to full disk.
maps = {m.detector: m.submap(SkyCoord([-1100, 1100], [-1100, 1100],
unit=u.arcsec, frame=m.coordinate_frame))
for m in sunpy.map.Map([AIA_193_JUN2012, STEREO_B_195_JUN2012])}
maps['AIA'].plot_settings['vmin'] = 0 # set the minimum plotted pixel value
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Now, let’s plot both maps, and we draw the limb as seen by AIA onto the EUVI image. We remove the part of the limb that is hidden because it is on the far side of the Sun from STEREO’s point of view.
fig = plt.figure(figsize=(10, 4))
ax1 = fig.add_subplot(121, projection=maps['AIA'])
maps['AIA'].plot(axes=ax1)
maps['AIA'].draw_limb(axes=ax1)
ax2 = fig.add_subplot(122, projection=maps['EUVI'])
maps['EUVI'].plot(axes=ax2)
visible, hidden = maps['AIA'].draw_limb(axes=ax2)
hidden.remove()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Let’s plot the helioprojective coordinate grid as seen by SDO on the STEREO image in a cropped view. Note that only those grid lines that intersect the edge of the plot will have corresponding ticks and tick labels.
fig = plt.figure()
ax = fig.add_subplot(projection=maps['EUVI'])
maps['EUVI'].plot(axes=ax)
# Crop the view using pixel coordinates
ax.set_xlim(500, 1300)
ax.set_ylim(100, 900)
# Shrink the plot slightly and move the title up to make room for new labels.
ax.set_position([0.1, 0.1, 0.8, 0.7])
ax.set_title(ax.get_title(), pad=45)
# Change the default grid labels and line properties.
stereo_x, stereo_y = ax.coords
stereo_x.set_axislabel("Helioprojective Longitude (STEREO B) [arcsec]")
stereo_y.set_axislabel("Helioprojective Latitude (STEREO B) [arcsec]")
ax.coords.grid(color='white', linewidth=1)
# Add a new coordinate overlay in the SDO frame.
overlay = ax.get_coords_overlay(maps['AIA'].coordinate_frame)
overlay.grid()
# Configure the grid:
x, y = overlay
# Wrap the longitude at 180 deg rather than the default 360.
x.set_coord_type('longitude', 180*u.deg)
# Set the tick spacing
x.set_ticks(spacing=250*u.arcsec)
y.set_ticks(spacing=250*u.arcsec)
# Set the ticks to be on the top and left axes.
x.set_ticks_position('tr')
y.set_ticks_position('tr')
# Change the defaults to arcseconds
x.set_major_formatter('s.s')
y.set_major_formatter('s.s')
# Add axes labels
x.set_axislabel("Helioprojective Longitude (SDO) [arcsec]")
y.set_axislabel("Helioprojective Latitude (SDO) [arcsec]")
plt.show()
Total running time of the script: (0 minutes 2.349 seconds)
Note
Go to the end to download the full example code.
Extracting intensity of a map along a line#
In this example we will extract the intensity values of all the pixels that intersect with a given set of coordinates.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_ROLL_IMAGE
First we construct a map, using some sample data.
aia_map = sunpy.map.Map(AIA_171_ROLL_IMAGE)
Next we define a path in a SkyCoord object.
In this example we are just going to use a straight line defined by two points.
However a path with any number of points and any shape can be used.
line_coords = SkyCoord([-1024, -908], [20, 633], unit=(u.arcsec, u.arcsec),
frame=aia_map.coordinate_frame)
Next we call the sunpy.map.pixelate_coord_path() function with the map
and the coordinate path to obtain the coordinates of the map pixels that
intersect that path. We pass those coordinates to
sunpy.map.sample_at_coords() to extract the values for those map
pixels.
intensity_coords = sunpy.map.pixelate_coord_path(aia_map, line_coords)
intensity = sunpy.map.sample_at_coords(aia_map, intensity_coords)
Next we will calculate the angular separation between the first point and every other coordinate we extracted. We are doing this to give us a meaningful x-axis for our line plot below.
angular_separation = intensity_coords.separation(intensity_coords[0]).to(u.arcsec)
Finally let’s plot the results.
fig = plt.figure(figsize=(10, 4))
ax1 = fig.add_subplot(121, projection=aia_map)
aia_map.plot(axes=ax1)
ax1.plot_coord(intensity_coords)
ax1.plot_coord(line_coords[0], marker="o", color="blue", label="start")
ax1.plot_coord(line_coords[1], marker="o", color="green", label="end")
ax1.legend()
ax2 = fig.add_subplot(122)
ax2.plot(angular_separation, intensity)
ax2.set_xlabel("Angular distance along slit [arcsec]")
ax2.set_ylabel(f"Intensity [{aia_map.unit}]")
plt.show()
Total running time of the script: (0 minutes 0.780 seconds)
Note
Go to the end to download the full example code.
Getting the location of the planets#
How to get the position of planetary bodies im the solar system using astropy’s solar system ephemeris information and sunpy.
import matplotlib.pyplot as plt
from astropy.time import Time
from sunpy.coordinates import get_body_heliographic_stonyhurst
Lets grab the positions of each of the planets in Heliographic Stonyhurst coordinates.
obstime = Time('2014-05-15T07:54:00.005')
planet_list = ['earth', 'venus', 'mars', 'mercury', 'jupiter', 'neptune', 'uranus', 'sun']
planet_coord = [get_body_heliographic_stonyhurst(
this_planet, time=obstime) for this_planet in planet_list]
Let’s plot the results. Remember the Sun is at the center of this coordinate system.
fig = plt.figure()
ax = fig.add_subplot(projection='polar')
for this_planet, this_coord in zip(planet_list, planet_coord):
ax.plot(this_coord.lon.to('rad'), this_coord.radius, 'o', label=this_planet)
ax.legend()
plt.show()
Total running time of the script: (0 minutes 0.244 seconds)
Note
Go to the end to download the full example code.
Getting the observer location from a Map#
How to access the observer location from a Map and interpret it.
import matplotlib.pyplot as plt
import numpy as np
from astropy.constants import R_earth
import sunpy.map
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.data.sample import AIA_171_IMAGE
We use the sunpy sample data.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
You can access the observer coordinate with:
print(aiamap.observer_coordinate)
<SkyCoord (HeliographicStonyhurst: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>
This provides the location of the SDO as defined in the header and is necessary to fully define the helioprojective coordinate system which depends on where the observer is. Let’s see where this is with respect to Earth. SDO is a geosynchronous orbit with a semi-major axis of 42,164.71 km and an inclination of 28.05 deg. We will convert it to Geocentric Celestial Reference System (GCRS) whose center is at the Earth’s center-of-mass.
sdo_gcrs = aiamap.observer_coordinate.gcrs
sun = get_body_heliographic_stonyhurst('sun', aiamap.date)
Let’s plot the results. The green circle represents the Earth. This looks like the Earth is in the way of SDO’s field of view but remember that it is also above the plane of this plot by its declination.
fig = plt.figure()
ax = fig.add_subplot(projection='polar')
circle = plt.Circle((0.0, 0.0), 1.0, transform=ax.transProjectionAffine + ax.transAxes, color="green",
alpha=0.4, label="Earth")
ax.add_artist(circle)
ax.text(0.48, 0.5, "Earth", transform=ax.transAxes)
ax.plot(sdo_gcrs.ra.to('rad'), sdo_gcrs.distance / R_earth, 'o', label=f'SDO {sdo_gcrs.dec:.2f}')
ax.plot(sun.lon.to('rad').value * np.ones(2), [1, 10], '-', label='to Sun', color='black')
ax.legend()
plt.show()
Total running time of the script: (0 minutes 0.277 seconds)
Note
Go to the end to download the full example code.
Identifying stars in a STEREO/SECCHI COR2 coronagraph image#
Since the field of view in SECCHI COR2 images can span 2 to 15 solar radii, we often observe stars in these image data. In this example, we will use the Astroquery package to query the VizieR star catalog for stars observed by the Gaia satellite within the SECCHI COR2 field of view. Then we will use the coordinates framework in SunPy and AstroPy to transform the coordinates returned by VizieR into SECCHI COR2 image coordinates. As a bonus, we’ll also identify Mars.”
This requires the installation of the astroquery
package, which can be installed on top of the existing sunpy conda
environment: conda install -c astropy astroquery and an active internet connection.
import hvpy
import matplotlib.pyplot as plt
from astroquery.vizier import Vizier
import astropy.units as u
from astropy.coordinates import Distance, SkyCoord
from astropy.time import Time
import sunpy.map
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.time import parse_time
from sunpy.util.config import get_and_create_download_dir
Let’s download a STEREO-A SECCHI COR2 image from Helioviewer.org which provide pre-processed images and load it into a Map. We download to the default sunpy download directory.
cor2_file = hvpy.save_file(hvpy.getJP2Image(parse_time('2014/05/15 07:54').datetime,
hvpy.DataSource.COR2_A.value),
get_and_create_download_dir() + "/COR2.jp2")
cor2_map = sunpy.map.Map(cor2_file)
To efficiently search the star field, we need to know what stars are near the Sun as observed by STEREO. We need the vector that points from STEREO to the Sun. The location of STEREO in HCRS provides the Sun-to-STEREO vector.
sun_to_stereo = cor2_map.observer_coordinate.transform_to('hcrs')
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
We next reflect the vector to get our search vector which points from STEREO to the Sun.
stereo_to_sun = SkyCoord(-sun_to_stereo.spherical, obstime=sun_to_stereo.obstime, frame='hcrs')
Let’s look up bright stars using the Vizier search capability provided by astroquery. We will search the GAIA2 star catalog for stars with magnitude brighter than 7.
vv = Vizier(columns=['**'], row_limit=-1, column_filters={'Gmag': '<7'}, timeout=1200)
vv.ROW_LIMIT = -1
result = vv.query_region(stereo_to_sun, radius=4 * u.deg, catalog='I/345/gaia2')
Let’s see how many stars we’ve found.
print(len(result[0]))
23
Now we load all stars into an array coordinate. The reference epoch for the
star positions is J2015.5,so we update these positions to the date of the
COR2 observation using astropy.coordinates.SkyCoord.apply_space_motion().
tbl_crds = SkyCoord(ra=result[0]['RA_ICRS'],
dec=result[0]['DE_ICRS'],
distance=Distance(parallax=u.Quantity(result[0]['Plx'])),
pm_ra_cosdec=result[0]['pmRA'],
pm_dec=result[0]['pmDE'],
radial_velocity=result[0]['RV'],
frame='icrs',
obstime=Time(result[0]['Epoch'], format='jyear'))
tbl_crds = tbl_crds.apply_space_motion(new_obstime=cor2_map.date)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/erfa/core.py:4998: RuntimeWarning: invalid value encountered in pmsafe
ra2, dec2, pmr2, pmd2, px2, rv2, c_retval = ufunc.pmsafe(
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "pmsafe" yielded 7 of "solution didn't converge (Note 8)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
One of the bright features is actually Mars, so let’s also get that coordinate.
mars = get_body_heliographic_stonyhurst('mars', cor2_map.date, observer=cor2_map.observer_coordinate)
INFO: Apparent body location accounts for 1269.95 seconds of light travel time [sunpy.coordinates.ephemeris]
Let’s plot the results. The coordinates will be transformed automatically
when plotted using plot_coord().
fig = plt.figure()
ax = fig.add_subplot(projection=cor2_map)
# Let's tweak the axis to show in degrees instead of arcsec
lon, lat = ax.coords
lon.set_major_formatter('d.dd')
lat.set_major_formatter('d.dd')
cor2_map.plot(axes=ax, vmin=0, vmax=600)
cor2_map.draw_limb(axes=ax)
# Plot the position of Mars
ax.plot_coord(mars, 's', color='white', fillstyle='none', markersize=12, label='Mars')
# Plot all of the stars
ax.plot_coord(tbl_crds, 'o', color='white', fillstyle='none')
ax.legend()
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Total running time of the script: (0 minutes 10.038 seconds)
Note
Go to the end to download the full example code.
Obtaining a spacecraft trajectory from JPL Horizons#
This example shows how to obtain the trajectory of a spacecraft from JPL Horizons and plot it relative to other bodies in the solar system.
JPL Horizons can return the locations of planets and minor bodies (e.g., asteroids) in the solar system, and it can also return the location of a variety of major spacecraft.
import matplotlib.pyplot as plt
import astropy.units as u
from sunpy.coordinates import get_body_heliographic_stonyhurst, get_horizons_coord
from sunpy.time import parse_time
We use get_horizons_coord() to query JPL Horizons
for the trajectory of Parker Solar Probe (PSP). Let’s request 50 days on
either side of PSP’s 14th closest approach to the Sun.
perihelion_14 = parse_time('2022-12-11 13:16')
psp = get_horizons_coord('Parker Solar Probe',
{'start': perihelion_14 - 50 * u.day,
'stop': perihelion_14 + 50 * u.day,
'step': '180m'})
INFO: Obtained JPL HORIZONS location for Parker Solar Probe (spacecraft) (-96) [sunpy.coordinates.ephemeris]
We also obtain the location of Earth at PSP perihelion. We could query
JPL Horizons again, but get_body_heliographic_stonyhurst() returns
a comparably accurate location using the Astropy ephemeris.
earth = get_body_heliographic_stonyhurst('Earth', perihelion_14)
For the purposes of plotting on a Matplotlib polar plot, we create a short convenience function to extract the necessary values in the appropriate units.
def coord_to_polar(coord):
return coord.lon.to_value('rad'), coord.radius.to_value('AU')
Finally, we plot the trajectory on a polar plot. Be aware that the
orientation of the Stonyhurst heliographic coordinate system rotates
over time such that the Earth is always at zero longitude.
Accordingly, when we directly plot the trajectory, it does not appear
as a simple ellipse because each trajectory point has a different
observation time and thus a different orientation of the coordinate
system. To see the elliptical orbit, the trajectory can be
transformed to the coordinate frame of Earth at the single time of
PSP perihelion (earth), so that the trajectory is represented in
a non-rotating coordinate frame.
fig = plt.figure()
ax = fig.add_subplot(projection='polar')
ax.plot(0, 0, 'o', label='Sun', color='orange')
ax.plot(*coord_to_polar(earth), 'o', label='Earth', color='blue')
ax.plot(*coord_to_polar(psp),
label='PSP (as seen from Earth)', color='purple')
ax.plot(*coord_to_polar(psp.transform_to(earth)),
label='PSP (non-rotating frame)', color='purple', linestyle='dashed')
ax.set_title('Stonyhurst heliographic coordinates')
ax.legend(loc='upper center')
plt.show()
There are other tools that enable a similar style of figure. solarmach is one such example.
Total running time of the script: (0 minutes 1.498 seconds)
Note
Go to the end to download the full example code.
Offsetting the north pole of a coordinate frame#
How to use NorthOffsetFrame to offset the north pole for a coordinate frame.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import NorthOffsetFrame
from sunpy.data.sample import AIA_171_IMAGE
For an AIA observation from the sample data, let’s create a modified Heliographic Stonyhurst coordinate frame with its north pole offset to a longitude of 45 degrees and a latitude of 20 degrees.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
north = SkyCoord(45*u.deg, 20*u.deg,
frame="heliographic_stonyhurst", obstime=aiamap.date)
new_frame = NorthOffsetFrame(north=north)
Let’s plot the normal Heliographic Stonyhurst coordinate frame as a white grid and the modified coordinate frame as a blue grid. If the north pole of this new frame were placed at a point of interest, the lines of longitude form great circles radially away from the point, and the lines of latitude measure angular distance from the point.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
aiamap.draw_grid(axes=ax)
overlay = ax.get_coords_overlay(new_frame)
overlay[0].set_ticks(spacing=30. * u.deg)
overlay.grid(ls='--', color='blue')
plt.show()
Total running time of the script: (0 minutes 1.009 seconds)
Note
Go to the end to download the full example code.
Overplotting the position of the Venus transit#
How to accurately plot the position of Venus as it transited in front of the Sun as observed by SDO/AIA.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord, solar_system_ephemeris
import sunpy.map
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.data.sample import AIA_1600_VENUS_IMAGE
Let’s use the sunpy sample data which has an image of the Venus transit.
aiamap = sunpy.map.Map(AIA_1600_VENUS_IMAGE)
For this example, we require high-precision ephemeris information. The built-in
ephemeris provided by astropy is not accurate enough. This call requires jplephem
to be installed. This will also trigger a download of about ~10 MB.
solar_system_ephemeris.set('de432s')
<ScienceState solar_system_ephemeris: 'de432s'>
Now we get the position of Venus and convert it into the SDO/AIA coordinates. The apparent position of Venus accounts for the time it takes for light to travel from Venus to SDO.
venus = get_body_heliographic_stonyhurst('venus', aiamap.date, observer=aiamap.observer_coordinate)
venus_hpc = venus.transform_to(aiamap.coordinate_frame)
INFO: Apparent body location accounts for 144.14 seconds of light travel time [sunpy.coordinates.ephemeris]
Let’s crop the image with Venus at its center.
fov = 200 * u.arcsec
bottom_left = SkyCoord(venus_hpc.Tx - fov/2, venus_hpc.Ty - fov/2, frame=aiamap.coordinate_frame)
smap = aiamap.submap(bottom_left, width=fov, height=fov)
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=smap)
smap.plot(axes=ax)
smap.draw_limb(axes=ax)
ax.grid(False)
ax.plot_coord(venus_hpc, 'x', color='deepskyblue', label='Venus')
ax.legend()
plt.show()
Total running time of the script: (0 minutes 1.810 seconds)
Note
Go to the end to download the full example code.
Setting the correct position for SOHO in a LASCO C3 Map#
How to get the correct location of SOHO using JPL HORIZONS and update the header.
import hvpy
import matplotlib.pyplot as plt
import numpy as np
import sunpy.map
from sunpy.coordinates.ephemeris import get_body_heliographic_stonyhurst, get_horizons_coord
from sunpy.time import parse_time
from sunpy.util.config import get_and_create_download_dir
Let’s download a SOHO/LASCO C3 image Helioviewer.org and load it into a Map. The reason to use Helioviewer.org is that they provide processed images. We download to the default sunpy download directory.
lasco_file = hvpy.save_file(hvpy.getJP2Image(parse_time('2000/02/27 07:42').datetime,
hvpy.DataSource.LASCO_C3.value),
get_and_create_download_dir() + "/LASCO_C3.jp2")
lasco_map = sunpy.map.Map(lasco_file)
A user warning let’s you know that there is missing metadata for the observer location. sunpy goes ahead and assumes that the observer is at Earth.
print(lasco_map.observer_coordinate)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/checkouts/latest/examples/units_and_coordinates/getting_lasco_observer_location.py:34: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
print(lasco_map.observer_coordinate)
<SkyCoord (HeliographicStonyhurst: obstime=2000-02-27T07:42:05.810, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, km)
(0., -7.18539124, 1.48139588e+08)>
To check that this worked let’s get the location of Mercury in this exposure and show that it is in the correct location.
mercury_wrong = get_body_heliographic_stonyhurst(
'mercury', lasco_map.date, observer=lasco_map.observer_coordinate)
mercury_hpc_wrong = mercury_wrong.transform_to(lasco_map.coordinate_frame)
print(mercury_hpc_wrong)
INFO: Apparent body location accounts for 330.72 seconds of light travel time [sunpy.coordinates.ephemeris]
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
<Helioprojective Coordinate (obstime=2000-02-27T07:42:05.810, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2000-02-27T07:42:05.810, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, m)
(0., -7.18539124, 1.48139588e+11)>): (Tx, Ty, distance) in (arcsec, arcsec, km)
(-23270.61112792, 13370.26466196, 99146796.1462121)>
Let’s plot how this looks with the incorrect observer information.
fig = plt.figure()
ax = fig.add_subplot(projection=lasco_map)
# Let's tweak the axis to show in degrees instead of arcsec
lon, lat = ax.coords
lon.set_major_formatter('d.dd')
lat.set_major_formatter('d.dd')
ax.plot_coord(mercury_hpc_wrong, 's', color='white',
fillstyle='none', markersize=12, label='Mercury')
lasco_map.plot(axes=ax)
plt.show()
SOHO is actually a halo orbit around the Sun–Earth L1 point, about 1 million km away from the Earth. The following functions queries JPL HORIZONS which includes positions of major spacecraft. This function requires an internet connection to fetch the ephemeris data.
soho = get_horizons_coord('SOHO', lasco_map.date)
INFO: Obtained JPL HORIZONS location for SOHO (spacecraft) (-21) [sunpy.coordinates.ephemeris]
For fun, let’s see how far away from the Earth SOHO is by converting to an Earth-centered coordinate system (GCRS).
print(soho.transform_to('gcrs').distance.to('km'))
1352065.7329907273 km
Let’s fix the header.
lasco_map.meta['HGLN_OBS'] = soho.lon.to('deg').value
lasco_map.meta['HGLT_OBS'] = soho.lat.to('deg').value
lasco_map.meta['DSUN_OBS'] = soho.radius.to('m').value
Let’s get the right position now.
mercury = get_body_heliographic_stonyhurst(
'mercury', lasco_map.date, observer=lasco_map.observer_coordinate)
mercury_hpc = mercury.transform_to(lasco_map.coordinate_frame)
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
INFO: Apparent body location accounts for 326.33 seconds of light travel time [sunpy.coordinates.ephemeris]
The difference between the incorrect position and the right one is:
r = np.sqrt((mercury_hpc.Tx - mercury_hpc_wrong.Tx) ** 2 +
(mercury_hpc.Ty - mercury_hpc_wrong.Ty) ** 2)
print(r)
0.4271595231009972 arcsec
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=lasco_map)
# Let's tweak the axis to show in degrees instead of arcsec
lon, lat = ax.coords
lon.set_major_formatter('d.dd')
lat.set_major_formatter('d.dd')
ax.plot_coord(mercury_hpc, 's', color='white', fillstyle='none', markersize=12, label='Mercury')
lasco_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 1.327 seconds)
Plotting#
Examples of visualizing supported data types
Overplotting SRS active region locations on a magnetograms
Note
Go to the end to download the full example code.
Blending maps using mplcairo#
This example shows how to blend two maps using mplcairo.
matplotlib by itself provides only alpha-based transparency for
superimposing one image onto another, which can be restrictive when trying to
create visually appealing composite images.
mplcairo is an enhancement for
matplotlib that provides support for
cairo’s compositing operators,
which include a wide range of
blend modes for image overlays.
Note
This example requires mplcairo
to be installed. Installation via pip will work in most cases, but you
may need to refer to
OS-specific installation notes.
We need to tell matplotlib to use a backend from mplcairo. The
backend formally needs to be set prior to importing matplotlib.pyplot.
The mplcairo.qt GUI backend should work on Linux and Windows, but
you will need to use something different on macOS or Jupyter (see
mplcairo’s usage notes).
import matplotlib
if matplotlib.get_backend() == "agg":
# This is the non-GUI backend for when building the documentation
matplotlib.use("module://mplcairo.base")
else:
# This is a GUI backend that you would normally use
matplotlib.use("module://mplcairo.qt")
We can now import everything else.
import matplotlib.pyplot as plt
from mplcairo import operator_t
import astropy.units as u
import sunpy.data.sample
import sunpy.map
from sunpy.coordinates import Helioprojective
Let’s load two maps for blending. We reproject the second map to the
coordinate frame of the first map for proper compositing, taking care to use
the assume_spherical_screen()
context manager in order to preserve off-disk data.
a171 = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
a131 = sunpy.map.Map(sunpy.data.sample.AIA_131_IMAGE)
with Helioprojective.assume_spherical_screen(a171.observer_coordinate):
a131 = a131.reproject_to(a171.wcs)
Let’s first plot the two maps individually.
fig1 = plt.figure(figsize=(10, 4))
ax1 = fig1.add_subplot(121, projection=a171)
ax2 = fig1.add_subplot(122, projection=a131)
a171.plot(axes=ax1, clip_interval=(1, 99.9995)*u.percent)
a131.plot(axes=ax2, clip_interval=(1, 99.95)*u.percent)
<matplotlib.image.AxesImage object at 0x7f6f4035d240>
We now plot the two maps on the same axes. If the plot were rendered at this
point, the second map would completely obscure the first map. We save the
matplotlib artist returned when plotting the second map (im131) for
future use.
fig2 = plt.figure()
ax = fig2.add_subplot(projection=a171)
a171.plot(axes=ax, clip_interval=(1, 99.9995)*u.percent)
im131 = a131.plot(axes=ax, clip_interval=(1, 99.95)*u.percent)
We invoke the mplcairo operator for the
screen blend mode
to modify the artist for the second map. The second map will
now be composited onto the first map using that blend mode.
operator_t.SCREEN.patch_artist(im131)
Finally, we set the title and render the plot.
ax.set_title('mplcairo composite using screen blending')
plt.show()
Total running time of the script: (0 minutes 3.104 seconds)
Note
Go to the end to download the full example code.
Combining off-limb and disk maps#
This example combines creating a composite plot with masking out the
solar disk. The purpose is to introduce users to the plotting process
required to overlay multiple maps that have been modified with other
sunpy functions. The resulting plot in this tutorial shows information
on the upper photosphere, quiet corona, and magnetogram contours.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
from sunpy.map import Map
from sunpy.map.maputils import all_coordinates_from_map, coordinate_is_on_solar_disk
Let’s import sample data representing the three types of data we want to overlay: AIA 1600 (upper photosphere), AIA 171 (quiet corona), and HMI data (magnetic fields in the photosphere).
aia_1600 = Map(sunpy.data.sample.AIA_1600_IMAGE)
aia_171 = Map(sunpy.data.sample.AIA_171_IMAGE)
hmi = Map(sunpy.data.sample.HMI_LOS_IMAGE)
Next, let’s mask out the solar disk of the AIA 171 image since we will be overlaying the AIA 1600 photosphere.
hpc_coords = all_coordinates_from_map(aia_171)
mask = coordinate_is_on_solar_disk(hpc_coords)
colormap = aia_171.cmap.copy()
colormap.set_bad('black')
scaled_map = Map(aia_171.data, aia_171.meta, mask=mask)
Before we plot the composite image, let’s define the contour levels that will be applied to the HMI image. Also, grab the AIA 171 colormap to use with the AIA 1600 image to better match the images.
levels = [-1000, -500, -250, 250, 500, 1000]*u.G
aia171_cmap = plt.get_cmap('sdoaia171')
Now, create the plot by overlaying the maps and contours. Note that
clip_interval is being defined to highlight the coronal features of the
scaled AIA 171 map and the AIA 1600 map is autoaligned to the AIA 171 WCS
Axes.
fig = plt.figure()
ax = fig.add_subplot(projection=scaled_map)
scaled_map.plot(axes=ax, clip_interval=(1, 99.95)*u.percent,
cmap=colormap, zorder=0)
aia_1600.plot(axes=ax, cmap=aia171_cmap, autoalign=True, zorder=1)
hmi.draw_contours(axes=ax, levels=levels, zorder=2)
ax.set_title("AIA 171 (off limb) + AIA 1600 (on disk) + HMI (contours)")
plt.show()
Total running time of the script: (0 minutes 3.876 seconds)
Note
Go to the end to download the full example code.
Creating a Composite Plot with Three Maps#
In this example, a composite plot is created with three maps. It demonstrates how to specify contour levels, transparency, and ordering when overlaying multiple maps.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
from sunpy.coordinates import Helioprojective
from sunpy.map import Map
First, we will import sample data from EIT, RHESSI, and AIA. The EIT data shows a hot region of the solar corona, while AIA shows the cooler upper region of the corona. RHESSI data is focused on a solar flare, and will be plotted using contours.
eit = Map(sunpy.data.sample.EIT_195_IMAGE)
rhessi = Map(sunpy.data.sample.RHESSI_IMAGE)
aia = Map(sunpy.data.sample.AIA_171_IMAGE)
Before we plot the image, let’s reproject the off-disk AIA coordinates onto a spherical screen at the same distance as the EIT map, so they can be overlaid. Next, zoom in around the solar flare so the RHESSI contours are visible. Also, specify the RHESSI contour levels to be plotted.
with Helioprojective.assume_spherical_screen(eit.observer_coordinate):
aia = aia.reproject_to(eit.wcs)
bottom_left = [200, -800] * u.arcsec
top_right = [1000, -200] * u.arcsec
eit_smap = eit.submap(SkyCoord(*bottom_left, frame=eit.coordinate_frame),
top_right=SkyCoord(*top_right, frame=eit.coordinate_frame))
aia_smap = aia.submap(SkyCoord(*bottom_left, frame=aia.coordinate_frame),
top_right=SkyCoord(*top_right, frame=aia.coordinate_frame))
levels = [5, 10, 20, 30, 40]*u.percent
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:2757: SunpyUserWarning: rsun mismatch detected: AIA 171.0 Angstrom 2011-06-07 06:33:02.rsun_meters=696000000.0 m; 2011-06-07 20:37:52.rsun_meters=696310061.3753 m. This might cause unexpected results during reprojection.
warn_user("rsun mismatch detected: "
When creating the plot, choose which Map will be used to create the WCS Axes
that the other Maps will be plotted with. The EIT map is plotted first,
followed by the AIA map, and lastly the RHESSI contours. Transparency is
changed to 70% on the AIA map by specifying the parameter alpha, and the
image data is autoaligned to the EIT WCS Axes. The parameter zorder
specifies how each plot is layered (0 is plotted first and 1 is layered on
top of 0, and so on).
fig = plt.figure()
ax = fig.add_subplot(projection=eit_smap)
eit_smap.plot(axes=ax, clip_interval=(1, 99.9)*u.percent, zorder=0)
aia_smap.plot(axes=ax, clip_interval=(1, 99.97)*u.percent, alpha=0.7,
autoalign=True, zorder=1)
rhessi.draw_contours(axes=ax, levels=levels, zorder=2)
ax.set_title("Three Map Composite")
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:631: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obs_coord = self.observer_coordinate
Total running time of the script: (0 minutes 2.709 seconds)
Note
Go to the end to download the full example code.
Drawing a latitude-longitude quadrangle#
How to draw a latitude-longitude quadrangle on a map.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import HeliographicStonyhurst
from sunpy.data.sample import AIA_171_IMAGE
The purpose of this example is to demonstrate how to draw a quadrangle on a
map using draw_quadrangle(). A quadrangle has
edges aligned with lines of constant latitude and longitude in some
coordinate system. We start with the sample AIA image.
aia = sunpy.map.Map(AIA_171_IMAGE)
Now let’s define the bottom-left corner of the quadrangle. Note that we define it in a different coordinate frame (heliographic Stonyhurst) than the coordinate frame of the map (which is helioprojective Cartesian).
bottom_left = SkyCoord(30*u.deg, -10*u.deg,
frame=HeliographicStonyhurst, obstime=aia.date)
Now let’s draw a quadrangle on the map, with a width of 20 degrees and a
height of 60 degrees. Because the coordinate frame of bottom_left is
in HeliographicStonyhurst, the width and height
correspond to the longitude and latitude directions, respectively, in that
coordinate frame.
fig = plt.figure()
ax = fig.add_subplot(projection=aia)
aia.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
aia.draw_grid(axes=ax)
aia.draw_quadrangle(bottom_left, axes=ax, width=20*u.deg, height=60*u.deg,
edgecolor='blue', linewidth=2)
ax.plot_coord(bottom_left, 'x', color='red')
plt.show()
Total running time of the script: (0 minutes 0.694 seconds)
Note
Go to the end to download the full example code.
Drawing a rotated rectangle on a map#
This example will demonstrate how to draw a rectangle that is rotated relative
to the axes on a map using draw_quadrangle().
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord, SkyOffsetFrame
import sunpy.data.sample
import sunpy.map
Let’s start with a sample image of AIA 171.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
Define the rotation angle and center coordinate of the rectangle, as well as the width and height in physical units.
rotation_angle = 30 * u.deg
center_coord = SkyCoord(-100 * u.arcsec, -300 * u.arcsec, frame=aia_map.coordinate_frame)
width = 1000 * u.arcsec
height = 300 * u.arcsec
We define a custom coordinate frame for the rotated rectangle by providing
the center in the AIA 171 coordinate frame and rotation angle to
SkyOffsetFrame. We then define a 2-element
SkyCoord in that custom coordinate frame for the
bottom-left and top-right corners of the rectangle.
offset_frame = SkyOffsetFrame(origin=center_coord, rotation=rotation_angle)
rectangle = SkyCoord(lon=[-1/2, 1/2] * width, lat=[-1/2, 1/2] * height, frame=offset_frame)
Finally, we will draw the rotated rectangle and its center.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99) * u.percent)
ax.plot_coord(center_coord, "o", color="red")
aia_map.draw_quadrangle(
rectangle,
axes=ax,
edgecolor="red",
linestyle="--",
linewidth=2,
)
plt.show()
Total running time of the script: (0 minutes 0.387 seconds)
Note
Go to the end to download the full example code.
Drawing and using a Great Arc#
How to define and draw a great arc on an image of the Sun, and to extract intensity values along that arc.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.visualization import quantity_support
import sunpy.map
from sunpy.coordinates.utils import GreatArc
from sunpy.data.sample import AIA_171_IMAGE
quantity_support()
<astropy.visualization.units.quantity_support.<locals>.MplQuantityConverter object at 0x7f6f406e88b0>
We start with the sample data.
m = sunpy.map.Map(AIA_171_IMAGE)
Let’s define the start and end coordinates of the arc.
start = SkyCoord(735 * u.arcsec, -471 * u.arcsec, frame=m.coordinate_frame)
end = SkyCoord(-100 * u.arcsec, 800 * u.arcsec, frame=m.coordinate_frame)
Create the great arc between the start and end points.
great_arc = GreatArc(start, end)
Plot the great arc on the Sun.
fig = plt.figure()
ax = fig.add_subplot(projection=m)
m.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
ax.plot_coord(great_arc.coordinates(), color='c')
[<matplotlib.lines.Line2D object at 0x7f6f3b5dbdf0>]
Now we can get the intensity along the great arc coordinates, along with the angular distance from the start of the arc
coords = great_arc.coordinates()
intensity_coords = sunpy.map.pixelate_coord_path(m, coords)
intensity = sunpy.map.sample_at_coords(m, intensity_coords)
separation = intensity_coords.separation(intensity_coords[0]).to(u.arcsec)
Plot the intensity along the arc from the start to the end point.
fig, ax = plt.subplots()
ax.plot(separation, intensity)
ax.set_xlabel(f'Separation from start of arc [{separation.unit}]')
ax.set_ylabel(f'Intensity [{intensity.unit}]')
plt.show()
Total running time of the script: (0 minutes 0.535 seconds)
Note
Go to the end to download the full example code.
Drawing heliographic longitude and latitude lines#
This example demonstrates how you can draw individual Stonyhurst longitude and latitude lines.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import frames
from sunpy.data.sample import AIA_171_IMAGE
We will first demonstrate the coordinate transformations that occur under the hood to show a heliographic grid line of longitude or latitude. Let’s start with a map using the sample data.
aia = sunpy.map.Map(AIA_171_IMAGE)
We can define a heliographic coordinate for a single point.
lon_value = 35 * u.deg
lat_value = 12 * u.deg
stonyhurst_frame = frames.HeliographicStonyhurst(obstime=aia.date)
point_in_stonyhurst = SkyCoord(lon=lon_value, lat=lat_value, frame=stonyhurst_frame)
Next we transform it into the coordinate frame of our map, which is helioprojective.
point_in_hpc = point_in_stonyhurst.transform_to(aia.coordinate_frame)
print(point_in_hpc)
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406234, 0.04787238, 1.51846026e+11)>): (Tx, Ty, distance) in (arcsec, arcsec, km)
(532.2061827, 196.57024961, 1.51289075e+08)>
Now let’s define two lines, one of constant longitude and one of constant latitude, that pass through the previously defined coordinate point. We don’t need to explicitly transform them to the coordinate frame of our map because they will be transformed automatically when plotted.
num_points = 100
constant_lon = SkyCoord(lon_value, np.linspace(-90, 90, num_points) * u.deg,
frame=stonyhurst_frame)
constant_lat = SkyCoord(np.linspace(-90, 90, num_points) * u.deg, lat_value,
frame=stonyhurst_frame)
Now let’s plot the single coordinate point and the individual lines of constant longitude and latitude. We’ll overlay the autogenerated lon/lat grid as well for comparison.
fig = plt.figure()
ax = fig.add_subplot(projection=aia)
aia.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
ax.plot_coord(constant_lon, color="lightblue")
ax.plot_coord(constant_lat, color="tomato")
ax.plot_coord(point_in_stonyhurst, marker="o")
aia.draw_grid(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.703 seconds)
Note
Go to the end to download the full example code.
Editing the colormap and normalization of a Map#
How to edit the display of a map.
import matplotlib
import matplotlib.colors as colors
import matplotlib.pyplot as plt
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
All plot settings for a map are stored in the plot_settings attribute.
How a Map is displayed is determined by its colormap, which sets the colors
, and the normalization, which sets how data values are translated to colors.
Let’s replace the colormap and normalization.
aiamap.plot_settings['cmap'] = matplotlib.colormaps['Greys_r']
aiamap.plot_settings['norm'] = colors.LogNorm(100, aiamap.max())
To see all of the colormaps sunpy provides see sunpy.visualization.colormaps.
Matplotlib provides a number of colormaps
and normalizations.
For more advanced normalizations see astropy.visualization.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
plt.colorbar()
plt.show()
Total running time of the script: (0 minutes 0.510 seconds)
Note
Go to the end to download the full example code.
Fading between two maps#
Often it is useful to plot two maps of the Sun on top of each other, so features observed in one map (e.g. strong magnetic fields in a magnetogram) can be identified with features in the same place in another map (e.g. active regions in a EUV image).
This example shows how to plot two maps on top of each other, with a slider to fade between them. Note that it assumes that the two maps are taken at the same time from the same observation point.
import matplotlib.pyplot as plt
from matplotlib.widgets import Slider
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE, AIA_1600_IMAGE
Start by loading two AIA maps from the sample data.
Note that these maps must be co-aligned. If they are taken at different times
or from different observation locations they will need to be aligned using
rotate or reproject.
map_171 = sunpy.map.Map(AIA_171_IMAGE)
map_1600 = sunpy.map.Map(AIA_1600_IMAGE)
First we create a figure, and add the axes that will show the maps. Then plot both of the images on the same axes. Finally, we add another axes that contains the slider.
fig = plt.figure()
# Add the main axes. Note this is resized to leave room for the slider axes
ax = fig.add_axes([0.1, 0.2, 0.9, 0.7], projection=map_171)
im_1600 = map_1600.plot(axes=ax, autoalign=True)
im_171 = map_171.plot(axes=ax, alpha=0.5, clip_interval=(1, 99.99)*u.percent)
ax.set_title('AIA 171 + 1600')
# Add the slider axes
ax_slider = fig.add_axes([0.25, 0.05, 0.65, 0.03])
slider = Slider(ax_slider, 'Alpha', 0, 1, valinit=0.5)
Finally, define what happens when the slider is changed and link this to the slider we set up above. In this case we just set the alpha (ie. transparency) of the 171 image.
def update(val):
alpha = slider.val
im_171.set_alpha(alpha)
slider.on_changed(update)
plt.show()
Total running time of the script: (0 minutes 3.689 seconds)
Note
Go to the end to download the full example code.
Finding Local Peaks in Solar Data#
Detecting intensity peaks in solar images can be useful, for example as
a simple flare identification mechanism. This example illustrates detection
of areas where there is a spike in solar intensity.
We use the peak_local_max function in the scikit-image library
to find those regions in the map data where the intensity values form a local maxima.
Then we plot those peaks in the original AIA plot.
import matplotlib.pyplot as plt
from skimage.feature import peak_local_max
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_193_IMAGE
from sunpy.map.maputils import all_pixel_indices_from_map
We will first create a Map using some sample data and display it.
aiamap = sunpy.map.Map(AIA_193_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
plt.colorbar()
<matplotlib.colorbar.Colorbar object at 0x7f6f402565c0>
Before we find regions of local maxima, we need to create some variables to store pixel values for the 2D SDO/AIA data we are using. These variables are used for plotting in 3D later on.
X, Y = all_pixel_indices_from_map(aiamap)
We will only consider peaks within the AIA data that have minimum intensity
value equal to threshold_rel * max(Intensity) which is 20% of the maximum intensity.
The next step is to calculate the pixel locations of local maxima
positions where peaks are separated by at least min_distance = 60 pixels.
This function comes from scikit image and the documentation is found
here peak_local_max.
coordinates = peak_local_max(aiamap.data, min_distance=60, threshold_rel=0.2)
We now check for the indices at which we get such a local maxima and plot those positions marked red in the aiamap data.
fig = plt.figure(figsize=(12, 8))
ax = fig.add_subplot(projection='3d')
ax.plot_surface(X, Y, aiamap.data)
ax.view_init(elev=39, azim=64)
peaks_pos = aiamap.data[coordinates[:, 0], coordinates[:, 1]]
ax.scatter(coordinates[:, 1], coordinates[:, 0], peaks_pos, color='r')
ax.set_xlabel('X Coordinates')
ax.set_ylabel('Y Coordinates')
ax.set_zlabel('Intensity')
plt.show()
Now we need to turn the pixel coordinates into the world location so they can be easily overlaid on the Map.
hpc_max = aiamap.wcs.pixel_to_world(coordinates[:, 1]*u.pixel, coordinates[:, 0]*u.pixel)
Finally we do an AIA plot to check for the local maxima locations which will be marked with a blue x-label.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
ax.plot_coord(hpc_max, 'bx')
plt.show()
Total running time of the script: (0 minutes 1.219 seconds)
Note
Go to the end to download the full example code.
Fine grained Plotting Features of Map#
How to control various plotting features of map.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
SkyCoord module provides flexible celestial coordinate representation and a draw_limb method draws an edge as seen against the dark sky background. Date of the image taken can also be displayed in the plot.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
bottom_left = SkyCoord(-400*u.arcsec, -900*u.arcsec, frame=aiamap.coordinate_frame)
top_right = SkyCoord(800*u.arcsec, 700*u.arcsec, frame=aiamap.coordinate_frame)
aiamap_sub = aiamap.submap(bottom_left, top_right=top_right)
title_obsdate = aiamap_sub.date.strftime('%Y-%b-%d %H:%M:%S')
The sunpy map peek method shows a helioprojective grid by default. This is sometimes not desired, and instead a heliographic Stonyhurst grid might be required. Although maps offer a simple way to include a Stonyhurst grid, often times we want more control over the Stonyhurst type. The example below illustrates some aspects of how a map plot can be altered as desired. For more information regarding the axis and grid settings, go to Ticks, tick labels and grid lines in astropy.
fig = plt.figure(figsize=(6, 6))
ax = fig.add_subplot(projection=aiamap_sub)
aiamap_sub.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
aiamap_sub.draw_limb(axes=ax, color='white', linewidth=2, linestyle='dashed')
# To have more control over the Heliographic Stonyhurst grid,
# the axes object properties can be changed directly
overlay = ax.get_coords_overlay('heliographic_stonyhurst')
lon = overlay[0]
lat = overlay[1]
lon.set_ticks_visible(False)
lat.set_ticks_visible(False)
lat.set_ticklabel_visible(False)
lon.set_ticklabel_visible(False)
lon.coord_wrap = 180 * u.deg
lon.set_major_formatter('dd')
# Plot the Heliographic Stonyhurst grid
overlay.grid(color='tab:blue', linewidth=2, linestyle='dashed')
# Switch off the helioprojective grid
ax.grid(False)
# Change how the helioprojective grid tick labels are formatted
tx, ty = ax.coords
# Use integer coordinates for either axis.
tx.set_major_formatter('s')
ty.set_major_formatter('s')
ax.set_title(fr'AIA 171 $\AA$ {title_obsdate}')
ax.set_ylabel('Helioprojective Latitude [arcsec]')
ax.set_xlabel('Helioprojective Longitude [arcsec]')
plt.colorbar(fraction=0.045, pad=0.03, label='DN', ax=ax)
plt.show()
Total running time of the script: (0 minutes 0.906 seconds)
Note
Go to the end to download the full example code.
Imshow and maps coordinates#
How to use imshow with a map and overplot points specified by pixel coordinates and map coordinates.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data and create a submap of a smaller region.
aia = sunpy.map.Map(AIA_171_IMAGE)
top_right = SkyCoord(0 * u.arcsec, 1000 * u.arcsec, frame=aia.coordinate_frame)
bottom_left = SkyCoord(-1000 * u.arcsec, 0 * u.arcsec, frame=aia.coordinate_frame)
smap = aia.submap(bottom_left, top_right=top_right)
By setting the projection of the axis, a WCSAxes is created and this enables the plot to be created with the map coordinates, in this case helioprojective coordinates. The standard plot command expects pixel coordinates. It is also possible to plot a point using the map coordinates if we pass the transformation but this transform expects the values to be in degrees and not arcseconds.
fig = plt.figure()
ax = fig.add_subplot(projection=smap)
ax.imshow(smap.data)
<matplotlib.image.AxesImage object at 0x7f6f4022aad0>
Add an overlay grid, showing the native coordinates of the map.
ax.coords.grid(color='yellow', linestyle='solid', alpha=0.5)
plot a point in the middle of the image
pixel_coord = [smap.data.shape[0]/2., smap.data.shape[1]/2.] * u.pix
ax.plot(pixel_coord[0], pixel_coord[1], 'x', color='white',
label=f'Pixel coordinate [{pixel_coord[0]}, {pixel_coord[1]}]')
[<matplotlib.lines.Line2D object at 0x7f6f3bf2ca00>]
Using the transform command expects coordinates in degrees and not arcseconds.
map_coord = ([-300, 200] * u.arcsec)
ax.plot(map_coord[0].to('deg'), map_coord[1].to('deg'), 'o', color='white',
transform=ax.get_transform('world'),
label=f'Map coordinate [{map_coord[0]}, {map_coord[1]}]')
ax.legend()
plt.show()
Total running time of the script: (0 minutes 0.379 seconds)
Note
Go to the end to download the full example code.
Loading an HMI synoptic map#
In this example we load a synoptic map produced by the HMI team. This data is an interesting demonstration of sunpy’s Map class as it is not in the more common Helioprojective coordinate system, but in heliographic Carrington coordinates and a cylindrical equal area (CEA) projection.
import matplotlib.pyplot as plt
from astropy.utils.data import download_file
import sunpy.map
Download the file and read it into a Map.
filename = download_file(
'http://jsoc.stanford.edu/data/hmi/synoptic/hmi.Synoptic_Mr.2191.fits', cache=True)
syn_map = sunpy.map.Map(filename)
Plot the results.
fig = plt.figure(figsize=(12, 5))
ax = plt.subplot(projection=syn_map)
im = syn_map.plot(axes=ax)
ax.coords[0].set_axislabel("Carrington Longitude [deg]")
ax.coords[1].set_axislabel("Latitude [deg]")
ax.coords.grid(color='black', alpha=0.6, linestyle='dotted', linewidth=0.5)
cb = plt.colorbar(im, fraction=0.019, pad=0.1)
cb.set_label(f"Radial magnetic field [{syn_map.unit}]")
# In order to make the x-axis ticks show, the bottom y-limit has to be adjusted slightly
ax.set_ylim(bottom=0)
ax.set_title(f"{syn_map.meta['content']},\n"
f"Carrington rotation {syn_map.meta['CAR_ROT']}")
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:631: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obs_coord = self.observer_coordinate
Total running time of the script: (0 minutes 2.042 seconds)
Note
Go to the end to download the full example code.
Overlay an AIA image on a LASCO C2 coronagraph#
This example shows the steps needed to overlay the disc and off-limb components of an AIA image within the masked occulter of a LASCO C2 image.
from datetime import datetime
import hvpy
import matplotlib.pyplot as plt
from hvpy.datasource import DataSource
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
from sunpy.coordinates import Helioprojective
from sunpy.map import Map
from sunpy.util.config import get_and_create_download_dir
First, we will acquire a calibrated LASCO C2 image from Helioviewer and create a map. hvpy uses the standard datetime instead of astropy.time.
lasco_jp2_file = hvpy.save_file(hvpy.getJP2Image(datetime(2011, 6, 7, 6, 34),
DataSource.LASCO_C2.value),
filename=get_and_create_download_dir() + "/LASCO_C2.jp2", overwrite=True)
lasco_map = Map(lasco_jp2_file)
aia_map = Map(sunpy.data.sample.AIA_171_IMAGE)
In order to plot off-limb features of the AIA image.
Note that off-disk AIA data are not retained by default because an
additional assumption is required to define the location of the AIA
emission in 3D space. We can use
assume_spherical_screen() to
retain the off-disk AIA data. See
Reprojecting Using a Spherical Screen
for more reference.
projected_coord = SkyCoord(0*u.arcsec, 0*u.arcsec,
obstime=lasco_map.observer_coordinate.obstime,
frame='helioprojective',
observer=lasco_map.observer_coordinate,
rsun=aia_map.coordinate_frame.rsun)
projected_header = sunpy.map.make_fitswcs_header(aia_map.data.shape,
projected_coord,
scale=u.Quantity(aia_map.scale),
instrument=aia_map.instrument,
wavelength=aia_map.wavelength)
# We use `assume_spherical_screen` to ensure that the off limb AIA pixels are reprojected
# otherwise it will only be the on disk pixels that are reprojected.
with Helioprojective.assume_spherical_screen(aia_map.observer_coordinate):
aia_reprojected = aia_map.reproject_to(projected_header)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/checkouts/latest/examples/plotting/lasco_overlay.py:45: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obstime=lasco_map.observer_coordinate.obstime,
Finally, we plot the images by layering the AIA image on top of the LASCO C2 image.
fig = plt.figure()
ax = fig.add_subplot(projection=lasco_map)
lasco_map.plot(axes=ax)
aia_reprojected.plot(axes=ax, clip_interval=(1, 99.9)*u.percent, autoalign=True)
ax.set_title("AIA and LASCO C2 Overlay")
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
Total running time of the script: (0 minutes 3.981 seconds)
Note
Go to the end to download the full example code.
Overlaying Two Maps#
This example demonstrates how to draw contours of one map on top of another to compare features.
import matplotlib.colors
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
from sunpy.map import Map
We start with the sample data. HMI shows the line-of-sight magnetic field at the photosphere while AIA 171 images show the resulting magnetic fields filled with hot plasma above, in the corona. We want to see what coronal features overlap with regions of strong line-of-sight magnetic fields.
aia_map = Map(sunpy.data.sample.AIA_171_IMAGE)
hmi_map = Map(sunpy.data.sample.HMI_LOS_IMAGE)
bottom_left = [0, 0] * u.arcsec
top_right = [800, 800] * u.arcsec
aia_smap = aia_map.submap(SkyCoord(*bottom_left,
frame=aia_map.coordinate_frame),
top_right=SkyCoord(*top_right,
frame=aia_map.coordinate_frame))
hmi_smap = hmi_map.submap(SkyCoord(*bottom_left,
frame=hmi_map.coordinate_frame),
top_right=SkyCoord(*top_right,
frame=hmi_map.coordinate_frame))
Let’s set the contours of the HMI map from a few hundred to a thousand Gauss, which is the typical field strength associated with umbral regions of pores and sunspots.
levels = [-1000, -500, -250, 250, 500, 1000] * u.G
Now let’s look at the result. Notice that we can see the coronal structures present on the AIA image and how they correspond to the line of sight magnetic field.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_smap)
aia_smap.plot(axes=ax)
hmi_smap.draw_contours(axes=ax, levels=levels, cmap="viridis")
norm = matplotlib.colors.Normalize(vmin=-1000, vmax=1000)
cbar = fig.colorbar(plt.cm.ScalarMappable(norm=norm, cmap="viridis"), ax=ax)
cbar.ax.set_title("Gauss", fontsize=10)
ax.set_title("AIA & HMI Composite Plot")
plt.show()
Total running time of the script: (0 minutes 0.699 seconds)
Note
Go to the end to download the full example code.
Overplotting HEK feature/event polygons on a map#
How to overplot HEK outlines on a map.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta
import sunpy.data.sample
import sunpy.map
from sunpy.coordinates import frames
from sunpy.net import attrs as a
from sunpy.net import hek
from sunpy.physics.differential_rotation import solar_rotate_coordinate
from sunpy.time import parse_time
We start with the sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
Look for coronal holes detected using the SPoCA feature recognition method:
hek_client = hek.HEKClient()
start_time = aia_map.date - TimeDelta(2*u.hour)
end_time = aia_map.date + TimeDelta(2*u.hour)
responses = hek_client.search(a.Time(start_time, end_time),
a.hek.CH, a.hek.FRM.Name == 'SPoCA')
Let’s find the biggest coronal hole within 80 degrees north/south of the equator:
area = 0.0
for i, response in enumerate(responses):
if response['area_atdiskcenter'] > area and np.abs(response['hgc_y']) < 80.0:
area = response['area_atdiskcenter']
response_index = i
Next let’s get the boundary of the coronal hole.
ch = responses[response_index]
p1 = ch["hpc_boundcc"][9:-2]
p2 = p1.split(',')
p3 = [v.split(" ") for v in p2]
ch_date = parse_time(ch['event_starttime'])
The coronal hole was detected at different time than the AIA image was taken so we need to rotate it to the map observation time.
ch_boundary = SkyCoord(
[(float(v[0]), float(v[1])) * u.arcsec for v in p3],
obstime=ch_date, observer="earth",
frame=frames.Helioprojective)
rotated_ch_boundary = solar_rotate_coordinate(ch_boundary, time=aia_map.date)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/physics/differential_rotation.py:152: SunpyUserWarning: Using 'time' assumes an Earth-based observer.
warn_user("Using 'time' assumes an Earth-based observer.")
Now let’s plot the rotated coronal hole boundary on the AIA map, and fill it with hatching.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
ax.plot_coord(rotated_ch_boundary, color='c')
ax.set_title('{:s}\n{:s}'.format(aia_map.name, ch['frm_specificid']))
plt.colorbar()
plt.show()
Total running time of the script: (0 minutes 1.361 seconds)
Note
Go to the end to download the full example code.
Overplotting SRS active region locations on a magnetograms#
How to find and plot the location of an active region on an HMI magnetogram.
import matplotlib.pyplot as plt
import numpy as np
import sunpy.coordinates
import sunpy.data.sample
import sunpy.map
from sunpy.io.special import srs
For this example, we will start with the sample data. We need an HMI file and
use it to create a map, and the SRS table which contains a list of active
regions. Both of these data can be downloaded with Fido.
smap = sunpy.map.Map(sunpy.data.sample.HMI_LOS_IMAGE)
srs_table = srs.read_srs(sunpy.data.sample.SRS_TABLE)
We only need the rows which have ‘ID’ = ‘I’ or ‘IA’.
srs_table = srs_table[np.logical_or(srs_table['ID'] == 'I', srs_table['ID'] == 'IA')]
Now we extract the latitudes, longitudes and the region numbers.
lats = srs_table['Latitude']
lngs = srs_table['Longitude']
numbers = srs_table['Number']
Let’s plot the results by defining coordinates for each location.
fig = plt.figure()
ax = fig.add_subplot(projection=smap)
# Passing vmin/vmax to ``plot`` does not work since
# a normalisation is set on the map. So we have to
# work around it like so:
smap.plot_settings["norm"].vmin = -150
smap.plot_settings["norm"].vmax = 150
smap.plot(axes=ax)
smap.draw_limb(axes=ax)
# Add a text box and arrow pointing to each active region
lat_text = -40
transparent_white = (1, 1, 1, 0.5)
for num, lng, lat in zip(numbers, lngs.value, lats.value):
ax.annotate(num, (lng, lat),
xytext=(320, lat_text),
xycoords=ax.get_transform('heliographic_stonyhurst'),
backgroundcolor=transparent_white,
color='red',
fontweight='bold',
arrowprops=dict(facecolor=transparent_white, width=1, headwidth=10),
horizontalalignment='right', verticalalignment='top')
lat_text += 10
plt.show()
Total running time of the script: (0 minutes 0.659 seconds)
Note
Go to the end to download the full example code.
Plot positions on a blank map#
This example showcases how to plot positions on a blank map. It is often useful to plot coordinate positions of events on a blank helioprojective coordinate map. In this example, we create an empty map with a WCS defined by a helioprojective frame as observed from Earth at a certain time, and show how you can plot different coordinates on it.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import frames
First we will create a blank map using with an array of zeros. Since there is no WCS information, we will need to construct a header to pass to Map.
data = np.full((10, 10), np.nan)
# Define a reference coordinate and create a header using sunpy.map.make_fitswcs_header
skycoord = SkyCoord(0*u.arcsec, 0*u.arcsec, obstime='2013-10-28',
observer='earth', frame=frames.Helioprojective)
# Scale set to the following for solar limb to be in the field of view
header = sunpy.map.make_fitswcs_header(data, skycoord, scale=[220, 220]*u.arcsec/u.pixel)
# Use sunpy.map.Map to create the blank map
blank_map = sunpy.map.Map(data, header)
Now we have constructed the map, we can plot it and mark important locations to it. Initialize the plot and add the map to it
fig = plt.figure()
ax = fig.add_subplot(projection=blank_map)
blank_map.plot(axes=ax)
blank_map.draw_limb(axes=ax, color="k")
blank_map.draw_grid(axes=ax, color="k")
<CoordinatesMap with 2 world coordinates:
index aliases type unit wrap format_unit visible
----- ------- --------- ---- --------- ----------- -------
0 lon longitude deg 180.0 deg deg yes
1 lat latitude deg None deg yes
>
Coordinates that are being plotted - (0, 0), (50, 100) and (400, 400).
xc = [0, 50, 400] * u.arcsec
yc = [0, 100, 400] * u.arcsec
Plot the blank map with the specified coordinates. Note that the marker for
(0, 0) in helioprojective coordinates is at the center of the solar disk, yet
the heliographic equator (zero degrees latitude) does not go through the disk
center and instead curves below it. The reason for that is the observer,
specified as Earth in this example, is almost always at non-zero heliographic
latitude, and disk center as seen by such an observer will have that same
heliographic latitude. The B0() function returns
Earth’s heliographic latitude at a specified time.
coords = SkyCoord(xc, yc, frame=blank_map.coordinate_frame)
ax.plot_coord(coords, 'o')
ax.set_title('Plotting fixed points on a blank map')
plt.show()
Total running time of the script: (0 minutes 0.555 seconds)
Note
Go to the end to download the full example code.
Plotting a coordinate grid#
This example demonstrates how you can draw a grid of solar coordinates on top of a map.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.data.sample
import sunpy.map
Let’s start with a sample AIA image.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
Now we can plot the image, and add grids in heliographic Stonyhurst and heliographic Carrington coordinates.
fig = plt.figure()
ax1 = fig.add_subplot(121, projection=aia_map)
aia_map.plot(axes=ax1, clip_interval=(1, 99.9)*u.percent)
stonyhurst_grid = aia_map.draw_grid(axes=ax1, system='stonyhurst')
ax2 = fig.add_subplot(122, projection=aia_map)
aia_map.plot(axes=ax2, clip_interval=(1, 99.9)*u.percent)
carrington_grid = aia_map.draw_grid(axes=ax2, system='carrington')
for ax in [ax1, ax2]:
ax.set_title('')
ax.set_xlim(0, 400)
ax.set_ylim(0, 400)
# Turn of the map grid and ticks
ax.grid(False)
for coord in [0, 1]:
ax.coords[coord].set_ticklabel_visible(False)
ax.coords[coord].set_ticks_visible(False)
stonyhurst_grid['lon'].set_ticks([-30, -45, -60] * u.deg)
carrington_grid['lon'].set_ticks([-45, -60, -75] * u.deg)
fig.subplots_adjust(wspace=0.5)
plt.show()
Total running time of the script: (0 minutes 1.711 seconds)
Note
Go to the end to download the full example code.
Plotting a map#
How to create a plot of a map.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
Let’s plot the result. Setting the projection is necessary to ensure that pixels can be converted accurately to coordinates values.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax)
aiamap.draw_limb(axes=ax)
aiamap.draw_grid(axes=ax)
plt.show()
The above image looks “dark” because the color scale is accounting for the
small set of pixels that are extremely bright. We can use the keyword
clip_interval to clip out pixels with extreme values. Here, we clip out
the darkest 1% of pixels and the brightest 0.01% of pixels.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
aiamap.draw_limb(axes=ax)
aiamap.draw_grid(axes=ax)
plt.show()
Total running time of the script: (0 minutes 1.237 seconds)
Note
Go to the end to download the full example code.
Plotting a solar cycle index#
This example demonstrates how to plot the solar cycle in terms of the number of sunspots and a prediction for the next few years.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.time import Time, TimeDelta
from astropy.visualization import time_support
import sunpy.timeseries as ts
from sunpy.net import Fido
from sunpy.net import attrs as a
from sunpy.time import TimeRange
The U.S. Dept. of Commerce, NOAA, Space Weather Prediction Center (SWPC) provides recent solar cycle indices which includes different sunspot numbers, radio flux, and geomagnetic index. They also provide predictions for how the sunspot number and radio flux will evolve. Predicted values are based on the consensus of the Solar Cycle 24 Prediction Panel.
We will first search for and then download the data.
time_range = TimeRange("2008-06-01 00:00", Time.now())
result = Fido.search(a.Time(time_range), a.Instrument('noaa-indices'))
f_noaa_indices = Fido.fetch(result)
result = Fido.search(a.Time(time_range.end, time_range.end + TimeDelta(4 * u.year)),
a.Instrument('noaa-predict'))
f_noaa_predict = Fido.fetch(result)
We then load them into individual TimeSeries objects.
noaa = ts.TimeSeries(f_noaa_indices, source='noaaindices').truncate(time_range)
noaa_predict = ts.TimeSeries(f_noaa_predict, source='noaapredictindices')
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "dtf2d" yielded 1 of "dubious year (Note 6)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "dtf2d" yielded 1 of "dubious year (Note 6)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
Finally, we plot both noaa and noaa_predict for the sunspot number.
In this case we use the S.I.D.C. Brussels International Sunspot Number (RI).
The predictions provide both a high and low values, which we plot below as
ranges.
time_support()
fig, ax = plt.subplots()
ax.plot(noaa.time, noaa.quantity('sunspot RI'), label='Sunspot Number')
ax.plot(
noaa_predict.time, noaa_predict.quantity('sunspot'),
color='grey', label='Near-term Prediction'
)
ax.fill_between(
noaa_predict.time, noaa_predict.quantity('sunspot low'),
noaa_predict.quantity('sunspot high'), alpha=0.3, color='grey'
)
ax.set_ylim(bottom=0)
ax.set_ylabel('Sunspot Number')
ax.set_xlabel('Year')
ax.legend()
plt.show()
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/erfa/core.py:133: ErfaWarning: ERFA function "dtf2d" yielded 144 of "dubious year (Note 6)"
warn(f'ERFA function "{func_name}" yielded {wmsg}', ErfaWarning)
Total running time of the script: (0 minutes 5.611 seconds)
Note
Go to the end to download the full example code.
Plotting points on a Map with WCSAxes#
This example demonstrates the plotting of a point, a line and an arch in pixel coordinates, world coordinates, and SkyCoords respectively when plotting a map with WCSAxes.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
from sunpy.coordinates.utils import GreatArc
We will start by creating a map using an AIA 171 image. Then we shall create a standard plot of this map.
my_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
fig = plt.figure(figsize=(8, 8))
ax = fig.add_subplot(projection=my_map)
my_map.plot(axes=ax, clip_interval=(1, 99.9)*u.percent)
<matplotlib.image.AxesImage object at 0x7f6f403bdba0>
Now we will plot a line on the map by using coordinates in arcseconds.
The array below xx and yy are the x and y coordinates that define a
line from the Sun center (at 0, 0) to the point (500, 500) in arcsecs.
When plotting a map a WCSAxes is created.
For plotting with WCSAxes, pixel coordinates are expected as a default, however,
we can plot world coordinates (i.e. arcsec) by using the transform keyword.
Its important to note that when transforming between world and pixel coordinates
the world coordinates need to be in degrees rather than arcsecs.
xx = np.arange(0, 500) * u.arcsec
yy = xx
# Note that the coordinates need to be in degrees rather than arcseconds.
ax.plot(xx.to(u.deg), yy.to(u.deg),
color='r',
transform=ax.get_transform("world"),
label=f'WCS coordinate [{0*u.arcsec}, {500*u.arcsec}]')
[<matplotlib.lines.Line2D object at 0x7f6f39a3bcd0>]
Here we will plot a point in pixel coordinates (i.e. array index).
Let’s define a pixel coordinate in the middle of the image, pixel_coord.
As this coordinate is in pixel space (rather than a world coordinates like arcsec)
we do not need to use the transform keyword.
pixel_coord = [my_map.data.shape[0]/2., my_map.data.shape[1]/2.] * u.pix
ax.plot(pixel_coord[0], pixel_coord[1], 'x', color='w',
label=f'Pixel coordinate [{pixel_coord[0]}, {pixel_coord[1]}]')
[<matplotlib.lines.Line2D object at 0x7f6f39a3b250>]
As well as defining a point and using GenericMap.plot(), you can also plot
a point with WCSAxes using the plot_coord
functionality using a coordinate as a SkyCoord.
We can demonstrate this by plotting a point and an arc on a map using two separate SkyCoords.
Here we will plot a point (at -250,-250) on the map using a SkyCoord.
ax.plot_coord(SkyCoord(-250*u.arcsec, -250*u.arcsec, frame=my_map.coordinate_frame), "o",
label=f'SkyCoord [{-250*u.arcsec}, {-250*u.arcsec}]')
[<matplotlib.lines.Line2D object at 0x7f6f4022c580>]
Finally, let’s create a great arc between a start and end point defined as SkyCoords.
start = SkyCoord(723 * u.arcsec, -500 * u.arcsec, frame=my_map.coordinate_frame)
end = SkyCoord(-100 * u.arcsec, 900 * u.arcsec, frame=my_map.coordinate_frame)
great_arc = GreatArc(start, end)
my_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
ax.plot_coord(great_arc.coordinates(), color='c',
label=f'SkyCoord [{723*u.arcsec}, {-500*u.arcsec}],\n \
[{-100*u.arcsec}, {900*u.arcsec}]')
ax.legend(loc="lower center")
plt.show()
Total running time of the script: (0 minutes 0.603 seconds)
Note
Go to the end to download the full example code.
Plotting the solar equator and prime meridian#
This example shows how you can draw the solar equator and prime meridian (zero Carrington longitude) on Maps as seen by the axes observer.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
from sunpy.visualization import drawing
Let’s import sample AIA data and plot it with the equator and prime meridian.
aia_map = sunpy.map.Map(AIA_171_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot()
drawing.equator(ax, color='blue')
drawing.prime_meridian(ax, color='red')
plt.show()
The draw methods also work with heliogaphic maps. To demonstrate this, let’s convert the AIA map to a heliographic Carrington map then plot it with the equator and prime meridian.
frame_out = SkyCoord(0, 0, unit=u.deg,
frame="heliographic_carrington",
obstime=aia_map.date,
observer=aia_map.observer_coordinate,
rsun=aia_map.coordinate_frame.rsun)
header = sunpy.map.make_fitswcs_header((720, 1440),
frame_out,
scale=(360 / 1440,
180 / 720) * u.deg / u.pix,
projection_code="CAR")
outmap = aia_map.reproject_to(header)
fig = plt.figure()
ax = fig.add_subplot(projection=outmap)
outmap.plot(axes=ax)
drawing.equator(ax, color='blue')
drawing.prime_meridian(ax, color='red')
plt.show()
Total running time of the script: (0 minutes 1.904 seconds)
Note
Go to the end to download the full example code.
Plotting the solar limb#
This example demonstrates how you can draw the limb as seen by an arbitrary observer.
import matplotlib.pyplot as plt
import sunpy.map
from sunpy.coordinates import get_body_heliographic_stonyhurst
from sunpy.visualization import drawing
Let’s download a magnetic field synoptic map and read it into a Map.
syn_map = sunpy.map.Map('http://jsoc.stanford.edu/data/hmi/synoptic/hmi.Synoptic_Mr.2191.fits')
syn_map.plot_settings['cmap'] = 'hmimag'
syn_map.plot_settings['norm'] = plt.Normalize(-1500, 1500)
Get coordinates for Earth and Mars at the date of the synoptic map
coords = {body: get_body_heliographic_stonyhurst(body, syn_map.date)
for body in ['Earth', 'Mars']}
Now we can plot the map the the solar limb seen from these two coordinates.
To create a legend for these limbs, we need to keep the patches returned by
limb() and provide them to
legend().
fig = plt.figure(figsize=(12, 5))
ax = fig.add_subplot(projection=syn_map)
im = syn_map.plot(axes=ax)
visible_limbs = []
for (body, coord), color in zip(coords.items(), ['tab:blue', 'tab:red']):
v, _ = drawing.limb(ax, coord, color=color, label=f'Limb seen from {body}', linewidth=2)
visible_limbs.append(v)
ax.legend(handles=visible_limbs)
plt.show()
INFO: Missing metadata for solar radius: assuming the standard radius of the photosphere. [sunpy.map.mapbase]
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:631: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hglt_obs,hgln_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: dsun_obs,crln_obs,crlt_obs
obs_coord = self.observer_coordinate
Total running time of the script: (0 minutes 2.511 seconds)
Note
Go to the end to download the full example code.
Set Axis Range When Plotting a Map#
In this example we are going to look at how to set the axes
range using Matplotlib’s set_xlim and set_ylim when plotting a
Map with WCSAxes.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
Lets start by creating a Map from the sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
Now lets say for example we are only interested in plotting a certain region
of this Map. One way this could be done is to create a submap over the region
of interest and then plotting that. Another useful way is to set the axes
range over which to plot using Matplotlib’s
set_xlim and set_ylim functionality.
The axes that Matplotlib uses is in pixel coordinates (e.g. of image data array)
rather than world coordinates (e.g. in arcsecs) so we need to define our limits that
are passed to set_xlim, set_ylim to pixel coordinates.
We can define our limits we want to use in world coordinates and then work out what pixel
coordinates these correspond to.
Lets choose x-limits and y-limits in arcsecs that we are interested in.
xlims_world = [500, 1100]*u.arcsec
ylims_world = [-800, 0]*u.arcsec
We can then convert these into a SkyCoord which can be passed to
GenericMap.wcs.world_to_pixel
to determine the corresponding pixel coordinates.
world_coords = SkyCoord(Tx=xlims_world, Ty=ylims_world, frame=aia_map.coordinate_frame)
pixel_coords_x, pixel_coords_y = aia_map.wcs.world_to_pixel(world_coords)
We can now plot this Map and then use pixel_coords_x and pixel_coords_y to set
the limits on the axes.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.9)*u.percent)
ax.set_xlim(pixel_coords_x)
ax.set_ylim(pixel_coords_y)
plt.show()
Total running time of the script: (0 minutes 0.335 seconds)
Note
Go to the end to download the full example code.
Simple Differential Rotation#
The Sun is known to rotate differentially, meaning that the rotation rate near the poles (rotation period of approximately 35 days) is not the same as the rotation rate near the equator (rotation period of approximately 25 days). This is possible because the Sun is not a solid body. Though it is still poorly understood, it is fairly well measured and must be taken into account when comparing observations of features on the Sun over time. A good review can be found in Beck 1999 Solar Physics 191, 47–70. This example illustrates solar differential rotation.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.time import TimeDelta
import sunpy.data.sample
import sunpy.map
from sunpy.physics.differential_rotation import solar_rotate_coordinate
from sunpy.sun.models import differential_rotation
Next lets explore solar differential rotation by replicating Figure 1 in Beck 1999.
latitudes = np.arange(0, 90, 1) * u.deg
dt = 1 * u.day
rotation_rate = differential_rotation(dt, latitudes) / dt
rotation_period = 360 * u.deg / rotation_rate
plt.figure()
plt.plot(np.sin(latitudes), rotation_period.value)
plt.ylim(38, 24)
plt.ylabel(f'Rotation Period [{rotation_period.unit}]')
plt.xlabel('Sin(Latitude)')
plt.title('Solar Differential Rotation Rate')
Text(0.5, 1.0, 'Solar Differential Rotation Rate')
Next let’s show how to this looks like on the Sun. Load in an AIA map:
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
Let’s define our starting coordinates.
hpc_y = np.arange(-700, 800, 100) * u.arcsec
hpc_x = np.zeros_like(hpc_y)
Let’s define how many days in the future we want to rotate to.
dt = TimeDelta(4*u.day)
future_date = aia_map.date + dt
Now let’s plot the original and rotated positions on the AIA map.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
ax.set_title(f'The effect of {dt.to(u.day).value} days of differential rotation')
aia_map.draw_grid(axes=ax)
for this_hpc_x, this_hpc_y in zip(hpc_x, hpc_y):
start_coord = SkyCoord(this_hpc_x, this_hpc_y, frame=aia_map.coordinate_frame)
rotated_coord = solar_rotate_coordinate(start_coord, time=future_date)
coord = SkyCoord([start_coord.Tx, rotated_coord.Tx],
[start_coord.Ty, rotated_coord.Ty],
frame=aia_map.coordinate_frame)
ax.plot_coord(coord, 'o-')
ax.set_ylim(0, aia_map.data.shape[1])
ax.set_xlim(0, aia_map.data.shape[0])
plt.show()
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/physics/differential_rotation.py:152: SunpyUserWarning: Using 'time' assumes an Earth-based observer.
warn_user("Using 'time' assumes an Earth-based observer.")
Total running time of the script: (0 minutes 1.280 seconds)
Note
Go to the end to download the full example code.
Using the sunpy Colormaps with matplotlib#
How you can use the sunpy colormaps with matplotlib.
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import sunpy.visualization.colormaps as cm
When the sunpy colormaps are imported, the sunpy colormaps are registered with matplotlib. It is now possible to access the colormaps with the following command.
sdoaia171 = matplotlib.colormaps['sdoaia171']
You can get the list of all sunpy colormaps with:
print(cm.cmlist.keys())
dict_keys(['goes-rsuvi94', 'goes-rsuvi131', 'goes-rsuvi171', 'goes-rsuvi195', 'goes-rsuvi284', 'goes-rsuvi304', 'sdoaia94', 'sdoaia131', 'sdoaia171', 'sdoaia193', 'sdoaia211', 'sdoaia304', 'sdoaia335', 'sdoaia1600', 'sdoaia1700', 'sdoaia4500', 'sohoeit171', 'sohoeit195', 'sohoeit284', 'sohoeit304', 'soholasco2', 'soholasco3', 'sswidlsoholasco2', 'sswidlsoholasco3', 'stereocor1', 'stereocor2', 'stereohi1', 'stereohi2', 'yohkohsxtal', 'yohkohsxtwh', 'hinodexrt', 'hinodesotintensity', 'trace171', 'trace195', 'trace284', 'trace1216', 'trace1550', 'trace1600', 'trace1700', 'traceWL', 'hmimag', 'irissji1330', 'irissji1400', 'irissji1600', 'irissji2796', 'irissji2832', 'irissji5000', 'irissjiFUV', 'irissjiNUV', 'irissjiSJI_NUV', 'kcor', 'rhessi', 'std_gamma_2', 'euvi171', 'euvi195', 'euvi284', 'euvi304', 'solar orbiterfsi174', 'solar orbiterfsi304', 'solar orbiterhri_euv174', 'solar orbiterhri_lya1216'])
Let’s now create a data array.
delta = 0.025
x = y = np.arange(-3.0, 3.0, delta)
X, Y = np.meshgrid(x, y)
Z1 = np.exp(-X**2 - Y**2)
Z2 = np.exp(-(X - 1)**2 - (Y - 1)**2)
Z = (Z1 - Z2) * 2
Let’s now plot the results with the colormap.
fig, ax = plt.subplots()
im = ax.imshow(
Z, interpolation='bilinear', cmap=sdoaia171,
origin='lower', extent=[-3, 3, -3, 3],
vmax=abs(Z).max(), vmin=-abs(Z).max()
)
plt.show()
Total running time of the script: (0 minutes 0.129 seconds)
Differential Rotation of the Sun#
Examples of accounting for differential rotation (i.e., the latitude-dependent rotation rate of the Sun) in the coordinates framework
Note
Go to the end to download the full example code.
Comparing differential-rotation models#
How to compare differential-rotation models.
The example uses the RotatedSunFrame coordinate
metaframe in sunpy.coordinates to apply differential rotation to a coordinate.
See Differential rotation using coordinate frames for more details on using
RotatedSunFrame.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import HeliographicStonyhurst, RotatedSunFrame
from sunpy.data.sample import AIA_335_IMAGE
from sunpy.sun.constants import sidereal_rotation_rate
First, we use an AIA observation primarily as a pretty background. We also define the meridian using a two-element coordinate array of the south pole and the north pole at zero longitude.
aiamap = sunpy.map.Map(AIA_335_IMAGE)
meridian = SkyCoord(0*u.deg, [-90, 90]*u.deg, frame=HeliographicStonyhurst,
obstime=aiamap.date)
Next, we calculate the sidereal rotation period of the Sun. This is the time for a full rotation relative to an inertial reference frame (e.g., distant stars), as opposed to the synodic period, which is the apparent rotation period as seen from an Earth-based observer. Since the Earth orbits the Sun in the same direction as the Sun rotates, the Sun appears to rotate slower for an Earth-based observer.
sidereal_period = 360*u.deg / sidereal_rotation_rate
print(sidereal_period)
25.379994924001014 d
We use RotatedSunFrame to rotate the
meridian by one sidereal period using each of the available
differential-rotation models. See
diff_rot() for details on each
model.
rotated_meridian = {}
for model in ['howard', 'snodgrass', 'allen', 'rigid']:
rotated_meridian[model] = SkyCoord(RotatedSunFrame(base=meridian,
duration=sidereal_period,
rotation_model=model))
Finally, we plot the differentially rotated meridians over the map, using
draw_quadrangle() to conveniently draw a line
of constant longitude in the original frame between two endpoints. (One
could instead use astropy.visualization.wcsaxes.WCSAxes.plot_coord(),
but then meridian would need to consist of a sequence of many points
spanning all latitudes between the two poles to render as desired.)
Note that the “rigid” model appears as the meridian again as expected for a
rotation of exactly one sidereal period.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(0.5, 99.9)*u.percent)
colors = {
'howard': 'red',
'snodgrass': 'green',
'allen': 'yellow',
'rigid': 'white',
}
for model, coord in rotated_meridian.items():
aiamap.draw_quadrangle(coord, axes=ax, edgecolor=colors[model],
label=model.capitalize())
ax.legend()
ax.set_title(f'{sidereal_period:.2f} of solar rotation')
plt.show()
Total running time of the script: (0 minutes 0.599 seconds)
Note
Go to the end to download the full example code.
Differentially rotating a coordinate#
How to differentially rotate a coordinate.
The example uses the RotatedSunFrame coordinate
metaframe in sunpy.coordinates to apply differential rotation to a
coordinate. See Differential rotation using coordinate frames for more details on
using RotatedSunFrame.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import RotatedSunFrame
from sunpy.data.sample import AIA_171_IMAGE
First, load an AIA observation and define a coordinate in its coordinate frame (here, helioprojective Cartesian). The appropriate rate of rotation is determined from the heliographic latitude of the coordinate.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
point = SkyCoord(187*u.arcsec, 283*u.arcsec, frame=aiamap.coordinate_frame)
We can differentially rotate this coordinate by using
RotatedSunFrame with an array of observation
times. Let’s define a daily cadence for +/- five days.
durations = np.concatenate([range(-5, 0), range(1, 6)]) * u.day
diffrot_point = SkyCoord(RotatedSunFrame(base=point, duration=durations))
To see what this coordinate looks like in “real” helioprojective Cartesian coordinates, we can transform it back to the original frame. Since these coordinates are represented in the original frame, they will not account for the changing position of the observer over this same time range.
transformed_diffrot_point = diffrot_point.transform_to(aiamap.coordinate_frame)
print(transformed_diffrot_point)
<SkyCoord (Helioprojective: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406234, 0.04787238, 1.51846026e+11)>): (Tx, Ty, distance) in (arcsec, arcsec, m)
[(-772.69378072, 282.77545584, 1.51502020e+11),
(-635.25851451, 282.86951972, 1.51373625e+11),
(-459.07089956, 282.9428274 , 1.51273829e+11),
(-254.82447782, 282.99074318, 1.51208725e+11),
( -34.98324566, 283.01021568, 1.51182296e+11),
( 397.53637817, 282.96074958, 1.51249466e+11),
( 583.76348605, 282.89496793, 1.51338955e+11),
( 734.3555842 , 282.80682531, 1.51459158e+11),
( 840.22448988, 282.70185963, 1.51602747e+11),
( 895.06498643, 282.58659113, 1.51760989e+11)]>
Let’s plot the original coordinate and the differentially rotated coordinates on top of the AIA observation.
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(1., 99.95)*u.percent)
ax.plot_coord(point, 'ro', fillstyle='none', label='Original')
ax.plot_coord(transformed_diffrot_point, 'bo', fillstyle='none',
label='Rotated')
ax.legend()
plt.show()
Total running time of the script: (0 minutes 0.395 seconds)
Note
Go to the end to download the full example code.
Differentially rotating a map#
How to apply differential rotation to a Map.
Note
This example requires reproject 0.6 or later to be installed.
The example uses the propagate_with_solar_surface()
context manager to apply differential rotation during coordinate
transformations.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
from astropy.wcs import WCS
import sunpy.map
from sunpy.coordinates import Helioprojective, propagate_with_solar_surface
from sunpy.data.sample import AIA_171_IMAGE
First, load an AIA observation.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
in_time = aiamap.date
Let’s define the output frame to be five days in the future for an observer at Earth (i.e., about five degrees offset in heliographic longitude compared to the location of AIA in the original observation).
out_time = in_time + 5*u.day
out_frame = Helioprojective(observer='earth', obstime=out_time,
rsun=aiamap.coordinate_frame.rsun)
Construct a WCS object for the output map. If one has an actual Map
object at the desired output time (e.g., the actual AIA observation at the
output time), one can use the WCS object from that Map object (e.g.,
mymap.wcs) instead of constructing a custom WCS.
out_center = SkyCoord(0*u.arcsec, 0*u.arcsec, frame=out_frame)
header = sunpy.map.make_fitswcs_header(aiamap.data.shape,
out_center,
scale=u.Quantity(aiamap.scale))
out_wcs = WCS(header)
Reproject the map from the input frame to the output frame. We use the
propagate_with_solar_surface() context manager so
that coordinates are treated as points that evolve in time with the
rotation of the solar surface rather than as inertial points in space.
with propagate_with_solar_surface():
out_warp = aiamap.reproject_to(out_wcs)
Let’s plot the differentially rotated Map next to the original Map.
fig = plt.figure(figsize=(12, 4))
ax1 = fig.add_subplot(121, projection=aiamap)
aiamap.plot(axes=ax1, vmin=0, vmax=20000, title='Original map')
plt.colorbar()
ax2 = fig.add_subplot(122, projection=out_warp)
out_warp.plot(axes=ax2, vmin=0, vmax=20000,
title=f"Reprojected to an Earth observer {(out_time - in_time).to('day')} later")
plt.colorbar()
plt.show()
Total running time of the script: (0 minutes 2.748 seconds)
Note
Go to the end to download the full example code.
Overlaying differentially rotated gridlines#
How to overlay differentially rotated gridlines on a Map.
The example uses the RotatedSunFrame coordinate
metaframe in sunpy.coordinates to overlay differentially rotated gridlines on
a Map. See Differential rotation using coordinate frames for more details on using
RotatedSunFrame.
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.map
from sunpy.coordinates import HeliographicStonyhurst, RotatedSunFrame
from sunpy.data.sample import AIA_171_IMAGE
Let’s use an AIA observation, and plot lines of constant longitude in heliographic Stonyhurst. We’ll plot the normal lines (prior to applying differential rotation) in white and the differentially rotated lines in blue.
aiamap = sunpy.map.Map(AIA_171_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(1., 99.95)*u.percent)
<matplotlib.image.AxesImage object at 0x7f6f4192fd30>
Lines of constant longitude prior to differential rotation
overlay1 = ax.get_coords_overlay('heliographic_stonyhurst')
overlay1[0].set_ticks(spacing=15. * u.deg)
overlay1[1].set_ticks(spacing=90. * u.deg)
overlay1.grid(ls='-', color='white')
Differentially rotating the lines of constant longitude by 27 days Be aware that the differentially rotated lines are plotted in the original coordinate frame, so it doesn’t account for any motion of the observer over 27 days.
rs_hgs = RotatedSunFrame(base=HeliographicStonyhurst(obstime=aiamap.date),
duration=27*u.day)
overlay2 = ax.get_coords_overlay(rs_hgs)
overlay2[0].set_ticks(spacing=15. * u.deg)
overlay2[1].set_ticks(spacing=90. * u.deg)
overlay2.grid(ls='-', color='blue')
plt.show()
Total running time of the script: (0 minutes 1.046 seconds)
Saving and Loading Data#
Examples of saving and loading data
Note
Go to the end to download the full example code.
Saving and loading coordinates with asdf#
In this example we are going to look at saving and loading collections of coordinates with asdf.
asdf is a modern file format designed to meet the needs of the astronomy community. It has deep integration with Python and sunpy and Astropy as well as implementations in other languages. It can be used to store known Python objects in a portable, well defined file format. It is primarily useful for storing complex Astropy and SunPy objects in a way that can be loaded back into the same form as they were saved.
Note
This example requires astropy>=3.2 and asdf>=2.3.0
import matplotlib.pyplot as plt
import numpy as np
import asdf
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.coordinates import frames
from sunpy.data.sample import AIA_171_IMAGE
from sunpy.sun import constants
To get started let’s use a function to get the coordinates of a semi-circular loop from this blog post by Will Barnes to generate ourselves some coordinates.
@u.quantity_input
def semi_circular_loop(length: u.cm, latitude: u.deg = 0*u.deg):
"""
Return HGS coordinates for a semi-circular loop
"""
angles = np.linspace(0, 1, 1000) * np.pi * u.rad
z = length / np.pi * np.sin(angles)
x = length / np.pi * np.cos(angles)
hcc_frame = frames.Heliocentric(
observer=frames.HeliographicStonyhurst(lon=0 * u.deg, lat=latitude, radius=constants.au))
return SkyCoord(
x=x,
y=np.zeros_like(x),
z=z + constants.radius,
frame=hcc_frame)
Use this function to generate a SkyCoord object.
loop_coords = semi_circular_loop(500*u.Mm, 30*u.deg)
print(loop_coords.shape)
# print the first and last coordinate point
print(loop_coords[[0, -1]])
(1000,)
<SkyCoord (Heliocentric: obstime=None, observer=<HeliographicStonyhurst Coordinate (obstime=None, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, m)
(0., 30., 1.49597871e+11)>): (x, y, z) in Mm
[( 159.15494309, 0., 695.7), (-159.15494309, 0., 695.7)]>
This is a regular coordinate object that can be transformed to other frames or overplotted on images. For instance we could overplot it on an AIA image
aiamap = sunpy.map.Map(AIA_171_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aiamap)
aiamap.plot(axes=ax, clip_interval=(1, 99.5) * u.percent)
ax.plot_coord(loop_coords, 'r')
plt.show()
We can now save these loop points to an asdf file to use later. The advantage
of saving them to asdf is that all the metadata about the coordinates will be
preserved, and when we load the asdf, we will get back an identical
SkyCoord object.
asdf files save a dictionary to a file, so to save the loop coordinates we need to put them into a dictionary. This becomes what asdf calls a tree.
tree = {'loop_points': loop_coords}
with asdf.AsdfFile(tree) as asdf_file:
asdf_file.write_to("loop_coords.asdf")
This asdf file is a portable file and can be safely loaded by anyone with Astropy and sunpy installed. We can reload the file like so:
with asdf.open("loop_coords.asdf") as input_asdf:
new_coords = input_asdf['loop_points']
print(new_coords.shape)
# print the first and last coordinate point
print(new_coords[[0, -1]])
(1000,)
<SkyCoord (Heliocentric: obstime=None, observer=<HeliographicStonyhurst Coordinate (obstime=None, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, m)
(0., 30., 1.49597871e+11)>): (x, y, z) in Mm
[( 159.15494309, 0., 695.7), (-159.15494309, 0., 695.7)]>
Total running time of the script: (0 minutes 1.015 seconds)
Note
Go to the end to download the full example code.
Saving and loading sunpy Maps with FITS#
In this example we are going to look at how to save and load a
GenericMap as FITS files.
FITS is a modern file format designed to meet the needs of the astronomy community. It has deep integration with Python, SunPy and Astropy as well as IDL and many other languages.
Here, even though we will be working with AIAMap
specifically, the process can be extended to any GenericMap.
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We begin by creating an AIAMap object using the
sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
aia_map.peek(clip_interval=(1, 99.99)*u.percent)
We can now save this object to a FITS file to use later. Saving it like this
allows us to preserve all of the metadata of the object along with the actual
array data. When we load the FITS file again, we get an identical
AIAMap object.
All changes to GenericMap are saved within the FITS file. For example,
if we rotate the image by 45 degrees and then save it to a FITS file, these changes
are reflected in both the image data and metadata when we load the data back in.
aia_map = aia_map.rotate(45*u.deg)
# Please be aware that if you try to save this twice,
# it will throw an exception rather than overwriting the file.
aia_map.save('aia_map.fits')
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/astropy/io/fits/hdu/image.py:627: VerifyWarning: Invalid 'BLANK' keyword in header. The 'BLANK' keyword is only applicable to integer data, and will be ignored in this HDU.
warnings.warn(msg, VerifyWarning)
This FITS file is portable and can be safely loaded by any FITS reader.
Using Map, we can easily read our rotated image back in:
aia_map_from_fits = sunpy.map.Map('aia_map.fits')
aia_map_from_fits.peek(clip_interval=(1, 99.99)*u.percent)
Total running time of the script: (0 minutes 1.368 seconds)
Note
Go to the end to download the full example code.
Saving and loading sunpy Maps with asdf#
In this example we are going to look at how we can save and load a
GenericMap with asdf.
asdf is a modern file format designed to meet the needs of the astronomy community. It has deep integration with Python, SunPy and Astropy as well as implementations in other languages. It can be used to store known Python objects in a portable, well defined file format. It is primarily useful for storing complex Astropy and SunPy objects in a way that can be loaded back into the same form as they were saved.
Here, even though we will be working with AIAMap
specifically, the process can be extended to any GenericMap,
including ones created using custom FITS files.
import asdf
import astropy.units as u
import sunpy.data.sample
import sunpy.map
We begin by creating an AIAMap object using the
sample data.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
aia_map.peek(clip_interval=(1, 99.99)*u.percent)
We can now save this object to an asdf file to use later. Saving it like this
allows us to preserve all of the metadata of the object along with the actual
array data. When we load the asdf again, we get an identical
AIAMap object.
asdf files work by saving a dictionary internally, so to save the object we need to put it into a dictionary. This becomes what asdf calls a “tree”.
tree = {'aia_map': aia_map}
We can now write the data to an asdf file like so:
with asdf.AsdfFile(tree) as asdf_file:
asdf_file.write_to("sunpy_map.asdf")
This asdf file is a portable file and can be safely loaded by anyone with Astropy, sunpy and asdf installed. We can reload it like so:
with asdf.open("sunpy_map.asdf") as asdf_file:
reloaded_aia_map = asdf_file['aia_map']
reloaded_aia_map.peek(clip_interval=(1, 99.99)*u.percent)
Total running time of the script: (0 minutes 0.847 seconds)
Computer Vision Techniques#
Examples of using computer vision techniques to analyze solar data
Note
Go to the end to download the full example code.
Edge Enhancing Coronal Loops#
How to edge enhance coronal loops in an SDO/AIA image.
import matplotlib.pyplot as plt
import numpy as np
from scipy import ndimage
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
We start with the sample data and crop it down to a region featuring coronal loops.
aia = sunpy.map.Map(AIA_171_IMAGE)
bottom_left = SkyCoord(750 * u.arcsec, -200 * u.arcsec, frame=aia.coordinate_frame)
top_right = SkyCoord(1500 * u.arcsec, 550 * u.arcsec, frame=aia.coordinate_frame)
aia_smap = aia.submap(bottom_left, top_right=top_right)
Next we apply an edge enhance filter to the data in both x and y directions and combine the two images together.
sx = ndimage.sobel(aia_smap.data, axis=0, mode='constant')
sy = ndimage.sobel(aia_smap.data, axis=1, mode='constant')
edge_enhanced_im = np.hypot(sx, sy)
Finally we create a new map with the edge enhanced data.
edge_map = sunpy.map.Map(edge_enhanced_im, aia_smap.meta)
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=edge_map)
edge_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.336 seconds)
Note
Go to the end to download the full example code.
Enhancing off-disk emission#
How to enhance emission above the limb.
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
from astropy.visualization.mpl_normalize import ImageNormalize
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
from sunpy.map.maputils import all_coordinates_from_map
We start with the sample data.
aia = sunpy.map.Map(AIA_171_IMAGE)
A utility function gives us access to the helioprojective coordinate of each pixels. We can use that to create a new array which contains the normalized radial position for each pixel.
hpc_coords = all_coordinates_from_map(aia)
r = np.sqrt(hpc_coords.Tx ** 2 + hpc_coords.Ty ** 2) / aia.rsun_obs
Let’s check how emission above the limb depends on distance.
rsun_step_size = 0.01
rsun_array = np.arange(1, r.max(), rsun_step_size)
y = np.array([aia.data[(r > this_r) * (r < this_r + rsun_step_size)].mean()
for this_r in rsun_array])
Next let’s plot it along with a fit to the data. We perform the fit in linear-log space.
params = np.polyfit(rsun_array[rsun_array < 1.5],
np.log(y[rsun_array < 1.5]), 1)
Let’s plot the results using LaTeX for all the text.
fontsize = 14
fig, ax = plt.subplots()
ax.plot(rsun_array, y, label='data')
best_fit = np.exp(np.poly1d(params)(rsun_array))
label = fr'best fit: {best_fit[0]:.2f}$e^{{{params[0]:.2f}r}}$'
ax.plot(rsun_array, best_fit, label=label)
ax.set_yscale('log')
ax.set_ylabel(r'mean DN', fontsize=fontsize)
ax.set_xlabel(r'radius r ($R_{\odot}$)', fontsize=fontsize)
ax.tick_params(axis='both', size=fontsize)
ax.set_title(r'observed off limb mean DN and best fit', fontsize=fontsize)
ax.legend(fontsize=fontsize)
fig.tight_layout()
plt.show()
We now create our normalization array. At the solar radius and below, the normalization is 1, while off-disk the normalization changes according to the function we fit above.
scale_factor = np.exp((r-1)*-params[0])
scale_factor[r < 1] = 1
Finally we create a new map with the normalized off-disk emission. We set the normalization of the new map to be the same as the original map to compare the two.
scaled_map = sunpy.map.Map(aia.data * scale_factor, aia.meta)
scaled_map.plot_settings['norm'] = ImageNormalize(stretch=aia.plot_settings['norm'].stretch)
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=scaled_map)
im = scaled_map.plot(axes=ax, clip_interval=(5, 99.9)*u.percent)
scaled_map.draw_limb(axes=ax)
fig.colorbar(im)
plt.show()
Total running time of the script: (0 minutes 1.286 seconds)
Note
Go to the end to download the full example code.
Finding and masking bright pixels#
How to find and overplot the location of the brightest pixel and then mask pixels around that region.
import matplotlib.pyplot as plt
import numpy as np
import numpy.ma as ma
import astropy.units as u
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
from sunpy.map.maputils import all_coordinates_from_map
We start with the sample data.
aia = sunpy.map.Map(AIA_171_IMAGE)
To find the brightest pixel, we find the maximum in the AIA image data then transform that pixel coordinate to a map coordinate.
pixel_pos = np.argwhere(aia.data == aia.data.max()) * u.pixel
hpc_max = aia.wcs.pixel_to_world(pixel_pos[:, 1], pixel_pos[:, 0])
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=aia)
aia.plot(axes=ax)
ax.plot_coord(hpc_max, color='white', marker='x', markersize=15)
plt.show()
A utility function gives us access to the helioprojective coordinate of each
pixels. We create a new array which contains the normalized radial position
for each pixel adjusted for the position of the brightest pixel
(using hpc_max) and then create a new map.
hpc_coords = all_coordinates_from_map(aia)
r_mask = np.sqrt((hpc_coords.Tx - hpc_max.Tx) ** 2 +
(hpc_coords.Ty - hpc_max.Ty) ** 2) / aia.rsun_obs
mask = ma.masked_less_equal(r_mask, 0.1)
scaled_map = sunpy.map.Map(aia.data, aia.meta, mask=mask.mask)
Let’s plot the results.
fig = plt.figure()
ax = fig.add_subplot(projection=scaled_map)
scaled_map.plot(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.883 seconds)
Note
Go to the end to download the full example code.
Masking out the solar disk#
How to mask out all emission from the solar disk.
import matplotlib.pyplot as plt
import sunpy.map
from sunpy.data.sample import AIA_171_IMAGE
from sunpy.map.maputils import all_coordinates_from_map, coordinate_is_on_solar_disk
We start with the sample data.
aia = sunpy.map.Map(AIA_171_IMAGE)
A utility function gives us access to the helioprojective coordinate of each pixels. We can use that to create a new array of all the coordinates that are on the solar disk.
hpc_coords = all_coordinates_from_map(aia)
Now, we can create a mask from the coordinates by using another utility
function that gives us a mask that has True for those coordinates that are
on the solar disk. We also make a slight change to the colormap so that
masked values are shown as black instead of the default white.
mask = coordinate_is_on_solar_disk(hpc_coords)
palette = aia.cmap.copy()
palette.set_bad('black')
Finally we create a new map with our new mask.
scaled_map = sunpy.map.Map(aia.data, aia.meta, mask=mask)
Let’s plot the results using our modified colormap.
fig = plt.figure()
ax = fig.add_subplot(projection=scaled_map)
scaled_map.plot(axes=ax, cmap=palette)
scaled_map.draw_limb(axes=ax)
plt.show()
Total running time of the script: (0 minutes 0.652 seconds)
Showcase#
Examples that use an advanced combination of capabilities in sunpy
Understanding Observer Orientation Relative to a Simulated Volume with astropy.coordinates
Note
Go to the end to download the full example code.
HMI Showcase: Cutout#
This example demonstrates how to plot a cutout region of a Map
with connector lines that indicate the region of interest in the full-disk
image.
Since this example deals with the creation of a specific style of image, there are multiple lines that deal directly with matplotlib axes.
import matplotlib.colors
import matplotlib.pyplot as plt
from matplotlib.patches import ConnectionPatch
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
from sunpy.data.sample import HMI_LOS_IMAGE
First, we use the sample HMI LOS image and focus the cutout over an active region near the solar center.
magnetogram = sunpy.map.Map(HMI_LOS_IMAGE).rotate()
left_corner = SkyCoord(Tx=-142*u.arcsec, Ty=50*u.arcsec, frame=magnetogram.coordinate_frame)
right_corner = SkyCoord(Tx=158*u.arcsec, Ty=350*u.arcsec, frame=magnetogram.coordinate_frame)
We clean up the magnetogram by masking off all data that is beyond the solar limb.
hpc_coords = sunpy.map.all_coordinates_from_map(magnetogram)
mask = ~sunpy.map.coordinate_is_on_solar_disk(hpc_coords)
magnetogram_big = sunpy.map.Map(magnetogram.data, magnetogram.meta, mask=mask)
We create the figure in two stages. The first stage is plotting the full-disk magnetogram.
fig = plt.figure(figsize=(7.2, 4.8))
We create a nice normalization range for the image.
norm = matplotlib.colors.SymLogNorm(50, vmin=-7.5e2, vmax=7.5e2)
Plot the full-disk magnetogram.
ax1 = fig.add_subplot(121, projection=magnetogram_big)
magnetogram_big.plot(axes=ax1, cmap='RdBu_r', norm=norm, annotate=False,)
magnetogram_big.draw_grid(axes=ax1, color='black', alpha=0.25, lw=0.5)
<CoordinatesMap with 2 world coordinates:
index aliases type unit wrap format_unit visible
----- ------- --------- ---- --------- ----------- -------
0 lon longitude deg 180.0 deg deg yes
1 lat latitude deg None deg yes
>
These lines deal with hiding the axis, its ticks and labels.
for coord in ax1.coords:
coord.frame.set_linewidth(0)
coord.set_ticks_visible(False)
coord.set_ticklabel_visible(False)
We draw the rectangle around the region we plan to showcase in the cutout image.
magnetogram_big.draw_quadrangle(left_corner, top_right=right_corner, edgecolor='black', lw=1)
<astropy.visualization.wcsaxes.patches.Quadrangle object at 0x7f6f3bc386a0>
The second stage is plotting the zoomed-in magnetogram.
magnetogram_small = magnetogram.submap(left_corner, top_right=right_corner)
ax2 = fig.add_subplot(122, projection=magnetogram_small)
im = magnetogram_small.plot(axes=ax2, norm=norm, cmap='RdBu_r', annotate=False,)
ax2.grid(alpha=0)
Unlike the full-disk image, here we just clean up the axis labels and ticks.
lon, lat = ax2.coords[0], ax2.coords[1]
lon.frame.set_linewidth(1)
lat.frame.set_linewidth(1)
lon.set_axislabel('Helioprojective Longitude',)
lon.set_ticks_position('b')
lat.set_axislabel('Helioprojective Latitude',)
lat.set_axislabel_position('r')
lat.set_ticks_position('r')
lat.set_ticklabel_position('r')
Now for the finishing touches, we add two lines that will connect the two images as well as a colorbar.
xpix, ypix = magnetogram_big.wcs.world_to_pixel(right_corner)
con1 = ConnectionPatch(
(0, 1), (xpix, ypix), 'axes fraction', 'data', axesA=ax2, axesB=ax1,
arrowstyle='-', color='black', lw=1
)
xpix, ypix = magnetogram_big.wcs.world_to_pixel(
SkyCoord(right_corner.Tx, left_corner.Ty, frame=magnetogram_big.coordinate_frame))
con2 = ConnectionPatch(
(0, 0), (xpix, ypix), 'axes fraction', 'data', axesA=ax2, axesB=ax1,
arrowstyle='-', color='black', lw=1
)
ax2.add_artist(con1)
ax2.add_artist(con2)
pos = ax2.get_position().get_points()
cax = fig.add_axes([
pos[0, 0], pos[1, 1]+0.01, pos[1, 0]-pos[0, 0], 0.025
])
cbar = fig.colorbar(im, cax=cax, orientation='horizontal')
For the colorbar we want it to have three fixed ticks.
cbar.locator = matplotlib.ticker.FixedLocator([-1e2, 0, 1e2])
cbar.set_label("LOS Magnetic Field [gauss]", labelpad=-40, rotation=0)
cbar.update_ticks()
cbar.ax.xaxis.set_ticks_position('top')
plt.show()
Total running time of the script: (0 minutes 1.703 seconds)
Note
Go to the end to download the full example code.
Obtaining solar-eclipse information#
How to obtain information about a solar eclipse
The function sunpy.coordinates.sun.eclipse_amount() returns how much of
the Sun is occulted by the Moon at the specified time(s). This example
showcases how one can use the the output of this function to calculate the
start/end times of an eclipse and to plot the eclipse amount as a function of
time.
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.dates import DateFormatter
import astropy.units as u
from astropy.coordinates import EarthLocation, solar_system_ephemeris
from astropy.time import Time
from sunpy.coordinates import sun
Define a location near San Antonio, Texas, that falls on both the path of the 2023 annular eclipse and the path of the 2024 total eclipse.
location = EarthLocation.from_geodetic(-98.5*u.deg, 29.6*u.deg)
max2023 = Time('2023-10-14 16:54')
max2024 = Time('2024-04-08 18:35')
Define a function to calculate start/end eclipse times and plot the eclipse timeseries within +/- 2 hours of the time of interest.
def plot_eclipse_timeseries(location, time):
# Define an array of observation times centered around the time of interest
times = time + np.concatenate([np.arange(-120, -5) * u.min,
np.arange(-300, 300) * u.s,
np.arange(5, 121) * u.min])
# Create an observer coordinate for the time array
observer = location.get_itrs(times)
# Calculate the eclipse amounts using a JPL ephemeris
with solar_system_ephemeris.set('de440s'):
amount = sun.eclipse_amount(observer)
amount_minimum = sun.eclipse_amount(observer, moon_radius='minimum')
# Calculate the start/end points of partial/total solar eclipse
partial = np.flatnonzero(amount > 0)
if len(partial) > 0:
print("Eclipse detected:")
start_partial, end_partial = times[partial[[0, -1]]]
print(f" Partial solar eclipse starts at {start_partial} UTC")
total = np.flatnonzero(amount_minimum == 1)
if len(total) > 0:
start_total, end_total = times[total[[0, -1]]]
print(f" Total solar eclipse starts at {start_total} UTC\n"
f" Total solar eclipse ends at {end_total} UTC")
print(f" Partial solar eclipse ends at {end_partial} UTC")
# Plot the eclipse timeseries
fig = plt.figure(layout="constrained")
ax = fig.add_subplot()
ax.plot(times.datetime64, amount)
ax.set_ylim(0, 105)
ax.xaxis.set_major_formatter(DateFormatter('%I:%M %p', tz='US/Central'))
ax.tick_params('x', rotation=90)
ax.set_title(f"{time.strftime('%Y %B %d')}")
ax.set_ylabel("Eclipse percentage")
ax.set_xlabel("Local time (US/Central)")
ax.grid()
Plot the timeseries for the 2023 annular eclipse. Note that the eclipse amount reaches a maximum of only ~90%, as expected.
plot_eclipse_timeseries(location, max2023)
Eclipse detected:
Partial solar eclipse starts at 2023-10-14 15:24:00.000 UTC
Partial solar eclipse ends at 2023-10-14 18:32:00.000 UTC
Plot the timeseries for the 2024 total eclipse. Since the eclipse amount reaches 100%, the above function also calculates the start/end of total eclipse.
plot_eclipse_timeseries(location, max2024)
plt.show()
Eclipse detected:
Partial solar eclipse starts at 2024-04-08 17:15:00.000 UTC
Total solar eclipse starts at 2024-04-08 18:33:44.000 UTC
Total solar eclipse ends at 2024-04-08 18:35:33.000 UTC
Partial solar eclipse ends at 2024-04-08 19:55:00.000 UTC
Total running time of the script: (0 minutes 3.976 seconds)
Note
Go to the end to download the full example code.
Reproducing the “Where is STEREO Today?” plot#
How to reproduce the “Where is STEREO Today?” plot.
This example uses capabilities in sunpy to reproduce the plot that can be found at https://stereo-ssc.nascom.nasa.gov/where.shtml. This example is not written as a tutorial, but there are related tutorials for learning about these capabilities:
import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import MultipleLocator
import astropy.units as u
from astropy.coordinates import Longitude
from sunpy.coordinates import HeliocentricEarthEcliptic, get_body_heliographic_stonyhurst, get_horizons_coord
from sunpy.time import parse_time
Define the time for the plot as the time when this script is run.
obstime = parse_time('now')
Define a convenience function to extract the first full orbit from a trajectory, assuming that the trajectory moves in the direction of positive ecliptic longitude.
hee_frame = HeliocentricEarthEcliptic(obstime=obstime)
def get_first_orbit(coord):
lon = coord.transform_to(hee_frame).spherical.lon
shifted = Longitude(lon - lon[0])
ends = np.flatnonzero(np.diff(shifted) < 0)
if ends.size > 0:
return coord[:ends[0]]
return coord
Obtain the locations and trajectories of the various planets and spacecraft. To ensure that each trajectory contains at least one full orbit, we request 700 days for each planet and 1 year for each spacecraft.
planets = ['Mercury', 'Venus', 'Earth', 'Mars']
times = obstime + np.arange(700) * u.day
planet_coords = {planet: get_first_orbit(get_body_heliographic_stonyhurst(planet, times))
for planet in planets}
stereo_a = get_horizons_coord('STEREO-A', obstime)
stereo_b = get_horizons_coord('STEREO-B', obstime)
missions = ['Parker Solar Probe', 'Solar Orbiter', 'BepiColombo']
mission_labels = {'Parker Solar Probe': 'PSP', 'Solar Orbiter': 'SO', 'BepiColombo': 'BEPICOLOMBO'}
mission_coords = {mission: get_first_orbit(get_horizons_coord(mission, {'start': obstime,
'stop': obstime + 1 * u.yr,
'step': '1d'}))
for mission in missions}
INFO: Obtained JPL HORIZONS location for STEREO-A (spacecraft) (-234) [sunpy.coordinates.ephemeris]
INFO: Obtained JPL HORIZONS location for STEREO-B (spacecraft) (-235) [sunpy.coordinates.ephemeris]
INFO: Obtained JPL HORIZONS location for Parker Solar Probe (spacecraft) (-96) [sunpy.coordinates.ephemeris]
INFO: Obtained JPL HORIZONS location for Solar Orbiter (spacecraft) (-144) [sunpy.coordinates.ephemeris]
INFO: Obtained JPL HORIZONS location for BepiColombo (Spacecraft) (-121) [sunpy.coordinates.ephemeris]
Define a convenience function for converting coordinates to plot positions in the ecliptic plane.
def coord_to_heexy(coord):
coord = coord.transform_to(hee_frame)
coord.representation_type = 'cartesian'
return coord.y.to_value('AU'), coord.x.to_value('AU')
Set Matplotlib settings to the desired appearance and initialize the axes.
mpl.rcParams.update({'figure.facecolor': 'black',
'axes.edgecolor': 'white',
'axes.facecolor': 'black',
'axes.labelcolor': 'white',
'axes.titlecolor': 'white',
'lines.linewidth': 1,
'xtick.color': 'white',
'xtick.direction': 'in',
'xtick.top': True,
'ytick.color': 'white',
'ytick.direction': 'in',
'ytick.right': True})
fig = plt.figure()
ax = fig.add_subplot()
ax.set_xlim(-2.15, 2.15)
ax.set_xlabel('Y (HEE)')
ax.xaxis.set_major_locator(MultipleLocator(1))
ax.xaxis.set_minor_locator(MultipleLocator(0.1))
ax.set_ylim(1.8, -1.8)
ax.set_ylabel('X (HEE)')
ax.yaxis.set_major_locator(MultipleLocator(1))
ax.yaxis.set_minor_locator(MultipleLocator(0.1))
ax.set_title(obstime.strftime('%d-%b-%Y %H:%M UT'))
ax.set_aspect('equal')
Draw the Sun-Earth line.
ax.plot([0, 0], [0, 2], linestyle='dotted', color='gray')
[<matplotlib.lines.Line2D object at 0x7f6f403b6d70>]
Draw Mercury, Venus, Earth, and Mars, with Earth formatted differently.
for planet, coord in planet_coords.items():
ax.plot(*coord_to_heexy(coord), linestyle='dashed', color='gray')
if planet == 'Earth':
color, markersize, offset = 'lime', 10, 0.1
else:
color, markersize, offset = 'gray', None, 0.05
x, y = coord_to_heexy(coord[0])
ax.plot(x, y, 'o', markersize=markersize, color=color)
ax.text(x + offset, y, planet, color=color)
Draw the STEREO spacecraft (without orbits), as well as Sun-STEREO lines.
for stereo, label, color in [(stereo_a, 'A', 'red'), (stereo_b, 'B', 'blue')]:
x, y = coord_to_heexy(stereo)
ax.plot([0, 5*x], [0, 5*y], linestyle='dotted', color='gray')
ax.plot(x, y, 'o', color=color)
ax.text(x + 0.1, y, label, color=color, fontsize=18)
Draw the Sun, which is at the origin by definition.
ax.plot(0, 0, 'o', markersize=15, color='yellow')
ax.text(0.12, 0, 'Sun', color='yellow')
Text(0.12, 0, 'Sun')
Finally, draw the various spacecraft, with Solar Orbiter colored differently.
for mission, coord in mission_coords.items():
color = 'magenta' if mission == 'Solar Orbiter' else 'orange'
ax.plot(*coord_to_heexy(coord), linestyle='dashed', color=color)
x, y = coord_to_heexy(coord[0])
ax.plot(x, y, 'o', color=color)
ax.text(x + 0.05, y, mission_labels[mission], color=color)
plt.show()
# This is necessary to reset the Matplotlib settings after plotting for our documentation.
# You don't need this in your own code.
mpl.rcParams.update(mpl.rcParamsDefault)
mpl.rcParams.update({'axes.titlecolor': 'black'})
Total running time of the script: (0 minutes 4.048 seconds)
Note
Go to the end to download the full example code.
Understanding Observer Orientation Relative to a Simulated Volume with astropy.coordinates#
This example demonstrates how to use the astropy.coordinates framework, combined with the solar
coordinate frameworks provided by sunpy, to find the intersection of a set of lines of sight
defined by an observer position with a simulated volume represented by a Cartesian box.
import itertools
import matplotlib.pyplot as plt
import numpy as np
import astropy.time
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.map
import sunpy.sun.constants
from sunpy.coordinates import Heliocentric, Helioprojective
First, we need to define the orientation of our box and the corresponding coordinate frame. We do this by specifying a coordinate in the Heliographic Stonyhurst frame (HGS) and then using that coordinate to define the coordinate frame of the simulation box in a Heliocentric Cartesian (HCC) frame. For more information on all of these frames, see Thompson (2006).
hcc_orientation = SkyCoord(lon=0*u.deg, lat=20*u.deg,
radius=sunpy.sun.constants.radius,
frame='heliographic_stonyhurst')
date = astropy.time.Time('2020-01-01')
frame_hcc = Heliocentric(observer=hcc_orientation, obstime=date)
Next, we need to define a coordinate frame for our synthetic spacecraft. This could be something like SDO or Solar Orbiter, but in this case we’ll just choose an interesting orientation at 1 AU. As before, we first need to define the observer position in HGS which we will then use to define the Helioprojective frame of our observer.
observer = SkyCoord(lon=45*u.deg, lat=10*u.deg, radius=1*u.AU, frame='heliographic_stonyhurst')
frame_hpc = Helioprojective(observer=observer, obstime=date)
Using our coordinate frame for synthetic observer, we can then create an empty map that represents our fake observation. We’ll create a function to do this since we’ll need to do it a few times. Note that we are using the center coordinate at the bottom boundary of our simulation box as the reference coordinate such that field-of-view of our synthetic simulation is centered on this point.
def make_fake_map(ref_coord):
instrument_data = np.nan * np.ones((25, 25))
instrument_header = sunpy.map.make_fitswcs_header(instrument_data,
ref_coord,
scale=u.Quantity([20, 20])*u.arcsec/u.pix)
return sunpy.map.Map(instrument_data, instrument_header)
instrument_map = make_fake_map(hcc_orientation.transform_to(frame_hpc))
Now that we have our map representing our fake observation, we can get the coordinates, in pixel space, of the center of each pixel. These will be our lines of sight.
map_indices = sunpy.map.all_pixel_indices_from_map(instrument_map).value.astype(int)
map_indices = map_indices.reshape((2, map_indices.shape[1]*map_indices.shape[2]))
We can then use the WCS of the map to find the associate world coordinate for each pixel coordinate. Note that by default, the “z” or distance coordinate in the HPC frame is assumed to lie on the solar surface. As such, for each LOS, we will add a z coordinate that spans from 99% to 101% of the observer radius. This gives us a reasonable range of distances that will intersect our simulation box.
lines_of_sight = []
distance = np.linspace(0.99, 1.01, 10000)*observer.radius
for indices in map_indices.T:
coord = instrument_map.wcs.pixel_to_world(*indices)
lines_of_sight.append(SkyCoord(Tx=coord.Tx, Ty=coord.Ty, distance=distance, frame=coord.frame))
We can do a simple visualization of all of our LOS on top of our synthetic map
fig = plt.figure()
ax = fig.add_subplot(projection=instrument_map)
instrument_map.plot(axes=ax, title=False)
instrument_map.draw_grid(axes=ax, color='k')
for los in lines_of_sight:
ax.plot_coord(los[0], color='C0', marker='.', ls='', markersize=1,)
The next step is to define our simulation box. We’ll choose somewhat arbitrary dimensions for our box and then compute the offsets of each of the corners from the center of the box.
box_dimensions = u.Quantity([100,100,200])*u.Mm
corners = list(itertools.product(box_dimensions[0]/2*[-1,1],
box_dimensions[1]/2*[-1,1],
box_dimensions[2]/2*[-1,1]))
We then define the edges of the box by considering all possible combinations of two corners and keeping only those combinations who vary along a single dimension.
edges = []
for possible_edges in itertools.combinations(corners,2):
diff_edges = u.Quantity(possible_edges[0])-u.Quantity(possible_edges[1])
if np.count_nonzero(diff_edges) == 1:
edges.append(possible_edges)
Finally, we can define the edge coordinates of the box by first creating a coordinate to represent the origin. This is easily computed from our point that defined the orientation since this is the point at which the box is tangent to the solar surface.
box_origin = hcc_orientation.transform_to(frame_hcc)
box_origin = SkyCoord(x=box_origin.x,
y=box_origin.y,
z=box_origin.z+box_dimensions[2]/2,
frame=box_origin.frame)
Using that origin, we can compute the coordinates of all edges.
edge_coords = []
for edge in edges:
edge_coords.append(SkyCoord(x=box_origin.x+u.Quantity([edge[0][0],edge[1][0]]),
y=box_origin.y+u.Quantity([edge[0][1],edge[1][1]]),
z=box_origin.z+u.Quantity([edge[0][2],edge[1][2]]),
frame=box_origin.frame))
Let’s overlay the simulation box on top of our fake map we created earlier to see if things look right.
fig = plt.figure()
ax = fig.add_subplot(projection=instrument_map)
instrument_map.plot(axes=ax, title=False)
instrument_map.draw_grid(axes=ax, color='k')
for edge in edge_coords:
ax.plot_coord(edge, color='k', ls='-', marker='')
ax.plot_coord(box_origin, color='r', marker='x')
ax.plot_coord(hcc_orientation, color='b', marker='x')
[<matplotlib.lines.Line2D object at 0x7f6f383efb80>]
Let’s combine all of these pieces by plotting the lines of sight and the simulation box on a single plot. We’ll also overlay the pixel grid of our fake image.
fig = plt.figure()
ax = fig.add_subplot(projection=instrument_map)
instrument_map.plot(axes=ax, title=False)
instrument_map.draw_grid(axes=ax, color='k')
ax.plot_coord(box_origin, marker='x', color='r', ls='', label='Simulation Box Center')
ax.plot_coord(hcc_orientation, color='b', marker='x', ls='', label='Simulation Box Bottom')
for i,edge in enumerate(edge_coords):
ax.plot_coord(edge, color='k', ls='-', label='Simulation Box' if i==0 else None)
# Plot the pixel center positions
for i,los in enumerate(lines_of_sight):
ax.plot_coord(los[0], color='C0', marker='.', label='LOS' if i==0 else None, markersize=1)
# Plot the edges of the pixel grid
xpix_edges = np.array(range(int(instrument_map.dimensions.x.value)+1))-0.5
ypix_edges = np.array(range(int(instrument_map.dimensions.y.value)+1))-0.5
ax.vlines(x=xpix_edges,
ymin=ypix_edges[0],
ymax=ypix_edges[-1],
color='k', ls='--', lw=.5, label='pixel grid')
ax.hlines(y=ypix_edges,
xmin=xpix_edges[0],
xmax=xpix_edges[-1],
color='k', ls='--', lw=.5,)
ax.legend(loc=2, frameon=False)
ax.set_xlim(-10, 35)
ax.set_ylim(-10, 35)
(-10.0, 35.0)
Finally, we have all of the pieces of information we need to understand whether a given LOS intersects the simulation box. First, we define a function that takes in the edge coordinates of our box and a LOS coordinate and returns to us a boolean mask of where that LOS coordinate falls in the box.
def is_coord_in_box(box_edges, coord):
box_edges = SkyCoord(box_edges)
coord_hcc = coord.transform_to(box_edges.frame)
in_x = np.logical_and(coord_hcc.x<box_edges.x.max(), coord_hcc.x>box_edges.x.min())
in_y = np.logical_and(coord_hcc.y<box_edges.y.max(), coord_hcc.y>box_edges.y.min())
in_z = np.logical_and(coord_hcc.z<box_edges.z.max(), coord_hcc.z>box_edges.z.min())
return np.all([in_x, in_y, in_z], axis=0)
Next we define another map, using a different orientation from our observer, so we can look at the intersection of the box and the many LOS from a different viewing angle.
new_obs = SkyCoord(lon=25*u.deg, lat=0*u.deg, radius=1*u.AU, frame='heliographic_stonyhurst')
earth_map = make_fake_map(
hcc_orientation.transform_to(Helioprojective(observer=new_obs, obstime=date))
)
Finally, we’ll create another visualization that combines the simulation box with the lines of sight and additional highlighting that shows where the LOS intersect the box.
fig = plt.figure()
ax = fig.add_subplot(projection=earth_map)
earth_map.plot(axes=ax, title=False)
earth_map.draw_grid(axes=ax, color='k')
for los in lines_of_sight:
ax.plot_coord(los, color='C0', marker='', ls='-', alpha=0.2)
for los in lines_of_sight:
inside_box = is_coord_in_box(edge_coords, los)
if inside_box.any():
ax.plot_coord(los[inside_box], color='C1', marker='', ls='-')
for edge in edge_coords:
ax.plot_coord(edge, color='k', ls='-')
ax.plot_coord(box_origin, marker='x', color='r')
ax.plot_coord(hcc_orientation, color='b', marker='x', ls='')
ax.set_xlim(-10, 35)
ax.set_ylim(-10, 35)
plt.show()
Total running time of the script: (0 minutes 33.834 seconds)
Note
Go to the end to download the full example code.
Visualizing 3D stereoscopic images#
How to make an anaglyph 3D image from a stereoscopic observation
We use a stereoscopic observation from July 2023, when the STEREO-A spacecraft and the SDO spacecraft were close in heliographic longitude. See the Wikipedia page on anaglyph 3D, which typically requires red-cyan glasses to visualize.
import copy
import matplotlib.pyplot as plt
from matplotlib.colors import Normalize
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
from sunpy.coordinates import Helioprojective
Download co-temporal SDO/AIA image STEREO/EUVI images. The EUVI map does not explicitly define their reference radius of the Sun, so we set it to be the same as for the AIA map to silence some informational messages.
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_STEREOSCOPIC_IMAGE)
euvi_map = sunpy.map.Map(sunpy.data.sample.EUVI_STEREOSCOPIC_IMAGE)
euvi_map.meta['rsun_ref'] = aia_map.meta['rsun_ref']
Verify that the angular separation between the two vantage points is a few degrees. If the angular separation much larger, the 3D effect will be bad.
print(euvi_map.observer_coordinate.separation(aia_map.observer_coordinate))
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/astropy/coordinates/baseframe.py:1781: NonRotationTransformationWarning: transforming other coordinates from <HeliographicStonyhurst Frame (obstime=2023-07-06T00:05:33.350, rsun=696000.0 km)> to <HeliographicStonyhurst Frame (obstime=2023-07-06T00:05:25.008, rsun=696000.0 km)>. Angular separation can depend on the direction of the transformation.
warnings.warn(NonRotationTransformationWarning(self, other_frame))
3d43m07.52520019s
Define a convenience function to reproject an input map to an observer in same direction, but at a distance of 1 AU, and then reproject both maps. The purpose is to make the Sun exactly the same size in pixels in both maps.
def reproject_to_1au(in_map):
header = sunpy.map.make_fitswcs_header(
(1000, 1000),
SkyCoord(
0*u.arcsec, 0*u.arcsec,
frame='helioprojective',
obstime=in_map.date,
observer=in_map.observer_coordinate.realize_frame(
in_map.observer_coordinate.represent_as('unitspherical') * u.AU
)
),
scale=(2.2, 2.2)*u.arcsec/u.pixel
)
with Helioprojective.assume_spherical_screen(in_map.observer_coordinate):
return in_map.reproject_to(header)
euvi_map = reproject_to_1au(euvi_map)
aia_map = reproject_to_1au(aia_map)
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:2757: SunpyUserWarning: rsun mismatch detected: EUVI-A 171.0 Angstrom 2023-07-06 00:05:25.rsun_meters=696000000.0 m; 2023-07-06 00:05:25.rsun_meters=695700000.0 m. This might cause unexpected results during reprojection.
warn_user("rsun mismatch detected: "
/home/docs/checkouts/readthedocs.org/user_builds/sunpy/conda/latest/lib/python3.10/site-packages/sunpy/map/mapbase.py:2757: SunpyUserWarning: rsun mismatch detected: AIA 171.0 Angstrom 2023-07-06 00:05:33.rsun_meters=696000000.0 m; 2023-07-06 00:05:33.rsun_meters=695700000.0 m. This might cause unexpected results during reprojection.
warn_user("rsun mismatch detected: "
Define linear scaling for the two images so that they look the same. The values here are empirical.
euvi_norm = Normalize(vmin=750, vmax=1e4, clip=True)
aia_norm = Normalize(vmin=0, vmax=2.1e3, clip=True)
Plot the two maps side by side. Those who are able to see 3D images by defocusing their eyes can see the 3D effect without the later anaglyph image.
fig = plt.figure(figsize=(8, 4))
fig.subplots_adjust(wspace=0)
ax1 = fig.add_subplot(121, projection=euvi_map)
euvi_map.plot(axes=ax1, cmap='gray', norm=euvi_norm)
ax1.grid(False)
ax1.set_title('STEREO/EUVI')
ax2 = fig.add_subplot(122, projection=aia_map)
aia_map.plot(axes=ax2, cmap='gray', norm=aia_norm)
ax2.coords[1].set_ticks_visible(False)
ax2.coords[1].set_ticklabel_visible(False)
ax2.grid(False)
ax2.set_title('SDO/AIA')
Text(0.5, 1.0, 'SDO/AIA')
We will make a color anaglyph 3D image by creating two colormaps based on an
initial colormap (here, 'sdoaia171'). The left-eye colormap is just the
red channel, and the right-eye colormap is just the green and blue channels.
Thus, we zero out the appropriate channels in the two colormaps.
cmap_left = copy.deepcopy(plt.get_cmap('sdoaia171'))
cmap_right = copy.deepcopy(cmap_left)
cmap_left._segmentdata['blue'] = [(0, 0, 0), (1, 0, 0)]
cmap_left._segmentdata['green'] = [(0, 0, 0), (1, 0, 0)]
cmap_right._segmentdata['red'] = [(0, 0, 0), (1, 0, 0)]
Finally, we build the anaglyph 3D image. To easily plot in RGB color
channels, we will be passing a NxMx3 array to matplotlib’s imshow. That
array needs to be manually normalized (to be between 0 and 1) and colorized
(mapped into the 3 color channels). The colorizing actually adds a fourth
layer for the alpha layer, so we discard that layer.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
euvi_data = euvi_norm(euvi_map.data)
aia_data = aia_norm(aia_map.data)
ax.imshow((cmap_left(euvi_data) + cmap_right(aia_data))[:, :, 0:3], origin='lower')
ax.set_xlabel('Helioprojective Longitude (Solar-X)')
ax.set_ylabel('Helioprojective Latitude (Solar-Y)')
plt.show()
Total running time of the script: (0 minutes 3.970 seconds)
How-To Guides#
These how-to guides provide examples of how to perform specific tasks with sunpy. They are recipes that do not provide much in-depth explanation and assume you have some knowledge of what sunpy is and how it works. If you’re starting fresh you might want to check out the The sunpy tutorial first.
Access coordinate components#
Individual coordinates can be accessed via attributes on the SkyCoord object, but the names of the components of the coordinates can depend on the the frame and the chosen representation (e.g., Cartesian versus spherical).
Helioprojective#
For the helioprojective frame, the theta_x and theta_y components are accessed Tx and Ty, respectively:
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
>>> from sunpy.coordinates import frames
>>> c = SkyCoord(-500*u.arcsec, 100*u.arcsec, frame=frames.Helioprojective)
>>> c.Tx
<Longitude -500. arcsec>
>>> c.Ty
<Latitude 100. arcsec>
Heliocentric#
Heliocentric is typically used with Cartesian components:
>>> c = SkyCoord(-72241.0*u.km, 361206.1*u.km, 589951.4*u.km, frame=frames.Heliocentric)
>>> c.x
<Quantity -72241. km>
>>> c.y
<Quantity 361206.1 km>
>>> c.z
<Quantity 589951.4 km>
HeliographicStonyhurst and HeliographicCarrington#
Both of the heliographic frames have the components of latitude, longitude and radius:
>>> c = SkyCoord(70*u.deg, -30*u.deg, 1*u.AU, frame=frames.HeliographicStonyhurst)
>>> c.lat
<Latitude -30. deg>
>>> c.lon
<Longitude 70. deg>
>>> c.radius
<Distance 1. AU>
Heliographic Stonyhurst, when used with Cartesian components, is known as Heliocentric Earth Equatorial (HEEQ).
Here’s an example of how to use HeliographicStonyhurst for HEEQ coordinates:
>>> c = SkyCoord(-72241.0*u.km, 361206.1*u.km, 589951.4*u.km,
... representation_type='cartesian', frame=frames.HeliographicStonyhurst)
>>> c.x
<Quantity -72241. km>
>>> c.y
<Quantity 361206.1 km>
>>> c.z
<Quantity 589951.4 km>
How to create a sunpy Map#
One of the primary goals of the Map interface is to make it as easy as possible to create a Map. As such, you can pass many different kinds of inputs to Map. These are listed below.
File name#
If you have a FITS file, this is the easiest and recommended way to create a Map.
This can be either a string or a Path.
>>> import pathlib
>>> import sunpy.map
>>> import sunpy.data.sample
>>> my_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
>>> my_map = sunpy.map.Map('file.fits')
>>> my_map = sunpy.map.Map(pathlib.Path('file.fits'))
>>> sub_dir = pathlib.Path('local_dir/sub_dir')
>>> my_map = sunpy.map.Map(sub_dir / 'another_file.fits')
Directory containing FITS files#
If there is more than one FITS file in the directory, this will return a list of Map objects.
>>> my_maps = sunpy.map.Map('local_dir/sub_dir')
>>> my_maps = sunpy.map.Map(sub_dir)
Array and astropy.io.fits.Header#
If needed, this way can be used to modify the header before passing it to Map.
>>> import astropy.io.fits
>>> with astropy.io.fits.open(sunpy.data.sample.AIA_171_IMAGE) as hdul:
... data = hdul[1].data
... header = hdul[1].header
>>> my_map = sunpy.map.Map(data, header)
These data header pairs can also be passed as a tuple,
>>> my_map = sunpy.map.Map((data, header))
Data array and a MetaDict object#
This includes any base class of MetaDict, including dict or collections.OrderedDict.
>>> import sunpy.util.metadata
>>> meta = sunpy.util.metadata.MetaDict(header)
>>> my_map = sunpy.map.Map(data, meta)
Data array and an astropy.wcs.WCS object#
>>> import astropy.wcs
>>> wcs = astropy.wcs.WCS(header=header)
>>> my_map = sunpy.map.Map(data, wcs)
Glob patterns#
If the glob pattern matches more than one FITS file, this will return a list of Map objects.
>>> my_map = sunpy.map.Map('eit_*.fits')
URL#
>>> sample_data_url = 'http://data.sunpy.org/sunpy/v1/AIA20110607_063302_0171_lowres.fits'
>>> my_map = sunpy.map.Map(sample_data_url)
Combinations of any of the above#
These can either be in a list or as separate arguments. As with the case of a directory or glob pattern, this will return multiple Map objects.
>>> my_map = sunpy.map.Map(['file1.fits', 'file2.fits', 'file3.fits', 'directory1/'])
>>> my_map = sunpy.map.Map((data, header), data, meta, 'file1.fits', sample_data_url, 'eit_*.fits')
How to draw a rectangle on a Map#
sunpy provides a convenient method called draw_quadrangle() to draw rectangles on maps.
In this guide, we will demonstrate four different methods to draw a rectangle on a sunpy.map.Map.
Specify corners with a single SkyCoord#
We will use one SkyCoord to represent the two opposite corners.
import matplotlib.pyplot as plt
import astropy.units as u
from astropy.coordinates import SkyCoord
import sunpy.data.sample
import sunpy.map
aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
coords = SkyCoord(Tx=(100, 500) * u.arcsec, Ty=(200, 500) * u.arcsec,frame=aia_map.coordinate_frame)
aia_map.draw_quadrangle(coords, axes=ax, edgecolor="blue")
plt.show()
(Source code, png, hires.png, pdf)
Specify corners with separate SkyCoord#
We will use two SkyCoord to represent the two opposite corners.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
bottom_left = SkyCoord(100 * u.arcsec, 200 * u.arcsec, frame=aia_map.coordinate_frame)
top_right = SkyCoord(500 * u.arcsec, 500 * u.arcsec, frame=aia_map.coordinate_frame)
aia_map.draw_quadrangle(bottom_left, axes=ax, top_right=top_right, edgecolor="green")
plt.show()
(Source code, png, hires.png, pdf)
Specify one corner with a width and height#
We will use one SkyCoord to represent the bottom left and supply a width and height to complete the rectangle.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
bottom_left = SkyCoord(100 * u.arcsec, 200 * u.arcsec, frame=aia_map.coordinate_frame)
width = 400 * u.arcsec
height = 300 * u.arcsec
aia_map.draw_quadrangle(bottom_left, axes=ax, width=width, height=height, edgecolor="yellow")
plt.show()
(Source code, png, hires.png, pdf)
Using pixel coordinates#
We will use a SkyCoord to work out the pixel coordinates instead of using coordinates as we do above.
fig = plt.figure()
ax = fig.add_subplot(projection=aia_map)
aia_map.plot(axes=ax, clip_interval=(1, 99.99)*u.percent)
bottom_left = aia_map.wcs.pixel_to_world(551 * u.pixel, 594 * u.pixel)
top_right = aia_map.wcs.pixel_to_world(717 * u.pixel, 719 * u.pixel)
aia_map.draw_quadrangle(bottom_left, axes=ax, top_right=top_right, edgecolor="red")
plt.show()
(Source code, png, hires.png, pdf)
Create Coordinate Objects#
The easiest interface to work with coordinates is through the SkyCoord class:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import frames
>>> coord = SkyCoord(-100*u.arcsec, 500*u.arcsec, frame=frames.Helioprojective)
>>> coord = SkyCoord(x=-72241.0*u.km, y=361206.1*u.km, z=589951.4*u.km, frame=frames.Heliocentric)
>>> coord = SkyCoord(70*u.deg, -30*u.deg, frame=frames.HeliographicStonyhurst)
>>> coord
<SkyCoord (HeliographicStonyhurst: obstime=None, rsun=695700.0 km): (lon, lat) in deg
(70., -30.)>
It is also possible to use strings to specify the frame but in that case make sure to explicitly import sunpy.coordinates as it registers sunpy’s coordinate frames the Astropy coordinates framework:
>>> import sunpy.coordinates
>>> coord = SkyCoord(-100*u.arcsec, 500*u.arcsec, frame='helioprojective', observer='earth')
>>> coord
<SkyCoord (Helioprojective: obstime=None, rsun=695700.0 km, observer=earth): (Tx, Ty) in arcsec
(-100., 500.)>
SkyCoord and all coordinate frames support array coordinates.
These work the same as single-value coordinates, but they store multiple coordinates in a single object.
When you’re going to apply the same operation to many different coordinates, this is a better choice than a list of SkyCoord objects, because it will be much faster than applying the operation to each SkyCoord in a for loop:
>>> coord = SkyCoord([-500, 400]*u.arcsec, [100, 200]*u.arcsec, frame=frames.Helioprojective)
>>> coord
<SkyCoord (Helioprojective: obstime=None, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec
[(-500., 100.), ( 400., 200.)]>
>>> coord[0]
<SkyCoord (Helioprojective: obstime=None, rsun=695700.0 km, observer=None): (Tx, Ty) in arcsec
(-500., 100.)>
Create Custom Maps by Hand#
It is possible to create Maps using custom data, e.g., from a simulation or an observation from a data source that is not explicitly supported by sunpy.
To do this, you need to provide sunpy.map.Map with both the data array as well as appropriate metadata.
The metadata informs sunpy.map.Map of the correct coordinate information associated with the data array and should be provided to sunpy.map.Map in the form of a header as a dict or MetaDict.
See this Generating a map from data array for a brief demonstration of generating a Map from a data array.
The keys required for the header information follow the FITS standard.
sunpy provides a Map header helper function to assist the user in creating a header that contains the correct meta information needed to create a Map.
The utility function make_fitswcs_header() will return a header with the appropriate FITS keywords once the Map data array and an astropy.coordinates.SkyCoord or sunpy.coordinates.frames is provided.
All the metadata keywords that a Map will parse along with their description are listed in the Meta Keywords at the end of this page.
make_fitswcs_header() also takes optional keyword arguments including reference_pixel and scale that describe the pixel coordinate at the reference coordinate (defined by the SkyCoord) and the spatial scale of the pixels, respectively.
Here’s an example of creating a header from some generic data and an astropy.coordinates.SkyCoord:
>>> import numpy as np
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
>>> from sunpy.coordinates import frames
>>> from sunpy.map.header_helper import make_fitswcs_header
>>> data = np.arange(0, 100).reshape(10, 10)
>>> reference_coord = SkyCoord(0*u.arcsec, 0*u.arcsec, obstime = '2013-10-28', observer = 'earth', frame = frames.Helioprojective)
>>> header = make_fitswcs_header(data, reference_coord)
>>> for key, value in header.items():
... print(f"{key}: {value}")
wcsaxes: 2
crpix1: 5.5
crpix2: 5.5
cdelt1: 1.0
cdelt2: 1.0
cunit1: arcsec
cunit2: arcsec
ctype1: HPLN-TAN
ctype2: HPLT-TAN
crval1: 0.0
crval2: 0.0
lonpole: 180.0
latpole: 0.0
mjdref: 0.0
date-obs: 2013-10-28T00:00:00.000
rsun_ref: 695700000.0
dsun_obs: 148644585949.49
hgln_obs: 0.0
hglt_obs: 4.7711570596394
naxis: 2
naxis1: 10
naxis2: 10
pc1_1: 1.0
pc1_2: -0.0
pc2_1: 0.0
pc2_2: 1.0
rsun_obs: 965.3829548285768
From this we can see now that the function returned a sunpy.util.MetaDict that populated the standard FITS keywords with information provided by the passed astropy.coordinates.SkyCoord, and the data array.
Since the reference_pixel and scale keywords were not passed in the example above, the values of the “CRPIXn” and “CDELTn” keys were set to the center of the data array and 1, respectively, by default.
These keywords can be passed to the function in the form of an astropy.units.Quantity with associated units.
Here’s another example of passing reference_pixel and scale to the function:
>>> reference_pixel = u.Quantity([5, 5], u.pixel)
>>> scale = u.Quantity([2, 2], u.arcsec/u.pixel)
>>> header = make_fitswcs_header(data, reference_coord, reference_pixel=reference_pixel, scale=scale)
>>> for key, value in header.items():
... print(f"{key}: {value}")
wcsaxes: 2
crpix1: 6.0
crpix2: 6.0
cdelt1: 2.0
cdelt2: 2.0
cunit1: arcsec
cunit2: arcsec
ctype1: HPLN-TAN
ctype2: HPLT-TAN
crval1: 0.0
crval2: 0.0
lonpole: 180.0
latpole: 0.0
mjdref: 0.0
date-obs: 2013-10-28T00:00:00.000
rsun_ref: 695700000.0
dsun_obs: 148644585949.49
hgln_obs: 0.0
hglt_obs: 4.7711570596394
naxis: 2
naxis1: 10
naxis2: 10
pc1_1: 1.0
pc1_2: -0.0
pc2_1: 0.0
pc2_2: 1.0
rsun_obs: 965.3829548285768
As we can see, a list of WCS and observer meta information is contained within the generated headers, however we may want to include other meta information including the observatory name, the wavelength and wavelength unit of the observation.
Any valid FITS WCS keywords can be passed to the make_fitswcs_header() and will then populate the returned MetaDict header.
Furthermore, the following observation keywords can be passed to the make_fitswcs_header function: observatory, instrument, telescope, wavelength, exposure.
An example of creating a header with these additional keywords:
>>> header = make_fitswcs_header(data, reference_coord, reference_pixel=reference_pixel, scale=scale,
... telescope='Test case',
... instrument='UV detector',
... wavelength=1000*u.angstrom)
>>> for key, value in header.items():
... print(f"{key}: {value}")
wcsaxes: 2
crpix1: 6.0
crpix2: 6.0
cdelt1: 2.0
cdelt2: 2.0
cunit1: arcsec
cunit2: arcsec
ctype1: HPLN-TAN
ctype2: HPLT-TAN
crval1: 0.0
crval2: 0.0
lonpole: 180.0
latpole: 0.0
mjdref: 0.0
date-obs: 2013-10-28T00:00:00.000
rsun_ref: 695700000.0
dsun_obs: 148644585949.49
hgln_obs: 0.0
hglt_obs: 4.7711570596394
instrume: UV detector
telescop: Test case
wavelnth: 1000.0
waveunit: Angstrom
naxis: 2
naxis1: 10
naxis2: 10
pc1_1: 1.0
pc1_2: -0.0
pc2_1: 0.0
pc2_2: 1.0
rsun_obs: 965.3829548285768
From these header MetaDict’s that are generated, we can now create a custom map:
>>> import sunpy.map
>>> my_map = sunpy.map.Map(data, header)
>>> my_map
<sunpy.map.mapbase.GenericMap object at ...>
SunPy Map
---------
Observatory: Test case
Instrument: UV detector
Detector:
Measurement: 1000.0 Angstrom
Wavelength: 1000.0 Angstrom
Observation Date: 2013-10-28 00:00:00
Exposure Time: Unknown
Dimension: [10. 10.] pix
Coordinate System: helioprojective
Scale: [2. 2.] arcsec / pix
Reference Pixel: [5. 5.] pix
Reference Coord: [0. 0.] arcsec
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
[20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
[40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
[50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
[60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
[70, 71, 72, 73, 74, 75, 76, 77, 78, 79],
[80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
[90, 91, 92, 93, 94, 95, 96, 97, 98, 99]])
Keyword |
Description |
|---|---|
cunit1 |
Units of the coordinate increments along naxis1 e.g. arcsec (required) |
cunit2 |
Units of the coordinate increments along naxis2 e.g. arcsec (required) |
crval1 |
Coordinate value at reference point on naxis1 (required) |
crval2 |
Coordinate value at reference point on naxis2 (required) |
cdelt1 |
Spatial scale of pixels for naxis1, i.e. coordinate increment at reference point |
cdelt2 |
Spatial scale of pixels for naxis2, i.e. coordinate increment at reference point |
crpix1 |
Pixel coordinate at reference point naxis1 |
crpix2 |
Pixel coordinate at reference point naxis2 |
ctype1 |
Coordinate type projection along naxis1 of data e.g. HPLT-TAN |
ctype2 |
Coordinate type projection along naxis2 of data e.g. HPLN-TAN |
hgln_obs |
Heliographic longitude of observation |
hglt_obs |
Heliographic latitude of observation |
dsun_obs |
distance to Sun from observation in metres |
rsun_obs |
radius of Sun in meters from observation |
dateobs |
date of observation e.g. 2013-10-28 00:00 |
date_obs |
date of observation e.g. 2013-10-28 00:00 |
rsun_ref |
reference radius of Sun in meters |
solar_r |
radius of Sun in meters from observation |
radius |
radius of Sun in meters from observation |
crln_obs |
Carrington longitude of observation |
crlt_obs |
Heliographic latitude of observation |
solar_b0 |
Solar B0 angle |
detector |
name of detector e.g. AIA |
exptime |
exposure time of observation, in seconds e.g 2 |
instrume |
name of instrument |
wavelnth |
wavelength of observation |
waveunit |
unit for which observation is taken e.g. angstom |
obsrvtry |
name of observatory of observation |
telescop |
name of telescope of observation |
lvl_num |
FITS processing level |
crota2 |
Rotation of the horizontal and vertical axes in degrees |
PC1_1 |
Matrix element PCi_j describing the rotation required to align solar North with the top of the image. |
PC1_2 |
Matrix element PCi_j describing the rotation required to align solar North with the top of the image. |
PC2_1 |
Matrix element PCi_j describing the rotation required to align solar North with the top of the image. |
PC2_2 |
Matrix element PCi_j describing the rotation required to align solar North with the top of the image. |
CD1_1 |
Matrix element CDi_j describing the rotation required to align solar North with the top of the image. |
CD1_2 |
Matrix element CDi_j describing the rotation required to align solar North with the top of the image. |
CD2_1 |
Matrix element CDi_j describing the rotation required to align solar North with the top of the image. |
CD2_2 |
Matrix element CDi_j describing the rotation required to align solar North with the top of the image. |
Create Custom TimeSeries#
Sometimes you will have data that you want to transform into a TimeSeries.
You can use the factory to create a GenericTimeSeries from a variety of data sources currently including pandas.DataFrame and astropy.table.Table.
Creating a TimeSeries from a Pandas DataFrame#
A TimeSeries object must be supplied with some data when it is created. The data can either be in your current Python session, in a local file, or in a remote file. Let’s create some data and pass it into a TimeSeries object:
>>> import numpy as np
>>> intensity = np.sin(np.arange(0, 12 * np.pi, ((12 * np.pi) / (24*60))))
This creates a basic numpy array of values representing a sine wave.
We can use this array along with a suitable time storing object (such as Astropy time or a list of datetime objects) to make a Pandas DataFrame.
A suitable list of times must contain the same number of values as the data, this can be created using:
>>> import datetime
>>> base = datetime.datetime.today()
>>> times = [base - datetime.timedelta(minutes=x) for x in range(24*60, 0, -1)]
The Pandas DataFrame will use the dates list as the index:
>>> from pandas import DataFrame
>>> data = DataFrame(intensity, index=times, columns=['intensity'])
This DataFrame can then be used to construct a TimeSeries:
>>> import astropy.units as u
>>> import sunpy.timeseries as ts
>>> header = {'key': 'value'}
>>> units = {'intensity': u.W/u.m**2}
>>> ts_custom = ts.TimeSeries(data, header, units)
Creating Custom TimeSeries from an Astropy Table#
A Pandas DataFrame is the underlying object used to store the data within a TimeSeries, so the above example is the most lightweight to create a custom TimeSeries, but being scientific data it will often be more convenient to use an Astropy Table to create a TimeSeries.
An advantage of this method is it allows you to include metadata and Astropy Quantity values, which are both supported in tables, without additional arguments.
For example:
>>> from astropy.time import Time
>>> from astropy.table import Table
>>> base = datetime.datetime.today()
>>> times = [base - datetime.timedelta(minutes=x) for x in range(24*60, 0, -1)]
>>> intensity = u.Quantity(np.sin(np.arange(0, 12 * np.pi, ((12 * np.pi) / (24*60)))), u.W/u.m**2)
>>> tbl_meta = {'t_key':'t_value'}
>>> table = Table([times, intensity], names=['time', 'intensity'], meta=tbl_meta)
>>> table.add_index('time')
>>> ts_table = ts.TimeSeries(table)
Note that due to the properties of the Time object, this will be a mixin column which since it is a single object, limits the versatility of the Table a little.
For more on mixin columns see the Astropy docs.
The units will be taken from the table quantities for each column, the metadata will simply be the table.meta dictionary.
You can also explicitly add metadata and units, these will be added to the relevant dictionaries using the dictionary update method, with the explicit user-given values taking precedence:
>>> from collections import OrderedDict
>>> from sunpy.util.metadata import MetaDict
>>> meta = MetaDict({'key':'value'})
>>> units = OrderedDict([('intensity', u.W/u.m**2)])
>>> ts_table = ts.TimeSeries(table, meta, units)
Fixing incorrect metadata#
There will be times where you will come across a FITS files with either incorrect, missing or unparsable metadata and reading these files into Map will cause an error.
Therefore, to load these files into a Map, you will need to correct the metadata beforehand.
In the example below, the units in the FITS header, as controlled by the CUNIT1 and CUNIT2 keywords, are incorrect.
Before loading the file into a Map, we will correct these keywords to have the correct units.
>>> from astropy.io import fits
>>> from sunpy.map import Map
>>> import sunpy.data.sample
>>> filepath = sunpy.data.sample.AIA_171_IMAGE
>>> data, header = fits.getdata(filepath, header=True)
>>> # Note that it is case insensitive for the keys
>>> header['cunit1'] = 'arcsec'
>>> header['cunit2'] = 'arcsec'
>>> updated_map = Map(data, header)
This applies for any keyword in the FITS standard.
Manipulate grids lines when plotting Maps#
Underneath the hood, sunpy.map.GenericMap.plot uses WCSAxes to have to ability to plot images in world coordinates.
This means that sometimes the standard matplotlib methods to manipulate grid lines may not work as expected.
So you have to access the WCSAxes object to manipulate the grid lines.
Astropy have a guide on how to manipulate grid lines in WCSAxes.
Here we provide a solar example of how to manipulate grid lines when plotting a GenericMap.
Turning off the Helioprojective grid#
By default sunpy.map.GenericMap.plot draws a grid tracing the lines of helioprojective latitude and longitude, as below:
import matplotlib.pyplot as plt
import astropy.units as u
import sunpy.map
from sunpy.data.sample import HMI_LOS_IMAGE
smap = sunpy.map.Map(HMI_LOS_IMAGE)
fig = plt.figure()
ax = fig.add_subplot(111, projection=smap)
smap.plot(axes=ax)
plt.show()
(Source code, png, hires.png, pdf)
In some cases it may be desirable to remove this grid.
Since the underlying axes is a WCSAxes, you will need to access the grid method on the axes’ object coord attribute:
fig = plt.figure()
ax = fig.add_subplot(111, projection=smap)
smap.plot(axes=ax)
ax.coords[0].grid(draw_grid=False) # Disable grid for 1st axis
ax.coords[1].grid(draw_grid=False) # Disable grid for 2nd axis
plt.show()
(Source code, png, hires.png, pdf)
Changing the appearance of Helioprojective grid#
As the Helioprojective grid is applied by WCSAxes, if you want to change the appeerance of the grid lines, you will need to access the grid method on the axes’ object coord attribute and pass the desired keywords to it:
fig = plt.figure()
ax = fig.add_subplot(111, projection=smap)
smap.plot(axes=ax)
ax.coords[0].grid(color='yellow', linestyle='solid', alpha=0.5)
ax.coords[1].grid(color='red', linestyle='dotted', alpha=0.5)
plt.show()
(Source code, png, hires.png, pdf)
Changing the appearance of Heliographic grid#
sunpy.map.GenericMap.draw_grid allows users to overlay a Heliographic grid on their solar image plot.
This method does not explicitly provide many options for the appearance of the grid lines.
Instead it accepts keyword arguments and transparently passes them onto the underlying infrastructure.
Therefore to change the width of the lines to say, 0.25, and their style to say, dotted, do the following:
fig = plt.figure()
ax = fig.add_subplot(111, projection=smap)
smap.plot(axes=ax)
smap.draw_grid(axes=ax, linewidth=0.25, linestyle="dotted")
plt.show()
(Source code, png, hires.png, pdf)
Parse times with parse_time#
>>> import time
>>> from datetime import date, datetime
>>> import numpy as np
>>> import pandas
>>> from sunpy.time import parse_time
The following examples show how to use sunpy.time.parse_time to parse various time formats, including both strings and objects, into an astropy.time.Time object.
Strings#
>>> parse_time('1995-12-31 23:59:60')
<Time object: scale='utc' format='isot' value=1995-12-31T23:59:60.000>
This also works with the scale= keyword argument (See this list: for the list of all allowed scales):
>>> parse_time('2012:124:21:08:12', scale='tai')
<Time object: scale='tai' format='isot' value=2012-05-03T21:08:12.000>
Tuples#
>>> parse_time((1998, 11, 14))
<Time object: scale='utc' format='isot' value=1998-11-14T00:00:00.000>
>>> parse_time((2001, 1, 1, 12, 12, 12, 8899))
<Time object: scale='utc' format='isot' value=2001-01-01T12:12:12.009>
time.struct_time#
>>> parse_time(time.gmtime(0))
<Time object: scale='utc' format='isot' value=1970-01-01T00:00:00.000>
datetime.datetime and datetime.date#
>>> parse_time(datetime(1990, 10, 15, 14, 30))
<Time object: scale='utc' format='datetime' value=1990-10-15 14:30:00>
>>> parse_time(date(2023, 4, 22))
<Time object: scale='utc' format='iso' value=2023-04-22 00:00:00.000>
pandas time objects#
pandas.Timestamp, pandas.Series and pandas.DatetimeIndex
>>> parse_time(pandas.Timestamp(datetime(1966, 2, 3)))
<Time object: scale='utc' format='datetime64' value=1966-02-03T00:00:00.000000000>
>>> parse_time(pandas.Series([[datetime(2012, 1, 1, 0, 0), datetime(2012, 1, 2, 0, 0)],
... [datetime(2012, 1, 3, 0, 0), datetime(2012, 1, 4, 0, 0)]]))
<Time object: scale='utc' format='datetime' value=[[datetime.datetime(2012, 1, 1, 0, 0) datetime.datetime(2012, 1, 2, 0, 0)]
[datetime.datetime(2012, 1, 3, 0, 0) datetime.datetime(2012, 1, 4, 0, 0)]]>
>>> parse_time(pandas.DatetimeIndex([datetime(2012, 1, 1, 0, 0),
... datetime(2012, 1, 2, 0, 0),
... datetime(2012, 1, 3, 0, 0),
... datetime(2012, 1, 4, 0, 0)]))
<Time object: scale='utc' format='datetime' value=[datetime.datetime(2012, 1, 1, 0, 0)
datetime.datetime(2012, 1, 2, 0, 0)
datetime.datetime(2012, 1, 3, 0, 0)
datetime.datetime(2012, 1, 4, 0, 0)]>
numpy.datetime64#
>>> parse_time(np.datetime64('2014-02-07T16:47:51.008288123'))
<Time object: scale='utc' format='isot' value=2014-02-07T16:47:51.008>
>>> parse_time(np.array(['2014-02-07T16:47:51.008288123', '2014-02-07T18:47:51.008288123'],
... dtype='datetime64'))
<Time object: scale='utc' format='isot' value=['2014-02-07T16:47:51.008' '2014-02-07T18:47:51.008']>
Formats handled by astropy.time.Time#
See this list of all the allowed formats.
>>> parse_time(1234.0, format='jd')
<Time object: scale='utc' format='jd' value=1234.0>
>>> parse_time('B1950.0', format='byear_str')
<Time object: scale='tt' format='byear_str' value=B1950.000>
anytim output#
Format output by the anytim routine in SolarSoft (see the documentation for TimeUTime for more information):
>>> parse_time(662738003, format='utime')
<Time object: scale='utc' format='utime' value=662738003.0>
anytim2tai output#
Format output by the anytim2tai routine in SolarSoft (see the documentation for TimeTaiSeconds for more information):
>>> parse_time(1824441848, format='tai_seconds')
<Time object: scale='tai' format='tai_seconds' value=1824441848.0>
Use the Remote Data Manager#
Often, data prep or analysis functions require files to be downloaded from a remote server.
The remote data manager handles the usage of remote data files including file verification using hashes.
For example, say a function, test_function, requires the remote data file: predicted-sunspot-radio-flux.txt, which has the SHA256 hash “4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11”.
To ensure that this exact version of the file is downloaded when test_function is called, use the require decorator.
>>> from sunpy.data import manager
>>> @manager.require('test_file',
... ['http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt'],
... '4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11')
... def test_function():
... pass
The first time the function is called, the file will be downloaded and then cached such that subsequent function calls will not trigger a download.
>>> test_function() # The file will be downloaded
>>> test_function() # No download
To access the downloaded file inside the function, use the get() method,
>>> @manager.require('test_file',
... ['http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt'],
... '4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11')
... def test_function():
... return manager.get('test_file')
To call a function that uses the data manager, but skip the hash check, use the skip_hash_check context manager.
>>> with manager.skip_hash_check():
... test_function()
PosixPath('.../sunpy/data_manager/predicted-sunspot-radio-flux.txt')
To replace the required file with another version, use the override_file context manager.
>>> with manager.override_file('test_file', 'http://data.sunpy.org/sample-data/AIA20110319_105400_0171.fits'):
... test_function()
PosixPath('.../sunpy/data_manager/AIA20110319_105400_0171.fits')
Search for multiple wavelengths with Fido#
Use the Wavelength to search for a particular wavelength:
>>> from astropy import units as u
>>> from sunpy.net import Fido
>>> from sunpy.net import attrs as a
>>> time_range = a.Time("2022-02-20 00:00:00", "2022-02-20 00:00:30")
>>> aia_search = Fido.search(time_range,
... a.Instrument.aia,
... a.Wavelength(171*u.angstrom))
>>> aia_search
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
2 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 135.578 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:09.000 2022-02-20 00:00:10.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:21.000 2022-02-20 00:00:22.000 SDO ... FULLDISK 64.64844
The “|” operator can be used to combine multiple wavelengths:
>>> aia_search = Fido.search(time_range,
... a.Instrument.aia,
... a.Wavelength(171*u.angstrom) | a.Wavelength(193*u.angstrom))
>>> aia_search
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
2 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 135.578 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:09.000 2022-02-20 00:00:10.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:21.000 2022-02-20 00:00:22.000 SDO ... FULLDISK 64.64844
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 203.366 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:04.000 2022-02-20 00:00:05.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:16.000 2022-02-20 00:00:17.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:28.000 2022-02-20 00:00:29.000 SDO ... FULLDISK 64.64844
When searching for more than two wavelengths, it is more practical to use the sunpy.net.attrs.AttrOr() function:
>>> wavelengths = [94, 131, 171, 193, 211]*u.angstrom
>>> aia_search = Fido.search(time_range,
... a.Instrument.aia,
... a.AttrOr([a.Wavelength(wav) for wav in wavelengths]))
>>> aia_search
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 5 Providers:
2 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 135.578 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:11.000 2022-02-20 00:00:12.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:23.000 2022-02-20 00:00:24.000 SDO ... FULLDISK 64.64844
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 203.366 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:06.000 2022-02-20 00:00:07.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:18.000 2022-02-20 00:00:19.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:30.000 2022-02-20 00:00:31.000 SDO ... FULLDISK 64.64844
2 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 135.578 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:09.000 2022-02-20 00:00:10.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:21.000 2022-02-20 00:00:22.000 SDO ... FULLDISK 64.64844
3 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 203.366 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:04.000 2022-02-20 00:00:05.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:16.000 2022-02-20 00:00:17.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:28.000 2022-02-20 00:00:29.000 SDO ... FULLDISK 64.64844
2 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 135.578 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2022-02-20 00:00:09.000 2022-02-20 00:00:10.000 SDO ... FULLDISK 64.64844
2022-02-20 00:00:21.000 2022-02-20 00:00:22.000 SDO ... FULLDISK 64.64844
Search the Virtual Solar Observatory (VSO)#
To search the Virtual Solar Observatory (VSO) for SDO AIA data in all channels over a given time range, use the timerange (Time) and the instrument (Instrument) attrs,
>>> import astropy.units as u
>>> from sunpy.net import Fido
>>> from sunpy.net import attrs as a
>>> time_range = a.Time('2020/03/04 00:00:15', '2020/03/04 00:00:30')
>>> result = Fido.search(time_range, a.Instrument.aia)
>>> result
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
11 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 745.677 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:16.000 2020-03-04 00:00:17.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:17.000 2020-03-04 00:00:18.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:18.000 2020-03-04 00:00:19.000 SDO ... FULLDISK 64.64844
... ... ... ... ... ...
2020-03-04 00:00:28.000 2020-03-04 00:00:29.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:29.000 2020-03-04 00:00:30.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:30.000 2020-03-04 00:00:31.000 SDO ... FULLDISK 64.64844
You can also search for specific physical observables using Physobs.
For example, you can search for line-of-sight (LOS) magnetic field measurements from HMI in the same time range,
>>> Fido.search(time_range, a.Instrument.hmi, a.Physobs.los_magnetic_field)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:26.000 2020-03-04 00:00:27.000 SDO ... FULLDISK -0.00098
You can also use relational operators when constructing queries. For example, the AIA query above can also be expressed using the AND (&) operator,
>>> Fido.search(time_range & a.Instrument.aia)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
11 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 745.677 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:16.000 2020-03-04 00:00:17.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:17.000 2020-03-04 00:00:18.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:18.000 2020-03-04 00:00:19.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:21.000 2020-03-04 00:00:22.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:21.000 2020-03-04 00:00:22.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:23.000 2020-03-04 00:00:24.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:24.000 2020-03-04 00:00:25.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:28.000 2020-03-04 00:00:29.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:28.000 2020-03-04 00:00:29.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:29.000 2020-03-04 00:00:30.000 SDO ... FULLDISK 64.64844
2020-03-04 00:00:30.000 2020-03-04 00:00:31.000 SDO ... FULLDISK 64.64844
Additionally, multiple operators can be chained together to, for example, search for only the 171 channel,
>>> Fido.search(time_range & a.Instrument.aia & a.Wavelength(171*u.angstrom))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:21.000 2020-03-04 00:00:22.000 SDO ... FULLDISK 64.64844
The OR operator (|) can also be used to construct queries.
For example, to search for AIA data in this same time range from both the 94 and 171 channels,
>>> Fido.search(time_range,
... a.Instrument.aia,
... a.Wavelength(171*u.angstrom) | a.Wavelength(94*u.angstrom))
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 2 Providers:
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:21.000 2020-03-04 00:00:22.000 SDO ... FULLDISK 64.64844
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:23.000 2020-03-04 00:00:24.000 SDO ... FULLDISK 64.64844
These relational operators are particularly useful when searching a given time interval for multiple instruments. For example, to find the HMI LOS magnetic field data and the AIA 94 and 171 data in the given time interval,
>>> aia_params = a.Instrument.aia & (a.Wavelength(171*u.angstrom) | a.Wavelength(94*u.angstrom))
>>> hmi_params = a.Instrument.hmi & a.Physobs.los_magnetic_field
>>> Fido.search(time_range, aia_params | hmi_params)
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 3 Providers:
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:21.000 2020-03-04 00:00:22.000 SDO ... FULLDISK 64.64844
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Total estimated size: 67.789 Mbyte
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:23.000 2020-03-04 00:00:24.000 SDO ... FULLDISK 64.64844
1 Results from the VSOClient:
Source: http://vso.stanford.edu/cgi-bin/search
Start Time End Time Source ... Extent Type Size
... Mibyte
----------------------- ----------------------- ------ ... ----------- --------
2020-03-04 00:00:26.000 2020-03-04 00:00:27.000 SDO ... FULLDISK -0.00098
Transform between coordinate frames#
Both SkyCoord and BaseCoordinateFrame instances have a transform_to method.
This can be used to transform the frame to any other frame, either implemented in sunpy or in Astropy (see also Transforming between Systems).
An example of transforming the center of the solar disk to Carrington coordinates is:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import frames
>>> coord = SkyCoord(0*u.arcsec, 0*u.arcsec, frame=frames.Helioprojective, obstime="2017-07-26",
... observer="earth")
>>> coord
<SkyCoord (Helioprojective: obstime=2017-07-26T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (Tx, Ty) in arcsec
(0., 0.)>
>>> coord.transform_to(frames.HeliographicCarrington)
<SkyCoord (HeliographicCarrington: obstime=2017-07-26T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, AU)
(283.95956776, 5.31701821, 0.00465047)>
It is also possible to transform to any coordinate system implemented in Astropy. This can be used to find the position of the solar limb in AltAz equatorial coordinates:
>>> from astropy.coordinates import EarthLocation, AltAz
>>> time = '2017-07-11 15:00'
>>> greenbelt = EarthLocation(lat=39.0044*u.deg, lon=-76.8758*u.deg)
>>> greenbelt_frame = AltAz(obstime=time, location=greenbelt)
>>> west_limb = SkyCoord(900*u.arcsec, 0*u.arcsec, frame=frames.Helioprojective,
... observer=greenbelt.get_itrs(greenbelt_frame.obstime), obstime=time)
>>> west_limb.transform_to(greenbelt_frame)
<SkyCoord (AltAz: obstime=2017-07-11 15:00:00.000, location=(1126916.53031967, -4833386.58391627, 3992696.62211575) m, pressure=0.0 hPa, temperature=0.0 deg_C, relative_humidity=0.0, obswl=1.0 micron): (az, alt, distance) in (deg, deg, m)
(111.40782056, 57.1660434, 1.51859559e+11)>
Quick Reference
The following table is meant as a quick reference. For more complete code examples, see the how-to guides above.
How do I… |
Solution |
|---|---|
create a map from a FITS file |
|
save a map to a FITS file |
|
get a quicklook summary of a map |
|
plot a map |
|
access the underlying data array of a map |
|
access the map metadata |
|
make a copy of the map data array |
|
make a copy of the whole map |
|
access the observer location |
|
remove the roll angle |
|
plot the solar limb |
|
overlay a heliographic grid |
|
create a time series |
|
plot a time series |
|
concatenate two time series together |
|
convert a time series to a |
|
convert a time series to an |
|
parse the string representation of a timestamp |
|
calculate the Carrington rotation number at a given time |
|
calculate the time corresponding to a given Carrington rotation |
|
see all of the available solar constants |
Topic Guides#
These topic guides provide a set of in-depth explanations for various parts of the sunpy core package.
They assumes you have some knowledge of what sunpy is and how it works - if you’re starting fresh you might want to check out the Tutorial first.
How to use the topic guides
The topic guides are designed to be read in a standalone manner, without running code at the same time. Although there are code snippets in various parts of each topic guide, these are present to help explanation, and are not structured in a way that they can be run as you are reading a topic guide.
Coordinates#
The sunpy.coordinates sub-package contains:
A robust framework for working with solar-physics coordinate systems
Functions to obtain the locations of solar-system bodies (
sunpy.coordinates.ephemeris)Functions to calculate Sun-specific coordinate information (
sunpy.coordinates.sun)
The sunpy coordinate framework extends the Astropy coordinates framework.
This page contains an overview of how coordinates work in sunpy. See the following pages for detailed discussion of specific topics:
Calculating Carrington longitude#
Carrington coordinates are a heliographic coordinate system, where the longitude is determined assuming that the Sun has rotated at a sidereal period of ~25.38 days starting at zero longitude at a specific time on 1853 November 9. For a given observer at a given observation time, the center of the disk of the Sun as seen by that observer (also known as the apparent [1] sub-observer point) has a Carrington latitude and longitude. When the observer is at Earth, the apparent sub-Earth [2] Carrington latitude and longitude are also known as B0 and L0, respectively.
Quick summary#
sunpy calculates the sub-observer Carrington longitude in a manner that enables co-alignment of images of the Sun’s surface from different observatories at different locations/velocities in the solar system, but that results in a ~20 arcsecond discrepancy with The Astronomical Almanac.
Observer effects#
An observer will perceive the locations and orientations of solar-system objects different from how they truly are. There are two primary effects for why the apparent sub-observer point is not the same as the “true” sub-observer point.
Light travel time#
It takes time for light to travel from the Sun to any observer (e.g., ~500 seconds for an observer at Earth). Thus, the farther the observer is from the Sun, the farther back in time the Sun will appear to be. This effect is sometimes called “planetary aberration”.
As an additional detail, the observer is closer to the sub-observer point on the Sun’s surface than to the center of the Sun (i.e., by the radius of the Sun), which corresponds to ~2.3 lightseconds.
Observer motion#
The motion of an observer will shift the apparent location of objects, and this effect is called “stellar aberration”. A more subtle question is whether observer motion also shifts the apparent sub-observer Carrington longitude.
Stellar aberration causes the apparent ecliptic longitude of an observer to be different from its true ecliptic longitude. This shift in ecliptic longitude also means a shift in the heliographic longitude of the observer. For example, for an Earth observer, the apparent heliographic longitude of Earth is ~0.006 degrees (~20 arcseconds) less than its true heliographic longitude. For some purposes, this shift in the heliographic longitude of the observer should also shift the apparent sub-observer Carrington longitude.
However, shifting the sub-observer Carrington longitude due to observer motion is not always appropriate. For certain types of observations (e.g., imagery of features on the surface of the Sun), stellar aberration does not shift the positions of solar features relative to the center of the disk of the Sun; everything shifts in tandem. Thus, for the purposes of co-aligning images from different observatories, it is critical to exclude any correction for observer motion.
Background information#
The IAU definition#
Since the original definition of Carrington coordinates [3], there have been a number of refinements of the definition, but there have been inconsistencies with how those definitions are interpreted. Possible points of contention include the reference epoch, the rotation period, and the handling of observer effects.
The most recent definition was essentially laid down in 2007 by the IAU [4]:
The reference epoch is J2000.0 TDB (2000 January 1 12:00 TDB). The difference between Barycentric Dynamical Time (TDB) and Terrestrial Time (TT) is very small, but both are appreciably different from Coordinated Universal Time (UTC).
The longitude of the prime meridian (W0) at the reference epoch is 84.176 degrees. This longitude is the “true” longitude, without accounting for any effects that would modify the apparent prime meridian for an observer (e.g., light travel time).
The sidereal rotation rate is exactly 14.1844 degrees per day. This is close to, but not exactly the same as, a sidereal rotation period of 25.38 days or a mean synodic period of 27.2753 days.
The ICRS coordinates of the Sun’s north pole of rotation is given as a right ascension of 286.13 degrees and a declination of 63.87 degrees.
The tuning of W0#
Importantly, the IAU parameters (specifically, W0) were tuned following an investigation by The Astronomical Almanac [5]. The prescription of The Astronomical Almanac included both the correction for light travel time and the correction for stellar aberration due to Earth’s motion. When using this prescription, The Astronomical Almanac found that a W0 value of 84.176 degrees minimized the differences between the modern approach and earlier approaches. However, for those purposes where one should use sub-observer Carrington longitudes without the stellar-aberration correction, there will be a discrepancy compared to The Astronomical Almanac and to calculations made using earlier approaches.
The approach in sunpy#
sunpy determines apparent sub-observer Carrington longitude by including the correction for the difference in light travel time, but excluding any correction for stellar aberration due to observer motion. The exclusion of a stellar-aberration correction is appropriate for purposes such as image co-alignment, where Carrington longitudes must be consistently associated with features on the Sun’s surface.
Comparisons to other sources#
Compared to The Astronomical Almanac and older methods of calculation#
The Astronomical Almanac publishes the apparent sub-Earth Carrington longitude (L0), and these values include the stellar-aberration correction. Consequently, sunpy values will be consistently ~0.006 degrees (~20 arcseconds) greater than The Astronomical Almanac values, although these discrepancies are not always apparent at the printed precision (0.01 degrees).
Since The Astronomical Almanac specifically tuned the IAU parameters to minimize the discrepancies with older methods of calculation under their prescription that includes the stellar-aberration correction, sunpy values will also be ~20 arcseconds greater than values calculated using older methods. Be aware that older methods of calculation may not have accounted for the variable light travel time between the Sun and the Earth, which can cause additional discrepancies of up to ~5 arcseconds.
The Astronomical Almanac does not appear to account for the difference in light travel time between the sub-Earth point on the Sun’s surface and the center of the Sun (~2.3 lightseconds), which results in a fixed discrepancy of ~1.4 arcseconds in L0. However, this additional discrepancy is much smaller than the difference in treatment of stellar aberration.
Compared to SPICE#
In SPICE, the apparent sub-observer Carrington longitude can be calculated using the function subpnt.
subpnt allows for two types of aberration correction through the input argument abcorr:
“LT”, which is just the correction for light travel time
“LT+S”, which is the correction for light travel time (“LT”) plus the correction for observer motion (“S”)
sunpy calculates the apparent sub-observer Carrington longitude in a manner equivalent to specifying “LT” (as opposed to “LT+S”). The discrepancy between sunpy values and SPICE values is no greater than 0.01 arcseconds.
Compared to JPL Horizons#
In JPL Horizons, one can request the “Obs sub-long & sub-lat”. JPL Horizons appears to start from the IAU parameters and to include the correction for light travel time but not the correction for observer motion (i.e., equivalent to specifying “LT” to subpnt in SPICE). The discrepancy between sunpy values and JPL Horizons values is no greater than 0.01 arcseconds.
Compared to SunSPICE#
In SunSPICE, one can convert to and from the “Carrington” coordinate system using the function convert_sunspice_coord. However, these Carrington longitudes are “true” rather than “apparent” because the observer is not specified, so there are no corrections for light travel time or for observer motion. For example, for an Earth observer, sunpy values will be consistently greater than SunSPICE’s coordinate transformation by ~0.82 degrees.
Footnotes#
Differential rotation using coordinate frames#
Normally, coordinates refer to a point in inertial space (relative to the barycenter of the solar system).
Transforming to a different observation time does not move the point itself, but if the coordinate frame origin and/or axis change with time, the coordinate representation is updated to account for this change.
In solar physics an example of a frame that changes with time is the HeliographicStonyhurst (HGS) frame.
Its origin moves with the center of the Sun, and its orientation rotates such that the longitude component of Earth is zero at any given time.
A coordinate in a HGS frame of reference transformed to a HGS frame defined a day later will have a different longitude:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import HeliographicStonyhurst
>>> hgs_coord = SkyCoord(0*u.deg, 0*u.deg, radius=1*u.au, frame='heliographic_stonyhurst', obstime="2001-01-01")
>>> new_frame = HeliographicStonyhurst(obstime="2001-01-02")
>>> new_hgs_coord = hgs_coord.transform_to(new_frame)
>>> hgs_coord.lon, new_hgs_coord.lon
(<Longitude 0. deg>, <Longitude -1.01372559 deg>)
but when transformed to an inertial frame of reference we can see that these two coordinates refer to the same point in space:
>>> hgs_coord.transform_to('icrs')
<SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, AU)
(101.79107615, 26.05004621, 0.99601156)>
>>> new_hgs_coord.transform_to('icrs')
<SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, AU)
(101.79107615, 26.05004621, 0.99601156)>
To evolve a coordinate in time such that it accounts for the rotational motion of the Sun, one can use the RotatedSunFrame “metaframe” class as described below.
This machinery will take into account the latitude-dependent rotation rate of the solar surface, also known as differential rotation.
Multiple models for differential rotation are supported (see diff_rot() for details).
Note
RotatedSunFrame is a powerful metaframe, but can be tricky to use correctly.
We recommend users to first check if the simpler propagate_with_solar_surface() context manager is sufficient for their needs.
In addition, one may want to account for the translational motion of the Sun as well, and that can be achieved by also using the context manager transform_with_sun_center() for desired coordinate transformations.
Basics of the RotatedSunFrame class#
The RotatedSunFrame class allows one to specify coordinates in a coordinate frame prior to an amount of solar (differential) rotation being applied.
That is, the coordinate will point to a location in inertial space at some time, but will use a coordinate system at a different time to refer to that point, while accounting for the differential rotation between those two times.
RotatedSunFrame is not itself a coordinate frame, but is instead a “metaframe”.
A new frame class is created on the fly corresponding to each base coordinate frame class.
This tutorial will refer to these new classes as RotatedSun* frames.
Creating coordinates#
RotatedSunFrame requires two inputs: the base coordinate frame and the duration of solar rotation.
The base coordinate frame needs to be fully specified, which means a defined obstime and, if relevant, a defined observer.
Note that the RotatedSun* frame that is created in this example is appropriately named RotatedSunHeliographicStonyhurst:
>>> from sunpy.coordinates import RotatedSunFrame
>>> import sunpy.coordinates.frames as frames
>>> base_frame = frames.HeliographicStonyhurst(obstime="2001-01-01")
>>> rs_hgs = RotatedSunFrame(base=base_frame, duration=1*u.day)
>>> rs_hgs
<RotatedSunHeliographicStonyhurst Frame (base=<HeliographicStonyhurst Frame (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km)>, duration=1.0 d, rotation_model=howard)>
Once a RotatedSun* frame is created, it can be used in the same manner as other frames.
Here, we create a SkyCoord using the RotatedSun* frame:
>>> rotated_coord = SkyCoord(0*u.deg, 0*u.deg, frame=rs_hgs)
>>> rotated_coord
<SkyCoord (RotatedSunHeliographicStonyhurst: base=<HeliographicStonyhurst Frame (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km)>, duration=1.0 d, rotation_model=howard): (lon, lat) in deg
(0., 0.)>
Transforming this into the original heliographic Stonyhurst frame, we can see that the longitude is equal to the original zero degrees, plus an extra offset to account for one day of differential rotation:
>>> rotated_coord.transform_to(base_frame).lon
<Longitude 14.32632838 deg>
Instead of explicitly specifying the duration of solar rotation, one can use the keyword argument rotated_time.
The duration will be automatically calculated from the difference between rotated_time and the obstime value of the base coordinate frame.
Here, we also include coordinate data in the supplied base coordinate frame:
>>> rs_hgc = RotatedSunFrame(base=frames.HeliographicCarrington(10*u.deg, 20*u.deg, observer="earth",
... obstime="2020-03-04 00:00"),
... rotated_time="2020-03-06 12:00")
>>> rs_hgc
<RotatedSunHeliographicCarrington Coordinate (base=<HeliographicCarrington Frame (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=2.5 d, rotation_model=howard): (lon, lat) in deg
(10., 20.)>
A RotatedSun* frame containing coordinate data can be supplied to SkyCoord as normal:
>>> SkyCoord(rs_hgc)
<SkyCoord (RotatedSunHeliographicCarrington: base=<HeliographicCarrington Frame (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=2.5 d, rotation_model=howard): (lon, lat) in deg
(10., 20.)>
The above examples used the default differential-rotation model, but any of the models available through sunpy.physics.differential_rotation.diff_rot() are selectable.
For example, instead of the default (“howard”), one can specify “allen” using the keyword argument rotation_model.
Note the slight difference in the “real” longitude compared to the output above:
>>> allen = RotatedSunFrame(base=frames.HeliographicCarrington(10*u.deg, 20*u.deg, observer="earth",
... obstime="2020-03-04 00:00"),
... rotated_time="2020-03-06 12:00", rotation_model="allen")
>>> allen.transform_to(allen.base)
<HeliographicCarrington Coordinate (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
(45.22266666, 20., 695700.)>
Transforming coordinate arrays#
For another transformation example, we define a meridian with a Carrington longitude of 100 degrees, plus 1 day of differential rotation.
Again, the coordinates are already differentially rotated in inertial space; the RotatedSun* frame allows one to represent the coordinates in a frame prior to the differential rotation:
>>> meridian = RotatedSunFrame([100]*11*u.deg, range(-75, 90, 15)*u.deg,
... base=frames.HeliographicCarrington(observer="earth", obstime="2001-01-01"),
... duration=1*u.day)
>>> meridian
<RotatedSunHeliographicCarrington Coordinate (base=<HeliographicCarrington Frame (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=1.0 d, rotation_model=howard): (lon, lat) in deg
[(100., -75.), (100., -60.), (100., -45.), (100., -30.), (100., -15.),
(100., 0.), (100., 15.), (100., 30.), (100., 45.), (100., 60.),
(100., 75.)]>
An easy way to “see” the differential rotation is to transform the coordinates to the base coordinate frame. Note that the points closer to the equator (latitude of 0 degrees) have evolved farther in longitude than the points at high latitudes:
>>> meridian.transform_to(meridian.base)
<HeliographicCarrington Coordinate (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
[(110.755047, -75., 695700.), (111.706972, -60., 695700.),
(112.809044, -45., 695700.), (113.682163, -30., 695700.),
(114.17618 , -15., 695700.), (114.326328, 0., 695700.),
(114.17618 , 15., 695700.), (113.682163, 30., 695700.),
(112.809044, 45., 695700.), (111.706972, 60., 695700.),
(110.755047, 75., 695700.)]>
In the specific case of HeliographicCarrington, this frame rotates with the Sun, but in a non-differential manner.
The Carrington longitude approximately follows the rotation of the Sun.
One can transform to the coordinate frame of 1 day in the future to see the difference between Carrington rotation and differential rotation.
Note that equator rotates slightly faster than the Carrington rotation rate (its longitude is now greater than 100 degrees), but most latitudes rotate slower than the Carrington rotation rate:
>>> meridian.transform_to(frames.HeliographicCarrington(observer="earth", obstime="2001-01-02"))
<HeliographicCarrington Coordinate (obstime=2001-01-02T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
[( 96.71777552, -75.1035280, 695509.61226612),
( 97.60193088, -60.0954217, 695194.47689542),
( 98.68350999, -45.0808511, 694918.44538999),
( 99.54760854, -30.0611014, 694697.75301952),
(100.03737064, -15.0375281, 694544.31380180),
(100.18622957, -0.01157236, 694467.21969767),
(100.03737064, 15.0151761, 694471.58239044),
( 99.54760854, 30.0410725, 694557.27090716),
( 98.68350999, 45.0645144, 694719.82847332),
( 97.60193088, 60.0838908, 694951.31065278),
( 96.71777552, 75.0975847, 695238.51302901)]>
Be aware that transformations with a change in obstime will also contend with a translation of the center of the Sun.
Note that the radius component above is no longer precisely on the surface of the Sun.
For precise transformations of solar features, one should also use the context manager transform_with_sun_center() to account for the translational motion of the Sun.
Using the context manager, the radius component stays as the solar radius as desired:
>>> from sunpy.coordinates import transform_with_sun_center
>>> with transform_with_sun_center():
... print(meridian.transform_to(frames.HeliographicCarrington(observer="earth", obstime="2001-01-02")))
<HeliographicCarrington Coordinate (obstime=2001-01-02T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
[( 96.570646, -75., 695700.), ( 97.52257 , -60., 695700.),
( 98.624643, -45., 695700.), ( 99.497762, -30., 695700.),
( 99.991779, -15., 695700.), (100.141927, 0., 695700.),
( 99.991779, 15., 695700.), ( 99.497762, 30., 695700.),
( 98.624643, 45., 695700.), ( 97.52257 , 60., 695700.),
( 96.570646, 75., 695700.)]>
Transforming multiple durations of rotation#
Another common use case for differential rotation is to track a solar feature over a sequence of time steps.
Let’s track an active region that starts at Helioprojective coordinates (-123 arcsec, 456 arcsec), as seen from Earth, and we will look both backwards and forwards in time.
When duration is an array, the base coordinate will be automatically upgraded to an array if it is a scalar.
We specify a range of durations from -5 days to +5 days, stepping at 1-day increments:
>>> durations = range(-5, 6, 1)*u.day
>>> ar_start = frames.Helioprojective(-123*u.arcsec, 456*u.arcsec,
... obstime="2001-01-01", observer="earth")
>>> ar = RotatedSunFrame(base=ar_start, duration=durations)
>>> ar
<RotatedSunHelioprojective Coordinate (base=<Helioprojective Frame (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=[-5. -4. -3. -2. -1. 0. 1. 2. 3. 4. 5.] d, rotation_model=howard): (Tx, Ty) in arcsec
[(-123., 456.), (-123., 456.), (-123., 456.), (-123., 456.),
(-123., 456.), (-123., 456.), (-123., 456.), (-123., 456.),
(-123., 456.), (-123., 456.), (-123., 456.)]>
Let’s convert to the base coordinate frame to reveal the motion of the active region over time:
>>> ar.transform_to(ar.base)
<Helioprojective Coordinate (obstime=2001-01-01T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (Tx, Ty, distance) in (arcsec, arcsec, AU)
[(-865.549563, 418.102848, 0.982512),
(-794.67361 , 429.259359, 0.981549),
(-676.999492, 439.158483, 0.980695),
(-519.354795, 447.212391, 0.980001),
(-330.98304 , 452.940564, 0.979507),
(-123. , 456. , 0.979244),
( 92.27676 , 456.207078, 0.979226),
( 302.081349, 453.54936 , 0.979455),
( 493.984308, 448.186389, 0.979917),
( 656.653862, 440.439434, 0.980585),
( 780.541211, 430.770974, 0.981419)]>
Be aware that these coordinates are represented in the Helioprojective coordinates as seen from Earth at the base time.
Since the Earth moves in its orbit around the Sun, one may be more interested in representing these coordinates as they would been seen by an Earth observer at each time step.
Since the destination frame of the transformation will now have arrays for obstime and observer, one actually has to construct the initial coordinate with an array for obstime (and observer) due to a limitation in Astropy.
Note that the active region moves slightly slower across the disk of the Sun because the Earth orbits in the same direction as the Sun rotates, thus reducing the apparent rotation of the Sun:
>>> ar_start_array = frames.Helioprojective([-123]*len(durations)*u.arcsec,
... [456]*len(durations)*u.arcsec,
... obstime=["2001-01-01"]*len(durations), observer="earth")
>>> ar_array = RotatedSunFrame(base=ar_start_array, duration=durations)
>>> earth_hpc = frames.Helioprojective(obstime=ar_array.rotated_time, observer="earth")
>>> ar_array.transform_to(earth_hpc)
<Helioprojective Coordinate (obstime=['2000-12-27 00:00:00.000' '2000-12-28 00:00:00.000'
'2000-12-29 00:00:00.000' '2000-12-30 00:00:00.000'
'2000-12-31 00:00:00.000' '2001-01-01 00:00:00.000'
'2001-01-02 00:00:00.000' '2001-01-03 00:00:00.000'
'2001-01-04 00:00:00.000' '2001-01-05 00:00:00.000'
'2001-01-06 00:00:00.000'], rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (Tx, Ty, distance) in (arcsec, arcsec, AU)
[(-853.35712 , 420.401517, 0.982294),
(-771.20926 , 429.298481, 0.981392),
(-650.31062 , 437.85932 , 0.980601),
(-496.634378, 445.519914, 0.97996 ),
(-317.863549, 451.731964, 0.9795 ),
(-123. , 456. , 0.979244),
( 78.103714, 457.916782, 0.979203),
( 275.263157, 457.194475, 0.97938 ),
( 458.500759, 453.689226, 0.979764),
( 618.572111, 447.417202, 0.980336),
( 747.448484, 438.560811, 0.981067)]>
Transforming into RotatedSun frames#
So far, all of the examples show transformations with the RotatedSun* frame as the starting frame.
The RotatedSun* frame can also be the destination frame, which can be more intuitive in some situations and even necessary in some others (due to API limitations).
Let’s use a coordinate from earlier, which represents the coordinate in a “real” coordinate frame:
>>> coord = rs_hgc.transform_to(rs_hgc.base)
>>> coord
<HeliographicCarrington Coordinate (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
(45.13354448, 20., 695700.)>
If we create a RotatedSun* frame for a different base time, we can represent that same point using coordinates prior to differential rotation:
>>> rs_frame = RotatedSunFrame(base=frames.HeliographicCarrington(observer="earth",
... obstime=coord.obstime),
... rotated_time="2020-03-06 12:00")
>>> rs_frame
<RotatedSunHeliographicCarrington Frame (base=<HeliographicCarrington Frame (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=2.5 d, rotation_model=howard)>
>>> new_coord = coord.transform_to(rs_frame)
>>> new_coord
<RotatedSunHeliographicCarrington Coordinate (base=<HeliographicCarrington Frame (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>)>, duration=2.5 d, rotation_model=howard): (lon, lat, radius) in (deg, deg, km)
(10., 20., 695700.)>
There coordinates are stored in the RotatedSun* frame, but it can be useful to “pop off” this extra layer and retain only the coordinate representation in the base coordinate frame.
There is a convenience method called as_base() to do exactly that.
Be aware the resulting coordinate does not point to the same location in inertial space, despite the superficial similarity.
Essentially, the component values have been copied from one coordinate frame to a different coordinate frame, and thus this is not merely a transformation between coordinate frames:
>>> new_coord.as_base()
<HeliographicCarrington Coordinate (obstime=2020-03-04T00:00:00.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate for 'earth'>): (lon, lat, radius) in (deg, deg, km)
(10., 20., 695700.)>
Example uses of RotatedSunFrame#
Here are the examples in our gallery that use RotatedSunFrame:
Coordinates with velocity information#
Velocity information can be added to any coordinate [1]. When the coordinate is transformed to a different coordinate frame, the velocity vector will be transformed as appropriate. Be aware that the transformation framework does not take any velocity information into account when transforming the position vector.
Creating a SkyCoord with velocity#
Velocity information can be added as keyword arguments to SkyCoord.
For sunpy’s frames, the names of the velocities components are the names of the position components prepended by “d_”, e.g.,:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> import sunpy.coordinates
>>> sc = SkyCoord(lon=10*u.deg, lat=20*u.deg, distance=1*u.AU,
... d_lon=3*u.arcmin/u.week, d_lat=4*u.arcmin/u.d, d_distance=5*u.km/u.min,
... frame='heliocentricinertial', obstime='2021-01-01')
>>> sc
<SkyCoord (HeliocentricInertial: obstime=2021-01-01T00:00:00.000): (lon, lat, distance) in (deg, deg, AU)
(10., 20., 1.)
(d_lon, d_lat, d_distance) in (arcsec / s, arcsec / s, km / s)
(0.00029762, 0.00277778, 0.08333333)>
See Working with Velocities in Astropy Coordinates for ways to add velocity information to existing coordinates.
Querying velocity information#
sunpy has functions to query the positions of planets or other objects (e.g., get_body_heliographic_stonyhurst()).
For any of these functions, if include_velocity=True is specified, the returned coordinate will include velocity information, e.g.,:
>>> from sunpy.coordinates import get_body_heliographic_stonyhurst
>>> get_body_heliographic_stonyhurst('mercury', '2021-01-01', include_velocity=True)
<HeliographicStonyhurst Coordinate (obstime=2021-01-01T00:00:00.000, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, AU)
(-156.46460438, -1.38836399, 0.43234904)
(d_lon, d_lat, d_radius) in (arcsec / s, arcsec / s, km / s)
(0.09138133, 0.00720229, -7.2513617)>
Transforming velocities#
The transformation of the velocity vector between two coordinate frames takes into account two effects:
The change in the direction of the velocity vector due to a change in the orientation of the axes between the two frames.
The “induced” velocity due to the time dependence of the frames themselves.
Orientation change#
To illustrate the orientation change, let’s start with the SkyCoord created at the beginning, which was defined in the HeliocentricInertial frame.
We transform to Astropy’s HCRS frame, which is a different inertial [2] frame that is also centered at the Sun:
>>> sc_hcrs = sc.transform_to('hcrs')
>>> sc_hcrs
<SkyCoord (HCRS: obstime=2021-01-01T00:00:00.000): (ra, dec, distance) in (deg, deg, AU)
(80.95428245, 44.31500877, 1.)
(pm_ra_cosdec, pm_dec, radial_velocity) in (mas / yr, mas / yr, km / s)
(-8828397.4990187, 87659732.13232313, 0.08333338)>
Even though the velocity vectors are oriented very differently in their respective spherical coordinates, their amplitudes are essentially the same:
>>> sc.velocity.norm()
<Quantity 2.02654081 km / s>
>>> sc_hcrs.velocity.norm()
<Quantity 2.02654083 km / s>
Induced velocity#
To illustrate “induced” velocity, consider the HeliographicStonyhurst frame, which is defined such that the Earth is always at zero degrees longitude.
That is, this frame rotates around the Sun over time to “follow” the Earth.
Accordingly, the longitude component of Earth’s velocity vector will be negligible in this frame:
>>> from sunpy.coordinates import get_earth
>>> earth = get_earth('2021-01-01', include_velocity=True)
>>> earth
<SkyCoord (HeliographicStonyhurst: obstime=2021-01-01T00:00:00.000, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, AU)
(0., -3.02983361, 0.98326486)
(d_lon, d_lat, d_radius) in (arcsec / s, arcsec / s, km / s)
(1.82278759e-09, -0.00487486, -0.01720926)>
Transforming this coordinate to the HeliocentricInertial frame, which does not rotate over time, confirms that the Earth is moving in inertial space at the expected ~1 degree/day in heliographic longitude:
>>> earth.heliocentricinertial
<SkyCoord (HeliocentricInertial: obstime=2021-01-01T00:00:00.000): (lon, lat, distance) in (deg, deg, AU)
(24.55623543, -3.02983361, 0.98326486)
(d_lon, d_lat, d_distance) in (arcsec / s, arcsec / s, km / s)
(0.0422321, -0.00487486, -0.01720925)>
>>> earth.heliocentricinertial.d_lon.to('deg/d')
<Quantity 1.01357048 deg / d>
Transforming over time#
As the transformation framework is currently implemented, transforming between frames with different values of obstime takes into account any time dependency for the definitions of the frames, but does not incorporate any notion of the coordinate itself moving in inertial space.
This behavior does not change even if there is velocity information attached to the coordinate.
For example, if we take the same coordinate created earlier for Earth, and transform it to one day later:
>>> from sunpy.coordinates import HeliographicStonyhurst
>>> earth.transform_to(HeliographicStonyhurst(obstime=earth.obstime + 1*u.day))
<SkyCoord (HeliographicStonyhurst: obstime=2021-01-02T00:00:00.000, rsun=695700.0 km): (lon, lat, radius) in (deg, deg, AU)
(-1.01416251, -3.02979409, 0.98326928)
(d_lon, d_lat, d_radius) in (arcsec / s, arcsec / s, km / s)
(-1.19375277e-05, -0.00487485, -0.01743006)>
Note that the location of the Earth in the new frame is ~ -1 degree in longitude, as opposed to zero degrees. That is, this coordinate represents the location of Earth on 2021 January 1 using axes that are defined using the location of Earth on 2021 January 2.
Footnotes#
Coordinates and WCS#
The sunpy.coordinates sub-package provides a mapping between FITS-WCS CTYPE convention and the coordinate frames as defined in sunpy.coordinates.
This is used via the astropy.wcs.utils.wcs_to_celestial_frame function, with which the sunpy frames are registered upon being imported.
This list is used by for example, wcsaxes to convert from astropy.wcs.WCS objects to coordinate frames.
The sunpy.map.GenericMap class creates astropy.wcs.WCS objects as amap.wcs, however, it adds some extra attributes to the WCS object to be able to fully specify the coordinate frame.
It adds heliographic_observer and rsun.
If you want to obtain a un-realized coordinate frame corresponding to a GenericMap object you can do the following:
>>> import sunpy.map
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> amap = sunpy.map.Map(AIA_171_IMAGE)
>>> amap.observer_coordinate
<SkyCoord (HeliographicStonyhurst: obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>
which is equivalent to:
>>> from astropy.wcs.utils import wcs_to_celestial_frame
>>> wcs_to_celestial_frame(amap.wcs)
<Helioprojective Frame (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07T06:33:02.770, rsun=696000.0 km): (lon, lat, radius) in (deg, deg, m)
(-0.00406308, 0.04787238, 1.51846026e+11)>)>
The role of rsun in coordinate transformations#
In the case of HeliographicCarrington, one can specify observer='self' to indicate that the coordinate itself should be used as the observer for defining the coordinate frame.
It is possible to convert from a Helioprojective frame with one observer location to another Helioprojective frame with a different observer location.
The transformation requires the coordinate to be 3D, so if the initial coordinate is only 2D, the default assumption maps that 2D coordinate to the surface of the Sun, as defined by the rsun frame attribute.
The conversion can be performed as follows:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import frames
>>> import sunpy.coordinates
>>> hpc1 = SkyCoord(0*u.arcsec, 0*u.arcsec, observer="earth", obstime="2017-07-26", frame=frames.Helioprojective)
>>> hpc_out = sunpy.coordinates.Helioprojective(observer="venus", obstime="2017-07-26")
>>> hpc2 = hpc1.transform_to(hpc_out)
For example if you have two maps, named aia and stereo:
>>> hpc1 = SkyCoord(0*u.arcsec, 0*u.arcsec, frame=aia.coordinate_frame)
>>> hpc2 = hpc1.transform_to(stereo.coordinate_frame)
Design of the Coordinates Sub-Package#
This sub-package works by defining a collection of “Frames” (sunpy.coordinates.frames), which exists on a transformation graph, where the transformations between the coordinate frames are then defined and registered with the transformation graph (sunpy.coordinates).
It is also possible to transform SunPy frames to Astropy frames.
Positions within these “Frames” are stored as a “Representation” of a coordinate, a representation being a description of a point in a Cartesian, spherical or cylindrical system (see Using and Designing Coordinate Representations). A frame that contains a representation of one or many points is said to have been ‘realized’.
For a more in depth look at the design and concepts of the Astropy coordinates system see Overview of astropy.coordinates Concepts
Frames and SkyCoord#
The SkyCoord class is a high level wrapper around the astropy.coordinates sub-package.
It provides an easier way to create and transform coordinates, by using string representations for frames rather than the classes themselves and some other usability improvements.
For more information see the SkyCoord documentation.
The main advantage provided by SkyCoord is the support it provides for caching Frame attributes.
Frame attributes are extra data specified with a frame, some examples in sunpy.coordinates are obstime or observer for observer location.
Only the frames where this data is meaningful have these attributes, i.e., only the Helioprojective frames have observer.
However, when you transform into another frame and then back to a projective frame using SkyCoord it will remember the attributes previously provided, and repopulate the final frame with them.
If you were to do transformations using the Frames alone this would not happen.
Add a new rotation method to Map#
It is possible to select from a number of rotation methods when using sunpy.image.transform.affine_transform() and sunpy.map.GenericMap.rotate().
You can add a custom rotation method using the decorator sunpy.image.transform.add_rotation_function():
from sunpy.image.transform import add_rotation_function
@add_rotation_function("my_rotate", allowed_orders={0, 1, 3},
handles_clipping=False, handles_image_nans=False, handles_nan_missing=False)
def _rotation_mine(image, matrix, shift, order, missing, clip):
# Rotation code goes here
return rotated_image
See the docstring for add_rotation_function() for a detailed explanation of each of the decorator parameters and each of the required input parameters to the rotation function
Then you can do:
>>> from sunpy.map import Map
>>> from sunpy.data import sample
>>> hmi_map = sunpy.map.Map(sample.HMI_LOS_IMAGE)
>>> rot_map = hmi_map.rotate(order=3, recenter=True, method="my_rotate")
The available rotation methods are all implemented using the add_rotation_function() decorator, so you can look in sunpy/image/transform.py for examples of how to use this decorator.
Adding new data sources to Fido#
sunpy’s data search and retrieval tool (Fido) is designed to be extensible, so that new sources of data or metadata can be supported, either inside or outside the sunpy core package.
There are two ways of defining a new client, depending on the complexity of the web service.
A “scraper” client inherits from GenericClient which provides helper methods for downloading from a list of URLs.
If the service you want to add has easily accessible HTTP or FTP URLs that have a well defined folder and filename structure, this is probably the best approach.
If the service you want to add requires making requests to an API with parameters for the search and getting a list of results in return, then you probably want to write a “full” client.
Before writing a new client, ensure you are familiar with how searches are specified by the sunpy.net.attr system, including combining them with logical operations.
When choosing a name for your new client it should have the form <name>Client as sunpy will split the name of the class to extract the name of your client.
The main place this is done is when constructing a UnifiedResponse object, where the name part can be used to index the response object.
Writing a new “scraper” client#
A “scraper” Fido client (also sometimes referred to as a “data retriever” client) is a Fido client which uses the URL Scraper to find files on remote servers.
If the data provider you want to integrate does not provide a tree of files with predictable URLs then a “full” client is more likely to provide the functionality you need.
A new “scraper” client inherits from GenericClient and requires a minimum of these three components:
A class method
register_values(); this registers the “attrs” that are supported by the client. It returns a dictionary where keys are the supported attrs and values are lists of tuples. Eachtuplecontains the “attr” value and its description.A class attribute
baseurl; this is a regular expression which is used to match the URLs supported by the client.A class attribute
pattern; this is a template used to extract the metadata from URLs matched bybaseurl. The extraction uses theparseformat.
For a simple example of a scraper client, we can look at the implementation of sunpy.net.dataretriever.sources.eve.EVEClient in sunpy.
A version without documentation strings is reproduced below:
class EVEClient(GenericClient):
baseurl = (r'http://lasp.colorado.edu/eve/data_access/evewebdata/quicklook/'
r'L0CS/SpWx/%Y/%Y%m%d_EVE_L0CS_DIODES_1m.txt')
pattern = '{}/SpWx/{:4d}/{year:4d}{month:2d}{day:2d}_EVE_L{Level:1d}{}'
@classmethod
def register_values(cls):
from sunpy.net import attrs
adict = {attrs.Instrument: [('EVE', 'Extreme ultraviolet Variability Experiment, which is part of the NASA Solar Dynamics Observatory mission.')],
attrs.Physobs: [('irradiance', 'the flux of radiant energy per unit area.')],
attrs.Source: [('SDO', 'The Solar Dynamics Observatory.')],
attrs.Provider: [('LASP', 'The Laboratory for Atmospheric and Space Physics.')],
attrs.Level: [('0', 'EVE: The specific EVE client can only return Level 0C data. Any other number will use the VSO Client.')]}
return adict
This client scrapes all the URLs available under the base url http://lasp.colorado.edu/eve/data_access/evewebdata/quicklook/L0CS/SpWx/.
Scraper is primarily focused on URL parsing based on time ranges, so the rest of the baseurl pattern specifies where in the pattern the time information is located, using strptime notation.
The pattern attribute is used to populate the results table from the URLs matched with the baseurl.
It includes some of the time definitions, as well as names of attrs (in this case “Level”).
The supported time keys are: ‘year’, ‘month’, ‘day’, ‘hour’, ‘minute’, ‘second’, ‘millisecond’.
The attrs returned in the register_values() method are used to match your client to a search, as well as adding their values to the attr.
This means that after this client has been imported, running print(a.Provider) will show that the EVEClient has registered a provider value of LASP.
In addition to this, a sanitized, lower cased version of the value will be available for tab completing, e.g. a.Provider.lasp or a.Level.zero.
More Complex Clients#
Sometimes the attr values may not exist identically in the required URLs, and therefore can not be simply extracted with pattern.
Say, for example, the Wavelength of a file is expressed in the URL as a passband by name; in this case conversion of the Quantity object to the pass band name would be needed.
This is done addressed with the two following methods:
pre_search_hook()which will convert the passed attrs to their representation in the URL.post_search_hook()which converts the retrieved metadata from a URL to the form in which they are desired to be represented in the response table.
A good example of the use of these two methods is the sunpy.net.dataretriever.sources.norh.NoRHClient in sunpy.
It may also be possible that the baseurl property needs to be customized based on attrs other than Time.
Since Scraper doesn’t currently support generating directories that have non-time variables, the search() needs to be customized.
The search method should in this case, generate a baseurl dependent on the values of these attrs, and then call super().search or Scraper for each baseurl generated.
For an example of a complex modification of the search() method see the implementation of SUVIClient.search.
Customizing the Downloader#
There is no method for a client creator to override the parfive.Downloader that is used to fetch the files.
This is because all downloads made by a single call to Fido.fetch share one instance of parfive.Downloader.
However, it is possible to pass keywords parfive.Downloader.enqueue_file(), which is important if there is a need to customize the requests to a remote server, such as setting custom HTTP headers.
This is done by setting the enqueue_file_kwargs attribute of the client class.
One example from the sunpy.net.dataretriever.sources.noaa.SRSClient is:
class SRSClient(GenericClient):
...
# Server does not support the normal aioftp passive command.
enqueue_file_kwargs = {"passive_commands": ["pasv"]}
...
These keywords are passed to each call to parfive.Downloader.enqueue_file(), so they will affect all files that are added for download by your client.
Examples#
Suppose any file of a data archive can be described by this URL https://some-domain.com/%Y/%m/%d/satname_{SatelliteNumber}_{Level}_%y%m%d%H%M%S_{any-2-digit-number}.fits:
baseurl becomes r'https://some-domain.com/%Y/%m/%d/satname_(\d){2}_(\d){1}_(\d){12}_(\d){2}\.fits'.
Note all variables in the filename are converted to regex that will match any possible value for it.
A character enclosed within () followed by a number enclosed within {} is used to match the specified number of occurrences of that special sequence.
For example, %y%m%d%H%M%S is a twelve digit variable (with 2 digits for each item) and thus represented by r'(\d){12}'.
Note that \ is used to escape the special character ..
pattern becomes '{}/{year:4d}/{month:2d}{day:2d}/satname_{SatelliteNumber:2d}_{Level:1d}_{:6d}{hour:2d}{minute:2d}{second:2d}_{:2d}.fits'.
Note the sole purpose of pattern is to extract the information from matched URL, using parse.
So the desired key names for returned dictionary should be written in the pattern within {}, and they should match with the attr.__name__.
register_values() can be written as:
@classmethod
def register_values(cls):
from sunpy.net import attrs
adict = {
attrs.Instrument: [("SatName", "The description of Instrument")],
attrs.Physobs: [('some_physobs', 'Phsyobs description')],
attrs.Source: [('some_source', 'Source description')],
attrs.Provider: [('some_provider', 'Provider description')],
attrs.Level: [("1", "Level 1 data"), ("2", "Level 2 data")],
attrs.SatelliteNumber: [("16", "Describe it"), ("17", "Describe it")]
}
return adict
Writing a “full” client#
In this section we will describe how to build a “full” Fido client. You should write a new “full” client if the data you are accessing can not be accessed via a URL template, for instance if you hit a web API with a query to return results for a search.
A new Fido client contains three major components:
A subclass of
BaseClientwhich implementssearch,fetch, and_can_handle_query.Zero or more new
Attrclasses to specify search parameters unique to your data source.An instance of
AttrWalkerwhich can be used to walk the tree ofAttrinstances and convert them into a form useful to your client’s search method.
Search Attrs#
As described in attr the attr system allows the construction of complex queries by the user.
To make these complex queries easily processable by the clients the AttrWalker converts these into a set of queries which can be processed separately.
It does this by converting the input query to a set of queries which are ORed, but are complete queries.
This means the list of queries is an OR of ANDs (technically called disjunctive normal form).
Each query in the list of ORs contains all the information about that query so for example if the user provided a query like
a.Time("2020/02/02", "2020/02/03") & (a.Instrument("AIA") | a.Instrument("HMI"))
it would be passed to the client as
(a.Time("2020/02/02", "2020/02/03") & a.Instrument("HMI")) | (a.Time("2020/02/02", "2020/02/03") & a.Instrument("AIA"))
So you can process each element of the OR in turn without having to consult any other part of the query.
If the query the user provided contains an OR statement you get passed an instance of AttrOr and each sub-element of that AttrOr will be AttrAnd (or a single other attr class).
If the user query doesn’t contain an OR you get a single Attr instance or an AttrAnd.
For example you could get any of the following queries (using & for AND and | for OR):
(a.Instrument("AIA") & a.Time("2020/02/02", "2020/02/03")) | (a.Instrument("HMI") & a.Time("2020/02/02", "2020/02/03"))a.Time("2020/02/02", "2020/02/03")a.Instrument("AIA") & a.Time("2020/02/02", "2020/02/03")(a.Time(..) & a.Instrument("AIA") & a.Wavelength(30*u.nm, 31*u.nm)) | (a.Time(..) & a.Instrument("AIA") & a.Wavelength(30*u.nm, 31*u.nm))
but you would not be passed queries which look like the following examples, even if that’s how the user specified them:
a.Time("2020/02/02", "2020/02/03") & (a.Instrument("AIA") | a.Instrument("HMI"))a.Time(..) & (a.Instrument("AIA") | a.Instrument("AIA")) & a.Wavelength(30*u.nm, 31*u.nm))
The Attr Walker#
Given the potential complexity of these combined attrs, converting them into other forms, such as query parameters or JSON etc involves walking the tree and converting each attr to the expected format in a given way.
This parsing and conversion of the query tree is deliberately not done using methods or attributes of the attrs themselves.
The attrs should be independent of any client in their implementation, so they can be shared between the different Fido clients.
A class is provided to facilitate this conversion, AttrWalker.
The AttrWalker class consists of three main components:
Creators: The
createmethod is one of two generic functions for which a different function is called for each Attr type. The intended use for creators is to return a new object dependent on different attrs. It is commonly used to dispatch onAttrAndandAttrOr.Appliers: The
applymethod is the same ascreatein that it is a generic function. The only difference between it andcreateis its intended use. Appliers are generally used to modify an object returned by a creator with the values or information contained in other Attrs.Converters: Adding a converter to the walker adds the function to both the creator and the applier. For the VSO client this is used to convert each supported attr into a
ValueAttrwhich is then later processed by the appliers and creators. This pattern can be useful if you would otherwise have to repeat a lot of logic in each of the applier functions for each type of Attr you support.
An Example of AttrWalker#
In this example we will write a parser for some simple queries which uses AttrWalker to convert the query to a dict of URL query parameters for a HTTP GET request.
Let’s imagine we have a web service which you can do a HTTP GET request to https://sfsi.sunpy.org/search for some imaginary data from an instrument called SFSI (Sunpy Fake Solar Instrument).
This GET request takes three query parameters startTime, endTime and level, so a request might look something like: https://sfsi.sunpy.org/search?startTime=2020-01-02T00:00:00&endTime=2020-01-02T00:00:00&level=1.
Which would search for level one data between 2020-01-01 and 2020-01-02.
As attrs has Time and Level we do not need to define any of our own attrs for this client.
We do however want to write our own walker to convert them to the form out client’s search() method wants to send them to the server.
The first step is to setup the walker and define a creator method which will return a list of dicts, one for each independent search.
import sunpy.net.attrs as a
from sunpy.net.attr import AttrWalker, AttrAnd, AttrOr, DataAttr
walker = AttrWalker()
@walker.add_creator(AttrOr)
def create_or(wlk, tree):
results = []
for sub in tree.attrs:
results.append(wlk.create(sub))
return results
@walker.add_creator(AttrAnd, DataAttr)
def create_and(wlk, tree):
result = dict()
wlk.apply(tree, result)
return [result]
The call wlk.apply(...) inside the creator will walk any nested attrs and add their values to the dictionary as defined by the applier registered to each attr type.
If we want our client to support searching by a.Time and a.Level as in the URL example above, we would need to register an applier for each of these attrs.
@walker.add_applier(a.Time)
def _(wlk, attr, params):
return params.update({'startTime': attr.start.isot,
'endTime': attr.end.isot})
@walker.add_applier(a.Level)
def _(wlk, attr, params):
return params.update({'level': attr.value})
This combination of creators and appliers would allow support of any combination of queries consisting of a.Time and a.Level.
Obviously, most clients would want to support more attrs than these two, and this could be done by adding more applier functions.
Adding “Attrs” to Registry#
Registering of “attrs” ensures discoverability of search attributes supported by the corresponding sunpy Client.
For adding them to the Registry, we need to define a class method register_values() that returns a dictionary of registered values.
This dictionary should have Attr classes as keys and a list of tuples corresponding to that key representing the possible values the key “attr” can take.
Each tuple comprises of two elements.
The first one is a value and the second element contains a brief description of that value.
An example of writing register_values() for GenericClient is provided above.
Please note that it can be defined in a similar way for full clients too.
An Example of register_values()#
@classmethod
def register_values(cls):
from sunpy.net import attrs
adict = {
attrs.Instrument: [("LASCO", "Large Angle and Spectrometric Coronagraph")],
attrs.Source: [('SOHO', 'Solar and Heliospheric Observatory')],
attrs.Provider: [('SDAC', 'Solar Data Analysis Center')],
attrs.Detector: [('C1', 'Coronograph 1'),
('C2', 'Coronograph 2'),
('C3', 'Coronograph 3')]
}
return adict
Registering custom attrs in the attrs namespace#
When you have custom attrs defined in a separate attrs module, you can add them to the namespace using the _attrs_module() class method.
The method returns a tuple of length 2, where the first element is the target module name under which you want to add the custom attrs to the main attrs namespace.
The second is the import path to the source module where the custom attrs are defined.
Note that the source module here need not be an internal sunpy module, it could very well be external.
An example for this can be seen as implemented in the JSOC client:
@classmethod
def _attrs_module(cls):
return 'jsoc', 'sunpy.net.jsoc.attrs'
This adds all attrs that exist within sunpy.net.jsoc.attrs, such as Keyword, to attrs.jsoc.
These can now be accessed via an import of the main attrs module, e.g., at a.jsoc.Keyword.
Writing a Search Method#
The search() method has the job of taking a set of user queries and returning an instance of QueryResponseTable containing the results.
The general flow of a search() method is:
Call your instance of an
AttrWalkerto convert the input into a form expected by your API.Make as many requests to your API as needed to fulfill the query. Generally one per element of the outer
sunpy.net.attrs.AttrOr.Process the response from your API into an instance of
QueryResponseTable.
To process the query with the AttrWalker, call the AttrWalker.create() method:
def search(self, query):
queries = walker.create(query)
Assuming the walker is the one we defined above, queries would be a list of dicts with the attrs processed into query parameters for the API URL.
Note
If you want your search method to be able to be called independently of Fido, then you should accept a variable number of positional arguments (*args) and they should have the AND operator applied to them.
This looks like:
def search(self, *args):
query = attr.and_(args)
queries = walker.create(query)
Once the walker has processed the query into a form designed to be passed to your API, your search() method then needs to iterate over these parameters, make the requests, and process the results into a table.
In the following example we pretend our client has a method _make_search(query_parameters) which takes the query parameters and makes a request to our API.
We also pretend that the response is a json object in the form of a Python dictionary, which we want to put into the table.
def search(self, query):
queries = walker.create(query)
results = []
for query_parameters in queries:
results.append(self._make_search(query_parameters))
return QueryResponseTable(results, client=self)
In reality, you probably want to post-process the results from your API before you put them in the table, they should be human readable first, with spaces and capitalization as appropriate.
Supporting file size estimates#
The base client has a method for automatically estimating the total size of files in a given query: total_size().
To enable to support for this, make sure the table returned by search has a column that contains filesizes as astropy quantities convertible to u.byte, and set the size_column class attribute to the name of this column.
The _can_handle_query method#
The next required method is _can_handle_query, this method tells Fido if your client might be able to return results for a given query.
If this method returns True, your clients search() method will be called for that query.
This method gets passed each query (in its independent form), and must either return True or False.
A simple example, which just checks the type of attrs and not their values would be
@classmethod
def _can_handle_query(cls, *query):
query_attrs = set(type(x) for x in query)
supported_attrs = {a.Time, a.Level}
return supported_attrs.issuperset(query_attrs)
Note, that this method is a class method, it gets called without instantiating your client to speed up the dispatching.
Writing a Fetch Method#
The fetch() method of a Fido client is responsible for converting a set of search results (possibly sliced by the user) into a set of URLs to be downloaded.
Due to the history of clients and how they were implemented in sunpy, some existing clients support use outside of the Fido wrapper, this makes them appear more complex.
In this example we are going to write a fetch() method which is designed only to be called from Fido.
The parameters for such a method should be:
def fetch(self, query_results, *, path, downloader, **kwargs):
...
The parameters here are:
query_resultswhich is an instance ofQueryResponseTableorQueryResponseRow, these are the results the user wants to download.path=This is the path that the user wants the file to be downloaded to, this can be a template string (i.e. expects to have.format()called on it).downloader=This is aparfive.Downloaderobject which should be mutated by thefetch()method.**kwargsIt is very important thatfetch()methods accept extra keyword arguments that they don’t use, as the user might be passing them to other clients viaFido.
Processing the query_results Argument#
The query_results argument can be of two types QueryResponseTable or QueryResponseRow, as the user can slice the results table down to a single row and then pass that to Fido.fetch().
If you do not wish to handle a single row any differently to a table, you can place the convert_row_to_table decorator on your fetch() method which will convert the argument to a length one table when it is a single row object.
The primary function of the fetch() method is for you to convert this results object into a set of URLs for Fido to download.
This logic will be specific to your client.
Formatting the path= Argument#
The path argument may contain format sections which are processed column names from the response table.
In addition to these it may contain the {file} format segment which is a placeholder for the filename.
Each row of the results table has a response_block_map property which is a dictionary of valid format keys to values for that row.
In addition to the response_block_map your fetch method also needs to be able to generate a filename for the file.
The simplest (but unlikely) scenario is that you know the filename for each file you are going to download before you do so, in this situation you would be able to generate the full filepath for each row of the response as follows
for row in query_results:
filename = self._calculate_filename(row)
filepath = path.format(file=filename, **row.response_block_map)
In the situation where you wish to be told the filename by the web server you are downloading the file from, it is a little more complex, you need to pass a callback function to parfive.Downloader.enqueue_file() which will calculate the full filename in the context of the download, where the headers can be inspected for the filename the web server provides.
The filename callback passed to parfive.Downloader.enqueue_file() accepts two arguments resp and url.
resp is an aiohttp.ClientResponse object which is returned when parfive requests the URL.
This response object allows us to inspect the headers of the response before the data is downloaded.
url is the URL that was requested to generate the resp response.
To combine the formatting of the row with the extraction of the filename from the headers it is common to use functools.partial to generate many functions with different fixed parameters.
In the following example we will define a function which takes 4 arguments which we will use to generate the filename for the row.
This function will be called by parfive with the resp and url arguments.
def make_filename(path, row, resp, url):
# Define a fallback filename based on the information in the search results
name = f"row['ID'].fits"
if resp:
cdheader = resp.headers.get("Content-Disposition", None)
if cdheader:
_, params = sunpy.util.net.parse_header(cdheader)
name = params.get('filename', "")
return path.format(file=name, **row.response_block_map)
To reduce this function down to the two arguments expected we pre-specify the first two of these with partial before passing the function to enqueue_file inside the fetch() method.
Our simple example above now becomes:
for row in query_results:
filepath = partial(make_filename, path, row)
Where the path variable is a pathlib.Path object provided as the path argument to fetch().
Adding URLs to be Downloaded#
For each file you wish for Fido to download (normally one per row of the query_results) you need to call the parfive.Downloader.enqueue_file() of the downloader argument.
Combining this with the simple example above it may look something like
for row in query_results:
filename = self._calculate_filename(row)
filepath = path.format(file=filename, **row.response_block_map)
url = self._calculate_url(row)
downloader.enqueue_file(url, filename=filepath)
If your filepath is a callback function, pass this to the filename= argument.
Your fetch method does not need to return anything, as long as enqueue_file is called for every file you want Fido to download.
Putting it all together#
An example client class may look something like
import sunpy.util.net
import sunpy.net.atrrs as a
from sunpy.net.attr import AttrWalker, AttrAnd, AttrOr, DataAttr
from sunpy.base_client import QueryResponseTable
walker = AttrWalker()
@walker.add_creator(AttrOr)
def create_or(wlk, tree):
results = []
for sub in tree.attrs:
results.append(wlk.create(sub))
return results
@walker.add_creator(AttrAnd, DataAttr)
def create_and(wlk, tree):
result = dict()
wlk.apply(tree, result)
return [result]
@walker.add_applier(a.Time)
def _(wlk, attr, params):
return params.update({'startTime': attr.start.isot,
'endTime': attr.end.isot})
@walker.add_applier(a.Level)
def _(wlk, attr, params):
return params.update({'level': attr.value})
class ExampleClient(BaseClient):
size_column = 'Filesize'
def search(self, query):
queries = walker.create(query)
results = []
for query_parameters in queries:
results.append(self._make_search(query_parameters))
return QueryResponseTable(results, client=self)
def _make_filename(path, row, resp, url):
# Define a fallback filename based on the information in the search results
name = f"row['ID'].fits"
if resp:
cdheader = resp.headers.get("Content-Disposition", None)
if cdheader:
_, params = sunpy.util.net.parse_header(cdheader)
name = params.get('filename', "")
return path.format(file=name, **row.response_block_map)
@convert_row_to_table
def fetch(self, query_results, *, path, downloader, **kwargs):
for row in query_results:
filepath = partial(self._make_filename, path, row)
url = f"https://sfsi.sunpy.org/download/{row['ID']}"
downloader.enqueue_file(url, filename=filepath)
@classmethod
def _can_handle_query(cls, *query):
query_attrs = set(type(x) for x in query)
supported_attrs = {a.Time, a.Level}
return supported_attrs.issuperset(query_attrs)
How “HISTORY” and “COMMENT” FITS Keys are Handled in sunpy#
In FITS files, there are often multiple entries for both the “HISTORY” and “COMMENT” keys.
For example, when applying a prep routine to an image, a “HISTORY” entry may be added to the FITS header for every step in the prep pipeline.
Because the metadata associated with each GenericMap acts like a dictionary, where each key must be unique, these repeated “HISTORY” and “COMMENT” keys cannot be represented as separate entries in MetaDict.
Thus, when a FITS file with multiple “HISTORY” keys is read into a Map object, all the values corresponding to “HISTORY” are joined together with newline characters \n and stored as a single “history” entry in MetaDict.
When writing the resulting Map to a FITS file, “history” is split along the \n characters and each entry is written to a separate “HISTORY” key in the resulting FITS header.
The same is true for “COMMENT” keys.
Advanced Installation#
Warning
This page has advanced instructions for installing sunpy.
If you are new to Python or sunpy, please read the installation instructions first.
The SunPy Project maintains a range of affiliated packages that leverage the wider ecosystem of scientific Python packages for solar physics.
This page focuses on how to install the sunpy core package, but these instructions should apply to most of the affiliated packages as well.
conda#
Full instructions for installing using conda are in Installation. This is the recommended way to install sunpy, because it creates a fresh Python environment that is independent from any other Python install or packages on your system, and allows the installation of non-python packages. The SunPy Project publishes many of it’s packages to the conda-forge channel. If you have an existing conda install, without the conda-forge channel added you can configure the conda-forge channel by following the instructions in the conda-forge documentation.
Updating a conda package#
You can update to the latest version of any package by running:
$ conda update <package_name>
pip#
This is for installing sunpy within a Python environment, where pip has been used to install all previous packages.
You will want to make sure you are using a Python virtual environment.
Once the environment active, to acquire a full sunpy installation:
$ pip install "sunpy[all]"
Warning
If you get a PermissionError this means that you do not have the required administrative access to install new packages to your Python installation.
Do not install sunpy or other Python packages using sudo.
This error implies you have an incorrectly configured virtual environment or it is not activated.
We strive to provide binary wheels for all of our packages.
If you are using a Python version or operating system that is missing a binary wheel,
pip will try to compile the package from source and this is likely to fail without a C compiler (e.g., gcc or clang).
Getting the compiler either from your system package manager or XCode (if you are using macOS) should address this.
If you have a reason to want a more minimal installation, you can install sunpy with no optional dependencies, however this means a lot of submodules will not import:
$ pip install "sunpy"
It is possible to select which “extra” dependencies you want to install, if you know you only need certain submodules:
$ pip install "sunpy[map,timeseries]"
The available options are: [asdf], [dask], [image], [jpeg2000], [map], [net], [timeseries], [visualization].
Updating a pip package#
You can update to the latest version of any package by running:
$ pip install --upgrade <package_name>
Logging system#
Overview#
The sunpy logging system is a subclass of AstropyLogger.
Its purpose is to provide users the ability to decide which log and warning messages to show, to capture them, and to send them to a file.
All messages provided by sunpy make use of this logging facility which is based on the Python logging module rather than print statements.
Messages can have one of several levels, in increasing order of importance:
DEBUG: Detailed information, typically of interest only when diagnosing problems.
INFO: A message conveying information about the current task, and confirming that things are working as expected.
WARNING: An indication that something unexpected happened, and that user action may be required.
ERROR: Indicates a more serious issue where something failed but the task is continuing.
CRITICAL: A serious error, indicating that the program itself may be unable to continue running.
By default, all messages except for DEBUG messages are displayed.
Configuring the logging system#
The default configuration for the logger is determined by the default sunpy configuration file.
To make permanent changes to the logger configuration see the [logger] section of the sunpy configuration file (sunpyrc).
If you’d like to control the logger configuration for your current session first import the logger:
>>> from sunpy import log
or by:
>>> import logging
>>> log = logging.getLogger('sunpy')
The threshold level for messages can be set with:
>>> log.setLevel('DEBUG')
This will display DEBUG and all messages with that level and above. If you’d like to see the fewest relevant messages you’d set the logging level to WARNING or above.
For other options such as whether to log to a file or what level of messages the log file should contain, see the the Sunpy configuration file (sunpyrc).
Context managers#
If you’d like to capture messages as they are generated you can do that with a context manager:
>>> with log.log_to_list() as log_list:
... # your code here
Once your code is executed, log_list will be a Python list containing all of the sunpy messages during execution.
This does not divert the messages from going to a file or to the screen.
It is also possible to send the messages to a custom file with:
>>> with log.log_to_file('myfile.log'):
... # your code here
which will save the messages to a local file called myfile.log.
Writing a new instrument Map class#
All instrument Map classes are subclasses of the generic GenericMap subclass.
GenericMap expects to be provided the data array and a header dictionary and will parse the header metadata to the best of its ability based on common standards.
The instrument subclass implements the instrument-specific code to parse the metadata, apply any necessary procedures on the data array, as well as defining other things such what color tables to use.
In practice, the instrument subclass is not directly accessed by users.
The Map factory is the primary interface for creating Map objects.
Any subclass of GenericMap which defines a method named is_datasource_for() will automatically be registered with the Map factory.
The is_datasource_for() method is used by the Map factory to check if a file should use a particular instrument Map class.
This function can run any test it needs to determine this.
For example, it might check the value of the “INSTRUMENT” key in the metadata dictionary.
The following example shows how this works and includes a sample doc string that is aligned with the documentation guidelines:
import sunpy.map
class NextGenerationTelescopeMap(sunpy.map.GenericMap):
"""
NextGenerationTelescope Map.
The Next Generation Telescope is a optical telescope on board the new space mission.
It operates in low Earth orbit with an altitude of 600 km and an inclination of 28.5 degrees.
It is designed to observe the mechanisms that are responsible for triggering the impulsive release of magnetic energy in the solar corona.
It observes in the following 3 different passbands, in visible light, wavelength A, wavelength B, wavelength C.
The focal plane consists of a detector with 4k x 4k pixels.
The plate scale is 0.1 arcsec per pixel.
The field of view is the whole Sun (1000 x 1000 arcsec).
It makes images in each passband every 1 second except for when it is in eclipse which occurs every approximately 80 minutes.
It began operating on 2100 November 1.
Notes
-----
Due to failure of the filter wheel, 2 of the different passbands are no longer functional.
References
----------
* Mission Paper
* Instrument Paper(s)
* Data Archive
* Mission documents
"""
def __init__(self, data, header, **kwargs):
# Will process the header according to FITS common standards
super().__init__(data, header, **kwargs)
# Any NextGenerationTelescope Instrument-specific manipulation.
# Any metadata changes should be done by overloading
# the corresponding attributes/methods.
# Used by the Map factory to determine if this subclass should be used
@classmethod
def is_datasource_for(cls, data, header, **kwargs):
"""
Determines if data, header corresponds to a NextGenerationTelescope image
"""
# Returns True only if this is data and header from NextGenerationTelescope
return header.get('instrume', '').startswith('NextGenerationTelescope')
A Detailed Look at the TimeSeries Metadata#
TimeSeries store metadata in a TimeSeriesMetaData object, this object is designed to be able to store multiple basic MetaDict (case-insensitive ordered dictionary) objects and able to identify the relevant metadata for a given cell in the data.
This enables a single TimeSeries to be created by combining/concatenating multiple TimeSeries source files together into one and to keep a reliable track of all the metadata relevant to each cell, column or row.
The metadata can be accessed by:
>>> import sunpy.timeseries
>>> import sunpy.data.sample
>>> my_timeseries = sunpy.timeseries.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES)
>>> meta = my_timeseries.meta
You can easily get an overview of the metadata, this will show you a basic representation of the metadata entries that are relevant to this TimeSeries.
>>> meta
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2011-06-06T23:59:59.961999 | xrsa | simple: True |
| to | xrsb | bitpix: 8 |
|2011-06-07T23:59:57.631999 | | naxis: 0 |
| | | extend: True |
| | | date: 26/06/2012 |
| | | numext: 3 |
| | | telescop: GOES 15 |
| | | instrume: X-ray Detector |
| | | object: Sun |
| | | origin: SDAC/GSFC |
| | | ... |
|-------------------------------------------------------------------------------------------------|
The data within a TimeSeriesMetaData object is stored as a list of tuples, each tuple representing the metadata from a source file or provided when creating the time series.
The tuple will contain a TimeRange telling us which rows the metadata applies to, a list of column name strings for which the metadata applies to and finally a MetaDict object for storing the key/value pairs of the metadata itself.
Each time a TimeSeries is concatenated to the original a new set of rows and/or columns will be added to the DataFrame and a new entry will be added into the metadata.
Note that entries are ordered chronologically based on start and generally it’s expected that no two TimeSeries will overlap on both columns and time range.
For example, it is not good practice for alternate row values in a single column to be relevant to different metadata entries as this would make it impossible to uniquely identify the metadata relevant to each cell.
If you want the string that is printed then you can use the to_string method.
This has the advantage of having optional keyword arguments that allows you to set the depth (number of rows for each entry) and width (total number of characters wide) to better fit your output.
For example:
>>> meta.to_string(depth=20, width=99)
"|-------------------------------------------------------------------------------------------------|\n|TimeRange | Columns | Meta |\n|-------------------------------------------------------------------------------------------------|\n|2011-06-06T23:59:59.961999 | xrsa | simple: True |\n| to | xrsb | bitpix: 8 |\n|2011-06-07T23:59:57.631999 | | naxis: 0 |\n| | | extend: True |\n| | | date: 26/06/2012 |\n| | | numext: 3 |\n| | | telescop: GOES 15 |\n| | | instrume: X-ray Detector |\n| | | object: Sun |\n| | | origin: SDAC/GSFC |\n| | | date-obs: 07/06/2011 |\n| | | time-obs: 00:00:00.000 |\n| | | date-end: 07/06/2011 |\n| | | time-end: 23:59:57.632 |\n| | | comment: Energy band information given in extensio|\n| | | history: |\n| | | keycomments: {'SIMPLE': 'Written by IDL: Tue Jun |\n|-------------------------------------------------------------------------------------------------|\n"
Similar to the TimeSeries, the metadata has some properties for convenient access to the global metadata details, including columns as a list of strings, and time_range of the data.
Beyond this, there are properties to get lists of details for all the entries in the TimeSeriesMetaData object, including timeranges, columns (as a list of string column names) and metas.
Similar to TimeSeries objects you can concatenate TimeSeriesMetaData objects, but generally you won’t need to do this as it is done automatically when actioned on the TimeSeries.
Note that when truncating a TimeSeriesMetaData object you will remove any entries outside of the given TimeRange.
You can also append a new entry (as a tuple or list), which will add the entry in the correct chronological position.
It is frequently necessary to locate the metadata for a given column, row or cell which can be uniquely identified by both, to do this you can use the find method, by adding “colname” and/or time/row keyword arguments you get a TimeSeriesMetaData object returned which contains only the relevant entries.
You can then use the metas property to get a list of just the relevant MetaDict objects.
For example:
>>> tsmd_return = my_timeseries.meta.find(colname='xrsa', time='2012-06-01 00:00:33.904999')
>>> tsmd_return.metas
[]
Note, the colname and time keywords are optional, but omitting both just returns the original.
A common use case for the metadata is to find out the instrument(s) that gathered the data and in this case you can use the get method.
This method takes a single key string or list of key strings with the optional filters and will search for any matching values.
This method returns another TimeSeriesMetaData object, but removes all unwanted key/value pairs.
The result can be converted into a simple list of strings using the values method:
>>> tsmd_return = my_timeseries.meta.get('telescop', colname='xrsa')
>>> tsmd_return.values()
['GOES 15']
Note values removes duplicate strings and sorts the returned list.
You can update the values for these entries efficiently using the update method which takes a dictionary argument and updates the values to each of the dictionaries that match the given colname and time filters, for example:
>>> my_timeseries.meta.update({'telescop': 'G15'}, colname='xrsa', overwrite=True)
Here we have to specify the overwrite=True keyword parameter to allow us to overwrite values for keys already present in the MetaDict objects, this helps protect the integrity of the original metadata and without this set (or with it set to False) you can still add new key/value pairs.
Note that the MetaDict objects are both case-insensitive for key strings and have ordered entries, where possible the order is preserved when updating values.
The role of “rsun” in sunpy#
Unlike a solid ball, the Sun is not defined by a single surface, and as such, the position from which a photon is emitted should best be defined not just with a 2 dimensional coordinate but with the inclusion of a height. The Solar radius is a common choice for this height, and is defined by the IAU as the distance from the center of the Sun to a layer in the photosphere where the optical depth is 2/3. However, different wavelengths of light can be emitted from different heights in the solar atmosphere, and so the radius at which a photon is emitted is not always the same as the solar radius. It is therefore a useful convention to define “rsun” attribute for a map, which is the radius at which the emission is presumed to originate. This has limitations: data will invariably consist of a range of wavelengths of light and even a single wavelength, especially for coronal emission, comes from a variety of heights The value of “rsun” is intended as an estimate for this height.
Data providers often include this information for end users via the fits keyword: “RSUN_REF”, where this keyword is not available, sunpy assumes the standard value of the Solar radius, as defined by sunpy.sun.constants.
Whether “rsun” is set by the data provider or not, we emphasize that this is little more than an approximation.
The exact value used for “rsun” has consequences for working in sunpy with coordinates and transforms, it provides 3 dimensional information about the location of a map pixel.
For example, when loading a fits file with sunpy.map.Map a map.wcs is constructed with a value for “rsun” collected for use in coordinate transformations.
When manipulating these coordinates, idiosyncrasies and unexpected behavior might be encountered, these can be preempted by an understanding of behaviors surrounding “rsun”.
Transforming between Helioprojective frames#
It is possible to convert from a Helioprojective frame with one observer location to another Helioprojective frame with a different observer location.
The transformation requires the coordinate to be 3D, so if the initial coordinate is only 2D, the default assumption maps that 2D coordinate to the surface of the Sun, at a radius defined by the “rsun” frame attribute.
The conversion can be performed as follows:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> import sunpy.coordinates
>>> from sunpy.coordinates import frames
>>> hpc1 = SkyCoord(0*u.arcsec, 0*u.arcsec, observer="earth", obstime="2017-07-26", frame=frames.Helioprojective)
>>> hpc_out = sunpy.coordinates.Helioprojective(observer="venus", obstime="2017-07-26")
>>> hpc2 = hpc1.transform_to(hpc_out)
An example with two maps, named aia and stereo:
>>> hpc1 = SkyCoord(0*u.arcsec, 0*u.arcsec, frame=aia.coordinate_frame)
>>> hpc2 = hpc1.transform_to(stereo.coordinate_frame)
Reprojecting between frames with different “rsun”#
When the values of “rsun” from two wcs instances are different, issues with reprojecting between those frames can be encountered.
reproject_to() by default, enforces a round-trip behavior, the idea being that you should only trust the reprojection if the 2D coordinates from each observer both resolve to the same 3D point (within a pixel volume).
When the values for “rsun” are different, that criterion fails towards the limb.
In other cases, this furthers results in banding as the criterion fails, then succeeds and then fails again.
Changing the value of “rsun” can fix this issue. Take the example Reprojecting Images to Different Observers. If we run the reprojection twice, once before fixing the discrepancy in the “rsun_ref” metadata and once after:
from matplotlib import pyplot as plt
from sunpy.map import Map
from sunpy.data.sample import AIA_193_JUN2012, STEREO_A_195_JUN2012
plt.rcParams['figure.figsize'] = (16, 8)
map_aia = Map(AIA_193_JUN2012)
map_euvi = Map(STEREO_A_195_JUN2012)
outmap1 = map_euvi.reproject_to(map_aia.wcs)
map_euvi.meta['rsun_ref'] = map_aia.meta['rsun_ref']
outmap2 = map_euvi.reproject_to(map_aia.wcs)
fig = plt.figure()
ax1 = fig.add_subplot(121, projection=outmap1); outmap1.plot(axes=ax1); plt.title('Without rsun Fix')
ax2 = fig.add_subplot(122, projection=outmap2); outmap2.plot(axes=ax2); plt.title('With rsun Fix')
plt.show()
(Source code, png, hires.png, pdf)
We can see the difference in the appearance of the reprojected maps near the limb.
Reference#
sunpy (sunpy)#
sunpy Package#
sunpy#
An open-source Python library for Solar Physics data analysis.
Homepage: https://sunpy.org
Documentation: https://docs.sunpy.org/en/stable/
Functions#
|
|
Prints ones' system info in an "attractive" fashion. |
|
Print current configuration options. |
Coordinates (sunpy.coordinates)#
This sub-package contains:
A robust framework for working with solar-physics coordinate systems
Functions to obtain the locations of solar-system bodies (
sunpy.coordinates.ephemeris)Functions to calculate Sun-specific coordinate information (
sunpy.coordinates.sun)Bridge module to enable the use of the
SkyCoordAPI to perform computations using SPICE kernels (sunpy.coordinates.spice)
The SunPy coordinate framework extends the Astropy coordinates framework. The coordinates guide provides in depth discussion of the structure and concepts underlying the coordinates framework.
Supported Coordinate Systems#
Coordinate system |
Abbreviation |
SunPy/Astropy equivalent |
Notes |
|---|---|---|---|
Heliocentric Aries Ecliptic (Mean) |
HAE (also HEC) |
Astropy’s |
|
Heliocentric Cartesian |
HCC |
||
Heliocentric Earth Ecliptic |
HEE |
||
Heliocentric Earth Equatorial |
HEEQ (also HEQ) |
Use a Cartesian representation |
|
Heliocentric Inertial |
HCI |
||
Heliocentric Radial |
HCR |
similar to |
Use a cylindrical representation, but with a 90-degree offset in |
Heliocentric/Heliographic Radial-Tangential-Normal |
HGRTN |
similar to |
The axes are permuted, with HCC X, Y, Z equivalent respectively to HGRTN Y, Z, X |
Heliographic Carrington |
HGC |
||
Heliographic Stonyhurst |
HGS |
||
Helioprojective Cartesian |
HPC |
||
Geocentric Earth Equatorial (Mean) |
GEI |
||
Geographic |
GEO |
Astropy’s |
The precise geographic definitions may differ |
Geocentric Solar Ecliptic |
GSE |
||
Geomagnetic |
MAG |
||
Solar Magnetic |
SM |
||
GeocentricSolarMagnetospheric |
GSM |
For a description of these coordinate systems, see Thompson (2006) and Franz & Harper (2002) (and corrected version).
Reference/API#
sunpy.coordinates Package#
This subpackage contains:
A robust framework for working with coordinate systems
Functions to obtain the locations of solar-system bodies (
sunpy.coordinates.ephemeris)Functions to calculate Sun-specific coordinate information (
sunpy.coordinates.sun)
The diagram below shows all of Sun-based and Earth-based coordinate systems
available through sunpy.coordinates, as well as the transformations between
them. Each frame is labeled with the name of its class and its alias (useful
for converting other coordinates to them using attribute-style access).
The frames colored in cyan are implemented in astropy.coordinates, and there
are other astronomical frames that can be transformed to that are not shown
below (see astropy.coordinates.builtin_frames).
-
AffineTransform: ➝
-
FunctionTransform: ➝
-
FunctionTransformWithFiniteDifference: ➝
-
StaticMatrixTransform: ➝
-
DynamicMatrixTransform: ➝
Built-in Frame Classes#
A coordinate or frame in the ICRS system. |
|
A coordinate or frame in the Altitude-Azimuth system (Horizontal coordinates) with respect to the WGS84 ellipsoid. |
|
A coordinate or frame in the Geocentric Celestial Reference System (GCRS). |
|
A coordinate or frame in the Celestial Intermediate Reference System (CIRS). |
|
A coordinate or frame in the International Terrestrial Reference System (ITRS). |
|
A coordinate or frame in a Heliocentric system, with axes aligned to ICRS. |
|
A coordinate frame defined in a similar manner as GCRS, but precessed to a requested (mean) equinox. |
|
Geocentric mean ecliptic coordinates. |
|
Heliocentric mean ecliptic coordinates. |
|
Geocentric true ecliptic coordinates. |
|
Heliocentric true ecliptic coordinates. |
Functions#
|
Return a |
|
Return a |
|
Queries JPL HORIZONS and returns a |
|
Context manager for coordinate transformations to automatically apply solar differential rotation for any change in observation time. |
|
For a given frame, this function returns the corresponding WCS object. |
This function registers the coordinates frames to their FITS-WCS coordinate type values in the |
|
Context manager for coordinate transformations to ignore the motion of the center of the Sun. |
Classes#
|
Base class for HeliographicCarrington (HGC) and HeliographicStonyhurst (HGS) frames. |
|
Base class for frames that rely on the Earth's magnetic model (MAG, SM, and GSM). |
|
A coordinate or frame in the Geocentric Earth Equatorial (GEI) system. |
|
A coordinate or frame in the Geocentric Solar Ecliptic (GSE) system. |
|
A coordinate or frame in the GeocentricSolarMagnetospheric (GSM) system. |
|
A coordinate or frame in the Geomagnetic (MAG) system. |
|
A coordinate or frame in the Heliocentric system, which is observer-based. |
|
A coordinate or frame in the Heliocentric Earth Ecliptic (HEE) system. |
|
A coordinate or frame in the Heliocentric Inertial (HCI) system. |
|
A coordinate or frame in the Carrington Heliographic (HGC) system. |
|
A coordinate or frame in the Stonyhurst Heliographic (HGS) system. |
|
A coordinate or frame in the Helioprojective Cartesian (HPC) system, which is observer-based. |
|
A frame which is offset from another frame such that it shares the same origin, but has its "north pole" (i.e., the Z axis) in a different direction. |
|
A frame that applies solar rotation to a base coordinate frame. |
|
A coordinate or frame in the Solar Magnetic (SM) system. |
|
Base class for sunpy coordinate frames. |
Class Inheritance Diagram#
sunpy.coordinates.ephemeris Module#
Ephemeris calculations using SunPy coordinate frames
Functions#
|
Return a |
|
Return a |
|
Queries JPL HORIZONS and returns a |
sunpy.coordinates.spice Module#
Bridge module to use the SkyCoord API for SPICE computations.
Note
This module requires the optional dependency spiceypy to be
installed.
The SPICE observation geometry information
system is being increasingly used by space missions to describe the locations of
spacecraft and the time-varying orientations of reference frames.
While the spiceypy package provides a Python interface for
performing SPICE computations, its API is very different from that of
SkyCoord.
This module “wraps” spiceypy functionality so that relevant SPICE
computations can be accessed using the SkyCoord API.
When loading a set of kernels, a frame class and corresponding transformations
are created for each SPICE frame. One can also query the location of a body
as computed via SPICE or retrieve the field of view (FOV) of an instrument.
To facilitate the conversion of a SPICE-based coordinate to the built-in frames
in sunpy.coordinates, every SPICE-based coordinate has the method
to_helioprojective().
This method returns a coordinate in the Helioprojective
frame with the observer frame attribute appropriately set.
Be aware that when converting a SPICE-based coordinate to/from a built-in frame, there can be small inconsistencies due to differing planetary ephemerides and models for various orientations.
Notes
2D coordinates can be transformed only if the to/from frames have the same SPICE body ID as their centers.
Transformations of velocities are not yet supported.
SPICE frame names are rendered as uppercase, except for plus/minus characters, which are replaced with lowercase
'p'/'n'characters.
Examples
Functions#
|
Get the location of a body via SPICE. |
|
Get the field of view (FOV) for an instrument via SPICE. |
|
Load one more more SPICE kernels and create corresponding frame classes. |
|
Install a specified SPICE frame. |
Classes#
|
Base class for all frames generated to represent SPICE frames. |
Class Inheritance Diagram#
sunpy.coordinates.sun Module#
Sun-specific coordinate calculations
Functions#
|
Return the angular radius of the Sun as viewed from Earth. |
|
Returns the apparent position of the Sun (right ascension and declination) on the celestial sphere using the equatorial coordinate system, referred to the true equinox of date (as default). |
Return the Carrington rotation number. |
|
|
Return the time of a given Carrington rotation. |
|
Returns the Sun's true geometric longitude, referred to the mean equinox of date. |
|
Returns the Sun's apparent longitude, referred to the true equinox of date. |
|
Returns the Sun's true geometric latitude, referred to the mean equinox of date. |
|
Returns the Sun's apparent latitude, referred to the true equinox of date. |
Returns the mean obliquity of the ecliptic, using the IAU 2006 definition. |
|
|
Returns the Sun's true geometric right ascension relative to Earth, referred to the mean equinox of date (as default). |
|
Returns the Sun's true geometric declination relative to Earth, referred to the mean equinox of date (as default). |
Returns the true obliquity of the ecliptic, using the IAU 2006 definition. |
|
|
Returns the Sun's apparent right ascension relative to Earth, referred to the true equinox of date (as default). |
|
Returns the Sun's apparent declination relative to Earth, referred to the true equinox of date (as default). |
|
Print out a summary of solar ephemeris. |
|
Return the B0 angle for the Sun at a specified time, which is the heliographic latitude of the of the center of the disk of the Sun as seen from Earth. |
|
Return the L0 angle for the Sun at a specified time, which is the apparent Carrington longitude of the Sun-disk center as seen from Earth. |
|
Return the position (P) angle for the Sun at a specified time, which is the angle between geocentric north and solar north as seen from Earth, measured eastward from geocentric north. |
|
Return the distance between the Sun and the Earth at a specified time. |
|
Return the orientation angle for the Sun from a specified Earth location and time. |
|
Return the percentage of the Sun that is eclipsed by the Moon. |
sunpy.coordinates.utils Module#
Miscellaneous utilities related to coordinates
Functions#
|
Specify a rectangular region of interest in longitude and latitude in a given coordinate frame. |
|
Return the equivalency to convert between a physical distance on the Sun and an angular separation as seen by a specified observer. |
|
Get coordinates for the solar limb as viewed by a specified observer. |
Classes#
|
Calculate the properties of a great arc at user-specified points between a start and end point on a sphere. |
Reference/API for supporting coordinates modules#
The parts of the following modules that are useful to a typical user are already imported into the sunpy.coordinates namespace.
sunpy.coordinates.frames Module#
Common solar physics coordinate systems.
This submodule implements various solar physics coordinate frames for use with
the astropy.coordinates module.
Classes#
|
Base class for sunpy coordinate frames. |
|
Base class for HeliographicCarrington (HGC) and HeliographicStonyhurst (HGS) frames. |
|
Base class for frames that rely on the Earth's magnetic model (MAG, SM, and GSM). |
|
A coordinate or frame in the Stonyhurst Heliographic (HGS) system. |
|
A coordinate or frame in the Carrington Heliographic (HGC) system. |
|
A coordinate or frame in the Heliocentric system, which is observer-based. |
|
A coordinate or frame in the Helioprojective Cartesian (HPC) system, which is observer-based. |
|
A coordinate or frame in the Heliocentric Earth Ecliptic (HEE) system. |
|
A coordinate or frame in the Geocentric Solar Ecliptic (GSE) system. |
|
A coordinate or frame in the Heliocentric Inertial (HCI) system. |
|
A coordinate or frame in the Geocentric Earth Equatorial (GEI) system. |
|
A coordinate or frame in the Geomagnetic (MAG) system. |
|
A coordinate or frame in the Solar Magnetic (SM) system. |
|
A coordinate or frame in the GeocentricSolarMagnetospheric (GSM) system. |
Class Inheritance Diagram#
sunpy.coordinates.metaframes Module#
Coordinate frames that are defined relative to other frames
Classes#
|
A frame which is offset from another frame such that it shares the same origin, but has its "north pole" (i.e., the Z axis) in a different direction. |
|
A frame that applies solar rotation to a base coordinate frame. |
Class Inheritance Diagram#
sunpy.coordinates.wcs_utils Module#
Functions#
This function registers the coordinates frames to their FITS-WCS coordinate type values in the |
|
|
For a given frame, this function returns the corresponding WCS object. |
Attribution#
Some of this documentation was adapted from Astropy under the terms of the BSD License.
This sub-package was initially developed by Pritish Chakraborty as part of GSOC 2014 and Stuart Mumford.
Data (sunpy.data)#
sunpy.data contains ways to access sample data and small test files for
running the sunpy test suite.
sunpy.data Package#
Variables#
This class provides a remote data manager for managing remote files. |
|
Cache provides a way to download and cache files. |
sunpy.data.sample Module#
This module provides the following sample data files. When a sample shortname
is accessed, the corresponding file is downloaded if needed. All files can be
downloaded by calling download_all().
Summary variables#
|
Dictionary of all sample shortnames and, if downloaded, corresponding
file locations on disk (otherwise, |
|
List of disk locations for sample data files that have been downloaded |
Sample shortnames#
Sample shortname |
Name of downloaded file |
|---|---|
|
AIA20110607_063305_0094_lowres.fits |
|
AIA20110607_063301_0131_lowres.fits |
|
AIA20110607_063305_1600_lowres.fits |
|
aia_lev1_1600a_2012_06_06t04_07_29_12z_image_lev1_lowres.fits |
|
AIA20110607_063302_0171_lowres.fits |
|
aiacalibim5.fits |
|
AIA20110607_063307_0193_cutout.fits |
|
AIA20110607_063931_0193_cutout.fits |
|
AIA20110607_064555_0193_cutout.fits |
|
AIA20110607_065219_0193_cutout.fits |
|
AIA20110607_065843_0193_cutout.fits |
|
AIA20110607_063307_0193_lowres.fits |
|
AIA20120601_000007_0193_lowres.fits |
|
AIA20110607_063302_0211_lowres.fits |
|
AIA20110607_063334_0304_lowres.fits |
|
AIA20110607_063303_0335_lowres.fits |
|
aia_lev1_171a_2023_07_06t00_05_33_35z_image_lev1.fits |
|
BIR_20110607_062400_10.fit |
|
eit_l1_20110607_203753.fits |
|
20230706_000525_n4eua.fts |
|
20110607_EVE_L0CS_DIODES_1m.txt |
|
glg_cspec_n5_110607_v00.pha |
|
go1520110607.fits |
|
HMI20110607_063211_los_lowres.fits |
|
LOFAR_70MHZ_20190409_131136.fits |
|
lyra_20110607-000000_lev3_std.fits |
|
tca110607.fits |
|
hsi_image_20110607_063300.fits |
|
hsi_obssumm_20110607_025.fits |
|
20110607SRS.txt |
|
20120601_000530_n4eua.fits |
|
20120601_000530_n4eub.fits |
|
swap_lv1_20110607_063329.fits |
Functions#
|
Download all sample data at once that has not already been downloaded. |
sunpy.data.test Package#
This package contains all of SunPy’s test data.
Functions#
|
Return the full path to a test file in the |
Return a list of all test files in |
|
|
Generate a dummy |
|
Convert a FITS file containing an image and a header to a plaintext file containing only the string representation of the header. |
sunpy.data.data_manager Package#
Classes#
|
Cache provides a way to download and cache files. |
|
This class provides a remote data manager for managing remote files. |
Concrete implementation of |
|
|
This provides a sqlite backend for storage. |
Class Inheritance Diagram#
sunpy.data.data_manager.downloader Module#
Classes#
Base class for remote data manager downloaders. |
|
Error to be raised when a download fails. |
|
Concrete implementation of |
Class Inheritance Diagram#
sunpy.data.data_manager.storage Module#
Storage module contains the abstract implementation of storage
for sunpy.data.data_manager.Cache and a concrete implementation
using sqlite.
Classes#
Base class for remote data manager storage providers. |
|
|
This provides a sqlite backend for storage. |
This provides a storage stored in memory. |
Class Inheritance Diagram#
Image processing (sunpy.image)#
sunpy.image contains routines to process images (i.e. numpy arrays). The
routines in this submodule are generally exposed through map-specific functions
in other places.
sunpy.image Package#
sunpy.image.resample Module#
Image resampling methods.
Functions#
|
Returns a new |
|
Re-shape the two dimension input image into a a four dimensional array whose first and third dimensions express the number of original pixels in the "x" and "y" directions that form one superpixel. |
sunpy.image.transform Module#
Functions for geometrical image transformation and warping.
Functions#
|
Decorator to add a rotation function to the registry of selectable implementations. |
|
Rotates, shifts and scales an image. |
Input/output (sunpy.io)#
sunpy.io contains readers for files that are commonly used in solar physics.
Unified File Readers#
sunpy.io Package#
Functions#
|
Attempts to determine the type of file a given filepath is. |
|
Automatically determine the filetype and read the file. |
|
Reads the header from a given file. |
|
Write a file from a data & header pair using one of the defined file types. |
sunpy.io.ana Module#
This module provides an ANA file Reader.
This is a modified version of pyana.
Warning
The reading and writing of ana files is not supported under Windows.
By default, this module is not installed on platforms other than Linux (x86-64) and macOS (x86-64 and ARM64). See the installation guide for more info.
Functions#
|
Loads an ANA file and returns the data and a header in a list of (data, header) tuples. |
|
Loads an ANA file and only return the header consisting of the dimensions, size (defined as the product of all dimensions times the size of the datatype, this not relying on actual filesize) and comments. |
|
Saves a 2D |
Special File Readers#
sunpy.io.special.genx Module#
This module implements a solarsoft genx file reader.
Functions#
|
solarsoft genx file reader. |
sunpy has a custom reader for NOAA SWPC Solar Region Summary (SRS) files:
sunpy.io.special.srs Module#
This module implements a SRS File Reader.
Functions#
|
Parse a SRS table from NOAA SWPC. |
asdf (Advanced Scientific Data Format)#
ASDF is a modern file format designed to meet the needs of the astronomy community. It has deep integration with Python, SunPy, and Astropy, as well as implementations in other languages. It can be used to store known Python objects in a portable, well defined file format. It is primarily useful for storing complex Astropy and SunPy objects in a way that can be loaded back into the same form as they were saved. It is designed to be an archive file format, with human readable metadata and a simple on-disk layout.
sunpy currently implements support for saving Map and
coordinate frame objects into asdf files. As asdf
tightly integrates into Python, saving a map to an asdf file will save the
metadata, data and mask. In comparison, the mask is not currently saved
to FITS. The following code shows to to save and load a sunpy Map to an asdf
file
>>> import asdf
>>> import sunpy.map
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> aiamap = sunpy.map.Map(AIA_171_IMAGE)
>>> tree = {'amap': aiamap}
>>> with asdf.AsdfFile(tree) as asdf_file:
... asdf_file.write_to("sunpy_map.asdf")
>>> input_asdf = asdf.open("sunpy_map.asdf")
>>> input_asdf['amap']
<sunpy.map.sources.sdo.AIAMap object at ...>
SunPy Map
---------
Observatory: SDO
Instrument: AIA 3
Detector: AIA
Measurement: 171.0 Angstrom
Wavelength: 171.0 Angstrom
Observation Date: 2011-06-07 06:33:02
Exposure Time: 0.234256 s
Dimension: [1024. 1024.] pix
Coordinate System: helioprojective
Scale: [2.402792 2.402792] arcsec / pix
Reference Pixel: [511.5 511.5] pix
Reference Coord: [3.22309951 1.38578135] arcsec
array([[ -95.92475 , 7.076416 , -1.9656711, ..., -127.96519 ,
-127.96519 , -127.96519 ],
[ -96.97533 , -5.1167884, 0. , ..., -98.924576 ,
-104.04137 , -127.919716 ],
[ -93.99607 , 1.0189276, -4.0757103, ..., -5.094638 ,
-37.95505 , -127.87541 ],
...,
[-128.01454 , -128.01454 , -128.01454 , ..., -128.01454 ,
-128.01454 , -128.01454 ],
[-127.899666 , -127.899666 , -127.899666 , ..., -127.899666 ,
-127.899666 , -127.899666 ],
[-128.03072 , -128.03072 , -128.03072 , ..., -128.03072 ,
-128.03072 , -128.03072 ]], dtype=float32)
>>> input_asdf.close()
When saving a Map to ASDF all maps are saved as a GenericMap and not a specific source class.
This comes with some trade-offs.
If you are using custom map sources defined outside of the sunpy core package, and these sources are imported after asdf has been invoked for the first time (used, not just imported), then they will not be registered with the asdf converter.
Also if the custom map subclass is not registered with sunpy.map.Map upon loading of the map, it will be returned as a GenericMap.
This approach has been chosen despite these limitations so that once a map is saved to an ASDF file it can always be loaded back into a map rather than the asdf library returning it as a Python dictionary.
It also follows the philosophy of the way maps are saved and loaded in the FITS format, where the components of the Map are serialised and the way meta data is handled depends solely on the contents of the .meta attribute.
Internal API#
These are readers and other utilities that are used internally by sunpy to read files.
They are not intended to be used directly by users and we do not guarantee that they will
work for all files of a given type nor will the API be stable.
sunpy.io._fits Module#
This module provides a FITS file reader for internal use.
We instead recommend users use the astropy.io.fits module,
which provides more generic functionality to read FITS files.
Notes
FITS files allow comments to be attached to every value in the header. This is implemented in this module as a KEYCOMMENTS dictionary in the sunpy header. To add a comment to the file on write, add a comment to this dictionary with the same name as a key in the header (upcased).
Due to the way
fitsworks with images, the header dictionary may differ depending on whether is accessed before or after the fits[0].data is requested. If the header is read before the data then the original header will be returned. If the header is read after the data has been accessed then the data will have been scaled and a modified header reflecting these changes will be returned: BITPIX may differ and BSCALE and B_ZERO may be dropped in the modified version.The verify(‘silentfix+warn’) call attempts to handle violations of the FITS standard. For example,
nanvalues will be converted to “nan” strings. Attempting to cast aastropy.io.fits.Headerto a dictionary while it contains invalid header tags will result in an error so verifying it early on makes the header easier to work with later.
Functions#
|
Convert a header dict to a |
|
Read a fits file. |
|
Read a fits file and return just the headers for all HDU's. |
|
Take a data header pair and write a FITS file. |
|
Attempt to read the wavelength unit from a given FITS header. |
|
Combine |
sunpy.io.header Module#
This module provides a generic FileHeader object for the readers.
Classes#
|
FileHeader is designed to provide a consistent interface to all other sunpy classes that expect a generic file. |
Class Inheritance Diagram#
sunpy.io._jp2 Module#
This module provides a JPEG 2000 file reader for internal use.
It was developed to read JPEG 2000 files created by the Helioviewer Project and not as a general JPEG 2000 file reader.
Functions#
|
Reads a JPEG2000 file. |
|
Reads the header from the file. |
|
Take a data header pair and write a JPEG2000 file. |
sunpy.io._file_tools Module#
This module provides a generic file reader for internal use.
Functions#
|
Automatically determine the filetype and read the file. |
|
Reads the header from a given file. |
|
Write a file from a data & header pair using one of the defined file types. |
|
Attempts to determine the type of file a given filepath is. |
CDF (common data format)#
CDF files are commonly used to store timeseries data observed by instruments taking in-situ measurements of plasmas throughout the heliosphere. sunpy provides support for reading in CDF files that conform to the Space Physics Guidelines for CDF.
sunpy.io._cdf Module#
This module provides a CDF file reader for internal use.
Functions#
|
Read a CDF file that follows the ISTP/IACG guidelines. |
Maps (sunpy.map)#
sunpy Map objects are constructed using the special factory class:
- class sunpy.map.map_factory.MapFactory(default_widget_type=None, additional_validation_functions=[], registry=None)[source]#
A factory for generating coordinate aware 2D images.
This factory takes a variety of inputs, such as file paths, wildcard patterns or (data, header) pairs.
Depending on the input different return types are possible.
- Parameters:
*inputs – Inputs to parse for map objects. See the examples section for a detailed list of accepted inputs.
sequence (
bool, optional) – Return asunpy.map.MapSequenceobject comprised of all the parsed maps.composite (
bool, optional) – Return asunpy.map.CompositeMapobject comprised of all the parsed maps.
- Returns:
sunpy.map.GenericMap– If the input results in a singular map object, then that is returned.listofGenericMap– If multiple inputs are given andsequence=Falseandcomposite=False(the default) then a list ofGenericMapobjects will be returned.sunpy.map.MapSequence– If the input corresponds to multiple maps andsequence=Trueis set, then aMapSequenceobject is returned.sunpy.map.CompositeMap– If the input corresponds to multiple maps andcomposite=Trueis set, then aCompositeMapobject is returned.
Examples
>>> import sunpy.map >>> from astropy.io import fits >>> import sunpy.data.sample >>> mymap = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
sunpy.map Package#
Map objects are constructed using the special factory class: Map.
All sunpy Maps are derived from sunpy.map.GenericMap, all the methods and attributes are documented in that class.
The result of a call to Map will be either a GenericMap object if no instrument matches, or a subclass of GenericMap which deals with a specific source of data, e.g., AIAMap or LASCOMap (see sunpy.map Package to see a list of all of them).
sunpy.map Package#
Functions#
|
Returns the coordinates of the center of every pixel in a map. |
Returns the coordinates of the pixel corners in a map. |
|
Returns pixel pair indices of every pixel in a map. |
|
|
Checks whether a coordinate falls within the bounds of a map. |
|
Checks if a map contains the full disk of the Sun. |
|
Checks if a map contains any part of the solar limb or equivalently whether the map contains both on-disk and off-disk pixels. |
|
Returns |
|
Checks if the helioprojective Cartesian coordinates are on the solar disk. |
|
Checks if none of the coordinates in the |
|
Checks if all of the coordinates in the |
|
Returns the pixel locations of the edges of an input map. |
Returns the the bottom left and top right coordinates of the smallest rectangular region that contains all the on disk coordinates of the input map. |
|
|
Return the pixel coordinates for every pixel that intersects with a coordinate path. |
|
Samples the data in a map at given series of coordinates. |
|
Calculates the solar angular radius as seen by the observer. |
Classes#
|
A Composite Map class |
|
A Generic spatially-aware 2D data array |
|
A series of Maps in a single object. |
|
Variables#
A factory for generating coordinate aware 2D images. |
Class Inheritance Diagram#
Header helpers#
The header_helper sub-module contains helper functions for generating FITS-WCS headers from Python objects.
sunpy.map.header_helper Module#
Functions#
|
Function to create a FITS-WCS header from a coordinate object ( |
|
Function to get observer meta from coordinate frame. |
|
Construct a FITS-WCS header for a full-Sun heliographic (Carrington or Stonyhurst) coordinate frame. |
Instrument Map Classes#
Defined in sunpy.map.sources are a set of GenericMap subclasses which convert the specific metadata and other differences in each instruments data to the standard GenericMap interface.
These ‘sources’ also define things like the colormap and default normalization for each instrument.
These subclasses also provide a method, which describes to the Map factory which data and metadata pairs match its instrument.
sunpy.map.sources Package#
Functions#
|
Test determining if the given metadata contains Helioviewer Project sourced data. |
|
Assign the correct source-dependent image stretching function. |
Classes#
|
AIA Image Map. |
|
STEREO-SECCHI CORonograph Image Map. |
|
SOHO EIT Image Map. |
|
EUI Image Map |
|
STEREO-SECCHI EUVI Image Map |
|
GONG H-Alpha Map. |
|
GONG Synoptic Map. |
|
STEREO-SECCHI Heliospheric Imager (HI) Map. |
|
HMI Image Map. |
|
SDO/HMI Synoptic Map. |
|
K-Cor Image Map. |
|
SOHO LASCO Image Map |
|
SOHO MDI Image Map |
|
SOHO MDI synoptic magnetogram Map. |
|
RHESSI Image Map. |
|
A 2D IRIS Slit Jaw Imager Map. |
|
Hinode SOT Image Map definition. |
|
SUVI Image Map. |
|
PROBA2 SWAP Image Map. |
|
Yohkoh SXT Image Map |
|
TRACE Image Map |
|
WISPR Map |
|
Hinode XRT map definition. |
Class Inheritance Diagram#
Remote data (sunpy.net)#
sunpy.net contains a lot of different code for accessing various solar
physics related web services. This submodule contains many layers. Most users
should use Fido, which is an interface to multiple sources
including all the sources implemented in dataretriever as well as
vso and jsoc. Fido can be used like so:
>>> from sunpy.net import Fido, attrs as a
>>> results = Fido.search(a.Time("2012/1/1", "2012/1/2"), a.Instrument.lyra)
>>> files = Fido.fetch(results)
sunpy.net Package#
Classes#
|
A Scraper to scrap web data archives based on dates. |
Variables#
Fido is a unified data search and retrieval tool. |
Class Inheritance Diagram#
sunpy.net.attrs Module#
‘attrs’ are parameters which can be composed together to specify searches to
sunpy.net.Fido. They can be combined
by using logical and (&) and logical or (|) operations to construct
very complex queries.
For example you could combine two instruments using or (|) with a time
specification and a sample cadence using:
>>> import astropy.units as u
>>> from sunpy.net import Fido, attrs as a
>>> a.Time("2011/01/01", "2011/01/02") & (a.Instrument.aia | a.Instrument.hmi) & a.Sample(1*u.day))
In addition to the core attrs defined here, other sunpy clients also provide attrs specific to them, under:
Classes#
|
Specify the time range of the query. |
|
Specifies the Instrument name for the search. |
|
|
|
Specifies the data processing level to search for. |
|
The type of Extent; for example, "FULLDISK", "SYNOPTIC", "LIMB", etc. |
|
Time interval for data sampling. |
|
The detector from which the data comes from. |
|
Resolution level of the data. |
|
Specifies the physical observable the VSO can search for. |
|
Data sources that Fido can search with. |
|
Specifies the data provider to search for data using Fido. |
|
Attribute representing attributes ANDed together. |
|
Attribute representing attributes ORed together. |
Class Inheritance Diagram#
sunpy.net.fido_factory Module#
This module provides the Fido instance of
sunpy.net.fido_factory.UnifiedDownloaderFactory it also provides the
UnifiedResponse class which
Fido.search returns and the
parfive.Results class that is returned by
Fido.fetch.
Classes#
|
The object used to store results from |
|
Fido is a unified data search and retrieval tool. |
Class Inheritance Diagram#
VSO#
sunpy.net.vso Package#
Classes#
|
Provides access to query and download from Virtual Solar Observatory (VSO). |
|
Class Inheritance Diagram#
sunpy.net.vso.attrs Module#
Attributes that can be used to construct VSO queries.
Attributes are the fundamental building blocks of queries that, together with the two operations of AND and OR (and in some rare cases XOR) can be used to construct complex queries. Most attributes can only be used once in an AND-expression, if you still attempt to do so it is called a collision. For a quick example think about how the system should handle Instrument(‘aia’) & Instrument(‘eit’).
Classes#
|
Specify the spatial field-of-view of the query. |
|
A subclass of the value attribute. |
|
Pixels are (currently) limited to a single dimension (and only implemented for SDO data) We hope to change this in the future to support TRACE, Hinode and other investigations where this changed between observations. |
|
This attribute is a placeholder for the future. |
|
Retrieve 'quicklook' data if available. |
|
Pixel Scale (PSCALE) is in arc seconds. |
Class Inheritance Diagram#
Dataretriever#
sunpy.net.dataretriever Package#
The sunpy.net.dataretriever submodule is a framework for downloading data
from “simple” web sources such as HTTP or FTP servers. Although it could be
used for more complex services as well. Following the example of sunpy.map
and sunpy.timeseries this module provides a base class
GenericClient from which specific services can
subclass. All these subclasses are then registered with the sunpy.net.Fido factory
class, so do not need to be called individually.
Classes#
Provides access to Level 0CS Extreme ultraviolet Variability Experiment (EVE) data. |
|
Provides access to data from the Gamma-Ray Burst Monitor (GBM) instrument on board the Fermi satellite. |
|
Provides access to the Magnetogram products of NSO-GONG synoptic Maps. |
|
Base class for simple web clients for the data retriever module. |
|
Provides access to the LYRA/Proba2 data archive. |
|
Provides access to the NOAA solar cycle indices. |
|
Provides access to the NOAA SWPC predicted sunspot Number and 10.7 cm radio flux values. |
|
Provides access to the Nobeyama RadioHeliograph (NoRH) averaged correlation time series data. |
|
|
|
Provides access to the RHESSI observing summary time series data. |
|
Provides access to the NOAA SWPC solar region summary data. |
|
Provides access to data from the GOES Solar Ultraviolet Imager (SUVI). |
|
Provides access to several GOES XRS files archive. |
Class Inheritance Diagram#
sunpy.net.dataretriever.attrs.goes Module#
Classes#
|
The GOES Satellite Number |
Class Inheritance Diagram#
JSOC#
sunpy.net.jsoc Package#
Classes#
|
Select a cutout region. |
Provides access to the JSOC Data Export service. |
|
|
|
|
Allows comparison filtering of the JSOC Keywords. |
|
An email address to get a notification to when JSOC has staged your request. |
|
Prime Keys |
|
The type of download to request one of ("FITS", "JPEG", "MPG", "MP4", or "as-is"). |
|
Segments choose which files to download when there are more than one present for each record e.g. 'image'. |
|
The JSOC Series to Download. |
Class Inheritance Diagram#
sunpy.net.jsoc.attrs Module#
Classes#
|
The JSOC Series to Download. |
|
The type of download to request one of ("FITS", "JPEG", "MPG", "MP4", or "as-is"). |
|
An email address to get a notification to when JSOC has staged your request. |
|
Segments choose which files to download when there are more than one present for each record e.g. 'image'. |
|
Prime Keys |
|
Select a cutout region. |
|
Allows comparison filtering of the JSOC Keywords. |
Class Inheritance Diagram#
HEK#
sunpy.net.hek Package#
Classes#
|
Provides access to the Heliophysics Event Knowledgebase (HEK). |
|
Handles the response from the HEK. |
|
A container for data returned from HEK searches. |
Class Inheritance Diagram#
sunpy.net.hek.attrs Module#
Attributes that can be used to construct HEK queries. They are different to the VSO ones in that a lot of them are wrappers that conveniently expose the comparisons by overloading Python operators. So, e.g., you are able to say AR & AR.NumSpots < 5 to find all active regions with less than 5 spots. As with the VSO query, you can use the fundamental logic operators AND and OR to construct queries of almost arbitrary complexity. Note that complex queries result in multiple requests to the server which might make them less efficient.
Classes#
|
|
|
|
|
This ensures the attr inspect magic works for registering in the client. |
|
Class Inheritance Diagram#
sunpy.net.hek2vso Package#
This module provides a translation layer between the HEK and the VSO. It allows you to acquire records of data that are available via the VSO, based on the data in HEK event entries.
Warning
This module is in beta and maybe unstable.
Functions#
|
Formulate VSO queries from HEK results. |
|
Parses VSO attributes from a HEK result. |
Classes#
Class to handle HEK to VSO translations |
Class Inheritance Diagram#
CDAWeb#
sunpy.net.cdaweb Package#
Functions#
|
Get a list of datasets for a given observatory. |
Get a list of observatory IDs for each observatory in CDAWeb. |
Classes#
Provides access to query and download from the Coordinated Data Analysis Web (CDAWeb). |
|
|
Dataset ID. |
Class Inheritance Diagram#
HELIO#
sunpy.net.helio Package#
A Module for accessing the HELIO web service
Classes#
|
Provides access to the HELIO webservices. |
|
A container for data returned from HEC searches. |
|
A tool to infer some information from chaincodes produced by HELIO Feature Catalogue or Heliophysics Events Knowledgebase. |
Class Inheritance Diagram#
sunpy.net.helio.attrs Module#
Classes#
|
The maximum number of desired records. |
|
The table to query from |
Class Inheritance Diagram#
Internal Classes and Functions#
These classes and functions are designed to be used to help develop new clients
for sunpy.net.Fido.
sunpy.net.base_client Module#
Functions#
|
A wrapper to convert any |
Classes#
|
A column subclass which knows about the client of the parent table. |
An Abstract Base Class for results returned from BaseClient. |
|
|
A row subclass which knows about the client of the parent table. |
|
A class to represent tables of heterogeneous data. |
This defines the Abstract Base Class for each download client. |
Class Inheritance Diagram#
sunpy.net.dataretriever.client Module#
Classes#
|
|
Base class for simple web clients for the data retriever module. |
Class Inheritance Diagram#
sunpy.net.attr Module#
Allow representation of queries as logic expressions. This module makes sure that attributes that are combined using the two logic operations AND (&) and OR (|) always are in disjunctive normal form, that is, there are only two levels - the first being disjunction and the second being conjunction. In other words, every combinations of attributes looks like this: (a AND b AND c) OR (d AND e).
Walkers are used to traverse the tree that results from combining attributes.
They are implemented using functools.singledispatch modified to dispatch on the second argument to the function.
Please note that & is evaluated first, so A & B | C is equivalent to (A & B) | C.
Functions#
|
Trick operator precedence. |
|
Trick operator precedence. |
Classes#
|
This is the base for all attributes. |
|
A base class for attributes classes which contain data. |
Empty attribute. |
|
|
An attribute that only has a single value. |
|
An attribute that represents a range of a value. |
|
Attribute representing attributes ANDed together. |
|
Attribute representing attributes ORed together. |
|
|
Traverse the Attr tree and convert it to a different representation. |
|
|
Allows a Attr to have a value and a comparison operator. |
Class Inheritance Diagram#
sunpy.net.scraper Module#
This module provides a web scraper.
Classes#
|
A Scraper to scrap web data archives based on dates. |
Class Inheritance Diagram#
Physics (sunpy.physics)#
sunpy.physics contains routines to calculate various physical parameters
and models for the Sun.
sunpy.physics Package#
sunpy.physics.differential_rotation Module#
Functions#
|
Deprecated since version 6.0. |
|
Given a coordinate on the Sun, calculate where that coordinate maps to as seen by a new observer at some later or earlier time, given that the input coordinate rotates according to the solar rotation profile. |
|
Warp a |
Solar properties (sunpy.sun)#
sunpy.sun contains constants, parameters and models of the Sun.
sunpy.sun.constants Module#
This module provides fundamental solar physical constants. The following constants are available:
Name |
Value |
Unit |
Description |
|---|---|---|---|
mass |
1.98847542e+30 |
kg |
Solar mass |
radius |
695700000 |
m |
Nominal solar radius |
luminosity |
3.828e+26 |
W |
Nominal solar luminosity |
mean distance |
1.49597871e+11 |
m |
Astronomical Unit |
perihelion distance |
1.471e+11 |
m |
Perihelion Distance |
aphelion distance |
1.521e+11 |
m |
Aphelion Distance |
age |
4.6e+09 |
year |
Age of the Sun |
solar flux unit |
1e-22 |
W m**-2 Hz**-1 |
Solar flux unit |
visual magnitude |
-26.75 |
Apparent visual magnitude |
|
average angular size |
959.63 |
arcsec |
Semidiameter |
surface area |
6.087e+18 |
m**2 |
Surface area |
average density |
1409 |
kg m**-3 |
Mean density |
surface gravity |
274 |
m s**-2 |
Surface gravity |
moment of inertia |
5.7e+54 |
kg m**-2 |
Moment of inertia |
volume |
1.4122e+27 |
m**3 |
Volume |
escape velocity |
617700 |
m s**-1 |
Escape velocity at surface |
oblateness |
8.01 |
marcsec |
oblateness |
metallicity |
0.0122 |
Metallicity |
|
sunspot cycle |
11.4 |
year |
Average duration of sunspot cycle |
average intensity |
20090000 |
W m**-2 sr**-1 |
Mean Intensity |
effective temperature |
5778 |
K |
Effective black-body temperature |
mass conversion rate |
4.3e+09 |
kg s**-1 |
Mass conversion rate |
center density |
162200 |
kg m**-3 |
Center density |
center temperature |
15710000 |
K |
Center temperature |
absolute magnitude |
4.83 |
Absolute magnitude |
|
mean energy production |
0.0001937 |
J kg**-1 |
mean energy production |
ellipticity |
5e-05 |
ellipticity |
|
GM |
132712000 |
km**3 s**-2 |
standard gravitational parameter |
W_0 |
84.176 |
deg |
longitude of the prime meridian (epoch J2000.0) |
sidereal rotation rate |
14.1844 |
deg day**-1 |
sidereal rotation rate |
first Carrington rotation (JD TT) |
2398167.4 |
day |
first Carrington rotation (JD TT) |
mean synodic period |
27.2752612 |
day |
mean synodic period |
alpha_0 |
286.13 |
deg |
right ascension (RA) of the north pole (epoch J2000.0) |
delta_0 |
63.87 |
deg |
declination of the north pole (epoch J2000.0) |
Functions#
|
Retrieve a constant by key. |
|
Return list of constants keys containing a given string. |
Provides a table of the complete list of constants. |
Variables#
Spectral classification |
|
Astronomical Unit |
|
Solar mass |
|
Nominal solar radius |
|
Volume |
|
Surface area |
|
Mean density |
|
Surface gravity |
|
Effective black-body temperature |
|
Nominal solar luminosity |
|
Mass conversion rate |
|
Escape velocity at surface |
|
Solar flux unit |
|
Semidiameter |
|
sidereal rotation rate |
|
Time of the start of the first Carrington rotation |
|
mean synodic period |
sunpy.sun.models Module#
Solar Physical Models#
This module contains models of the Sun from various sources:
interior:QTableof the structure of the solar interior as defined in Table 7 of Turck-Chieze et al. (1988)evolution:QTableof the evolution of the Sun over time as defined in Table 6 of Turck-Chieze et al. (1988)differential_rotation(): Function for calculating solar differential rotation for different models
References
Functions#
|
Computes the change in longitude over a duration for a given latitude. |
Variables#
A class to represent tables of heterogeneous data. |
|
A class to represent tables of heterogeneous data. |
Time (sunpy.time)#
sunpy.time contains helpers for converting strings to astropy.time.Time
objects and handling common operations on these objects. As well as this a
TimeRange object is provided for representing a period of time
and performing operations on that range.
sunpy.time Package#
Functions#
|
Return iterator of occurrences of date formatted with format in string. |
|
Returns true if the input is a valid date/time representation. |
|
Work around for astropy/astropy#6970. |
|
Tests whether a time string is formatted according to the given time format. |
|
Returns the number of Julian centuries since J1900.0 (noon on 1900 January 0). |
|
Takes a time input and will parse and return a |
Classes#
|
A class to create and handle time ranges. |
|
SI seconds from 1958-01-01 00:00:00, which includes UTC leap seconds. |
|
UT seconds from 1979-01-01 00:00:00 UTC, ignoring leap seconds. |
Class Inheritance Diagram#
Timeseries (sunpy.timeseries)#
One of the core classes in sunpy is a timeseries.
A number of instruments are supported through subclasses of the base GenericTimeSeries class.
See Instrument TimeSeries Classes for a list of them.
A timeseries can be created by calling TimeSeries.
Instrument TimeSeries Classes#
The generic method to create an instrument-specific TimeSeries is to call TimeSeries with a file path and the instrument-specific source keyword argument.
In some cases the source can be determined automatically if a FITS file is being loaded.
The following example shows the factory loading a sample file:
>>> import sunpy.timeseries as ts
>>> import sunpy.data.sample
>>> goes = ts.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES, source='XRS')
The TimeSeries factory will load the file and create the timeseries instance.
The following instrument classes are supported:
sunpy.timeseries Package#
Classes#
|
Used to store metadata for |
|
A generic time series object. |
Variables#
A factory for generating solar timeseries objects. |
sunpy.timeseries.sources Package#
This module provides a collection of datasource-specific
TimeSeries classes.
Each mission should have its own file with one or more classes defined.
Typically, these classes will be subclasses of the
sunpy.timeseries.TimeSeries.
Classes#
|
SDO EVE/ESP Level 1 data. |
|
SDO EVE LightCurve for level 0CS data. |
|
Fermi/GBM Summary lightcurve TimeSeries. |
|
GOES XRS Time Series. |
|
Proba-2 LYRA Lightcurve TimeSeries. |
|
NOAA Solar Cycle monthly indices. |
|
NOAA Solar Cycle Predicted Progression. |
|
Nobeyama Radioheliograph Correlation lightcurve TimeSeries. |
|
RHESSI X-ray Summary lightcurve TimeSeries. |
Class Inheritance Diagram#
CDF files#
GenericTimeSeries can load a single CDF file, or a list of CDF files if concatenate=True is passed.
Units#
The physical units of different columns in CDF files do not conform to a standard that astropy.units understands.
sunpy internally stores a set of common mappings from unit strings to Unit, but you may see a warning about unrecognised unit strings when reading a CDF file.
To register the correct unit definition astropy.units.add_enabled_units() can be used.
For example, to register ‘deg K’ as representing Kelvin and ‘#/cc’ as 1/cm^3:
>>> import astropy.units as u
>>> _ = u.add_enabled_units([u.def_unit('deg K', represents=u.K), u.def_unit('#/cc', represents=u.cm**-3)])
Utilities (sunpy.util)#
sunpy.util Package#
Functions#
|
Used to mark a function or class as deprecated. |
|
Makes sure that a list of dictionaries all have the same keys. |
|
Expand a list of lists or tuples. |
|
List installed and missing dependencies. |
|
Returns a new documentation string such that there are notes section duplication in in |
|
Returns a set of keyword names from |
|
Returns a set of keyword names that can be handled by an object's |
Gets the width of the current terminal. |
|
|
Returns the SHA-256 hash of a file. |
|
Get all the specified extras for a package and report any missing dependencies. |
|
Return a replacement path if input path is currently in use. |
|
A decorator that tracks the entry and exit of a context manager, setting the key's value to True on entry and False on exit. |
Prints ones' system info in an "attractive" fashion. |
|
|
Return only unique elements of a sequence. |
|
Raise a |
|
Raise a |
|
Raise a |
|
Raise a |
Classes#
|
A class to hold metadata associated with a |
An error raised when a file is opened and no maps are found. |
|
A warning class to indicate a connection warning. |
|
A warning class to indicate a deprecated feature. |
|
Warning class for cases metadata is missing. |
|
A warning class to indicate a soon-to-be deprecated feature. |
|
The primary warning class for Sunpy. |
|
The base warning class from which all Sunpy warnings should inherit. |
Class Inheritance Diagram#
sunpy.util.config Module#
This module provides SunPy’s configuration file functionality.
Functions#
Read the "sunpyrc" configuration file. |
|
|
Copies the default sunpy config file to the user's config directory. |
Print current configuration options. |
sunpy.util.datatype_factory_base Module#
This module provides the base registration factory used for all SunPy data/net factories.
Classes#
|
Generalized registerable factory type. |
Exception for when no candidate class is found. |
|
Exception for when too many candidate classes are found. |
|
Exception for when no candidate class is found. |
Class Inheritance Diagram#
sunpy.util.net Module#
This module provides general net utility functions.
Functions#
|
Parse a Content-type like header. |
|
Slugify given unicode text. |
|
Get the content disposition filename from given header. |
|
Get filename from given |
|
Get filename from given |
|
Download a file from a url into a directory. |
|
Download a file from a url into a directory. |
sunpy.util.xml Module#
This module provides XML helper functions.
Functions#
|
Converts an XML string to a Python dictionary. |
|
Scans through the children of the node and makes a dictionary from the content. |
|
Scans through all children of |
Classes#
Class Inheritance Diagram#
sunpy.util.sphinx Package#
Helpers and extensions for sphinx.
This subpackage contains two sphinx directives:
.. generate::which includes raw output generated by a Python script into a.. raw::block. This can be used to for example, include HTML output into the built docs... changelog::which renders the latest changelog with towncrier and includes it in the documentation.
To use them add 'sunpy.util.sphinx.generate' and
'sunpy.util.sphinx.changelog' to the extensions list in your conf.py
file.
sunpy.util.sphinx.generate Module#
Classes#
|
Custom directive to include raw output generated using supplied Python code |
Class Inheritance Diagram#
sunpy.util.functools Module#
This file defines wrappers and variants of things in the functools standard lib.
Functions#
|
A variant of |
Visualization (sunpy.visualization)#
sunpy.visualization contains plotting helpers and functions.
sunpy.visualization Package#
Functions#
|
Returns axis labels for the given coordinate type and unit. |
|
A decorator to place on |
sunpy.visualization.colormaps Package#
The following colormaps are provided by this module.
(Source code, png, hires.png, pdf)
‘goes-rsuvi94’
‘goes-rsuvi131’
‘goes-rsuvi171’
‘goes-rsuvi195’
‘goes-rsuvi284’
‘goes-rsuvi304’
‘sdoaia94’
‘sdoaia131’
‘sdoaia171’
‘sdoaia193’
‘sdoaia211’
‘sdoaia304’
‘sdoaia335’
‘sdoaia1600’
‘sdoaia1700’
‘sdoaia4500’
‘sohoeit171’
‘sohoeit195’
‘sohoeit284’
‘sohoeit304’
‘soholasco2’
‘soholasco3’
‘sswidlsoholasco2’
‘sswidlsoholasco3’
‘stereocor1’
‘stereocor2’
‘stereohi1’
‘stereohi2’
‘yohkohsxtal’
‘yohkohsxtwh’
‘hinodexrt’
‘hinodesotintensity’
‘trace171’
‘trace195’
‘trace284’
‘trace1216’
‘trace1550’
‘trace1600’
‘trace1700’
‘traceWL’
‘hmimag’
‘irissji1330’
‘irissji1400’
‘irissji1600’
‘irissji2796’
‘irissji2832’
‘irissji5000’
‘irissjiFUV’
‘irissjiNUV’
‘irissjiSJI_NUV’
‘kcor’
‘rhessi’
‘std_gamma_2’
‘euvi171’
‘euvi195’
‘euvi284’
‘euvi304’
‘solar orbiterfsi174’
‘solar orbiterfsi304’
‘solar orbiterhri_euv174’
‘solar orbiterhri_lya1216’
Functions#
|
Displays a plot of the custom color maps supported in SunPy. |
sunpy.visualization.colormaps.color_tables Module#
This module provides dictionaries for generating
LinearSegmentedColormap, and a dictionary of these
dictionaries.
Functions#
|
Returns one of the fundamental color tables for SDO AIA images. |
|
Returns one of the SSWIDL-defined color tables for SOHO LASCO images. |
|
Returns one of the fundamental color tables for SOHO EIT images. |
|
Returns one of the fundamental color tables for Yokhoh SXT images. |
Returns the color table used for all Hinode XRT images. |
|
|
Returns one of the standard color tables for TRACE JP2 files. |
|
Returns one of the standard color tables for SOT files (following osdc convention). |
Returns an alternate HMI Magnetogram color table; from Stanford University/JSOC. |
|
|
Returns one of the fundamental color tables for SUVI images. |
|
|
sunpy.visualization.animator Package#
Classes#
|
Create an interactive viewer for a |
Class Inheritance Diagram#
sunpy.visualization.wcsaxes_compat Module#
This module provides functions to make WCSAxes work in SunPy.
Functions#
|
Tests a |
|
Get the current axes, or create a new |
|
Get the transformation to world coordinates. |
|
Apply some default |
|
Create a heliographic overlay using |
sunpy.visualization.drawing Module#
This module provides functions that draw on Astropy’s WCSAxes.
Functions#
|
Draws the solar limb as seen by the specified observer. |
|
Draws the solar equator as seen by the axes observer. |
|
Draws the solar prime meridian (zero Carrington longitude) as seen by the axes observer. |
Customizing sunpy#
The sunpyrc file#
The sunpy core package uses a sunpyrc configuration file to customize
certain properties. You can control a number of key features of sunpy such as
where your data will download to. sunpy looks for the sunpyrc file
in a platform specific directory, which you can see the path for by running:
>>> import sunpy
>>> sunpy.print_config()
FILES USED:
...
CONFIGURATION:
[general]
time_format = %Y-%m-%d %H:%M:%S
working_dir = ...
[downloads]
download_dir = ...
remote_data_manager_dir = ...
cache_expiry = 10
sample_dir = ...
[database]
url = sqlite:////...
[logger]
log_level = INFO
use_color = True
log_warnings = True
log_exceptions = False
log_to_file = False
log_file_level = INFO
log_file_format = %(asctime)s, %(origin)s, %(levelname)s, %(message)s
Do not edit the default file (the first in the “FILES USED:” list above) directly as every time you install or update sunpy, this file will be overwritten.
To maintain your personal customizations place a copy of the default “sunpyrc” file into the user configuration path. To find this path, use:
>>> from sunpy.extern.appdirs import AppDirs
>>> AppDirs('sunpy', 'sunpy').user_config_dir
You can use sunpy.util.config.copy_default_config to write the default config into the correct place.
The user configuration path can also be set using an environment variable SUNPY_CONFIGDIR.
Depending on your system, it may be useful to have a site-wide configuration file.
If it is used, it will be on the “FILES USED:” list below the default file.
To find your system’s site configuration path for sunpy, use:
>>> from sunpy.extern.appdirs import AppDirs
>>> AppDirs('sunpy', 'sunpy').site_config_dir
In Unix, the site and user configuration paths follow the XDG specifications.
The site configuration is applied before your personal user configuration, thus your configuration file will override the site configuration settings. For this reason, when you create or edit your personal configuration file, you may want to replicate the site configuration items into your own configuration file, or comment out the items in your configuration file that are set in the site configuration file.
See below for the example config file.
Dynamic settings#
You can also dynamically change the default settings in a python script or
interactively from the python shell. All of the settings are stored in a
Python ConfigParser instance called sunpy.config, which is global to
the sunpy package. Settings can be modified directly, for example:
import sunpy
sunpy.config.set('downloads', 'download_dir', '/home/user/Downloads')
A sample sunpyrc file#
;
; SunPy Configuration
;
; This is a sample sunpy configuration file - you can find a copy
; of it on your system in site-packages/sunpy/data/sunpyrc. If you edit it
; there, please note that it will be overridden in your next install.
; If you want to keep a permanent local copy that will not be
; over-written, we use AppDirs https://pypi.org/project/appdirs/
; to find the right place for each OS to place it.
; So if you open the link, you can find the user_config_dir
; or print(sunpy.util.config.CONFIG_DIR) to find it on your own system.
; Note that any relative filepaths specified in the SunPy configuration file
; will be relative to SunPy's working directory.
;;;;;;;;;;;;;;;;;;;
; General Options ;
;;;;;;;;;;;;;;;;;;;
[general]
; The SunPy working directory is the parent directory where all generated
; and download files will be stored.
; Default Value: <user's home directory>/sunpy
; working_dir = /home/$USER/sunpy
; Time Format to be used for displaying time in output (e.g. graphs)
; The default time format is based on ISO8601 (replacing the T with space)
; note that the extra '%'s are escape characters
time_format = %Y-%m-%d %H:%M:%S
;;;;;;;;;;;;;
; Downloads ;
;;;;;;;;;;;;;
[downloads]
; Location to save download data to. Path should be specified relative to the
; SunPy working directory.
; Default value: data/
download_dir = data
; Location to save remote data manager (sunpy.data.manager) data to.
; Path should be specified relative to the SunPy working directory.
; Default value: data_manager/
remote_data_manager_dir = data_manager
; Time interval for cache expiry in days.
; Default value: 10
cache_expiry = 10
; Location where the sample data will be downloaded. If not specified, will be
; downloaded to platform specific user data directory.
; The default directory is specified by appdirs (https://github.com/ActiveState/appdirs)
; sample_dir = /data/sample_data
;;;;;;;;;;;;
; Logger ;
;;;;;;;;;;;;
[logger]
# Threshold for the logging messages. Logging messages that are less severe
# than this level will be ignored. The levels are 'DEBUG', 'INFO', 'WARNING',
# 'ERROR'
log_level = INFO
# Whether to use color for the level names
use_color = True
# Whether to log warnings.warn calls
log_warnings = True
# Whether to log exceptions before raising them
log_exceptions = False
# Whether to always log messages to a log file
log_to_file = False
# The file to log messages to
# log_file_path = sunpy.log
# Threshold for logging messages to log_file_path
log_file_level = INFO
# Format for log file entries
log_file_format = %(asctime)s, %(origin)s, %(levelname)s, %(message)s
Troubleshooting and Bugs#
Obtaining sunpy version#
To find out your sunpy version number, import it and print the __version__ attribute:
>>> import sunpy
>>> sunpy.__version__
System Info#
To quickly collect information on your system, you can use our convenience function system_info which you can run through:
>>> import sunpy
>>> sunpy.util.system_info()
The output should look something like:
==========================================================
sunpy Installation Information
Sunday, 18. November 2012 11:06PM UT
==========================================================
###########
General
###########
OS: Mac OS X 10.8.2 (i386)
Python: 2.7.3 (64bit)
####################
Required libraries
####################
sunpy: 0.1
NumPy: 1.6.2
SciPy: 0.10.1
Matplotlib: 1.2.x
PyFITS: 3.0.8
pandas: 0.8.1
#######################
Recommended libraries
#######################
beautifulsoup4: 4.1.1
PyQt: 4.9.4
SUDS: 0.4'
This information is especially useful if you are running into a bug and need help.
Making use of the logger#
For information on configuring and using sunpy's logging system, a useful tool for troubleshooting, see Logger Objects.
sunpy install location#
You can find what directory sunpy is installed in by importing it and printing the __file__ attribute:
>>> import sunpy
>>> sunpy.__file__
.sunpy directory location#
Each user should have a .sunpy/ directory which should contain a sunpyrc file.
To locate your .sunpy/ directory, use sunpy.print_config():
>>> import sunpy as sun
>>> sun.print_config()
We use appdirs to work out the location depending on your operating system.
If you would like to use a different configuration directory, you can do so by specifying the location in your SUNPY_CONFIGDIR environment variable.
Reporting Bugs#
If you are having a problem with sunpy, search the mailing list or the github issue tracker. It is possible that someone else has already run into your problem.
If not, please provide the following information in your e-mail to the mailing list or to the github issue tracker:
your operating system; (Linux/UNIX users: post the output of
uname -a)sunpy version:
>>> import sunpy >>> sunpy.util.system_info()how you obtained sunpy.
any customizations to your
sunpyrcfile (see Customizing sunpy).Please try to provide a minimal, standalone Python script that demonstrates the problem. This is the critical step. If you can’t post a piece of code that we can run and reproduce your error, the chances of getting help are significantly diminished. Very often, the mere act of trying to minimize your code to the smallest bit that produces the error will help you find a bug in your code that is causing the problem.
SSWIDL/sunpy Cheat Sheet#
SolarSoft (SSWIDL) is a popular IDL software library for solar data analysis, and in fact, many parts of sunpy are inspired by data structures and functions in SSWIDL. Though IDL and Python are very different it sometimes helps to consider how to translate simple tasks between the two languages. The primary packages which provide much of the functionality for scientific data analysis in Python are NumPy and SciPy. In the following we assume that those packages are available to you and that you are imported Numpy is imported as np with the following import statement:
import numpy as np
import scipy as sp
In the following examples, a and b could be arrays. For python the arrays must be numpy arrays which can be created simply through:
np.array(a)
where a is a python list of numbers.
Relational Operators
IDL |
Python |
|---|---|
a EQ b |
a == b |
a LT b |
a < b |
a GT b |
a > b |
a GE b |
a >= b |
a LE b |
a <= b |
a NE b |
a != b |
Logical Operators
IDL |
Python |
|---|---|
a and b |
a and b |
a or b |
a or b |
Math Functions
IDL |
Python |
|---|---|
cos(a) |
np.cos(a) |
alog(a) |
np.log(a) |
alog10(a) |
np.alog(a) |
exp(a) |
np.exp(a) |
Math Constants
IDL |
Python |
|---|---|
!pi |
np.pi |
exp(1) |
np.e |
Arrays Sequences
IDL |
Python |
|---|---|
indgen(10) |
np.arange(0,10) |
findgen(10) |
np.arange(0,10,dtype=np.float) |
Array Creation
IDL |
Python |
|---|---|
dblarr(3,5) |
np.zeros((3,5)) |
intarr(3,5) |
np.zeros((3,5),dtype=np.int) |
dblarr(3,5)+1 |
np.ones((3,5)) |
intarr(3,5)+9 |
np.zeros((3,5),dtype=np.int) + 9 |
boolarr(10) |
np.zeros(10,dtype=bool) |
identity(3) |
np.identity(3) |
Many more examples can be found on this page
Release History#
A new version of the sunpy core package is released twice a year, with target release months of May and November.
Versions x.0 are long term support (LTS) releases, supported for 12 months until the next LTS release.
Versions x.1 are non-LTS releases, supported for 6 months until the next LTS release.
Bugfix releases (versions x.y.1, x.y.2, etc.) are released monthly, with a target of the last Friday of each month.
Full Changelog#
6.0.dev521+gaabc56841 (2024-05-19)#
Breaking Changes#
Arguments for
reproject_to()after the target WCS are now keyword-only. (#7339)Arguments for
sunpy.timeseries.GenericTimeSeries.peek()are now keywords only. (#7340)
Deprecations#
sunpy.physics.differential_rotation.diff_rot()has been deprecated and replaced bysunpy.sun.models.differential_rotation(). (#7409)Deprecated all positional arguments in
sunpy.map.GenericMap.plot()method. Theannotate,axes,title,clip_intervalarguments should be passed as keyword arguments (e.g.,..., title=True, ...) instead. (#7421)The keyword
response_formatinsunpy.net.vso.VSOClient.search()has been deprecated. This was introduced to preserve legacy behaviour of the VSO client, to returnsunpy.net.vso.legacy_response.QueryResponseinstead ofsunpy.net.vso.table_response.VSOQueryResponseTableobjects. This behaviour has been the default for over 4 years and the keyword is no longer needed. This keyword and the oldersunpy.net.vso.legacy_response.QueryResponseclass will be removed in sunpy 7.0. The keywordprogressinsunpy.net.hek2vso.H2VClient.full_query()has been deprecated and will be removed in sunpy 7.0. (#7468)
Removals#
sunpy.databasehas been removed. (#7320)sunpy.map.header_helper.meta_keywordshas been removed. (#7337)sunpy.net.helioviewer.HelioviewerClienthas been removed. Use the hvpy package instead. (#7338)There was a private “Maxwell” unit within
sunpy.mapto register it before astropy had support for it. This has now been removed in favour of using the astropy version. (#7383)
New Features#
sunpy.io.read_file()will now try to detect the filetype based on the content and then fallback to using the file extension. (#6736)It is now possible to read the comments in a header from a JPEG2000 file. (#6841)
Added the ability for
sunpy.map.Mapto load files from a generator. (#7024)Added the ability to pass
clip_intervaltosunpy.map.MapSequence.plot. (#7253)Add support for the
fillkeyword indraw_contours()to allow for filled contours. (#7281)get_horizons_coord()now supports time arrays with up to 10,000 elements. (#7319)Add an example of plotting a rectangle on a map with a rotation angle relative to the axes (Drawing a rotated rectangle on a map). (#7348)
Added testing and explicit support for Python 3.12. (#7351)
Added warning when importing a submodule without installing that submodules extra dependencies. (#7369)
Added a warning message for
rsunmismatch inreproject_to()method. (#7370)Added a new optional extra group to install “opencv” if you want to it for affine transforms.
pip install sunpy[opencv] (`#7383 <https://github.com/sunpy/sunpy/pull/7383>`__)
Increased minimum versions for:
asdf >= 2.12.0
asdf-astropy >= 0.2.0
astropy >= 5.2.0
beautifulsoup4 >= 4.11.0
cdflib >= 0.4.4
dask >= 2022.5.2
h5netcdf > =1.0.0
h5py >= 3.7.0
lxml >= 4.9.0
opencv-python >= 4.6.0.66
pandas >= 1.4.0
python >= 3.10
reproject >= 0.9.0
requests >= 2.28.0
scikit-image >= 0.19.0
scipy >= 1.8.0
spiceypy >= 5.0.0
tqdm >= 4.64.0
zeep >= 4.1.0 (#7383)
sunpy.map.GenericMap.draw_contours()don’t run internal transform code iftransformkeyword is provided. (#7427)Update ASDF schemas for upcoming ASDF standard 1.6.0. (#7432)
Add a new map source
GONGHalphaMapfor GONG H-Alpha data. (#7451)Allow units to be passed to
make_fitswcs_headeras strings. (#7454)sunpy.net.jsoc.JSOCClientqueries now return the SUMS directory paths as the segment key value in the results table. (#7469)
Bug Fixes#
Long object names are no longer truncated in the logging output of
get_horizons_coord(). (#7319)When calling
sunpy.map.GenericMap.rotate()on an integer data array, withmissingset to NaN (the default value), the method will now itself raise an informative error message instead deferring to NumPy to raise the error. (#7344)Fixed the appearance of a double “Notes” heading in
Mapsubclasses. (#7376)Mapwith UINT8 data will now not error on plotting due to normalization. We now skip adding a normalization. (#7422)When calling
reproject_to()along with both context managerspropagate_with_solar_surface()andassume_spherical_screen()now raises a warning. (#7437)Fix a bug which caused
Fido.searchto crash due to SSL certificate verification error for theHECClientnow returns no results and logs a warning in this case. (#7446)Fixed the sanitization of the names of files downloaded via VSO so that periods are no longer replaced and case is no longer forced to be lowercase. (#7453)
The creation of the series string for a JSOC query was not adding the correct escape characters for comparison values for keywords. This was causing the JSOC to error. (#7467)
The EVE L0CS client now uses the new URLs for the data from LASP. (#7483)
JPEG2000 files are now saved with the correct orientation. Previously they would be vertically flipped when saved. (#7486)
Fixed a very minor inaccuracy in three
sunpy.maputility functions (contains_full_disk(),coordinate_is_on_solar_disk(), andis_all_off_disk()) resulting from the accidental use of the small-angle approximation. (#7512)The
rotate()function now correctly updates the NAXISi. (#7522)Fixed an inaccuracy in the implementation of
HeliocentricEarthEclipticandGeocentricSolarEclipticsuch that the Earth was not exactly in the XY plane, but rather had an error of up ~10 meters. (#7530)Fixed a bug with any coordinate transformation starting in
GeocentricEarthEquatorial(GEI) returning output with AU as the length unit, rather than preserving the length unit of the initial coordinate. (#7545)Fixed a bug that interfered with
astropy.wcs.utils.celestial_frame_to_wcs()when working with a custom subclass ofSunPyBaseCoordinateFrame. (#7594)Fixed bug where conversion of results from the HEKClient to Astropy Time failed when some values where empty or missing for the values of event_strattime, event_endtime or event_peaktime (#7627)
Documentation#
Added a how-to guide for manipulating grid lines on
GenericMap. (#6978)Created a how to guide on fixing metadata that is either missing or incorrect before passing the header into the
Mapclass. (#7262)Fixed the usage of
superpixel()in Resampling Maps. (#7316)Added Clarification on setting JSOC Email. (#7329)
Added explanation text to Plot positions on a blank map about the offset between “(0, 0)” in helioprojective coordinates and the heliographic equator. (#7352)
Convert draw rectangle gallery example into a how-to guide(How to draw a rectangle on a Map) (#7435)
Fix a VSO doctest due to VSO now returning level one EIT data. (#7483)
Add an example gallery entry demonstrating how to use the coordinates framework to compute intersections between instrument lines of sight and a simulation domain. (#7491)
Internal Changes#
sunpy.net.jsoc.JSOCClient.fetch()calleddrmsAPI that passed aprogresskeyword which added extra print statements to the console. This has been removed indrms0.7.0, which had breaking API changes within this release. As a result, we increased the minimum required version ofdrmsto 0.7.1.This specifically refers to the following information that was printed to the console by default:
"Export request pending. [id=X, status=X]""Waiting for X seconds...""Request not found on server, X retries left."These were handled by
drmsand are now logging messages.If you want to silence these messages, you can set the logging level to
WARNINGor higher.import logging drms_logger = logging.getLogger("drms") drms_logger.setLevel(logging.WARNING) from sunpy.net import fido, attrs
Note, you have to do it before you import
fido. (#7307)The
requestspackage is a now formally a core dependency.requestswas already commonly installed as an implied dependency ofsunpy.netor for building documentation. (#7319)The function
get_horizons_coord()no longer calls theastroquerypackage, soastroqueryis no longer a dependency. (#7319)Notifychecks that a valid email address has been given as a value. (#7342)The
delim_whitespacekeyword inpandas.read_csvis deprecated and was updated withsep='\s+'. This should have no affect on the output of the code. (#7350)Fixed an environment-specific failure of a unit test for
sunpy.coordinates.Helioprojective.is_visible(). (#7356)Moved to
pyproject.tomland removedsetup.pyandsetup.cfg. (#7384)pyerfais now a new direct dependency. It has been an indirect dependency from sunpy 3.1, over two years ago. (#7397)Increased Python minimum version to be >= 3.10. (#7402)
Fixed an unnecessary division computation when performing a unsupported division operation using a
Map. (#7551)
5.1.0 (2023-11-20)#
New Features#
Added the ability to skip over errors raised for invalid fits files when passing a list of files to map using the existing keyword argument
silence_errors. (#7018)Added a
sunpy.coordinates.Helioprojective.is_visible()method to return whether the coordinate is visible (i.e., not obscured from the observer assuming that the Sun is an opaque sphere). (#7118)Added a keyword option (
quiet) forget_body_heliographic_stonyhurst()to silence the normal reporting of the light-travel-time correction whenobserveris specified. (#7142)Added the function
sunpy.coordinates.sun.eclipse_amount()to calculate the solar-eclipse amount for an observer. (#7142)Add a keyword (
map_center_longitude) tomake_heliographic_header()for centering the heliographic map at a longitude other than zero longitude. (#7143)The minimum required version of
Glymur(an optional dependency for reading JPEG2000 files) has been increase to 0.9.1. (#7164)Added new default colormap scalings for WISPR Maps. Plots are now clipped at zero, and
AsinhStretchis used for the scaling to ensure coronal details are visible despite the much-brighter stars. Parsing of thedetectorandlevelfields of the FITS headers is also improved. (#7180)When creating a coordinate or coordinate frame without specifying
obstime, theobstimevalue from theobserverframe attribute will be used if present. (#7186)Added a GONG synoptic map class which fixes non-compliant FITS metadata (#7220)
Added the module
sunpy.coordinates.spiceto enable the use of theSkyCoordAPI to perform computations using SPICE kernels. (#7237)Added three coordinate frames that depend on the orientation of Earth’s magnetic dipole:
Geomagnetic(MAG),SolarMagnetic(SM), andGeocentricSolarMagnetospheric(GSM). (#7239)
Bug Fixes#
Fix RHESSI (
RHESSIClient) fallback server detection. (#7092)Fix bug in
get_horizons_coord()when specifying a time range via a dictionary that could cause the returned times to be slightly different from the supplied times. (#7106)Updated the url of the
GBMClientto match on files other than those that end with version 0 (i.e., V0.pha). (#7148)When directly instantiating a
WCSfrom a FITS header that contains both Stonyhurst and Carrington heliographic coordinates for the observer location, the Stonyhurst coordinates will now be prioritized. This behavior is now consistent with theMapclass, which has always prioritized Stonyhurst coordinates over Carrington coordinates. (#7188)Fixed a bug with
sample_at_coords()where sampling outside the bounds of the map would sometimes not error and instead return strange pixel values. (#7206)Improved code when loading CDF files to improve performance and avoid raising of pandas performance warnings. (#7247)
Fixed a bug with
sunpy.map.GenericMap.plot()where settingnormtoNonewould result in an error. (#7261)
Documentation#
Removed the specification of a non-identity rotation matrix in two reprojection examples. (#7114)
Added an example (Visualizing 3D stereoscopic images) for how to make an anaglyph 3D (i.e., red-cyan) image from a stereoscopic observation. (#7123)
Added an example (Obtaining solar-eclipse information) to show how to obtain information about a solar eclipse using
sunpy.coordinates.sun.eclipse_amount(). (#7142)Changed the Masking HMI based on the intensity of AIA to reproject AIA to HMI instead of the other way around. This is to avoid interpolating the HMI LOS magnetic field data. (#7160)
Fixed the timeseries peak finding example. Previously there was a bug when plotting the data with pandas. (#7199)
Added an example (Coordinates computations using SPICE kernels) for how to perform SPICE computations using the
SkyCoordAPI. (#7237)
Deprecations#
Deprecated
silence_errorsin Map and Timeseries. This has been replaced withallow_errorskeyword. (#7021)The
sunpy.coordinates.transformationsmodule is now slated for removal from the public API as it consists of internal functions used by coordinate transformations. The context managerssunpy.coordinates.transform_with_sun_center()andsunpy.coordinates.propagate_with_solar_surface()should be accessed undersunpy.coordinates. (#7113)
Removals#
sunpy.map.extract_along_coord()has been removed. Instead, usepixelate_coord_path(), and then pass its output tosample_at_coords().pixelate_coord_pathuses a different line algorithm by default, but you can specifybresenham=Trueas an argument to use the same line algorithm asextract_along_coord. (#7200)sunpy.visualisation.limb.draw_limb()has been removed. Usesunpy.visualization.drawing.limb()instead. (#7202)Removed
GenericTimeSeries.index. UseGenericTimeseries.timeinstead as a direct replacement. (#7203)Removed the deprecated
sunpy.io.cdfsubmodule, which is not intended to be user facing. (#7240)Removed the deprecated
sunpy.io.jp2, which is not intended to be user facing. (#7241)Removed the deprecated
sunpy.io.file_tools, which is not intended to be user facing. (#7242)The deprecated
sunpy.data.download_sample_data()has been removed Usesunpy.data.sample.download_all()instead. (#7250)
Internal Changes#
Removed the Binder configuration and link in README. This is because the configuration was untested, and does not currently work. (#7062)
Add a Dependabot config file to auto-update GitHub action versions. (#7068)
Add tests to check whether various
Mapmethods preserve laziness when operating on Maps backed by adask.array.Array. (#7100)Added missing support to find GOES-18 XRS data in
XRSClient. (#7108)Raise an error with a helpful message when
sunpy.map.GenericMap.plot()is called with a non-boolean value for theannotatekeyword, because the user is probably trying to specify the axes. (#7163)Fixed our ASDF manifest having the incorrect ID. (#7282)
Fix example formatting in a few asdf schemas. (#7292)
Pinned the
drmsrequirement to< 0.7to avoid breaking changes indrmsversion 0.7. (#7308)
5.0.0 (2023-06-14)#
Breaking Changes#
XRSClientnow provides the re-processed GOES-XRS 8-15 data from NOAA. These files are now all NetCDF and not FITS files. (#6737)Changed the output of
sunpy.map.sample_at_coords()to return the sampled values asQuantitywith the appropriate units instead of merely numbers. (#6882)
Deprecations#
Using
sunpy.map.header_helper.meta_keywordsis deprecated. Please see Meta Keywords for the list of metadata keywords used byMap. (#6743)The utility function
sunpy.map.extract_along_coordis deprecated. Usesunpy.map.pixelate_coord_path(), and then pass its output tosunpy.map.sample_at_coords(). (#6840)Parsing SDO/EVE level 0CS average files is deprecated, and will be removed in sunpy 6.0. Parsing this data is untested, and we cannot find a file to test it with. If you know where level 0CS ‘averages’ files can be found, please get in touch at https://community.openastronomy.org/c/sunpy/5. (#6857)
Fully deprecated
sunpy.database, with an expected removal version of sunpy 6.0. (#6869)sunpy.io.cdf,sunpy.io.file_toolsandsunpy.io.jp2sub-modules have been deprecated, and will be removed in version 5.1. This because they are designed for internal use only, and removing it from the public API gives the developers more flexibility to modify it without impacting users. (#6895)
New Features#
A pure Python
sunpywheel is now published on PyPI with each release.pipwill now default to installing the pure Python wheel instead of the source distribution on platforms other than Linux (x86-64) and macOS (x86-64 and ARM64). This should mean simpler and faster installs on such platforms, which includes the Raspberry Pi as well as some cloud computing services.This wheel does not contain the
sunpy.io.anacompiled extension. If you need this extension (not available on Windows) you can install thesunpysource distribution withpip install --no-binary sunpy "sunpy[all]". (#6175)Added three tutorials which replicate
CompositeMapfunctionality (Overlaying Two Maps, Combining off-limb and disk maps, Creating a Composite Plot with Three Maps). (#6459)exposure_timenow looks for the exposure time in theXPOSUREkey first and then theEXPTIMEkey. (#6557)make_fitswcs_headernow includes the keyword argumentdetectorfor setting theDETECTORFITS keyword in the resulting header. (#6558)Adds two tutorials that demonstrate how to use LASCO data in overlaying maps (Overlay an AIA image on a LASCO C2 coronagraph) and how to create a custom mask for a LASCO C2 image (Creating a mask for LASCO C2 data). (#6576)
Able to run the
sunpytests doingpython -m sunpy.tests.self_test. (#6600)Able to detect gzip-compressed FITS files even if they don’t have the
.gzextension in the filename.detect_filetypenow looks for the right file signature while checking for gzipped FITS files. (#6693)Added
AttrAndandAttrOrto the namespace insunpy.net.attrs. This allows users to to avoid|or&when creating a query a larger query. (#6708)SUVIClientnow provides GOES-18 SUVI data. (#6737)The minimum required versions of several core dependencies have been updated:
Python 3.9
astropy 5.0.1
numpy 1.21.0
The minimum required versions of these optional dependencies has also been updated:
Matplotlib 3.5.0
dask 2021.4.0
pandas 1.2.0
scikit-image 0.18.0
scipy 1.7.0 (#6742)
Added the utility function
sunpy.map.pixelate_coord_path()to fully pixelate a coordinate path according to the pixels of a given map. (#6840)The minimum version of h5netcdf required by sunpy has been bumped to version 0.11.0. (#6859)
Able to download files from REST/TAP Data Providers from the VSO. (#6887)
Adding data unit into html repr for
sunpy.map.Map(#6902)Joined
HISTORYkeys with newline characters when parsingHISTORYcards from FITS header. (#6911)Added the ability to query for the GOES-XRS 1 minute average data with the
XRSClient. (#6925)Increased minimum version of
parfiveto 2.0.0.We are aware the change in the
parfiveminimum version is a release earlier than our dependency policy allows for. However, due to significant issues thatparfivev2.0.0 solves and changes to remote servers, we have decided to increase it to improve the user experience when downloading files. (#6942)
Bug Fixes#
Fixed the incorrect calculation in
make_fitswcs_header()of the rotation matrix from a rotation angle when the pixels are non-square. (#6597)Return code from
self_testis now non-zero if it stops due to missing dependencies. (#6600)Fixed an issue with loading old EIT fits files with
sunpy.map.Mapwhere the date could not be parsed. (#6605)Fixed a bug where the
exposure_timereturnedNonewhen the exposure time key was set to zero. (#6637)Fixed a bug that prevented specifying a
BaseCoordinateFrame(as opposed to aSkyCoord) tosunpy.map.GenericMap.draw_quadrangle(). (#6648)HMI JPEG2000 files from Helioviewer could not be loaded due to a bug in setting the plotting normalization. This has been fixed. (#6710)
The
data_managerwas not raising failed downloads correctly and would continue as if the file existed locally. Now it will raise any errors fromparfive. (#6711)XRTMapwill now set the unit for XRT files if theBUNITkey is missing. (#6725)XRSClientupdate use the new url for which the GOES-XRS 8-15 data is provided by NOAA. (#6737)Updated
sunpy.databaseto be compatible withSQLAlchemyversions >=2.0 (#6749)When using
autoalign=Truewhen plotting maps, the result was misaligned by half a pixel. (#6796)sunpy.map.GenericMap.submap()can now handle aBaseCoordinateFrameas input. (#6820)Multi-line
HISTORYandCOMMENTkeys metadata dictionaries are now correctly split into multiple history and comment cards when writing a FITS file. (#6911)Pass in “max_splits” to Parfive to prevent multi connections to JSOC for JSOC only queries. (#6921)
When converting an
astropy.wcs.WCSobject to a solar coordinate frame theDATE-AVGkey will be used before theDATE-OBSkey, previously onlyDATE-OBSwas checked. (#6995)sunpy.map.GenericMap.rotation_matrixnow applies the default values if any FITS rotation matrix keywords are missing from the header. (#7004)Modified
sunpy.io.special.srs.read_srs()to correctly handle uppercase SRS files and supplementary sections occurring after the main data sections (I, IA, II). (#7035)
Documentation#
Added an example of how to search for multiple wavelengths attributes for AIA data using
sunpy.net.attrs.AttrOr. (#6501)Added
sunpy.map.PixelPairto the reference documentation. (#6620)Split the installation docs into a new Installation tutorial, and an installation guide. (#6639)
Added an example (Creating a TimeSeries from GOES-XRS near real time data) to download GOES NRT data and load it into
TimeSeries. (#6744)Added an example gallery (Querying and loading SHARP data) for querying SHARP data and loading it into a
Map. (#6757)Added an example (Obtaining a spacecraft trajectory from JPL Horizons) to plot the trajectory of Parker Solar Probe. (#6771)
Created a “Showcase” section of the gallery, which includes a new example (Reproducing the “Where is STEREO Today?” plot) and a relocated example (HMI Showcase: Cutout). (#6781)
Updated examples in the gallery to always explicitly create an Axes and use that for plotting, instead of using the Matplotlib pyplot API. (#6822)
Added an example (Masking HMI based on the intensity of AIA) of how to mask a HMI map based on the intensity of AIA. (#6825)
Added an example (Blending maps using mplcairo) to blend two maps using
mplcairo. (#6835)Changed the reprojecting images to different observers example (Reprojecting Images to Different Observers) to avoid using custom wcs headers where possible. (#6853)
Added a note in examples Auto-Aligning AIA and HMI Data During Plotting and Aligning AIA and HMI Data with Reproject suggesting to use
assume_spherical_screen()to retain off-disk HMI data. (#6855)Moved the Helioviewer migration guide from the tutorial to guide section of the docs. (#6868)
Moved the plotting section of the tutorial into the map section of the tutorial. (#6870)
Reorganized “Units” section of the Tutorial into smaller sections and added a section about unit equivalencies. (#6879)
Added clarifying detail (in the
TimeUTimedocstring) for how theutimetime format handles seconds on a day with a leap second. (#6894)Fixed a series of broken URLS and typos in examples and documentation strings. (#6903)
Improved the time tutorial. (#6920)
Add a “how-to” guide section to the documentation. (#6926)
Redesigned the landing page to highlight the different sections of the documentation. (#6938)
Significantly revised and improved the Maps part of the tutorial. This included moving the section on custom maps to the How-To Guides section (see How to create a sunpy Map). (#6944)
Migrated example gallery entries for searching the VSO, using
parse_time, using the data manager, and using solar constants to the how-to guide. (#6948)Reorganized some parts of the coordinates topic guide into multiple how-to guides. (#6954)
Move examples of how to create a Map from reference pages to a how-to guide. (#6977)
Cleaned up and simplified the Timeseries section of the tutorial. (#6990)
Added a topic-guide to aid understanding the role, “rsun” plays in sunpy coordinate transformations and
sunpy.map.GenericMap.reproject_to(). (#7000)Updated all of the sphinx anchors to be more consistent. This means that any use of the old anchors (intersphinx links to sunpy doc pages) will need to be updated. (#7032)
Internal Changes#
When determining which VSO servers to use for queries,
VSOClientwill now attempt to check if the cgi endpoint referenced by the WDSL file is accessible, and try the next endpoint if it can’t be reached. This should mean that a small category of connection issues with the VSO are now automatically bypassed. (#6362)
4.1.0 (2022-11-11)#
Breaking Changes#
Updated the sample data file,
AIA_171_ROLL_IMAGEto be rice compressed instead of gzip compressed. This means that the data is now stored in the second HDU. (#6221)
Deprecations#
Passing positional arguments to all
timeseriespeek()methods is now deprecated, and will raise an error in sunpy 5.1. Pass the arguments with keywords (e.g.title='my plot title') instead. (#6310)Using
sunpy.timeseries.GenericTimeSeries.index`is deprecated. Usetimeto get an astropy Time object, orts.to_dataframe().indexto get the times as a pandasDataTimeIndex. (#6327)Deprecated the
sunpy.visualization.limbmodule. Thesunpy.visualization.limb.draw_limbfunction has been moved intodrawingaslimb(). (#6332)The
sunpy.net.helioviewermodule is deprecated and will be removed in version 5.1. The Helioviewer Project now maintains a replacement Python library called hvpy. As such, in consultation with the Helioviewer Project, we have decided to deprecate theHelioviewerClientclass. (#6404)Passing the
algorithm,return_footprintarguments as positional arguments is deprecated. Pass them as keyword arguments (e.g...., return_footprint=True, ...) instead. (#6406)sunpy.data.download_sample_data()is now deprecated. Usesunpy.data.sample.download_all()instead. (#6426)The sunpy.database module is no longer actively maintained and has a number of outstanding issues. It is anticipated that sunpy.database will be formally deprecated in sunpy 5.0 and removed in sunpy 6.0. If you are using sunpy.database and would like to see a replacement, please join the discussion thread at https://community.openastronomy.org/t/deprecating-sunpy-database/495. (#6498)
Removals#
The
sunpy.io.fitssub-module has been removed, as it was designed for internal use. Use theastropy.io.fitsmodule instead for more generic functionality to read FITS files. (#6432)The
sunpy.physics.solar_rotationsub-module has been removed, having been moved tosunkit_image.coalignment. (#6433)Most of the
sunpy.visualization.animatorsubpackage has been removed, with the exception ofMapSequenceAnimatorIt has been moved into the standalone mpl-animators package Please update your imports to replacesunpy.visualization.animatorwithmpl_animators. (#6434)Remove
GenericMap.shiftmethod and theGenericMap.shifted_value. Useshift_reference_coordinstead. (#6437)sunpy.util.scraperhas been removed. Usesunpy.net.scraperinstead. (#6438)sunpy.image.coalignmenthas been removed. Usesunkit_image.coalignmentinstead, which contains all the same functionality. (#6440)sunpy.map.GenericMap.draw_limb()can no longer be used to draw the limb on a non-WCS Axes plot. (#6533)sunpy.image.resample()no longer accepts “neighbour” as an interpolation method. Use “nearest” instead. (#6537)sunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate()no longer accepts theuse_scipykeyword. (#6538)
New Features#
Updated and expanded the HTML representation for
TimeSeries. (#5951)When reading CDF files, any columns with a floating point data type now have their masked values converted to NaN. (#5956)
Add support for saving
GenericMapas JPEG 2000 files. (#6153)Add a function
sunpy.map.extract_along_coordthat, for a given set of coordinates, finds each array index that crosses the line traced by those coordinates and returns the value of the data array of a given map at those array indices. (#6189)Three new maps have been added to the sample data from STEREO A and STEREO B at 195 Angstrom, and AIA at 193 Angstrom. These images are from a time when the three spacecraft were equally spaced around the Sun, and therefore form near complete instantaneous coverage of the solar surface.
Users upgrading to this version will find this three files download when they use the sample data for the first time. (#6197)
Added a SDO/AIA 1600 file of the Venus transit to the sunpy sample data. (#6242)
Created the
sunpy.visualization.drawingmodule which includes newequator()andprime_meridian()functions. (#6251)Expose GOES quality flags in order to allow filtering corrupt values when using the
XRSTimeSeries. (#6260)All TimeSeries plotting methods now consistently set the same formatter and locator for the x-axis. (#6264)
sunpy.timeseries.GenericTimeSeries.peek()now takes atitleargument to set the title of the plot. (#6304)Added the
sunpy.timeseries.GenericTimeSeries.timeproperty to get the times of a timeseries as aTimeobject. (#6327)Added the Plotting the solar equator and prime meridian example to the Example Gallery. (#6332)
Added a new function
sunpy.map.header_helper.make_heliographic_header()to help with generating FITS-WCS headers in Carrington or Stonyhurst coordinate systems that span the entire solar surface. (#6415)Sample data files provided through
sunpy.data.sampleare now downloaded individually on demand rather than being all downloaded upon import of that module. To download all sample data files, callsunpy.data.sample.download_all(). (#6426)XRSTimeSeriesis now able to parse the primary detector information from the GOES-R XRS data if available. (#6454)sunpy.net.Scrapernow includes treats files as spanning a full interval equal to the smallest increment specified in the file pattern. For example, a pattern like"%Y.txt"that only contains a year specifier will be considered to span that full year.This means searches that fall entirely within the whole interval spanned by a pattern will return that file, where previously they did not. As an example, matching
"%Y.txt"withTimeRange('2022-02-01', '2022-04-01')will now return["2022.txt"]where previously no files were returned. (#6472)Implemented site configuration for sunpyrc, and modified documentation for sunpy customization. (#6478)
make_fitswcs_header()now includes the keyword argumentunitfor setting theBUNITFITS keyword in the resulting header. This will take precedence over any unit information attached todata. (#6499)If the
dataargument tomake_fitswcs_header()is anQuantity, the associated unit will be used to set theBUNITFITS keyword in the resulting header. (#6499)Added a 304 sample data file called
AIA_304_IMAGE. (#6546)
Bug Fixes#
Fix a bug that prevented EUI maps with missing wavelength metadata loading. (#6199)
The
sunpy.net.dataretriever.sources.noaa.SRSClientwas not correctly setting the passive mode for FTP connection resulting in a permission error. This has been fixed. (#6256)Fixed
XRSTimeSeriesinability to read leap-second files for GOES. It floors the leap-second timestamp to be59.999, so that Python datetime does not raise an exception. (#6262)Changed the default scaling for
EUIMapfrom a linear stretch to a asinh stretch.To revert to the previous linear stretch do the following:
from astropy.visualization import ImageNormalize, LinearStretch euimap.plot_settings["norm"] = ImageNormalize(stretch=LinearStretch()) (`#6285 <https://github.com/sunpy/sunpy/pull/6285>`__)
Fixed bugs when working with a coordinate frame where the observer is specified in
HeliographicStonyhurstwith a Cartesian representation, which is equivalent to Heliocentric Earth Equatorial (HEEQ). Now, the observer will always be converted to spherical representation when the coordinate frame is created. (#6311)Fixed an error when Fido returns zero results from the VSO and some results from at least one other data source. This (now fixed) error is only present when using numpy version >= 1.23. (#6318)
If a level 1 XRT file does not specify the heliographic longitude of the spacecraft, a silent assumption is made that the spacecraft is at zero Stonyhurst heliographic longitude (i.e., the same longitude as Earth). (#6333)
The sample data retry was failing under parfive 2.0.0. (#6334)
Fixed bug that prevented
RotatedSunFrameinstances from being pickled. (#6342)Fix a bug in loading
XRSTimeSeriesdue to unsupported quality flag column names. (#6410)Adds units (dimensionless units) to the quality columns in
XRSTimeSeries. (#6423)Refactored
SXTMapto use ITRS observer coordinate information in header rather than incorrect HGS keywords. TheSXTMapalso now uses the defaultdsunproperty as this information can be derived from the (now corrected) observer coordinate. (#6436)In
sunpy.map.GenericMap.coordinate_systemandsunpy.map.GenericMap.date, the default values will now be used if the expected key(s) used to derive those properties are empty. Previously, empty values of these keys were not treated as missing and thus the default values were not correctly filled in. (#6436)Fixed a bug where the observer coordinate was incorrectly determined for
KCorMap. (#6447)Trying to download an empty search response from the JSOC now results in an empty results object. Previously the results object contained the path to the sunpy download directory. (#6449)
Removed an error when searching CDAWEB using
sunpy.net.Fidoand no results are returned. An empty response table is now returned. (#6450)Fix a bug to parse the GOES “observatory” number in
XRSTimeSeriesfor GOES 13, 14, 15 and for the 1 minute GOES-R data. (#6451)Changed the default scaling for
XRTMapfrom a linear stretch toLogStretch.To revert to the previous linear stretch do the following:
from astropy.visualization import ImageNormalize, LinearStretch xrtmap.plot_settings["norm"] = ImageNormalize(stretch=LinearStretch()) (`#6480 <https://github.com/sunpy/sunpy/pull/6480>`__)
Fix the
detectorproperty ofSOTMapto return “SOT”. (#6480)The right-hand y-axis of the GOES-XRS timeseries plots with labelled flare classes now automatically scales with the left-hand y-axis. (#6486)
Add support for Python 3.11.
The deprecated
cgi.parse_headeris now available assunpy.util.net.parse_header. (#6512)Fixed the metadata handling of
resample()andsuperpixel()so that the CDELTi values are scaled and the PCi_j matrix (if used) is modified in the correct manner for asymmetric scaling. The previous approach of having the PCi_j matrix store all of the scaling resulted in non-intuitive behaviors when accessing thescaleandrotation_matrixproperties, and when de-rotating a map viarotate(). (#6571)Fixd a bug with the
sunpy.map.GenericMap.scaleproperty for maps containing only the CDij matrix where the scale was not being determined from the CDij matrix. (#6573)Fixed a bug with the
sunpy.map.GenericMap.rotation_matrixproperty for maps using the CDij matrix formulism where the rotation matrix would be calculated incorrectly for non-square pixels. (#6573)Fixed a bug where
parse_time()would always disregard the remainder of a time string starting with the final period if it was followed by only zeros, which could affect the parsing of the time string. (#6581)
Documentation#
Improved annotations in the SRS active regions plotting example. (#6196)
Updated gallery examples that use STEREO data to use sample data instead of searching for and downloading data via Fido. (#6197)
Added the current bugfix release policy to the docs. (#6336)
The Maps and Timeseries have been reviewed and updated. (#6345)
Adds a pull request check list to the Developer’s Guide. (#6346)
Improved the plotting guide. (#6430)
Slight improvements to the downloading data with Fido part of the guide. (#6444)
Split the units and coordinate guides on to separate pages, and made minor improvements to them. (#6462)
Added a how-to guide
conda_for_dependenciesfor usingcondato set up an environment with the complete set of dependencies to use all optional features, build the documentation, and/or run the full test suite. The guide also describes how best to have an editable installation ofsunpyin this environment. (#6524)
Internal Changes#
Added a
columnskeyword to each plot method for allsunpy.timeseries.GenericTimeSeriessources. (#6056)Added a script in the
sunpy/toolsthat will update all the Python libraries insunpy/extern. (#6127)Added automatic conversion of unit strings in CDF files to astropy unit objects for the following instruments: PSP/ISOIS, SOHO/CELIAS, SOHO/COSTEP-EPHIN, and SOHO/ERNE. (#6159)
Add an environment variable
SUNPY_NO_BUILD_ANA_EXTENSIONwhich when present will cause sunpy to not compile the ANA C extension when building from source. (#6166)sunpynow uses the Limited Python API. Therefore, one binary distribution (wheel) per platform is now published and it is compatible with all Python versionssunpysupports. (#6171)Add support for upcoming parfive 2.0 release. (#6243)
The primary sample-data URL will be changing from
https://github.com/sunpy/sample-data/raw/master/sunpy/v1/tohttps://github.com/sunpy/data/raw/main/sunpy/v1/. We expect GitHub to redirect from the old URL for sometime but will eventually expire it. Thedata.sunpy.orgmirror will continue to be available. (#6289)Add support for downloading sample data from more than two mirror locations. (#6295)
Timeseries data sources can now set the
_peek_titleclass attribute to set the default plot title produced when.peek()is called and the user does not provide a custom title. (#6304)Add maintainer documentation on the backport bot (#6355)
Switched to using the standard matrix-multiplication operator (available in Python 3.5+) instead of a custom function. (#6376)
Fixed a colormap deprecation warning when importing the sunpy colormaps with Matplotlib 3.6. (#6379)
Removed custom tick label rotation from Lyra, EVE, and Norh timeseries sources, and grid drawing from NOAA and RHESSI sources. (#6385)
Fixed a bug where the private attribute
_default_observer_coordinateforGenericMapwas being used even when there was sufficient observer metadata in the header. (#6447)Tidy the GOES XRSTimesSeries tests and add two new XRS files to test. (#6460)
Added a pre-commit hook for codespell, and applied spelling fixes throughout the package. (#6574)
v4.0.0 (2022-05-06)#
Breaking Changes#
When rotating images using the SciPy rotation method, the default behavior is now to clip the output range to the input range, which matches the default behavior of the scikit-image rotation method. (#5867)
Any NaNs are now preserved by
sunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate(). (#5867)sunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate()now default to using SciPy for rotation instead of scikit-image, so rotation results may be slightly different. (#5867)The math convenience methods of
sunpy.map.GenericMap-max(),mean(),min(), and ,std()- now ignore NaNs in the image data. (#5867)sunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate()now default to using NaN instead of zero for themissingvalue, the value used for pixels in the output array that have no corresponding pixel in the input array. To obtain the previous behavior,missingshould be explicitly specified as zero. (#5867)The
JSOCClientand everysunpy.net.dataretriever.GenericClientwas passing all**kwargstoparfive.Downloader.enqueue_file, this was unintended and has been removed. (#6052)Changed the default interpolation order for
sunpy.map.GenericMap.rotate()from 4 to 3, with the precise meaning of these interpolation orders depending on the selected rotation method. For the default rotation method, which usesscipy.ndimage.affine_transform(), this changes the default interpolation from biquartic to bicubic, which reduces the computation time without reducing the quality of the output below what a typical user needs. (#6089)
Deprecations#
Deprecate
sunpy.image.coalignmentas the code has now been moved tosunkit_image.coalignmentwith an identical API. This module will be removed in sunpy 4.1. (#5957)The
sunpy.map.GenericMap.shiftmethod has been renamed tosunpy.map.GenericMap.shift_reference_coordandshifthas been deprecated. (#5977)The
sunpy.map.GenericMap.shifted_valueproperty has been deprecated. Modifications to the reference coordinate can be found in theCRVAL1andCRVAL2keys ofsunpy.map.GenericMap.meta.modified_items. (#5977)The
sunpy.io.fitsmodule is deprecated, as it was designed for internal use only. Use theastropy.io.fitsmodule instead for more generic functionality to read FITS files. (#5983)sunpy.physics.solar_rotation.mapsequence_solar_derotateis deprecated and will be removed in version 4.1. This function has been moved tosunkit_image.coalignment.mapsequence_coalign_by_rotationand has an identical API and functionality. (#6031)sunpy.physics.solar_rotation.calculate_solar_rotate_shiftis deprecated and will be removed in version 4.1. This function has been moved tosunkit_image.coalignment.calculate_solar_rotate_shiftand has an identical API and functionality. (#6031)Deprecated using
sunpy.map.GenericMap.draw_limbon an Axes that is not a WCSAxes. (#6079)
New Features#
Added support for Python 3.10 (#5568)
Added support for
"%Y.%m.%d_%H:%M:%S_UTC"and"%Y.%m.%d_%H:%M:%S"time formats insunpy.time.parse_time. (#5647)The
rsunargument toget_observer_meta()is now optional. (#5655)Added the
total_size(), which estimates the total size of the results from a Fido query. If this is supported by a client, the total size is printed alongside the results.To add support for this in external clients, make sure one column contains the individual filesizes as
Quantity, and set thesize_columnclass attribute to the name of this column. (#5659)Added the ability to specify the use of Carrington coordinates with
sunpy.map.GenericMap.draw_grid(). (#5703)Printing a
MetaDictwill now show each entry on a new line. (#5765)Removed support for Python 3.7. (#5773)
The ‘event_endtime’, ‘event_starttime’ and ‘event_peaktime’ columns in a HEK query are now returned as
Timeobjects. Previously they were timestamp strings. (#5806)Added a helpful warning message when converting a 2D Helioprojective coordinate will return all NaNs. (#5817)
The colorbar limits on HMI magnetic field maps are now automatically set to be symmetric about zero. (#5825)
Added a
clipkeyword tosunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate()to enable or disable whether the range of the output image is clipped to the range of the input range. (#5867)Created the decorator
sunpy.image.transform.add_rotation_function()for registering new rotation functions for use bysunpy.image.transform.affine_transform()andsunpy.map.GenericMap.rotate(). (#5867)sunpy.image.transform.affine_transformandsunpy.map.GenericMap.rotate()have both had theiruse_scipyarguments deprecated. Instead use the newmethodargument to select from the available rotation methods. (#5916)Added a Maxwell unit and any places where a conversion to Gauss occurs has been removed. (#5998)
Add a basic HTML representation for
TimeSeries. (#6032)The minimum supported asdf version has been increased to 2.8.0 to allow future compatibility with the breaking changes planned for asdf 3.0. In addition to this the asdf-astropy package is now required to serialise and deserialise the sunpy coordinate frame classes to ASDF. (#6057)
Added the option to rotate using OpenCV when using
sunpy.image.transform.affine_transform()orsunpy.map.GenericMap.rotate()by specifyingmethod='cv2'. The OpenCV Python package must be installed on the system. (#6089)
Bug Fixes#
Fixed reading CDF files when a column has no entries. If this is the case the column will be ignored, and a message logged at DEBUG level. (#5664)
Fixed the units of
sunpy.map.sources.HMISynopticMap.scaleandsunpy.map.sources.MDISynopticMap.scale. (#5682)Fixed a bug where custom values in the
plot_settingsdictionary were not being propagated to new map instances created when calling map methods (e.g..submap). (#5687)Added automatic conversion of some common but non-standard unit strings in CDF files to astropy unit objects. If sunpy does not recognise the unit string for a particular column, units of
u.dimensionless_unscaledare applied to that column and a warning raised.If you think a given unit should not be dimensionless and support should be added for it in sunpy, please raise an issue at sunpy/sunpy#issues. (#5692)
The default
id_typeinsunpy.coordinates.get_horizons_coord()is nowNoneto match the defaultid_typein astroquery 0.4.4, which will search major bodies first, and if no major bodies are found, then search small bodies. For older versions of astroquery the defaultid_typeused byget_horizons_coord()is still'majorbody'. (#5707)In consultation with JSOC, we now limit all JSOC downloads to one connection. This will override all connection user settings passed to the downloader. (#5714)
Updated the
plotmethods on some timeseries classes to correctly label and format the time axis. (#5720)Fixed a long-standing bug where our logger could intercept Astropy warnings in addition to SunPy warnings, and thus could conflict with Astropy’s logger. (#5722)
Update asdf schemas so that references use URIs not tags as this is not supported by the new asdf extensions API. (#5723)
Increased the default maximum amount of records returned from HEC to 500 from 10. If the maximum number of records are returned, a message is shown. (#5738)
Reading a series of CDF files where at least one of them is empty no longer raises an error. A message for each empty file is logged at the DEBUG level. (#5751)
sunpy.map.header_helper.make_fitswcs_header()now includes a PC_ij matrix in the returned header if no rotation is specified. (#5763)In the case where a map header has no PC_ij values, CROTA2 != 0, and CDELT1 != CDELT2, the calculation of the map rotation matrix has been fixed. This bug only affected maps with non-zero rotation, no PC matrix in the header, and un-equal scales along the two image axes. (#5766)
Maps created from
resample()andsuperpixel()have been fixed in the case where the resampling was not square, and the PCi_j matrix (often a rotation matrix) was not a multiple of the identity matrix. When the PCi_j or CDi_j formalisms are used in the metadata these are now correctly modified, and the CDELT values are left unchanged. (#5786)The
__repr__of severalsunpy.databaseclasses have been updated to remove angular brackets and add equals signs. As an example,'<DatabaseEntry(id 3)>'has changed to'DatabaseEntry(id=3)'(#5790)Fixed a bug when rotating a map by a matrix that is not purely a rotation. The likely way to inadvertently encounter this bug was when de-rotating a map with rectangular pixels that were not aligned with the coordinate axes. (#5803)
Fixed a bug where rotating a map while simultaneously scaling it could result in some of the map data being cropped out. (#5803)
Symmetric colorbar limits are no longer set on intensity images from MDI. (#5825)
Fixed plotting and peeking NORH timeseries data with
pandas1.4.0. (#5830)In the case where
sunpy.database.Database.fetch()successfully downloads only some of the search results, asunpy.database.PartialFetchErroris raised. This fixes a bug where the successful downloads would have been added to the database, but sometimes with incorrect metadata. (#5835)When getting IRIS files from the VSO, Fido was incorrectly labelling them as XML files. (#5868)
HMIMapnow looks for'INSTRUME'instead of'TELESCOP'in order to support Helioviewer JPEG2000 versions of HMI data which do not preserve the'TELESCOP'keyword as expected in the JSOC standard. (#5886)Fixes a bug where the
cmapandnormkeyword arguments were ignored when callingplot. (#5889)Fix parsing of the GOES/XRS netcdf files to ignore leap seconds. (#5915)
Fixed compatibility with
h5netcdf>0.14when loading GOES netcdf files. (#5920)Fixed bugs with the rebinning and per-keV calculation for Fermi/GBM summary lightcurves (
GBMSummaryTimeSeries). (#5943)Fixed the unintentionally slow parsing of Fermi/GBM files (
GBMSummaryTimeSeries). (#5943)Fixes a bug in
SJIMapwhere undefined variable was used when parsing the wavelength. Also fixes the unit parsing by removing the “corrected” string from theBUNITkeyword as “corrected DN” cannot be parsed as a valid FITS unit. (#5968)Fixed unit handling issue with
GenericMapand lowercasing the unit before it submits it toastropy.units. (#5970)Fixed reading CDF files when a variable has more than 2 dimensions. If this is the case the variable will be ignored, and a user warning is provided. (#5975)
Fixed
sunpy.system_infoso it returns the extra group when an optional dependency is missing. (#6011)Relax condition check for a HMI Synoptic map source. (#6018)
VSOClientwas not passing**kwargsthrough each download method. (#6052)Fixed the inability to rotate images and maps with byte ordering that is different from the native byte order of the system (e.g., big-endian values on a little-endian system) for certain interpolation orders when internally using
scikit-image. (#6064)Fixed a crash for dask arrays when displaying the
GenericMaphtml representation. (#6088)Constructing the color map name for a
KCorMapno longer requires the “detector” key in the metadata. This allows for reading files that are missing this keyword, as in the KCor JPEG2000 files. (#6112)We now correctly pass keyword arguments in our internal FITS reader to
astropy.io.fits.open. (#6123)
Documentation#
Fixed various plotting issues with the gallery example Drawing the AIA limb on a STEREO EUVI image. (#5534)
Improved the gallery example AIA to STEREO coordinate conversion to better illustrate how coordinate transformations interact with submaps and coordinate plotting. (#5534)
Tidy the API Reference section of the documentation and improve the landing page for the docs. (#5623)
Add info about loading CDF files to the API documentation. (#5735)
Added a known issues entry about
scikit-imagepackage version pinning. (#5865)Edited entries in the example gallery to have a consistent plotting style. Added said style guidelines to the example gallery page in the dev guide. (#5870)
Added the gallery example Reprojecting to a Map Projection with a Custom Origin, which specifically showcases the azimuthal equidistant projection (also known as the Postel projection). (#5961)
Remove the part of the
SJIMapdocstring that says it only works on L1 as the data work for L2 and the level checking was not being enforced. (#5968)Updated the timeseries documentation to make it clear that you can pass in a numpy array. (#6024)
Internal Changes#
Sped up the parsing of results from the VSO. For large queries this significantly reduces the time needed to perform a query to the VSO. (#5681)
sunpy.map.GenericMap.wcsnow checks that the scale property has the correct units whilst constructing the WCS. (#5682)Added packaging as a core dependency as distutils is now deprecated. (#5713)
SunpyWarningis no longer a subclass ofAstropyWarning. (#5722)Running the tests now requires the
pytest-xdistpackage. By default tests are not run in parallel, but can be configured to do so usingpytest-xdistcommand line options. (#5827)Migrate the asdf infrastructure to the new style converters etc added in asdf 2.8.0. This makes sure sunpy will be compatible with the upcoming asdf 3.0 release. (#6057)
Declare in our dependencies that we are not compatible with asdf 3.0.0 until we are. (#6077)
Improved performance of the code that parses dates in clients that use the
Scraperto get available files. (#6101)
3.1.0 (2021-10-29)#
Breaking Changes#
sunpy.timeseries.sources.NOAAIndicesTimeSeries.peek()acceptsplot_typeas an argument instead oftype. (#5200)Fill values are now set to
numpy.naninsunpy.timeseries.sources.noaafile parsers. They were previously set to a fill value of-1. (#5363)sunpy.map.GenericMap.datenow looks for more metadata than just DATE-OBS, using new FITS keywords defined in version 4 of the standard.sunpy.map.GenericMap.datenow returns, in order of preference:The DATE-OBS FITS keyword
The current time.
If DATE-OBS is present alongside DATE-AVG or DATE-BEG and DATE-END, this results in a behaviour change to favour the new (more precisely defined) keywords. It is recommended to use
date_average,date_start, ordate_endinstead if you need one of these specific times. (#5449)sunpy.io.fits.get_headerno longer automatically tries to add the WAVEUNIT keyword if it isn’t present in the header. To replicate the original behaviour do:header = sunpy.io.fits.get_header(...) waveunit = sunpy.io.fits.extract_waveunit(header) if waveunit is not None: header['WAVEUNIT'] = waveunit
The
sunpy.map.GenericMap.waveunitproperty still usessunpy.io.fits.extract_waveunit`to try and get the waveunit if the WAVEUNIT key isn’t present. (#5501)sunpy.map.GenericMap.wcsno longer passes the whole.metadictionary toastropy.wcs.WCSwhen constructing.wcs. Instead each metadata value is manually taken from various map properties, which allows fixes to be made to the WCS without modifying the original map header. We think thatwcscorrectly sets all the keys needed for a full WCS header, but if you find anything missing please open an issue on the sunpy issue tracker. (#5501)
Deprecations#
sunpy.util.scraper.Scraperhas been moved intosunpy.net, please update your imports to befrom sunpy.net import Scraper. (#5364)Using “neighbour” as a resampling method in
sunpy.image.resample.resample()is deprecated. Use “nearest” instead, which has the same effect. (#5480)The
sunpy.visualization.animatorsubpackage has been spun out into the standalone mpl-animators package, with the exception ofMapSequenceAnimator. Please update your imports to replacesunpy.visualization.animatorwithmpl_animators.This is primarily because the
ndcubepackage now relies on the animator classes as well assunpy. (#5619)
Removals#
The deprecated
sunpy.roi.chaincode.Chaincodehas been removed in favour ofsunpy.net.helio.Chaincode. (#5304)The deprecated
sunpy.roi.roiwas removed, there is no direct replacement but astropy-regions is something to consider. (#5304)The deprecated
sunpy.instrhas been removed, please use sunkit_instruments. (#5304)The deprecated
sunpy.map.GenericMap.sizehas been removed, please usesunpy.map.GenericMap.data.size. (#5304)The deprecated ability to read txt files from
sunpy.timeseries.sources.noaa.NOAAIndicesTimeSeriesandsunpy.timeseries.sources.noaa.NOAAPredictIndicesTimeSerieshas been removed as the data provided by NOAA is now provided as JSON files. (#5304)Removed various deprecated methods on our Fido clients and responses:
UnifiedResponse.build_table,UnifiedResponse.tables,UnifiedResponse.responses,UnifiedResponse.get_responseandUnifiedResponse.blocksasUnifiedResponseis now anastropy.table.Tablethat is sliceable.UnifiedResponse.response_block_propertiesasUnifiedResponse.path_format_keyswas added as a better replacement.HECClient.time_queryas you can now useFido.searchdirectly.sunpy.net.jsoc.attrs.Keyswas not used for querying JSOC.sunpy.net.jsoc.JSOCClient.search_metadataas the functionality this provided was merged intosunpy.net.jsoc.JSOCClient.search.sunpy.net.vso.VSOClient.linkas better search support in the client replaces this method. (#5304)
The deprecated
sunpy.map.GenericMap.draw_rectangle()has been removed, the replacement issunpy.map.GenericMap.draw_quadrangle()(#5304)sunpy now errors if the unused
.rsunor.heliographic_observerattributes are set on aWCS. (#5348)Support for passing non-unit levels to
sunpy.map.GenericMap.draw_contours()when map data has units set has been removed, and with now raise an error. (#5352)The
originargument tosunpy.map.GenericMap.world_to_pixel()andsunpy.map.GenericMap.pixel_to_world()has been removed. (#5353)Support for plotting or contouring
GenericMapon axes that are notWCSAxeshas been removed. To create aWCSAxes, use theprojectionargument when the axes is created, e.g.fig.add_subplot(111, projection=my_map). (#5354)The following search attributes in
sunpy.net.vso.attrshave been removed:['Time', 'Instrument', 'Wavelength', 'Source', 'Provider', 'Level', 'Sample', 'Detector', 'Resolution', 'Physobs']. Use the equivalent attribute fromsunpy.net.attrsinstead. (#5355)The default response format from the VSO client is now a table. (#5355)
sunpy.net.hek.attrs.Timehas been removed, usesunpy.net.attrs.Timeinstead. (#5355)
New Features#
Ensured that
plotandpeekwill output the same figures for allsunpy.timeseries.TimeSeriessources. (#5200)Added hook file and tests for using PyInstaller with sunpy. (#5224)
Allows
sunpy.map.GenericMap.draw_quadrangle()to accept pixel units as input to enable plotting boxes in the pixel space of the map, which can be different from the plot axes. (#5275)Added the
propagate_with_solar_surface()context manager for transformations, which will automatically apply solar differential rotation when transforming a coordinate between frames with a change in time (obstime). (#5281)Add support for parsing the observer location from a
WCSobject when using the ‘OBSGEO’ formulation. This is the recommended way to define the observer location of a ground based observer. (#5315)Added a new function,
sunpy.visualization.draw_limb, that draws the solar limb as seen from an arbitrary observer coordinate on a world coordinate system aware Axes. (#5414)sunpy.map.GenericMap.rsun_metersnow usessunpy.map.GenericMap.rsun_obsas a fallback to calculate the assumed radius of emission if RSUN_REF metadata isn’t present but metadata forrsun_obsis. (#5416)Added
sunpy.coordinates.utils.get_limb_coordinates()to get the solar limb coordinates as seen from a given observer. (#5417)Printing the response from a
Fidoquery now includes the URL where the data files are sourced from.If you develop a third-party
Fidoclient, support for this can be automatically enabled by adding ainfo_urlproperty to yourBaseClientthat returns a URL as a string. (#5431)TimeSeriescan now read CDF files that conform to theISTP/IACG guidelines (https://spdf.gsfc.nasa.gov/sp_use_of_cdf.html). (#5435)
The properties
date_start,date_end, anddate_averagehave been added to be drawn from the relevant FITS metadata, if present in the map header. (#5449)Add default color map and normalization for
HMISynopticMapThe default color map is ‘hmimag’ and the default normalization is linear between -1.5e-3 and +1.5e3, the expected normalization for this particular color map. (#5464)The headers produced by
make_fitswcs_header()now includeNAXIS,NAXIS1, andNAXIS2keywords. (#5470)The
WCSinstance returned by thesunpy.map.GenericMap.wcsproperty now includes the shape of the data array. (#5470)Added the method
sunpy.map.GenericMap.reproject_to()for reprojecting aMapto a different WCS. This method requires the optional packagereprojectto be installed. (#5470)Registered the time format
tai_secondsforastropy.time.Time(viaTimeTaiSeconds) to support parsing the numerical time format of TAI seconds since 1958-01-01 00:00:00. This format includes UTC leap seconds, and enables equivalent functionality to theanytim2tairoutine in SSW. (#5489)Added
sunpy.map.sources.WISPRMapas a map source for WISPR on Parker Solar Probe. This improves thenameof the map and adds correct information for theprocessing_levelandexposure_time. (#5502)sunpy.io.fits.writecan now update thedataandheaderof an existing HDU instance, as an alternative to creating a new instance of a specified HDU type. This adds support for writing a HDU (such asCompImageHDU) initialised with non-default keyword arguments. (#5503)Added
observatoryto provide observatory information for the timeseries e.g. specific goes satellite number. (#5556)sunpy.timeseries.GenericTimeSeries.plot()andsunpy.timeseries.GenericTimeSeries.peek()will now automatically label the y-axis if all the columns being plotted have the same units. (#5557)sunpy.timeseries.GenericTimeSeries.plot()andsunpy.timeseries.GenericTimeSeries.peek()now have an optioncolumnsthat allows plotting a subset of the columns present. (#5557)Added a new CDAWeb client, along with helper utilities to
sunpy.net.cdaweb. (#5558)Support for filtering searches with JSOC keywords has been added to
Fido.search. (#5566)Added support for arithmetic operations between`~sunpy.map.GenericMap` and array-like objects. (#5614)
Added
quantityattribute toGenericMapto expose thedataattribute as aQuantityusing theunitattribute. (#5614)
Bug Fixes#
sunpy.map.GenericMap.superpixel()now keeps the reference coordinate of the WCS projection the same as the input map, and updates the reference pixel accordingly. This fixes inconsistencies in the input and output world coordinate systems when a non-linear projection is used. (#5295)Inputs to the
dimensionsandoffsetarguments tosunpy.map.GenericMap.superpixel()in units other thanu.pix(e.g.`u.kpix) are now handled correctly. (#5301)Fractional inputs to the
dimensionsandoffsetarguments tosunpy.map.GenericMap.superpixel()were previously rounded usingintin the superpixel algorithm, but not assigned integer values in the new metadata. This has now been changed so the rounding is correctly reflected in the metadata. (#5301)Remove runtime use of
astropy.tests.helper.assert_quantity_allclosewhich introduces a runtime dependency onpytest. (#5305)sunpy.map.GenericMap.resample()now keeps the reference coordinate of the WCS projection the same as the input map, and updates the reference pixel accordingly. This fixes inconsistencies in the input and output world coordinate systems when a non-linear projection is used. (#5309)Fix saving
GenericMapto an asdf file with version 2.8.0 of the asdf package. (#5342)When the limb is entirely visible,
sunpy.map.GenericMap.draw_limb()no longer plots an invisible patch for the hidden part of the limb and now returnsNoneinstead of the invisible patch. Similarly, when the limb is entirely invisible, no patch is drawn for the visible part andNoneis returned instead of the visible patch. (#5414)sunpy.map.GenericMap.plot()now correctly sets axis labels based on the coordinate system of the axes, and not the coordinate system of the map being plotted. This was previously only an issue if usingautoalign=Truewhen the Map coordinate system was different to the axes coordinate system. (#5432)sunpy.map.GenericMap.plot()no longer adds a unit string to the axis labels if the axes being plotted on is a WCSAxes. For a WCSAxes, angular units are indicated in the tick labels, and automatically change when the zoom level changes from e.g. degrees to arc-minutes. This could previously lead to situations where the axis label units were incorrect. (#5432)Implement automatic fallback to helioviewer mirrors if API is non-functional. (#5440)
Fixed the incorrect value for the FITS WCS
LONPOLEkeyword when usingmake_fitswcs_header()for certain combinations of WCS projection and reference coordinate. (#5448)The date returned by
datefor Solar Orbiter/EUI maps has been adjusted to be taken from the DATE-AVG keyword (the middle of the image acquisition period), instead of the DATE-OBS keyword (the beginning of the image acquisition period). This means the observer coordinate now has the correct date. (#5462)The
.unitattribute for HMI synoptic maps has been fixed. (#5467)When “TAI” is in the date string,
sunpy.map.GenericMap.datenow only raises a warning if the TIMESYS keyword is present and different to “TAI”. Previously a warning was raised all the time when “TAI” was in the date string. (#5468)Fixed a bug where the property
sunpy.map.GenericMap.rsun_meterswould always internally determine the observer location, even when it is not needed, particularly for Stonyhurst heliographic maps, which have no notion of an observer. Thus, when working with a Stonyhurst heliographic map, a user could get an irrelevant warning message about having to assume an observer location (Earth center). (#5478)Fixed the unintended insertion of (assumed) observer location information when accessing the property
sunpy.map.GenericMap.wcsfor Stonyhurst heliographic maps. (#5478)Fixed an incorrect value for the FITS WCS
LONPOLEkeyword when usingmake_fitswcs_header()forHelioprojectivemaps with certain values of latitude for the reference coordinate. (#5490)A non-standard
CROTAkeyword included in asunpy.map.sources.EUIMapFITS header is now renamed to the recommendedCROTA2so a warning is no longer raised. (#5493)The plotting x-limits of
sunpy.timeseries.sources.NOAAIndicesTimeSeries.plot()are now adjusted to only include finite points in the timeseries data. (#5496)The Hinode/XRT map source now corrects the TIMESYS keyword, fixing the
.wcsproperty that was previously broken for Hinode/XRT maps. (#5508)Updated
sunpy.map.CompositeMap.plotto support thelinestylesandcolorsarguments, in addition to the existinglinewidthsargument. (#5521)Fixed a bug where rotating a
Mapcould result in an extremely small shift (at the numerical-precision level) in the mapping from world coordinates to pixels. (#5553)Fixed a bug where rotating a
Mapthat is missing observation-time metadata could result in an incorrect reference coordinate. (#5553)Fix a bug where saving a helioprojective or heliocentric coordinate to an asdf file didn’t work due to a schema version mismatch if the observer location was a fully specified Stonyhurst heliographic coordinate. (#5584)
XRTMapuppercases theTIMESYSkey before checking if the key needs to be fixed. (#5592)Fixed passing a URL to
sunpy.io.read_file()on windows. (#5601)Fixed a bug where the
dateproperty onHMISynopticMapreturnedNoneif theDATE-OBSkey was present. (#5648)
Documentation#
Added the gallery example Comparing differential-rotation models to visualize the differences between models of solar differential rotation. (#5527)
Added an example to how to save out maps as FITS files and load them back in, Saving and loading sunpy Maps with FITS. (#5544)
Internal Changes#
The
Helioprojectiveframe now has the convenience propertyangular_radiusto return the angular radius of the Sun as seen by the observer. (#5191)Online tests can now report back status of remote urls and will XFAIL if the remote server is unreachable. (#5233)
Re-enabled the unit test to check for coordinates consistency with JPL HORIZONS when the matching ephemeris can be specified. (#5314)
The
TimeSeriesfactory has been refactored to improve readability and maintainability of the internal code. (#5411)sunpy.map.GenericMap.rsun_obsno longer emits a warning if the metadata it looks for is not present. Instead the standard photospheric radius is assumed and a log message emitted at the ‘info’ level. (#5416)Nearest-neighbour and linear (the default for
sunpy.map.GenericMap.resample()) resampling have been significantly sped up. (#5476)sunpy.map.Mapnow raises a clear error when the map is constructed if units of either two axes are not angular units. (#5602)
3.0.1 (2021-07-03)#
Bug Fixes#
Fixed a bug where
GenericMapused to break with keyword arguments. (#5392)Fixed a bug where calling
sunpy.map.GenericMap.draw_contours()on a different WCS could result in an unnecessary expansion of the plot limits. (#5398)Fixed incorrect return values from
all_corner_coords_from_map()if a rectangular map was provided. (#5419)Do not trigger a pytest import in the asdf plugin for saving sunpy coordinate frames. (#5429)
Constructing a 2D coordinate in the
HeliographicCarringtonframe withobserver='self'now raises an error upon creation. When specifyingobserver='self', theradiuscoordinate component serves as the Sun-observer distance that is necessary to fully define the Carrington heliographic coordinates. (#5358)Fixed two bugs with handling the motion of the Sun when transforming between coordinate frames with a change in
obstime. These bugs did not affect any results if the context managertransform_with_sun_center()had been used. (#5381)Fixed a bug where the
rsunframe attribute could be unintentionally reset to the default value during transformation. This bug primarily affected the transformation of aHelioprojectivecoordinate to aHeliographicStonyhurstframe. (#5395)Fixed a bug where creating a
HeliographicStonyhurstframe or aHeliographicCarringtonframe from WCS information failed to make use of any specifiedrsun_refvalue. (#5395)SXTMapnow always returnsNonefor thewavelengthattribute. Previously this raised an error. (#5401)
Added/Improved Documentation#
3.0.0 (2021-05-14)#
Backwards Incompatible Changes#
sunpy.instrhas been deprecated and will be removed in sunpy 3.1 in favour ofsunkit_instruments. The code that is undersunpy.instris imported viasunkit_instrumentsto ensure backwards comparability. (#4526)Several
sunpy.map.GenericMapattributes have been updated to returnNonewhen the relevant piece of FITS metadata is missing. These are:exposure_time, previously defaulted to zero seconds.measurement, previously defaulted to zero.waveunit, previously defaulted tou.one.wavelength, previously defaulted to zero. (#5126)
HeliographicStonyhurstandHeliographicCarringtonno longer automatically convert 2D input to a 3D coordinate during instantiation. Instead, the 2D-to-3D conversion is deferred until the coordinate is transformed to a different frame, or with a call to the methodmake_3d(). (#5211)Changed URL for the
sunpy.net.dataretriever.sources.noaa.SRSClientfrom “ftp://ftp.swpc.noaa.gov/pub/warehouse/” to “ftp://ftp.ngdc.noaa.gov/STP/swpc_products/daily_reports/”. The old URL is unsupported and we expect the files will be the same but we can not say with 100% certainty. (#5173)Changed
sunpy.net.attrs.Sourcetosunpy.net.attrs.Providerfor thesunpy.net.dataretriever.sources.gong.GONGClient. (#5174)The
rsunframe attribute ofHelioprojectivenow converts any input to kilometers. (#5211)sunpy.map.CompositeMap.plot()now internally callssunpy.map.GenericMap.plot()andsunpy.map.GenericMap.draw_contours(), which may affect the plot output of existing user code. (#5255)Removed the
basic_plotkeyword argument fromsunpy.map.CompositeMap.peek()due to its unreliability. (#5255)sunpy.util.sphinx.changelogandsunpy.util.towncrierhave been removed and are now in a standalone package sphinx-changelog. (#5049)
Deprecations and Removals#
Deprecated
sunpy.map.GenericMap.draw_rectanglein favor ofdraw_quadrangle(). (#5236)Using
GenericMapplotting methods on anAxesthat is not aWCSAxesis deprecated. This previously raised a warning, but is now formally deprecated, and will raise an error in sunpy 3.1. (#5244)Deprecated
sunpy.roi.chaincode.Chaincodeand created a replacement atsunpy.net.helio.Chaincode.This replacement has the following changes:
Added support for numpy array as an input (it was broken before).
Renamed
BoundingBoxtoboundingboxRenamed
subBoundingBoxtosub_boundingboxNow area and length raise
NotImplementedError(#5249)
Deprecated
sunpy.roi.roi, as it currently has no obvious use and has never seen any real development work. (#5249)
Features#
sunpy.coordinates.get_horizons_coord()can now be given a start time, end time, and number of intervals (or interval length) to query a evenly spaced set of times. See the documentation string for more information and an example. (#4698)Added
sunpy.map.GenericMap.draw_quadrangle()for drawing a quadrangle on a map. A quadrangle has edges that are aligned with lines of constant latitude and longitude, but these can be in a different coordinate system than that of the map. (#4809)Added a
longitudekeyword argument tocarrington_rotation_time()as an alternate way to specify a fractional Carrington rotation. (#4879)Colorbar in
sunpy.map.GenericMap.peeknow has a unit label. (#4930)The default axes used by
BaseFuncAnimator.get_animation()is nowBaseFuncAnimator.axes, instead of the currently active axes (accessed via.matplotlib.pyplot.gca()). The allows animations to be created on figures created directly usingmatplotlib.figure.Figure.To revert to the previous behaviour of using the current axes, give
axes=plt.gca()toget_animation(). (#4968)Added colormaps for Solar Orbiter EUI images. These are used automatically when an EUI image is loaded. (#5023)
Added the ability to dynamically scale
sunpy.visualization.animatorinstances. By specifying theclip_intervalkeyword, it will now clip the minimum and maximum at each slider step to the specified interval. (#5025)Added a
sunpy.time.timerange.TimeRange.__contains__method tosunpy.time.TimeRangethat tests if two time ranges overlap. (#5093)Added the ability to namespace files downloaded using
sunpy.data.data_manager.manager.DataManagerby prepending the file name with module name. (#5111)Added a rigid rotation model to
diff_rot()viarot_type=rigid, where the rotation rate does not vary with latitude. (#5132)Added a
save()method tosunpy.map.MapSequencethat saves each map of the sequence. (#5145)The allowable
levelinputs tosunpy.map.GenericMap.contour()andsunpy.map.GenericMap.draw_contours()have been consolidated. Both methods now accept - Scalars, if the map has no units - Quantities, if the map has units - Percentages (#5154)Added support for corrected NOAA SWPC solar region summary data files. (#5173)
Updated
sunpy.util.sysinfo.system_infoto return all optional dependencies of sunpy. (#5175)sunpy.map.Mapnow supports the EUI instrument on Solar Orbiter. (#5210)HeliographicStonyhurstandHeliographicCarringtonnow have anrsunframe attribute to specify the radius of the Sun, which defaults to the photospheric radius defined insunpy.sun.constants. This frame attribute is used when converting a 2D coordinate (longitude and latitude, with no specified radial distance) to a 3D coordinate by setting the radial distance torsun(i.e., the assumption is that the coordinate is on the surface of the Sun). (#5211)Enhanced
sunpy.map.GenericMap.draw_limb()so that the solar limb can be plotted on axes that correspond to a different map (e.g., with a different observer). The part of the limb that is not visible to the axes’s observer because it is on the far side of the Sun is shown as dotted rather than solid. (#5237)MetaDictnow saves a copy of the metadata on creation, which can be accessed using theoriginal_metaproperty. Three new properties have also been added to query any changes that have been made to metadata:As an example,
my_map.meta.modified_itemswill return a dictionary mapping keys to their original value and current value. (#5241)Added
sunpy.map.contains_coordinate()which provides a quick way to see if a world coordinate is contained within the array bounds of a map. (#5252)Added an optional keyword argument
autoaligntosunpy.map.GenericMap.plot()for plotting a map to axes that correspond to a different WCS. See Auto-Aligning AIA and HMI Data During Plotting. (#5255)sunpy.map.CompositeMap.plot()now properly makes use of WCS information to position and orient maps when overlaying them. (#5255)
Bug Fixes#
Fixed the drawing methods of
sunpy.map.GenericMap(e.g.,~sunpy.map.GenericMap.draw_rectangle) so that any text labels will appear in the legend. (#5019)Fixed bug in
sunpy.until.scraper.Scraperwhich caused URL patterns containing backslashes to be incorrectly parsed on Windows. (#5022)Constructing a
MetaDictis now more lenient, and accepts any class that inherits fromcollections.abc.Mapping. This fixes a regression where headers read withastropy.io.fitsraised an error when passed to individualmapsources. (#5047)Added warning to
sunpy.map.GenericMap.rotate()when specifiedmissingvalue is not compatible with the number type of the data array. (#5051)Prevented some colormaps being accidentally modified depending on the order and method through which they were accessed. (#5054)
Reverted change for
sunpy.map.GenericMap.draw_limbthat made it use “add_artist” as it was changing the FOV of the plotted image. (#5069)Fixed a bug where some
RotatedSunFrametransformations could fail with anobserver=Noneerror. (#5084)Fixed bug where
sunpy.data.data_manager.DataManagerwould fail to recover upon deleting the sqlite database file. (#5089)Fixed a bug where coordinate frames were considered different due to an unintended time difference during time handling at the level of numerical precision (i.e., tens of picoseconds). This resulted in the unexpected use of transformation machinery when transforming a coordinate to its own coordinate frame. (#5127)
Fixed a bug with failing downloads in 2010 with the
SRSClient. (#5159)If the property
sunpy.map.GenericMap.rsun_obsneeds to calculate the solar angular radius from header information, it now properly uses thersun_refkeyword if it is present and does not emit any warning. (#5172)Added a “rsun_obs” keyword to the output of
sunpy.map.header_helper.make_fitswcs_header()if the coordinate argument has a “rsun” frame attribute. (#5177)Fixed small inaccuracies in the grid plotted by
draw_grid()for maps that specify a radius of the Sun that is different from the constant insunpy.sun.constants. (#5211)Fixed
sunpy.map.GenericMap.draw_contours()so that the contours from a map can be plotted on axes with a different coordinate system. (#5239)When using the cylindrical representation of
Heliocentricto work in the Heliocentric Radial coordinate frame, thepsicomponent now goes from 0 to 360 degrees instead of -180 to 180 degrees. (#5242)Changed
MDIMapto use the “CONTENT” keyword to identify the measurement, similar toHMIMap, and removed the special-case nickname. This fixes the broken title on plots. (#5257)sunpy.coordinates.solar_frame_to_wcs_mapping()now sets the observer auxiliary information when aHeliographicCarringtonframe withobserver='self'is passed. (#5264)Calling
sunpy.map.header_helper.make_fitswcs_header()with aHeliographicCarringtoncoordinate that withobserver='self'set now correctly sets the observer information in the header. (#5264)sunpy.map.GenericMap.superpixel()now keeps the reference coordinate of the WCS projection the same as the input map, and updates the reference pixel accordingly. This fixes inconsistencies in the input and output world coordinate systems when a non-linear projection is used. (#5295)Inputs to the
dimensionsandoffsetarguments tosunpy.map.GenericMap.superpixel()in units other thanu.pix(e.g.`u.kpix) are now handled correctly. (#5301)Fractional inputs to the
dimensionsandoffsetarguments tosunpy.map.GenericMap.superpixel()were previously rounded usingintin the superpixel algorithm, but not assigned integer values in the new metadata. This has now been changed so the rounding is correctly reflected in the metadata. (#5301)Remove runtime use of
astropy.tests.helper.assert_quantity_allclosewhich introduces a runtime dependency onpytest. (#5305)sunpy.map.GenericMap.resample()now keeps the reference coordinate of the WCS projection the same as the input map, and updates the reference pixel accordingly. This fixes inconsistencies in the input and output world coordinate systems when a non-linear projection is used. (#5309)Fix saving
GenericMapto an asdf file with version 2.8.0 of the asdf package. (#5342)
Added/Improved Documentation#
Added a gallery example for drawing rectangles on maps. (#4528)
Added an example (Plotting points on a Map with WCSAxes) of how pixel and SkyCoords work when plotted with
wcsaxes. (#4867)Added a gallery example (Plot positions on a blank map) on how to create a blank map and mark locations. (#5077)
Added a gallery example (HMI Showcase: Cutout) demonstrating how to add a HMI zoomed-in region next to a full disk HMI image. (#5090)
Updated the Masking out the solar disk example to generate the mask using
sunpy.map.coordinate_is_on_solar_disk(). (#5114)Added a gallery example (Segmenting a Map based on transformation of coordinates) demonstrating how to create a segment of a particular map from transformed coordinates. (#5121)
For the various subclasses of
GenericMap(e.g.,AIAMap), the online documentation now shows all of the inherited attributes and methods. (#5142)Added a documentation string to
HMISynopticMap. (#5186)Added a new gallery example showcasing how to overlay HMI contours on an AIA image. (#5229)
Trivial/Internal Changes#
Replaced the old test runner with a new version that adds a dependency check before the test suite is run. (#4596)
The testing suite now raises a warning if the
pyplotfigure stack is not empty prior to running a test, and it closes all open figures after finishing each test. (#4969)Improved performance when moving the slider in
sunpy.visualisation.animator.ArrayAnimatorWCS. (#4971)Added some basic logging to HEK searches, at the ‘debug’ logging level. (#5020)
Refactored
RotatedSunFrametransformations for improved performance. (#5084)Re-ordered keyword-only arguments of
sunpy.map.GenericMap.draw_rectangleto matchsunpy.map.GenericMap.submap(). (#5091)Significantly sped up calls to
parse_time()for string arguments. This will have knock on effects, including improved performance of querying the VSO. (#5108)Added tests for
sunpy.visualization.animator.mapsequenceanimatorandsunpy.map.MapSequence.plot(). (#5125)The
CROTAkeywords are no longer set onsunpy.map.GenericMap.wcs, as thePC_ijkeywords are always set and the FITS standard says that these keywords must not co-exist. (#5166)Temporarily disabled the unit test to check for coordinates consistency with JPL HORIZONS due to the inability to choose a matching ephemeris. (#5203)
wcsaxes_heliographic_overlay()now acceptsobstimeandrsunoptional arguments. This function is not typically called directly by users. (#5211)GenericMapplotting methods now have consistent argument checking for theaxesargument, and will raise the same warnings or errors for similaraxesinput. (#5223)Calling
sunpy.map.GenericMap.plot()on aWCSAxeswith a different World Coordinate System (WCS) to the map now raises a warning, as the map data axes may not correctly align with the coordinate axes. This happens if anAxesis created with a projection that is a different map to the one being plotted. (#5244)Re-enabled the unit test to check for coordinates consistency with JPL HORIZONS when the matching ephemeris can be specified. (#5314)
2.1.0 (2020-02-21)#
Backwards Incompatible Changes#
Support for Python 3.6 and Numpy 1.15 has been dropped in line with NEP 29. The minimum supported version of Astropy is now 4.0, and the minimum version of scipy is now 1.2. (#4284)
Changed
sunpy.coordinates.sun.B0()return type fromAngletoLatitude. (#4323)An error is now raised if
vminorvmaxare passed to tosunpy.map.GenericMap.plotand they are already set on the mapnorm. This is consistent with upcoming Matplotlib changes. (#4328)Previously slicing the result of
Fido.search()(aUnifiedResponseobject) so that it had a length of one returned anotherUnifiedResponseobject. Now it will return aQueryResponseTableobject, which is a subclass ofastropy.table.Table. (#4358)The
.sizeproperty of a coordinate frame with no associated data will now raise an error instead of returning 0. (#4577)The following
Mapmethods have had support for specific positional arguments removed. They must now be passed as keyword arguments (i.e.m.method(keyword_arg=value)).The sunpy specific attributes
.heliographic_observerand.rsunare no longer set on theWCSreturned bysunpy.map.GenericMap.wcs. (#4620)Due to upstream changes, the parsing logic for the
HECClientnow returns strings and not bytes forget_table_names(). (#4643)Reduced the selection of dependent packages installed by default via
pip, which means that some of our sub-packages will not fully import when sunpy is installed withpip install "sunpy". You can install all dependencies by specifyingpip install "sunpy[all]", or you can install sub-package-specific dependencies by specifying, e.g.,[map]or[timeseries]. (#4662)The class inheritance for
RotatedSunFrameand the frames it creates has been changed in order to stop depending on unsupported behavior in the underlying machinery. The return values for someisinstance()/issubclass()calls will be different, but the API forRotatedSunFrameis otherwise unchanged. (#4691)Fix a bug in
submapwhere only the top right and bottom left coordinates of the input rectangle in world coordinates were considered when calculating the pixel bounding box. All four corners are once again taken into account now, meaning thatsubmapcorrectly returns the smallest pixel box which contains all four corners of the input rectangle.To revert to the previous 2.0.0 behaviour, first convert the top right and bottom left coordinates to pixel space before calling submap with:
top_right = smap.wcs.world_to_pixel(top_right) * u.pix bottom_left = smap.wcs.world_to_pixel(bottom_left) * u.pix smap.submap(bottom_left=bottom_left, top_right=top_right)
This will define the rectangle in pixel space. (#4727)
VSO results where the size was
-1(missing data) now returnNonerather than-1to be consistent with other missing data in the VSO results. (#4798)All result objects contained within the results of a
Fido.search()(aUnifiedResponseobject) are nowQueryResponseTableobjects (or subclasses thereof). These objects are subclasses ofastropy.table.Tableand can therefore be filtered and inspected as tabular objects, and the modified tables can be passed toFido.fetch.This, while a breaking change for anyone accessing these response objects directly, will hopefully make working with
Fidosearch results much easier. (#4798)Results from the
NOAAIndicesClientand theNOAAPredictClientno longer hasStart TimeorEnd Timein their results table as the results returned from the client are not dependent upon the time parameter of a search. (#4798)The
sunpy.net.vso.QueryResponse.searchmethod has been removed as it has not worked since the 1.0 release of sunpy. (#4798)The
sunpy.net.hek.hek.HEKColumnclass has been removed, theHEKTableclass now uses the standardastropy.table.Columnclass. (#4798)The keys used to format file paths in
Fido.fetchhave changed. They are now more standardised across all the clients, as they are all extracted from the names of the columns in the results table.For results from the VSO the keys are no longer separated with
., and are based on the displayed column names. For results from thedataretrieverclients the only main change is that the keys are now lower case, where they were capitalized before. You can use the.sunpy.net.fido_factory.UnifiedResponse.path_format_keysmethod to see all the possible keys for a particular search. (#4798)The time returned from
carrington_rotation_number()has been changed from the TT scale to the more common UTC scale. To undo this change, usetime_out = time_out.tton the outputted time. (#4819)response_block_propertieshas been renamed to.BaseQueryResponse.path_format_keys, on the return objects from allsearch()methods on all clients and fromFido.search(). (#4798)
Removals#
Removed deprecated functions:
sunpy.coordinates.frames.Helioprojective.calculate_distance, alternative issunpy.coordinates.frames.Helioprojective.make_3d.sunpy.image.coalignment.repair_image_nonfinite- if you wish to repair the image, this has to be done manually before calling the varioussunpy.image.coalignmentfunctions.The
repair_nonfinitekeyword argument tocalculate_shiftandcalculate_match_template_shifthas been removed.sunpy.instr.lyra.download_lytaf_database- this just downloaded the file athttp://proba2.oma.be/lyra/data/lytaf/annotation_ppt.db, which can be done manually.sunpy.util.net.check_download_file, no alternative.sunpy.visualization.animator.ImageAnimatorWCS, alternative issunpy.visualization.animator.ArrayAnimatorWCS. (#4350)
Removed deprecated function
sunpy.instr.aia.aiaprep. Alternative isregisterfor converting AIA images from level 1 to level 1.5. (#4485)sunpy.cmhas been removed. All of the functionality in this module can now be found insunpy.visualization.colormaps. (#4488)sunpy.test.hashhas been removed, the functionality has been moved into the pytest-mpl package. (#4605)sunpy.util.multimethodhas been removed. (#4614)The
lytaf_pathargument (which previously did nothing) has been removed from -sunpy.instr.lyra.remove_lytaf_events_from_timeseries-sunpy.instr.lyra.get_lytaf_events-sunpy.instr.lyra.get_lytaf_event_types(#4615)
Deprecations#
Deprecated
sunpy.net.vso.attrs.Sourceandsunpy.net.vso.attrs.Provider. They are nowsunpy.net.attrs.Sourceandsunpy.net.attrs.Providerrespectively. (#4321)Deprecated the use of the
sunpy.map.GenericMap.sizeproperty, usesunpy.map.Map.data.sizeinstead. (#4338)sunpy.net.helio.HECClient.time_queryis deprecated,searchis the replacement. (#4358)sunpy.net.jsoc.attrs.Keysis deprecated; all fields are returned by default and can be filtered post search. (#4358)sunpy.net.hek.attrs.Timeis deprecated;Timeshould be used instead. (#4358)Support for
sunpy.coordinates.wcs_utils.solar_wcs_frame_mapping()to use the.heliographic_observerand.rsunattributes on aWCSis deprecated. (#4620)The
originargument tosunpy.map.GenericMap.pixel_to_worldandsunpy.map.GenericMap.world_to_pixelis deprecated.If passing
0, not using theoriginargument will have the same effect.If passing
1, manually subtract 1 pixel from the input topixel_to_world, or manually add 1 pixel to the output ofworld_to_pixel, and do not use theoriginargument. (#4700)
The
.VSOClient.linkmethod is deprecated as it is no longer used. (#4789)The
.UnifiedResponse.get_response,.UnifiedResponse.tablesand.UnifiedResponse.responsesattributes of.UnifiedResponsehave been deprecated as they are no longer needed now the object returns the table objects it contains when sliced. (#4798)sunpy.net.vso.VSOClient.search()has a new keyword argumentresponse_type=which controls the return type from thesearch()method. In sunpy 2.1 and 3.0 it will default to the"legacy"response format, in 3.1 it will default to the new"table"response format, and the"legacy"format may be deprecated and removed at a later date.Searches made with
Fidowill use the new"table"response format, so this only affects users interacting with theVSOClientobject directly. (#4798)
Features#
For
sunpy.map.GenericMap.quicklook()andsunpy.map.MapSequence.quicklook()(also used for the HTML representation shown in Jupyter notebooks), the histogram is now shaded corresponding to the colormap of the plotted image. Clicking on the histogram will toggle an alternate version of the histogram. (#4931)Add an
SRS_TABLEfile to the sample data, and use it in the magnetogram plotting example. (#4993)Added a
sunpy.map.GenericMap.contour()method to find the contours on a map. (#3909)Added a context manager (
assume_spherical_screen()) to interpretHelioprojectivecoordinates as being on the inside of a spherical screen instead of on the surface of the Sun. (#4003)Added
sunpy.map.sources.HMISynopticMapfor handling the Synoptic maps from HMI. (#4053)Added a
MDISynopticMapmap source class. (#4054)Created
GONGClientfor accessing magnetogram synoptic map archives of NSO-GONG. (#4055)All coordinate frames will now show the velocity if it exists in the underlying data. (#4102)
The ephemeris functions
get_body_heliographic_stonyhurst(),get_earth(), andget_horizons_coord()can now optionally return the body’s velocity as part of the output coordinate. (#4102)MetaDictnow maintains coherence between its keys and their corresponding keycomments. Callingdelon aMetaDictobject key is now case-insensitive. (#4129)Allow
sunpy.visualization.animator.ArrayAnimatorWCSto disable ticks for a coordinate, by settingticks: Falsein thecoord_paramsdictionary. (#4270)Added a
show()method forBaseQueryResponsewhich returnsTablewith specified columns for the Query Response. (#4309)Added
_extract_files_metamethod insunpy.util.scraper.Scraperwhich allows scraper to extract metadata from the file URLs retrieved for a given time range. (#4313)Refactoring of
dataretrieverwhich adds these capabilities toQueryResponse:Any
attrshall not be defaulted to a hard-coded value in all subclasses ofGenericClient; thus records for all possibleattrsshall be returned if it is not specified in the query.QueryResponsecan now show more columns; thus all metadata extractable from matching file URLs shall be shown and for a client, non-supportedattrsshall not be shown in the response tables. (#4321)
New class attributes added to
GenericClient:Additions in
sunpy.util.scraperto support the refactoring ofGenericClient: -sunpy.util.scraper.Scraper.findDatewith_extractorthat parses the url using extractor to return its start time. - Amatcherinsunpy.util.scraper.Scraper._extract_files_metawhich validates the extracted metadata by using the dictionary returned from_get_match_dict(). (#4321)Added methods
pre_search_hook()andpost_search_hook()which helps to translate the attrs for scraper before and after the search respectively. (#4321)sunpy.timeseries.sources.RHESSISummaryTimeSeries.peekhas had the following minor changes:Colors from the default matplotlib color cycle are now used (but the colors remain qualitatively the same)
The default matplotlib linewidth is now used
It is now possible to pass in a user specified linewidth
Seconds have been added to the x-axis labels (previously it was just hours and minutes) (#4326)
HECClientandHEKClientnow inheritBaseClientwhich makes them compatible with theUnifiedDownloaderFactory(Fido). (#4358)MaxRecordsandTableNameadded as “attrs” for HELIO searches. (#4358)Add the ability to download new GOES 16 & 17 data alongside the reprocessed GOES 13, 14 and 15 data via the GOES-XRS Fido client. (#4394)
sunpy.net.jsoc.JSOCClient.request_datanow support additional parameter “method” which allows user to download staged data as single .tar file. (#4405)Added
sunpy.util.get_timerange_from_exdictwhich finds time range for a URL using its metadata. Addedsunpy.util.scraper.Scraper.isvalid_timethat checks whether the file corresponds to a desired time range. (#4419)Colormap data has been moved to individual .csv files in the
sunpy/visualization/colormaps/datadirectory. (#4433)Added
solar_angle_equivalencyto convert between a physical distance on the Sun (e.g., km) to an angular separation as seen by an observer (e.g., arcsec). (#4443)sunpy.map.Mapinstances now have their.unitattribute set from the'BUNIT'FITS keyword. If the keyword cannot be parsed, or is not present the unit is set toNone. (#4451)The
sunpy.map.GenericMap.wcsproperty is now cached, and will be recomputed only if changes are made to the map metadata. This improves performance of a number of places in the code base, and only one warning will now be raised about WCS fixes for a given set of metadata (as opposed to a warning each time.wcsis accessed) (#4467)Extended
concatenate()andconcatenate()to allow iterables. (#4499)Enable
RotatedSunFrameto work with non-SunPy frames (e.g.,HeliocentricMeanEcliptic). (#4577)Add support for
pathlib.Pathobjects to be passed tosunpy.timeseries.TimeSeries. (#4589)Add support for GOES XRS netcdf files to be read as a
sunpy.timeseries.sources.XRSTimeSeries. (#4592)Add
Cutoutattr for requesting cutouts from JSOC viaJSOCClientandFido. (#4595)sunpy now sets auxiliary parameters on
sunpy.map.GenericMap.wcsusing theastropy.wcs.Wcsprm.auxattribute. This stores observer information, along with the reference solar radius if present. (#4620)The
HeliographicCarringtonframe now accepts the specification ofobserver='self'to indicate that the coordinate itself is also the observer for the coordinate frame. This functionality greatly simplifies working with locations of observatories that are provided in Carrington coordinates. (#4659)Add two new colormaps (
rhessiandstd_gamma_2) that are used for plotting RHESSI maps. (#4665)If either ‘CTYPE1’ or ‘CTYPE2’ are not present in map metadata, sunpy now assumes they are ‘HPLN-TAN’ and ‘HPLT-TAN’ (previously it assumed ‘HPLN- ‘ and ‘HPLT- ‘). In addition, a warning is also now raised when this assumption is made. (#4702)
Added a new
all_corner_coords_from_mapfunction to get the coordinates of all the pixel corners in aGenericMap. (#4776)Added support for “%Y/%m/%dT%H:%M” to
sunpy.time.parse_time(). (#4791)Added the STEREO EUVI instrument specific colormaps called” ‘euvi171’, ‘euvi195’, ‘euvi284’, ‘euvi304’. (#4822)
Bug Fixes#
sunpy.map.GenericMap.datenow has its time scale set from the ‘TIMESYS’ FITS keyword, if it is present. If it isn’t present the time scale defaults to ‘UTC’, which is unchanged default behaviour, so this change will only affect maps with a ‘TIMESYS’ keyword that is not set to ‘UTC’. (#4881)Fixed the
sunpy.net.dataretriever.sources.noaa.SRSClientwhich silently failed to download the SRS files when the tarball for the previous years did not exist. Client now actually searches for the tarballs and srs files on the ftp archive before returning them as results. (#4904)No longer is the WAVEUNIT keyword injected into a data source if it is missing from the file’s metadata. (#4926)
Map sources no longer overwrite FITS metadata keywords if they are present in the original metadata. The particular map sources that have been fixed are
SJIMap,KCorMap,RHESSIMap,EITMap,EUVIMap,SXTMap. (#4926)Fixed a handling bug in
sunpy.map.GenericMap.draw_rectanglewhen the rectangle is specified in a different coordinate frame than that of the map. A couple of other minor bugs insunpy.map.GenericMap.draw_rectanglewere also fixed. (#4929)Improved error message from
sunpy.net.Fido.fetch()when no email has been supplied for JSOC data. (#4950)Fixed a bug when transforming from
RotatedSunFrameto another frame at a different observation time that resulted in small inaccuracies. The translational motion of the Sun was not being handled correctly. (#4979)Fixed two bugs with
differential_rotate()andsolar_rotate_coordinate()that resulted in significant inaccuracies. Both functions now ignore the translational motion of the Sun. (#4979)The ability to to filter search results from the
VSOClientwas broken. This has now been restored. (#4011)Fixed a bug where transformation errors were not getting raised in some situations when a coordinate frame had
obstimeset to the default value ofNoneandSkyCoordwas not being used. Users are recommended to useSkyCoordto manage coordinate transformations unless they have a specific reason not to. (#4267)Fixed a bug in
_get_url_for_timerangewhich returned incorrect URLs because of not using**kwargsin the client’s_get_overlap_urls()method. (#4288)Data products from
NOAAIndicesClientandNOAAPredictClienthave been updated to download new JSON files. The old text files which the data used to come in no longer exist. The new JSON files forNOAAIndicesClientnow do not have the following columns: - Geomagnetic Observed and Smoothed - Sunspot Numbers Ratio (RI/SW)Both
sunpy.timeseries.sources.noaa.NOAAIndicesTimeSeriesandsunpy.timeseries.sources.noaa.NOAAPredictIndicesTimeSerieshave been updated to support the new JSON files. Loading the old text files is still supported, but support for this will be removed in a future version of sunpy. (#4340)Fixed a bug due to which
sunpy.net.helio.parser.wsdl_retrieverignored previously discovered Taverna links. (#4358)The flare class labels in GOES
peek()plots are now drawn at the center of the flare classes. Previously they were (ambiguously) drawn on the boundaries. (#4364)sunpy.map.GenericMap.rsun_obsno longer assumes the observer is at Earth ifrsun_obswas not present in the map metadata. The sun-observer distance is now taken directly from the observer coordinate. If the observer coordinate is not present, this defaults to the Earth, retaining previous behaviour. (#4375)
Nanosecond precision is now retained when using
parse_timewith aTimestamp. (#4409)Fixed a bug where SunPy could not be successfully imported if the default text encoding of the running environment was unable to handle non-ASCII characters. (#4422)
sunpy.net.dataretriever.sources.noaa.SRSClientnow correctly returns zero results for queries in the future or before 1996, which is when data is first available. (#4432)Fixes issue where NAXISn is not updated after invoking
GenericMap.resample()(#4445)The floating point precision of input to
sunpy.image.transform.affine_transformis now preserved. Previously all input was cast tonumpy.float64, which could cause large increases in memory use for 32 bit data. (#4452)Fixed
affine_transform()to scale images to [0, 1] before passing them toskimage.transform.warp()and later rescale them back. (#4477)Several
warnings.simplefilter('always', Warning)warning filters insunpy.timeserieshave been removed. (#4511)All calculations of the angular radius of the Sun now use the same underlying code with the accurate calculation. The previous inaccuracy was a relative error of ~0.001% (0.01 arcseconds) for an observer at 1 AU, but could be as large as ~0.5% for Parker Solar Probe perihelia. (#4524)
Fixed an issue in
sunpy.time.TimeRange.get_dates()where the function would return the wrong number of days if less than 24 hours had passed (#4529)Several functions in
sunpy.mapnow properly check if the provided coordinate is in the expectedHelioprojectiveframe. (#4552)Fixes a bug which occurs in setting the
ylimsbysunpy.visualization.animator.line.LineAnimatorwhen there are non-finite values in the data array to be animated. (#4554)Clear rotation metadata for SOHO/LASCO Helioviewer JPEG2000 images, as they are already rotated correctly. (#4561)
The
max_connargument toFido.fetch()is now correctly respected by the JSOC client. Previously the JSOC client would default to 4 connections no matter what the value passed toFido.fetch()was. (#4567)sunpy.time.parse_time()now correctly parses lists of time strings that have one of the built in sunpy time formats. (#4590)Fixes the SRSClient to search for files of correct queried time and now allows a path keyword to be downloaded in fetch. (#4600)
Fixed
sunpy.net.helio.parser.wsdl_retriever, which previously ignored discovered Taverna links. (#4601)The transformations between
HCRSandHeliographicStonyhursthave been re-implemented to enable the proper transformations of velocities. All ephemeris functions (e.g.,get_body_heliographic_stonyhurst()) now return properly calculated velocities wheninclude_velocity=Trueis specified. (#4613)The maximum number of connections opened by the JSOC downloader has been reduced from 4 to 2. This should prevent downloads of large numbers of files crashing. (#4624)
Fixed a significant performance bug that affected all coordinate transformations. Transformations have been sped up by a factor a few. (#4663)
Fixed a bug with the mapping of a WCS header to a coordinate frame if the observer location is provided in Carrington coordinates. (#4669)
sunpy.io.fits.header_to_fitsnow excludes any keys that have associated NaN values, as these are not valid in a FITS header, and throws a warning if this happens. (#4676)Fixed an assumption in
sunpy.map.GenericMap.pixel_to_worldthat the first data axis is longitude, and the second is latitude. This will affect you if you are using data where the x/y axes are latitude/longitude, and now returns correct values in methods and properties that callpixel_to_world, such asbottom_left_coord,top_right_coord,center. (#4700)Added a warning when a 2D
Helioprojectivecoordinate is upgraded to a 3D coordinate and the number type is lower precision than the native Python float. This 2D->3D upgrade is performed internally when transforming a 2DHelioprojectivecoordinate to any other coordinate frame. (#4724)All columns from a
sunpy.net.vso.VSOClient.searchwill now be shown. (#4788)The search results object returned from
Fido.search(UnifiedResponse) now correctly counts all results in it’sfile_numproperty. Note that because someFidoclients now return metadata only results, this is really the number of records and does not always correspond to the number of files that would be downloaded. (#4798)Improved the file processing logic for EVE L0CS files, which may have fixed a bug where the first line of data was parsed incorrectly. (#4805)
Fixing the
CROTAmeta keyword in EUVI FITS toCROTAnstandard. (#4846)
Added/Improved Documentation#
Added a developer guide for writing a new
Fidoclient. (#4387)Added an example of how to use Matplotlib’s axes range functionality when plotting a Map with WCSAxes. (#4792)
Add links to Thompson 2006 paper on solar coordinates to synoptic map example. (#3549)
Clarified the meaning of
.bottom_left_coordand.top_right_coordinsunpy.map.GenericMap. (#3706)Added a list of possible signatures to
sunpy.timeseries.metadata.TimeSeriesMetaData. (#3709)Added
sunpy.data.manager,sunpy.data.cache,sunpy.net.Fido,sunpy.map.Map, andsunpy.timeseries.TimeSeriesto the docs. (#4098)Clarified spline option for
sunpy.map.GenericMap.resample. (#4136)Updated the gallery example Plotting a solar cycle index to retrieve data using
Fido. (#4169)Fixed example usage of
sunpy.io.fits.readto account for the fact that it returns a list of data-header pairs rather than the data-header pairs directly. (#4183)Added example of how to create a
sunpy.map.GenericMapfrom observations in RA-DEC coordinates. (#4236)Added
sunpy.coordinates.SunPyBaseCoordinateFrameandsunpy.coordinates.BaseHeliographicto the documentation. (#4274)sunpy.time.TimeRangehad a.__contains__method and this is now documented. (#4372)Revamped sunpy pull request review developer documentation. (#4378)
Revamped sunpy installation documentation. (#4378)
Fixed broken documentation links in the guide. (#4414)
Fixed miscellaneous links in the API documentation. (#4415)
Added
sunpy.data.data_manager.downloader,sunpy.data.data_manager.storage, andsunpy.net.hek.HEKTableto the docs. (#4418)Added documentation for copying Map objects using the copy module’s deepcopy method. (#4470)
Added a gallery example for saving and loading sunpy Maps using asdf. (#4494)
Added description for a counter-intuitive section in the Differentially rotating a map example. (#4548)
Added Coordinates with velocity information to explain how to use velocity information in the coordinates framework. (#4610)
New gallery example of searching and downloading GOES XRS data (with GOES 15, 16 and 17). (#4686)
Created the new gallery example Offsetting the north pole of a coordinate frame for
NorthOffsetFrame. (#4709)Added more information on which FITS keywords are used for various
sunpy.map.GenericMapproperties. (#4717)Improved documentation for
sunpy.physics.differential_rotation.diff_rot(). (#4876)
Documentation Fixes#
The keyword
clip_intervalis now used more extensively in gallery examples when plotting the sample AIA image (e.g., Plotting a map). (#4573)Modified Overplotting SRS active region locations on a magnetograms to use HMI file from sample data instead of downloading it with Fido. (#4598)
Removed unnecessary transformations of coordinates prior to plotting them using
plot_coord. (#4609)Ensure that all attrs are documented and clean the
sunpy.net.hek.attrsnamespace of non-attr objects. (#4834)Fixed miscellaneous issues with the gallery example Aligning AIA and HMI Data with Reproject. (#4843)
Fixed the display of arguments in the documentation for
Fidoattributes (sunpy.net.attrs). (#4916)
Trivial/Internal Changes#
Fido.fetchnow always specifies apath=argument of typepathlib.Pathto thefetchmethod of the client. This path will default to the configured sunpy download dir, will have the user directory expanded, will have the{file}placeholder and will be tested to ensure that it is writeable. (#4949)Added information on what went wrong when
sunpy.map.GenericMap.wcsfails to parse a FITS header into a WCS. (#4335)Fixed the
Helioprojectivedocstring to be clear about the names of the coordinate components. (#4351)Raise a better error message if trying to load a FITS file that contains only one dimensional data. (#4426)
The following functions in
sunpy.maphave had their performance greatly increased, with runtimes typically improving by a factor of 20x. This has been achieved by improving many of the checks so that they only require checking the edge pixels of a map as opposed to all of the pixels.Improved the output when you print a sunpy Map. (#4464)
Creating a
MetaDictwith dictionary keys that are not strings now raises as user-friendlyValueErrorwhich prints all the non-compliant keys. (#4476)Maps created directly via.
sunpy.map.GenericMapnow have their metadata automatically converted to aMetaDict, which is the same current behaviour of thesunpy.map.Mapfactory. (#4476)If the
top_rightcorner given tosunpy.map.GenericMap.submap()is below or to the right of thebottom_leftcorner, a warning is no longer raised (as the rectangle is still well defined), but a message is still logged at the debug level to the sunpy logger. (#4491)Added test support for Python 3.9 (no wheels yet). (#4569)
sunpy.sunfunctions now make use of theGeocentricTrueEclipticframe to simplify internal calculations, but the returned values are unchanged. (#4584)Change the format of the time returned from
carrington_rotation_number()from'jd'to'iso', so printing theTimereturned will now print an ISO timestamp instead of the Julian days. (#4819)The listings for the sample data (
sunpy.data.sample) are now sorted. (#4838)Changed the implementation of a
hypothesis-based test so that it does not raise an error withhypothesis6.0.0. (#4852)
2.0.0 (2020-06-12)#
Backwards Incompatible Changes#
The frames
HeliographicStonyhurstandHeliographicCarringtonnow inherit from the new base classBaseHeliographic. This changes means thatisinstance(frame, HeliographicStonyhurst)is no longerTruewhenframeisHeliographicCarrington. (#3595)aia_color_table,eit_color_tableandsuvi_color_tablenow only takeastropy.unitsquantities instead of strings. (#3640)sunpy.map.Mapis now more strict when the metadata of a map cannot be validated, and an error is now thrown instead of a warning if metadata cannot be validated. In order to load maps that previously loaded without error you may need to passsilence_errors=Truetosunpy.map.Map. (#3646)Fido.searchwill now return results from all clients which match a query, you no longer have to make the query specific to a single client. This means that searches involving the ‘eve’ and ‘rhessi’ instruments will now potentially also return results from the VSO. ForRHESSIClientyou can now specifya.Physobs("summary_lightcurve")to only include the summary lightcurve data products not provided by the VSO. (#3770)The objects returned by the
searchmethods onVSOClient,JSOCClientandGenericClienthave been changed to be based onsunpy.net.base_client.BaseQueryResponse. This introduces a few subtle breaking changes for people using the client search methods directly (notFido.search), or people usingsunpy.net.fido_factory.UnifiedResponse.get_response. When slicing an instance ofQueryResponseit will now return an instance of itself,QueryResponse.blockscan be used to access the underlying records. Also, the.clientattribute of the response no longer has to be the instance of the class the search was made with, however, it often is. (#3770)HeliographicCarringtonis now an observer-based frame, where the observer location (specifically, the distance from the Sun) is used to account for light travel time when determining apparent Carrington longitudes. Coordinate transformations using this frame now require an observer to be specified. (#3782)To enable the precise co-alignment of solar images from different observatories, the calculation of Carrington coordinates now ignores the stellar-aberration correction due to observer motion. For an Earth observer, this change results in an increase in Carrington longitude of ~20 arcseconds. See Calculating Carrington longitude for more information. (#3782)
Fixed a bug where some of the coordinate transformations could raise
ValueErrorinstead ofConvertErrorwhen the transformation could not be performed. (#3894)Astropy 3.2 is now the minimum required version of that dependency. (#3936)
Deprecations and Removals#
Fido search attrs available as
sunpy.net.attrsi.e,a.Time,a.Instrumentetc are now deprecated as VSO attrs (sunpy.net.vso.attrs). (#3714)sunpy.util.multimethod.MultiMethodis deprecated,functools.singledispatchprovides equivalent functionality in the standard library. (#3714)sunpy.net.vso.attrs.Physobshas been moved tosunpy.net.attrs.Physobsand the original deprecated. (#3877)Deprecate
sunpy.instr.aia.aiaprepin favor of theaiapy.calibrate.registerfunction in the [aiapy](LMSAL_HUB/aia_hub/aiapy) package.sunpy.instr.aia.aiaprepwill be removed in version 2.1 (#3960)Removed the module
sunpy.sun.sun, which was deprecated in version 1.0. Use the modulesunpy.coordinates.suninstead. (#4014)Removed Sun-associated functions in
sunpy.coordinates.ephemeris, which were deprecated in 1.0. Use the corresponding functions insunpy.coordinates.sun. (#4014)Remove the deprecated
sunpy.net.vso.vso.VSOClient.query_legacyand.latestmethods. (#4109)Removed the sample datasets NOAAINDICES_TIMESERIES and NOAAPREDICT_TIMESERIES because they will invariably be out of date. Up-to-date versions of these NOAA indices can be downloaded using
Fido(see Plotting a solar cycle index). (#4169)
Features#
Added
RotatedSunFramefor defining coordinate frames that account for solar rotation. (#3537)Added a context manager (
transform_with_sun_center) to ignore the motion of the center of the Sun for coordinate transformations. (#3540)Updated the gallery example titled ‘Downloading and plotting an HMI magnetogram’ to rotate the HMI magnetogram such that solar North is pointed up. (#3573)
Creates a function named
sunpy.map.sample_at_coordsthat samples the data from the map at the given set of coordinates. (#3592)Enabled the discovery of search attributes for each of our clients. (#3637)
Printing
sunpy.net.attrs.Instrumentor other “attrs” will show all attributes that exist under the corresponding “attr”. (#3637)Printing
sunpy.net.Fidowill print out all the clients that Fido can use. (#3637)Updates
draw_gridto allow disabling the axes labels and the ticks on the top and right axes. (#3673)Creates a
tablesproperty forUnifiedResponse, which allows to access theBaseQueryResponseas anTable, which then can be used for indexing of results. (#3675)Change the APIs for
sunpy.map.GenericMap.draw_rectangleandsunpy.map.GenericMap.submap()to be consistent with each other and to use keyword-only arguments for specifying the bounding box. (#3677)Updates the
observer_coordinateproperty to warn the user of specific missing metadata for each frame. Omits warning about frames where all metadata is missing or all meta is present. (#3692)Added
sunpy.util.config.copy_default_configthat copies the default config file to the user’s config directory. (#3722)sunpy.databasenow supports adding database entries and downloading data fromHEKquery (#3731)Added a helper function (
get_rectangle_coordinates) for defining a rectangle in longitude and latitude coordinates. (#3737)Add a
.dataproperty inGenericTimeSeries, so that users are encouraged to useto_dataframe()to get the data of the timeseries. (#3746)It is now possible to turn on or off various corrections in
L0()(the apparent Carrington longitude of Sun-disk center as seen from Earth). (#3782)Made skimage.transform import lazy to reduce import time of
sunpy.image.transformby ~50% (#3818)Add support for parfive 1.1. This sets a limit on the number of open connections to JSOC when downloading files to 10. (#3822)
Fido clients (subclasses of
sunpy.net.base_client.BaseClient) can now register their own attrs modules withsunpy.net.attrs. This allows clients which require attr classes specific to that client to register modules that can be used by the user i.e.a.vso. It also allows clients implemented externally to sunpy to register attrs. (#3869)Added the methods
sunpy.map.GenericMap.quicklook()andsunpy.map.MapSequence.quicklook()to display an HTML summary of the instance, including interactive controls. When using Jupyter notebooks, this HTML summary is automatically shown instead of a text-only representation. (#3951)Added
_localfilelistmethod insunpy.util.scraper.Scraperto scrap local data archives. (#3994)Added extra constants to
sunpy.sun.constants:Longitude of the prime meridian (epoch J2000.0) :
sunpy.sun.constants.get('W_0')Sidereal rotation rate :
sunpy.sun.constants.sidereal_rotation_rateFirst Carrington rotation (JD TT) :
sunpy.sun.constants.first_carrington_rotationMean synodic period :
sunpy.sun.constants.mean_synodic_periodRight ascension (RA) of the north pole (epoch J2000.0) :
sunpy.sun.constants.get('alpha_0')Declination of the north pole (epoch J2000.0) :
sunpy.sun.constants.get('delta_0')(#4013)
Adds to
sunpy.util.scraper.Scraperthe ability to include regular expressions in the URL passed. (#4107)
Bug Fixes#
Added support for passing
TimeSeriesMetaDataobject totimeseries_factoryand associated validation tests. (#3639)Now when
GenericMapfails to load a file, the filename that failed to load will now be part of the error message. (#3727)Work around incorrect Content-Disposition headers in some VSO downloads, which were leading to mangled filenames. (#3740)
Fido.searchcan now service queries withouta.Timebeing specified. This is currently only used by thesunpy.net.jsoc.JSOCClient. (#3770)Fixed a bug with the calculation of Carrington longitude as seen from Earth where it was using an old approach instead of the current approach (for example, the varying Sun-Earth distance is now taken into account). The old approach resulted in errors no greater than 7 arcseconds in Carrington longitude when using
L0andHeliographicCarrington. (#3772)Updated
sunpy.map.CompositeMap.plotto support a linewidths argument. (#3792)Fix a bug in
sunpy.net.jsoc.JSOCClientwhere requesting data for export would not work if a non-time primekey was used. (#3825)Add support for passing paths of type
pathlib.Pathinsunpy.net.jsoc.JSOCClient.fetch. (#3838)Add explicit support for dealing with download urls for files, under ‘as-is’ protocol in
sunpy.net.jsoc.JSOCClient.get_request. (#3838)Updated the method used to filter time in the VSO post-search filtering function. (#3840)
Fix failing of fetching of the indexed JSOCResponses using
sunpy.net.fido_factory.UnifiedDownloaderFactory.fetch. (#3852)Prevented
sunpy.map.GenericMap.plotmodifying in-place any items passed asimshow_kwargs. (#3867)Changed the format of DATE-OBS in
sunpy.map.GenericMap.wcsfrom iso to isot (ie. with a “T” between the date and time) to conform with the FITS standard. (#3872)Fixed a minor error (up to ~10 arcseconds) in the calculation of the Sun’s position angle (
sunpy.coordinates.sun.P()). (#3886)Updated to HTTPS for HEK. (#3917)
The accuracy of the output of
sunpy.coordinates.ephemeris.get_horizons_coord()is significantly improved. (#3919)Fixed a bug where the longitude value for the reference coordinate in the Map repr would be displayed with the unintended longitude wrapping. (#3959)
It is now possible to specify a local file path to
sunpy.data.data_manager.DataManager.override_filewithout having to prefix it withfile://. (#3970)Closed the session in the destructor of VSOClient thus solving the problem of socket being left open (#3973)
Fixed a bug of where results of VSO searches would have inconsistent ordering in
sunpy.net.vso.vso.QueryResponseby always sorting the results by start time. (#3974)Fixes two bugs in
sunpy.util.deprecated: correctly calculates the removal version and does not override the default and/or alternative functionality message. Providing a custom deprecation message now suppresses any mention of the removal version. Additionally, apendingkeyword argument is provided to denote functions/classes that are pending deprecation. (#3982)Correctly generate labels for sliders in
~sunpy.visualization.animator.ArrayAnimatorWCSwhen the number of pixel dimensions and the number of world dimensions are not the same in the WCS. (#3990)Updated VSOClient.response_block_properties to check if “None” is in the return. (#3993)
Fix a bug with
sunpy.visualization.animator.ArrayAnimatorWCSwhere animating a line with a masked array with the whole of the initial line masked out the axes limits for the x axis were not correctly set. (#4001)Fixed passing in a list of URLs into
sunpy.map.GenericMap, before it caused an error due to the wrong type being returned. (#4007)Fixed a bug with
transform_with_sun_center()where the global variable was sometimes restored incorrectly. This bug was most likely encountered if there was a nested use of this context manager. (#4015)Fixes a bug in fido_factory to allow path=”./” in fido.fetch(). (#4058)
Prevented
sunpy.io.fits.header_to_fitsmodifying the passed header in-place. (#4067)Strip out any unknown unicode from the HEK response to prevent it failing to load some results. (#4088)
Fixed a bug in
get_body_heliographic_stonyhurst()that resulted in a error when requesting an array of locations in conjunction with enabling the light-travel-time correction. (#4112)sunpy.map.GenericMap.top_right_coordandcenterhave had their definitions clarified, and both have had off-by-one indexing errors fixed. (#4121)Fixed
sunpy.map.GenericMap.submap()when scaled pixel units (e.g.u.mpix) are used. (#4127)Fixed bugs in
sunpy.util.scraper.Scraper.filelistthat resulted in error when the HTML page of URL opened by the scraper contains some “a” tags without “href” attribute and resulted in incorrect file urls when any href stores filepath relative to the URL’s domain instead of just a filename. (#4132)Fixed inconsistencies in how
submapbehaves when passed corners in pixel and world coordinates. The behavior for submaps specified in pixel coordinates is now well-defined for pixels on the boundary of the rectangle and is consistent for all boundaries. Previously pixels on the lower left boundary were included, but excluded on the upper and right boundary. This means the shape of a submap may now be 1 pixel larger in each dimension. Added several more tests forsubmapfor a range of cutout sizes in both pixel and world coordinates. (#4134)sunpy.map.on_disk_bounding_coordinatesnow fully propagates the coordinate frame of the input map to the output coordinates. Previously only the observer coordinate, and no other frame attributes, were propagated. (#4141)Fix an off-by-one error in the reference pixel returned by
sunpy.map.header_helper.make_fitswcs_header(). (#4152)sunpy.map.GenericMap.reference_pixelnow uses zero-based indexing, in order to be consistent with the rest of thesunpy.mapAPI. (#4154)Previously
sunpy.map.GenericMap.resamplewithmethod='linear'was using an incorrect and constant value to fill edges when upsampling a map. Values near the edges are now correctly extrapolated using thefill_value=extrapolateoption toscipy.interpolate.interp1d. (#4164)Fixed a bug where passing an
intorlistvia thehduskeyword argument tosunpy.io.fits.readthrew an exception because the list of HDU objects was no longer of typeHDUList. (#4183)Fix attr printing when the attr registry is empty for that attr (#4199)
Improved the accuracy of
angular_radius()by removing the use of the small-angle approximation. The inaccuracy had been less than 5 milliarcseconds. (#4239)Fixed a bug with the
observerframe attribute for coordinate frames where an input that was not supplied as aSkyCoordwould sometimes result in a transformation error. (#4266)
Improved Documentation#
Fixed an issue with the scaling of class-inheritance diagrams in the online documentation by blocking the versions of graphviz containing a bug. (#3548)
A new example gallery example “Plotting a difference image” has been added, which can be used for base difference or running difference images. (#3627)
Removed obsolete Astropy Helpers submodule section in
CONTRIBUTING.rst; Also removed mentions of astropy_helpers in all files of the project. (#3676)Corrected misleading
TimeSeriesMetaDatadocumentation about optional parameters. (#3680)Added an example for
world_to_pixelfunction in the Units & Coordinates guide. (#3776)Added a page describing how SunPy calculates Carrington longitudes. (#3782)
Changed padding value of an example in the example gallery to fix the overlap of titles and x-label axes. (#3835)
More information and links about how to create changelogs. (#3856)
Clarified some inputs to
sunpy.map.GenericMap.plot. (#3866)Changed quoted sentence (that we suggest authors add to their research papers) in CITATION.rst (#3896)
Add example of how to use SunPy’s HEK client to search for the GOES flare event list. (#3953)
Improved the doc layout of
sunpy.data.sample. (#4034)Made improvements to STEREO starfield gallery example. (#4039)
Improved the documentation of
sunpy.map.GenericMap.resample. (#4043)Updated the STEREO starfield example to use all of the information in the star catalog. (#4116)
Mini-galleries are now easier to create in the documentation thanks to a custom Sphinx directive (
minigallery). The page Differential rotation using coordinate frames has an example of a mini-gallery at the bottom. (#4124)Added
sunpy.visualization.colormaps.color_tablesto the docs. (#4182)Made minor improvements to the map histogramming example. (#4205)
Add a warning to
sunpy.iodocs to recommend not using it for FITS (#4208)
Trivial/Internal Changes#
Removed un-used and un-tested code paths in the private
_remove_lytaf_eventsfunction insunpy.instr.lyra. (#3570)Removed
astropy_helpersand this means thatpython setup.py <test,build_docs>no longer works. So if you want to:Run the tests: Use
tox -e <env name>or callpytestdirectlyBuild the docs: Use
tox -e docsor cd into the docs folder and runmake htmlorsphinx-build docs docs/_build/html -W -b html -d docs/_build/.doctrees(#3598)
Cleaned up test warnings in sunpy.coordinates. (#3652)
Fix Python version for requiring importlib_resources (#3683)
sunpy.net.attr.AttrWalkerno longer usessunpy.util.multimethod.MultiMethodit uses a derivative offunctools.singledispatchsunpy.util.functools.seconddispatchwhich dispatches on the second argument. (#3714)Errors from a VSO search will now be raised to the user. (#3719)
Fixed the transformation test for
NorthOffsetFrame, which would intermittently fail. (#3775)earth_distance()is now computed without using coordinate transformations for better performance. (#3782)Created a helper function for testing the equality/closeness of longitude angles (i.e., angles with wrapping). (#3804)
Bump the astropy version figure tests are run with from 3.1.2 to 3.2.3 (#3925)
Used
urllib.parse.urlsplitinsunpy.util.scraperfor file scraping functionalities. (#3956)Added
sunpy.net.base_client.BaseClient.check_attr_types_in_queryas a helper method to check if a query contains a set of required attributes, and is a subset of optional attributes. (#3979)Removes appending login details for ftp urls from scraper. (#4020)
Re-factored the
sunpy.map.Mapfactory to dispatch argument parsing based on type. (#4037)Improved the error message raised by the Map factory when a map matches multiple source map types. (#4052)
Added log messages when the sample data fails to download. (#4137)
Remove an Astropy 3.1 compatibility wrapper for
Quantity.to_string. (#4172)Refactor the sphinx config to no longer depend on astropy-sphinx and more closely match the new sunpy package template (#4188)
1.1.0 (2020-01-10)#
Backwards Incompatible Changes#
The
sunpy.net.vso.vso.get_online_vso_urlfunction has been broken into two components, the newsunpy.net.vso.vso.get_online_vso_urlfunction takes no arguments (it used to take three) and now only returns an online VSO mirror or None. The construction of azeep.Clientobject is now handled bysunpy.net.vso.vso.build_clientwhich has a more flexible API for customising thezeep.Clientinterface. (#3330)Importing
sunpy.timeseries.timeseriesbaseno longer automatically imports Matplotlib. (#3376)sunpy.timeseries.sources.NOAAIndicesTimeSeries.peek()now checks that thetypeargument is a valid string, and raises aValueErrorif it isn’t. (#3378)Observer-based coordinate frames (
HeliocentricandHelioprojective) no longer assume a default observer (Earth) if no observer is specified. These frames can now be used with no observer specified, but most transformations cannot be performed for such frames. This removal of a default observer only affectssunpy.coordinates, and has no impact on the default observer insunpy.map. (#3388)The callback functions provided to
BaseFuncAnimatorbutton_funckeyword argument now take two positional arguments rather than one. The function signature is now(animator, event)where the first arg is the animator object, and the second is the matplotlib mouse event. (#3407)The colormap stored in SunPy’s Map subclasses (ie.
map.plot_settings['cmap']) can now be colormap string instead of the fullmatplotlib.colors.Colormapobject. To get the fullColormapobject use the new attributemap.cmap. (#3412)Fix a warning in
sunpy.map.GenericMap.rotatewhere the truth value of an array was being calculated. This changes the behaviour ofrotatewhen theangle=parameter is not anQuantityobject to raiseTypeErrorrather thanValueError. (#3456)
Deprecations and Removals#
Removed the step of reparing images (replacing non-finite entries with local mean) before coaligning them. The user is expected to do this themselves before coaligning images. If NaNs/non-finite entries are present, a warning is thrown. The function
sunpy.image.coalignment.repair_image_nonfiniteis deprecated. (#3287)The method to convert a
Helioprojectiveframe from 2D to 3D has been renamed fromcalculate_distancetomake_3d. This method is not typically directly called by users. (#3389)sunpy.visualization.animator.ImageAnimatorWCSis now deprecated in favour ofArrayAnimatorWCS. (#3407)sunpy.cmhas been moved tosunpy.visualization.colormapsand will be removed in a future version. (#3410)
Features#
Add a new
sunpy.data.managerandsunpy.data.cachefor dealing with versioned remote data within functions. Please see the Remote Data Manager guide. (#3124)Added the coordinate frames
HeliocentricEarthEcliptic(HEE),GeocentricSolarEcliptic(GSE),HeliocentricInertial(HCI), andGeocentricEarthEquatorial(GEI). (#3212)Added SunPy Map support for GOES SUVI images. (#3269)
Support APE14 for
ImageAnimatorWCSin SunPy’s visualization module (#3275)
Add ability to disable progressbars when downloading files using
sunpy.net.helioviewerand edited docstrings to mention this feature. (#3280)Adds support for searching and downloading SUVI data. (#3301)
Log all VSO XML requests and responses to the SunPy logger at the
DEBUGlevel. (#3330)Transformations between frames in
sunpy.coordinatescan now provide detailed debugging output. Set thelogginglevel toDEBUGto enable this output. (#3339)Added the
sunpy.coordinates.sun.carrington_rotation_timefunction to compute the time of a given Carrington rotation number. (#3360)A new method has been added to remove columns from a
sunpy.timeseries.GenericTimeSeries. (#3361)Add
shapeproperty to TimeSeries. (#3380)Added ASDF schemas for the new coordinate frames (
GeocentricEarthEquatorial,GeocentricSolarEcliptic,HeliocentricEarthEcliptic,HeliocentricInertial). See the gallery for an example of usingasdfto save and load a coordinate frame. (#3398)sunpy.visualization.animator.ArrayAnimatorWCSwas added which uses the WCS object to get the coordinates of all axes, including the slider labels. It also provides the ability to customise the plot by specifying arguments toWCSAxesmethods and supports animation of WCS aware line plots with Astroy 4.0. (#3407)The returned list of
Mapobjects is now sorted by filename when passing a directory or glob pattern toMapFactory. (#3408)Single character wildcards and character ranges can now be passed as glob patterns to
Map. (#3408)Mapnow accepts filenames and directories aspathlib.Pathobjects. (#3408)GenericMapobjects now have a.cmapattribute, which returns the fullColormap. object. (#3412)sunpy.io.write_file()now acceptsPathobjects as filename inputs. (#3469)sunpy.map.header_helper.make_fitswcs_header()now accepts atuplerepresenting the shape of an array as well as the actual array as thedataargument. (#3483)Made a couple of module imports lazy to reduce the import time of sunpy.map by ~40%. (#3495)
sunpy.map.GenericMap.wcsnow uses the full FITS header to construct the WCS. This adds support for instruments with more complex projections, such as WISPR, however does mean that Map will be more sensitive to incorrect or invalid FITS headers. If you are using custom headers with SunPy Map you might encounter issues relating to this change. (#3501)sunpy.visualization.animator.BaseFuncAnimatornow takes an optionalslider_labelskeyword argument which draws text labels in the center of the sliders. (#3504)Added a more helpful error message when trying to load a file or directory that doesn’t exist with
sunpy.map.Map. (#3568)Add
__repr__forMapSequenceobjects so that users can view the critical information of all theMapobjects, in a concise manner. (#3636)
Bug Fixes#
Fixed accuracy issues with the calculations of Carrington longitude (
L0) and Carrington rotation number (carrington_rotation_number). (#3178)Updated
sunpy.map.header_helper.make_fitswcs_header()to be more strict on the inputs it accepts. (#3183)Fix the calculation of
rsun_refinmake_fitswcs_header()and and ensure that the default reference pixel is indexed from 1. (#3184)Fixed the missing transformation between two
HeliographicCarringtonframes with different observation times. (#3186)sunpy.map.sources.AIAMapandsunpy.map.sources.HMIMapwill no longer assume the existence of certain header keys. (#3217)sunpy.map.header_helper.make_fitswcs_header()now supports specifying the map projection rather than defaulting toTAN. (#3218)Fix the behaviour of
sunpy.coordinates.frames.Helioprojective.calculate_distanceif the representation isn’t Spherical. (#3219)Fixed a bug where the longitude of a coordinate would not wrap at the expected angle following a frame transformation. (#3223)
Fixed a bug where passing a time or time interval to the differential rotation function threw an error because the new observer was not in HGS. (#3225)
Fixed bug where
get_horizons_coordwas unable to acceptTimearrays as input. (#3227)Fix the ticks on the default heliographic grid overlay so they are not white (and normally invisible) by default. (#3235)
Fixed a bug with
sunpy.net.hek.HEKClientwhen the results returned were a mixed dataset. (#3240)Fix
sunpy.physics.differential_rotation.differential_rotateto rotate in the correct direction and to account for the rotation of the heliographic coordinate frame with time. (#3245)Fixed a bug with the handling of changing observation times for transformations between
HCRSandHeliographicStonyhurst, which also indirectly affected other transformations when changing observation times. (#3246)Fixed all coordinate transformations to properly handle a change in observation time. (#3247)
Fixed the handling of coordinates with velocity information when transforming between Astropy frames and SunPy frames. (#3247)
Fixed
sunpy.physics.solar_rotation.calculate_solar_rotate_shiftso that it does not calculate a shift between the reference layer and itself, which would sometimes incorrectly result in a shift of a pixel due to numerical precision. (#3255)Stop crash when
LineAnimatoraxes_rangesentry given as1Darray when data is>1D, i.e. as an independent axis. (#3283)Fixed a
sunpy.coordinatesbug where a frame using the default observer of Earth could have its observer overwritten during a transformation. (#3291)Fixed a bug where the transformation from
HelioprojectivetoHeliocentricused the Sun-observer distance from the wrong frame when shifting the origin, and thus might not give the correct answer if the observer was not the same for the two frames. (#3291)Fixed a bug with the transformations between
HeliocentricandHeliographicStonyhurstwhen the frame observation time was not the same as the observer observation time. The most common way to encounter this bug was when transforming fromHelioprojectiveto any non-observer-based frame while also changing the observation time. (#3291)VSO client
fetchshould not download whenwaitkeyword argument is specified. (#3298)Fixed a bug with
solar_frame_to_wcs_mappingthat assumed that the supplied frame was a SunPy frame. (#3305)Fixed bugs with
solar_frame_to_wcs_mappingif the input frame does not include an observation time or an observer. (#3305)GreatArcnow accounts for the start and end points of the arc having different observers. (#3334)Fixed situations where 2D coordinates provided to
HeliographicStonyhurstandHeliographicCarringtonwere not converted to 3D as intended. Furthermore, the stored data will always be the post-conversion, 3D version. (#3351)Fix off by one error in
sunpy.map.header_helper.make_fitswcs_header()where when using the defaultreference_pixel=Nonekeyword argument the pixel coordinate of the reference pixel was off by +1. (#3356)Updated both GOES XRS and LYRA dataretriever clients to use
sunpy.util.scraper.Scraper, to make sure that files are actually on the servers being queried. (#3367)Fixing the ordering of lon and lat inputs into make_fitswcs_header (#3371)
Updated the URL for Fermi spacecraft-pointing files to use an HTTPS connection to HEASARC. (#3381)
Fixed a bug where permission denied errors when downloading files are very verbose by adding an error message in
fetch. (#3417)Fixed a malformed call to
astropy.time.Timein a test, which resulted in an incorrect time scale (UTC instead of TT). (#3418)Fix incorrect files being included in the tarball, and docs missing from the tarball (#3423)
Fixed a bug where clipping behavior had been enabled by default in the plotting normalizers for
Mapobjects. Clipping needs to be disabled to make use of the over/under/masked colors in the colormap. (#3427)Fix a bug with observer based frames that prevented a coordinate with an array of obstimes being transformed to other frames. (#3455)
sunpy.map.GenericMapwill no longer raise a warning if the position of the observer is not known for frames that don’t need an observer, i.e. heliographic frames. (#3462)Apply
os.path.expandusertosunpy.map.map_factory.MapFactoryinput before passing toglob.glob(#3477)Fix multiple instances of
sunpy.mapsources assuming the type of FITS Header values. (#3497)Fixed a bug with
NorthOffsetFramewhere non-spherical representations for the north pole produced an error. (#3517)Fixed
map.__repr__when the coordinate system information contained in theCUNIT1/2metadata is not set to a known value. (#3569)Fixed bugs with some coordinate transformations when
obstimeisNoneon the destination frame but can be assumed to be the same as theobstimeof the source frame. (#3576)Updated
sunpy.map.mapsequence.MapSequenceso that calling_derotate()raisesNotImplementedError. Added associated tests. (#3613)Fixed pandas plotting registration in
sunpy.timeseries. (#3633)Correctly catch and emit a warning when converting a map metadata to a FITS header and it contains a keyword with non-ascii characters. (#3645)
Improved Documentation#
Clean up the docstring for
sunpy.physics.differential_rotation.solar_rotate_coordinateto make the example clearer. (#2708)Added new gallery examples and cleaned up various gallery examples. (#3181)
Cleaned and expanded upon the docstrings for each Fido Client. (#3220)
Added clarifying hyperlinks to the gallery example
getting_lasco_observer_locationto link to astroquery docs page. (#3228)Added more details to docstrings in
sunpy.coordinates.frames. (#3262)Added a link to package maintainer list in the API Stability page. (#3281)
Improved the contributing guide by updating commands and highlighting text. (#3394)
Removing
.fitsfrom the end of path kwargs insunpy.net.fido_factory.UnifiedDownloaderFactory.fetchdocs to change output file extension from{file}.fits.fitsto{file}.fits. (#3399)A new example gallery section “Using SunPy with Other Packages” has been added, which contains a set of new examples using the reproject with solar data. (#3405)
Added a table of supported coordinate systems and other miscellaneous improvements to the coordinates documentation. (#3414)
Clarified the meaning of
sunpy.map.GenericMap.dsun. (#3430)Fixed the plots with multiple subplots in the
Mapuser guide to properly usewcsaxesand to be appropriately sized. (#3454)Fixed various issues with the gallery example of saving/loading coordinates using
asdf. (#3473)Added
sunpy.__citation__with a BibTex entry for citing sunpy. (#3478)Added an example showing how to display two maps and fade between them. (#3488)
Clarified the meaning of some
GenericMapobserver properties. (#3585)Added inherited members of
sunpy.mapclasses to the docs. (#3587)Fixed documentation of
sunpy.database.Database.searchby addingReturnsdocstring. (#3593)Updated the docstring for the parameter
sortbyinMapSequencewith the default value, valid value and how to disable sorting. (#3601)Updated the tour guide to reflect that the time series is not random data. (#3603)
Fixes bold type and extra line breaks of remote data manager example. (#3615)
Trivial/Internal Changes#
Allow running our sphinx-gallery examples as Jupyter notebooks via Binder (#3256)
Improve error messages and type checking in
sunpy.visualization.animator.image.ImageAnimatorWCS. (#3346)Copy the library
distrointosunpy/extern: replaces the deprecatedplatform/linux_distribution(#3396)The version of Matplotlib used to generate figure tests has been bumped from 3.0.3 to 3.1.1. (#3406)
Corrected spelling of ‘plotting’ in timeseries method (changed ‘ploting’ to ‘plotting’). (#3429)
Switched to “importlib_metadata” to get package version to speed up import of SunPy. (#3449)
Fix tests for
sunpy.data.data_managerand ensure they are correctly executed with pytest. (#3550)
1.0.0 (2019-06-01)#
Backwards Incompatible Changes#
Move the matplotlib animators from
sunpy.visualisation.imageanimatorandsunpy.visualization.mapcubeanimatortosunpy.visualization.animator. (#2515)Make
sunpy.time.parse_timereturnastropy.time.Timeinstead ofdatetime.datetime. (#2611)The properties and methods of
sunpy.time.TimeRangereturnsastropy.time.Timeandastropy.time.TimeDeltainstead ofdatetime.datetimeanddatetime.timedeltarespectively. (#2638)The
sunpy.instr.goesmodule now accepts and returnssunpy.timeseries.sources.XRSTimeSeriesobjects only. (#2666)obstimekeyword param ofsunpy.instr.goes._goes_lxtakes a non-scalarastropy.time.Timeobject instead ofnumpy.ndarray. The precision of times contained insunpy.timeserieshas been increased to 9 from 6. (#2676)Removed
sunpy.net.jsoc.attrs.Timebecause it served the same purpose assunpy.net.attrs.Timeafter the switch toastropy.time.Time. (#2694)Remove unused
**kwargswithin TimeSeries functions. (#2717)Rotation matrices inside map objects were previously stored as numpy matrices, but are now stored as numpy arrays, as numpy will eventually remove their matrix datatype. See https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html for more information. (#2719)
The
sunpy.cm.show_colormapsfunction now accepts the keyword ‘search’ instead of ‘filter’. (#2731)The keyword arguments to all client
.fetchmethods have been changed to support the new parfive downloader and to ensure consistency across all Fido clients. (#2797)The Helioviewer client has been switched to using the newer Helioviewer API. This has meant that we have changed some of the keywords that were passed into client’s methods. We have enforced that several keywords (observatory,instrument,detector,measurement) need to be defined otherwise the functions cannot return any data. (#2801)
Maps no longer assume that the pixel units are arcseconds if the units aren’t explicitly set. In addition to this if critical metadata is missing from when creating a map, the map will fail to initialize and will raise an error. (#2847)
axis_ranges kwarg of
sunpy.visualization.animator.base.ArrayAnimator,sunpy.visualization.animator.image.ImageAnimatorandsunpy.visualization.animator.line.LineAnimatornow must be entered as None, [min, max] or pixel edges of each array element. Previously, pixel centers were expected. This change removes ambiguity in interpretation and ensures the extent of the plot can always be accurately derived. (#2867)All keywords have been added (with defaults) to each
~sunpy.net.helioviewer.HelioviewerClientfunction. This means that there will be some changes to the style of the PNG screenshot that is returned. Returns for the JPEG 2000 and the other functions should be the same but not guaranteed. (#2883)Changed
sunpy.sun.models.interiorandsunpy.sun.models.evolutionfrompandas.DataFrametoastropy.table.QTable(#2936)Minimum numpy version is now >=1.14.5 (#2954)
Removed
sunpy.time.julian_day,sunpy.time.julian_centuries,sunpy.time.day_of_year,sunpy.time.break_time,sunpy.time.get_day. (#2999)Updated the solar values in
sunpy.sun.constantsto IAU 2015 values. (#3001)Renamed
eccentricity_sunearth_orbittoeccentricity_sun_earth_orbit. (#3001)Renamed
sunpy.image.rescaletosunpy.image.resample. (#3044)Remove the
basic_plotkeyword argument frompeek. An example has been added to the gallery showing how to make a plot like this. (#3109)sunpy.map.GenericMapwill no longer use the keysolar_b0as a value for heliographic latitude. (#3115)sunpy.map.GenericMapnow checks for a complete observer location rather than individually defaulting coordinates (lat, lon, distance) to Earth position. If any one of the three coordinates is missing from the header the observer will be defaulted to Earth and a warning raised. (#3115)sunpy.sun.sunfunctions have been re-implemented using Astropy for significantly improved accuracy. Some functions have been removed. (#3137)All of the functions in
sunpy.sun.sunand all of the Sun-specific functions insunpy.coordinates.ephemerishave been moved to the new modulesunpy.coordinates.sun. (#3163)
Deprecations and Removals#
The deprecated
sunpy.lightcurve,sunpy.wcsandsunpy.spectramodules have now been removed. (#2666)sunpy.instr.rhessi.get_obssumm_dbase_filesunpy.instr.rhessi.get_obssum_filename,sunpy.instr.rhessi.get_obssumm_filehave been removed.Fidoshould be used to download these files. (#2808)Removed
heliographic_solar_centerin favour ofsunpy.coordinates.get_sun_L0andsunpy.coordinates.get_sun_B0(#2830)Removed
GenericClient.queryin favour ofsunpy.net.dataretriever.GenericClient.search(#2830)Removed
sunearth_distancein favour ofget_sunearth_distance(#2830)Removed
remove_lytaf_events_from_lightcurvein favour ofsunpy.instr.lyra.remove_lytaf_events_from_timeseries(#2830)Removed
sunpy.cm.get_cmapin favour ofplt.get_cmap(#2830)Removed
database.queryin favour ofsunpy.database.Database.search(#2830)Removed
sunpy.net.vso.InteractiveVSOClient(#2830)Removed
MapCubein favour ofMapSequence(#2830)Removed
solar_northin favour ofget_sun_P(#2830)Removed
database.downloadin favour ofsunpy.database.Database.fetch(#2830)Removed
sunpy.map.GenericMap.pixel_to_datain favour ofsunpy.map.GenericMap.pixel_to_world(#2830)Removed
GenericClient.getin favour ofsunpy.net.dataretriever.GenericClient.fetch. This changes applies to the other clients as well. (#2830)Removed
Map.xrangeandMap.yrange(#2830)Removed
sunpy.net.attrs.Wavein favour ofsunpy.net.vso.attrs.Wavelength(#2830)Removed
JSOCClient.check_requestin favour ofdrms.ExportRequest.status(#2830)sunpy.net.vso.VSOClient.query_legacyandsunpy.net.vso.VSOClient.latesthave been deprecated as we strongly recommend people usesunpy.net.Fidofor all queries. (#2866)The deprecated
sunpy.physics.transformsmodule has been removed, it is replaced bysunpy.physics.solar_rotationandsunpy.physics.differential_rotation. (#2994)Removed
sunpy.sun.sun.solar_cycle_numberbecause it was fundamentally flawed (#3150)
Features#
Change arguments to
sunpy.testfromoffline=andonline=toonlineandonline_only. This matches the behavior of the figure keyword arguments and comes as a part of a move to using a modified version of the Astropy test runner. (#1983)asdf schemas and tags were added for the SunPy coordinate frames and
GenericMapallowing these objects to be saved to and restored from asdf files. (#2366)The images from image tests are now saved in a local folder for easy access. (#2507)
sunpy.map.MapCubehas been renamed tosunpy.map.MapSequenceto better reflect its use as a collection of map objects. (#2603)Net search attributes now support tab completion of values and display a table of possible values when printed, to allow easier discoverability of possible search values. (#2663)
Running the figure tests now creates a page showing the differences between the expected figures and the figures produced from running the tests. (#2681)
Add support for Dask arrays in
sunpy.map.Map. The map factory now checks a whitelist of array types rather than strictly checking if the array is of typenumpy.ndarray. (#2689)Persist the name of a coordinate, i.e. “earth” even though a concrete coordinate object has been calculated and use this string representation to change the way the sunpy frames are printed. This is primarily to facilitate displaying the name of the body rather than the concrete coordinate when printing a
SkyCoord. (#2723)searchnow returns anastropy.table.Tableinstead of list of adict. (#2759)Add a downscaled HMI image to the sample data. (#2782)
Now able to create a
sunpy.map.Mapusing an array and aastropy.wcs.WCSobject. (#2793)The download manager for
Fidohas been replaced with parfive. This provides advanced progress bars, proper handling of overwriting and the ability to retry failed downloads. (#2797)sunpy.map.GenericMapcan now save out rice compressed FITS files. (#2826)Now any SunPyDeprecationWarnings will cause an error when using pytest. (#2830)
Added full Tox support for SunPy tests, documentation build and figure tests. (#2839)
Transition the
sunpy.net.vso.VSOClientfrom using suds to zeep as the SOAP library. This is a more actively maintained library, and should provide better support for the VSOs https endpoints. This change should have no effect on the public API of thesunpy.net.vso.VSOClient. (#2866)Provided access to the Helioviewer header information using
~sunpy.net.helioviewer.HelioviewerClient.get_jp2_headerfunction. (#2904)Add a new WSDL URL and port to support SunPy use of VSO instance at SDAC. (#2912)
Add support for COSMO K-Coronograph (KCOR) FITS data. (#2916)
Add logger messaging system based on
AstropyLogger, cleaned up all warnings, removed all print statements. (#2980)The function
sunpy.image.coalignment.get_correlation_shiftsnow issues an error when the number of dimensions are not correct instead of a warning and returning None. (#2980)The default location of the sunpy sample data has changed to be in the platform specific data directory as provided by appdirs. (#2993)
Add timeseries support for EVE/ESP level 1 data in
sunpy.timeseries.sources(#3032)The default style for Map plots have changed to reflect the changes in Astropy 3.2. (#3054)
sunpy.coordinates.ephemeris.get_body_heliographic_stonyhurstcan now account for light travel time when computing the (apparent) body position, as long as the observer location is provided. (#3055)Added a helper function (
sunpy.map.header_helper.make_fitswcs_header()) that allows users to create a meta header for custom createdsunpy.map.GenericMap. (#3083)Map plotting now accepts the optional keyword
clip_intervalfor specifying a percentile interval for clipping. For example, if the interval (5%, 99%) is specified, the bounds of the z axis are chosen such that the lowest 5% of pixels and the highest 1% of pixels are excluded. (#3100)The new function
get_horizons_coordenables querying JPL HORIZONS for the locations of a wide range of solar-system bodies, including spacecraft. (#3113)
Bug Fixes#
Fix the bug that prevented VSO queries for HMI data from downloading file without specifying
a.Physobs. (#2621)Fix
sunpy.map.mapcube.MapCube.plot. The code had not been updated to support the changes to the wcsaxes helper functions. (#2627)Replace all use of the deprecated
sunpy.cm.get_cmapwithmatplotlib.cm.get_cmapto prevent deprecation warnings being raised. (#2635)Fix generation of the coordinate transformation graph with Astropy 3.1.dev (#2636)
Prevent helioviewer from erroring when downloading file to a directory that does not exist. It will now create the directory when required. (#2642)
Fix transformations into/out of Heliographic Stonyhurst frame when the coordinate representation is Cartesian. (#2646)
Running the figure tests with
setup.py testnow saves the figures and the hashes to the same directory as setup.py. (#2658)sunpy.instr.fermi.met_to_utcnow returns the correct utc time which takes into account the leap seconds that have passed. (#2679)Support passing Python file objects to
sunpy.io.fits.write. (#2688)Added DRMS to setup.py so sunpy[all] installs it as a dependency. (#2693)
Fix eve 0cs timeseries separator regex to support Python 3.7 (#2697)
Fix the bug which crashes
LASCOMapfor when ‘date-obs’ is reformatted again from a self applied function. (#2700)Change all instances of quantity_allclose to
astropy.units.allclosethis prevents pytest being needed to importsunpy.coordinateson Astropy 3 (#2701)Fix RHESSI obssum file downloading to include the final day in the time range. (#2714)
Raise an error when transforming between HPC and HCC frames if the observer is not the same. (#2725)
Replaces the existing LASCO C2 and C3 color maps with new ones that perform better with JP2 and Level 0.5, 1 data. (#2731)
Do not attempt to save a FITS header comment for a keyword which is not in the header. This prevents an error on saving some maps after the metadata had been modified but not the comments. (#2748)
Add support for
HMIMapobjects as input tosunpy.instr.aia.aiaprep. (#2749)User can convert between HPC and HCC coordinates with different observers. This is implemented by automatically transforming the coordinate into HGS and then changing observer, and then transforming back to HCC. (#2754)
Changed default file type for Helioviewer to prevent decode errors. (#2771)
Increase figure size to avoid cutting off longer colormap names in
sunpy.cm.show_colormaps. (#2824)The sample data directory will no longer be created until files are downloaded to it. (#2836)
Timeseries and lightcurve will now respect updated config values for download directory. (#2844)
Always use _default_wrap_angle rather than hard coding a wrap angle in the init of a sunpy coordinate frame (#2853)
Ensure imageanimators only slice arrays with integers (#2856)
Fixed
sunpy.io.fits.writeto handle the keywordCOMMENTcorrectly. (#2880)If Carrington longitude (“crln_obs”) is found in the FITS header,
Mapconverts this to the correct Heliographic longitude. (#2946)sunpy.net.helio.hec.HECClient.time_querynow resolves the correct input time format. (#2969)Fixes the calculation of the solar rotation of coordinates and the differential rotation of
sunpy.map.GenericMap. (#2972)Added back the FERMI GBM client to
sunpy.net.dataretriever. (#2983)Fix bug in
sunpy.net.hekwhich raised and error if a search returned zero results, now returns an emptysunpy.net.hek.HEKTable. (#3046)AIAMapnow uses the provided HAE coordinates instead of the provided HGS coordinates to determine the observer location. (#3056)Correctly zero pad milliseconds in the
sunpy.util.scraper.Scraperformatting to prevent errors when the millisecond value was less than 100. (#3063)Fix
sunpy.util.scraper.Scraperfailing if a directory is not found on a remote server. (#3063)Correctly extract observer location from MDI and EIT data (#3067)
Fix HGS <> HCRS test due to Ecliptic frame changes in astropy 3.2 (#3075)
Fixes bug when creating a timeseries from a URL and bug when creating a TimeSeries from older GOES/XRS fits files. (#3081)
Added
rsun_obs. It returns a quantity in arcsec consistent with othersunpy.map.GenericMapand overwrites mapbase’s assumption of a photospheric limb as seen from Earth. (#3099)Fixed bugs related to using
plotandpeekwith theinlineMatplotlib backend in Jupyter notebook. (#3103)Make a correction to
sunpy.coordinates.wcs_utils.solar_wcs_frame_mappingso thatastropy.wcs.WCSobjects are correctly converted tosunpy.coordinates.framesobjects irrespective of the ordering of the axes. (#3116)The
solar_rotate_coordinatefunction returns a coordinate that accounts for the location of the new observer. (#3123)Add support for rotation parameters to
sunpy.map.header_helper.make_fitswcs_header(). (#3139)Improve the implementation of
differential_rotatethe image warping when transforming Maps for differential rotation and change in observer position. (#3149)Fix a bug where new helioviewer sources potentially cause
~sunpy.net.helioviewer.HelioviewerClient.data_sourcesto error. (#3162)
Improved Documentation#
Organise the gallery into sections based on example type and tidy up a little. (#2624)
Added gallery example showing the conversion of Helioprojective Coordinates to Altitude/Azimuth Coordinates to and back. (#2656)
Add contribution guidelines for the sunpy example gallery. (#2682)
Added a gallery example for “Downloading and plotting a HMI image” and “Creating a Composite map”. (#2746)
Added an example for
sunpy.visualization.animator.ImageAnimatorWCS. (#2752)Minor changes to the developer guide regarding sprint labels. (#2765)
Copyedited and corrected the solar cycles example. (#2770)
Changed “online” mark to “remote_data” and made formatting of marks consistent. (#2799)
Add a missing plot to the end of the units and coordinates guide. (#2813)
Added gallery example showing how to access the SunPy colormaps (#2865)
Added gallery example showing how to access the SunPy solar physics constants. (#2882)
Major clean up of the developer documentation. (#2951)
Overhaul of the install instructions for the guide section of our documentation. (#3147)
Trivial/Internal Changes#
parse_timenow usessingledispatchunderneath. (#2408)Revert the handling of
quantity_allclosenow that astropy/astropy#7252 is merged. This also bumps the minimum astropy version to 3.0.2. (#2598)Replace the subclasses of matplotlib Slider and Button in
sunpy.visualizationwith partial functions. (#2613)Sort the ana C source files before building to enable reproducible builds. (#2637)
We are now using towncrier to generate our changelogs. (#2644)
Moved figure tests to Python 3.6. (#2655)
Removed old metaclass used for Map and TimeSeries as we have now moved to Python 3.6. (#2655)
Updated astropy_helpers to v3.0.2. (#2655)
When running image tests, a comparison HTML page is now generated to show the generated images and expected images. (#2660)
Change to using pytest-cov for coverage report generation to enable support for parallel builds (#2667)
Use of
textwrapto keep source code indented when multiline texts is used (#2671)Fix misspelling of private attribute
_default_heliographic_latitudein map. (#2730)Miscellaneous fixes to developer docs about building sunpy’s documentation. (#2825)
Changed
sunpy.instr.aia.aiaprepto update BITPIX keyword to reflect the float64 dtype. (#2831)Remove warning from
GenericMap.submapwhen using pixelQuantitiesas input. (#2833)Remove the usage of six and all
__future__imports (#2837)Fix SunPy Coordinate tests with Astropy 3.1 (#2838)
Stores entries from directories into database sorted by name. It adds mocks to the database user guide examples. (#2873)
Fix all DeprecationWarning: invalid escape sequence. (#2885)
Used
unittest.mockfor creating offline tests for simulating online tests fortest_noaa.py(#2900)Fix support for pip 19 and isolated builds (#2915)
Moved to using AppDirs as the place to host our configuration file. (#2922)
Users can now use fewer keywords in our
~sunpy.net.helioviewer.HelioviewerClientto access the available sources. Either byobservatoryandmeasurementorinstrumentandmeasurementas this much information is enough to get the source ID for most of the cases. (#2926)Remove the pytest dependency on the
GenericMapasdf tag. (#2943)Fix initialization of
VSOClientwhen no WSDL link is found. (#2981)
0.9.0#
New Features#
Added TimeUTime class to support utime. [#2409]
Example for fine-grained use of ticks and grids [#2435]
Maintiners Workflow Guide [#2411]
Decorator to append and/or prepend doc strings [#2386]
Adding
python setup.py test --figure-only[#2557]Fido.fetch now accepts pathlib.Path objects for path attribute.[#2559]
The
HeliographicStonyhurstcoordinate system can now be specified using a cartesian system, which is sometimes known as the “Heliocentric Earth equatorial” (HEEQ) coordinate system. [#2437]
API Changes#
sunpy.coordinates.representationhas been removed. Longitude wrapping is now done in the constructor of the frames. [#2431]Propagation of
obstimein the coordinate frame transformation has changed, this means in general when transforming directly between frames (notSkyCoord) you will have to specifyobstimein more places. [#2461]Transforming between Heliographic Stonyhurst and Carrington now requires that
obstimebe defined and the same on both the input and output frames. [#2461]Removed the figure return from .peek() [#2487]
Bug Fixes#
Improve TimeSeriesBase docstring [#2399]
Validate that pytest-doctestplus is installed [#2388]
Fix use of self.wcs in plot in mapbase [#2398]
Updated docstring with pointer to access EVE data for other levels [#2402]
Fix broken links and redirections in documentation [#2403]
Fixes Documentation changes due to NumPy 1.14 [#2404]
Added docstrings to functions in download.py [#2415]
Clean up database doc [#2414]
rhessi.py now uses sunpy.io instead of astropy.io [#2416]
Remove Gamma usage in Map [#2424]
Changed requirements to python-dateutil [#2426]
Clarify coordinate system definitions [#2429]
Improve Map Peek when using draw_grid [#2442]
Add HCC –> HGS test [#2443]
Testing the transformation linking SunPy and Astropy against published values [#2454]
Fixed title bug in sunpy.timeseries.rhessi [#2477]
Allow LineAnimator to accept a varying x-axis [#2491]
Indexing Bug Fix to LineAnimator [#2560]
Output sphinx warnings to stdout [#2553]
Docstring improvement for LineAnimator [#2514]
move the egg_info builds to circleci [#2512]
Added tests for TraceMap [#2504]
Fix HGS frame constructor and HPC
calculate_distancewith SkyCoord constructor. [#2463]removed
wavelnthkeyword in meta desc of Maps to avoid using non standard FITS keyword likenan[#2456]The documentation build now uses the Sphinx configuration from sphinx-astropy rather than from astropy-helpers.[#2494]
Migrate to hypothesis.strategies.datetimes [#2368]
Prevent a deprecation warning due to truth values of Quantity [#2358]
Print a warning when heliographic longitude is set to it’s default value of 0 [#2480]
parse_time now parses numpy.datetime64 correctly. [#2572]
0.8.5#
Bug Fixes#
Removed AstropyDeprecationWarning from sunpy.coordinates.representation [#2476]
Fix for NorthOffsetFrame under Astropy 3.0 [#2486]
Fix lightcurve tests under numpy dev [#2505]
Updated depecration link of radiospectra [#2481]
Fixed Padding values in some of the documentation pages [#2497]
Move documentation build to circleci [#2509]
Fix Issue #2470 hgs_to_hcc(heliogcoord, heliocframe) [#2502]
Fixing CompositeMap object so that it respects masked maps [#2492]
0.8.4#
Bug Fixes#
Improve detection of
SkyCoordframe instantiation when distance is1*u.one. This fixes a plotting bug withWCSAxesin Astropy 3.0 [#2465]removed
wavelnthkeyword in meta desc of Maps to avoid using non standard FITS keyword likenan[#2427]Change the default units for HPC distance from
u.kmtoNone. [#2465]
0.8.3#
Bug Fixes#
XRSClientnow reports time ranges of files correctly. [#2364]Make parse_time work with datetime64s and pandas series [#2370]
CompositeMap axes scaling now uses map spatial units [#2310]
Moved license file to root of repository and updated README file [#2326]
Fix docstring formatting for net.vso.attrs [#2309]]
Fix coloring of ticks under matplotlib 2.0 default style [#2320]
Always index arrays with tuples in
ImageAnimator[#2320]Added links to possible attrs for FIDO in guide [#2317] [#2289]
Updated GitHub Readme [#2281] [#2283]
Fix matplotlib / pandas 0.21 bug in examples [#2336]
Fixes the off limb enhancement example [#2329]
Changes to masking hot pixels and picking bright pixels examples [#2325] [#2319]
Travis CI fix for numpy-dev build [#2340]
Updated masking brightest pixel example [#2338]
Changed TRAVIS cronjobs [#2338]
Support array values for
obstimefor coordinates and transformations [#2342] [#2346]Updated Gallery off limb enhance example [#2337]
Documentation fixes for VSO [#2354] [#2353]
All tests within the documentation have been fixed [#2343]
Change to using pytest-remotedata for our online tests [#2345]
Fixed upstream astropy/numpy documentation issues [#2359]
Documentation for Map improved [#2361]
Fix the output units of pixel_to_world [#2362]
Documentation for Database improved [#2355]
Added test for mapsave [#2365]
Documentation for Sun improved [#2369]
0.8.2#
Bug Fixes#
Shows a warning if observation time is missing [#2293]
Updates MapCube to access the correct properties of the namedtuple SpatialPair [#2297]
0.8.1#
Bug fixes#
Fixed TimeSeries test failures due to missing test files [#2273]
Refactored a GOES test to avoid a Py3.6 issue [#2276]
0.8.0#
New Features#
Solar differential rotation for maps and submaps included.
Solar rotation calculation and mapcube derotation now use sunpy coordinates.
Sample data now downloads automatically on import if not available and is now pluggable so can be used by affiliated packages. Shortcut names have been normalized and all LIGHTCURVE shortcuts have changed to TIMESERIES.
Calculation of points on an arc of a great circle connecting two points on the Sun.
Removed
extract_timefunction fromsunpy.timeand also tests related to the function fromsunpy.time.testsUser can now pass a custom time format as an argument inside
sunpy.database.add_from_dir()in case thedate-obsmetadata cannot be read automatically from the files.Add time format used by some SDO HMI FITS keywords
Now the
sunpy.database.tables.display_entries()prints an astropy table.Additional methods added inside the
sunpy.databaseclass to make it easier to display the database contents.Remove unused
sunpy.visualization.plottingmodulePort the pyana wrapper to Python 3
Map.peek(basic_plot-True)no longer issues warningsRemove the
sunpy.map.nddata_compatmodule, this makesMap.dataandMap.metaread only.Add a
NorthOffsetFrameclass for generating HGS-like coordinate systems with a shifted north pole.Remove deprecated
VSOClient.showmethod.Deprecate
sunpy.wcs:sunpy.coordinatesandsunpy.mapnow provide all that functionality in a more robust manner.Added hdu index in
sunpy.database.tables.DatabaseEntryas a column in the table.Removed
HelioviewerClientfrom thesunpy.netnamespace. It should now be imported withfrom sunpy.net.helioviewer import HelioviewerClient.Removed compatibility with standalone
wcsaxesand instead depend on the version in astropy 1.3. SunPy now therefore depends on astropy>-1.3.Update to
TimeRange.__repr__; now includes the qualified name andidof the object.A new
sunpy.visualization.imageanimator.LineAnimatorclass has been added to animate 1D data. This has resulted in API change for thesunpy.visualization.imageanimator.ImageAnimatorclass. The updateimage method has been renamed to update_plot.Drop support for Python 3.4.
SunPy now requires WCSAxes and Map.draw_grid only works with WCSAxes.
HelioprojectiveandHelioCentricframes now have anobserverattribute which itself is a coordinate object (SkyCoord) instead ofB0,L0andD0to describe the position of the observer.GenericMap.draw_gridnow usesWCSAxes, it will only work on aWCSAxesplot, this may be less performant than the previous implementation.GenericMap.world_to_pixelandGenericMap.pixel_to_worldnow accept and returnSkyCoordobjects only.GenericMaphas a new propertyobserver_coordinatewhich returns aSkyCoorddescribing the position of the observer.GenericMap.submapnow takes arguments of the formbottom_leftandtop_rightrather thanrange_aandrange_b. This change enables submap to properly handle rotated maps and take input in the form ofSkyCoordobjects.When referring to physical coordinates
Pair.xhas been replaced withSpatialPair.axis1. This means values returned byGenericMapnow differentiate between physical and pixel coordinates.The physical radius of the Sun (length units) is now passed from Map into the coordinate frame so a consistent value is used when calculating distance to the solar surface in the
HelioprojectiveFramecoordinate frame.A new
sunpy.visualization.imageanimator.ImageAnimatorWCSclass has been added to animate N-Dimensional data with the associated WCS object.Moved Docs to docs/ to follow the astropy style
Added SunPy specific warnings under util.
SunPy coordinate frames can now be transformed to and from Astropy coordinate frames
The time attribute for SunPy coordinate frames has been renamed from
dateobstoobstimeEphemeris calculations with higher accuracy are now available under
sunpy.coordinates.ephemerisAdd support for SunPy coordinates to specify observer as a string of a major solar-system body, with the default being Earth. To make transformations using an observer specified as a string,
obstimemust be set.Added VSO query result block level caching in the database module. This prevents re-downloading of files which have already been downloaded. Especially helpful in case of overlapping queries.
Change the default representation for the Heliographic Carrington frame so Longitude follows the convention of going from 0-360 degrees.
All Clients that are able to search and download data now have a uniform API that is
searchandfetch. The older functions are still there but are deprecated for 0.8.
Bug fixes#
Add tests for RHESSI instrument
Maps from Helioviewer JPEG2000 files now have correct image scaling.
Get and set methods for composite maps now use Map plot_settings.
Simplified map names when plotting.
Fix bug in
wcs.convert_data_to_pixelwhere crpix[1] was used for both axes.Fix some leftover instances of
GenericMap.unitsFixed bugs in
sunequationssunpy.io.fits.readwill now return any parse-able HDUs even if some raise an error.VSOClientno longer prints a lot of XML junk if the query fails.Fix Map parsing of some header values to allow valid float strings like ‘nan’ and ‘inf’.
Fix Map parsing of some header values to allow valid float strings like ‘nan’ and ‘inf’.
0.7.8#
The SunPy data directory “~/sunpy” is no longer created until it is used (issue #2018)
Change the default representation for the Heliographic Carrington frame so Longitude follows the convention of going from 0-360 degrees.
Fix for surface gravity unit.
Support for Pandas 0.20.1
0.7.7#
Fix errors with Numpy 1.12
0.7.6#
Add Astropy 1.3 Support
0.7.5#
Fix test failure (mapbase) with 1.7.4
Restrict supported Astropy version to 1.0<astropy<1.3
Add Figure test env to SunPy repo.
0.7.4#
Remove Map always forcing warnings on.
Map.centernow usesMap.wcsto correctly handle rotation.Fix link in coordinates documentation.
Update helioviewer URL to HTTPS (fixes access to Helioviewer).
Fix processing of TRACE and YOHKOH measurement properties.
Remove warnings when using
Map.peek(basic_plot-True)Update docstrings for HPC and HCC frames.
0.7.3#
Fix ConfigParser for Python 3.5.2 - This allows SunPy to run under Python 3.5.2
Fix incorrect ordering of keys in
MapMetaAdd
sunpy.util.scraperto the API documentation.
0.7.2#
Fixed bugs in
sunequations
0.7.1#
Fix bug in
wcs.convert_data_to_pixelwhere crpix[1] was used for both axes.Fix some leftover instances of
GenericMap.unitsFixed bugs in
sunequationsNow the
sunpy.database.tables.display_entries()prints an astropy table.Additional methods added inside the
sunpy.databaseclass to make it easier to display the database contents.sunpy.io.fits.readwill now return any parse-able HDUs even if some raise an error.VSOClientno longer prints a lot of XML junk if the query fails.Remove unused
sunpy.visualization.plottingmoduleMap.peek(basic_plot-True)no longer issues warningsRemove the
sunpy.map.nddata_compatmodule, this makesMap.dataandMap.metaread only.Add a
NorthOffsetFrameclass for generating HGS-like coordinate systems with a shifted north pole.Remove deprecated
VSOClient.showmethod.Deprecate
sunpy.wcs:sunpy.coordinatesandsunpy.mapnow provide all that functionality in a more robust manner.Added hdu index in
sunpy.database.tables.DatabaseEntryas a column in the table.Removed
HelioviewerClientfrom thesunpy.netnamespace. It should now be imported withfrom sunpy.net.helioviewer import HelioviewerClient.Removed compatibility with standalone
wcsaxesand instead depend on the version in astropy 1.3. SunPy now therefore depends on astropy>-1.3.Update to
TimeRange.__repr__; now includes the qualified name andidof the object.Change the default representation for the Heliographic Carrington frame so Longitude follows the convention of going from 0-360 degrees.
Fix Map parsing of some header values to allow valid float strings like ‘nan’ and ‘inf’.
0.7.0#
Fixed test failures with numpy developer version.[#1808]
Added
timeoutparameter insunpy.data.download_sample_data()Fixed
aiaprepto return properly sized map.Deprecation warnings fixed when using image coalignment.
Sunpy is now Python 3.x compatible (3.4 and 3.5).
Added a unit check and warnings for map metadata.
Added IRIS SJI color maps.
Updated
show_colormaps()with new string filter to show a subset of color maps.Fixed MapCube animations by working around a bug in Astropy’s ImageNormalize
Remove
vso.QueryResponse.num_records()in favour oflen(qr)Add a
draw_rectanglehelper toGenericMapwhich can plot rectangles in the native coordinate system of the map.Added the ability to shift maps to correct for incorrect map location, for example.
Bug fix for RHESSI summary light curve values.
Mapcube solar derotation and coalignment now pass keywords to the routine used to shift the images, scipy.ndimage.interpolation.shift.
Add automatic registration of
GenericMapsubclasses with the factory as long as they define anis_datasource_formethod.Added functions
flareclass_to_fluxandflux_to_flareclasswhich convert between GOES flux to GOES class numbers (e.g. X12, M3.4).Removed old
sunpy.util.goes_flare_class()Bug fix for RHESSI summary light curve values.
The
MapCube.as_arrayfunction now returns a masked numpy array if at least one of the input maps in the MapCube has a mask.Map superpixel method now respects maps that have masks.
Map superpixel method now accepts numpy functions as an argument, or any user-defined function.
Map superpixel method no longer has the restriction that the number of original pixels in the x (or y) side of the superpixel exactly divides the number of original pixels in the x (or y) side of the original map data.
sunpy.physics.transformshas been deprecated and the code moved intosunpy.physics.Add the
sunpy.coordinatesmodule, this adds the core physical solar coordinates frame within the astropy coordinates framework.Added ability of maps to draw contours on top of themselves (
draw_contours)Added concatenate functionality to lightcurve base class.
Fix Map to allow astropy.io.fits Header objects as valid input for meta arguments.
Added an examples gallery using
sphinx-gallery.API clean up to constants. Removed constant() function which is now replaced by get().
Prevent helioviewer tests from checking access to the API endpoint when running tests offline.
GenericMap.unitsis renamed toGenericMap.spatial_unitsto avoid confusion withNDData.unit.GenericMapnow has acoordinate_frameproperty which returns anastropy.coordinatesframe with all the meta data from the map populated.GenericMapnow has a_mpl_axesmethod which allows it to be specified as a projection tomatplotlibmethods and will return aWCSAxesobject withWCSprojection.
0.6.5#
The draw_grid keyword of the peek method of Map now accepts booleans or astropy quantities.
Fix bug in
wcs.convert_data_to_pixelwhere crpix[1] was used for both axes.Fixed bugs in
sunequations
0.6.4#
Bug fix for rhessi summary lightcurve values.
Fix docstring for
pixel_to_dataanddata_to_pixel.Fix the URL for the Helioviewer API. (This fixes Helioviewer.)
Fix the way
reshape_image_to_4d_superpixelchecks the dimension of the new image.Fix Map to allow astropy.io.fits Header objects as valid input for meta arguments.
Prevent helioviewer tests from checking access to API when running tests in offline mode.
0.6.3#
Change setup.py extras to install suds-jurko not suds.
0.6.2#
Changed start of GOES 2 operational time range back to 1980-01-04 so data from 1980 can be read into GOESLightCurve object
Fix bug with numpy 1.10
update astropy_helpers
Added new sample data
0.6.1#
Fixed MapCube animations by working around a bug in Astropy’s ImageNormalize
Small fix to RTD builds for Affiliated packages
SunPy can now be installed without having to install Astropy first.
MapCubes processed with
coalignment.apply_shiftsnow have correct metadata.Multiple fixes for WCS transformations, especially with solar-x, solar-y CTYPE headers.
0.6.0#
Enforced the use of Astropy Quantities through out most of SunPy.
Dropped Support for Python 2.6.
Remove old style string formatting and other 2.6 compatibility lines.
Added vso like querying feature to JSOC Client.
Refactor the JSOC client so that it follows the .query() .get() interface of VSOClient and UnifedDownloader.
Provide
__str__and__repr__methods on vsoQueryResponsedeprecate.show().Downloaded files now keep file extensions rather than replacing all periods with underscores.
Update to TimeRange API, removed t1 and t0, start and end are now read-only attributes.
Added ability to download level3 data for lyra Light Curve along with corresponding tests.
Added support for gzipped FITS files.
Add STEREO HI Map subclass and color maps.
Map.rotate() no longer crops any image data.
For accuracy, default Map.rotate() transformation is set to bi-quartic.
sunpy.image.transform.affine_transformnow casts integer data to float64 and sets NaN values to 0 for all transformations except scikit-image rotation with order <- 3.CD matrix now updated, if present, when Map pixel size is changed.
Removed now-redundant method for rotating IRIS maps since the functionality exists in Map.rotate()
Provide
__str__and__repr__methods on vsoQueryResponsedeprecate.show()SunPy colormaps are now registered with matplotlib on import of
sunpy.cmsunpy.cm.get_cmapno longer defaults to ‘sdoaia94’Added database url config setting to be setup by default as a sqlite database in the sunpy working directory
Added a few tests for the sunpy.roi module
Added capability for figure-based tests
Removed now-redundant method for rotating IRIS maps since the functionality exists in Map.rotate().
SunPy colormaps are now registered with matplotlib on import of
sunpy.cm.sunpy.cm.get_cmapno longer defaults to ‘sdoaia94’.Added database url config setting to be setup by default as a sqlite database in the sunpy working directory.
Added a few tests for the sunpy.roi module.
Refactored mapcube co-alignment functionality.
Removed sample data from distribution and added ability to download sample files
Changed start of GOES 2 operational time range back to 1980-01-04 so data from 1980 can be read into GOESLightCurve object
Require JSOC request data calls have an email address attached.
Calculation of the solar rotation of a point on the Sun as seen from Earth, and its application to the de-rotation of mapcubes.
Downloaded files now keep file extensions rather than replacing all periods with underscores
Fixed the downloading of files with duplicate names in sunpy.database
Removed sample data from distribution and added ability to download sample files.
Added the calculation of the solar rotation of a point on the Sun as seen from Earth, and its application to the de-rotation of mapcubes.
Changed default for GOESLightCurve.create() so that it gets the data from the most recent existing GOES fits file.
Map plot functionality now uses the mask property if it is present, allowing the plotting of masked map data
Map Expects Quantities and returns quantities for most parameters.
Map now used Astropy.wcs for world <-> pixel conversions.
map.world_to_pixel now has a similar API to map.pixel_to_world.
map.shape has been replaced with map.dimensions, which is ordered x first.
map.rsun_arcseconds is now map.rsun_obs as it returns a quantity.
Map properties are now named tuples rather than dictionaries.
Improvement for Map plots, standardization and improved color tables, better access to plot variables through new plot_settings variable.
Huge improvements in Instrument Map doc strings. Now contain instrument descriptions as well as reference links for more info.
net.jsoc can query data series with time sampling by a Sample attribute implemented in vso.
MapCube.plot and MapCube.peek now support a user defined plot_function argument for customising the animation.
Added new sample data file, an AIA cutout file.
Moved documentation build directory to doc/build
0.5.5#
Changed default for GOESLightCurve.create() so that it gets the data from the most recent existing GOES fits file.
Improvements to the Map documentation.
Typo fixes in sunpy.wcs documentation.
0.5.4#
sunpy.image.transform.affine_transformnow casts integer data to float64 and sets NaN values to 0 for all transformations except scikit-image rotation with order <- 3.Updated SWPC/NOAA links due to their new website.
Exposed the raw AIA color tables in
sunpy.cm.color_tables.Fixes
mapcompatibility with Astropy 1.0.x.
0.5.3#
Goes peek() plot now works with matplotlib 1.4.x
The ANA file reading C extensions will no longer compile under windows. Windows was not a supported platform for these C extensions previously.
0.5.2#
If no CROTA keyword is specified in Map meta data, it will now default to 0 as specified by the FITS WCS standard.
Map now correctly parses and converts the CD matrix, as long as CDELT is specified as well. (Fixes SWAP files)
Fix of HELIO webservice URLs
MapCube.plot() is now fixed and returns a matplotlib.animation.FuncAnimation object.
0.5.1#
MAJOR FIX: map.rotate() now works correctly for all submaps and off center rotations.
HELIO URL updated, queries should now work as expected.
All tabs removed from the code base.
All tests now use tempfile rather than creating files in the current directory.
Documentation builds under newer sphinx versions.
ANA and JP2 tests are skipped if dependencies are missing.
ANA tests are skipped on windows.
0.5.0#
Added additional functionality to the GOES module i.e. the ability to calculate GOES temperature and emission measure from GOES fluxes.
changed _maps attribute in MapCube to a non-hidden type
Added Nobeyama Radioheliograph data support to Lightcurve object.
Fixed some tests on map method to support Windows
Added a window/split method to time range
Updates to spectrogram documentation
Added method Database.add_from_hek_query_result to HEK database
Added method Database.download_from_vso_query_result
GOES Lightcurve now makes use of a new source of GOES data, provides metadata, and data back to 1981.
Removed sqlalchemy as a requirement for SunPy
Added support for NOAA solar cycle prediction in lightcurves
Some basic tests for GenericLightCurve on types of expected input.
Fix algorithm in sunpy.sun.equation_of_center
Added Docstrings to LightCurve methods.
Added tests for classes in sunpy.map.sources. Note that some classes (TRACE, RHESSI) were left out because SunPy is not able to read their FITS files.
Added functions that implement image coalignment with support for MapCubes.
Cleaned up the sunpy namespace, removed .units, /ssw and .sphinx. Also moved .coords .physics.transforms.
Added contains functionality to TimeRange module
Added t-‘now’ to parse_time to provide utcnow datetime.
Fixed time dependent functions (.sun) to default to t-‘now’
Fixed solar_semidiameter_angular_size
Improved line quality and performances issues with map.draw_grid()
Remove deprecated make_map command.
0.4.2#
Fixes to the operational range of GOES satellites
Fix the URL for HELIO queries.
0.4.1#
Fix map.rotate() functionality
Change of source for GOES data.
Fix EIT test data and sunpy FITS saving
Some documentation fixes
fix file paths to use os.path.join for platform independence.
0.4.0#
Major documentation refactor. A far reaching re-write and restructure.
Add a SunPy Database to store and search local data.
Add beta support for querying the HELIO HEC
Add beta HEK to VSO query translation.
Add the ability to download the GOES event list.
Add support for downloading and querying the LYTAF database.
Add support for ANA data.
Updated sun.constants to use astropy.constants objects which include units, source, and error instide. For more info check out https://docs.astropy.org/en/latest/constants/index.html
Add some beta support for IRIS data products
Add a new MapCubeAnimator class with interactive widgets which is returned by mapcube.peek().
The Glymur library is now used to read JPEG2000 files.
GOESLightCurve now supports all satellites.
Add support for VSO queries through proxies.
Fix apparent Right Ascension calculations.
LightCurve meta data member now an OrderedDict Instance
0.3.2#
Pass draw_limb arguments to patches.Circle
Pass graw_grid arguments to pyplot.plot()
Fix README code example
Fix Documentation links in potting guide
Update to new EVE data URL
Update LogicalLightcurve example in docs
Improved InteractiveVSOClient documentation
GOESLightCurve now fails politely if no data is available.
Known Bugs:
sunpy.util.unit_conversion.to_angstrom does not work if ‘nm’ is passed in.
0.3.1#
Bug Fix: Fix a regression in CompositeMap that made contor plots fail.
Bug Fix: Allow Map() to accept dict as metadata.
Bug Fix: Pass arguments from Map() to io.read_file.
0.3.0#
Removal of Optional PIL dependency
Parse_time now looks through nested lists/tuples
Draw_limb and draw_grid are now implemented on MapCube and CompositeMap
Calculations for differential rotation added
mapcube.plot() now runs a mpl animation with optional controls
A basic Region of Interest framework now exists under sunpy.roi
STEREO COR colour maps have been ported from solarsoft.
sunpy.time.timerange has a split() method that divides up a time range into n equal parts.
Added download progress bar
pyfits is deprecated in favor of Astropy
spectra:
Plotting has been refactorted to use a consistent interface
spectra now no-longer inherits from numpy.ndarray instead has a .data attribute.
Map: * map now no-longer inherits from numpy.ndarray instead has a .data attribute. * make_map is deprecated in favor of Map which is a new factory class * sunpy.map.Map is now sunpy.map.GenericMap * mymap.header is now mymap.meta * attributes of the map class are now read only, changes have to be made through map.meta * new MapMeta class to replace MapHeader, MapMeta is not returned by sunpy.io * The groundwork for GenericMap inheriting from astropy.NDData has been done, there is now a NDDataStandin class to provide basic functionality.
io: * top level file_tools improved to be more flexible and support multiple HDUs * all functions in sunpy.io now assume multiple HDUs, even JP2 ones. * there is now a way to override the automatic filetype detection * Automatic fits file detection improved * extract_waveunit added to io.fits for detection of common ways of storing wavelength unit in fits files.
A major re-work of all internal imports has resulted in a much cleaner namespace, i.e. sunpy.util.util is no longer used to import util.
Some SOHO and STEREO files were not reading properly due to a date_obs parameter.
Sunpy will now read JP2 files without a comment parameter.
Memory leak in Crotate patched
Callisto: Max gap between files removed
0.2.0#
Completely re-written plotting routines for most of the core datatypes.
JPEG 2000 support as an input file type.
Improved documentation for much of the code base, including re-written installation instructions.
New lightcurve object
LYRA support
GOES/XRS support
SDO/EVE support
New Spectrum and Spectrogram object (in development)
Spectrogram plotting routines
Callisto spectrum type and support
STEREO/SWAVES support
Map Object
Added support for LASCO, Yohkoh/XRT maps
A new CompositeMap object for overlaying maps
Resample method
Superpixel method
The addition of the rotate() method for 2D maps.
What’s New in sunpy 6.0?#
The SunPy Project is pleased to announce the 6.0 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
sunpy 6.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
PUT STATS HERE
Updates to minimum dependencies#
The minimum required version of Python has been updated to 3.10.
The minimum required versions of core dependencies have been updated:
astropy >= 5.2.0
The minimum required versions of these optional dependencies has also been updated:
asdf >= 2.12.0
asdf-astropy >= 0.2.0
beautifulsoup4 >= 4.11.0
cdflib >= 0.4.4
dask >= 2022.5.2
h5netcdf >= 1.0.0
h5py >= 3.7.0
lxml >= 4.9.0
opencv-python >= 4.6.0.66
pandas >= 1.4.0
reproject >= 0.9.0
requests >= 2.28
scikit-image >= 0.19.0
scipy >= 1.8.0
spiceypy >= 5.0.0
tqdm >= 4.64.0
zeep >= 4.1.0
Removal of sunpy.database#
The sunpy.database module has not been actively maintained for over a year now and has a number of outstanding issues.
It has been deprecated since sunpy 4.1, and since there have been no major objections from the community since then we have completely removed sunpy.database in sunpy 6.0.
If you are interested in seeing a replacement for sunpy.database, either inside sunpy or as a third-party package, please join the discussion thread at https://community.openastronomy.org/t/deprecating-sunpy-database/495.
Arguments for reproject_to()#
Arguments for this method have been changed to be keyword only after the target WCS argument. This was raising a warning since sunpy 4.1.
Removal of sunpy.net.helioviewer.HelioViewerClient#
sunpy.net.helioviewer has been deprecated since sunpy v4.1 and has now been removed.
Users should instead use the hvpy package.
This package provides a Python wrapper around the Helioviewer API and is maintained by the Helioviewer Project.
The hvpy package supersedes the sunpy.net.helioviewer module.
Arguments for sunpy.timeseries.GenericTimeSeries.peek() are now keywords only#
The arguments for sunpy.timeseries.GenericTimeSeries.peek() have been changed to be keyword only.
This means that you must now specify the arguments by name, rather than by position and has been done to make the API more consistent.
This has been raising a warning since sunpy 4.1 and is now an error.
Fix filename sanitization for downloaded files from the VSO#
The VSOClient has been sanitizing filenames to ensure that they are valid on all platforms.
However, we have now fixed the sanitization to be more conservative, to follow the NFKC Unicode normalization, which is the recommended normalization for filenames on most platforms.
This now does not replace periods, does not change letter case and do not leave Unicode characters decomposed.
An example of this is that the filename ä will now be replaced with ä instead of being left out of the filename.
Another example is that the filename “aia.lev1.171A_2020_06_07T06_33_09.35Z.image_lev1.fits” used to be replaced with “aia_lev1_171a_2020_06_07t06_33_09_35z_image_lev1.fits” and now will not be.
Deprecate positional arguments in sunpy.map.GenericMap.plot()#
The arguments for sunpy.map.GenericMap.plot() have been changed to being keyword only.
Pass them as keyword arguments (e.g., ..., title=True, ...) instead.
What’s New in sunpy 5.1?#
The SunPy Project is pleased to announce the 5.1 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
sunpy 5.1 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
This release of sunpy contains 361 commits in 102 merged pull requests closing 27 issues from 19 people, 7 of which are first-time contributors to sunpy.
361 commits have been added since 5.0
27 issues have been closed since 5.0
102 pull requests have been merged since 5.0
19 people have contributed since 5.0
7 of which are new contributors
The people who have contributed to the code for this release are:
Alasdair Wilson
Albert Y. Shih
Brett J Graham *
David Stansby
Hannah Collier *
Jan Gieseler
Kurt McKee *
Laura Hayes
Nabil Freij
Paul Wright *
Samuel Badman *
Samuel J. Van Kooten *
Shane Maloney
Steven Christe
Stuart Mumford
Trestan Simon *
Will Barnes
Where a * indicates that this release contains their first contribution to sunpy.
Calculating the amount of solar eclipse#
In anticipation of the upcoming “Great North American Eclipse” on April 8, 2024, there is a new function sunpy.coordinates.sun.eclipse_amount() that returns how much of the Sun is occulted by the Moon at the specified observer location and time.
The output can be used to determine the start/end times of partial eclipse and of totality.
Computations using SPICE kernels#
The SPICE observation geometry information system is being increasingly used by space missions to describe the locations of spacecraft and the time-varying orientations of reference frames.
The new sunpy.coordinates.spice module enables the use of the SkyCoord API to perform SPICE computations such as the location of bodies or the transformation of a vector from one coordinate frame to another coordinate frame.
Although SPICE kernels can define coordinate frames that are very similar to the frames that sunpy.coordinates already provides, there will very likely be slight differences.
Using sunpy.coordinates.spice will ensure that the definitions are exactly what the mission specifies and that the results are identical to other implementations of SPICE (e.g., CSPICE or Icy).
Note
sunpy.coordinates.spice requires the optional dependency spiceypy to be installed.
Support for GONG Synoptic Data#
The Global Oscillation Network Group (GONG) are a network of observatories located around the Earth that produce photospheric magnetograms every hour.
These magnetograms are used to derive synoptic maps which show a full-surface picture of the solar magnetic field.
A new map source, GONGSynopticMap, has been added to support these synoptic data products.
This means that GONG synoptic data products can now be loaded directly into Map objects.
If you have pfsspy installed, this new map source will be used instead of the one built into pfsspy.
New Method for Determining Visibility of Helioprojective Coordinates#
A new method has been added to the Helioprojective frame to determine whether a coordinate is visible or not.
Visibility is determined as whether or not the coordinate is behind the limb relative to the observer coordinate.
See the documentation for is_visible() for more information.
Improved Support for WISPR Data#
The default colorbar limits and stretching for WISPRMap have been improved to better emphasize the coronal structures present in many images.
Metadata handling has also been improved, including handling non-integer processing levels (e.g. “L2b”) and adding more intuitive labels for the different detectors (e.g. “inner” and “outer” instead of “1” and “2”).
What’s New in sunpy 5.0?#
The SunPy project is pleased to announce the 5.0 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
sunpy 5.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
This release of sunpy contains 1279 commits in 182 merged pull requests closing 115 issues from 36 people, 21 of which are first-time contributors.
1279 commits have been added since 4.1
115 issues have been closed since 4.1
182 pull requests have been merged since 4.1
36 people have contributed since 4.1
21 of which are new contributors
The people who have contributed to the code for this release are:
Akash Verma
Akhoury Shauryam *
Akshit Tyagi *
Alasdair Wilson
Albert Y. Shih
Ansh Dixit *
Aritra Sinha *
Aryan Shukla *
Conor MacBride
Daniel Garcia Briseno
David Stansby
Ed Behn *
Jan Gieseler
Krish Agrawal
Laura Hayes
Lazar Zivadinovic
Marius Giger *
Matt Wentzel-Long *
Nabil Freij
Naveen Srinivasan *
Nick Murphy
Nischal Singh *
OussCHE *
Stuart Mumford
Suleiman Farah *
Syed Md Mihan Chistie *
Tan Jia Qing *
Timo Laitinen *
Will Barnes
William Jamieson *
ejm4567 *
pradeep *
ryuusama *
tal66 *
Where a * indicates that this release contains their first contribution to sunpy.
Updates to minimum dependencies#
The minimum required versions of several core dependencies have been updated:
Python 3.9
astropy 5.0.1
numpy 1.21.0
parfive 2.0.0
We are aware the change in the parfive minimum version is a release earlier than our dependency policy allows for.
However, due to significant issues that parfive v2.0.0 solves and changes to remote servers, we have decided to increase it to improve the user experience when downloading files.
The minimum required versions of these optional dependencies has also been updated:
Matplotlib 3.5.0
dask 2021.4.0
h5netcdf 0.11.0
pandas 1.2.0
scikit-image 0.18.0
scipy 1.7.0
cdflib 0.3.20
New documentation structure#
There has been a major overhaul of the documentation structure. The documentation is now organised into four major distinct sections:
Tutorial, a step-by-step introduction to sunpy.
How-To Guides, a set of small self-contained guides for getting specific tasks done.
Topic Guides, in-depth explanations that detail how and why different parts of sunpy work.
Reference, a listing of the contents of the sunpy package.
OS agnostic packaging#
A pure Python sunpy distribution is now published on PyPI with each release. This distribution contains all the functionality of sunpy apart from the code for reading ANA files. pip will default to installing the pure Python distribution instead of the source distribution on platforms other than Linux (x86-64) and macOS (x86-64 and ARM64). This should mean simpler and faster installs on such platforms, which include the Raspberry Pi as well as some cloud computing services.
Getting pixels along a coordinate path#
A new function, sunpy.map.pixelate_coord_path(), has been added to return all pixels that are intersected by a given coordinate path.
This replaces the now-deprecated sunpy.map.extract_along_coord.
See the following pages for examples of how to use pixelate_coord_path(),
1 minute GOES data now available#
Searching for GOES data now returns 1 minute averaged data in addition to the high-cadence data for GOES 13-17.
As with all Fido clients, the results of a search will return all available data.
For example, searching over a single day will provide multiple results for the different satellites available and the different resolution data.
This can be seen in the Resolution column below.
Here, the flx1s refers to the high-cadence 1s data and the avg1m refers to the averaged 1 minute sampling data.
In the past, sunpy only provided a search over the high-cadence data.
>>> from sunpy.net import Fido, attrs as a
>>> res = Fido.search(a.Time("2022-02-15", "2022-02-15"),
... a.Instrument("XRS"))
>>> res
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
4 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA flx1s
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA avg1m
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA flx1s
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA avg1m
If you want to download just the 1 minute data, you can do so by specifying the resolution in the query by passing the a.Resolution attribute.
If you want the 1s resolution data, you would instead pass a.Resolution("flx1s") instead.
>>> res = Fido.search(a.Time("2022-02-15", "2022-02-15"),
... a.Instrument("XRS"), a.Resolution("avg1m"))
>>> res
<sunpy.net.fido_factory.UnifiedResponse object at ...>
Results from 1 Provider:
2 Results from the XRSClient:
Source: <8: https://umbra.nascom.nasa.gov/goes/fits
8-15: https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/
16-17: https://data.ngdc.noaa.gov/platforms/solar-space-observing-satellites/goes/
Start Time End Time ... Provider Resolution
----------------------- ----------------------- ... -------- ----------
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA avg1m
2022-02-15 00:00:00.000 2022-02-15 23:59:59.999 ... NOAA avg1m
sunpy.database deprecation#
The sunpy.database module is no longer actively maintained and has a number of outstanding issues.
It has now been formally deprecated, and will be removed in sunpy 6.0.
If you are using sunpy.database and would like to see a replacement, please join the discussion thread at https://community.openastronomy.org/t/deprecating-sunpy-database/495.
Deprecation of IO readers#
sunpy.io.cdf, sunpy.io.file_tools and sunpy.io.jp2 sub-modules have been deprecated, and will be removed in version 5.1.
These are designed for internal use only, and removing them from the public API gives the developers more flexibility to modify them without impacting users.
What’s New in sunpy 4.1?#
The SunPy project is pleased to announce the 4.1 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
sunpy 4.1 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
This release of sunpy contains 1408 commits in 191 merged pull requests closing 86 issues from 28 people, 18 of which are first-time contributors.
1408 commits have been added since 4.0
86 issues have been closed since 4.0
191 pull requests have been merged since 4.0
28 people have contributed since 4.0
18 of which are new contributors
The people who have contributed to the code for this release are:
Akash Verma *
Alasdair Wilson
Albert Y. Shih
Alex Kaszynski *
Alex W *
Andy Tang *
Chris Bard *
Conor MacBride
Daniel Garcia Briseno *
David Stansby
Jan Gieseler *
Jia Qing *
Karthikeyan Singaravelan *
Krish Agrawal *
Laura Hayes
Lazar Zivadinovic
Marius Giger *
Matt Wentzel-Long *
Nabil Freij
Naveen Srinivasan *
Stuart Mumford
Timo Laitinen *
Will Barnes
William Jamieson *
William Russell *
tal66 *
Where a * indicates that this release contains their first contribution to sunpy.
Extracting map values along coordinate paths#
It is now easy to extract data values from a GenericMap along a curve specified by set of coordinates using the new sunpy.map.extract_along_coord function.
This is done by applying Bresenham’s line algorithm between the consecutive coordinates, in pixel space, and then indexing the data array of the map at those points.
Easier reprojecting to Carrington and Stonyhurst coordinates#
sunpy.map.header_helper has a new make_heliographic_header() function that simplifies creating map headers that span the whole solar surface in Carrington or Stonyhurst coordinates.
When used in combination with sunpy.map.GenericMap.reproject_to() this greatly simplifies reprojecting helioprojective maps into heliographic full-Sun maps.
Drawing the solar equator and prime meridian#
The new sunpy.visualization.drawing module can be used to draw the solar equator and prime meridian (zero Carrington longitude) on a plot with Astropy WCS Axes.
Sample data downloaded on demand#
The sample data files provided through sunpy.data.sample are now downloaded individually on demand rather than being all downloaded upon import of that module.
All the sample data files can still be downloaded at once by calling sunpy.data.sample.download_all().
sunpy.database deprecation#
The sunpy.database module is no longer actively maintained and has a number of outstanding issues.
It is anticipated that sunpy.database will be formally deprecated in sunpy 5.0 and removed in sunpy 6.0.
If you are using sunpy.database and would like to see a replacement, please join the discussion thread at https://community.openastronomy.org/t/deprecating-sunpy-database/495.
sunpy.visualization.limb moved#
sunpy.visualization.limb.draw_limb has been moved into the sunpy.visualization.drawing module and renamed sunpy.visualization.drawing.limb().
The sunpy.visualization.limb module will remain working, but is deprecated and will be removed in version 5.1.
HelioviewerClient deprecated#
The Helioviewer Project now maintains a Python Wrapper called hvpy.
As such, in consultation with the Helioviewer Project, the sunpy.net.helioviewer module is deprecated and will be removed in sunpy 5.1.
Changes to timeseries plotting#
To make plotting a timeseries as source-independent as possible, a number of source-specific plot customisations in peek() and plot() methods have been removed.
See the changelog for full details on what has changed.
To harmonize different peek() and plot() signatures, all non-keyword arguments to these methods are deprecated.
To avoid a warning pass all arguments with keywords (e.g. plot(title='my plot title')) instead.
Updated data downloader dependency#
The package that handles downloading data from remote sources, parfive, has had a recent major release to version 2.0.
This new version comes with major usability improvements: removal of incomplete files and major error reporting upgrades.
sunpy users are encouraged to upgrade parfive to benefit from these improvements.
To upgrade you can use pip:
$ pip install -U parfive
or conda:
$ conda update parfive
What’s New in SunPy 4.0?#
The SunPy project is pleased to announce the 4.0 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
SunPy 4.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
This release of sunpy contains 1310 commits in 182 merged pull requests closing 83 issues from 29 people, 19 of which are first-time contributors to sunpy.
1310 commits have been added since 3.1
83 issues have been closed since 3.1
182 pull requests have been merged since 3.1
29 people have contributed since 3.1
19 of which are new contributors
The people who have contributed to the code for this release are:
Akash Verma *
Alasdair Wilson *
Albert Y. Shih
Alex Kaszynski *
Alex W *
Andy Tang *
Anubhav Sinha *
Conor MacBride
Daniel Garcia Briseno *
David Stansby
Devansh Shukla *
Jan Gieseler *
Jeffrey Aaron Paul
Jia Qing *
Karthikeyan Singaravelan *
Krish Agrawal *
Laura Hayes
Lazar Zivadinovic
Nabil Freij
Noah Altunian *
Rohan Sharma *
Samriddhi Agarwal
Stuart Mumford
Thomas Braccia *
Tim Gates *
Will Barnes
William Russell *
cbard *
pre-commit-ci[bot] *
Where a * indicates that this release contains their first contribution to sunpy.
Increase in required package versions#
We have bumped the minimum version of several packages we depend on; these are the new minimum versions:
python >= 3.8
Improvements to map and image rotation#
There have been significant improvements to sunpy.map.GenericMap.rotate() and sunpy.image.transform.affine_transform().
Most notably, there is now comprehensive support for having not-a-number (NaN) values in pixels, with NaNs now preserved across image rotation.
Accordingly, the default behavior for these functions has been changed to use NaN for the pixels in the output image that are beyond the extent of the input image.
To obtain the prior behavior, which used zero by default for such pixels, the missing needs to be explicitly set to zero.
The default rotation function used internally has been switched to scipy.ndimage.affine_transform() from skimage.transform.warp() due to changes in the latest release of scikit-image, although the differences between the rotation results are small.
Since bicubic interpolation by scipy.ndimage.affine_transform() is sufficient in quality for the typical user, we have changed the default interpolation order for sunpy.map.GenericMap.rotate() to bicubic interpolation (order=3) instead of biquartic interpolation (order=4) for improved performance under default settings.
sunpy.image.transform.affine_transform() has always defaulted to bicubic interpolation (order=3).
Also, there is now the option to rotate using OpenCV.
The rotation function to use can be selected via the method keyword argument.
New rotation functions beyond these three can be added using the new decorator add_rotation_function().
Improved return types of HEK queries#
The ‘event_endtime’, ‘event_starttime’ and ‘event_peaktime’ columns in a HEK
query are now returned as Time objects.
Drawing Carrington coordinate grids#
It is now easy to draw Carrington coordinate grids on top of maps using
sunpy.map.GenericMap.draw_grid() by specifying system='carrington'.
See Plotting a coordinate grid for an example.
Better printing of metadata#
Printing a MetaDict now prints each entry on a new line, making it much easier to read:
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> from sunpy.map import Map
>>> m = Map(AIA_171_IMAGE)
>>> print(m.meta)
simple: True
bitpix: -32
naxis: 2
naxis1: 1024
naxis2: 1024
...
Deprecation of sunpy.image.coalignment#
The sunpy.image.coalignment module has been deprecated and will be removed in version 4.1.
Users should instead use sunkit_image.coalignment which includes identical functionality and
an identical API.
The reason for deprecating and moving sunpy.image.coalignment is twofold.
First, the scope of the core sunpy package has increasingly narrowed, with more analysis-specific
functionality moved out to affiliated packages.
Second, the module has seen little development in several years and by moving
it to sunkit_image.coalignment, we hope to increase its visibility and attract a larger number
of contributors.
Deprecation of sunpy.io.fits#
The sunpy.io.fits module is deprecated, and will be removed in version 4.1.
This because it was designed for internal use only, and removing it from the public API gives the developers more flexibility to modify it without impacting users.
The astropy.io.fits module can be used instead as a replacement that is designed to be user-facing.
Deprecation of sunpy.physics.solar_rotation#
sunpy.physics.solar_rotation.calculate_solar_rotate_shift and sunpy.physics.solar_rotation.mapsequence_solar_derotate have been deprecated and will be removed in version 4.1.
Both of these functions have been moved to sunkit_image.coalignment and have identical functionality and API.
Note that sunpy.physics.solar_rotation.mapsequence_solar_derotate has been renamed to mapsequence_coalign_by_rotation to more accurately reflect its functionality.
New dependency on asdf-astropy for ASDF support#
With the 4.0 release of sunpy the asdf plugin has been updated to support future versions of the asdf Python library.
This has lead to no significant changes to how ASDF files are handled on save or load, however the plugin code is significantly simpler.
When updating sunpy to 4.0 it is important that the asdf-astropy package is installed if asdf is installed.
If installing sunpy with sunpy[all] or sunpy[asdf] when using pip this will happen automatically, however, if you update sunpy with pip install -U sunpy and you have previously installed asdf (implicitly or explicitly) you will need to install the asdf-astropy package.
If you have installed sunpy with conda, you don’t need to do anything as the conda package has been updated to depend on asdf and asdf-astropy.
If asdf-astropy is not installed then sunpy’s asdf plugin will fail to load and emit a warning, this will happen every time for any ASDF file irrespective of if it contains a sunpy object in its tree.
Deprecation of shift and shifted_value#
The method sunpy.map.GenericMap.shift has been renamed
sunpy.map.GenericMap.shift_reference_coord and shift has been
deprecated.
The method has been renamed to make it clear that it is the reference coordinate that is
being shifted and not the image itself.
Additionally, the sunpy.map.GenericMap.shifted_value property, which keeps track of
the shifts applied by shift, has been deprecated.
Users should instead use the CRVAL1 and CRVAL2 keys in
sunpy.map.GenericMap.meta.modified_items to see how the reference coordinate has been
modified.
Note that shift_reference_coord does not modify
shifted_value.
Contributors to this Release#
The people who have contributed to the code for this release are:
TODO: fill this in at release time.
Where a * indicates that this release contains their first contribution to SunPy.
What’s New in SunPy 3.1?#
The SunPy project is pleased to announce the 3.1 release of the sunpy core package.
On this page, you can read about some of the big changes in this release.
SunPy 3.1 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
610 commits have been added since 3.0
56 issues have been closed since 3.0
145 pull requests have been merged since 3.0
16 people have contributed since 3.0
8 of which are new contributors
Increase in required package versions#
We have bumped the minimum version of several packages we depend on; these are the new minimum versions for sunpy 3.1:
astropy >= 4.2
matplotlib >= 3.2.0
numpy >= 1.17.0
pandas >= 1.0.0
Increased in-situ data support#
Two new features have significantly increased support for in-situ data returned by heliospheric missions. See Getting data from CDAWeb for a full example of searching for, downloading, and loading a CDF file into sunpy.
New CDAWeb client#
A new Fido client to search the Coordinated Data Analysis Web (CDAWeb) has been added to sunpy.net.
This allows one to search for and download data from many space physics missions that take in-situ data (including Parker Solar Probe and Solar Orbiter).
In combination with the new CDF reading abilities of sunpy.timeseries.TimeSeries, this provides a full workflow for searching for, downloading, and analysing in-situ data contained within CDF files.
Support for .cdf files#
sunpy.timeseries.TimeSeries now supports loading CDF files if the external library cdflib is installed.
New limb drawing function#
The solar limb as seen from an arbitrary observer coordinate can now be drawn on a world coordinate system aware
Axes using the sunpy.visualization.draw_limb function.
New WISPR map source#
A new map source for the WISPR instrument on Parker Solar Probe has been added.
This improves the name of the map and adds correct
information for the processing_level and
exposure_time.
Changes to map metadata fixes#
The GenericMap map sources are primarily used to modify metadata values to either fix known incorrect values in a given data source, or make the metadata comply with the FITS standard.
There are two ways in which the sunpy map sources can do this:
Directly edit the values stored in the FITS metadata.
Overload properties (e.g.
sunpy.map.GenericMap.unit) such that they return the correct information without modifying the underlying metadata.
In previous versions of sunpy there has been no consistency in which of these two approaches is taken.
As of sunpy 3.1, the second approach is now consistently taken, and sunpy no longer edits any FITS metadata values when constructing a Map.
GenericMap properties now consistently provide corrected information.
For details of any corrections applied, the docstrings of different map sources (e.g., HMIMap - see Instrument Map Classes for a list of map sources) provide information on any assumptions made beyond the original FITS metadata when constructing the map properties.
For all maps, the following fixes are no longer made:
DATE-OBS is no longer replaced by DATE_OBS as a fallback
NAXIS, NAXIS1, NAXIS2, BITPIX are no longer populated if not present
BUNIT is no longer corrected to be a FITS compliant unit string
WAVEUNIT is no longer automatically populated from the header comments if it is not present.
For specific map sources, the following keywords are no longer modified or added:
KCorMap: OBSERVATORY, DETECTOR, WAVEUNIT, DSUN_OBS, HGLN_OBSSWAPMap: OBSRVTRY, DETECTORRHESSIMap: CUNIT1, CUNIT2, CTYPE1, CTYPE2, WAVEUNIT, WAVELNTHAIAMap: BUNIT, DETECTORHMIMap: DETECTOR, CRDER1, CRDER2HMISynopticMap: CUNIT1, CUNIT2, CDELT1, CDELT2, DATE-OBSEITMap: WAVEUNIT, CUNIT1, CUNIT2LASCOMap: DATE-OBS, DATE_OBS, CROTA, CROTA1, CROTA2, CUNIT1, CUNIT2MDIMap: CUNIT1, CUNIT2MDISynopticMap: CUNIT1, CUNIT2, CDELT2, DATE-OBS, CRDER1, CRDER2EUVIMap: WAVEUNIT, DATE-OBS, CROTA, CROTA2CORMap: DATE-OBSHIMap: DATE-OBSSUVIMap: DETECTOR, TELESCOPTRACEMap: DETECTOR, OBSRVTRY, CUNIT1, CUNIT2SXTMap: DETECTOR, TELESCOP, DSUN_APPARENTXRTMap: DETECTOR, TELESCOP, TIMESYSSOTMap: DETECTOR, TELESCOPSJIMap: DETECTOR, WAVEUNIT, WAVELNTH, CUNIT1, CUNIT2EUIMap: CROTA, CROTA2
Changes to map date/time handling#
New date properties#
The properties date_start,
date_end, and date_average have
been added to be drawn from the relevant FITS metadata, if present in the map
header. These are from new keywords defined in version 4 of the FITS standard,
which have precise meanings compared to the previously ill-defined DATE-OBS.
Changes to date#
sunpy.map.GenericMap.date now looks for more metadata than just DATE-OBS.
This property can return any one of the new properties (see above) depending
on the metadata present in the map. It now draws from, in order of preference:
The DATE-OBS FITS keyword
The current time.
If DATE-OBS is present alongside DATE-AVG or DATE-BEG and DATE-END, this results
in a behaviour change to favour the new (more precisely defined) keywords.
It is recommended
to use date_average,
date_start, or date_end
instead if you need one of these specific times.
Addition of new time format TimeTaiSeconds#
The new TimeTaiSeconds format is the number of
SI seconds from 1958-01-01 00:00:00, which includes UTC leap seconds.
1958-01-01 00:00:00 is the defined time when International Atomic Time (TAI)
and Universal Time (UT) are synchronized.
This format is equivalent to the output of the SSW anytim2tai routine, and
related routines, for times after 1972-01-01. Be aware that the SSW routines
are not written to provide valid results for times before 1972-01-01.
This format is equivalent to TimeUnixTai, except that the epoch
is 12 years earlier.
Propagating solar-surface coordinates in time#
There is now an easy-to-use context manager (propagate_with_solar_surface()) to enable coordinate transformations to take solar rotation into account.
Normally, a coordinate refers to a point in inertial space, so transforming it to a different observation time does not move the point at all.
Under this context manager, a coordinate will be treated as if it were referring to a point on the solar surface.
Coordinate transformations with a change in observation time will automatically rotate the point in heliographic longitude for the time difference, with the amount of rotation depending on the specified differential-rotation model.
Convenient reprojection of maps#
Map objects now have the reproject_to() method to easily reproject the map to a new WCS.
The returned map will be of type GenericMap, with no metadata preserved from the original map, so copy over any desired metadata from the original map.
This method requires the optional package reproject to be installed.
Reprojecting to a Map Projection with a Custom Origin
JSOC keyword filtering with Fido#
Support for filtering searches with JSOC keywords has been added to Fido.search:
>>> from sunpy.net import Fido, attrs as a
>>> import astropy.units as u
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
a.jsoc.Series('aia.lev1_euv_12s'), a.Wavelength(304*u.AA), a.jsoc.Keyword("EXPTIME") > 1)
<sunpy.net.fido_factory.UnifiedResponse object at 0x7fe16a5d20d0>
Results from 1 Provider:
301 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
-------------------- -------- -------- -------- -------
2014-01-01T00:00:01Z SDO/AIA AIA_4 304 2145
2014-01-01T00:00:13Z SDO/AIA AIA_4 304 2145
2014-01-01T00:00:25Z SDO/AIA AIA_4 304 2145
2014-01-01T00:00:37Z SDO/AIA AIA_4 304 2145
... ... ... ... ...
2014-01-01T00:59:25Z SDO/AIA AIA_4 304 2145
2014-01-01T00:59:37Z SDO/AIA AIA_4 304 2145
2014-01-01T00:59:49Z SDO/AIA AIA_4 304 2145
2014-01-01T01:00:01Z SDO/AIA AIA_4 304 2145
Length = 301 rows
>>> Fido.search(a.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'),
a.jsoc.Series('aia.lev1_euv_12s'), a.Wavelength(304*u.AA), a.jsoc.Keyword("EXPTIME") == 1)
<sunpy.net.fido_factory.UnifiedResponse object at 0x7fe16a5d20d0>
Results from 1 Provider:
0 Results from the JSOCClient:
Source: http://jsoc.stanford.edu
Please be aware of two caveats:
We do not validate the value used for comparison.
Passing in a keyword without comparison to a value (e.g.
==0,< 10) will error.
Arithmetic operations with maps#
GenericMap objects now support arithmetic operations (i.e. addition, subtraction, multiplication, division) with array-like quantities.
This includes scalar quantities as well as Numpy arrays.
Notably, arithmetic operations between two GenericMap objects are not supported.
Contributors to this Release#
The people who have contributed to the code for this release are:
Alasdair Wilson *
Albert Y. Shih
Anubhav Sinha *
Conor MacBride
David Stansby
Devansh Shukla *
Jeffrey Aaron Paul
Nabil Freij
Noah Altunian *
Rohan Sharma *
Samriddhi Agarwal
Stuart Mumford
Thomas Braccia *
Tim Gates *
Will Barnes
Where a * indicates that this release contains their first contribution to SunPy.
What’s New in SunPy 3.0?#
Overview#
The SunPy project is pleased to announce the 3.0 release of the sunpy package.
On this page, you can read about some of the big changes in this release:
SunPy 3.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
526 commits have been added since 2.1
53 issues have been closed since 2.1
109 pull requests have been merged since 2.1
26 people have contributed since 2.1
10 of which are new contributors
Please find below a selection of what we consider to be the biggest changes or features with this release.
Improvements to Visualization of Maps#
In this release a number of Map visualisation methods have been improved to be more coordinate aware than previous releases.
The first new feature is sunpy.map.GenericMap.draw_quadrangle() which replaces the old draw_rectangle method.
draw_quadrangle() draws rectangles where the edges follow lines of constant longitude and latitude in the coordinate system in which the box is drawn.
The next change is to sunpy.map.GenericMap.draw_limb(), which now supports drawing the limb as seen by any observer on any image.
This means that it is possible to visualize the sections of the limb which are visible in one map on another, or to draw the limb as seen from helioprojective map observer on a synoptic map.
For a demonstration of this new functionality see the first figure in the Reprojecting Images to Different Observers example.
The last major change is to sunpy.map.GenericMap.plot(), which now has an autoalign=True keyword argument.
This option when set to True will use pcolormesh() to transform the image being plotted the correct coordinates of the axes it is being plotted on.
This allows for plots of two images with different projections to be visualized together correctly.
It is worth noting that using autoalign=True is computationally expensive, and when used with interactive plots there is a significant performance penalty every time the plot is modified.
See the Auto-Aligning AIA and HMI Data During Plotting example for details.
Improved Support for Solar Orbiter’s EUI Instrument in Map#
A new map source to support data from the Extreme Ultraviolet Imager (EUI) instrument on the Solar Orbiter (SolO) spacecraft has been added.
This source improves the accuracy of the observer position by using the heliocentric inertial coordinates as well as correctly setting the processing level, exposure time and colormap.
Data from EUI will automatically load using this source via sunpy.map.Map.
Inspect history of map metadata changes#
The .meta property of a GenericMap now keeps a record of the contents of the metadata (normally a FITS header) when it was created.
This can be accessed via the original_meta property.
This allows any changes made by sunpy or by the user directly to be tracked with the following properties:
See the new Map metadata modification example for details.
sunpy.instr moved to sunkit-instruments#
The sunpy.instr subpackage has been moved to a separate affiliated package called sunkit-instruments.
This has been done to make the core package align with the goal that instrument specific analysis and processing code should live in affiliated packages.
Increase in required package versions#
We have bumped the minimum version of several packages we depend on; these are the new minimum versions for sunpy 3.0:
asdf>=2.6.0
astropy >= 4.1.0
beautifulsoup4>=4.8.0
dask[array]>=2.0.0
drms>=0.6.1
glymur>=0.8.18,!=0.9.0
h5netcdf>=0.8.1
matplotlib>=3.1.0
numpy >= 1.16.0
pandas>=0.24.0
parfive >= 1.2.0
python-dateutil>=2.8.0
scipy >= 1.3.0
scipy>=1.3.0
sqlalchemy>=1.3.4
tqdm>=4.32.1
zeep>=3.4.0
Contributors to this Release#
The people who have contributed to the code for this release are:
Abhijeet Manhas
Abhishek Pandey
Adwait Bhope *
Albert Y. Shih
Amarjit Singh Gaba *
Aryan Chouhan
David Stansby
Jeffrey Aaron Paul
Kateryna Ivashkiv
Kaustubh Chaudhari *
Kritika Ranjan
Laura Hayes
Megh Dedhia *
Monica Bobra
Mouloudi Mohamed Lyes *
Nabil Freij
Nakul Shahdadpuri
Ratul Das *
Samriddhi Agarwal *
Shane Maloney
Stuart Mumford
Tathagata Paul
Thomas A Caswell
Varun Bankar *
Will Barnes
Yukie Nomiya *
Where a * indicates that this release contains their first contribution to sunpy.
What’s New in SunPy 2.1?#
Overview#
The SunPy project is pleased to announce the 2.1 release of the sunpy package.
On this page, you can read about some of the big changes in this release:
SunPy 2.1 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
1148 commits have been added since 2.0
202 issues have been closed since 2.0
306 pull requests have been merged since 2.0
38 people have contributed since 2.0
19 of which are new contributors
Please find below a selection of what we consider to be the biggest changes or features with this release.
Support for GOES-16 and GOES-17 X-Ray Sensor (XRS) data#
We have now included support for the GOES-16 and GOES-17 XRS 1-second data. This data can now be queried and downloaded with sunpy.net.Fido and the files read in and analysed as a sunpy.timeseries.TimeSeries.
Retrieving and analyzing GOES X-Ray Sensor (XRS) data
Fido Metadata#
Fido provides a unified interface to download data from several services but lacked support for search for metadata provided by catalogues or data providers. This has now changed and you can query JSOC, HEK and HEC with Fido:
>>> from sunpy.net import Fido
>>> from sunpy.net import attrs as a
>>> results = Fido.search(timerange,
a.hek.FL & (a.hek.FL.PeakFlux > 1000) |
a.jsoc.Series('hmi.m_45s'))
>>> results
Results from 2 Providers:
2 Results from the HEKClient:
gs_thumburl ...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100731_205448_25495028/www/EDS_FlareDetective-TriggerModule_20100801T033001-20100801T035225_AIA_171_S21W87_ssw_cutout_20100801_033013_AIA_171_S21W87_20100801_033012_context_0180.gif ...
http://sdowww.lmsal.com/sdomedia/ssw/ssw_client/data/ssw_service_100801_234037_25860951/www/EDS_FlareDetective-TriggerModule_20100801T033008-20100801T035232_AIA_193_S21W87_ssw_cutout_20100801_033020_AIA_193_S21W87_20100801_033019_context_0180.gif ...
1 Results from the JSOCClient:
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
----------------------- -------- ---------- -------- -------
2010.08.01_03:40:30_TAI SDO/HMI HMI_FRONT2 6173.0 2099
Alongside this, there is easier access to the results from a client by indexing the results by the client name:
>>> hek_results, jsoc_results = results['hek'], results['jsoc']
In the first example you can see the search results from HEK are not too useful by default.
The results are truncated as the long urls for the thumbnail take up too many characters and HEK can return up to 100 columns of information.
With a new method show() on the results itself, you directly select the columns to display:
>>> hek_results.show('event_peaktime', 'obs_instrument', 'fl_peakflux')
event_peaktime obs_instrument fl_peakflux
------------------- -------------- -----------
2010-08-01T03:40:37 AIA 1027.64
2010-08-01T03:40:44 AIA 1441.78
To find the keys for all of the columns you can do:
>>> hek_results.keys()
['gs_thumburl',
'comment_count',
'hpc_bbox',
...
'area_unit',
'obs_lastprocessingdate',
'refs']
In addition, if you pass the entire search query into Fido.fetch() and it will ignore results that have no corresponding data files to retrieve.
Fido Results Refactor#
While working on the above change, several changes were made to the way that results are returned from sunpy.net.Fido.search and the search methods of the underlying clients.
We have tried to minimize any breaking changes here and we believe that for most users the difference between 2.0 and 2.1 will be minor.
The key highlights you will want to be aware of are:
Previously slicing the result of
Fido.search()(aUnifiedResponseobject) so that it had a length of one returned anotherUnifiedResponseobject, now it will return aQueryResponseTableobject, which is a subclass ofastropy.table.Table.All result objects contained within the results of a
Fido.search()are nowQueryResponseTableobjects (or subclasses thereof). These objects are subclasses ofastropy.table.Tableand can therefore be filtered and inspected as tabular objects, and the modified tables can be passed toFido.fetch().The keys available to be used when formatting the
path=argument toFido.fetch()have changed. This is to standardise them over the results from more clients and make them easier to use. You can use the~.UnifiedResponse.path_format_keysmethod to see all the possible keys for a particular search.The search results object returned from
Fido.searchnow correctly counts all results in itsfile_numproperty.Results from the
NOAAIndicesClientand theNOAAPredictClientno longer haveStart TimeorEnd Timein their results table as the results returned from the client are not dependent upon the time parameter of a search.
New synoptic map sources and clients#
MDISynopticMap and HMISynopticMap have been added as new data sources, and automatically fix common issues with FITS metadata from these sources. You do not need to change any code to use these, as sunpy automatically detects and uses the appropriate map sources for each file.
It is now possible to search for GONG synoptic maps within sunpy.net.Fido, using a.Instrument('GONG').
Requesting cutouts from the JSOC#
Fido can now be used to request cutouts from JSOC via the new a.jsoc.Cutout attr. This includes the ability to adjust the requested field of view to “track” a feature as it moves across the solar disk, perform sub-pixel image registration, and mask off-disk pixels.
Coordinates with velocities#
It is now supported to transform coordinates with attached velocities, and the various ephemeris functions can optionally include velocity information.
Transformations between coordinate frames will account for both any change in orientation of the velocity vector and any induced velocity due to relative motion between the frames.
For example, consider Mars’s position/velocity in HeliographicStonyhurst:
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import get_body_heliographic_stonyhurst
>>> mars = SkyCoord(get_body_heliographic_stonyhurst('mars', '2021-01-01',
... include_velocity=True))
>>> mars
<SkyCoord( HeliographicStonyhurst: obstime=2021-01-01T00:00:00.000): (lon, lat, radius) in (deg, deg, AU)
(-34.46752135, 1.77496469, 1.50936573)
(d_lon, d_lat, d_radius) in (arcsec / s, arcsec / s, km / s)
(-0.00048971, 0.00060976, 19.54950062)>
>>> mars.velocity.norm()
<Quantity 19.56823076 km / s>
However, HeliographicStonyhurst is a non-inertial frame that rotates over time.
By transforming this coordinate to HeliocentricInertial, we can see that Mars’s actual velocity is larger:
>>> mars.heliocentricinertial
<SkyCoord (HeliocentricInertial: obstime=2021-01-01T00:00:00.000): (lon, lat, distance) in (deg, deg, AU)
(-9.91128592, 1.77496469, 1.50936573)
(d_lon, d_lat, d_distance) in (arcsec / s, arcsec / s, km / s)
(0.04174239, 0.00060976, 19.54950058)>
>>> mars.heliocentricinertial.velocity.norm()
<Quantity 49.68592218 km / s>
See Coordinates with velocity information for more information.
Alternatives for reprojecting a Helioprojective map#
The typical observation in Helioprojective coordinates does not contain full 3D information for the sources of emission, so an assumption needs to be made when transforming such coordinates to other coordinate frames.
By default, SunPy assumes that the emission is coming from the surface of the Sun, which enables reprojections such as in the example Reprojecting Images to Different Observers.
However, this assumption is not appropriate for some observations, e.g., from coronagraphs.
There is now a context manager (assume_spherical_screen()) to override the default assumption such that any 2D coordinates are interpreted as being on the inside of a large spherical screen.
See the following example for how this context manager enables alternative reprojections.
Finding map contours#
The new sunpy.map.GenericMap.contour() method can be used to extract contours from a map. It returns contours as a SkyCoord, allowing contours to be easily overplotted on the original or other maps.
Performance improvements#
Several functions in sunpy.map have been significantly sped up with improved algorithms.
In addition, sunpy.map.GenericMap.wcs is now cached when the map metadata remains unchanged, significantly improving performance in applications which make multiple requests for the map WCS (e.g. plotting), and reducing the number of repeated warnings thrown when metadata is missing.
Increase in required package versions#
We have bumped the minimum version of several packages we depend on; these are the new minimum versions for sunpy 2.1:
python 3.7
astropy 4.0
scipy 1.2
parfive 1.1
drms 0.6.1
matplotlib 2.2.2
Contributors to this Release#
The people who have contributed to the code for this release are:
Abhijeet Manhas
Abhishek Pandey *
Adrian Price-Whelan
Albert Y. Shih
Aryan Chouhan *
Conor MacBride *
Daniel Ryan
David Pérez-Suárez
David Stansby
Dipanshu Verma *
Erik Tollerud *
Jai Ram Rideout
Jeffrey Aaron Paul *
Johan L. Freiherr von Forstner *
Kateryna Ivashkiv *
Koustav Ghosh *
Kris Akira Stern
Kritika Ranjan *
Laura Hayes
Lazar Zivadinovic
Nabil Freij
Rutuja Surve
Sashank Mishra
Shane Maloney
Shubham Jain *
SophieLemos *
Steven Christe
Stuart Mumford
Sudeep Sidhu *
Tathagata Paul *
Thomas A Caswell *
Will Barnes
honey
mridulpandey
nakul-shahdadpuri *
platipo *
resakra *
sophielemos *
Where a * indicates that this release contains their first contribution to SunPy.
What’s New in SunPy 2.0?#
Overview#
The SunPy project is pleased to announce the 2.0 release of the sunpy package. On this page, you can read about some of the big changes in this release:
SunPy 2.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
1044 commits have been added since 1.1
144 issues have been closed since 1.1
290 pull requests have been merged since 1.1
33 people have contributed since 1.1
16 new contributors
Increase in required package versions#
We have bumped the minimum version of several packages we depend on:
numpy>=1.15.0
scipy>=1.0.0
matplotlib>=2.2.2
astropy>=3.2
parfive>=1.1.0
Search Attributes#
To search with Fido, you need to specify attributes to search against.
Before sunpy 2.0, you had to supply values, as the following example demonstrates:
>>> from sunpy.net import Fido, attrs as a
>>> Fido.search(a.Time('2012/3/4', '2012/3/6'), a.Instrument("norh"),
... a.Wavelength(17*u.GHz))
There was no way to know if the value was correct, but now we have a extenstive list of supported values from the clients and servers we can request data from.
Using Instrument as an example, if you print the object:
>>> print(a.Instrument)
sunpy.net.attrs.Instrument
Specifies the Instrument name for the search.
Attribute Name Client Full Name Description
--------------------------- ----------- ------------------------ --------------------------------------------------------------------------------
aia VSO AIA Atmospheric Imaging Assembly
bbi VSO BBI None
bcs VSO BCS Bragg Crystal Spectrometer
bic_hifi VSO BIC-HIFI None
bigbear VSO Big Bear Big Bear Solar Observatory, California TON and GONG+ sites
...
This will list the name of value you should use, what data source will supply that data and a description.
Furthermore, you can use tab completion to auto-fill the attribute name, for example by typing a.Instrument.<TAB>.
So now you can do the following instead:
Fido.search(a.Time('2012/3/4', '2012/3/6'), a.Instrument.norh, a.Wavelength(17*u.GHz))
aiaprep is now deprecated#
With the release of the new aiapy package, sunpy.instr.aia.aiaprep will be removed in version 2.1.
Equivalent functionality is provided by the register() function in aiapy. For more
details, see the example on registering and aligning level 1 AIA images
in the aiapy documentation.
Fixes and clarification to pixel indexing#
sunpy uses zero-based indexing when referring to pixels, where the center of the bottom left pixel of a map is at [0, 0] * u.pix.
Several parts of the API have been updated to make sure this is consistently the case across the package.
In particular:
sunpy.map.GenericMap.top_right_coordpreviously had an off-by-one error in the calculation of the top right coordinate. This has been fixed.sunpy.map.GenericMap.centerpreviously had an off-by-one error in the calculation of the coordinate of the center of a map. This has been fixed.sunpy.map.GenericMap.reference_pixelnow returns a zero-based reference pixel. This is one pixel less than the previously returned value. Note that this means thereference_pixelnow does not have the same value as the FITSCRPIXvalues, which are one-based indices.sunpy.map.header_helper.make_fitswcs_header()now correctly interprets thereference_pixelargument as being zero-based, in previous releases it incorrectly interpreted thereference_pixelas one-based.
Standardization of submap and sunpy.map.GenericMap.draw_rectangle#
Both submap and sunpy.map.GenericMap.draw_rectangle allow specification of “rectangles” in world (spherical) coordinates.
In versions prior to 2.0 you passed the coordinates of the rectangle to draw_rectangle as a bottom left coordinate, and a height and width, but for submap you passed it as a bottom left and a top right.
In 2.0 the way you call both methods has changed, to accept a bottom left and then either width and height or a top right coordinate.
As part of this change, the top_right, width, and height arguments must always be keyword arguments, i.e. width=10*u.arcsec
This change allows you to give the same rectangle specification to submap as to sunpy.map.GenericMap.draw_rectangle.
Which is especially useful when you wish to plot a cropped area of a map, along with it’s context in the parent map:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> import matplotlib.pyplot as plt
>>> import sunpy.map
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> aia = sunpy.map.Map(AIA_171_IMAGE)
>>> bottom_left = SkyCoord(-100 * u.arcsec, -100 * u.arcsec, frame=aia.coordinate_frame)
>>> width = 500 * u.arcsec
>>> height = 300 * u.arcsec
>>> sub_aia = aia.submap(bottom_left, width=width, height=height)
>>> fig = plt.figure()
>>> ax1 = fig.add_subplot(1, 2, 1, projection=aia)
>>> aia.plot(axes=ax1)
>>> aia.draw_rectangle(bottom_left, width=width, height=height)
>>> ax2 = fig.add_subplot(1, 2, 2, projection=sub_aia)
>>> sub_aia.plot(axes=ax2)
Both these methods delegate the input parsing to a new utility function sunpy.coordinates.utils.get_rectangle_coordinates.
Graphical overview for Map and MapSequence#
There are new methods to produce graphical overviews for Map and MapSequence instances: quicklook() and quicklook(), respectively.
This graphical overview opens the default web browser and uses HTML to show a table of metadata, a histogram of the pixel values in the data, and a histogram-equalized image of the data.
Here’s an example of the output for a MapSequence instance:
<sunpy.map.mapsequence.MapSequence object at 0x7f6f15b25b40> MapSequence of 3 elements, with maps from HMIMap, AIAMap, EITMap
If you are using Jupyter Notebook, there is no need to call these methods explicitly to see this graphical overview. If you type just the name of the instance, the graphical overview is shown within the notebook itself as a rich representation of the instance, instead of the typical text representation.
Differential rotation in the coordinate framework#
The rotation rate of solar features varies with heliographic latitude, this rotation is called “differential rotation”.
SunPy has already included functionality in the sunpy.physics.differential_rotation module to transform coordinates and Maps to account for the rotation of the Sun.
SunPy now provides differential-rotation functionality integrated directly into the coordinate framework using the RotatedSunFrame class.
Here are examples of using this class:
A detailed write-up of how to use RotatedSunFrame can be found at the RotatedSunFrame documentation.
Changes to Carrington coordinates#
We have refined our approach for heliographic Carrington coordinates to best support high-resolution imagery of the Sun, including from observatories that are at distances from the Sun that is significantly different from 1 AU (e.g., Solar Orbiter).
Our HeliographicCarrington coordinate frame is now expressly intended for the co-alignment of images of the Sun’s surface from different observatories.
HeliographicCarrington now requires the specification of the observer location (Earth or otherwise) because the light travel time between the Sun and the observer is accounted for.
SunPy output now matches the calculations by JPL Horizons and SPICE.
There may be small differences compared to Carrington coordinates computed by groups that do not use modern parameter values or the same assumptions for the methodology.
Importantly, the Carrington longitude that is now calculated (including using sunpy.coordinates.sun.L0()) will not match earlier versions of SunPy.
A detailed write-up of the calculation approach and comparisons to other resources can be found at carrington’s functionality documentation.
Download behind proxies#
With the release of parfive 1.1, sunpy has been patched to be able to utilize proxy servers when downloading files.
Proxy URL is read from the environment variables
HTTP_PROXYorHTTPS_PROXY.Proxy Authentication
proxy_authshould be passed as aaiohttp.BasicAuthobject, explicitly by the user.Proxy Headers
proxy_headersshould be passed asdictobject, explicitly by the user.
For example if you use a bash terminal:
$ HTTP_PROXY=http://user:password@proxyserver.com:3128
$ HTTPS_PROXY=https://user:password@proxyserver.com:3128
$ export HTTP_PROXY
$ export HTTPS_PROXY
these will be used to enable downloads through a proxy.
Citation update#
A paper discussing sunpy 1.0 was accepted in The Astrophysical Journal and you can find the bibtex for it by running:
>>> import sunpy
>>> sunpy.__citation__
Previous update: sunpy 1.1#
In case you never updated to the intermediate release (sunpy 1.1) the whatsnew contains the major changes from that release: What’s New in SunPy 1.1?
What’s New in SunPy 1.1?#
Overview#
The SunPy project is pleased to announce the 1.1 release of the sunpy package.
The headline changes in 1.1 are:
The
coordinatessubpackage now supports four additional coordinate frames (HCI, HEE, GSE, and GEI).A new subpackage
sunpy.data.data_managerhas been added to support versioned data for functions and methods.Support in
sunpy.mapandsunpy.netfor the SUVI instrument on GOES satellites.Initial support for WISPR data from Parker Solar Probe in
sunpy.map.The import times for
sunpyand some subpackages are significantly shorter, with no loss of functionality.
On this page, you can read about some of the big changes in this release:
SunPy 1.1 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
1137 commits have been added since 1.0
106 issues have been closed since 1.0
242 pull requests have been merged since 1.0
24 people have contributed since 1.0
10 new contributors
Supported versions of Python#
Like SunPy 1.0, 1.1 comes with support for Python versions 3.6, 3.7 and support for 3.8 has been added.
New coordinate frames#
The coordinates subpackage now supports four additional coordinate frames of interest to solar physics:
The following transformation graph illustrates how all of these coordinate frames can be transformed to any other frame in sunpy.coordinates or astropy.coordinates:
See our coordinates documentation for a table of the currently supported coordinate systems and the corresponding frame classes.
Manager for Versioned Data Files#
SunPy 1.1 provides a data manager for versioning and caching remote files. The objective of this is to provide a way for data required for functions, such as instrument correction routines, to depend on non-local data in a reliable way. The data manager also guarantees that a specific version of the code uses a specific data file, with the ability for users to specify updated files.
This works by providing the URL of a remote file and a SHA256 hash to the sunpy.data.manager.require decorator which can be added to functions that require these specific data files from remote sources.
If the specified hash does not match that of the remote version, an exception is raised to make the user aware of any changes on the remote server or corruption of local files.
Additionally, sunpy.data.cache can be used to avoid re-downloading files that already exist on a user’s local machine, thus saving disk space and internet bandwidth.
Support for SUVI Data#
The Solar Ultraviolet Imager (SUVI) is a EUV instrument imaging the full disk of the Sun in six passbands, and is onboard the latest of the Geostationary Operational Environmental Satellite (GOES) missions.
SUVIMap provides SunPy map support for loading SUVI FITS image data, and the SUVIClient adds support to search for SUVI data hosted by NOAA via Fido. It supports searching for wavelength, level of data (level 2 data which consists of stacked level 1b images and original level 1b files), as well as GOES satellite number (>= GOES 16).
Initial Support for WISPR Images#
Following the first data release from Parker Solar Probe, SunPy 1.1 supports loading WISPR imaging data into a GenericMap.
Due to the complex projections in the WISPR data this involved changing the way sunpy converts FITS headers into astropy.wcs.WCS objects.
It is expected that sunpy 2.0 will include more complete support for WISPR data.
Speeding up import times#
We know that the initial import of sunpy or its subpackages can feel like it takes a long time, particularly on slower machines.
Some of that import time can be the result of importing other modules or external packages that are required for specialized functionality that a user may not ever actually use.
We have identified the most egregious cases and deferred those imports of dependencies to when they are actually needed.
For example, the initial import of sunpy.map is now ~40% faster, with no loss of functionality.
We will continue to look for ways to improve import times.
Notable Breaking Changes or Removed functionality#
Importing
sunpy.timeseriesno longer automatically imports Matplotlib. (#3376)sunpy.timeseries.sources.NOAAIndicesTimeSeries.peeknow checks that thetypeargument is a valid string, and raises aValueErrorif it isn’t. (#3378)Observer-based coordinate frames (
HeliocentricandHelioprojective) no longer assume a default observer (Earth) if no observer is specified. These frames can now be used with no observer specified, but most transformations cannot be performed for such frames. This removal of a default observer only affectssunpy.coordinates, and has no impact on the default observer insunpy.map. (#3388)The colormap stored in SunPy’s Map subclasses (ie.
map.plot_settings['cmap']) can now be colormap string instead of the fullmatplotlib.colors.Colormapobject. To get the fullColormapobject use the new attributemap.cmap. (#3412)Fix a warning in
sunpy.map.GenericMap.rotatewhere the truth value of an array was being calculated. This changes the behavior ofrotatewhen theangle=parameter is not anQuantityobject to raiseTypeErrorrather thanValueError. (#3456)Removed the step of repairing images (replacing non-finite entries with local mean) before coaligning them. The user is expected to do this themselves before coaligning images. If NaNs/non-finite entries are present, a warning is thrown. The function
sunpy.image.coalignment.repair_image_nonfiniteis deprecated. (#3287)The method to convert a
Helioprojectiveframe from 2D to 3D has been renamed fromsunpy.coordinates.frames.Helioprojective.calculate_distancetomake_3d. This method is not typically directly called by users. (#3389)sunpy.visualization.animator.ImageAnimatorWCSis now deprecated in favour ofsunpy.visualization.animator.ArrayAnimatorWCS. (#3407)sunpy.cmhas been moved tosunpy.visualization.colormapsand will be removed in a future version. (#3410)
Full Change Log#
To see a detailed list of all changes in version v1.1, including changes in API, please see the Full Changelog.
What’s New in SunPy 1.0?#
Overview#
The SunPy project is pleased to announce the 1.0 release of the sunpy package. This release is the result of over 14 months of work and marks a shift in the core library with the removal of a lot of legacy code and a focus on usability and stability. The headline changes in 1.0 are:
A complete transition of the whole code base to use
astropy.time.Time, which was implemented by Vishnunarayan K I as part of Google Summer of Code 2018.A rewrite of how all the clients in
sunpy.netdownload files from the internet. This means vastly improved progress bars, skipping downloads if files are present, and better visibility and retrying of failed downloads.A rewrite of the differential rotation and image warping code to correctly account for observer location using the Astropy coordinate functionality.
Removal of many deprecated functions and submodules; we have used the 1.0 release as a chance to clean out SunPy, reducing the number of lines of Python code in the package by almost 3,000!
The first release of SunPy to be Python 3 only, requiring Python 3.6+. This is because Python 2 will not be maintained after 2019.
On this page, you can read about some of the big changes in this release:
There have been numerous improvements to large parts of SunPy, notably in the content of the documentation, continuous integration and testing. SunPy 1.0 also includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
1913 commits have been added since 0.9
582 issues have been closed since 0.9
332 pull requests have been merged since 0.9
46 people have contributed since 0.9
25 new contributors
Supported versions of Python#
SunPy 1.0 has dropped Python 2 support as Python 2 is nearing the end of its support cycle. To port your code to Python 3, there is some good advice here and a breakdown of the relevant changes in Python 3 on the Python 3 for Scientists page. If you still need Python 2 support, SunPy 0.9.X will see bug fixes until the end of 2019, but no new features. As a result, SunPy’s minimum Python version has been raised to Python 3.6 and is routinely tested against Python 3.6 and 3.7.
Astropy Time is used everywhere#
SunPy now uses Astropy’s Time object everywhere to represent time.
This comes with numerous benefits:
Support for non-UTC time scales. UTC as well as non-UTC time scales like TAI, TT, UT1 etc. can be used with
astropy.time.Time.>>> t = Time('2012-06-18T02:00:05.453', scale='tai') >>> t <Time object: scale='tai' format='isot' value=2012-06-18T02:00:05.453>
Timealso provides easy conversion between different scales.>>> t.utc <Time object: scale='utc' format='isot' value=2012-06-18T01:59:31.453>
Support for high precision times.
Timecan provide sub-nanosecond precision for time objects while pythondatetimewas restricted to microseconds.>>> t = Time('2012-06-18T02:00:05.453123123') >>> t <Time object: scale='utc' format='isot' value=2012-06-18T02:00:05.453> >>> t.precision = 9 >>> t <Time object: scale='utc' format='isot' value=2012-06-18T02:00:05.453123123>
Support for leap seconds This was one of the biggest motivations for the transition to
astropy.time.Time.datetimehas no support for leap seconds whileTimesupports them. A leap second is a one-second adjustment applied to UTC to keep it close to the mean solar time.>>> Time('2016-12-31T23:59:60') <Time object: scale='utc' format='isot' value=2016-12-31T23:59:60.000> >>> Time('2016-12-31T23:59:59') + 1 * u.s <Time object: scale='utc' format='isot' value=2016-12-31T23:59:60.000>
Support for numerous formats
Timecan parse numerous formats including pythondatetime.>>> list(Time.FORMATS) ['jd', 'mjd', 'decimalyear', 'unix', 'cxcsec', 'gps', 'plot_date', 'datetime', 'iso', 'isot', 'yday', 'fits', 'byear', 'jyear', 'byear_str', 'jyear_str']
>>> import datetime >>> Time(datetime.datetime.now()) <Time object: scale='utc' format='datetime' value=2018-10-20 15:36:16.364089>
Changes in return values
All functions which previously returned
datetime.datetimenow returnTimeand all functions which returneddatetime.timedeltanow returnastropy.time.TimeDelta. For example, the properties ofsunpy.time.TimeRangewhich used to returndatetime.datetimeanddatetime.timedeltanow returnastropy.time.Timeandastropy.time.TimeDelta.Changes to
parse_timeparse_timehas been reduced to a tiny wrapper overTime. The API ofparse_timeis almost the same asTime, however,parse_timesupports conversion of a few more formats thanTime, which arenumpy.datetime64,pandas.Series,pandas.DatetimeIndex, utime and a few other time string formats.
Improved file downloading capability#
The file download capability has been re-written to use the parfive package. This brings more visually appealing and informative progress bars, better reporting of download errors and the ability to re-download failed files.
It is possible to retry any downloads which fail with:
>>> files = Fido.fetch(results) # Some downloads fail
>>> files = Fido.fetch(files) # Retry the downloads which failed
Improvements to coordinates functionality#
Accurate Sun-specific coordinates calculations
Sun-specific coordinates calculations have been grouped together in
sunpy.coordinates.sun, and the underlying implementations have been re-written to use Astropy rather than approximate expressions. Nearly all of the returned values now match published values in the Astronomical Almanac to published precision (e.g., the hundredth of an arcsecond for apparent right ascension). For times that are provided to these functions, the user should take care to specify whether the time is other than UT (e.g., TT), which can be done usingTime(see above).Improved tools to get positions of bodies in the solar system
The existing function
get_body_heliographic_stonyhursthas been enhanced to be able to correct for light travel time. When one specifies an observer, the function determines the emission time in the past that results in photons arriving at the observer at the observation time. The function then returns the location of the requested body at that emission time.>>> t = '2012-06-05 22:34:48.350' >>> without_correction = get_body_heliographic_stonyhurst('venus', t) >>> print(without_correction) <HeliographicStonyhurst Coordinate (obstime=2012-06-05T22:34:48.350): (lon, lat, radius) in (deg, deg, AU) (359.92620234, 0.02752007, 0.72602872)> >>> with_correction = get_body_heliographic_stonyhurst('venus', t, observer=get_earth(t)) INFO: Apparent body location accounts for 144.06 seconds of light travel time [sunpy.coordinates.ephemeris] >>> print(with_correction) <HeliographicStonyhurst Coordinate (obstime=2012-06-05T22:34:48.350): (lon, lat, radius) in (deg, deg, AU) (359.92355609, 0.02734159, 0.72602853)>
There is a new function
get_horizons_coordthat queries JPL HORIZONS for the location of solar-system bodies. JPL HORIZONS includes not only planets and other natural bodies in the solar system, but also major spacecraft. This function requires the Astroquery package and an Internet connection.Query the location of Venus
>>> get_horizons_coord('Venus barycenter', '2001-02-03 04:05:06') INFO: Obtained JPL HORIZONS location for Venus Barycenter (2) [sunpy.coordinates.ephemeris] <SkyCoord (HeliographicStonyhurst: obstime=2001-02-03T04:05:06.000): (lon, lat, radius) in (deg, deg, AU) (326.06844114, -1.64998481, 0.71915147)>
Query the location of the SDO spacecraft
>>> get_horizons_coord('SDO', '2011-11-11 11:11:11') INFO: Obtained JPL HORIZONS location for Solar Dynamics Observatory (spac [sunpy.coordinates.ephemeris] <SkyCoord (HeliographicStonyhurst: obstime=2011-11-11T11:11:11.000): (lon, lat, radius) in (deg, deg, AU) (0.01018888, 3.29640407, 0.99011042)>
Query the location of the SOHO spacecraft via its ID number (-21)
>>> get_horizons_coord(-21, '2004-05-06 11:22:33', 'id') INFO: Obtained JPL HORIZONS location for SOHO (spacecraft) (-21) [sunpy.coordinates.ephemeris] <SkyCoord (HeliographicStonyhurst: obstime=2004-05-06T11:22:33.000): (lon, lat, radius) in (deg, deg, AU) (0.2523461, -3.55863351, 0.99923086)>
Logging used to record SunPy notices#
All messages provided by SunPy use a new logging facility which is based on the Python logging module rather than print statements.
Messages can have one of several levels, in increasing order of importance:
DEBUG: Detailed information, typically of interest only when diagnosing problems.
INFO: A message conveying information about the current task, and confirming that things are working as expected.
WARNING: An indication that something unexpected happened, and that user action may be required.
ERROR: An indication that a more serious issue has occurred, where something failed but the task is continuing.
CRITICAL: A serious error, indicating that the program itself may be unable to continue running.
By default, all messages except for DEBUG messages are displayed. Messages can also be sent to a file and time stamped.
See the Logger Objects documentation for instructions on how to control the verbosity of the logger.
Improvements to differential rotation#
Applying the effect of solar differential rotation to coordinates now properly takes into account the changing position of the observer. For example, since the Earth moves, observers on the Earth must take into account the solar differential rotation of the Sun and the motion of the Earth when calculating a location on the Sun.
Support for applying solar differential rotation to coordinates.
Solar differential rotation of on-disk coordinates can be specified using either time or a new observer. If time is specified, then the new observer is assumed to be located on the Earth:
>>> import astropy.units as u >>> from astropy.coordinates import SkyCoord >>> from sunpy.coordinates import Helioprojective >>> from sunpy.physics.differential_rotation import solar_rotate_coordinate >>> from sunpy.time import parse_time >>> start_time = '2010-09-10 12:34:56' >>> duration = 25*u.hour >>> c = SkyCoord(-570*u.arcsec, 120*u.arcsec, obstime=start_time, frame=Helioprojective) >>> solar_rotate_coordinate(c, time=duration) <SkyCoord (Helioprojective: obstime=2010-09-11T13:34:56.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2010-09-11T13:34:56.000): (lon, lat, radius) in (deg, deg, AU) (-5.08888749e-14, 7.24318962, 1.00669016)>): (Tx, Ty, distance) in (arcsec, arcsec, km) (-363.04027419, 104.87807178, 1.499598e+08)>
Due to the ellipticity of the Earth’s orbit, the amount of solar rotation is different at different times in the year:
>>> start_time = '2010-06-10 12:34:56' >>> duration = 25*u.hour >>> c = SkyCoord(-570*u.arcsec, 120*u.arcsec, obstime=start_time, frame=Helioprojective) >>> solar_rotate_coordinate(c, time=duration) <SkyCoord (Helioprojective: obstime=2010-06-10T12:34:56.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2010-06-11T13:34:56.000): (lon, lat, radius) in (deg, deg, AU) (0., 0.58398742, 1.01539908)>): (Tx, Ty, distance) in (arcsec, arcsec, km) (-359.11576773, 117.18020622, 1.51263627e+08)>
The user can also explicitly specify an observer at a different time and location in space. The amount of solar rotation applied depends on the time difference between the observation time of the`~astropy.coordinates.SkyCoord` and the time of the observer:
>>> import astropy.units as u >>> from astropy.coordinates import SkyCoord >>> from sunpy.coordinates import Helioprojective, HeliographicStonyhurst >>> from sunpy.physics.differential_rotation import solar_rotate_coordinate >>> from sunpy.time import parse_time >>> start_time = parse_time('2010-06-10 12:34:56') >>> duration = 25*u.hour >>> c = SkyCoord(-570*u.arcsec, 120*u.arcsec, obstime=start_time, frame=Helioprojective) >>> new_observer = SkyCoord(lon=20*u.deg, lat=8*u.deg, radius=0.9*u.au, obstime=end_time, frame=HeliographicStonyhurst) >>> solar_rotate_coordinate(c, observer=new_observer) <SkyCoord (Helioprojective: obstime=2010-06-10T12:34:56.000, rsun=695700.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2010-06-11T13:34:56.000): (lon, lat, radius) in (deg, deg, AU) (20., 8., 0.9)>): (Tx, Ty, distance) in (arcsec, arcsec, km) (-715.77862011, 31.87928146, 1.34122226e+08)>
Experimental support for applying solar differential rotation to maps.
Applying solar differential rotation to maps also accounts for changing observer position. This functionality is still experimental. For example, to differentially rotate a map back 23 hours:
>>> import astropy.units as u >>> import sunpy.map >>> from sunpy.data.sample import AIA_171_IMAGE >>> from sunpy.physics.differential_rotation import differential_rotate
>>> aia = sunpy.map.Map(AIA_171_IMAGE) >>> differential_rotate(aia, time=-23*u.hour)
differential_rotatealso accepts a new observer keyword. The amount of solar differential rotation is calculated using the time difference between the map date and observation time of the new observer. For example:>>> import astropy.units as u >>> import sunpy.map >>> from sunpy.data.sample import AIA_171_IMAGE
>>> from sunpy.physics.differential_rotation import differential_rotate >>> aia = sunpy.map.Map(AIA_171_IMAGE) >>> new_observer = SkyCoord(lon=-15*u.deg, lat=-4*u.deg, radius=1*u.au, obstime=aia.date-34*u.hour, frame=HeliographicStonyhurst) >>> differential_rotate(aia, observer=new_observer)
Map utility functions#
A set of new utility functions have been added to sunpy.map which act on sunpy.map.GenericMap instances.
For example, getting the world coordinates for every pixel:
>>> import sunpy.map
>>> from sunpy.data.sample import AIA_171_IMAGE
>>> import astropy.units as u
>>> from sunpy.physics.differential_rotation import differential_rotate
>>> from sunpy.map import contains_full_disk, all_coordinates_from_map
>>> aia = sunpy.map.Map(AIA_171_IMAGE)
>>> contains_full_disk(aia)
True
>>> coordinates = all_coordinates_from_map(aia) # The coordinates for every map pixel
>>> coordinates.shape
(1024, 1024)
or generating a new FITS header for a custom map:
>>> import numpy as np
>>> import astropy.units as u
>>> from sunpy.coordinates import frames
>>> from astropy.coordinates import SkyCoord
>>> data = np.arange(0,100).reshape(10,10)
>>> coord = SkyCoord(0*u.arcsec, 0*u.arcsec, obstime = '2013-10-28', observer = 'earth', frame = frames.Helioprojective)
>>> header = sunpy.map.header_helper.make_fitswcs_header(data, coord)
>>> for key, value in header.items():
... print(f"{key}: {value}")
wcsaxes: 2
crpix1: 5.5
crpix2: 5.5
cdelt1: 1.0
cdelt2: 1.0
cunit1: arcsec
cunit2: arcsec
ctype1: HPLN-TAN
ctype2: HPLT-TAN
crval1: 0.0
crval2: 0.0
lonpole: 180.0
latpole: 0.0
date-obs: 2013-10-28T00:00:00.000
hgln_obs: 0.0
hglt_obs: 4.7711570596394
dsun_obs: 148644585949.49176
rsun_ref: 695700.0
rsun_obs: 965.3723815059902
Config File Location Moved#
If you have customised your The sunpyrc file you will need to move it to the new config file location.
Your old file should be in ~/.sunpy/sunpyrc file and the new location, which is now platform specific, can be found by running sunpy.print_config.
We recommend that your take a look at the new file as available configuration options have increased.
Renamed/removed functionality#
This is just some of the renamed or removed functionality.
sunpy.sun.sunfunctions have been re-implemented using Astropy for significantly improved accuracy and moved tosunpy.coordinates.sun.Removed
sunpy.time.julian_day,sunpy.time.julian_centuries,sunpy.time.day_of_year,sunpy.time.break_time,sunpy.time.get_day.Move the matplotlib animators from
sunpy.visualisation.imageanimatorandsunpy.visualization.mapcubeanimatortosunpy.visualization.animator.axis_rangeskwarg ofsunpy.visualization.animator.ArrayAnimator,sunpy.visualization.animator.ImageAnimatorandsunpy.visualization.animator.LineAnimatornow must be entered asNone,[min, max]or pixel edges of each array element.The Helioviewer client has been switched to using the newer Helioviewer API. This has meant that we have changed some of the keywords that were passed into client’s methods.
Removed
sunpy.net.jsoc.attrs.Timebecause it served the same purpose assunpy.net.attrs.Timeafter the switch toastropy.time.Time.The deprecated
sunpy.lightcurve(replaced bysunpy.timeseries),sunpy.wcsandsunpy.spectra(replaced by theradiospectrapackage) modules have now been removed.
Full change log#
To see a detailed list of all changes in version v1.0, including changes in API, please see the Full Changelog.
What’s New in SunPy 0.9?#
Overview#
SunPy 0.9 brings improved support for downloading data from the JSOC and bugfixes compared to the 0.8.x series of releases.
The 0.9.x series will be the last series of SunPy releases to support Python 2.
This is because Python 2 will not be maintained after 2019.
The 0.9.x series will receive bugfixs only up until the and of life of Python 2 (around 18 months).
No new functionality will be added to the 0.9.x series, which will also be the last version to include
sunpy.spectra, sunpy.lightcurve and sunpy.wcs, all of which were deprecated in 0.8.
SunPy 1.0 and higher will support Python 3 only. All new functionality will be available only in SunPy 1.0 and higher.
On this page, you can read about some of the big changes in this release:
SunPy 0.9 includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
807 commits have been added since 0.8
310 issues have been closed since 0.8
147 pull requests have been merged since 0.8
34 people have contributed since 0.8
19 new contributors
There have been numerous improvements to large parts of SunPy, notably in the content of SunPy’s documentation, and the continuous integration testing of SunPy.
In addition, SunPy coordinate system functionality has been improved, and the transformation between SunPy specific coordinate systems and those implemented by Astropy is now tested.
The sunpy.map.CompositeMap object has received numerous bugfixes, improving its functionality.
Bugfixes for various animation functionality have also been implemented.
Supported versions of Python#
SunPy is tested against Python 2.7, 3.5 and 3.6.
Improvements to JSOC functionality#
JSOC search capabilities have been improved. It is now possible to search using any JSOC prime key, to search the metadata only, or to search by any series. SunPy’s JSOC functionality uses the DRMS library, which is now a SunPy affiliated package. We would like to thank Kolja Glogowski for the DRMS package and for his help with the JSOC project.
When using a JSOC query in Fido, you must provide a JSOC series and at least one PrimeKey (for example a start time and end time):
>>> from sunpy.net import Fido, attrs as a
>>> result = Fido.search(a.jsoc.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'), a.jsoc.Series('hmi.v_45s'), a.jsoc.Notify('me@email.org')))
>>> result
<sunpy.net.fido_factory.UnifiedResponse object at 0x7fca7050d6a0>
Results from 1 Provider:
81 Results from the JSOCClient:
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
str23 str7 str10 float64 int64
----------------------- -------- ---------- -------- -------
2014.01.01_00:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:01:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:02:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:03:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:04:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
... ... ... ... ...
2014.01.01_00:56:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:57:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:58:30_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_00:59:15_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:00_TAI SDO/HMI HMI_FRONT2 6173.0 2145
2014.01.01_01:00:45_TAI SDO/HMI HMI_FRONT2 6173.0 2145
>>> result = Fido.search(a.jsoc.Time('2014-01-01T00:00:00', '2014-01-01T01:00:00'), a.jsoc.Series('aia.lev1_euv_12s'), a.jsoc.Notify('me@email.org'), a.jsoc.PrimeKey('WAVELNTH', '171'))
>>> result
<sunpy.net.fido_factory.UnifiedResponse object at 0x7fca2981e1d0>
Results from 1 Provider:
301 Results from the JSOCClient:
T_REC TELESCOP INSTRUME WAVELNTH CAR_ROT
str20 str7 str5 int64 int64
-------------------- -------- -------- -------- -------
2014-01-01T00:00:01Z SDO/AIA AIA_3 171 2145
2014-01-01T00:00:13Z SDO/AIA AIA_3 171 2145
2014-01-01T00:00:25Z SDO/AIA AIA_3 171 2145
2014-01-01T00:00:37Z SDO/AIA AIA_3 171 2145
2014-01-01T00:00:49Z SDO/AIA AIA_3 171 2145
2014-01-01T00:01:01Z SDO/AIA AIA_3 171 2145
... ... ... ... ...
2014-01-01T00:58:49Z SDO/AIA AIA_3 171 2145
2014-01-01T00:59:01Z SDO/AIA AIA_3 171 2145
2014-01-01T00:59:13Z SDO/AIA AIA_3 171 2145
2014-01-01T00:59:25Z SDO/AIA AIA_3 171 2145
2014-01-01T00:59:37Z SDO/AIA AIA_3 171 2145
2014-01-01T00:59:49Z SDO/AIA AIA_3 171 2145
2014-01-01T01:00:01Z SDO/AIA AIA_3 171 2145
Data is downloaded using:
>>> files = Fido.fetch(result)
which returns a set of filepaths to the downloaded data.
For more information on accessing JSOC data using SunPy please consult Finding and Downloading Data from JSOC.
Renamed/removed functionality#
sunpy.coordinates.representations#
The package sunpy.coordinates.representations has been removed.
Full change log#
To see a detailed list of all changes in version v0.9, including changes in API, please see the Full Changelog.
What’s New in SunPy 0.8?#
Overview#
SunPy 0.8 is a major release that adds significant new functionality since the 0.7.x series of releases.
On this page, you can read about some of the big changes in this release:
In addition to these major changes, SunPy 0.8 includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog.
By the numbers:
1442 commits have been added since 0.7
163 issues have been closed since 0.7
200 pull requests have been merged since 0.7
35 distinct people have contributed code
17 new contributors
Supported versions of Python#
SunPy is tested against Python 2.7, 3.5 and 3.6. SunPy no longer supports Python 3.4.
New data downloading interface#
The new package Fido is the primary interface to search for and download observational data. Fido is a unified interface for searching and fetching solar physics data irrespective of the underlying client or webservice through which the data is obtained, e.g. VSO, JSOC, etc. It therefore supplies a single, easy and consistent way to obtain most forms of solar physics data. It unifies data search and download from multiple sources such as the VSO and non-VSO sources:
>>> from sunpy.net import Fido, attrs as a
>>> attrs_time = a.Time('2005/01/01 00:10', '2005/01/01 00:15')
>>> result = Fido.search(attrs_time, a.Instrument.eit)
>>> result
<sunpy.net.fido_factory.UnifiedResponse object at 0x7f581489bb70>
Results from 1 Provider:
1 Results from the VSOClient:
Start Time [1] End Time [1] Source Instrument Type Wavelength [2]
Angstrom
str19 str19 str4 str3 str8 float64
------------------- ------------------- ------ ---------- -------- --------------
2005-01-01 00:12:10 2005-01-01 00:12:22 SOHO EIT FULLDISK 195.0 .. 195.0
>>> attrs_time = a.Time('2016/10/22 00:00', '2016/10/25 23:59')
>>> result = Fido.search(attrs_time, a.Instrument.lyra)
>>> result
<sunpy.net.fido_factory.UnifiedResponse object at 0x11dfd8048>
Results from 1 Provider:
4 Results from the LYRAClient:
Start Time End Time Source Instrument Wavelength
str19 str19 str6 str4 str3
------------------- ------------------- ------ ---------- ----------
2016-10-22 00:00:00 2016-10-25 23:59:00 Proba2 lyra nan
2016-10-22 00:00:00 2016-10-25 23:59:00 Proba2 lyra nan
2016-10-22 00:00:00 2016-10-25 23:59:00 Proba2 lyra nan
2016-10-22 00:00:00 2016-10-25 23:59:00 Proba2 lyra nan
Data is downloaded using:
>>> files = Fido.fetch(result)
which returns a set of filepaths to the data.
New coordinate package#
The SunPy coordinates package describes locations in physical space, and coordinate frames. It provides a way to transform coordinates from one frame like helioprojective to another such as heliographic.
Coordinates can be defined very simply:
>>> import astropy.units as u
>>> from astropy.coordinates import SkyCoord
>>> from sunpy.coordinates import frames
>>> a = SkyCoord(200*u.arcsec, 300*u.arcsec, frame=frames.Helioprojective)
>>> a
<SkyCoord (Helioprojective: obstime=None, rsun=695508.0 km, observer=earth): (Tx, Ty) in arcsec
( 200., 300.)>
>>> b = SkyCoord(-20*u.degree, -56*u.degree, frame=frames.HeliographicStonyhurst)
>>> b
<SkyCoord (HeliographicStonyhurst: obstime=None): (lon, lat, radius) in (deg, deg, km)
(-20., -56., 695508.)>
The coordinate a is in the helioprojective coordinate system, and the coordinate b is in the heliographic Stonyhurst system.
Maps can also be used to define coordinate frames:
>>> import sunpy.map
>>> import sunpy.data.sample
>>> aia_map = sunpy.map.Map(sunpy.data.sample.AIA_171_IMAGE)
>>> c = SkyCoord(-45*u.arcsec, 600*u.arcsec, frame=aia_map.coordinate_frame)
>>> c
<SkyCoord (Helioprojective: obstime=2011-06-07 06:33:02.770000, rsun=696000000.0 m, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07 06:33:02.770000): (lon, lat, radius) in (deg, deg, m)
( 0., 0.048591, 1.51846026e+11)>): (Tx, Ty) in arcsec
(-45., 600.)>
The coordinate c is now defined with respect to the coordinate frame derived from the map.
The observer attribute:
>>> c.observer
<HeliographicStonyhurst Coordinate (obstime=2011-06-07 06:33:02.770000): (lon, lat, radius) in (deg, deg, m)
( 0., 0.048591, 1.51846026e+11)>
defines the location from which the coordinate was observed.
Transformation between solar physics coordinate systems#
Transformation between solar physics coordinate frames is simple:
>>> c.transform_to(frames.HeliographicStonyhurst)
<SkyCoord (HeliographicStonyhurst: obstime=2011-06-07 06:33:02.770000): (lon, lat, radius) in (deg, deg, km)
(-3.51257477, 39.27459767, 696000.00000088)>
Transformation to astropy coordinate systems#
Solar physics coordinates can also be transformed into astrophysical coordinates. For example, to convert to the International Celestial Reference System (ICRS):
>>> c.transform_to('icrs')
<SkyCoord (ICRS): (ra, dec, distance) in (deg, deg, km)
( 224.85859731, 10.52568476, 998439.00599877)>
Specification of observer at any major solar system body#
Major solar system bodies can be used to specify observer locations in SkyCoord:
>>> d = SkyCoord(-45*u.arcsec, 600*u.arcsec, observer='Mars', obstime='2011-06-07 06:33:02', frame=frames.Helioprojective)
>>> d
<SkyCoord (Helioprojective: obstime=2011-06-07 06:33:02, rsun=695508.0 km, observer=<HeliographicStonyhurst Coordinate (obstime=2011-06-07 06:33:02): (lon, lat, radius) in (deg, deg, AU)
( 135.78519602, 4.47598707, 1.43448427)>): (Tx, Ty) in arcsec
(-45., 600.)>
New timeseries data object#
The TimeSeries object is used to represent columns of time-ordered scalar values, and is source-aware, just like the Map object. This object supersedes the LightCurve object, which is now deprecated in 0.8.
The TimeSeries object can be instantiated by passing in a file:
>>> import sunpy.timeseries
>>> import sunpy.data.sample
>>> goes = sunpy.timeseries.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES)
TimeSeries objects can have more than one column:
>>> goes.columns
['xrsa', 'xrsb']
and have convenient plotting methods.
import sunpy.timeseries
import sunpy.data.sample
goes = sunpy.timeseries.TimeSeries(sunpy.data.sample.GOES_XRS_TIMESERIES)
goes.peek()
(Source code, png, hires.png, pdf)
TimeSeries objects have a ‘meta’ property that stores the metadata of the timeseries:
>>> goes.meta
|-------------------------------------------------------------------------------------------------|
|TimeRange | Columns | Meta |
|-------------------------------------------------------------------------------------------------|
|2011-06-06 23:59:59.961999 | xrsa | simple: True |
| to | xrsb | bitpix: 8 |
|2011-06-07 23:59:57.631999 | | naxis: 0 |
| | | extend: True |
| | | date: 26/06/2012 |
| | | numext: 3 |
| | | telescop: GOES 15 |
| | | instrume: X-ray Detector |
| | | object: Sun |
| | | origin: SDAC/GSFC |
| | | ... |
|-------------------------------------------------------------------------------------------------|
and the data can be accessed as a pandas dataframe using:
>>> goes.data
xrsa xrsb
2011-06-06 23:59:59.961999 1.000000e-09 1.887100e-07
2011-06-07 00:00:02.008999 1.000000e-09 1.834600e-07
2011-06-07 00:00:04.058999 1.000000e-09 1.860900e-07
...
2011-06-07 23:59:55.584999 1.000000e-09 1.624800e-07
2011-06-07 23:59:57.631999 1.000000e-09 1.598500e-07
Data sources that do not provide FITS files need to have a source keyword to help with the identification and interpretation of the data:
>>> eve = sunpy.timeseries.TimeSeries(sunpy.data.sample.EVE_TIMESERIES, source='EVE')
Differential rotation of maps#
Maps can now be transformed using solar differential rates using from sunpy.physics.differential_rotation import diffrot_map.
Renamed/removed functionality#
Several sub-packages have been moved or removed, and these are described in the following sections.
sunpy.lightcurve#
The package sunpy.lightcurve has been deprecated in favor of timeseries, and will be removed in a future version of SunPy.
sunpy.physics.transforms#
The modules in sunpy.physics.transforms have been moved to physics.
sunpy.net#
HelioviewerClient has been removed from the sunpy.net namespace. It
should now be imported with from sunpy.net.helioviewer import HelioviewerClient.
Full change log#
To see a detailed list of all changes in version v0.8, including changes in API, please see the Full Changelog.
Acknowledging or Citing SunPy#
If you use SunPy in your scientific work, we would appreciate citing it in your publications. The continued growth and development of SunPy is dependent on the community being aware of SunPy.
Citing SunPy in Publications#
Please add the following line within your methods, conclusion or acknowledgements sections:
This research used version X.Y.Z (software citation) of the SunPy open source software package (project citation).
The project citation should be to the SunPy paper, and the software citation should be the specific Zenodo DOI for the version used in your work.
@ARTICLE{sunpy_community2020,
doi = {10.3847/1538-4357/ab4f7a},
url = {https://iopscience.iop.org/article/10.3847/1538-4357/ab4f7a},
author = {{The SunPy Community} and Barnes, Will T. and Bobra, Monica G. and Christe, Steven D. and Freij, Nabil and Hayes, Laura A. and Ireland, Jack and Mumford, Stuart and Perez-Suarez, David and Ryan, Daniel F. and Shih, Albert Y. and Chanda, Prateek and Glogowski, Kolja and Hewett, Russell and Hughitt, V. Keith and Hill, Andrew and Hiware, Kaustubh and Inglis, Andrew and Kirk, Michael S. F. and Konge, Sudarshan and Mason, James Paul and Maloney, Shane Anthony and Murray, Sophie A. and Panda, Asish and Park, Jongyeob and Pereira, Tiago M. D. and Reardon, Kevin and Savage, Sabrina and Sipőcz, Brigitta M. and Stansby, David and Jain, Yash and Taylor, Garrison and Yadav, Tannmay and Rajul and Dang, Trung Kien},
title = {The SunPy Project: Open Source Development and Status of the Version 1.0 Core Package},
journal = {The Astrophysical Journal},
volume = {890},
issue = {1},
pages = {68-},
publisher = {American Astronomical Society},
year = {2020}
}
You can also get this information with sunpy.__citation__.
Other SunPy publications#
The SunPy v1.0.8 code was reviewed by The Journal of Open Source Software (JOSS).
@ARTICLE{Mumford2020,
doi = {10.21105/joss.01832},
url = {https://doi.org/10.21105/joss.01832},
year = {2020},
publisher = {The Open Journal},
volume = {5},
number = {46},
pages = {1832},
author = {Stuart Mumford and Nabil Freij and Steven Christe and Jack Ireland and Florian Mayer and V. Hughitt and Albert Shih and Daniel Ryan and Simon Liedtke and David Pérez-Suárez and Pritish Chakraborty and Vishnunarayan K and Andrew Inglis and Punyaslok Pattnaik and Brigitta Sipőcz and Rishabh Sharma and Andrew Leonard and David Stansby and Russell Hewett and Alex Hamilton and Laura Hayes and Asish Panda and Matt Earnshaw and Nitin Choudhary and Ankit Kumar and Prateek Chanda and Md Haque and Michael Kirk and Michael Mueller and Sudarshan Konge and Rajul Srivastava and Yash Jain and Samuel Bennett and Ankit Baruah and Will Barnes and Michael Charlton and Shane Maloney and Nicky Chorley and Himanshu and Sanskar Modi and James Mason and Naman9639 and Jose Rozo and Larry Manley and Agneet Chatterjee and John Evans and Michael Malocha and Monica Bobra and Sourav Ghosh and Airmansmith97 and Dominik Stańczak and Ruben De Visscher and Shresth Verma and Ankit Agrawal and Dumindu Buddhika and Swapnil Sharma and Jongyeob Park and Matt Bates and Dhruv Goel and Garrison Taylor and Goran Cetusic and Jacob and Mateo Inchaurrandieta and Sally Dacie and Sanjeev Dubey and Deepankar Sharma and Erik Bray and Jai Rideout and Serge Zahniy and Tomas Meszaros and Abhigyan Bose and André Chicrala and Ankit and Chloé Guennou and Daniel D'Avella and Daniel Williams and Jordan Ballew and Nick Murphy and Priyank Lodha and Thomas Robitaille and Yash Krishan and Andrew Hill and Arthur Eigenbrot and Benjamin Mampaey and Bernhard Wiedemann and Carlos Molina and Duygu Keşkek and Ishtyaq Habib and Joseph Letts and Juanjo Bazán and Quinn Arbolante and Reid Gomillion and Yash Kothari and Yash Sharma and Abigail Stevens and Adrian Price-Whelan and Ambar Mehrotra and Arseniy Kustov and Brandon Stone and Trung Dang and Emmanuel Arias and Fionnlagh Dover and Freek Verstringe and Gulshan Kumar and Harsh Mathur and Igor Babuschkin and Jaylen Wimbish and Juan Buitrago-Casas and Kalpesh Krishna and Kaustubh Hiware and Manas Mangaonkar and Matthew Mendero and Mickaël Schoentgen and Norbert Gyenge and Ole Streicher and Rajasekhar Mekala and Rishabh Mishra and Shashank Srikanth and Sarthak Jain and Tannmay Yadav and Tessa Wilkinson and Tiago Pereira and Yudhik Agrawal and Jamescalixto and Yasintoda and Sophie Murray},
title = {SunPy: A Python package for Solar Physics},
journal = {Journal of Open Source Software}
}
A paper about version v0.5 of SunPy was published in Computational Science & Discovery.
Acknowledging SunPy in Posters and Talks#
Please include the Sunpy logo on the title, conclusion slide, or about page. For websites please link the image to sunpy.org. Other versions of the logo are available in the sunpy-logo repository.
Known Issues#
This page documents commonly known issues, issues here is defined broadly and refers to oddities or specifics of how sunpy or the Python ecosystem works that could anyone catch out.
Disagreement between astropy.time.Time and SSW for pre-1972 dates#
Conversions between time scales for pre-1972 dates may not agree between astropy.time.Time and SSW.
Prior to 1972, the length of a Universal Time (UT) second was different from the length of an SI second, and astropy.time.Time correctly handles this difference.
However, SSW does not (at least in the commonly used routines), and thus SSW conversions for pre-1972 dates between UT and International Atomic Time (TAI, based on SI seconds) can be incorrect by multiple seconds.
The SSW routine utc2tai notes that its output is invalid for pre-1972 dates, but other SSW routines are missing such a disclaimer.
JSOCClient time filtering#
The JSOC API filters on T_OBS for some series, instead of T_REC which is what is used most of the time.
There is not anything we can do to fix this on our side.
The specific section of the SDO users guide explaining this.
Current status of sub-packages#
sunpy has variations of stability across different sub-packages. This document summarizes the current status of the sunpy sub-packages, so that users understand where they might expect changes in future, and which sub-packages they can safely use for production code. For help or to discuss specific sub-packages, refer to the maintainer list to find who to contact.
The classification is as follows:
| Planned | |
| Actively developed, be prepared for possible significant changes. | |
| Reasonably stable, any significant changes/additions will generally include backwards-compatibility. | |
| Mature. Additions/improvements possible, but no major changes planned. | |
| Pending deprecation. Might be deprecated in a future version. | |
| Deprecated. Might be removed in a future version. |
The current planned and existing sub-packages are:
| Sub-Package | Comments | |
|---|---|---|
| sunpy.coordinates | Any future changes should largely maintain backwards compatibility. | |
| sunpy.data | Should see no major changes. | |
| sunpy.image | Any future changes should largely maintain backwards compatibility. | |
| sunpy.io | Should see no major changes. | |
| sunpy.map | Any future changes should largely maintain backwards compatibility. | |
| sunpy.net | Any future changes should largely maintain backwards compatibility. | |
| sunpy.physics | While it has not changed recently, it could be subject to major revisions in future. | |
| sunpy.sun | Sub-module ``constants`` is stable. Sub-module ``models`` will likely have major changes. | |
| sunpy.time | Any future changes should largely maintain backwards compatibility. | |
| sunpy.timeseries | Any future changes should largely maintain backwards compatibility. | |
| sunpy.util | This sub-module is not user facing, but any changes will still undergo a deprecation period. | |
| sunpy.visualization | Any future changes should largely maintain backwards compatibility. |
Taken with love from the Astropy project.
Developer’s Guide#
This section contains the various guidelines to be followed by anyone working on sunpy.
Getting started
Newcomers’ Guide#
If you have come across this page, you just might be new to the SunPy project - thanks for your interest in contributing to SunPy! SunPy is an open project that encourages anyone to contribute in any way possible. The people who help develop or contribute to SunPy are varied in ability and experience, with the vast majority being volunteers.
If you have any questions, comments, or just want to say hello, we have an active chat room, a mailing list and a community forum.
How to Contribute to sunpy#
Not Code#
A common misconception (which applies to any package) is that all we really want is some Python code, in fact, we do not require only code contributions! If you do not have the time or the desire to code, we have severals of areas where we can use help.
Reporting Issues#
If you use sunpy and stumble upon a problem, the best way to report it is by opening an issue on our GitHub issue tracker. This way we can help you work around the problem and hopefully fix the problem!
You will need to sign into GitHub to report an issue and if you are not already a member of Github, you will have to join. Joining GitHub will make it easier to report and track issues in the future.
If you do not want to join Github, then another way to report your issue is email the SunPy developers list sunpy-dev@googlegroups.com.
When reporting an issue, please try to provide a short description of the issue with a small code sample, this way we can attempt to reproduce the error. Also provide any error output generated when you encountered the issue, we can use this information to debug the issue. For a good example of how to do this see issue #2879.
If there is functionality that is not currently available in sunpy you can make a feature request. Please write as much information as possible regarding the feature you would like to see in sunpy.
When you go to open either an issue or a feature request, there is a GitHub template that will guide you on the type of information we are seeking. Please be sure to read over the comments in the GitHub text box.
Documentation#
sunpy has online documentation and we try to make sure its as comprehensive as possible. This documentation contains the API of sunpy but also a user guide, an example gallery and developer documents.
However, documentation for any project is a living document. It is never complete and there are always areas that could be expanded upon or could do with proof reading to check if the text is easy to follow and understandable. If parts are confusing or difficult to follow, we would love suggestions or improvements!
Reviewing a Pull Request#
We at any one time have a variety of pull requests open and getting reviews is important. Generally the more people that can look over a pull request the better it will turn out and we encourage everyone to do so.
Code#
If you would prefer to code instead, the best way to start is to work on an existing and known issues. We have several repositories you can investigate. The main one is the sunpy repository with where all the known issues with sunpy are detailed. Each issue should have a series of labels that provide information about the nature of the issue. If you find an issue you’d like to work on, please make sure to add a comment to let people know that you are working on it! This will make it less likely that effort is duplicated.
Note
sunpy is Free and open-source software (FOSS), under the BSD-2 license. By contributing you are stating that you have the right to and agree to have your work distributed under the terms of this license.
This applies to everyone who wants to contribute during work time no matter who their employer is. You should start by checking if there is a Open Source Software Policy (e.g., Standford’s policy) for your work place. If not, OSS-Watch summaries what you will need to check and who to ask, however this resource is aimed at a UK readers. As an example, Standford’s guidance allows someone to contribute and open source their code. If you are unsure if your university or institution allows you to contribute under the BSD-2 license, you should contact the relevant department or administrator that deals with copyright at your institution.
If you are unsure where to start we suggest the Good First Issue label. These are issues that have been deemed a good way to be eased into sunpy and are achievable with little understanding of the sunpy codebase. Please be aware that this does not mean the issue is “easy”, just that you do not need to be aware of the underlying structure of sunpy.
We also tag issues for specific events such as Hacktoberfest under the Hacktoberfest label. The scope of the issues should be appropriate for that specific event. We do participate in several other events but right now we do not have dedicated labels. So please use the above labels for starting issues!
In addition, we have several other repositories that have open issues and you might find these more interesting than the main repository.
Python:
CSS/HTML:
If you already have code that you’ve developed or already have your own idea of code you’d like to work on please first have a look at the issue list to see if any existing issues are related to your idea. If you find nothing then create your own issue to stimulate the addition of your code and make sure to let people know about it chat room or by email. Creating an issue creates a permanent record. It may be possible your idea may be outside of the scope of the repository you are trying to contribute to and the issue comments are a great place to have that discussion where potential future contributors can also see.
Setting up a development environment#
The instructions in this following section are based upon three resources:
We strongly recommend that you read these links.
These links are more in-depth than this guide but you will need to replace astropy with sunpy.
In order to start coding you will need a local Python environment and we would recommend using Anaconda or miniconda (shortened to conda from here on). This method will bypass your operating system Python packages and makes the entire process easier.
The first step is to install the version of conda that corresponds to your operating system and instructions are here. Next we will want to setup the conda environment and we will need to add the conda-forge channel as a prerequisite:
$ conda config --add channels conda-forge
# Note you might need to add python=<version> if a new release of Python has come out very recently.
# Typically it will take around 3 months before we can support the latest version of Python.
$ conda create -n sunpy-dev pip
$ conda activate sunpy-dev
This will create a new conda environment called “sunpy-dev” and install the latest version of pip from the conda-forge channel. The next step is get a development version of sunpy. This will require that git be installed. If you have a GitHub account, we suggest that you fork the sunpy repository (the fork button is to the top right) and use that url for the clone step. This will make submitting changes easier in the long term for you:
Warning
Do not clone the sunpy repository into $HOME/sunpy. Depending on the operating system this location is used to store downloaded data files.
This will cause conflicts later on, so the last argument (sunpy-git) on the git clone line will become the local folder name of the cloned repository.
Otherwise you are free to clone to any other location.
$ git clone https://github.com/<username>/sunpy.git sunpy-git
$ cd sunpy-git
# This adds the main sunpy repository as a remote called "upstream".
# This will help you keep your fork up to date with the main sunpy repository.
$ git remote add upstream https://github.com/sunpy/sunpy.git
# This retrieves the tags from the main sunpy repository, which are used for determining the version of your fork.
$ git fetch --tags upstream
$ pip install -e ".[dev]"
Note
If this does not work, it could be due to a missing C compiler (e.g., gcc or clang) that is required to build sunpy at install.
Getting the compiler either from your system package manager, XCode or Anaconda should address this.
Now you have the latest version of sunpy installed and are ready to work on it using your favorite editor! Ideally, when you start making changes you want to create a git branch:
$ git checkout -b my_fix
You can change my_fix to anything you prefer.
If you get stuck or want help, just ask here!
Checking the code you have written#
Now that you have written some code to address an issue. You will need to check two things:
The changes you have made are correct, i.e., it fixes a bug or the feature works. This requires you to run the code either manually or by writing/running a test function. pytest is the framework we use for this.
The changes you have made follow the correct coding style. We follow the PEP8 style for all Python code and depending on your setup, you can use a linter program to check your code. For documentation, we follow the numpydoc style.
We provide more more detail about our test suite and how to write tests, and how to create and style documentation.
Send it back to us#
Once you have some changes you would like to submit, you will need to commit the changes. This is a three stage process:
Use
git statusto see that the only changes locally are the right ones.Use
git add <path to file>to add the changes togit.Use
git commit -m <message>to label those changes.Use
git pushto update your fork (copy) of sunpy on GitHub.
Here you replace <message> with some text of the work you have done.
We strongly recommend having a good commit message and this commit guide is worth reading.
Next step is to open a pull request on GitHub. If you are new to pull requests, here is the GitHub guide that is a detailed walkthrough. Go to the “pull requests” tab on your fork and pressing the large green “New pull request” button. Now on the right side from the box marked “compare” you can select your branch. Do one final check to make sure the code changes look correct and then press the green “Create pull request” button.
When you open your pull request, we have a GitHub template that will guide you on what to write in the message box. Please fill this in and title the pull request. Now the final step is to press the green “Create pull request” button.
As soon as you do this, you will be greeted by a message from the “sunpy bot” as well as several continuous integration checks. These are explained on our Pull Request Review page. But what is important to know is that these run a series of tests to make sure that the changes do not cause any new errors. We strongly recommend that any code changes you have had, follow the PEP8 style and that you have ran the code locally to make sure any changes do not break any existing code. We provide an overview on how to run the test suite here. Now we (the sunpy community) can review the code and offer suggestions and once we are happy, we can merge in the pull request.
If you do not have time to finish what you started on or ran out of time during a sprint and do not want to submit a pull request, you can create a git patch instead:
$ git format-patch main --stdout > my_fix.patch
You can rename my_fix to something more relevant.
This way, you still get acknowledged for the work you have achieved.
Now you can email this patch to the Google Group .
Just remember, if you have any problems get in touch!
Summer of Code(s)#
If you are interested in a “Summer of Code” project with sunpy, we have information on our wiki which has guidelines, advice, application templates and more! Our projects are located on our umbrella’s organization website, OpenAstronomy.
Pull Request Check List#
The pull request (commonly referred to as a PR) check list below is an outline of the steps that should be taken when making a contribution to a SunPy repository on Github.
New to Git and Github? Check out the Newcomers’ Guide and Git Cheat Sheets.
If you would prefer a visual Git interface, you can try Github Desktop or GitKraken.
- Review and test changes locally on your machine (see Testing Guidelines).
Double check that a pull request does not exist for the changes you are making. Ideally, check that there is an issue that details what you want to change and why.
If you are contributing code, review the Coding Standards page.
See the Developer’s Guide for guidelines regarding code tests, documentation, and other types of contributions.
Have you tested your changes with the latest version of
sunpy? If not, update your local copy from your remote repository on Github.
- Push your changes to Github.
Create a new branch in your fork of
sunpy.Give this branch a name that reflects the changes you are making.
Create commits that describe the changes.
Push your changes to the new branch on your remote fork.
- Compare your branch with
sunpy/main. Resolve any conflicts that occur ideally with a git rebase.
- Compare your branch with
- Create a pull request.
Create a pull request from the branch you have created/updated.
Add a description to the pull request that describes the changes you have made. Remember to delete the preamble within the message box.
Link to any relevant issues, pull requests, or comments in your description.
- Add a changelog to your pull request.
A changelog is a short record of the type of changes made in your pull request. Other users are the intended audience, and you can have multiple logs per pull request.
- Maintainers will review your pull request Pull Requests and GitHub Teams.
Tweak anything that others highlight and push the changes to your branch. You can also commit suggestions either in bulk or single commits via the Github user interface.
Discuss possible changes or improvements in the comments with the reviewers.
- Review the Continuous Integration (CI) What runs on our Continuous Integration tests and fix any errors or warnings that are found.
If you are confused by an error that the continuous integration is giving you, submit a comment in your pull request.
- Ask questions if you get stuck or confused at any point!
Open-source projects are about communication and collaboration.
This guide is partially based on Astropy’s Development Workflow.
Using conda for Development Dependencies#
The conda package for sunpy specifies only those dependencies that are relevant for the typical user.
A power user or a developer will instead need the complete set of dependencies to use all optional features, build the documentation, and/or run the full test suite.
We provide a conda environment file (sunpy-dev-env.yml) in the base directory of the GitHub repository.
You can create a conda environment (here named sunpy-dev) from this file:
$ conda env create -n sunpy-dev -f sunpy-dev-env.yml
$ conda activate sunpy-dev
Alternatively, if you want to add the complete set of dependencies to an existing conda environment, you can use this file to update that environment (here named existing-env):
$ conda env update -n existing-env -f sunpy-dev-env.yml
The above calls assume that you have already downloaded sunpy-dev-env.yml, either by itself or along with the whole repository.
You can alternatively supply the URL for the file, as hosted on GitHub, to either of the above calls, e.g.:
$ conda env create -n sunpy-dev -f https://raw.githubusercontent.com/sunpy/sunpy/main/sunpy-dev-env.yml
Note
This conda environment file intentionally does not specify restrictions on release versions for any dependency.
This conda environment file specifies only the dependencies for sunpy, and not sunpy itself.
Depending on your needs, continue with one of the following two ways to install sunpy in this conda environment.
Normal Installation of sunpy#
If you do not plan to modify the code of sunpy itself, you can simply install sunpy via conda:
$ conda install sunpy
Since the conda environment already has the complete set of dependencies for sunpy, this call should install only sunpy and no additional packages.
Editable Installation of sunpy#
If you plan to modify the code of sunpy itself, you will want to perform an “editable install” of your local repository, via pip, so that Python will link to your local repository.
Normally it is discouraged to have an environment that mixes package installations via conda with package installations via pip because it can lead to environment states that confuse the conda solver.
That is the reason why our instructions for new developers recommends that dependencies be installed exclusively via pip.
However, some of the dependencies in the complete set are difficult or even impossible to install via pip alone, yet are straightforward to install via conda.
Using the above conda environment, combined with a little care, it is possible to minimize that chance for any conda/pip conflicts.
From the base directory of your local repository, install sunpy with some additional pip options:
$ pip install --no-deps --no-build-isolation -e .
The --no-deps and --no-build-isolation options ensure that pip does not itself install any dependencies.
Since the conda environment is designed to already have the complete set of dependencies, the pip installation should succeed.
You now have a conda environment with an editable installation of sunpy and with (nearly) all dependencies managed by conda.
As you install other packages in this environment, a package that depends on sunpy will trigger conda to install sunpy.
That is fine!
This conda installation of sunpy will simply mask the pip installation of sunpy.
All you need to do is to remove the conda installation with the --force option so that dependencies are left undisturbed:
$ conda remove --force sunpy
Once the conda installation of sunpy is removed, the pip installation of sunpy will automatically be accessible again.
Note
For those who use mamba instead of conda, most conda commands can be translated by simply substituting “mamba” for “conda”. However, mamba remove does not support the --force option, so you do in fact have to call conda remove.
As a tip, you can follow a similar procedure to incorporate editable installations of other packages (e.g., astropy) in a conda environment.
You first install the package via conda to ensure its dependencies are present, then you remove the package alone without disturbing the dependencies, and finally you perform the editable install of the package from the base directory of your local repository:
$ conda install astropy
$ conda remove --force astropy
$ pip install --no-deps --no-build-isolation -e .
Conventions
Coding Standards#
The purpose of the page is to describe the standards that are expected of all the code in the SunPy project. All potential developers should read and abide by the following standards. Code which does not follow these standards closely will not be accepted.
We try to closely follow the coding style and conventions proposed by Astropy.
Language Standard#
All code must be compatible with Python 3.7 and later. Usage of
six,__future__, and2to3is no longer acceptable.The new Python 3 formatting style should be used (i.e.
"{0:s}".format("spam")instead of"%s" % "spam").The core package and affiliated packages should be importable with no dependencies other than components already in the sunpy core package, the Python Standard Library, and packages already required by the sunpy core package. Adding dependencies to sunpy core will be considered but are highly discouraged. Such optional dependencies should be recorded in the
pyproject.tomlfile in theproject.optional-dependenciessection.
Coding Style/Conventions#
The code will follow the standard PEP8 Style Guide for Python Code. In particular, this includes using only 4 spaces for indentation, and never tabs.
Follow the existing coding style within a file and avoid making changes that are purely stylistic. Please try to maintain the style when adding or modifying code.
Following PEP8’s recommendation, absolute imports are to be used in general. We allow relative imports within a module to avoid circular import chains.
The
import numpy as np,import matplotlib as mpl, andimport matplotlib.pyplot as pltnaming conventions should be used wherever relevant.from packagename import *should never be used (except in__init__.py)Classes should either use direct variable access, or Python’s property mechanism for setting object instance variables.
Classes should use the builtin
superfunction when making calls to methods in their super-class(es) unless there are specific reasons not to.supershould be used consistently in all subclasses since it does not work otherwise.Multiple inheritance should be avoided in general without good reason.
__init__.pyfiles for modules should not contain any significant implementation code.__init__.pycan contain docstrings and code for organizing the module layout.
Private code#
It is often useful to designate code as private, which means it is not part of the user facing API, only used internally by sunpy, and can be modified without a deprecation period. Any classes, functions, or variables that are private should either:
Have an underscore as the first character of their name, e.g.,
_my_private_function.If you want to do that to entire set of functions in a file, name the file with a underscore as the first character, e.g.,
_my_private_file.py.
If these might be useful for other packages within the sunpy ecosphere, they should be made public.
Utilities in sunpy#
Within sunpy, it might be useful to have a set of utility classes or functions that are used by internally to help with certain tasks or to provide a certain level of abstraction.
These should be placed either:
sunpy.{subpackage}.utils.py, if it is only used within that sub-package.sunpy.utilif it is used across multiple sub-packages.
These can be private (see section above) or public. The decision is up to the developer, but if these might be useful for other packages within the sunpy ecosphere, they should be made public.
Formatting#
We enforce a minimum level of code style with our continuous integration (the name is sunpy.sunpy (python_codestyle [linux]).
This runs a tool called pre-commit.
The settings and tools we use for the pre-commit can be found in the file .pre-commit-config.yaml at the root of the sunpy git repository.
Some of the checks are:
* Checks (but doesn’t fix) various PEP8 issues with flake8.
* Sort all imports in any Python files with isort.
* Remove any unused variables or imports with autoflake.
We suggest you use “tox” (which is used to run the sunpy test suite) to run these tools without having to setup anything within your own Python virtual environment:
$ tox -e codestyle
What you will see is this output (heavily condensed):
codestyle create: /home/<USER>/GitHub/sunpy/.tox/codestyle
codestyle run-test: commands[0] | pre-commit install-hooks
codestyle run-test: commands[1] | pre-commit run --verbose --all-files --show-diff-on-failure
flake8...................................................................Passed
- hook id: flake8
- duration: 1.35s
0
Check for case conflicts.................................................Passed
- hook id: check-case-conflict
- duration: 0.08s
Trim Trailing Whitespace.................................................Failed
- hook id: trailing-whitespace
- duration: 0.08s
- exit code: 1
- files were modified by this hook
Fixing docs/dev_guide/code_standards.rst
pre-commit hook(s) made changes.
If you are seeing this message in CI, reproduce locally with: `pre-commit run --all-files`.
To run `pre-commit` as part of git workflow, use `pre-commit install`.
All changes made by hooks:
diff --git a/docs/dev_guide/code_standards.rst b/docs/dev_guide/code_standards.rst
index bed700d90..c6b5df977 100644
--- a/docs/dev_guide/code_standards.rst
+++ b/docs/dev_guide/code_standards.rst
@@ -59,6 +59,8 @@ Instead of installing this, you can use "tox" (which is used to run the sunpy te
$ tox -e codestyle
+What you will see
+
If you want to setup the pre-commit locally, you can do the following::
$ pip install pre-commit
diff --git a/docs/dev_guide/documentation.rst b/docs/dev_guide/documentation.rst
index 5cd914047..b1017f77a 100644
--- a/docs/dev_guide/documentation.rst
+++ b/docs/dev_guide/documentation.rst
@@ -39,9 +39,9 @@ If there are multiple code elements with the same name (e.g. ``peek()`` is a met
.. code-block:: rst
- `GenericMap.peek` or `CompositeMap.peek`
+ `.GenericMap.peek` or `.CompositeMap.peek`
-These will show up as `GenericMap.peek` or `CompositeMap.peek`.
+These will show up as `.GenericMap.peek` or `.CompositeMap.peek`.
To still show only the last segment you can add a tilde as prefix:
ERROR: InvocationError for command /home/nabil/GitHub/sunpy/.tox/codestyle/bin/pre-commit run --verbose --all-files --show-diff-on-failure (exited with code 1)
___________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR: codestyle: commands failed
This will inform you of what checks failed and why, and what changes (if any) the command has made to your code.
If you want to setup the pre-commit locally, you can do the following:
$ pip install pre-commit
Now you can do:
$ pre-commit run --all-files
which will run the tools on all files in the sunpy git repository. The pre-commit tools can change some of the files, but in other cases it will report problems that require manual correction. If the pre-commit tool changes any files, they will show up as new changes that will need to be committed.
Automate#
Instead of running the pre-commit command each time you can install the git hook:
$ pre-commit install
which installs a command to .git/hooks/pre-commit which will run these tools at the time you do git commit and means you don’t have to run the first command each time.
We only suggest doing the install step if you are comfortable with git and the pre-commit tool.
Documentation and Testing#
American English is the default language for all documentation strings and inline commands. Variables names should also be based on English words.
Documentation strings must be present for all public classes/methods/functions, and must follow the form outlined in the Documentation page. Additionally, examples or tutorials in the package documentation are strongly recommended.
Write usage examples in the docstrings of all classes and functions whenever possible. These examples should be short and simple to reproduce–users should be able to copy them verbatim and run them. These examples should, whenever possible, be in the doctests format and will be executed as part of the test suite.
Unit tests should be provided for as many public methods and functions as possible, and should adhere to the standards set in the Testing Guidelines document.
Data and Configuration#
We store test data in
sunpy/data/testas long as it is less than about 100 kB. These data should always be accessed via thesunpy.data.test.get_test_filepath()andsunpy.data.test.get_test_data_filenames()functions.We store data used for examples in the sample-data repository. This data should not be used for unit tests but can be within our documentation.
All persistent configuration should use the Customizing sunpy mechanism. Such configuration items should be placed at the top of the module or package that makes use of them, and supply a description sufficient for users to understand what the setting changes.
Standard output, warnings, and errors#
The built-in print(...) function should only be used for output that is explicitly requested by the user, for example print_header(...) or list_catalogs(...).
Any other standard output, warnings, and errors should follow these rules:
For errors/exceptions, one should always use
raisewith one of the built-in exception classes, or a custom exception class. The nondescriptExceptionclass should be avoided as much as possible, in favor of more specific exceptions (IOError,ValueError, etc.).For warnings, one should always use the functions in
sunpy.util.exceptionsand notwarnings.warn. This ensures we are always raising a sunpy specific warning type.
Including C Code#
C extensions are only allowed when they provide a significant performance enhancement over pure Python, or a robust C library already exists to provided the needed functionality.
The use of Cython is strongly recommended for C extensions.
If a C extension has a dependency on an external C library, the source code for the library should be bundled with sunpy, provided the license for the C library is compatible with the sunpy license. Additionally, the package must be compatible with using a system-installed library in place of the library included in sunpy.
In cases where C extensions are needed but Cython cannot be used, the PEP 7 Style Guide for C Code is recommended.
C extensions (Cython or otherwise) should provide the necessary information for building the extension.
Testing Guidelines#
This section describes the testing framework and format standards for tests in sunpy. Here we have heavily adapted the Astropy version, and it is worth reading that link.
The testing framework used by sunpy is the pytest framework, accessed through the pytest command.
Note
The pytest project was formerly called py.test, and you may
see the two spellings used interchangeably.
Dependencies for testing#
Since the testing dependencies are not actually required to install or use sunpy, they are not included in “install_requires” in “pyproject.toml”.
Developers who want to run the test suite will need to install the testing packages using pip:
$ pip install -e ".[tests]"
If you want to see the current test dependencies, you check “extras_require” in “pyproject.toml”.
Running Tests#
There are currently two different ways to invoke the sunpy tests.
However, we strongly suggest using tox as the default one.
Each method uses the widely-used pytest framework and are detailed below.
tox#
The primary method is to use tox, which is a generic virtualenv management and test command line tool. We have several environments within our “tox.ini” file and you can list them:
$ tox -v -l
Then you can run any of them doing:
$ tox -e <name of env>
This will create a test environment in “.tox” and build, install sunpy and runs the entire test suite. This is the method that our continuous integration uses.
pytest#
The test suite can be run directly from the native pytest command.
In this case, it is important for developers to be aware that they must manually rebuild any extensions by running pip install -e . before testing.
To run the entire suite with pytest:
$ pytest
will use the settings in pyproject.toml.
If you want to run one specific test file:
$ pytest sunpy/map/tests/test_mapbase.py
or one specific test in a test file:
$ pytest sunpy/map/tests/test_mapbase.py::<test_name>
(This does not work with tox and is a known issue.)
If a test errors, you can use pdb to create a debugging session at the moment the test fails:
$ pytest --pdb
If you see mention of:
UserWarning: FigureCanvasAgg is non-interactive, and thus cannot be shown
You will have to either export MPLBACKEND = agg as an environmental variable or pass it as a command line pre-argument to pytest.
This comes from the figure tests (see below).
self_test#
Another method is to use sunpy.self_test():
import sunpy
sunpy.self_test()
You will see something like the following in your terminal:
Starting sunpy self test...
Checking for packages needed to run sunpy:
All required and optional sunpy dependencies are installed.
Starting the sunpy test suite:
...
The tests will run and will report any fails. You can report these through the `sunpy issue tracker <https://github.com/sunpy/sunpy/issues>`__ and we will strive to help.
It is possible to run this command in a situation where not all packages are installed. If this is the case, you will see the following when you run the test suite:
Starting sunpy self test...
Checking for packages needed to run sunpy:
The following packages are not installed for the sunpy[database] requirement:
* sqlalchemy
...
You do not have all the required dependencies installed to run the sunpy test suite.
If you want to run the sunpy tests install the 'tests' extra with `pip install "sunpy[tests]"`
This does not mean sunpy is broken, but you will need to install the extra packages to ensure a “complete” installation of sunpy and run the entire test suite. It is quite likely that you will run into not having the tests dependencies installed.
Remote data#
By default, no online tests are selected and so to run the online tests you have to:
$ tox -e py311-online
or:
$ pytest --remote-data=any
Figure tests#
In order to avoid changes in figures due to different package versions, we recommend using tox to run the figure tests:
$ tox -e py311-figure
This will ensure that any figures created are checked using the package versions that were used to create the original figure hashes. Running this will create a folder, “figure_test_images”, within your work folder (“<local clone location>/figure_test_images”), which is ignored by git. Inside this folder will be all the images created, as well as a json file with the hashes of the figures created by the test run. The current hashes are located within “sunpy/tests/figure_hashes_mpl_<ver>_ft_<ver>_astropy_<ver>.json” and this will be where you will need to update old hashes or create new figure entries if anything changes. The filenames are the versions of Matplotlib, freetype and astropy used. If these versions differ to your local setup, the figure tests will not run. In theory, the Python version does not change the results as we have pinned the packages that cause the hash to vary.
Running tests in parallel#
It is possible to speed up sunpy’s tests using the pytest-xdist plugin. This plugin can be installed using pip:
pip install pytest-xdist
Once installed, tests can be run in parallel using the --parallel commandline option.
For example, to use 4 processes:
$ tox -e <name of environment> -- -n=4
or:
$ pytest -n 4 ./sunpy
Coverage reports#
sunpy can use pytest-cov generate test coverage reports and settings are stored in pyproject.toml.
This plugin can be installed using pip:
$ pip install pytest-cov
To generate a test coverage report, use:
$ pytest --cov ./sunpy
This will print to the terminal a report of line coverage of our test suite. If you want to create a report in html, you can run:
$ pytest --cov-report xml:cov.xml --cov ./sunpy
$ coverage html
Writing tests#
pytest has the following test discovery rules:
* ``test_*.py`` or ``*_test.py`` files
* ``Test`` prefixed classes (without an ``__init__`` method)
* ``test_`` prefixed functions and methods
We use the first one for our test files, test_*.py and we suggest that developers follow this.
A rule of thumb for unit testing is to have at least one unit test per public function.
Simple example#
The following example shows a simple function and a test to test this function:
def func(x):
"""Add one to the argument."""
return x + 1
def test_answer():
"""Check the return value of func() for an example argument."""
assert func(3) == 5
If we place this in a test.py file and then run:
$ pytest test.py
The result is:
============================= test session starts ==============================
python: platform darwin -- Python 3.8.3 -- pytest-3.2.0
test object 1: /Users/username/tmp/test.py
test.py F
=================================== FAILURES ===================================
_________________________________ test_answer __________________________________
def test_answer():
> assert func(3) == 5
E assert 4 == 5
E + where 4 = func(3)
test.py:5: AssertionError
=========================== 1 failed in 0.07 seconds ===========================
Sometimes the output from the test suite will have xfail meaning a test has passed although it has been marked as @pytest.mark.xfail), or skipped meaning a test that has been skipped due to not meeting some condition (online and figure tests are the most common).
You need to use the option -rs for skipped tests and -rx for xfailed tests, respectively.
Or use -rxs for detailed information on both skipped and xfailed tests.
Where to put tests#
Each package should include a suite of unit tests, covering as many of the public methods/functions as possible. These tests should be included inside each package, e.g:
sunpy/map/tests/
“tests” directories should contain an __init__.py file so that the tests can be imported.
Online Tests#
There are some tests for functions and methods in sunpy that require a working connection to the internet.
pytest is configured in a way that it iterates over all tests that have been marked as pytest.mark.remote_data and checks if there is an established connection to the internet.
If there is none, the test is skipped, otherwise it is run.
Marking tests is pretty straightforward, use the decorator @pytest.mark.remote_data to mark a test function as needing an internet connection:
@pytest.mark.remote_data
def func(x):
"""Add one to the argument."""
return x + 1
Tests that create files#
Tests may often be run from directories where users do not have write permissions so tests which create files should always do so in temporary directories. This can be done with the pytest tmpdir function argument or with Python’s built-in tempfile module.
Tests that use test data#
We store test data in “sunpy/data/test” as long as it is less than about 100 kB.
These data should always be accessed via the sunpy.data.test.get_test_filepath() and sunpy.data.test.get_test_data_filenames() functions.
This way you can use them when you create a test.
You can also use our sample data but this will have to be marked as an online test (see above):
import sunpy.data.sample
@pytest.mark.remote_data
def func():
"""Returns the file path for the sample data."""
return sunpy.data.sample.AIA_131_IMAGE
Generally we do not run the tests on our sample data, so only do this if you have a valid reason.
Figure unit tests#
Note
The figure tests and the hashes they use are only checked on Linux and might be different on other platforms. We should suggest if you do not use a Linux, to add a fake hash to the json files and then CircleCi (ran on a PR) will tell you the real hash to use.
You can write sunpy unit tests that test the generation of Matplotlib figures by adding the decorator sunpy.tests.helpers.figure_test.
Here is a simple example:
import matplotlib.pyplot as plt
from sunpy.tests.helpers import figure_test
@figure_test
def test_simple_plot():
plt.plot([0,1])
The current figure at the end of the unit test, or an explicitly returned figure, has its hash (currently SHA256) compared against an established hash collection (more on this below).
If the hashes do not match, the figure has changed, and thus the test is considered to have failed.
If you are adding a new figure test you will need to generate a new hash library:
$ tox -e py311-figure -- --mpl-generate-hash-library=sunpy/tests/figure_hashes_mpl_332_ft_261_astropy_42.json
The filename changes if the version of astropy or Matplotlib or freetype gets updated. So you might need to adjust this command. For the development figure tests:
$ tox -e py311-figure-devdeps -- --mpl-generate-hash-library=sunpy/tests/figure_hashes_mpl_dev_ft_261_astropy_dev.json
This will run the figure test suite and update the hashes stored.
If you want to check what the images look like, you can do:
$ tox -e py311-figure -- --mpl-generate-path=baseline
The images output from the tests will be stored in a folder called .tmp/py311-figure/baseline or baseline in the sunpy folder, so you can double check the test works as you expected.
doctests#
Code examples in the documentation will also be run as tests and this helps to validate that the documentation is accurate and up to date. sunpy uses the same system as Astropy, so for information on writing doctests see the astropy documentation.
You do not have to do anything extra in order to run any documentation tests.
Within our pyproject.toml file we have set default options for pytest, such that you only need to run:
$ pytest <file to test>
to run any documentation test.
Bugs discovered#
In addition to writing unit tests new functionality, it is also a good practice to write a unit test each time a bug is found, and submit the unit test along with the fix for the problem. This way we can ensure that the bug does not re-emerge at a later time.
Documentation#
Overview#
All code must be documented and we follow these style conventions described here:
We recommend familiarizing yourself with this style.
Referring to other code#
To link to other methods, classes, or modules in sunpy you have to use backticks, for example:
`sunpy.map.GenericMap`
generates a link like this: sunpy.map.GenericMap.
We use the sphinx setting default_role = 'obj' so that you do not nor SHOULD NOT use the :class: qualifier, but :func:, :meth: are different (more on this below).
Often, you don’t want to show the full package and module name. As long as the target is unambiguous you can simply leave them out:
`.GenericMap`
and the link still works: GenericMap.
If there are multiple code elements with the same name (e.g. peek() is a method in multiple classes), you’ll have to extend the definition:
`.GenericMap.peek` or `.CompositeMap.peek`
These will show up as GenericMap.peek or CompositeMap.peek.
To still show only the last segment you can add a tilde as prefix:
`~.GenericMap.peek` or `~.CompositeMap.peek`
Other packages can also be linked via intersphinx:
`numpy.mean`
will return this link: numpy.mean.
This works for Python, Numpy and Astropy (full list is in docs/conf.py).
With Sphinx, if you use :func: or :meth:, it will add closing brackets to the link.
If you get the wrong pre-qualifier, it will break the link, so we suggest that you double check if what you are linking is a method or a function.
:class:`numpy.mean()`
:meth:`numpy.mean()`
:func:`numpy.mean()`
will return two broken links (“class” and “meth”) but “func” will work.
sunpy-Specific Rules#
For all RST files, we enforce a one sentence per line rule and ignore the line length.
Standards on docstring length and style are enforced using docformatter:
docformatter -r -i --pre-summary-newline --make-summary-multi-line
Heading style is the following for all RST files:
* with overline, for titles
= for sections
- for subsections
^ for subsubsections
" for paragraphs
Anchors for each page should follow this format:
sunpy-<section>-<subsection>-<summary of title>., e.g.,sunpy-tutorial-acquiring-data-index.Use of
.. code-block:is required for all code examples.
Documenting Data Sources#
Subclasses of GenericMap or TimeSeries must provide a detailed docstring providing an overview of the data source that the object represents.
In order to maintain consistency and completeness, the following information must be provided by a data source docstring, if available, and preferably in the following order:
the name of the mission and instrument and the institution that built it
short description of the instrument (e.g. Cassegrain reflector, Wolter-1 grazing incidence x-ray, coronagraph) including the type of detector
description of the platform (e.g. satellite in 28 deg inclined orbit, a telescope on the summit of Mauna Kea in Hawaii)
description of the primary purpose or science goals of the instrument.
list of all wavelength(s) or passbands in appropriate units
description of the emission processes which dominate in those passbands
appropriate measurement properties such as field of view, angular resolution, time resolution
description of the operational concept (e.g. operates 24/7, observes from 7 am to 5 pm UT) including mention of unusual operations scenarios (e.g. calibration seasons, eclipse seasons)
the start and end of the data set
In addition, a reference section must be provided with links to the following resources, if available,
the mission web page
the instrument web page
relevant wikipedia page(s)
relevant user guide(s)
the mission paper and instrument paper
information to interpret metadata keywords such as FITS header reference
the data archive
Examples can be found in any class defined in any Python file in sunpy/map/sources/ such as AIAMap.
Sphinx#
All of the sunpy documentation (like this page) is built by Sphinx, which is a tool especially well-suited for documenting Python projects. Sphinx works by parsing files written using a a Mediawiki-like syntax called reStructuredText. In addition to parsing static files of reStructuredText, Sphinx can also be told to parse code comments. In fact, in addition to what you are reading right now, the Python documentation was also created using Sphinx.
Usage#
All of the sunpy documentation is contained in the “docs” folder and code documentation strings. The examples from the example gallery can be found in the “examples” folder.
In the root directory run:
tox -e build_docs-gallery
This will generate HTML documentation for sunpy in the “docs/_build/html” directory. You can open the “index.html” file to browse the final product. The gallery examples are located under “docs/_build/html/generated/gallery”. Sphinx builds documentation iteratively, only adding things that have changed.
If you want to build the documentation without executing the gallery examples, i.e. to reduce build times while working on other sections of the documentation you can run:
tox -e build_docs
If you’d like to start from scratch (i.e., remove the tox cache) then change to the docs/ directory and run:
make clean
cd ..
tox -r -e build_docs-gallery
To build the documentation in your current python environment you must have all the dependencies specified in pyproject.toml installed (pip install -e .[docs,docs-gallery]).
Then change to the docs/ directory and run:
make html
For more information on how to use Sphinx, consult the Sphinx documentation.
Special Sphinx directives#
minigallery directive#
Sphinx will automatically record which functions, classes, etc. are used in each gallery example. In the documentation, you can insert a mini-gallery of the subset of the gallery examples that uses a particular function, class, etc. For example, the following RST block:
.. minigallery:: sunpy.coordinates.RotatedSunFrame
produces this mini-gallery:
If you want to specify more than one object, separate them by spaces. This is particularly useful if you need to cover multiple namespaces in which an object may be accessed, e.g.:
.. minigallery:: sunpy.coordinates.RotatedSunFrame sunpy.coordinates.metaframes.RotatedSunFrame
generate directive#
In rare circumstances, one may want to insert “raw” HTML directly into the pages written by Sphinx.
For HTML that is statically available (i.e., already written in some form), one can use the “raw” directive.
For HTML that is generated by Python code, sunpy provides the custom directive generate.
Here’s an example RST block:
.. generate:: html
:html_border:
import os
from sunpy.data.sample import file_dict
print("<table>")
for key, value in file_dict.items():
print(f"<tr><th>{key}</th><td>{os.path.basename(value)}</td></tr>")
print("</table>")
to insert the following HTML table:
| AIA_094_IMAGE | AIA20110607_063305_0094_lowres.fits |
|---|---|
| AIA_131_IMAGE | AIA20110607_063301_0131_lowres.fits |
| AIA_1600_IMAGE | AIA20110607_063305_1600_lowres.fits |
| AIA_1600_VENUS_IMAGE | aia_lev1_1600a_2012_06_06t04_07_29_12z_image_lev1_lowres.fits |
| AIA_171_IMAGE | AIA20110607_063302_0171_lowres.fits |
| AIA_171_ROLL_IMAGE | aiacalibim5.fits |
| AIA_193_CUTOUT01_IMAGE | AIA20110607_063307_0193_cutout.fits |
| AIA_193_CUTOUT02_IMAGE | AIA20110607_063931_0193_cutout.fits |
| AIA_193_CUTOUT03_IMAGE | AIA20110607_064555_0193_cutout.fits |
| AIA_193_CUTOUT04_IMAGE | AIA20110607_065219_0193_cutout.fits |
| AIA_193_CUTOUT05_IMAGE | AIA20110607_065843_0193_cutout.fits |
| AIA_193_IMAGE | AIA20110607_063307_0193_lowres.fits |
| AIA_193_JUN2012 | AIA20120601_000007_0193_lowres.fits |
| AIA_211_IMAGE | AIA20110607_063302_0211_lowres.fits |
| AIA_304_IMAGE | AIA20110607_063334_0304_lowres.fits |
| AIA_335_IMAGE | AIA20110607_063303_0335_lowres.fits |
| AIA_STEREOSCOPIC_IMAGE | aia_lev1_171a_2023_07_06t00_05_33_35z_image_lev1.fits |
| CALLISTO_SPECTRUM | BIR_20110607_062400_10.fit |
| EIT_195_IMAGE | eit_l1_20110607_203753.fits |
| EUVI_STEREOSCOPIC_IMAGE | 20230706_000525_n4eua.fts |
| EVE_TIMESERIES | 20110607_EVE_L0CS_DIODES_1m.txt |
| GBM_TIMESERIES | glg_cspec_n5_110607_v00.pha |
| GOES_XRS_TIMESERIES | go1520110607.fits |
| HMI_LOS_IMAGE | HMI20110607_063211_los_lowres.fits |
| LOFAR_IMAGE | LOFAR_70MHZ_20190409_131136.fits |
| LYRA_LEVEL3_TIMESERIES | lyra_20110607-000000_lev3_std.fits |
| NORH_TIMESERIES | tca110607.fits |
| RHESSI_IMAGE | hsi_image_20110607_063300.fits |
| RHESSI_TIMESERIES | hsi_obssumm_20110607_025.fits |
| SRS_TABLE | 20110607SRS.txt |
| STEREO_A_195_JUN2012 | 20120601_000530_n4eua.fits |
| STEREO_B_195_JUN2012 | 20120601_000530_n4eub.fits |
| SWAP_LEVEL1_IMAGE | swap_lv1_20110607_063329.fits |
Troubleshooting#
Sphinx can be very particular about formatting, and the warnings and errors aren’t always obvious.
Below are some commonly-encountered warning/error messages along with a human-readable translation:
WARNING: Duplicate explicit target name: “xxx”.
If you reference the same URL, etc more than once in the same document sphinx will complain. To avoid, use double-underscores instead of single ones after the URL.
ERROR: Malformed table. Column span alignment problem at line offset n
Make sure there is a space before and after each colon in your class and function docs (e.g. attribute : type, instead of attribute: type). Also, for some sections (e.g. Attributes) numpydoc seems to complain when a description spans more than one line, particularly if it is the first attribute listed.
WARNING: Block quote ends without a blank line; unexpected unindent.
Lists should be indented one level from their parents.
ERROR: Unknown target name: “xxx”
In addition to legitimate errors of this type, this error will also occur when variables have a trailing underscore, e.g., xxx_.
WARNING: Explicit markup ends without a blank line; unexpected unindent.
This usually occurs when the text following a directive is wrapped to the next line without properly indenting a multi-line text block.
WARNING: toctree references unknown document ‘…’ / WARNING: toctree contains reference to nonexisting document
This pair of errors is due to the way numpydoc scrapes class members.
Example Gallery#
The purpose of the page is to describe the contribution guidelines for the sunpy Example Gallery.
All potential contributors to the sunpy example gallery should read and abide by the following guidelines.
Contribution Guidelines#
The title of the example should be short yet descriptive and emphasize the goal of the example. Try to make the title appeal to a broad audience and avoid referencing a specific instrument, catalog, or anything wavelength dependent.
Each example should begin with a paragraph that gives a brief overview of the entire example, including relevant astronomy concepts, and motivates the described functionality.
The examples must be compatible with the versions supported by the last major release of the
sunpycore package.All the examples must be fully PEP8 compliant, the
pre-commithook should be used to ensure this.Wherever possible, the examples should include linked references with links pointing to the appropriate DOI or ADS entry.
The example should include links (URL or sphinx intersphinx) to relevant documentation pages.
Each example should, where possible, include at least one image, map, or plot to use as the icon in the example gallery.
The examples should avoid using acronyms without defining them first (e.g. Virtual Solar Observatory, or VSO). Similarly complex jargon should be avoided unless clearly explained.
There should be a good variety of examples for each section (simple and more complex to cater for different levels).
When creating a plot, particularly of a map, the example should follow these rules of thumb to minimize verbosity and maintain consistency with the other gallery examples:
Always create a figure instance using
plt.figure()prior to creating a plot.Always create an Axes instance, and where possible use this to modify the plot instead of using the
pyplotinterface (for example useax.set_xlabel()`instead ofplt.xlabel()).If an axes instance is created, it should be explicitly passed as a keyword argument wherever possible (e.g. in
plotordraw_grid).At the end of the example, call
plt.show(). While not explicitly needed for the gallery to render, this ensures that when run as a script, the plots will appear.If you need to use
astropy.visualization.{quantity_support, time_support}, import these functions at the top of the example, and call them directly before the first plot that needs them.
We recommend checking the other examples in the gallery to see how they are structured and what they contain.
Pull Requests and GitHub Teams#
This document describes the standards required for a pull request to sunpy and an explanation of our automated tests.
Each pull request must meet the following criteria before it is considered for merge:
The code must be PEP 8 compliant and meet the ill-defined sunpy quality standards. We have these in the Coding Standards page.
The PR must contain a changelog entry if it changes the behavior of any code.
The test coverage should not decrease, and for new features should be at or very close to 100%.
All code must be properly documented. Each function and each class must have an associated documentation string in the correct format.
Review Process#
Before the “merge” button is clicked the following criteria must be met:
All the continuous integration must pass unless there is a known issue.
At least two members (excluding the PR author) of the “sunpy-developers” group must have approved the PR. Exceptions can be made for minor changes, see below.
All comments posted on the thread must be resolved.
It is important that approval for merging the PR is always done by explicitly approving the PR through the GitHub UI before merging, so a record of approval is left in the PR thread.
Minor changes#
If a PR only makes minor changes, it can be merged by the first reviewer, if they are confident they fully understand the changes.
If this happens, the minor-change label should be added to the PR to indicate that it has been considered minor enough to need only one reviewer.
The PR author can add this label as a suggestion, but the first reviewer can remove the label as part of their evaluation.
Exactly what constitutes a minor-change is left up to the the reviewer but some examples might include: - Improvements to existing documentation - Small bugfixes - Changes in code style rather than substance
As a guideline, minor-changes don’t include:
New features
New documentation pages
Changes to the public facing API
Continuous Integration#
Currently we have a variety of services that respond or activate on an opened pull request.
Comments from bots:
pep8speaks: Performs a PEP8 check on any submitted code. This is updated as the code changes.
Checks that appear at the bottom of a pull request:
or at the top under the “Checks” tab:
figure-tests (CircleCi): Runs two figure tests environments (“ci/circleci: py3_-figure”, “ci/circleci: py3_-figure-devdeps”).
figure_report (Giles): Show the final results and download updated hashes of the figure tests.
figure_report_devdeps (Giles): Show the final results and download updated hashes of the figure tests using development packages.
changelog: absent | found (Giles): If a changelog is needed, this will check and will pass if a changelog with the correct number is found.
docs/readthedocs.org:sunpy (Read the Docs): This builds our documentation. This primary check is to ensure the documentation has rendered correctly. Warnings are not checked on this build but under GitHub Actions (see below).
CI (GitHub Actions): Runs our test suite on multiple operating systems. If the minimal “CI / core” tests are successful, the indepth “CI / test”, documentation “CI / docs” test and remote data “CI / online” tests will be run. You will see multiple jobs within each group. Each job corresponds to a tox environment being run on a particular operating system.
codecov/patch (CodeCov): Checks how many lines of the code lack test coverage for the submitted code in the pull request.
codecov/project (CodeCov): Checks how many lines of the code lack test coverage in sunpy overall.
pre-commit.ci - pr: Checks the code style checks have passed. This CI will automatically fix style issues by commenting
pre-commit.ci autofixon its own line in a comment on the PR.
It is common to see some of these checks fail. This can be happen due to a change that has broken a test (should be fixed) or a remote server has failed (might have to wait for it to come back). Therefore it is important to check why a task failed and if has a pre-existing issue, it can be safe to ignore a failing check on that pull request. However, you should try to ensure that as many checks pass before merging.
Understanding GitHub Actions#
The vast majority of our tests are run on GitHub Actions and this means you might have to navigate to the results if you want to check why the tests failed. The tests for GitHub Actions are split into multiple phases to reduce the number of builds running at one time. If your PR fails the minimal initial stage, the subsequent stages tests will not run.
The Azure checks on GitHub manifest:
This is the main form. There will be one check per GitHub Actions job ran. The publish and notify jobs are skipped in PRs, and each stage has an additional “Load tox environments” job to configure set up the stage. The “Details” link will show you the log output of the particular check:
On the left you should see the entire list of GitHub Actions checks. You can navigate between the jobs here. You can also see a flow diagram for the jobs by clicking on “Summary”.
For each of the jobs you can see each step that is undertaken. Normally the “Run tox” step will be red if the tests have failed. You will need to click on this so it will load the output from the test suite.
Our test suite is very verbose, so there will be a lot of text outputted. The important bits of information should be at the bottom as “pytest” prints out a test summary at the end. For example:
============================================================================= short test summary info =============================================================================
SKIPPED [1] d:\a\1\s\.tox\py37\lib\site-packages\pytest_doctestplus\plugin.py:178: unable to import module local('d:\\a\\1\\s\\.tox\\py37\\lib\\site-packages\\sunpy\\io\\setup_package.py')
SKIPPED [213] d:\a\1\s\.tox\py37\lib\site-packages\pytest_remotedata\plugin.py:87: need --remote-data option to run
SKIPPED [18] d:\a\1\s\.tox\py37\lib\site-packages\_pytest\doctest.py:387: all tests skipped by +SKIP option
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\map\sources\tests\test_source_type.py:21: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\map\sources\tests\test_source_type.py:30: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:22: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:31: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:40: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:49: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:58: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_ana.py:67: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_filetools.py:54: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_filetools.py:73: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_filetools.py:106: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_filetools.py:115: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_filetools.py:122: ANA is not available.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_jp2.py:11: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_jp2.py:21: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\io\tests\test_jp2.py:31: Glymur can not be imported.
SKIPPED [1] .tox\py37\lib\site-packages\sunpy\net\tests\test_fido.py:298: Windows.
FAILED ..\..\.tox\py37\lib\site-packages\sunpy\timeseries\sources\noaa.py::sunpy.timeseries.sources.noaa.NOAAGoesSXRTimeSeries
If you want to find the full test output, you can search the tab for the name of the test out of the ~3 results, one will be that output.
SunPy GitHub Groups#
This document has already referred to two SunPy groups, namely “developers” and “maintainers” there is also a third primary SunPy group “owners”.
SunPy owners#
The SunPy owners group is the group of people who have total control over the SunPy GitHub organization. The SunPy board have control over who is in this group, it has been decided that generally it will be the Lead Developer and the SunPy board chair and vice-chair.
sunpy Maintainers#
This is the group of people who have push access to the main sunpy repository. The membership of this group is at the discretion of the Lead Developer, but shall generally be made up of people who have demonstrated themselves to be trust worthy and active contributors to the project.
This group has subgroups for each section of the repository that has maintainers. The members of these groups will automatically be requested to review all PRs which change files in that subpackage.
sunpy Developers#
The members of this group have “read” access to the sunpy repository. As all these repository are open anyway, what this effectively means is that these people can be assigned to issues. The members of this group are people who are involved in the development of sunpy at a good frequency, they are people who’s opinions have been demonstrated to be constructive and informative.
Use of quantities and units#
Much code perform calculations using physical quantities. SunPy uses astropy’s quantities and units implementation to store, express and convert physical quantities. New classes and functions should adhere to the SunPy project’s quantity and unit usage guidelines.
This document sets out SunPy’s reasons and requirements for the usage of quantities and units. Briefly, SunPy’s policy is that all user-facing function/object arguments which accept physical quantities as input **MUST* accept astropy quantities, and ONLY astropy quantities*.
Developers should consult the Astropy Quantities and Units page for the latest updates on using quantities and units. The astropy tutorial on quantities and units also provides useful examples on their capabilities.
Astropy provides the decorator quantity_input that checks the units of the input arguments to a function against the expected units of the argument.
We recommend using this decorator to perform function argument unit checks.
The decorator ensures that the units of the input to the function are convertible to that specified by the decorator, for example
>>> import astropy.units as u
>>> @u.quantity_input
... def myfunction(myangle: u.arcsec):
... return myangle**2
This function only accepts arguments that are convertible to arcseconds. Therefore:
>>> myfunction(20 * u.degree)
<Quantity 400. deg2>
returns the expected answer but:
>>> myfunction(20 * u.km)
Traceback (most recent call last):
...
astropy.units.core.UnitsError: Argument 'myangle' to function 'myfunction' must be in units convertible to 'arcsec'.
raises an error.
The following is an example of a use-facing function that returns the area of a square, in units that are the square of the input length unit:
>>> @u.quantity_input
... def get_area_of_square(side_length: u.m):
... """
... Compute the area of a square.
...
... Parameters
... ----------
... side_length : `~astropy.units.quantity.Quantity`
... Side length of the square
...
... Returns
... -------
... area : `~astropy.units.quantity.Quantity`
... Area of the square.
... """
...
... return (side_length ** 2)
This more advanced example shows how a private function that does not accept quantities can be wrapped by a function that does:
>>> @u.quantity_input
... def some_function(length: u.m):
... """
... Does something useful.
...
... Parameters
... ----------
... length : `~astropy.units.quantity.Quantity`
... A length.
...
... Returns
... -------
... length : `~astropy.units.quantity.Quantity`
... Another length
... """
...
... # the following function either
... # a] does not accept Quantities
... # b] is slow if using Quantities
... result = _private_wrapper_function(length.convert('meters').value)
...
... # now convert back to a quantity
... result = Quantity(result_meters, units_of_the_private_wrapper_function)
...
... return result
In this example, the non-user facing function _private_wrapper_function requires a numerical input in units of meters, and returns a numerical output.
The developer knows that the result of _private_wrapper_function is in the units units_of_the_private_wrapper_function, and sets the result of some_function to return the answer in those units.
Repo management
Workflow for Maintainers#
This page is for maintainers who can merge our own or other peoples’ changes into the upstream repository.
Seeing as how you’re a maintainer, you should be completely on top of the basic git workflow in Newcomers’ Guide and Astropy’s git workflow.
Integrating changes via the web interface (recommended)#
Whenever possible, merge pull requests automatically via the pull request manager on GitHub. Merging should only be done manually if there is a really good reason to do this!
Make sure that pull requests do not contain a messy history with merges, etc. If this is the case, then follow the manual instructions, and make sure the fork is rebased to tidy the history before committing.
Integrating changes manually#
First, check out the “sunpy” repository. Being a maintainer, you’ve got read-write access.
It’s good to have your upstream remote have a scary name, to remind you that it’s a read-write remote:
$ git remote add upstream-rw git@github.com:sunpy/sunpy.git
$ git fetch upstream-rw
Let’s say you have some changes that need to go into trunk (upstream-rw/main).
The changes are in some branch that you are currently on. For example, you are looking at someone’s changes like this:
$ git remote add someone git://github.com/someone/sunpy.git
$ git fetch someone
$ git branch cool-feature --track someone/cool-feature
$ git checkout cool-feature
So now you are on the branch with the changes to be incorporated upstream. The rest of this section assumes you are on this branch.
If you prefer not to add remotes, you can make git fetch all pull requests opened to Sunpy.
Locate the section for your git remote in the .git/config file.
It looks like this:
[remote "upstream"]
url = git@github.com:sunpy/sunpy.git
fetch = +refs/heads/*:refs/remotes/upstream/*
Now add the line fetch = +refs/pull/*/head:refs/remotes/upstream/pr/* to this section.
It ends up looking like this:
[remote "upstream"]
url = git@github.com:sunpy/sunpy.git
fetch = +refs/heads/*:refs/remotes/upstream/*
fetch = +refs/pull/*/head:refs/remotes/upstream/pr/*
Now fetch all the pull requests:
$ git fetch upstream
From github.com:sunpy/sunpy
* [new ref] refs/pull/1000/head -> upstream/pr/1000
* [new ref] refs/pull/1002/head -> upstream/pr/1002
* [new ref] refs/pull/1004/head -> upstream/pr/1004
* [new ref] refs/pull/1009/head -> upstream/pr/1009
To check out a particular pull request:
$ git checkout pr/999
Branch pr/999 set up to track remote branch pr/999 from upstream.
Switched to a new branch 'pr/999'
When to remove or combine/squash commits#
In all cases, be mindful of maintaining a welcoming environment and be helpful with advice, especially for new contributors. It is expected that a maintainer would offer to help a contributor who is a novice git user do any squashing that that maintainer asks for, or do the squash themselves by directly pushing to the PR branch.
Pull requests must be rebased and at least partially squashed (but not necessarily squashed to a single commit) if large (approximately >10KB) non-source code files (e.g. images, data files, etc.) are added and then removed or modified in the PR commit history (The squashing should remove all but the last addition of the file to not use extra space in the repository).
Combining/squashing commits is encouraged when the number of commits is excessive for the changes made. The definition of “excessive” is subjective, but in general one should attempt to have individual commits be units of change, and not include reversions. As a concrete example, for a change affecting < 50 lines of source code and including a changelog entry, more than a two commits would be excessive. For a larger pull request adding significant functionality, however, more commits may well be appropriate.
As another guideline, squashing should remove extraneous information but should not be used to remove useful information for how a PR was developed. For example, 4 commits that are testing changes and have a commit message of just “debug” should be squashed. But a series of commit messages that are “Implemented feature X”, “added test for feature X”, “fixed bugs revealed by tests for feature X” are useful information and should not be squashed away without reason.
When squashing, extra care should be taken to keep authorship credit to all individuals who provided substantial contribution to the given PR, e.g. only squash commits made by the same author.
When to rebase#
Pull requests must be rebased (but not necessarily squashed to a single commit) if:
There are commit messages include offensive language or violate the code of conduct (in this case the rebase must also edit the commit messages)
Pull requests may be rebased (either manually or with the rebase and merge button) if:
There are conflicts with main
There are merge commits from upstream/main in the PR commit history (merge commits from PRs to the user’s fork are fine)
Asking contributors who are new to the project or inexperienced with using git is discouraged, as is maintainers rebasing these PRs before merge time, as this requires resetting of local git checkouts.
A few commits#
If there are only a few commits, consider rebasing to upstream:
# Fetch upstream changes
$ git fetch upstream-rw
# Rebase
$ git rebase upstream-rw/main
A long series of commits#
If there are a longer series of related commits, consider a merge instead:
$ git fetch upstream-rw
$ git merge --no-ff upstream-rw/main
Note the --no-ff above.
This forces git to make a merge commit, rather than doing a fast-forward, so that these set of commits branch off trunk then rejoin the main history with a merge, rather than appearing to have been made directly on top of trunk.
Check the history#
Now, in either case, you should check that the history is sensible and you have the right commits:
$ git log --oneline --graph
$ git log -p upstream-rw/main..
The first line above just shows the history in a compact way, with a text representation of the history graph.
The second line shows the log of commits excluding those that can be reached from trunk (upstream-rw/main), and including those that can be reached from current HEAD (implied with the .. at the end).
So, it shows the commits unique to this branch compared to trunk.
The -p option shows the diff for these commits in patch form.
Push to open pull request#
Now you need to push the changes you have made to the code to the open pull request:
$ git push git@github.com:<username>/sunpy.git HEAD:<name of branch>
You might have to add --force if you rebased instead of adding new commits.
Using Milestones and Labels#
Current milestone guidelines:
Only confirmed issues or pull requests that are release critical or for some other reason should be addressed before a release, should have a milestone. When in doubt about which milestone to use for an issue, do not use a milestone and ask other the maintainers.
Current labelling guidelines:
Issues that require fixing in main, but that also are confirmed to apply to supported stable version lines should be marked with a “Affects Release” label.
All open issues should have a “Priority <level>”, “Effort <level>” and “Package <level>”, if you are unsure at what level, pick higher ones just to be safe. If an issue is more of a question or discussion, you can omit these labels.
If an issue looks to be straightforward, you should add the “Good first issue” and “Hacktoberfest” label.
For other labels, you should add them if they fit, like if an issue affects the net submodule, add the “net” label or if it is a feature request etc.
Using Projects#
Projects allow us to layout current pull requests and issues in a manner that enables a more “meta” view regarding major releases. We categorize pull requests and issues into several levels of priorities and whether these can be classed as blockers before a release can be attempted. Further we can add general notes that someone deems important for a release.
Updating and Maintaining the Changelog#
The changelog will be read by users, so this description should be aimed at sunpy users instead of describing internal changes which are only relevant to the developers.
The current changelog is kept in the file “CHANGELOG.rst” at the root of the repository. You do not need to update this file as we use towncrier to update our changelog. This is built and embedded into our documentation.
Towncrier will automatically reflow your text, so it will work best if you stick to a single paragraph, but multiple sentences and links are OK and encouraged.
You can install towncrier and then run towncrier --draft if you want to get a preview of how your change will look in the final release notes.
Releasing sunpy#
We have a step by step checklist on the SunPy Wiki on how to release the sunpy core package.
Dependency Support Policy#
Note
This policy is based on NEP-0029.
sunpy has a short list of core dependencies (Python, numpy, astropy, parfive) and a long list of optional dependencies. The minimum version of these packages that we enforce follows this policy.
Python: Released in the prior 42 months from the anticipated release date.
astropy: Released in the prior 12 months from the anticipated release date.
Everything else: Released in the prior 24 months from the anticipated release date.
Sponsored affiliated packages will support at least the sunpy LTS version at the time of their release.
For dependencies only needed to run our tests we will support versions released in the prior 12 months to the current date.
What runs on our Continuous Integration#
Overall the aim of the Continuous Integration is to provide a way for contributors to test their code on multiple platforms and help in the development process. While we can run many jobs, we aim to limit the selection depending in the context to ensure that the time taken is not wasted or ends up delaying a pull request or release.
The goal is that, the builds we do not run on a pull request should primarily be ones which are unlikely to be the fault of the change in the pull request if they fail.
Currently we have several stages of CI jobs, some run on a pull request and some will only run on a schedule.
- “core” - Pull Request, Scheduled and Release
This runs a basic offline test suite on Linux for the latest version of Python we support. It ensures that basic functionality works and that we don’t have any regressions.
- “test” - Pull Request, Scheduled and Release
This runs a basic test suite on Windows and Mac OS for older versions of Python we support. It ensures that basic functionality works and that we don’t have any regressions. This stage needs to wait for the “core” stage to complete. Furthermore, we run:
“oldestdeps” - Check the offline test suite with the oldest dependencies installed.
- “docs” - Pull Request, Scheduled and Release
This runs a documentation build (without executing gallery examples) to test that the HTML documentation can be generated without any errors or warnings. The build is cached and the gallery examples are then tested during the “online” stage.
- “online” - Pull Request, Scheduled
This runs a full test suite on Linux for a version of Python. This build can fail (due to a range of reasons) and can take up to 45 minutes to run. We want to ensure that the offline tests are passing before we consider running this. Therefore this stage needs to wait for the “core” stage to complete.
In addition, we run the documentation build to execute the gallery examples which can fail due to the same reasons as the online build. The documentation build from the “docs” stage is cached and restored for this documentation build. Therefore this stage also needs to wait for the “docs” stage to complete.
As these are not tested when we build the distribution or wheels, we skip these on a release. These should be checked based on the last commit for that release branch instead before a tag.
- “cron” - Scheduled
Here we put builds that are useful to run on a schedule but not so useful on a pull request. This allows us to run more “exotic” or focused builds that day to day, should not affect a pull request.
These are:
“base_deps” - Check that sunpy does not error if you only have the base dependencies installed.
“devdeps” - Check the offline test suite with the development dependencies installed. Likely to break due to upstream changes we can’t fix and have to wait to be resolved.
“conda” - Check the offline test suite when using conda instead of pip/pypi. Likely to break due to packaging upstream we can’t fix and have to wait to be resolved.
Making Changes to Released Versions#
When changes need to be made in a bugfix release of an already released version of sunpy this is done by a process called “backporting”. The process is as follows:
Open a Pull Request (PR) as normal targeting the
mainbranch.Apply a label to that PR (or get a maintainer to do it) of the format
Backport X.Y.Once the PR has been merged a bot will open a new PR with the same changes targeting the
X.Ybranch.
Managing backports is done by the sunpy maintainers, as a contributor you generally shouldn’t have to worry about it. The following documentation is for maintainers on how to interact with the backport bot.
Controlling the Backport Bot#
The backport bot in use on the sunpy repo is MeeseeksDev.
Upon merging a PR the bot will look for the existence of a label with the following template in the label description field on-merge: backport to X.Y.
If the decision to backport a PR is taken after the merge of the PR, then a command needs to be added to a comment on the PR: @MeeseeksDev backport [to] {branch}
Manual Backports#
If a backport fails, meeseeks will add a comment to the PR and add a label named “Still Needs Manual Backport”. If you then manually backport the PR, please remove this label when the backport PR is open so that the label is an accurate list of PRs in need of manual backporting.
When doing a manual backport please do not forget to put the PR number you are backporting in the description of the PR so that GitHub links the two PRs together. (This automatic linking does not happen if the number is in the title).
Warning
HEAVY WIP
Submodule and funding#
Here we list whatever parts of the SunPy project have been funded by external sources such as grants or Google Summer of Code (GSoC) and the like.
Database#
Google Summer of Code (2013)#
Parts of the now removed sunpy.database module.
Net#
Google Summer of Code (2014)#
fido_factory.py, vso.py, most of the dataretriever submodule
ESA Summer of Code in Space (2011)#
attr.py tools/hektemplate.py tools/hek_mkcls.py
Extending sunpy
sunpy’s Public API#
Convention in the Python ecosystem is to add an underscore to the start of a function to denote if a function or method is “private” e.g., _angular_radius.
If it is considered to be private, there is no guarantee that the API or behavior will change with a warning, so external use of these functions or methods are strongly discouraged.
sunpy follows this convention but with one extra caveat.
Within each python file, we have a __all__ that defines what is imported into the namespace if you do e.g., from sunpy.coordinates.sun import *.
This is the “public” API of that module.
These functions are the ones listed within our API documentation: Reference.
If you do import sunpy.coordinates.sun, you can still access the “private” functions.
This means that all of the public API will follow the deprecation policy detailed below with the exception of sunpy.util which is considered to be for internal sunpy use only.
Deprecation Policy and Breaking Changes#
All public API within the SunPy project (the sunpy core package and stable affiliated packages) will enforce strict standards when it comes to either changing, updating or breaking the API.
Deprecations#
If you want to deprecate anything within in sunpy, you should do the following:
from sunpy.util.decorators import deprecated
@deprecated(since="3.1", message="We will be moving this", alternative="sunpy.net.Scraper")
class Scraper:
The deprecation warning has to be in one LTS release before the deprecated code can be removed. So in the above example, the warning will be in sunpy 3.1 but it can not be removed until sunpy 4.1 after the 4.0 LTS release.
There should be a “deprecation” changelog entry to accompany the deprecation warning. When the code is actually removed, a “removal” changelog should be added.
The same applies if you want to change the default value of a keyword argument for a function or method, e.g.:
from sunpy.util.exceptions import warn_deprecated
if response_format is None:
response_format = "legacy"
warn_deprecated("The default response format from the VSO client will "
"be changing to 'table' in version 3.1. "
"To remove this warning set response_format='legacy' "
"to maintain the old behaviour or response_format='table'"
" to use the new behaviour.")
Note
This is a summary of SEP-0009 which is the formal SunPy project deprecation policy.
Breaking Changes#
Every attempt is made to avoid breaking any public API in sunpy but in the case it does happen.
There should be a “breaking” changelog entry to accompany the change with as much detail on how to update a user’s code due to this breaking change.
Logging, Warnings, and Exceptions#
Overview#
sunpy makes use of a logging system to deal with messages (see Logger Objects). This provides the users and
developers the ability to decide which messages to show, to capture them, and to optionally also send
them to a file. The logger will log all messages issued directly to it but also warnings issued
through warnings.warn as well as exceptions.
The logger is configured as soon as sunpy is imported. You can access it by importing it explicitly:
from sunpy import log
Messages can be issued directly to it with the following levels and in the following way:
log.debug("Detailed information, typically of interest only when diagnosing problems.")
log.info("A message conveying information about the current task, and confirming that
things are working as expected.")
log.warning("An indication that something unexpected happened, or indicative of
some problem in the near future (e.g. disk space low).
The software is still working as expected.")
log.error("Due to a more serious problem, the software has not been able to
perform some function but the task is still continuing.")
log.critical("A serious error, indicating that the program itself may be unable to
continue running. A real error may soon by issued and the task will fail.")
The difference between logging a warning/error/critical compared to issuing a Python warning or raising an exception are subtle but important.
Use Python warnings.warn in library code if the issue is avoidable and the user code should be
modified to eliminate the warning.
Use log.warning() if there is likely nothing the user can do about the situation, but the event
should still be noted. An example of this might be if the input data are all zeros. This may be unavoidable or
even by design but you may want to let the user know.
True exceptions (not log.error()) should be raised only when there is no way for the function to proceed.
Regardless of the type (log message or warning or exception) messages should be one or two complete sentences that fully describe the issue and end with a period.
Issuing Warnings#
sunpy warnings are provided by the sunpy.util module. The primary warning which
should be used is sunpy.util.exceptions.SunpyUserWarning. For deprecation use sunpy.util.exceptions.SunpyDeprecationWarning or
sunpy.util.exceptions.SunpyPendingDeprecationWarning.
These three warning types have corresponding functions to raise them:
>>> from sunpy.util.exceptions import warn_user
>>> from sunpy.util.exceptions import warn_deprecated
>>> from sunpy.util.exceptions import warn_metadata
These warning functions must be used to interact correctly with the logging system. A warning can be issued in the following way:
>>> from sunpy.util.exceptions import warn_user
>>> warn_user("You have been warned about something you did not do correctly.")
See the section above for a discussion about the distinction between log.warn() and raising a warning.
Raising Exceptions#
Raising errors causes the program to halt. Likely the primary error that a sunpy developer will want to use is
ValueError: should be raised if the input is not what was expected and could not be used. Functions should not return anything (like None) in that case or issue a warning.
Exceptions are raised simply with:
>>> raise ValueError("The following error occurred.")
For more information on exceptions see the Built-in Exceptions documentation.
Tests using the Remote Data Manager#
A pytest fixture (sunpy_cache) is provided for ease of mocking network requests when using cache.
The following example demonstrates the usage of the fixture:
@manager.require('test_file',
['http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt'],
'4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11')
def test_function():
return manager.get('test_file')
@pytest.fixture()
def local_cache(sunpy_cache):
sunpy_cache = sunpy_cache('sunpy.test_module.cache')
sunpy_cache.add('http://example.com/test_file',
'test_data_path')
The above snippet creates a pytest fixture called local_cache.
This fixture can be used in wherever the files have to be mocked.
An example is given below:
def test_test_function(local_cache):
# inside this function the mocked cache is used
# test_function uses 'http://example.com/test_file'
assert test_function() == True