The MAST Portal
- Initiating a Search
- Exploring Search Results
- Refining the Results with Filters
- Retrieving Results
- The AstroView Tool
- The Plotting Tool
- The Cross-Match Tool
- Available Data
- Tips & Notes
- FAQ/Known Issues
- Video Demos
The Barbara A. Mikulski Archive for Space Telescopes (MAST) Data Discovery Portal (DDP, or Portal) is a web application for searching repositories of information about Astronomical Data, Publications and Images.
|Fig. 1 —Anatomy of the MAST Portal as it appears prior to the first search. Circled numbers in orange are keyed to descriptions in the text.|
The initial screen layout consists of the following components, numbered as in Fig. 1:
- 1. "Select Collection" Box and other basic tools
- Use the dropdown menu to choose the collection to search (results vary depending on the information contained within each particular collection). Also has a link to the "about" screen, allows a user to import a CSV or VO table they have saved on their local machine, a button that brings up the user's download basket (if populated), and a button for the user's guide.
- 2. Search Box
- Depending on the collection chosen, a hint appears describing the kind of query that should be performed. Also contains examples that populate the search box for you.
- 3. Data View Panel
- Presents data and images returned from the query or queries, sometimes allowing multiple views of a dataset, such as a grid layout or an album view. If filtering is available for the data panel, applicable filters are grouped into facets by semantic type, and will appear in this panel. Note that the Filters panel can be resized or (un)hidden using screen widgets.
- 4. Astroview Panel
- This powerful all-sky viewer provides visualization context and imagery for either aribtrary positions specified within the viewer itself or while being tied to data from Portal queries.
All searches begin by choosing a collection and entering either the name of an astronomical object, or the RA/Dec coordinates of a sky position in the search entry box in decimal degrees, sexagesimal or galactic coordinates (on the left in Fig. 2). A radius can be appended to the search by adding "r=" and a radius, with units of "d" (degrees), "m" (minutes) or "s" (seconds). Some collections have search radius limitations, and the default radius depends on the size of the object, or is usually .2 degrees for other targets. The search can be initiated by clicking the Search button or by pressing the Enter key on your keyboard while the cursor is in the search entry box.
|Fig. 2 —Search panel includes the Collection pulldown menu, Search box and button, examples of valid search text, and basic tool links.|
Names of objects are passed to a name resolver (one of many Virtual Observatory services that support the Portal), which associates known object names with sky coordinates. A wide variety of catalog and common names can be resolved to known objects. Coordinates can be given in a variety of formats; right ascensions must be positive and southern declinations require a leading negative sign. Examples of valid names and coordinate formats are given below.
||Objects from standard catalogs such as Messier and NGC will be resolved|
||Common names often work|
||Variable star names often work|
||Star catalogs with coordinate symbols|
||Other catalogs with coordinate and decimal symbols|
||All-sky catalogs with coordinate symbols|
||All-sky satellite catalogs with restricted symbols|
||Sexagesimal coordinates delimited with spaces|
||Sexagesimal coordinates delimited with colons; decimal minutes/arcminutes|
||Sexagesimal coordinates with explicit hms/dms|
||Galactic coordinates of the form g[lon][+/-lat] with no spaces|
||Coordinates in decimal degrees|
Further rules and caveats for valid searches may be found in Tips & Notes.
A successful search, such as that shown for M57 below, populates a new Search Results panel with a table of resources, a corresponding Filters panel applicable to these resources, and focuses the AstroView component on the position. If the software detects image outlines (footprints), it will draw them, and will attempt to indicate non-images, such as catalog objects, by marking the position listed for that row. The results of subsequent searches will be stored in new tables, and can be accessed by clicking on the tabs at the top of the Results panel. To eliminate a results table (and discard the results for that search), click the "x" icon on the tab. Concurrent searches are possible. If a new search is initiated prior to the last one completing, a new tab will be created while the previous search continues. Just below the tabs is a status panel with the following items from left to right:
- The number of rows retrieved (and update information for searches in progress).
- Controls for AstroView (more on these later).
- An Export Table button which can write the table of results to your local storage. This works in conjunction with the Import Table button at the top of the page.
- A button with the canonical name from the name resolver and the search radius, which can be expanded to show the coordinates as well.
|Fig. 3 —Appearance of the Portal after a search for m57 within radius = 0.2 degrees, showing the filters (left panel) that can be applied to the Search Results (center panel), and the AstroView component. The results of a prior search/tab may be viewed by clicking the tab (labelled Start Page in this case) at the top of the Search Results panel.|
The records are listed in the table, one per row. Each record represents data about a particular image, catalog or spectra, with the columns defining each datum about the record. In the example above, each displayed record is about a SWIFT image that was found within the bounds of the search.
Clicking the info button in the Search Results panel pops up a panel with summary information about the resource. Clicking the details tab at the top of this panel exposes all of the attributes for the resource.
|Fig. 4 —Summary of a m57 image, showing the summary description of the resource (left). Clicking the details tab at the top brings up all available details. Different details panel formats exist for the various collections. For collections where the tool knows, for example, that there will be an image and a URL that links to a large-format version, it will pop up a details panel such as this one that links to it. Other image records may be displayed differently. Sometimes the format is unavailable and the tool can only provide a generic view of the data.|
Virtual Observatory (VO) Search Results
For search results from the VO, the initial list returned is only the first half of the search. It contains a list of resources from the VO that claim to have data within the searched cone. These data are catalogs, images and spectra. From this list, the user must click the desired object to view a summary of its details. The user may then load the full data from that object by clicking on the Load button in the Summary Panel or in the Actions Column, which will display all of its associated data records into a new tab for further browsing of that resource's objects, images or spectra. This action is referred to as a "drilldown", and is a key component of using the Portal to explore multi-tiered data hierarchies such as the VO. Additionally, the user may download exactly the table, including all metadata, to a file on their local machine, or initiate another search using that resource. In order to search the VO, change the "Select Collection" dropdown to the "All Virtual Observatory Collections" selection.
|Fig. 4.1 —Summary of a m57 VO search, with the underlying catalog having been partially loaded. The details panel for the search allows the user to load the records, which places it into a new tab (marked VO: m57 as shown). The Export button will download the entire VO table to the user's machine as a file, including all metadata. In this example, the user has prepared to initiate a new M83 search using the CADC Image Search resource, rather than querying all of the VO again.|
Many searches, particularly on popular targets or of a sufficiently large area of sky, generate hundreds or even thousands of rows. The Filters panel allows the user to restrict the list of search results to those of most interest.
The list of potential filters is quite large, and includes as many attributes of the matched data collections as the data providers offer. The filters in the panel are grouped into semantically related facets depending on the data. You can view the list of allowed facets by clicking the Edit Facets button at the top of the panel; select the desired facets by clicking the checkboxes in the pop-up window (middle) and clicking the Apply button. Note that not all facets are displayed or even available by default.
There are three types of filters:
- Discrete filters - Used for text and most integer-based facets (columns of data). A filter can be applied by clicking the checkbox next to the desired facet values.
- A special discrete filter can exist when the data rows are selectable, allowing the user to choose whether to filter selected or unselected rows.
- Ranged sliders - Used for continuous numeric data, some integer data, and dates. Each facet presents the user with a histogram approximating the distribution of the data, a 2 handled slider for constraining endpoints, and a min/max box displaying the current endpoints, which also allows a user to manually type the desired endpoints. Some numeric facets are known to have certain values that represent a null or undefined entry, and the facet will alert the user that it is ignoring such a value.
- A search box, which allows regular expression matching by searching all records and columns of the data (even ones not visible in the display grid).
|Fig. 5 — The search has been filtered for rows where the Proposal ID is "PNMRD" with RA between the given values. This updates the counts of the other facet values based on how many other rows can be selected. Regular expressions may also be used as filters by entering a regular expression in the dialog box near the top of the panel. In the numeric facet above, the data were recognized as RA coordinates and is presented in these boxes in sexagesimal format, but a user may define his or her own bounds using sexagesimal or degrees.|
A core capability of the MAST Portal is to enable users to download resources, or to direct them to other desktop applications for analysis. As used here, resources is a rather generic term used to refer to individual data files (images, tables, and the like), or collections of them, or to services that provide data upon request. Any of these resources may be of use scientifically, and the Portal has simple mechanisms for retrieving them.
Continuing with the example shown in Fig. 5 (details of a M57 image), clicking the button will bring up a large-scale version of the image. In some cases, the larger image may be an interactive display using the Hubble Legacy Archive's FITS viewer.
Downloading catalogs or tables is somewhat simpler than downloading images, in that any table displayed in the results panel may be saved with the following steps:
- Click the Export button.
- Select the output format and other parameters from the pop-up panel.
- Click the Export button on the exporter dialog.
Broadcasting Data via SAMP
Many popular applications, including TOPCAT and DS9, have the ability to broadcast/receive in-memory data to another desktop application that is similarly enabled. The Portal can also broadcast a table, via the Simple Application Messaging Protocol (SAMP). The procedure is exactly the same as downloading a table, except that instead of clicking Export as the last step, click the Broadcast button. Note that at least one other SAMP-enabled application, such as TOPCAT or Aladin must already be running in order for data to be transferred anywhere. When a table is broadcast (in this case, from the Portal to TOPCAT) for the first time during a session a security notification may appear.
Finally, note that the Portal does not currently support receiving broadcasts of data from other applications.
Using the Download Basket
For MAST data, the rows of the grid are only a summary of the data products available for download. Select the rows you wish to investigate by checking the box on the far left of the row, and add the associated products to the download basket by clicking the in the grid. Alternatively, each row has action icons to add the data to the basket or to immediately download the data Adding data to the basket pops up a window containing the current download basket, which itself is filterable like any other grid. Most summaries represent many actual data products, so each product gets its own row in the grid. These can then be then be further selected and downloaded using the download button in the upper-right of the grid. Some of these data, such as proprietary data, can only be accessed via the "classic" MAST interface, which requires a login. When this situation occurs, the user may use the "Open DADS" button , which will open a new window/tab to navigate to the MAST page.
|Fig. 6 — Adding the products associated with a record to the download basket. Note that the 3 rows translated into 23 actual data products. Frequently a single row can translate into dozens of products.|
The AstroView Tool is an all-sky-viewing component. It was developed as a standalone application, but was later afforded a means of interaction with the Portal. It uses the same positional search and name- resolving capabilities as the Portal, but instead of displaying tabular results and image files it presents a view of the sky using imagery from various surveys. This view can be panned, zoomed and searched, and the imagery displayed can be changed based on the survey selected using the tool.
One interaction between the Portal and AstroView is the notion of projecting image "footprints", i.e. the detector shape, size and orientation for a given image, onto a background sky map from all-sky survey images. Selected rows in the grid are highlighted in this tool. Clicking footprints or markers in AstroView will not only highlight them, but will select them in the grid and bubble them to the top. Selecting is somewhat inexact by design - many footprints overlay each other exactly, so many rows may be highlighted in the grid with a single click. Figure 7 demonstrates footprints and highlighting.
|Fig. 7 — Footprints from WFPC2 were selected in the AstroView component, so they were highlighted in AstroView, then selected and bubbled to the top of the grid.|
The AstroView controls listed for each grid contain 3 components:
- Display - A toggle/drop-down menu indicating which footprints are to be displayed
- None - show no footprints
- All - show all footprints. Unselected footprints will be the color of the swatch, and selected footprints will appear slightly highlighted.
- Selected - show only footprints that are selected in the grid. This acts as a filter for the tool, and can help viewing a crowded area.
- *It is important to note that the selection from this dropdown works on a grid-by-grid basis. To hide all footprints across several tabs, each tab must be set to "None".
- Color - a swatch indicating the color of footprints associated with this search. Each search/grid may have its own color to allow a user to distinguish between different grid interactions.
- Palette dropdown - allows user to pick the color for this grid's footprints and points in the plotting tool.
The Plotting Tool, activated by using the charts button in the upper right of the grid, allows the user to scatter plot two numeric facets against each other. It provides a panel to allow the user to configure the X and Y axes. The axes may also be set to "Combine Columns" (the last value in each dropdown), which allows for arithmetic between two columns. Adjust the sliders or type the endpoints into the range boxes, then click "Apply Range" to zoom the chart to the new extrema. The user may also shift-click and drag a box on the plot to specify new bounds. The "Reset Range" button will restore the default bounds for the plot. Additionally, selection may be performed in two ways: 1) clicking and dragging (without holding shift) will remove any existing selection and select all points within the box bounds and 2) clicking without dragging will toggle selection of points at that position without removing the existing selection.
|Fig. 8 — Plotting Tool - The plot has been zoomed in from the original bounds, and points selected in AstroView/grid are also highlighted in the plot using the same color as in AstroView. The Y-axis has a value that represents a null (in this case NaN) and explains that any row with this value is excluded from the chart.|
The Cross-Match Tool allows the user to match the positions in their current grid against other catalogs. Press the cross-match button at the upper-right of the grid to bring up the cross-match dialog. Select the resource against which to match, specify the match radius in arcseconds and press the cross-match button to begin the search. The returned results will contain all the columns from the original search, then all the columns from the resource against which it was matched. It is important to note that in order to perform a cross-match, the software must have detected an RA and Declination column.
|Fig. 9 — Cross-Match Tool - There are multiple resources against which the user may match an arbitrary table.|
What data can you find using the MAST Data Discovery Portal?
The MAST DDP uses, as its source for searching a database, a model called the Common Archive Observation Model (CAOM), which will eventually include all the MAST holdings. Currently, not all of MAST's holdings and products are in this database.
Available Data (as of November, 2013):
- HST and HSTONLINE - All HST observations through August 7, 2013
- Hubble Legacy Archive - HLA enhanced products released for HLA 7.1
- GALEX - GR6/7 images and spectra, excepting spectral image files
- FUSE - All FUSE observations
- Kepler - All lightcurves and associated target-pixel files; one set of FFIs (Full Frame Images)
- SwiftUVOT - Most observations through July 2013. There are 143 missing observations that will be added in soon.
- IUE - All low dispersion data - High dispersion coming soon
- EUVE - All data
- HUT - All data
- WUPPE - All data
- TUES - All data
- BEFS - All data
Data that are not yet available through the MAST DDP:
- Kepler - most of the FFI sets
- IUE - High Dispersion
- GALEX - Spectral Image files
- The most recent HST and SwiftUVOT data.
- Standard HST listings/products - The initial load of HST observations into the new database includes data acquired through July 2013. The Portal will display the highest-level exposure or association that is available for a set of data, although the exposure data are also available for download. The download manager has several useful filters to let you choose the data products you need. At present, most of the HST data will be retrieved via the standard HST retrievals also known as DADS. The data available only through DADS is labeled as a being in the "DADS" project. When data are available for download from DADS, the download basket manager interface will have an icon indicating data are available from DADS.
- HST Online data - The FOC, FOS, GHRS and pre-servicing mission 4 STIS data are online and will be downloaded without having to submit a batch request. In spring of 2014, all HST holdings are planned to be available in an online cache. At that time, users will be able to download the data directly. These are flagged as project "HSTONLINE" in the Portal display.
- Hubble Legacy Archive data - The Hubble Legacy Archive Project has created enhanced data products for ACS, WFPC2, & NICMOS images, ACS & WFPC2 source lists; ACS & NICMOS extracted grism spectra, prototype ACS mosaics. As of DR 7.1. See the HLA page for a complete list of the holdings. In the Portal these observations are labeled as being in the "HLA" project.
When formulating a search, note the following:
- Leading zeros are ignored in the name. For example, M5 returns results for object Messier 005.
- Object names are not case-sensitive, and spaces between the characters (e.g. M51 and M 51) are ignored unless the space indicates a real character in the name.
- If an object name is incomplete the name resolvers from NED and SIMBAD return the closest match. This can lead to surprising results however.
- All coordinates are interpreted as J2000.
- Maximum search radius is limited on a collection-by-collection basis.
- VO resources may have arbitrary limits on the number of rows returned, on a resource-by-resource basis.
- Queries are currently limited to a single object or position. However, multiple queries may be submitted sequentially, even if one or more queries are still executing.
When downloading an image, note the following:
- Large images may take a while to download. Download progress can only be monitored using browser tools.
- Downloading is disabled for data products that are embargoed by the provider. It may not be obvious when this happens, except that no file will be downloaded to your computer.
When downloading a table, note the following:
- The Export Table As... operation refers to the currently visible table, and not to a selected row within the table.
When broadcasting a table, note the following:
- At least one other SAMP-enabled application, such as TOPCAT or Aladin must already be running a SAMP hub in order for data to be sent anywhere.
- The MAST Portal cannot currently receive data sent from other applications.
Tabs of search results are not saved between sessions; once the application has been abandoned, returning to it will not bring up previously searched objects. This means that clicking the browser back button will cause the session to be lost.
- The contents of the Download Basket are not persisted across sessions, nor is any customization of the grids.
- Circular footprints (e.g., GALEX) are currently defined as polygons with a finite number of vertices. When zoomed in close in Astroview, you can sometimes see the footprint line segments, rather than a curved trace of the true circular footprint. As such, targets that are very, very close to the edge of the footprint will appear to lie "outside" the footprint when they are actually within it.
- For Galex, background images used in AstroView are from data release 4/5, but the data available from MAST are from release 6/7. This may cause footprints to be placed in the viewer over objects that are not visible in the viewer.
- For Galex, the FUV and NUV observations are typically downloaded as a pair, rather than separately.
Searching Using the Virtual Observatory
Visualization with AstroView
Visual Form Search (SID interface)
Charting and Selection Interactivity
Importing and Exporting Files
To help improve the MAST Portal, please send feedback to The MAST Portal. Please note: the above link may cause you to leave the current page if your browser does not have a default email client setup! In this case, please send email to archive[at]stsci.edu with the subject "Portal Feedback".