Archive for category Development

NEW: Solar features and events from the Heliophysics Event Knowledgebase

The Sun has many different features and events of great scientific interest. It’s useful to be able to catalog those features and measure their properties. By doing so, we can build up more knowledge about the Sun.

The Heliophysics Event Knowledgebase (HEK) is one such catalog. The HEK collects and stores information about many different types of solar feature – active regions, flares, etc, from many different sources around the world. Each solar feature and event can be detected in different ways. Some features and events are detected by people looking at the data, and some are detected by specialized computer vision algorithms.

We’ve taken the information in the HEK and designed a simple interface to allow you to find out what features and events occurred on the Sun at any given time. We’ve organized the information in the HEK by feature/event. You may need to reload helioviewer.org to get the latest version which includes the HEK.

HV intro: Screenshot from 2013-07-16 10:18:37

The numbers at the end of each line tell you the total number of features/events of each type on the Sun at that time. You can select any combination of features and events you want (green tick marks), or you can select none. If you’ve selected a particular type of feature/event but the text is faded out, this means that there are none of those particular feature/events at the time you’ve requested. If you browse forward and backwards and time and those feature/events are in the HEK, helioviewer.org will display them.

We then break down the total number of features/events by feature recognition method. We do this because different feature recognition methods can give different results for the same feature/event type. You can select any combination of the available feature recognition methods, or you can select none. For example, the active regions on the Sun at this time were detected using two different feature recognition methods:

HV Intro: Screenshot from 2013-07-16 10:19:05

The numbers at the end of each line tell you how many active regions were detected by SPoCA (a computer vision algorithm) and the NOAA SWPC Observer (people looking at the data).

Here’s a typical view of some AIA 304 data with the HEK features and events overplotted.

HV Intro: Screenshot from 2013-07-16 10:29:31

Each of the marker pins corresponds to the feature/event detected by a feature recognition method. Clicking on them pulls up much more information on each individual event. Each of the marker pins also has a small label attached to it with an important piece of information concerning that feature/event. We’ve also extended the movie and screenshot capability so that your selected feature/event markers and labels appear in any movies and screenshots you make.

Finally, in the bottom left-hand corner of the viewer window you’ll see a small image of the Earth. This is the size of the Earth on the same scale as the solar and heliospheric images. This also appears in movies and screenshots of the Sun. Full information on using helioviewer.org can be found by clicking the help link at the bottom of the helioviewer.org webpage.

The HEK is the result of much work by many different people around the world. We are happy to be able to present data from this great solar and heliospheric resource in Helioviewer.org.

Did you like this? Share it:

11 Comments

Yohkoh SXT full disk images now available

We are happy to announce the availability of full disk Yohkoh Soft X-ray Telescope (SXT) images on helioviewer.org. SXT images x-rays from the Sun, and therefore looked at some of the hottest plasmas on the Sun. These data are important in trying to understand solar flares and the heating of the Sun’s corona.

Yohkoh (“Sunshine”) was launched in August 30, 1991, from the Kagoshima Space Center (Uchinoura) in Japan, and was a project of the Japanese Institute of Space and Astronautical Science (ISAS). The scientific objective was to observe the energetic phenomena taking place on the Sun, specifically solar flares in x-ray and gamma-ray emissions.

satlist2

Yohkoh carried four instruments to detect energetic emissions from the Sun: the Soft X-Ray Telescope (SXT), the Hard X-Ray Telescope (HXT), the Bragg Crystal Spectrometer (BCS) and the Wide Band Spectrometer (WBS). A team from the United States collaborated on SXT, and teams from the United States and the United Kingdom collaborated on BCS.

SXT imaged X-rays in the 0.25 – 4.0 keV range. SXT used thin metallic filters to acquire images in restricted portions of the energy range. We are making images from the thin aluminium filter (thin-Al), and the aluminium-magnesium-manganese filter (AlMgMn) available. White-light images are also available up until November 1992. An example thin-aluminium image is shown below.

1993_02_07_03_43_31_SXT_thin-Al

SXT could resolve features down to 2.5 arc seconds in size. Information about the temperature and density of the plasma emitting the observed x-rays was obtained by comparing images acquired with the different filters. Flare images could be obtained every 2 seconds. Smaller images with a single filter could be obtained as frequently as once every 0.5 seconds.

Yohkoh ceased operations on December 14, 2001. The SXT images we are making available cover portions of Solar Cycles 22 and 23 (we are currently somewhere close to the maximum of Solar Cyce 24). This historical data allows us to compare current solar behavior to previous solar behavior. Such studies allow us to better understand how the Sun operates on timescales of decades and longer.

Yohkoh SXT images are the first images of soft X-ray data available on helioviewer.org. We hope you enjoy examining this different view of the Sun on helioviewer.org.

Did you like this? Share it:

2 Comments

Coding Opportunities for Students: JHelioviewer and SunPy join ESA’s Summer of Code in Space

ESA Summer of Code in Space 2012 (SOCIS) is a program run by the European Space Agency. It offers student developers stipends to write code for various space-related open source software projects. Through SOCIS, students will be paired with mentors from participating project teams, thus gaining exposure to real-world software development. The program is inspired by (but not affiliated or related in any way to) Google’s Summer of Code initiative.

JHelioviewer, in collaboration with the start-up company Luxunda, and SunPy have been selected as mentoring organizations, and students are encouraged to apply until 27 July 2012 (11 am UTC).

Did you like this? Share it:

5 Comments

JHelioviewer 2.2.1 Released

A new version of JHelioviewer is available for download. What’s new? This update release contains improved movie export functionality, an updated LASCO C2 coronagraph mask, the new SDO Cutout Service plug-in plus various bug fixes.

The new movie export menu makes it easier to set the exact scaling of the area you are interested in, and the processing itself is now performed on the graphics card using OpenGL:

New movie export menu

The SDO Cutout Service plug-in allows you to request science-quality image data from the SDO/AIA and HMI instruments for the region of interest and time range selected in JHelioviewer:

SDO Cutout Service plug-in

Did you like this? Share it:

No Comments

Helioviewer.org 2.3.0 Released

A new version of Helioviewer.org has been released including better movie customization, support for embedding Helioviewer.org in remote sites, and a number of performance and bug fixes.

Screenshot showing new movie settings added in Helioviewer.org 2.3.0.

Support has been added for embedding Helioviewer.org into third-party websites, and JSONP support makes it easier for new versions of the front-end to be created which interact with the main Helioviewer.org back-end. Similarly, the front-end has been rewritten to allow for easier creation of custom front-end clients without having to re-implement a tiling system, etc.

New configuration options added in Helioviewer.org 2.3.0.

The back-end movie queuing system has been ported from Ruby to PHP to allow for better integration with the rest of the back-end, and the movies table structure has been modified for improved time estimation and similarity searching. Additional options (frame-rate and movie length) are offered to allow users further control over the movies they create and the duration option has been moved to a more obvious location.

Let us know what you think, or if you have any suggestions. Feedback is always welcome.

RELEASE NOTES:

Helioviewer.2.3.0 includes several new features to give users more control over how the site behaves. Let us know what you think, or if you have any suggestions. Feedback is always welcome.

New features:

* JSONP support
* Added option display date from last visit when returning to Helioviewer.org
* Added setting to automatically update images every 5 minutes
* Added support for embedding Helioviewer.org in other websites
* Added support for specifying frame-rate or duration during movie creation
* Added support for PROBA-2 SWAP data
* Created an installer diagnostic script
* Added support for tracking custom events in Google Analytics
* Movie and screenshot selection rectangle preserved during visit
* Data availability information included in getDateSources response

Bug fixes:

* Fixed bug #691356 JPX Summary file does not exist
* Fixed bug #783497 Port Helioqueuer to PHP
* Fixed bug #903360 Error occurs for certain layer orders when attempting to create AIA/LASCO
* Fixed bug #925542 The minimum width of the display window is too big
* Fixed bug #624857 After clicking “clear history” unfinished requests are still processed, and download links displayed
* Fixed bug #885795 Add image attribution to about dialog
* Fixed bug #888269 Attempt to normalize movie frame-rate instead of duration when possible
* Fixed bug #909795 Normalize date strings for API requests
* Fixed bug #909897 Mark movies that have not finished in less than x hours as Error
* Fixed bug #930628 Improve movie creation time estimation
* Fixed bug #942547 Validate value for dsun before attempting to process in front-side
* Fixed bug #609219 API should return an error message when an invalid parameter is specified in a request
* Fixed bug #783481 Report mouse coordinates immediately upon activation
* Fixed bug #787744 Add a checkSettings method to the UserSettings class to verify user settings integrity
* Fixed bug #876707 Included creation_time in FFmpeg metadata for mp4/webm movies
* Fixed bug #789515 Reduce filesize of WebM movies

Library updates:

* Flowplayer (3.2.7 => 3.2.8)
* jQuery (1.7.0 => 1.7.2)
* jQuery UI (1.8.16 => 1.8.18)
* jQuery.JSON (2.2 => 2.3)
* jQuery imgAreaSelect (0.9.5 => 0.9.8)

Did you like this? Share it:

55 Comments

Movie queue reset this morning

Well, we were hoping it wouldn’t come to this, but unfortunately we had to reset the movie queue this morning, deleting many requests which were made during the last couple days. The reason for the reset is that our server was overwhelmed with requests, to the point where new requests had estimated wait times in numbers of days, which is unreasonable. We looked into many different options, and tried some (including reducing the size of the movies made), but nothing was able to completely solve our problems: the time it would have taken to completely catch-up would have meant that even if other new exciting solar events were to occur, videos of those events would not be processed until days later!

In the future we will try and take measures to keep this from happening again. The last couple of days have made us look at the services we provide in a whole new way, thanks to the huge interest from you, our users, and we are grateful for that. We are constrained by our resources, so we may not be able to process many more movies each minute any time soon, but what we can do is make it easier for users to find other movies that are out there that have already been made. Thousands of movies have been made by users on Helioviewer.org, but right now there is no simple way to search through those movies and find the ones you are interested in. Further, we will also take some measures to make the users more aware of what the current wait time is before they submit a movie request, and also provide them with an option to cancel the request if they decide they do not want to wait for the movie. All of this will make it easier for you to get to the movies you want.

We apologize to all of the users who have queued movies during the last couple days that did not get processed. If you are still interested in making a movies please go ahead and queue it up again. Hopefully we will not have to take such extreme measures again. Thanks to you all for your patience.

Did you like this? Share it:

2 Comments

Helioviewer.org 2.2.0 Released

Helioviewer.org has been updated to include several new features and a number of bug fixes. Although the update is focused primarily on improving the reliability and performance, several new features have been added, and over the next 2-3 weeks you should begin to see new types of images from the STEREO mission.

Try it out! If you have any comments or suggestions, or run into any problems, report a bug or feature request.

RELEASE NOTES:

Helioviewer.org 2.2.0 is a major release including a number of stability and performance improvements, as well as a large amount of bug fixes. The main focus of this release is the recently added movie and screenshot creation capabilities. A large amount of work has been done to improve the reliability of those features, and also to pave the way for new functionality which will be added in upcoming releases and will be directed at giving users more fine-tuned control over the movie creation process.

New features:

* STEREO EUVI/COR image support (images will begin to appear online in the next 2-3 weeks!)
* WebM support
* IE9 HTML5 video support added
* YouTube video sharing support added
* New flash video player (Flowplayer)
* Improved screenshot quality
* Better mobile support (drag and drop, working sliders, etc)
* Added a data monitor page (http://www.helioviewer.org/status)

Bug fixes:

* Fixed bug #693901 Enable lookup of video properties using only its id
* Fixed bug #332020 Double-click zoom functioning incorrectly
* Fixed bug #602756 Optimize movie generation when near end of available data
* Fixed bug #662607 HTML5 Video format choice should be based on browser and not OS
* Fixed bug #662897 Avoid using compressed images during movie frame generation
* Fixed bug #692574 Generate WebM videos for each movie request
* Fixed bug #701509 Use RSUN instead of CDELT when scaling EIT images
* Fixed bug #730701 Speed up time with which shared videos appear on Helioviewer.org
* Fixed bug #735617 Store meta-information for tiles and screenshots generated by Helioviewer.org
* Fixed bug #781148 Use current time as default observation time
* Fixed bug #193782 Cursor bug in IE
* Fixed bug #609225 When attempting to use the Flash video browser, perform check for Flash support
* Fixed bug #612506 Movie pop-up menu returns NaN for older movies
* Fixed bug #621798 Use jQuery instead of Shadowbox when displaying Helioviewer.org Link
* Fixed bug #783483 IE: background image disappears in fullscreen mode
* Fixed bug #785792 When changing one field in a layer (e.g. observatory) attempt to preserve all other properties
* Fixed bug #691081 Add a “Share on Youtube” button
* Fixed bug #696346 Add metadata to generated movies
* Fixed bug #624817 Media History clear button stops working
* Fixed bug #666756 First getDataSources query after new data is added becomes very slow
* Fixed bug #605020 Helioviewer.org Movies should not include redundant frames
* Fixed bug #614001 Movie history does not disappear when creating movie
* Fixed bug #712095 Helioviewer-specific header tags not displayed in information dialog for newer images
* Fixed bug #725539 When building movies, ignore failures limited to a single frame
* Fixed bug #387365 Slider bars do not work on Safari on iPod touch
* Fixed bug #666809 Timestamp text overlaps icons on some Linux systems
* Fixed bug #669895 For movie generation failures specify whether failure occured during image creation or movie encoding
* Fixed bug #697191 High-quality video download button is cut-off on some browsers/display screens
* Fixed bug #700024 Metalinks not easily readable on large screens
* Fixed bug #704980 Suggestion for a better movie player dialog box title
* Fixed bug #376995 Make JP2 creation information available via helioviewer.org

Library updates:

* jQuery (1.4.2 => 1.6)
* jQuery imgAreaSelect (0.9.2 => 0.9.5)
* jQuery jGrowl (1.2.4 => 1.2.5)
* jQuery qTip (1.0 r54 => 2.0.0pre)
* jQuery UI (1.8 => 1.8.12)

Did you like this? Share it:

No Comments

Sneak peak at STEREO data…

We’re working on including data from NASA’s STEREO (Solar TErrestrial RElations Observatory, stereo.gsfc.nasa.gov) mission. It’s a mission consisting of two spacecraft, one drifting ahead of the Earth, and drifting behind, taking images of the Sun and the inner heliosphere. The concept behind the mission is to view the Sun as a three-dimensional object, from which we can better understand its surface structures and how it influences the inner heliosphere. This is the view from STEREO-A, and this is the view of the same event from STEREO-B. Both movies are of coronagraph data taken with the COR2 instrument; both STEREO spacecraft have the same instrument suites onboard.

We hope to have a stream of the very latest STEREO images very soon. Watch this space!

Did you like this? Share it:

No Comments

JHelioviewer 2.1.3 is out

The latest JHelioviewer update adds support for SDO/HMI data and features a new contrast filter, as well as an improved plugin to access the Heliophysics Event Knowledgebase (HEK).

Did you like this? Share it:

No Comments

Helioviewer.org update: Improved movie quality

Helioviewer.org has been updated this morning to include some recent improvement to the movie generation process. The result of this update is that the quality of the movies that you see on Helioviewer.org has been greatly improved.

While Helioviewer.org has offered High-definition H.264 movies for several months now (encoded using the excellent x264 library), the amount of compression used was fairly high. The result of this was very small file sizes (around 1-5MB), but some noticeable compression artifacts; the effect of which was especially noticeable for larger movies.

For example, the below movie was generated on Helioviewer.org several days ago:

Example 1: In-browser movie before update (download video)

A number of changes were made to the H.264 encoding parameters in order to improve the quality, for example, whereas movies were previously generated using a constant variable bit-rate (-b 2048K), the newer movies use a different rate-control method called “Constant Ratefactor (CRF)” in order to achieve a desired level of quality.

Here is an example of a movie created for viewing in the browser using the new code:

Example 2: In-browser movie after update (download video)

What’s more, the “high-quality” download option is now much higher quality than ever before. Previously, when users clicked on the link below in the in-browser movie that says “Click here to download a high-quality version“, what they got was actually the same movie that was already playing in their browser, but packed in a container format compatible with the user’s operating system. With the update this morning, however, the high-quality download link now points to a separate and visibly higher-quality movie from what is shown in the browser. The high-quality option available now is actually a lossless movie, with respect to the underlying JPEG 2000 data archive used by Helioviewer.org.

Example 3: High quality movie after update (download video)

(Note: If you are having trouble viewing the above video, you can download an MP4 version directly from here.)

Of course, nothing comes for free, and that is true in the case of these improvements as well. The improvements made to quality come at the cost of increased movie filesize. For the standard-quality movie that is shown directly in the browser, the filesize has increased from by a factor of around 1.5-10x with the largest files around 50MB each. The real behemoths, however, are the high-quality (lossless) movies which range from around 50-300MB. It’s a lot, but try watching a few of high-quality AIA movies and you won’t ever want to go back again. :)

Update 2010/12/31: Thanks to Dark_Shikari on #x264 for some help making sense of some of the many rate control options available when encoding using x264.

Did you like this? Share it:

2 Comments

Helioviewer.org 2.1.0 Released

Helioviewer.org has been updated to include a number of new features and bug fixes. Although many of the features included in this release have already been available online for some time now, their performance and reliability has been greatly improved during the past several weeks.

Try it out! If you have any comments or suggestions, or run into any problems, report a bug or feature request.

Stayed tuned for some exciting new features next week :)

RELEASE NOTES:

Helioviewer.org 2.1.0 is a major release which includes many new features and a large number of bug fixes. Major features added include: full support for AIA images, on the fly movie and screenshot creation and a custom resource management and queuing system. Other changes include:

New features:

* Overhauled error back-end error handling
* Support for HEK FRM/Event querying
* PHP IMagick used in place of convert when available for better performance (See: http://valokuva.org/?p=40 for more information)
* Custom FFmpeg wrapper used in place of PHPVideoToolkit
* FirePHP support
* Documentation for each back-end module now handled at the module level rather than globally
* IE8 local storage support
* Conversion from tile coordinates to spatial coordinates now handled on front-end
* Removed dependencies on image archive directory structure
* Added a JHelioviewer JNLP generation method so that users can easily jump from Helioviewer.org to JHelioviewer
* Color of image layer timestamps now ranges on a scale from green (near requested time) to red (far from requested time). Absolute difference is used so that it does not matter whether actual image is ahead or behind requested one

Bug fixes:

* Fixed bug #544338 Edge effects for tiles viewed at higher magnification
* Fixed bug #605412 Helioviewer-generated Flash movies degrade in quality as the movie progresses
* Fixed bug #614558 Use of IMagick#extentImage results in images with incorrect dimensions
* Fixed bug #619944 Movie history panel occasionally stays in viewport
* Fixed bug #312205 Sandbox dimensions are too large for C2 and C3 Images
* Fixed bug #383939 Disable tiling when layer is hidden
* Fixed bug #508723 Check to see if data exists for each source, or provide option to disable a data source
* Fixed bug #605398 Improve JPX request cadence optimization for non-evenly distributed data
* Fixed bug #605411 Fullscreen, etc. button tooltips are hidden by movie/screenshot creation dialogs
* Fixed bug #608265 Chrome appends .html on movies
* Fixed bug #610456 FITS header dialog not updated when image is changed
* Fixed bug #645836 shortcuts for time steps “<” or “>” not working
* Fixed bug #535645 Add link to uncompressed JavaScript and CSS for better “view source” function
* Fixed bug #544352 backfill JP2s from SOHO mission up to most recent
* Fixed bug #322857 Sandbox dimensions do not update right away when layers are removed
* Fixed bug #415455 helioviewer logo issues/request

Library updates:

* jQuery imgAreaSelect (0.8 => 0.9.2)
* jQuery jGrowl (1.2.0 => 1.2.4)
* jQuery qTip (1.0 r34 => 1.0 r54)

Did you like this? Share it:

No Comments

Converting your FITS and other image data to JPEG2000 for use with the Helioviewer Project

Many institutions host many different kinds of image data, either from observations or from simulations.  I have put instructions on how to convert image data to a format the Helioviewer Project uses on our wiki page. The wiki is where we keep our must up-to-date instructions on the installation and use of all the software we are developing.

Contact me if you encounter any problem with the JP2Gen instructions , or with implementing JP2Gen on your local system (or any other part of the wiki). I’d be very happy to get your feedback!

Did you like this? Share it:

No Comments