2008-03-18: DataFed-GoogleEarth Mashup - Take 2
From Datafedwiki
Back to DataFed Development Events
[edit] Accomplishments
[edit] 2008-06-17: Point Layers for Views Do not Work
Procedure:
- Open the view ARC/VIEWS_84perc_diff_SO4f
- Click the View Google Earth button
- Choose dynamic points or static points and View Current KMZ
Result:
Static points generates the image of the view, which is the wrong image to begin with. The image generated does not represent any of the layers in the view.
Dynamic points generates an error conversion source CAPITA.Generic.HttpDataImageBitmap target CAPITA.Generic.HttpDataTableDSSchema is not yet supported.
Fixed stuff:
Url overflowing is fixed by removing all the useless tags from the ge.aspx page.
You cannot anymore choose points from this page for two reasons:
- There are two candidate layers to get the points, we need selection for that.
- view is cached, so before any point access happens, the image is already retrieved.
So point options are disabled from view that have imagecachefile tag
--Hoijarvi 10:50, 19 June 2008 (CDT)
[edit] 2008-06-17: Global Datasets Cannot Be Viewed and Viewable Datasets Have Incorrect Lat/Lons
Procedure:
- View the dataset AVHRR or Ciesin GPW in the DataFed Viewer
- Click View Google Earth and View Current KMZ
and
- View the dataset ABBI in the DataFed Viewer
- Click View Google Earth and View Current KMZ
Result:
Error message stating validation failed and The MaxInclusive constraint failed. In the KML, the lat/lons go beyond -90/90 and -180/180.
and
The data and map do not line up with the Google Earth world.
Fixed --Hoijarvi 15:22, 18 June 2008 (CDT) Confirmed --MarkK 17:27, 18 June 2008 (CDT)
[edit] 2008-06-11: Computed Data Layer not Viewable in Views
Procedure:
- Open the view ARC/VIEWS_84perc_diff_OCfCombined
- Click on View Google Earth and Save the KMZ
- Click on View Current KMZ
Result: The time instance picture is displayed in GE even though the view is saved out as showing the computed data layer only.
time_min and time_max are set, so the time_range layer is screwed. Need to set loop_time_min and loop_time_max. --Hoijarvi 13:15, 13 June 2008 (CDT)
[edit] 2008-05-30: Certain Views Do not Load into Google Earth
Procedure:
- Open the view ARC/VIEWS_84perc_diff_OCfCombined
- Click on View Google Earth and View Current KMZ
Result:
THe image shows differently from original, because time_min and time_max are used in the url, and data range is used in the view. I have to use different tags from time_min and time_max, like loop_time_min and loop_time_max. --Hoijarvi 12:02, 12 June 2008 (CDT)
[edit] 2008-05-30: Data from Saved Views does not Show Up in Google Earth
Procedure:
- Open the saved view ARC/VIEWS_84perc_diff_Chem
- Click on View Google Earth and View Current KMZ
Result:
The picture in Google Earth is blank. The outline of the states is there, but none of the data in the DataFed Viewer appears.
Fixed the time_min and time_max problem --Hoijarvi 10:55, 19 June 2008 (CDT)
[edit] Improving Current Tools
[edit] Bugs and Issues
[edit] 2008-06-19: Image Layer Snippets Need to be Removed
The Snippets in the Image ground overlays are not related to the ground overlays. CTRL + left click does not do anything when you click on the image. The snippets should be blank.
Done --Hoijarvi 11:17, 23 June 2008 (CDT)
[edit] 2008-06-19: Date Screen Overlay for Dynamic Points Missing
Dynamic Points need a screen overlay to show the date
[edit] 2008-06-17: Changing Layers in Certain Views Creates Different Titles
Procedure:
- Open the view ARC/VIEWS_84perc_diff_SIf
- Change the view layer from time_instance to a different view such as computed.
Result:
The title and date show up incorrectly.
Reason
dataset_abbr = 'computed', there is no param_abbr for the layer and no datetime.
Titles like this need to be moved to the instance layer, or something similar.
--Hoijarvi 12:33, 18 June 2008 (CDT)
[edit] 2008-06-17: Point Layers for Views Do not Work
Procedure:
- Open the view ARC/VIEWS_84perc_diff_SO4f
- Click the View Google Earth button
- Choose dynamic points or static points and View Current KMZ
Result:
Static points generates the image of the view, which is the wrong image to begin with. The image generated does not represent any of the layers in the view.
Dynamic points generates an error conversion source CAPITA.Generic.HttpDataImageBitmap target CAPITA.Generic.HttpDataTableDSSchema is not yet supported.
Fixed stuff:
Url overflowing is fixed by removing all the useless tags from the ge.aspx page.
You cannot anymore choose points from this page for two reasons:
- There are two candidate layers to get the points, we need selection for that.
- view is cached, so before any point access happens, the image is already retrieved.
So point options are disabled from view that have imagecachefile tag
--Hoijarvi 10:50, 19 June 2008 (CDT)
[edit] 2008-06-18: page with static points does not work with time range
ARC/VIEWS_84perc_diff_OCfCombined
-- disabled static points because of image caches --Hoijarvi 10:52, 19 June 2008 (CDT)
[edit] 2008-06-10: ABBI Missing Metadata
Procedure:
- Open the ABBI dataset from the DataFed Catalog.
- Click on View Google Earth and View Current KMZ
Result: No metadata is in the KMZ
http://wiki.esipfed.org/index.php/Special:ExportRDF/ABBI does not export metadata, http://wiki.esipfed.org/index.php/Special:ExportRDF/AIRNOW does
Good metadata is only here: http://datafedwiki.wustl.edu/index.php/ABBI
--Hoijarvi 11:04, 12 June 2008 (CDT)
[edit] 2008-06-06: DataTypes Input
DataTypes in the DataFed Catalog have a specific set of entries that are allowable. The DataType SeqImage is not recognized but Image is. TRAJ is not recognized but TRAJECTORY is.
We need to put this list somewhere: GRID IMAGE POINT TRAJECTORY TABLE URLCUBE --Hoijarvi 11:06, 12 June 2008 (CDT)
[edit] 2008-05-30: Description Balloons generated from saved views have no metadata
Procedure:
- Open the saved view ARC/FRMPM25_84perc
- Click on View Google Earth and View Current KMZ
- In Google Earth, click on the Document description
Result:
No metadata exists in the description balloon
seems to be duplicate with esipfed metadata problem --Hoijarvi 11:17, 12 June 2008 (CDT)
[edit] 2008-05-29: Opened Views Viewed in Google Earth Unable to Change Settings
Procedure:
- Open a View in the Datafed Browser (I used ARC/VIEWS_NO3f)
- Click on View Google Earth and View Current KMZ
- In Google Earth, click on the Document description in the Places sidebar and click Change Settings
Result:
The browser does not take the user to the URL to change the settings of the KMZ. If the URL in the description is copy and pasted into the browser, the following error appears:
The url is fine, but GE refuses to use it. I have had some bad experiences with too long url's before, we may have to result to tinyurl or some other mechanism. --Hoijarvi 11:23, 12 June 2008 (CDT)
Same result, even after trimming useless tags off the url--Hoijarvi 10:58, 19 June 2008 (CDT).
[edit] Points Need Default View
Procedure:
- Make sure Google Earth is closed
- View AIRNOW in the DataFed Viewer and click on View Google Earth
- Select dynamic points
- Click on View Current KMZ
- Open KMZ with Google Earth
Result:
Generates uninformative polygons when Google Earth opens where the polygons are flat and cover a large portion of land.
Suggestion:
Make a default view for the dynamic points similar to the static points view
Implements Changes
I changed dyn and static points to have almost same defaults, to get same defs is not possible, since dynamic point rendering depends on zoom level.
Dynamic point render badly first time if GE is not open, that's a GE bug. --Hoijarvi 15:41, 13 June 2008 (CDT)
[edit] Scaling Issues
Procedure:
- View any MNEI dataset in the DataFed Viewer and click on View Google Earth
- Select static or dynamic points
- Select View Current KMZ
Result:
Polygons go out to space regardless of position except when viewing dynamic points using the procedure that causes flat polygons described above.
Suggestion:
Reevaluate the scaling of this dataset. There is also a possibility that other point datasets could have similar scaling issues.
Conclusion True. I changed MNEI_Point scaling to min=1 max=10000 mode=log10 and it renders fine --Hoijarvi 15:47, 13 June 2008 (CDT)
This change only affects the MNEI_Point dataset. MNEI_Area, Mobile, Nonroad, and Total still have this problem. --MarkK 11:05, 19 June 2008 (CDT)
[edit] Suggested Improvements
[edit] 2008-06-16: Additional KMZ Code Needed for Console Views
The following changes to the KML code for the KML outside of the individual layers should take place for the Console views:
Add a folder to house the documents that contain the ground overlays. <Folder> before the first document and </Folder> after the last document. The folder should have a descriptive name and be given a place for a description with no Snippet. <Snippet /><description>Description here</description>. datasets.kmz shows these additions.
Make the folder open. <open>1</open>
[edit] 2008-06-16: Changes to Data Layers for Console Views
The following changes to the KML code for the individual layers should take place for the Console views:
Remove the Snippets from the Documents and make them blank with <Snippet />. Those Snippets only apply for the polygon KMZs.
Add the visibility tag to the Documents that are invisible. Invisible layers should have <visibility>0</visibility>. Visible layers do not need the visibility tags, but in case they are needed they should have <visibility>1</visibility>.
An example of these changes can be found in datasets.kmz
A file that can be used for comparison is AIRNOW.
[edit] Unnecessary Parameters
Several parameters in the user interface seem as though they are not applicable or unnecessary to the KMZs we are creating.
- top, bottom, left, right
- These parameters seem unnecessary as they eliminate the accuracy of the images. If they are necessary they should be listed next to each other.
- Height and width
- Because the pictures must fit in a lat/lon box, these parameters seem unnecessary. If they are necessary they should be listed next to each other. Right now they are at the top and bottom of the list
- format
- An image format seems unnecessary, but this may be a toss up
- bgcolor
- If transparent colors were added as a function, this could be removed
- margin_bgcolor
- This parameter does not seem to apply to the KMZs
- time_max, time_min
- These parameters are at the top of the page already. With these towards the bottom of the page, users may get confused
- param_abbr
- This parameter is necessary, but it would be a great tool to maybe have these in a checkbox or multiple drop down menus so they can be loaded at once. This can be discussed if needed
- width in the point rendering settings
- Changing this parameter under the symbol section does not seem to do anything
Removed
top, bottom, left, right, format, bgcolor, margin_bgcolor, imagecachefile, kmlcachefile
The rest must stay for now --Hoijarvi 12:05, 19 June 2008 (CDT)
[edit] Notes for the parameters
Notes on each of the parameters and what they do would be helpful especially on the point rendering page. Otherwise, the users who are not familiar with the rendering sections such as pen and brush will be confused.
[edit] Change Settings without Generating New KMZ
To increase our abilities for the server to "talk" to Google Earth, it may be helpful for the "change settings" link in the description of the Document to be similar to the "change point rendering" link. As of right now, this would only apply to the dynamic points option as it is the only network link available. However, if we are offering a network link as dynamic and useful as this, saving settings for the KMZ without generating a new one would be a helpful addition to this service.
[edit] Adding Descriptions to Polygons
Adding a description to the polygons in point datasets would enable users to view explicit information about a point of interest. The new description should list the numerical value of the measured parameter and a time series of the dataset within the specified time window.
http://webapps.datafed.net/cgi.wsfl?view_id=time&format=image/png&dataset_abbr=AIRNOW¶m_abbr=pm10&loc_filter=loc_code&loc_code=480290053&time_filter=datetime_range&time_min=2002-07-01T00:00:00&time_max=2003-07-01T00:00:00&scale_min=0&scale_max=30&width=300&height=100 is a link to a time series generated by the DataFed service. The only part missing from the generated picture is an indicator of what part of the time series the user is looking at. Furthermore, these descriptions will have the ability to speak back to the server and ask for additional parameters to view in the time series graph. This type of communication has been done with the dynamic points rendering settings.
Partly Done --Hoijarvi 15:49, 13 June 2008 (CDT) --MarkK 10:22, 19 June 2008 (CDT)
[edit] Integrating Content
This is just a thought about how we could make content more available to users. If we were to include in our KMZs information about datasets that provide similar information to the dataset the user is currently viewing, the web service could become more valuable than it already is.
For example, air quality can be measured in many ways. If a user were looking at a dataset such as AIRNOW, our service would inform them of other datasets that provided air quality information such as OMI. With this kind of service, data would be easily accessible and our tool would be easier to use.
[edit] Methods for Fixing Bugs and Problems
Directory for Testing Google Earth Capabilities
[edit] Investigating Description Balloon Capabilities
The description balloon in Google Earth is one of the most important features in Google Earth to mash data with metadata. Limitations that exist in this feature limit the depth of information that can be delivered to the user, and as more content enters these balloons, new problems arise that require investigation. One such problem is the failure of previous versions of Google Earth to show pictures generated by an uncached web service. Versions 4.1 and 4.2 of Google Earth have demonstrated a failure to generate the image from this time series picture in the description balloon, while the new version, 4.3, does generate the image. An investigation on the cause of this problem was done by inserting the same html string in the description balloon using different links to pictures from the internet. The results showed that Google Earth has no difficulty loading direct images from the internet, which gives the implication that the web service image generator for the time series fails in the previous versions. A further investigation will continue as more web services are called into the description balloon. This AIRNOW static point KMZ is the current test subject for the investigation.
A solution to the previous versions of Google Earth not rendering web service images was found. Adding a fake extension to the end of the URL that calls the web service will help Google Earth render the image inside the balloon. In this case, adding &dummy=/file.png as an extension works.
[edit] Using the Spreadsheet Mapper Tool
A new KMZ file has been made using the spreadsheet mapper tool that Google offers as part of their Google Outreach program. This spreadsheet creates placemarks in Google Earth with descriptions made from templates that are customizable by the workgroup using the spreadsheet. One limitation of this tool is the inability to create logo URLs as unique variables, so anyone who wants to use a different logo for their placemark other than the logo specified in the template must use a different template altogether. However, this tool provides an easily maintained file in which anyone from anywhere can edit.
Note that the placemarks generated by the spreadsheet are viewable in Google Maps, but Google Maps may have a slow connection to the spreadsheet and will time out. In case of this happening, reload the link.
[edit] Issues with Spreadsheet and Implications
[edit] 2008-06-05: Missing Source URLs and Desynched Metadata
The spreadsheet requires the source URL from the metadata in order to provide the location of the server to users. Without this entry, the lat/lon coordinates cannot be identified. Approximately 30 of the datasets lack a working source URL:
- G5FCST
- HTAP_CAMCHEM-3311m13
- HTAP_ECHAM5-HAMMOZ-v21
- HTAP_FRSGUCI-v01
- HTAP_GEMAQ-v1p0
- HTAP_GEOSChem-v07
- HTAP_GEOSChem-v45
- HTAP_GISS-Puccini-modelA
- HTAP_GISS-Puccini-modelE
- HTAP_INCA-vSSz
- HTAP_LLNL-IMPACT-T5a
- HTAP_MOZARTTGFDL-v2
- HTAP_OsloCTM2
- HTAP_UM-CAM-v01
- MODIS_Fire
- MODIS4_AOT
- SCIAMACHYm
- VIEWS
- CMAQ_CENRAP
- CMAQ_EPA
- CMAQ_VISTAS
- CMAQ_WRAP
- HMS_Fire
- NOAA_HMS_WFS
- EMEPAir
- EMEPPrec
- THREDDS_GFS
- GIOVANNI_OL
- THREDDS_TEST
Also, some entries in the DataFed catalog are different from the wiki catalog. The implied problem with this is the method of synching up the wiki and DataFed. Some entries are correct in DataFed while other entries are correct in the wiki. One possible solution is to go through each dataset by hand and verify the metadata. Once this is done, the verified metadata can overwrite all other metadata and synch everything up.
[edit] KML and Google Maps
KML files are viewable in Google Maps. If the user types the URL to the KML or KMZ, it will display in Google Maps.
2007-06-01:_DataFed-GoogleEarth_Mashup - Take 1
[edit] Using XSL and Python to Mash
There is an XSL Directory containing all of the xsl files and python scripts that have been written to import the wiki pages. The most notable xsl files are the SQL2WikiNew, which creates a new wiki page from the SQL table and SQL2WikiEdit, which replaces the structured metadata with the SQL table metadata without changing the added text. The most notable python script is SQL2Wiki.py, which uses the dataset_abbr to generate new wiki pages from a specified xsl file.
Also, another xsl file was created to pull from the SQL table for the metadata in the Google Earth description balloons. Although this may not be necessary, it provides a resource for generating the description balloons using another metadata source. The xsl file is in the xslt folder in DataFed, the same location as the other xsl files that generate the description balloons.
Categories: DevEvents | Atomic | Yymmdd | GoogleEarth
