2007-05-20 Web2.0 and Mashup Unidata e-letter/Background

From Datafedwiki

Jump to: navigation, search

The content of cyberspace is increasingly generated and served by individuals, rather than organized and delivered through centralized servers. This architectural, technological and cultural transformation of the internet is good news for the Unidata community. It offers new possibilities for sharing and harvesting community-provided content and to collaboratively create new things. However, the use of the Web 2.0 technologies to our Earth Science community is not obvious. In this note we share a few of our early experiences with the new, participatory Internet in the context of May 2007 Georgia Fires as the use case.

Find and Filter. The fires in southern Georgia began in late April 2007 and continued throughout May. The Google Blog Search yielded several thousand entries, many of them repetitive. Visual scanning of the blog entries yielded a number of interesting posts, which were bookmarked and tagged with 070508+Florida+Smoke in the social bookmarking tool Del.ici.ous. Google Image and Video search results were also tagged in Del.icio.us. Additional smoke photos in the photo-sharing service, Flickr were also found. This procedure uses technologies for filtering and harvesting user-contributed content and yielded a rich array of descriptive material on the Georgia Fires, but of marginal value to quantitative analysis.

The tools and methods of quantitative analysis are also changing. Large monolythic, 'do-it-all' software tools are giving way to a modular approach of web service modules which are integrating through service chaining. Application software can now be created using the "publish, find, bind" triad of the Service Oriented Architecture, (SOA). Service modules are published and found through repositories and catalogs. Adherance to standard service interfaces then allows binding (loose coupling) and dynamic integration of distributed, reusable services. Service orientation not only allows the creation of flexible user-defined applications, but also lowers the entry resistance for service providers. One of the most impressive mashups is Weather Bonk, within own field. It is and integration of many services including Weather Service Forecasts, Yahoo Geocoding/Maps/Traffic and Google Maps. We have also been using the workflow approach to build air quality analysis applications as part of DataFed.

A wiki is a general workspace to organize web content, collaboratively create and discuss documents and it is a flexible client for mashups through incorporation of RSS feeds, YouTube Video, Google Maps etc. For example, the Georgia Smoke Event wiki page contains structured metadata that describes the event itself (top section), has links to all of the harvesting queries discussed above, access to data resources viewable in Google Earth as well as a summary of the smoke event written collaboratively by the community. The discussion page is a forum for a free exchange on the smoke event.

We hope that this short teaser illustration of Web 2.0 will encourage the Unidata community to explore, apply, and even to improve these emerging technologies for betterment of Earth Science research and education.

Contents

[edit] Web 2.0, Mashups and the Unidata Community (draft)

The content of cyberspace is increasingly generated and served by individuals, rather than organized and delivered through centralized servers. This architectural, technological and cultural transformation of the internet is good news for the Unidata community. It offers new possibilities for sharing and harvesting community-provided content and to collaboratively create new things. However, the use of the Web 2.0 technologies to our Earth Science community is not obvious. In this note we share a few of our early experiences with the new, participatory Internet in the use case of the May 2007 Georgia Fires.

The fires in southern Georgia began in late April 2007 and continued throughout May. The Google Blog Search yielded several thousand entries, many of them repetitive. Visual scanning of the blog entries yielded a number of interesting posts, which were bookmarked and tagged with 070508+Florida+Smoke in the social bookmarking tool Del.ici.ous. Google Image and Video search results were also tagged in Del.icio.us. Additional smoke photos in the photo-sharing service, Flickr were also found. Filtering and harvesting user-contributed content yielded a rich, but qualitative description of the Smoke Event.

The tools and methods of quantitative analysis are also changing. Large monolythic, 'do-it-all' software tools are giving way to a modular approach through web services, integrated through service chaining and workflow. Application software can now be created using the "publish, find, bind" triad of the Service Oriented Architecture, (SOA). Service modules are published and found through repositories and catalogs. Adherance to standard service interfaces then allows binding (loose coupling) and dynamic integration of distributed, reusable services. Service orientation not only allows the creation of flexible user-defined applications, but also lowers the entry resistance for service providers.

A manifestation of SOA is the emergence of software mashups, where the components themselves are independent software applications with purpose and characteristics of their own. One of the most impressive mashups is Weather Bonk, an integration of many services including Weather Service Forecasts, Yahoo Geocoding/Maps/Traffic, Webcams and Google Maps. We have also been using the workflow approach to build air quality analysis applications as part of DataFed, which includes mashup with Google Earth.

A wiki is a general workspace to (1) organize web content, (2) collaboratively create and discuss documents and it is a (3) flexible client for mashups through incorporation of RSS feeds, Google Maps etc. For example, the Georgia Smoke Event wiki page contains structured metadata that describes the event itself (top section), has links to the harvesting queries discussed above, access to data resources as well as a summary of the smoke event written collaboratively by the community. The discussion page is a forum for a free exchange on the smoke event.

We hope that this short teaser illustration of Web 2.0 will encourage the Unidata community to explore, apply, and even to improve these emerging technologies for betterment of Earth Science research and education.


A typical service chain for Earth Science application consists of a data access service, a processing service and a portrayal service. A unique manifestation of SOA is the emergence of software mashups, where the components themselves are independent software applications with purpose and characteristics of their own. The classical mashup examples involves Google Maps and Google Earth for rendering and user interface, while the data access and processing is performed by other autonomous software components.



s and their integration using service chaining and work flow integration tools. In other words, application software can now be created by "published" service components that can be found through service registries. Adherance to standard service interfaces then allows "loose coupling" and dynamic integration of distributed, reusable services. A typical service chain for Earth Science application consists of a data access service, a processing service and a portrayal service. Service orientation not only allows the creation of flexible user-defined applications, but also lowers the entry resistance for service providers. For instance, having a unique processing service is sufficient to become a useful community member without offering a full, monolithic, end-to-end system. A unique manifestation of Service Oriented Architecture is the emergence of software mashups, where the components themselves are independent software applications with purpose and characteristics of their own. The classical mashup example involves Google Maps for rendering and user interface, while the data access and processing is performed by other autonomous software components.

Another form of mashing is through the wiki workspace. ... The new wiki technology offer ... The wiki can serve as an organizer of the vast array of information resources on the Georgia Smoke Event as seen on the Georgia Smoke Event wiki page. It contains structured metadata that describes the event itself (top section). The page has links to all of the harvesting queries discussed above. The wiki page also contains a summary of the smoke event as written collaboratively by the community. The discussion page is a forum for a free exchange on the smoke event. An additional technological feature of the wiki is that it can serve as a client which incorporates content provided by multiple distributed applications, RSS feeds, YouTube Video, Google Map etc. In other words, the open architecture of the wiki makes it a general tool for mashing with other mature applications.

Mashup Google Earth/DataFed Wiki - RSS/YouTube/GoogleMaps/DataFed...


e quantification of the smoke spatial-temporal pattern and it's impact on environment is accomplished by analyzing the available satellite and surface measurements and the output of simulation models. The list of smoke-relavent data resources is also gathered and tagged with [070508+Florida+Smoke+data] using del.icio.us. In order to "see" the smoke spatial pattern Google Earth is the most useful rendering tool, this merge of two applications is known as a mashup.

The content of cyberspace is increasingly generated and served by individuals, rather than organized and delivered through centralized servers. This transformation along with an array of new technologiesis a key feature of the Web 2.0 phenomena. The architectural, technological and cultural transformation of the internet is good news for the Unidata community. It offers new possibilities for sharing and harvesting community-provided content and collaboratively creating new things. However, the use of the Web 2.0 technologies to our Earth Science community are not obvious. In this note we share a few of our early experiences with the new, participatory Internet. We will be providing the illustrations through the May 2007 Georgia Fires as the use case.


[edit] Roadblocks to Web2.0

Tame the technologies with standard controlled sets of tags we lose communal contributions. Tagging and tag-based search-organization of content is a key aspect of Web 2.0. We find that for our science applications structured tagging with agreed upon keywords is necessary, as opposed to the loose folksonomy prevalent in the current social software. The use of tags to organize content is particularly difficult in the case of tagged wiki pages where the content is highly diverse, but needs to be connected in some "natural" way. We have a long way to go to understand the various ways of structuring the wiki. For instance you can't just let me tag from memory

[edit] Report Outline

[edit] Intro

Evolution that could make the Unidata community more effective. Architecture of participation

[edit] Moving personal workspace onto the web

  • From Pull to Push --- content flows in - no structure
  • Magic submit/edit button - information flow in
  • outside in - edges of web

[edit] Reasons people moved to web workspace

  • independent of the computer, move content onto the web - communal space
  • More powerful tools than the desktop applications
  • Tools are constantly updated (perpetual beta) and free

[edit] Identifying the features of the Web2.0 Architecture

  • tagging - adding meaning after the fact, allows multiple structures
  • Applications of tagging - del.icio.us, flickr, blogs, wiki
    • flickr - easier picture organization and sharing with family
    • del.icio.us - get to your bookmarks anywhere, tagging/notes for own memory
    • blogs - keeping your diary, sharing with interested
  • applications useful for their own sake

[edit] Consequences that content is on the web

  • User-generated content can be found and reused by:
    • connected to,
    • compared to,
    • organized
    • harvested
  • Applications
    • flickr - find smoke pictures with tags: smoke, florida
    • Del.icio.us - find related links through other people's tags; Social ontology
      • who else links to the same link? Person also linked to these other sites; Learning to tag from others (i.e. action tags - things you may want to do with this page toread, for:rhusar...)
    • Technorati - linking bloggers together
    • Hard links between blogs
    • Wikipedia - harvest human intellect
    • Wiki - communal workspace for organizing, storing, communication, collaboration, coordination

KML is our metadata packaging/viewer because its structured langauage KMZ file - explicit link to wiki page that generates meta data and viewer that generates view.


  • unitentional connections
  • All of these above Web2.0 applications are clearly open on the input side to harvest, but also open for connections to other Web2.0 applications in form of mash-ups.

[edit] Mash-ups

Open make new things sponteneously

Machup engineer nea things from large grained pieces Facebook - personal portal

Social ontology ---

wiki - screencast


  • Google Earth/DataFed - small piece that shows workflow with picture of google earth/AIRNOW on otherside - point and image rendering.

AIRNOW in DataFed

  • Googlemap/wiki
  • Del.icio.us, flickr, RSS feed/Wiki

</feed>http://api.flickr.com/services/feeds/photos_public.gne?tags=georgia&lang=en-us&format=rss_200</feed> - Shared resources (i.e. unique tag combination)

  • Youtube Screencast/Wiki - put/get
  • DataFed/Wiki - WMS call into wiki produces image
  • organizing user-gen. content - semantic wiki, dynamic page list, transclusion,
  • Finding through tagging search by tag, geo tag, date
  • wiki/forum through discussion pages - easily give feedback
  • Forms and subpages - allow for structured content
  • Upload files - content management system with versioning 






 changing from the traditional client-server configuration  its form and function. The traditional form consisting of central servers providing content to many clients is now augmented by numerous servers run by “individuals” … These new distributed individual providers are now contributing an increasing fraction of the content in cyberspace.

These transformations are visible in mainstream applications like ebay, Amazon and Google.


A defining characteristic of Unidata is the strong sense of community among its members. The combination of the tools and methods provided by the Unidata program office, along with the active participation of the Unidata community has created a clear example of cyberinfrastructure serving the meteorological community. Recent development in the configuration in the use of the internet is dramatically changing the nature of the cyberinfrastructure. In particular, the newly generated content of the internet is now contributed largely from the "periferee" of the internet by individuals which is then connected in various ways by social software. The result is emergence of a network content that connects shared content in blogs, images, videos, web bookmarks, wikis, etc. This type of content creation and usuage is .... different from the old, central, server-based which is then distributed radially to largely passive users. This transition towards "architecture of participation" has been accompanied by the emergence of new technologies that support such participatory, cyberinfrastructure. Given the participatory nature and the technical savviness of the Unidata community, these technologies could further enable the Unidata community... The purpose of this brief note is to highlight several of the key technologies that characterize key aspects of the emerging Web 2.0.

[edit] Connectivity of Machines

Given that both Earth Science data and processing capabilities are distributed, Service Oriented Architecture (SOA) offers an approach to expose and connect the content... Everything is a service, workflow connects the services. ... Reduces entry point resistance, i.e. cheap to enter because you can contribute small, but reusable components. Technologies data access services, THREDDS, W*S, workflow, mash-ups.

[edit] Sharing and Connectivity of Humans

The key technologies of the Web 2.0 era are aimed at helping users adding structured content to the communal space.

Perform standard organziation of digital content Perform these operations on the Web. Being on the Web allows linking, filtering, aggregation and fusion of the communal content.

[edit] Del.icio.us

Web applications have begun to be more widely used because they are better than their desktop counterparts. Favorites are one simple example of this change. Initially, favorites were built in to the browser, accessible from one computer and only organized through one hierarchical set of folders. Through the online bookmarking tool, del.icio.us, the user can save the same favorites on to the web allowing access to the users favorites from any computer (RussRew links). Instead of a folder system Del.icio.us has implemented a tag-based strucutring so that users can sort and filter their bookmarks in multiple ways (RussRew links with netcdf tag). Because Del.icio.us is web-based each user benefits from an entire community of tagging allowing questions like who else has tagged this link, what other tags are associated, what are the popular links for this tag... (generic rss options)

[edit] Flicker

Photosharing on the web has increased dramatically with the web-based tool Flickr. Flickr like del.icio.us allows pictures or sets of pictures to be uploaded to the web and tagged for easy navigation. Again because flicker is web-based, by searching for a set of tags like Florida Smoke the return issmoke pictures from flickr users. Similarly, the February west Texas dust event returns pictures of the event.

[edit] Blogs

Blogging is more and more becoming a valuable resource pool for real-time, first-hand accounts of events going on all over the world. Technorati realizing the resource potential of blogs has created a blog search engine to harvest blog posts based on specific queries. Because bloggers write in near-real time finding blogs about specific air pollution events has become feasible. However with any search, some blogs that aren't applicable slip through - to remedy this we have tagged the best blogs for the Florida smoke event in Del.icio.us for reuse later (blogs collected from technorati).

[edit] Wikis

Wikis are a web-based workspace that allow resources to be pooled and shared through common standard tags and RSS feeds -(pictures from flickr, links from Del.icio.us along with other types of web content), collaboratively create new content and to serve as a forum for workgroups to interact through discussion. The wiki workspace is flexible enough to allow each group to define what content should be included and structured - from Wikipedia's narrow focus on encyclopedia articles to other wiki workspaces that accomodate diverse multimedia content.

We are clearly in the beginning of the emerging phenomena of Web 2.0 whose full features are yet to be seen. However, it is clear that these architectural changes and the tools can have profound effect on enhancing the Unidata community ...



Wikis are much harder to define. While the original Wikipedia promoted the creation of a user-generated online encyclopedia, wikis are now used as a general workspace for communities to collect and organize shared content, to collaboratively create new content and to serve as a forum for workgroups to interact through discussion... Modern wikis are host environments which allows each community to adapt the wiki workspace to their needs. Some wikis are for relatively narrow activity like Wikipedia, creating encyclopedia articles, while other wiki workspaces accomodate diverse multimedia content.

Wikis can encorporate RSS feeds from del.icio.us bookmark queries, google maps, youtube videos ... The organically growing wiki pages... are also enhanced with tags which then allows both searching and structuring of the content based on those tags.


Wiki - del.icio.us mashup via RSS http://datafedwiki.wustl.edu/index.php/AIRNOW#Papers

http://webapps.datafed.net/datafed.aspx?dataset_abbr=AIRNOW - Datafed/GoogleEarth Mashup


http://del.icio.us/popular/mashup

The emerging array of open services and their mashups are are being cataloged an the Programmable Web site.


[edit] Del.icio.us extra stuff

The key extension to the personal bookmarks is that:

  1. the online version of the bookmark is maintained in a central database and
  2. each bookmark can be marked with user-defined tags.


The open, shareable bookmarks, enhanced by tags allows a whole range of connections to be made, for instance the system can answer questions such as Who else bookmarked this page? What are the bookmarks of RussRew? Which articles are the most bookmarked? These facilities allow "orthogonal navigation" through the bookmarks based on the page bookmarker user... the tags attached to each bookmark page facilitates the searching for webpages since the tags contain more meaning then arbitrary words in the text. Finally, the collection of tags given by the community provides natural, socially generated naming convention. The role of del.icio.us is simply find webpages on a particualr topic by tag-based searching and then to allow the exploration of the rest of the community that is interested in the same topic. These features of del.icio.us are characteristic for most of the new social software including myspace and Facebook, where person-person connections are made based on profile.

Retrieved from "http://datafedwiki.wustl.edu/index.php/2007-05-20_Web2.0_and_Mashup_Unidata_e-letter/Background"
Personal tools
Workspaces
Clicky Web Analytics