Please explain the flow of project

README.md

@@ -5,12 +5,14 @@ A demo framework for Shiny + D3 including ggplot-like aesthetic mapping and geom
 
 Written by Alex B Brown at Intel Corp, 2012-2013
 
-Core idea: Pick a dataset, then describe how that data is mapped into a graph, using the handy-dangy ggplot2 like format.
+Core idea: Pick a dataset, then describe how that data is mapped into a graph, using the handy-dandy ggplot2 like format.
 
 Includes interactive features such as click and drag.
 
 Add Shiny inputs to the app to control graph filtering and more.
 
+Now supports layers
+
 Licence
 -------
 
@@ -19,6 +21,10 @@ See the files LICENCE and NOTICE for licence terms.
 Usage as a demo app
 -------------------
 
+Recommended packages to install:
+
+shiny, plyr, httr, hmisc, reshape2, stringr, lubridate, ggplot2 (plus dependencies), 
+
 To run g3plot in demo mode, 
 
 1) start R in this directory.
@@ -36,16 +42,63 @@ Look in plot.R and add some functions for other data sets.  If that gets old, go
 
 If *that* gets old, start to create new graph types in javascript, or fix the html table logic.
 
-Usage as your own app
----------------------
+Writing a new g3plot application
+--------------------------------
+
+You can start with this super simple application and extend it:
+
+mkdir myproject
+cd myproject
+git clone <g3plot repo> g3plot
 
-Checkout g3plot as a subdirectory of your Shiny Application and add the line
+Create the file `server.R` with the contents:
 
+```
+require(shiny)
+source("g3plot/shiny_extend_g3.R")
 addResourcePath("js",tools:::file_path_as_absolute("./g3plot/js"))
+shinyServer(function(input,output,session){
+  output$testplot = renderG3Plot(function() {
+    dataSet = data.frame(x=1:10,y=1:10) 
+    list(type="plot",
+     table=forceTableVector(dataSet),
+     structure=list(sX="x",sY="y"), 
+     aesthetic=list(X="sX",Y="sY"),
+     geom="point")})
+})
+```
+
+Create the file `ui.R` with the contents:
+
+```
+require(shiny)
+source("g3plot/shiny_extend_g3.R")
+shinyUI(
+  pageWithSidebar(div(),div(),
+  div(includeHTML("g3plot/g3widget.html"),
+  svgOutput("testplot")))
+)
+```
+
+Updating an existing shiny application to use g3plot
+----------------------------------------------------
+
+Checkout g3plot as a subdirectory of your Shiny Application
+
+```
+cd myproject
+git clone <g3plot repo> g3plot
+```
+
+and add the line
+
+```
+addResourcePath("js",tools:::file_path_as_absolute("./g3plot/js"))
+```
 
-To your server.R.
+To your `server.R`.
 
-Then follow some of the examples in server.R and ui.R and friends to add 
+Then follow some of the examples in the demo `server.R` and `ui.R` and friends to add 
 javascript plots to your Shiny Application.
 
 Note that you can still test the demo app by using
@@ -57,12 +110,24 @@ Geoms
 
 Currently supports:
 
-point
-bar
-point_bar
-point_range_bar
-line
-voronoi (doesn't display but makes UI better)
+name            | required aesthetics | optional aesthetics | axis | description
+----------------|---------------------|---------------------|--------|---
+point           | X,Y                 | Color               | cont** | a small round point
+line            | X,Y,Group           | Group,Color         | cont   | a line for each Group (or Color)
+area            | X,Y,Group           | Group,Color         | cont   | an area underneath the line for each group (stacking?)
+bar             | X,Y                 | Color               | ordinal| a bar starting at 0, can be stacked
+point_bar       | X,Y                 | Color               | ordinal| instead of a whole bar, just the tip
+point_range_bar | X,DX,Y              | Color               | cont   | like point bar but each can have a unique width (DX)
+voronoi         | X,Y,Label           |                     | cont   | Use with points to extend click/hover halo around point
+
+** *cont* is `linear` or `log`
+
+Other aesthetics supported by most geoms include:
+
+* Label - What appears when you hover.  Default is cloned from XCluster or X aesthetic.
+* XCluster - Compound X axis - see examples for more details.
+* YFacet - Facet plot into rows with separate synchronised Y axes.
+* Key - improve animation by giving each node a unique key.
 
 Layout Structure
 ----------------
@@ -74,13 +139,15 @@ Common Name | Entity | Arity | Class           | File        | Message Part
 Document    | HTML   | 1     |                 | UI.R        | -                 | the web page
 report      | DIV*   | n     | g3plotMultiPlex | g3widget.js | Array of arrays   | the shiny output
 section     | DIV    | n     | pane            | g3report.js | Array of list(name=?)    | a single formatted d3 object - one of "plot" "list" or other text
-figure+data | DIV    | 1     | plot            | g3plot.js   | ditto             | A combination of a drawing region with linked html table
+figure+grid | DIV    | 1     | plot            | g3plot.js   | ditto             | A combination of a drawing region with linked html table
 figure      | SVG    | 1     | d3svg           | g3plot.js   | ditto             | A single drawing region with any contents
 subfigure     | G      | n     | subplot         | g3subfigure.js   | List(name=?)      | container for a plot with distinct axes, data, legends
 plot        | G      | 1     | plot            | g3plot.js   | ditto             | the bit inside the axes
 y facet     | G      | J     | facet           | g3plot.js   | aesthetic(YFacet=?)          | the Jth horizontal slice with a personal clone of the Y scale
 x facet     | G      | K     | y facet         | g3plot.js   | aesthetic(XCluster=?)| the Kth vertical slice of the Jth horizontal
+layer       | G      | L     | layer           | g3plot.js   | Array of list(name=?)    | a single formatted d3 object - one of "layer"
 geom        | ?      | many  | dot/bar/...     | g3geom.js   | aesthetic(geom=?) | an actual drawing component
+grid        | TABLE  | 1     | XX?table        | ?           | like layer but grid | An html table
 
 TODO
 ----
@@ -98,16 +165,16 @@ Fun improvements:
 
  * Standardised way to add dynamic tooltips 
  * Standardised way to hover highlight nodes
- * Improved click dropzones (voronoi?)
+ * Improved click dropzones (voronoi?) (now implemented)
  * click drag on axes to scale (near ends) or pan (in middle)
+ * Mouseover cursor with tooltip coordinates of intersecting line / point and selection like brushes.
 
-Some ideas to develop the product
+Some ideas to develop the tool
 
  * Sparklines - providing a very simple 3-layer structure without faceting
- * Layers - multiple geoms on top of each other (another layer!)
+ * Layers - multiple geoms on top of each other (another layer!) (now implemented)
  * g3autoplot - looks at data and does something sensible, then tells you how to repeat / customise it
 
-
 Constraints
 -----------
 

dynamicUI.R

@@ -16,7 +16,7 @@ output$dataSetControls <- renderUI({
 
   if (is.null(hashParts())) return(NULL)
   hashDataSelected <- hashParts()$dataSet
-  selectedDataSets <- NULL
+  selectedDataSets <- "airquality"
   if (!is.null(hashDataSelected)) {
     selectedDataSets <- as.numeric(strsplit(hashDataSelected,"\\|")[[1]])
     selectedDataSets <- intersect(selectedDataSets,groupedDataSets)

g3widget.html

@@ -1,5 +1,7 @@
 <!-- g3widget.html is included in the HTML uploaded by UI.R.  
      It handles included javascript and extended css -->
+<!--<script src="external/d3.v3.js"></script>-->
+<!--<script src="external/underscore-min.js"></script>-->
 <script src="http://d3js.org/d3.v3.js"></script>
 <script src="http://underscorejs.org/underscore-min.js"></script>
 <!-- the original version -->
@@ -13,14 +15,15 @@
 <script src="js/g3report.js" type="text/javascript"></script>
 <script src="js/g3htmlPanes.js" type="text/javascript"></script>
 <script src="js/g3geoms.js" type="text/javascript"></script>
+<script src="js/g3widget.js" type="text/javascript"></script>
 <script src="js/g3brush.js" type="text/javascript"></script>
 <script src="js/g3events.js" type="text/javascript"></script>
 <script src="js/g3math.js" type="text/javascript"></script>
 <script src="js/g3stats.js" type="text/javascript"></script>
 <script src="js/g3legends.js" type="text/javascript"></script>
 <script src="js/g3xcluster.js" type="text/javascript"></script>
 <script src="js/g3functional.js" type="text/javascript"></script>
-<script src="js/g3plotDataUtils.js" type="text/javascript"></script>
+<script src="js/g3figureDataUtils.js" type="text/javascript"></script>
 <script src="js/g3patterns.js" type="text/javascript"></script>
 <script src="js/d3extend.js" type="text/javascript"></script>
 <script src="js/g3URL.js" type="text/javascript"></script>
@@ -34,10 +37,11 @@
 .axis path,
 .axis line {
     fill: none;
-    stroke: black;
+    stroke: lightgrey;
     stroke-width: 1px;
-    shape-rendering: crispEdges; // warning: zoom out in chrome disappears 1px crisp lines.
+    // shape-rendering: crispEdges; // warning: zoom out in chrome disappears 1px crisp lines.
 }
+
 rect.haxis {
 //  stroke: white;
 //  stroke-width: 1;

js/aestheticUtils.js

@@ -15,10 +15,10 @@
       return recordWalker.other(
         // for each string S in structure, pull out the indexed element
         // of the array named S in data.
-        recordWalker.indexedMemberOf(message.table,index))(structure);
+        recordWalker.indexedMemberOf(message.data,index))(structure);
     }
     // the number of records:
-    var recordCount = _.values(message.table)[0].length
+    var recordCount = _.values(message.data)[0].length
     // use the walker to map the structure onto the data
     var records = d3.range(0,recordCount).map(buildRecord)
 
@@ -63,5 +63,25 @@
       return _.isUndefined(filterWalker.walk(aesFilterSpec,record))
     }    
   }
+
+  // utilities - shouldn't be here really since it's about the concrete plan containing aesthetics.
+
+  // returns length of (first?) valid aesthetic or undefined if none
+  exports.hasAesthetic = function hasAesthetic(plan,aesthetic) {
+    // look in global aesthetic structure first
+    var aesthetics = []
+    if (plan.metaData && plan.aestheticStructure && plan.metaData.aestheticStructure[aesthetic]) {
+      aesthetics.push(plan.metaData.aestheticStructure[aesthetic])
+    }
+    // look in local aesthetic next
+    _.map(plan.layers,function(l){
+      if (l.metaData.aestheticStructure[aesthetic]) {
+        aesthetics.push(l.metaData.aestheticStructure[aesthetic])         
+      }
+    })
+    // return the length of the olength of the first aesthetic (or 1 for atomics or objects)
+    return _.isObject(aesthetics[0]) ?
+      _.keys(aesthetics[0]).length : !_.isUndefined(aesthetics[0])
+  }
 
 })(typeof exports === 'undefined'? this['aestheticUtils']={}: exports);

js/g3figure.js

@@ -163,18 +163,21 @@
         g3figure.filter.addWidget(d.subfigure.filterHandle());
       })
 
-    if(plans.length == 0 || plans[plans.length-1].data.message.grid == null){ 
+    // find first layer with grid
+    var grid_layers = _.chain(plans).pluck("layers").flatten(true).filter(function(layer){return !_.isUndefined(layer.data.message.grid)}).value()
+
+
+    if(grid_layers.length==0){ 
       // need to remove the table
       d3.select(el).select(".d3Table").select("table").remove()
       // and turn off the filters.  um, why?
       //exports.filter.clear()
     } else {
-      // It should mean - draw a table for each subFigure that wants one.  But I only
-      // know how to draw one table at the moment so only the last one that wants one
+      // It should mean - draw a table for each subfigure and layer that wants one.  But I only
+      // know how to draw one table at the moment so only the last layer that wants one
       // right now it does 'if the last plan wants a table' do it.
-      if(plans[plans.length-1].data.message.grid != null) {
-        exports.filter.addWidget(g3figure.table(d3.select(el).select(".d3Table"),plans[plans.length-1])); 
-      }
+      var grid_layer=_.last(grid_layers)
+      exports.filter.addWidget(g3figure.table(d3.select(el).select(".d3Table"),grid_layer)); 
     }
   }
 
@@ -194,20 +197,25 @@
     })
     subFigure.exit().remove();
 
-
-    // calculate the inner sizes for the subFigure
+    // adjust figure margins where aesthetics require guides
     subFigure
       .each(function(plan,i){
         var xAxisHeight = 0
         var xClusterAxisHeight = 0
-        if (plan.metaData.aestheticStructure.X) {
+        var legendWidth = 20
+        if (aestheticUtils.hasAesthetic(plan,"X")) {
           xAxisHeight += 25
         }
-        if (plan.metaData.aestheticStructure.XCluster) {
+        if (aestheticUtils.hasAesthetic(plan,"XCluster")) {
           // 1+length because there's always an outer cluster to select all
-          xClusterAxisHeight = (1+_.keys(plan.metaData.aestheticStructure.XCluster).length) * 20
+          xClusterAxisHeight = (1+aestheticUtils.hasAesthetic(plan,"XCluster")) * 20
+        }
+        if (aestheticUtils.hasAesthetic(plan,"Color") || aestheticUtils.hasAesthetic(plan,"Fill")) {
+          // 1+length because there's always an outer cluster to select all
+          legendWidth += 100
         }
 
+        // TODO: enable user to disable showing guides
 
         var s=d3.select(this);
         // calculate the inner sizes.  Should go in d3subfigure since
@@ -218,7 +226,7 @@
         var dimensions = plan.dimensions
         dimensions.margin = {
           top: (widgets.size=="history")?0:20, 
-          right: 160,
+          right: legendWidth,
           bottom: (widgets.size=="mini")?10:
                   (widgets.size=="history")?25:(xAxisHeight + xClusterAxisHeight),
           xcluster: xClusterAxisHeight,
@@ -256,8 +264,11 @@
   }  
 
   // filter is a per-plot (possibly global) filter collection that
-  // sends filter messages amongst graphs.  Should rebuild to use
-  // d3.dispatch (but what about new graphs - how do they get filters?)
+  // sends filter messages amongst graphs.  
+
+  // TODO: Should rebuild to use either:
+  // * d3.dispatch (but what about new graphs - how do they get filters?)
+  // * just using css selectors to find filterable things, then invoke update on them?
   exports.filter = function() {
     exports.filter.widgets = []
     exports.filter.unfilter = function() {

js/g3plotDataUtils.js → js/g3figureDataUtils.js

@@ -22,6 +22,9 @@
     })
       .entries(dataToFacet)
 
+    if (_.isUndefined(coordAesthetic)) return dnest;
+
+    // not convinced the code below is used.  not needed for cell facets, anyway
     var coordAccessor = function(d){return d[coordAesthetic]}
 
     // now decorate the nested entries with yscale

js/g3functional.js

@@ -9,6 +9,8 @@
     return function repeatedly() { return v[i=(1+i)%l] }
   }
 
+  exports.negate = function negate(f) { return function(x){ return !f(x) } }
+
   // using an arbitrary accessor as the next pointer, return the list thus formed from the input node
   exports.followChain=function(accessor){
     var followChainAccessor = function(node) {    
@@ -19,6 +21,13 @@
     }
     return followChainAccessor
   }
+
+  // other utilities
+  exports.pluralise = function(x){return typeof(x)==="undefined"?[]:_.isArray(x)?x:[x]}
 
+  // convert an array of objects to an object of arrays [{a:1},{a:2}] => {a:[1,2]}
+  // unused so far - just handy
+  var twoargs=function(f){return function(x,y){return f(x,y)}}
+  exports.objectArraysToArrayObjects=function(d){return (function(k){return _.object(k,_.map(k,function(x){return _.pluck(d,x)}))})(_.chain(d).map(_.keys).reduce(twoargs(_.unique)).value())}
 
 })(typeof exports === 'undefined'? this['g3functional']={}: exports);