Supporting a javascript editor enables:

• more structured and easy-to-read code, especially for multi-view visualizations
• comments & annotations in the code example

@sehilyi, I am planning to update docs about the responsive design. Going through the responsive examples, I have some thoughts and questions and would like to hear your opinions.

• I am wondering whether we can specify the conditions using a more unified expression, e.g., something like alt.condtion in altair. Instead of specifying a spec at two different places,

      "layout": "circular",
      "responsiveSpec": [
        {
          "spec": {"layout": "linear"},
          "selectivity": [
            {"measure": "aspectRatio", "operation": "GT", "threshold": 1.5}
          ]
        }
      ],

maybe we can use something like (assuming now the type layout = string|condition)

  "layout": {
    "ifTrue": "linear", 
    "ifFalse": "circular",
    "condition":  {"measure": "aspectRatio", "operation": "GT", "threshold": 1.5}
  }

• the same goes for the responsiveSize. instead of specifying it at two places

"responsiveSize": {"width": true},
 "width": 400,

do you think it will be more convenient to write the below?

"width": {
  "ifTrue": 400,
  "ifFalse": "vw", // not sure what is the best way to express the screen size here
  "condition": {"measure": "width", "operation": "GT", "threshold": 1000}
}

or just "width": "vw" if the width is always the width of the screen.

This definition is also consistent with the type definition of the visibility object, by considering the default value as "ifTrue": "visible", "ifFalse": "invisible".

• In a circular layout, the height of a track has no meaning in terms of the number of pixels. Will it be confusing if a user wants a circular chart to respond to the change of (screen) height? What is the proper way to define that?

Builds off of #494, but replaces the webpack-dev-server/build with vite. I think ideally we could just use Vite to unify the build, but the build.js script gives us better control over the "package".

Vitest is a unit-testing framework that is intended to be a drop-in replacement for Jest. The main benefit is that is reuses Vite's config, transformers, resolvers, and plugins. This means that the testing environment more closely mimics the final build, and we no longer need to worry about making things work in Jest and our build.

Fix #506 Fix #482

Summary of Changes

In addition to #533, this PR changes grammar around visual channels.

This PR contains a relatively large amount of changes, but many parts are changes on the examples (src/example/*.ts) and tests (*.test.ts) reflecting the grammatical changes that have been introduced in this PR. This PR introduces the following major grammatical changes.

C1. Axis channels are combined

from:

x: { field: 'start', type: 'genomic', axis: 'top', linkingId: 'overview' },
xe: { field: 'end', type: 'genomic', axis: 'top' }

to:

x: { startField: 'start', endField: 'end', type: 'genomic', axis: 'top', linkingId: 'overview' }

This way, axis channels convey a more clear meaning:

• multiple axes do not exist for the x-axis (i.e., x and xe) or the y-axis (i.e., y and ye).
• Instead, multiple fields can be used for a single axis (i.e., startField and endField in an x channel).

More importantly, this prevents users from making uncertain specs, such as

• using different types for the single axis (e.g., nominal field for y and then quantitative field for ye)
• defining properties multiple times for a single axis that should be defined only once (e.g., axis: 'top' or linkingId: 'overview'):

x: { field: 'start', type: 'genomic', axis: 'top', linkingId: 'overview' },
xe: { field: 'end', type: 'genomic', axis: 'bottom', linkingId: 'ab' }

C2. The encoding property is added

Track now has a encoding property that wraps all channels:

tracks: [{
   mark: 'line',
   encoding: {
      x: { startField: 'start', endField: 'end', type: 'genomic', axis: 'top', linkingId: 'overview' },
      color: { value: 'black' }
   },
   width: 100,
   height: 30
}]

This is consistent with the Vega-Lite and is beneficial in gos (https://github.com/gosling-lang/gos/issues/34#issuecomment-904798525).

C3. TemplateTrack modified accordingly

As we made a grammatical change to channels (i.e., x and xe --> x: { startField: ...., endField: ..., ...}), the template mapper logic was changed as well:

// part of a TemplateTrackDef spec
x: { base: { startField: 'startPosition', endField: 'endPosition', ... }

C4. DataTrack is removed entirely

We did not support DataTrack anymore, but it was still included in our code. I removed all related codes. We could support a similar feature with Track Templates in Gosling in the future:

{ template: 'vector-track', data: { ... }, width: 1000, height: 30}

To Test Locally

Run the following command to update your local gosling.schema.json since this PR updates the grammar.

yarn schema

BREAKING CHANGE

This PR introduces breaking changes. To change your existing specification for previous versions (~v0.9.9), you need to change your spec following the instructions described below.

If you are using multiple fields for a single axis in your spec:

x: { field: 'start', type: 'genomic', axis: 'top', linkingId: 'overview' },
xe: { field: 'end', type: 'genomic', axis: 'top' }

You need to combine them to a single axis using up to four fields (i.e., startField, endField, startField2, endField2):

x: { startField: 'start', endField: 'end', type: 'genomic', axis: 'top', linkingId: 'overview' }

Single-field axes are not changed (i.e., field properties are used):

x: { field: 'position', type: 'genomic', axis: 'top', linkingId: 'overview' }

Add an encoding property to all tracks.

From:

tracks: [{
   data: ...,
   mark: ...,
   x: ...,
   y: ...,
   ...
}]

To:

tracks: [{
   data: ...,
   mark: ...,
   encoding: {
      x: ...,
      y: ...,
      ...
   },
   ...
}]

TODO

• [x] Update all example specs that use multiple fields for a single axis
• [x] Update encoding parts of all marks
• [x] Update all tests
• [ ] Rename '|xe - x|' threshold to 'assignedWidth'?

There are 3 views in the following spec: 1) ideogram, 2) CN point plot, 3) gene annotation. The views are linked using the same linkingId. zoomToGene jumps to a specific gene, the first views go to the right region, but the last one keeps intact.

I built a search bar for jumping to any region. How can I use higlass auto-completion to get gene suggestion and then go to the corresponding location?

export const getOverviewSpec = (hoveredGene, hotGenes, enabled) => {
  return {
    "arrangement": "vertical",
    "title": "Large Rearrangment Auditing",
    "assembly": "hg19",
    "spacing": 0,
    "views": [
      {
        //"xDomain": { "chromosome": "1" },
        "linkingId": "mychoics_plus",
        "width": 1000,
        "height": 20, // reduce the track height
        "data": {
          "url": "https://dataviz.brbiotech.com/shared/cytoBand.hg19.tsv",
          "type": "csv",
          "separator": "\t",
          "chromosomeField": "chrom",
          "genomicFields": ["chromStart", "chromEnd"]
        },
        "x": {
          "field": "chromStart",
          "type": "genomic",
          "axis": "none"
        },
        "xe": { "field": "chromEnd", "type": "genomic" },
        "alignment": "overlay",
        "tracks": [
          {
            id: "ov-track-1",
            "mark": "text",
            "dataTransform": [{ "type": "filter", "field": "gieStain", "oneOf": ["acen"], "not": true }],
            "text": { "field": "name", "type": "nominal" },
            "color": {
              "field": "gieStain",
              "type": "nominal",
              "domain": ["gneg", "gpos25", "gpos50", "gpos75", "gpos100", "gvar"],
              "range": ["black", "black", "black", "black", "white", "black"]
            },
            "visibility": [
              {
                "operation": "less-than",
                "measure": "width",
                "threshold": "|xe-x|",
                "transitionPadding": 10,
                "target": "mark"
              }
            ],
            "style": { "textStrokeWidth": 0 }
          },
          {
            id: "ov-track-2",
            "mark": "rect",
            "dataTransform": [{ "type": "filter", "field": "gieStain", "oneOf": ["acen"], "not": true }],
            "color": {
              "field": "gieStain",
              "type": "nominal",
              "domain": ["gneg", "gpos25", "gpos50", "gpos75", "gpos100", "gvar"],
              "range": [
                "white",
                "#D9D9D9",
                "#979797",
                "#636363",
                "black",
                "#A0A0F2"
              ]
            }
          },
          {
            id: "ov-track-3",
            "mark": "triangleRight",
            "dataTransform": [
              { "type": "filter", "field": "gieStain", "oneOf": ["acen"] },
              { "type": "filter", "field": "name", "include": "q" }
            ],
            "color": { "value": "#B40101" }
          },
          {
            id: "ov-track-4",
            "mark": "triangleLeft",
            "dataTransform": [
              { "type": "filter", "field": "gieStain", "oneOf": ["acen"] },
              { "type": "filter", "field": "name", "include": "p" }
            ],
            "color": { "value": "#B40101" }
          }
        ],
        "size": { "value": 20 },
        "stroke": { "value": "gray" },
        "strokeWidth": { "value": 0.5 }
      },
      {
        alignment: "overlay",
        "linkingId": "mychoics_plus",
        genomePositionSearchBox: {
            autocompleteServer: 'https://higlass.io/api/v1',
            autocompleteId: 'P0PLbQMwTYGy-5uPIQid7A',
            chromInfoServer: 'https://higlass.io/api/v1',
            chromInfoId: 'hg19'
        },
				data: {
					url: "https://dataviz.brbiotech.com/RS20210907013FFP.panelData.tsv",
					type: "csv",
					separator: "\t",
					sampleLength: 10000,
					chromosomeField: "chr",
					genomicFields: ["uniProbeStart"],
					quantitativeFields: ["CN"]
				},
        mark: "point",
        x: { field: "uniProbeStart", type: "genomic" },
        y: { field: "CN", type: "quantitative" },
        size: { value: 4 },
        tooltip: [
          { field: "chr", type: "nominal", alt: "Chromosome" },
          { field: "uniProbeStart", type: "genomic", alt: "Position" },
          { field: "gene", type: "nominal", alt: "Gene" },
        ],
        opacity: { value: 0.5 },
        tracks: [
          {
            id: "ov-track-5",
            color: { value: "#C7D2FE" },
            //overlayOnPreviousTrack: true,
          },
          { 
            "id": "ov-track-6",
            dataTransform: [
              { type: 'filter', field: 'gene', oneOf: [hoveredGene] }
            ],
            "color": { "value": "#6366F1" },
            //overlayOnPreviousTrack: true,
          },
          {
            id: "ov-track-6-2",
            dataTransform: [
              { type: 'filter', field: 'gene', oneOf: hotGenes }
            ],
            "color": {
              "field": "gene",
              "type": "nominal",
              legend: true,
              "domain": hotGenes,
            },
            //overlayOnPreviousTrack: true,
          }
        ],
        style: { inlineLegend: true },
        width: 1000,
        height: 400
      },
      {
        "alignment": "overlay",
        "title": "hg19 | Genes",
        "linkingId": "mychoics_plus",
        //"xDomain": { "chromosome": "1" },
        "data": {
          "url": "https://higlass.io/api/v1/tileset_info/?d=OHJakQICQD6gTD7skx4EWA",
          "type": "beddb",
          "genomicFields": [
            {"index": 1, "name": "start"},
            {"index": 2, "name": "end"}
          ],
          "valueFields": [
            {"index": 5, "name": "strand", "type": "nominal"},
            {"index": 3, "name": "name", "type": "nominal"}
          ],
          "exonIntervalFields": [
            {"index": 12, "name": "start"},
            {"index": 13, "name": "end"}
          ]
        },
        "tracks": [
          {
            id: "ov-track-8",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["gene"]}
            ],
            "mark": "text",
            "text": {"field": "name", "type": "nominal"},
            "x": {"field": "start", "type": "genomic"},
            "xe": {"field": "end", "type": "genomic"},
            "style": {"dy": -15, "outline": "black", "outlineWidth": 0}
          },
          {
            id: "ov-track-7",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["gene"]},
              {"type": "filter", "field": "strand", "oneOf": ["+"]}
            ],
            "mark": "triangleRight",
            "x": {"field": "end", "type": "genomic", "axis": "none"},
            "size": {"value": 15}
          },
          {
            id: "ov-track-9",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["gene"]},
              {"type": "filter", "field": "strand", "oneOf": ["-"]}
            ],
            "mark": "triangleLeft",
            "x": {"field": "start", "type": "genomic"},
            "size": {"value": 15},
            "style": {
              "align": "right",
              "outline": "black",
              "outlineWidth": 0
            }
          },
          {
            id: "ov-track-10",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["exon"]}
            ],
            "mark": "rect",
            "x": {"field": "start", "type": "genomic"},
            "size": {"value": 15},
            "xe": {"field": "end", "type": "genomic"}
          },
          {
            id: "ov-track-11",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["gene"]},
              {"type": "filter", "field": "strand", "oneOf": ["+"]}
            ],
            "mark": "rule",
            "x": {"field": "start", "type": "genomic"},
            "strokeWidth": {"value": 2},
            "xe": {"field": "end", "type": "genomic"},
            "style": {
              "linePattern": {"type": "triangleRight", "size": 3.5},
              "outline": "black",
              "outlineWidth": 0
            }
          },
          {
            id: "ov-track-12",
            "dataTransform": [
              {"type": "filter", "field": "type", "oneOf": ["gene"]},
              {"type": "filter", "field": "strand", "oneOf": ["-"]}
            ],
            "mark": "rule",
            "x": {"field": "start", "type": "genomic"},
            "strokeWidth": {"value": 2},
            "xe": {"field": "end", "type": "genomic"},
            "style": {
              "linePattern": {"type": "triangleLeft", "size": 3.5},
              "outline": "black",
              "outlineWidth": 0
            }
          }
        ],
        "row": {
          "field": "strand",
          "type": "nominal",
          "domain": ["+", "-"]
        },
        "color": {
          "field": "strand",
          "type": "nominal",
          "domain": ["+", "-"],
        },
        "visibility": [
          {
            "operation": "less-than",
            "measure": "width",
            "threshold": "|xe-x|",
            "transitionPadding": 10,
            "target": "mark"
          }
        ],
        "width": 1000,
        "height": 100
      },
    ]
  };
};

This PR adds native support of matrix data. Example of drawing matrix using matrix format data:

{
     data: {
          url: GOSLING_PUBLIC_DATA.matrixMicroC,
          type: 'matrix'
      },
      mark: 'bar',
      x: { field: 'xs', type: 'genomic', axis: 'none' },
      xe: { field: 'xe', type: 'genomic', axis: 'none' },
      y: { field: 'ys', type: 'genomic', axis: 'none' },
      ye: { field: 'ye', type: 'genomic', axis: 'none' },
      color: { field: 'value', type: 'quantitative', range: 'warm' },
      width: 600,
      height: 600
}

Screenshots

https://user-images.githubusercontent.com/9922882/146459349-a7f10f91-2476-40f6-8f3e-e99b488946bc.mov

There are multiple follow up things to do:

• [ ] Axis on 2D tracks seem to be broken. This needs to be fixed.
• [ ] Improving the rendering performance. Zooming is slow with dense Hi-C data.
• [ ] Supporting log scale color (e.g., color: { ..., scale: 'log' })
• [ ] Supporting horizontal rules. Currently, we support vertical rules (see the MATRIX example).
• [ ] Somehow, the right-top part along the diagonal is zero values. Need to confirm if data itself only contains the left-bottom part to reduce performance. In this case, we need to properly manipulate data when loaded.

Adding a custom-defined genome is easy for higlass, while doing this is hard for gosling. Could you please guide me in modifying the source code for gosling to achieve this? After finishing the modification, zooming, panning and zoom_to are expected to function correctly. Besides, sequence abstraction can be easily implemented by add a custom-defined genome to gosling.

Hi,

I'm playing around with using gosling to display information w/o a chromosome (a random contig and annotations), and was wondering if it's possible to remove the "chrN: " from the axis? e.g.

Thanks!

Apologies for the lack on context behind the choice, but I don't know if I understand the motivation for including quantitativeFields, genomicFields, and chromosomeField in the CSV data definition. These fields aren't marked as required in the docs, but every example I've seen includes their use.

Motivation

To my knowledge, these columns inform how the CSV is parsed but this interpretation is also captured elsewhere in the track definition (type: genomic, quantitative, categorical, etc), so really it's an abstraction leak. I see how the chromosomeField is currently necessary, but I'm curious if that information could also be captured in the "genomic" type rather than the data definition.

As a motivating example, what is the expected behavior if I use a field type that differs from the data definition? I assume the track type takes precedent, and if so we don't need the data definition since a type is required on all tracks.

import gosling as gos

data = gos.csv('./data.csv', genomicFields=['start', 'end'], chromosomeField='chr', quantitativeFields=['value'])

gos.Track(data).encode(x=gos.Channel('start:G'), y=gos.Channel('value:N')) # value doesn't match data definition

Proposal

Remove quantitativeFields, genomicFields, and maybe chromosomeField from CSV definition. This would make specifying CSV data more concise and avoid the case where data definition does not match track definition.

import gosling as gos
import pandas as pd

data = gos.csv('./data.csv')
data = pd.read_csv('./data.csv').gos.csv() # no arguments needed

Approach

Use build-in (d3?) auto-parsing of CSV into memory. We can coerce any data-types that are mis-interpreted based on the track definition. Perhaps as an extension to #575, we can think about if there is a way to include chromosome field in the X encoding definition.

interface X {
  type: 'genomic';
  // tuples represent chromosome position OR chromosome region
  field: [chrom: string, start: string] | [chrom: string, start: string, end: string];
  // ...
}

This PR uses more precise types of individual channels, like X, Y, and Color. This allows checking gosling.js specs more precisely, e.g., row cannot be encoded with a genomic field:

row: { ..., type: "genomic" } // Error

or x is always mapped to a genomic field:

x: { ..., type: "quantitative" } // Error

Genomic domain is used for only x-axis channels:

y: { ..., domain: { chromosome: '1' } } // Error

Towards #506

Bumps json5 from 1.0.1 to 1.0.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps decode-uri-component from 0.2.0 to 0.2.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps qs from 6.10.1 to 6.11.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Here is a screenshot:

When the "background" parameter is set in mark_line(), the gos.overlay() does not overlay the marks inside "tracks". Instead it does a simple line graph using the LAST gosling track inside "tracks"

Using streamlit, the output looks like this.

When I remove the "background parameter from mark_line() as such:

I get the intended display. However, if I want the background to be anything other than white I cannot.

Hi, I am currently using the python implementation of gos. As far as I'm aware, I don't believe there is an option to "label" a set of vertical views, or a several vertically stacked tracks. Please correct me if I'm wrong!

This could be a potentially important feature in the future.