Jorge demoed the ChRIS plugin registration UI, Gideon demoed the updated add node UI with the CCC students matrix plugin, we had a discussion about CPU vs GPU processing outside of OpenShift, we had a discussion about sharing finer-grained state information about plugins as they run in the feed UI, and we talked about enabling PACS queries in the system in a short-term simplified way to hasten deployment.

• Admin now can upload a text file and register plugins from it.
• Text file contains plugin name, version, and compute environment it will be assigned, could also contain URI
• If version is left out, assumes the latest
• Plugin must be in the ChRIS store to work
• Handles errors eg version not found or not present in store or already installed

Demo screenshots (using txt file list of multiple):

Click on "Add plugins from file" in upper right

Create txt file with list of plugins

Plugins must be present in the ChRIS store. If version number not listed, it'll grab the latest available.

Select + upload txt file with list of plugins into admin panel

System processes text file and registers plugins

Post registration status page

Demo screenshots for adding a single plugin:

Click on "add plugin" in the upper right

Fill out the form

View plugin in list, select to modify

Plugin modification screen

Bill Status

• DevConf talk was published on YouTube: https://www.youtube.com/watch?v=ap88G5FbRoE
• Mo reached out to potential intern candidate

Gideon

• Reached out to cloud computing group students
• Students created a workflow - matrix multiply plugin
  • Simple app to test the GPU
  • Send it some parameters that define the square size of matrices with steps - sorts it by 32x32, 64x64, 96x96, 128x128 as a quick dummy test. Shows the time to multiply those random matrices on the GPU
  • This plugin is supposed to run on the GPU, this particular interface is running on a machine at BCH with nice GPUs on board, Rudolph confused how it's working... when you run this as a normal docker run, you would have to pass --runtime nvidia flag in the command line to have it run on the GPU. (It ran because Gideon created the tree in the CLI.)

Demo Screenshots

Gideon applied all of the feedback we gave last week around the design of the add new node dialog, and we agreed during this demo it appeared to flow much better. Some of the changes:

• The text field config and form field configs are now two distinct screens in the wizard rather than two modes on a single screen. You fill the form out first, then can modify the form's output in the full text mode.
• The full text mode now has the output from the form config option by default.
• The full text mode detects if you add an unsupported option and raises an error message
• The full text mode has a documentation area on the bottom that outlines the options that can be used.

CPU vs. GPU issue with the user interface:

• since it's not getting the runtime flag, it's running on the CPU, and it doesn't really run on the CPU
• one of the students discovered by accident, if you call the REST API ChRIS with those additional flags (--runtime nvidida),

pfurl --auth chris:chris1234 --verb POST \ --http ${HOST_IP}:${HOST_PORT}/api/v1/plugins/19/instances/ \ --content-type application/vnd.collection+json \ --jsonwrapper 'template' --msg ' {"data": [{"name":"runtime", "value":"nvidia"}, {"name":"coenum", "value":"32,32,128"}, {"name":"previous_id", "value":"1"} ] }'

• The Docker daemon intercepts this flag that's incorrect for the plugin and interprets it for itself, and this allows the plugin to run in the GPU

• How could the UI elegantly handle GPU vs CPU compute

  • Should be handled plugin by plugin or by feed?
  • Should be plugin by plugin

• Not sure how OpenShift handles the GPU.... this isn't going through OpenShift right now, it's going through Docker.

• mizmo recommends adding an 'environment' variable into the 'add a new node' wizard when you add a plugin to a feed.

• danmcp says plugins should have a flag to indicate GPU desired with max, min for OpenShift. We don't use it for docker runs, we can't use in docker environment.

• danmcp says the flag is on the plugin itself, to specify GPU desired, in that JSON. It was, but not sure if in the latest version.

• Jorge says when you run plugin instance can specify GPU limit with an integer, they use in OpenShift in one way, could use it in another way in Docker.

• Rudolph: In OpenShift it's used one way, in Docker it's used another way.

• Danmcp, not sure why it's a problem. If you run on OpenShift run this way, if you run on swarm run this way.

• Rudolph: Makes perfect sense, what has to happen here at the level - handled well at the OpenShift case but not handled elegantly at the Docker/Swarm case. So todo on our side is to look at how the config is processed in the OpenShift case and use that logic to have it work in the non OpenShift case.

• Danmcp: What should we show in this sort of UI from the user's perspective. This probably isn't a great thing for a user to answer, better for the backend to decide. If GPUs are available and plugin can use them, let it do so. If no GPUs available, and it can run on CPU, have it run.

• Jorge: plugin defines what it prefers but user can override.

• Danmcp: user should just indicate if it's ok for it to run slow or really prefer fast

• Rudolph: user necessarily shouldn't have to make choice, would be good to have code that analyzes metadata and makes determination on whether to have GPU running or not

• Rudolph: this plugin only runs on GPUs. If specify min of 1, then has to get a GPU

• mizmo concerned about messaging - error / notification status messages.... eg if can run on CPU and it runs then another one waits on a GPU, would be good to show that it's specifically waiting on a GPU so no confusion over why one feed is running and the other isn't

Finer-grained status information in the UI possible

Transparent to the user, pfcon regularly polling the resource to ask about job status this json structure comes back each time it polls

It's a file but not fed back to UI. But we do get this info back every second or two seconds or something. This does afford a much richer potential feedback to the user in the UI. a perfect example - right now in the other feed just has the spinning wheel of death and have no clue about what's going on.

Four stages of processing:

• push path, moving data to remote env
• compute
• returns, pulling back from remote resource
• putting info into swift

Mizmo to design notification for status

• What makes sense, n number of stages.
• Timeline status horizontal bar for the 4

Mizmo status: Library UI mockup progress

• want to do particular type of analysis - mprage
  • either want to do one off
  • or want to do gathering a bunch of data

Three thoughts:

1. Some way to somehow automate a kind of hunting and gathering thing... can imagine the case, you have 300 patients, manually clicking through them to hunt and gather. Can achieve with search across google books style interface. Confused by chapter dropdown. Show ref data vs patient data
2. Image data is just one potential data type for ChRIS. this interface very specifically catered for going thru open subset of potential data spaces. e.g. if its genomic data... maybe have different library types, MRI vs genomic data
3. ChRIS being held up for more wider deployment bc of lack of PACS collection and choosing stuff in swift tree for quick analyses... finishing up pacs query app... should have pacs lookup that works in ChRIS interface... but that'll take a long time to deploy coudl take months. Want something, maybe already available in js to do this, how could we jury rig onto the system, so someone could click something to run it. Would be super useful in library, can we apply it to just be able to do quick and dirty navigating through swift

Production chris - uses apache runs fast, Gideon has to take look at UI

Post-meeting, Mo and Gideon met up in Slack and came up with the following rough mockup for a quick & dirty browser for swift in the Library tab: