The Journey of Bringing webR into Shiny: Boost Performance, Improve Scalability

webR has been a trending topic ever since the release of webR 0.1.1 in March 2023. With the introduction of webR, developers can run R commands directly on top of their clients’ browsers. This eliminates the need for users to install R separately to run R scripts, allowing them to leverage their own devices for computation without waiting for a server to process and return results for each user’s request. While the topic of WebAssembly and webR might not be commonly utilized by Shiny developers, having this skill in your portfolio proves valuable when the need arises.

TOC:

• What Is webR?
• Why Does Shiny Need webR?
• How Does webR Work?
• How to Setup webR on a Shiny Application
• Tips and Tricks with webR and Shiny
• Practical webR Shiny Demo with Rhino – The Traveling Salesman Problem
• Hosting webR Shiny Applications on Posit Connect

What Exactly Is webR?

As mentioned earlier, webR enables R to run entirely on the browser. It represents a version of the open-source R interpreter compiled for WebAssembly, accompanied by a supporting TypeScript library that facilitates interaction with the console and R objects from a JavaScript environment.

What is WebAssembly?

WebAssembly, introduced in March 2017, empowers developers to incorporate tools and libraries developed in other languages directly into JavaScript. This eliminates the need to reinvent the wheel by learning JavaScript and rewriting the logic in it.

By compiling R to WebAssembly, users gain the ability to visit a website and execute R code directly within their web browsers, without the need to have R installed on their devices or rely on a supporting computational R server. All that’s necessary is a regular web server, which can include the type of cloud hosting service offered by Github Pages or Netlify. This streamlined setup allows users to leverage the power of R in a web-based environment without any additional infrastructure requirements.

Is webR the Future of Shiny?

The next question that may arise is, “Doesn’t Shiny already provide support for users to run R code on web browsers?”. In reality, webR is unlikely to replace Shiny in the near future. However, at present, webR can be used to complement the capabilities of Shiny. In other words, webR can work alongside Shiny, enhancing and extending its functionalities rather than serving as a direct replacement.

Why Does Shiny Need webR?

R is a single-threaded programming language, which means it must complete one task before moving on to the next. Likewise, Shiny is designed to serve one request after another. However, if a particular request involves a lengthy computation, it may result in users experiencing prolonged waiting times. By incorporating webR, this issue can be addressed effectively.

With webR, it becomes possible to delegate the resource-intensive computation to the user’s web browser. This approach allows Shiny to continue serving other users’ requests while the browser handles the time-consuming computation. As a result, users experience reduced waiting times, leading to a more responsive and efficient user experience. By leveraging webR, Shiny can enhance its performance and overall usability, making it a valuable addition.

Shiny, webR, and Scalability

Having limited scalability is a consistent issue being addressed with Shiny. With the integration of webR, this limitation can be further resolved. By leveraging webR, each new user gains the capability to harness the processing power of their own device to execute resource-intensive computations.

This distributed approach to computation not only alleviates the strain on the central server but also enables each user to independently handle their share of heavy computations. As a result, the overall scalability of the Shiny application is significantly improved. By making the most of users’ own device capabilities, webR empowers Shiny to accommodate a larger number of users without sacrificing performance or responsiveness. This enhanced scalability contributes to a more efficient and robust Shiny experience for all users.

How Does webR Work?

webR operates by utilizing a SharedArrayBuffer or, in cases where servers support cross-origin isolation, ServiceWorkers for communication between the R environment and the Javascript environment.

ServiceWorkers and Cross-Origin Isolation

Currently, due to the absence of a straightforward approach to implementing cross-origin isolation within Shiny servers, this article will mainly focus on ServiceWorkers. ServiceWorkers might be a novel concept for R users as they fall under the advanced realm of Javascript.

ServiceWorkers for Multi-Threaded Processing

In simple terms, ServiceWorkers provide a method to implement multi-threaded processing in Javascript, which, like R, is also a single-threaded language.

By employing a ServiceWorker, a separate thread is responsible for downloading the WebAssembly binary and data files from the webR file servers. It’s important to note that this initial download can entail around 20 – 30 MBs of data, which may lead to a slight delay during the first webpage load. However, once downloaded and initialized, the webR binary becomes available to execute R commands through Javascript.

Requirements and Setup

A crucial requirement of using ServiceWorkers is that the ServiceWorker file must reside at the same scope and origin as the page. Consequently, both the ServiceWorker and Worker script utilized for communicating with webR need to be stored at the root level of the web server.

Asynchronous Computation for Responsiveness

As mentioned earlier, since Javascript is single-threaded, executing long-running commands can render the webpage unresponsive to user inputs. To address this issue, asynchronous computation through promises was introduced. This allows files required for webR to be downloaded while the main thread continues to run freely, thereby maintaining webpage responsiveness during processing.

How to Setup webR on a Shiny Application

The most basic Shiny applications are typically written using an app.R file, which contains the UI and server components of the application. However, to ensure that the application we build is production-ready and sustainable throughout the development life cycle, we can organize our files in a single app folder with the following folder structure.

app/
├─ js/
├─ logic/
├─ styles/
├─ view/
├─ www/
│  ├─ scripts/
│  ├─ styles/
global.R
server.R
ui.R

In the above folder structure, the contents of the www folder will be hosted at the root level of the web server. This choice is essential because it allows us to provide the ServiceWorker and Worker scripts in this location, creating the webR environment seamlessly.

Downloading and Setting Up webR

At the time of publication, the latest version of webR is 0.1.0, and you can download the package contents from their Github releases section. Once downloaded and extracted, copy the webr.mjs file into the www/scripts folder and rename it to webr.js. The reason behind this file extension change is that even though the .mjs extension contains the javascript code in plain text, Shiny serves the .mjs file as a binary stream since the Shiny server cannot identify the MIME (Multipurpose Internet Mail Extensions) of the .mjs file.

Copying Additional Files

After adding the webr.js file, you will then need to copy the webr-serviceworker.js, webr-worker.js, R.bin.js, R.bin.data, R.bin.wasm, libRblas.so, and libRlapack.so files from the unzipped package to the www folder of your Shiny application.

Testing webR Integration

Once you’ve copied the necessary files, create an index.js file in the www/scripts folder with the following contents to test whether webR can be successfully loaded through Shiny.

$(() => {
  $(document).on("shiny:connected", () => {
    import('./webr.js').then(
      async ({ WebR }) => {
        globalThis.webR = new WebR({channelType: 0});
        await globalThis.webR.init();
        let result = await (await globalThis.webR.evalR("rnorm(100)")).toJs()
        console.log(result);
      })
  })

To import this Javascript file into your Shiny application you will need to add a ui.R file to the app folder with the following code:

tagList(
  tags$head(
      tags$script(src = "scripts/index.js", type="text/javascript")
  )
)

The server.R file can contain an empty function with the following signature for the time being:

function(input, output, session){
}

The final folder structure should look as follows:

app/
├─ js/
├─ logic/
├─ styles/
├─ view/
├─ www/
│  ├─ scripts/
│  │  ├─ index.js
│  │  ├─ webr.js
│  ├─ styles/
│  ├─ webr-serviceworker.js
│  ├─ webr-worker.js
global.R
server.R
ui.R

Final Steps to Run webR on Shiny

To run webR on the Shiny application, follow these steps:

1. Run the Shiny application: Use shiny::shinyAppDir("app") to launch the Shiny application.
2. Viewing the Output in the Web Browser: Open the resulting Shiny application in a web browser. To access the Javascript console, right-click on the web page and select “Inspect” (on Chrome and Firefox) to open the Developer tools of your browser. The outputs of the Javascript console should be visible under the “Console” tab of the Developer tools.
3. Setting up index.js: In the index.js file, wait for the document to be ready using $(() => ), and for Shiny to be connected with the client using $(document).on("shiny:connected",). Import the contents of the webr.js file, which contains the required functions to set up the service workers and create the environment for webR initialization.
4. Using an Asynchronous Function: Create an asynchronous function to be executed once the import of webr.js is done. Using an asynchronous function allows us to use the await keyword within the function, ensuring that downloading and initializing webR does not disrupt the execution of the main thread.
5. Initializing webR and Evaluating an R Command: Inside the async function, initialize webR, and then evaluate a simple R command. The result of the R command is converted to a Javascript-friendly object to be displayed on the web page. In this case, the result is displayed through the javascript console using console.log(). This confirms that webR is successfully running and executing R commands within the web browser.

A Few Tips and Tricks with webR and Shiny

Enhancing R Code Execution in webR

The previous setup requires passing R commands as a string to the evalR function, which can be limiting and cumbersome for complex functions, leading to inline R commands in Javascript. To address this issue, we can take advantage of a solution where R code is written in a separate source file, and webR sources that file using source("foo.R"). As webR runs in its own environment, it possesses its own file system that can be accessed through the webRFS module in webR.

Using download.file() to Access the File System

The simplest way to get a file into the webR file system is by using the download.file() function in R to download files and then sourcing that file to load all functions into the global R environment. Several methods can be used to host the file for download. In this case, as our web page is served through a Shiny web server, we can host the R file within the app/www folder to enable downloading the file to the webR file system.

Workflow for Loading and Sourcing Files in webR

To load and source a file, the workflow would look like this:

const SCRIPT_URL = "static/tsp.R"
const SCRIPT_FILE_NAME = "tsp.R"

$(() => {
  $(document).on("shiny:connected", () => {
    import('./webr.js').then(({ WebR }) => {
      console.log("registering webr");
      globalThis.webR = new WebR({channelType: 0});
      console.log("registering webr - done");
      globalThis.webR.init().then((init) => {
        console.log("webR is ready!");
        globalThis.webR.evalR(
          "download.file('" + 
            SCRIPT_URL + 
            "', '" + 
            SCRIPT_FILE_NAME + 
            "')"
        )
      }).then(file => {
        console.log("file downloaded");
        globalThis.webR.FS.lookupPath("/home/web_user/")
      }).then(wd => {
        console.log(wd)
        globalThis.webR.evalRVoid("source('" + SCRIPT_FILE_NAME + "')")
      }).then(source => {
        console.log("file sourced")
      })
    })

Promise Chaining and JSON

In the above example, we used a technique called promise chaining to pass promises once evaluated making the code more readable and understandable.

Now that you have your function ready you can pass data from Javascript to R and vice versa. However a caveat with transmitting data between these two languages is that, Sending and receiving matrices in R can be problematic as it seems to be flattened to a vector everytime. The solution is to use JSON as the transportation medium after sourcing the function.

To Receive Data from webR

await webR.installPackages(["jsonlite"])
let a  = await webR.evalR("jsonlite::toJSON(matrix(rnorm(10),nrow=5))")
let b = await a.toJs()
let c = JSON.parse(b.values[0])

To Send Data to webR

let cfn = await webR.evalR("function(d) jsonlite::fromJSON(d) |> nrow()")
await cfn(JSON.stringify([[1,2],[3,4]]))

Practical webR Shiny Demo with Rhino – The Traveling Salesman Problem

Currently, we can execute simple R commands using webR. However, the same can be achieved by communicating with the Shiny server. The real advantage of webR emerges when heavy computations are required to generate outputs.

To showcase webR’s practical application in Shiny, we will demonstrate solving the Traveling Salesman problem written in R, executed on the client’s web browser. This example highlights the power and efficiency of webR in handling complex computations directly within the browser environment.

The Traveling Salesman Problem

The Traveling Salesman problem is like a game where a salesman must find the shortest route to visit multiple cities and return to the starting city. It’s similar to planning the most efficient trip to various locations, just like going to the store, park, library, and a friend’s house. Mathematicians and computer scientists use special techniques and algorithms to solve this puzzle and determine the best route for the salesman. It’s like solving a smart trip puzzle for the salesman’s journey.

What Is the Algorithm Used to Solve It?

The 2-opt algorithm is a clever technique to enhance a route for a salesman visiting multiple cities. It works by examining pairs of cities in the route and swapping them to potentially find a shorter route. This process is iterated until further improvements are not possible, ultimately yielding a more efficient and shorter route for the salesman. It’s like solving a puzzle by rearranging the cities to discover the best possible travel route for the salesman.

What Are the Changes in Setting Up webR in a Rhino Application?

In a Rhino application, you can create a “www” folder in the root directory and place the webr-serviceworker.js, webr-worker.js, and other files mentioned in the setup, inside it. The JavaScript file responsible for loading webR (let’s call it “connect_webr.js”) can be stored in the “app/static/js/” folder. To add this file to the application’s UI, include a script tag with the “src” attribute set to “static/js/connect_webr.js” as shown below:

tagList(
  tags$script(
    src = "static/js/connect_webr.js",
    type = "text/javascript"
  )
)

The contents of the connect_webr.js will be as follows at this stage:

const SCRIPT_URL = "static/tsp.R"
const SCRIPT_FILE_NAME = "tsp.R"

$(() => {
  $(document).on("shiny:connected", () => {
    import('./webr.js').then(({ WebR }) => {
      console.log("registering webr");
      globalThis.webR = new WebR({ serviceWorkerUrl: serviceWorkerUrl() });
      console.log("registering webr - done");
      globalThis.webR.init().then((init) => {
        console.log("webR is ready!");
        globalThis.webR.evalR("download.file('" + SCRIPT_URL + "', '" + SCRIPT_FILE_NAME + "')")
      }).then(file => {
        console.log("file downloaded");
        globalThis.webR.FS.lookupPath("/home/web_user/")
      }).then(wd => {
        console.log(wd)
        globalThis.webR.evalRVoid("source('" + SCRIPT_FILE_NAME + "')")
      }).then(source => {
        console.log("file sourced")
        globalThis.webR.installPackages(["jsonlite"])
      }).then(finish => {
        console.log(“packages installed and webR is ready”);
      });
    }, (error) => {
      console.log(error)
    }
    )
})
})

Asynchronous webR Computation in Shiny

With the webR function set up, we can now register an event handler in JavaScript that responds to a reactive value in Shiny. This allows us to execute the webR computation process asynchronously whenever a user performs an action.

After the R function returns its results, we can parse the JSON data and use it to set an input value in Shiny, reflecting the resulting JSON. By using an observeEvent on the Shiny server, we can then retrieve and parse this JSON to display the computed results on the web page. This seamless process ensures that the webR computation is executed in response to user actions, and the outcomes are promptly showcased within the Shiny application’s UI.

Shiny.addCustomMessageHandler("send_dataset", function (message) {
   console.log("Got message: ", message)
   globalThis.webR.evalR("handle_two_opt")
     .then((twoOpt) => {
        return twoOpt(JSON.stringify(message))
      })
     .then((result) => {
        let obj = {}
        result.values.forEach((x, i) => { 
             obj[result.names[i]] = x.values 
        });
        let result_json = JSON.stringify(obj)
        console.log(result_json)
        Shiny.setInputValue(
          "app-page_travel-webr_results",
           result_json,
           { priority: 'event' }
        )
     })
})

Hosting webR Shiny Applications on Posit Connect

When hosting a Shiny application on Posit Connect, Posit Connect injects a <base> tag with a worker ID, which can disrupt webR functionality. To work around this issue, you can pass a proper serviceWorkerUrl argument to webR().

Alternatively, you can delete the <base> tag entirely using JavaScript, although this approach may introduce unknown complications.

function serviceWorkerUrl() {
  const base = document.querySelector('base');
  if (base && base.hasAttribute('href')) {
    return '../';
  }
    return '';
  }
… // other boilerplate code
import('./webr.js').then(({ WebR }) => {
  console.log("registering webr");
  globalThis.webR = new WebR({ 
    serviceWorkerUrl: serviceWorkerUrl() 
  });
  … // the rest of the computation
})

Integrating webR into Shiny: Unleash the Power of R on the Web

With webR’s release in March 2023, developers gained the ability to execute R commands directly in clients’ web browsers, reducing the need for server-side computation and installation of R on user devices. While webR doesn’t replace Shiny, it enhances scalability by offloading heavy computations to the browser.

For improved app performance with technical guidance and domain expertise, reach out to Appsilon to harness the full potential of webR in your Shiny applications. Better performing apps and cutting-edge tools can make all the difference for your team.

If you want to learn more about serverless Shiny, let’s talk.