Levelling-up Assets Management

You may have recently noticed a change in how your assets are managed in our magic-infused, web-based video editor. We've released a new Assets Explorer, an in-app file browser that enables users more control over how their assets are organized.

Assets Management V2: More Control & Flexibility

Assets are fundamental to every Runway project. An asset is created anytime a user uploads a file, as well as whenever a user exports a project. With continued usage, it is easy for users to gather a sizeable amount of assets over time.

Currently, we display assets as tiles in a grid. So far, it has served us well by providing users with a to-the-point, visual representation of their assets. However, its days were limited. From the insights we'd gathered from user feedback, it became clear that we needed a solution that enables users to organize their assets at scale.

With this in mind, we set out to kick off the Assets Explorer: a file browser for assets in Runway. This would enable users to organize their assets into folders, as well as provide search and filtering functionality for easy access.

Interaction Milestones: Collaboration between Design & Engineering

File browsers are commonplace in operating systems and professional apps. Despite this, implementing it turned out to be more complex than we had originally anticipated. They afford a variety of user interactions; dragging files between folders, selecting multiple files, expanding and collapsing folders, etc. Each of these interactions is nuanced and impose their own set of edge cases. As such, it wasn't feasible to spec out the minutiae of all these interactions accurately in advance.

To ensure that we were able to iterate on the design as the implementation evolved, we broke down the task of building the Assets Explorer into "interaction milestones". Each of these milestones represented a discreet piece of functionality such as:

• representing folders in a tree structure
• selecting and moving multiple files
• searching and filtering

Storybook provided us with an isolated environment to build and test the interactions without the need for in-app authentication and API interfacing. It also allowed us to build our File Explorer using mock data. We'd taken advantage of this by plugging in a large, nested list to stress-test our UI's performance.

Milestones were picked up by the frontend team. Using existing File Explorers as references, a first pass would be built in Storybook for the specific interaction. This is then handed off to the design team for feedback. The granularity of these milestones allowed us to focus on specific interactions and achieve consensus between design and engineering.

A nice side-effect of this approach was that it also allowed us to reevaluate the scope of the initial release. In the middle of the mission, we'd realized that we had an inventory of milestones that were already signed-off and immediately valuable to users. We were able to ship these ahead of the other milestones and provide users with something they could make use of. With that said, stay tuned for more improvements soon 😎

Optimizing for User Experience

The Assets Explorer interfaces with the backend through a series of endpoints to a) fetch the assets to render, and b) keep the backend up-to-date with the client's interactions.

With the Assets Explorer, we were vigilant that users with a lot of assets may experience longer waiting times for their asset requests to resolve, and also that users can perform updates more frequently than the speed at which the requests resolve.

In most cases on the web, the client can handle this simply — send out the request, show a loader in the meantime, and render the results when the request resolves. However, for the Assets Explorer, this pattern could lead to long and frequent loading times which block user interactions, and ultimately an unpleasant user experience.

With this in mind, we came up with these solutions to keep the backend and client in sync.

Fetch Assets Proactively

As users may have a lot of assets, it could take a while to retrieve all the user's assets. We considered two approaches to solve this, either lazy loading subsets of a user's assets only as the user needs them, or eager loading all the assets at once.

Lazy Loading Assets as Needed

Since the Assets Explorer contains collapsible folders, the client only needs to know assets which visible, or in other words, are children of expanded folders. This presents us with the opportunity to only lazy load the required subsets of a user's assets whenever they expand a folder, instead of loading all the assets at once.

Fetching data in this piecemeal manner would result in shorter waiting times, in theory. This could save a significant amount of time and memory during the user's session especially in cases where the user has a large number of assets.

However, this increase in speed comes at the cost of more requests needing to be made, and at times where the user is actively interacting with the Assets Explorer. In the end, it seemed that although the initial load would be faster, this approach would impose loading times throughout the user's interactions at points where it would be beneficial to see the data immediately.

Eager Loading All Assets

Loading the whole list at once seemed to be preferable for user experience. We fetch all the assets at once, instead of fetching parts of the list frequently. With this pattern, users would only see one initial loading state instead of many shorter ones throughout their interactions with the Assets Explorer.

To minimize the impact of the long initial load on the user's experience, we load the assets eagerly, as soon as the user opens the application. This gives us a head-start in loading the assets in the hope that the complete list of assets would be loaded by the time users navigate to Runway from the homepage.

On top of this, we cache the user's assets on the client and follow a stale-while-revalidate pattern. Caching minimizes the need to re-fetch data, and the SWR allows us to make use of data we already have while we are fetching new data.

This combination of minimizing the number of network requests from the client and abstracting away loading times allowed us to reduce the impact of network latency on the user's experience of the Assets Explorer.

Optimistic Updates

Users can perform a range of actions with the Assets Explorer, — creating folders, renaming folders, moving files — all of which need to be communicated to the backend via a set of endpoints. These actions can be performed quickly by the users and at a rate quicker than it takes for requests to resolve. A user can create a folder, move assets into it, and delete it all in a matter of seconds.

We perform updates optimistically, assuming that it will resolve successfully instead of waiting for a request to resolve. This assumption frees the client to reflect the user's changes immediately in the app without waiting for the network roundtrip to resolve (since the client knows what the resulting output would be). In the case where a request fails, we simply re-fetch the tree from the backend to keep the client in sync with the backend.

A caveat here is that both the client and backend logic needs to be maintained in tandem for each action. If there were inconsistencies in the logic, we could run into errors and unexpected results. Given the benefits of optimistic updating to the user experience, we thought this was a cost worth taking on. Unit tests and pair programming sessions between the backend and frontend were helpful here to make sure we were aligned in our implementation.

Order is also important in these updates, and actions must be performed sequentially. Take this sequence of actions for example:

1. the user creates a folder
2. the user moves assets into the new folder
3. the user deletes the folder

These updates can only happen in that order — we can't move assets into a folder that doesn't exist, nor can we delete a folder that doesn't exist. To ensure race conditions don't occur, we queue client requests to make sure they resolve sequentially. This is handled neatly by setting up a PQueue with concurrency set to one.

Taking time to consider what we implement, and how we implement it has served us well at Runway to build user interfaces that put user experience first. We hope you enjoyed getting a peek into how we work at Runway. If this sounds like the type of project you'd be interested to work on, we're hiring! Join us on our journey to build the future of video editing.