Meet the Creators: George Profenza

We recently spoke with George Profenza who developed an open source library for integrating RunwayML with Processing, an easy-to-learn, open source programming language for creating with interactive media. George’s contribution provides a simple way for Processing users to connect to RunwayML from their Processing sketches, enabling an entirely new form of interaction with state-of-the-art machine intelligence capabilities.

Thanks for chatting with us! Can you talk a little bit about yourself and your background?

Hi! I’m a London-based, Romanian-born, Creative Technologist.

I picked up creative coding toolkits such as Processing, OpenFrameworks, and MaxMSP, during my undergraduate studies in Creative Computing at Goldsmiths, University of London, and I did a bit more Processing during my Architectural Computation MSc at University College London, a long time ago. I have used all of the above and much more (Python, C++ for embedded devices, etc.) in the past eight years working at Hirsch & Mann.

I also work on freelance projects and collaborations where I explore things outside of my daily work comfort area, as long as the scope is finite and manageable.

Can you talk a bit about the new RunwayML Processing library?
What’s there and what inspired you to work on it?

Processing makes it easy to learn programming in a simple visual way, especially for those from creative backgrounds. For example, you can write a short line of code such as point(50,50); and press play to immediately run a program that draws a dot on the screen.

The barrier for entry into machine learning — especially from a creative (arts/design) background — is quite high. Typically you have to learn enough about Terminal/Command Prompt to set up Python and many, many dependency libraries, configure neural networks, train or use checkpoints, and so on and so on. (Don’t even get me started on compiling TensorFlow from source with CUDA support on OSX.) There are many steps until you finally get to the fun part: feeding new data to a neural network to see what it produces.

RunwayML does an excellent job at completely removing the high barrier to entry by offering an intuitive user interface for running machine learning models that would otherwise be very complex to set up and use. In addition, there are so many creatives with wildly different ideas for how to use open source machine learning models. These two reasons inspired the library, and I’m happy for the opportunity to contribute.

Running code examples is like playing with LEGO bricks: they provide instant gratification, you can tweak them and break them to understand how the code works, and often, you can remix/collage them into their own mini-projects.

The RunwayML Processing library includes over 80 examples because examples are key to any successful creative coding library. For example, Three.js, openFrameworks, and Processing itself, all provide a plethora of useful examples to help people learn those programming languages and environments. Books and documentation are great for gaining in-depth knowledge, but I’ve always learned faster by doing, especially tinkering with examples. Running code examples is like playing with LEGO bricks: they provide instant gratification, you can tweak them and break them to understand how the code works, and often, you can remix/collage them into their own mini-projects. For this reason, it was important to include as many examples in the library as possible to help people get started. The examples may not be exciting and inspiring, but it’s something to strive for. I also had fun sneaking references to both male artists and female computer scientists and mathematicians who had radical ways of looking at the world.

Do you use machine learning in your own creative and professional practices?

I do use machine learning for my own creative practice and strive to include it into my professional practices as well. I had the pleasure of collaborating with Jessica In for her project NORAA: an interactive physical drawing machine that invites dialogue between humans and a machine learning algorithm. I had worked with convolutional neural networks before (for emotion classification, for example), but my work with Jessica motivated and challenged me to learn about recurrent neural networks. NORAA uses two such neural networks: a classifier based on the QuickDraw dataset (and the TensorFlow tutorial) and David Ha’s Sketch-RNN network to predict drawings from simple input strokes. I must admit I had to read the Sketch-RNN paper quite a few times until the network architecture started to click for me.

Do you have any recommendations for other who might wish to contribute? Either to RunwayML’s examples or another open source space?

The most important recommendation: do it because it’s fun!

I would love to see everybody’s ideas, projects, and hopefully even more examples. New models are frequently added to RunwayML, both from the RunwayML team and the community, so new Processing library examples are more than welcome.

Regarding other open source contributions, once your understanding of programming fundamentals is in place, find a library that you like and try to contribute examples or new features.

I love neat, clean code that is super easy to read and understand. Contributing to an open source project written in a style of code that you admire is an excellent way to learn and deepen your understanding and skills.

Can you talk about a project you’re working on right now? Or, what’s the best way for folks to get in touch with you if they want to learn more about your work?

I’m afraid I can’t share details about my commercial work right now, however feel free to follow updates on my website, Twitter, and Github feeds. I also contribute answers on Stack Overflow (especially with the Processing tag) and in the RunwayML Slack.

To get started with RunwayML and Processing, download RunwayML Beta and watch this quick video for instructions on how to install and run examples in the RunwayML Processing library.

If you’d like to contribute to RunwayML’s development, visit our GitHub account for our official open source projects. Any and all examples, tutorials, and additional libraries are welcome!

Connect with us through our Support Channel or via Twitter and Slack for questions and to share your creations.