About

In this document I try to narrate how I approached the #fxhackathon2023 event, how I took the initial prompt and set up my goals for it. Part mood piece, part technical breakdown, I describe my process and the various challenges I encountered throughout production.

I hope this will shed light on where the project comes from and what I’m aiming for.

Prompt analysis

Here’s the prompt, from the event page :

The end result of your work over the six-week time period should be to mint a novel, creative, and innovative fx(params) project that takes advantage of the code-driven params field to develop a custom minting interface. The emphasis of this project is to make sure your design encourages meaningful interactions between the artist, artwork, and collector.

There was also a workshop on Discord with @ciphrd early on where I grasped a few more tidbits of what was expected from the event, particularly in terms of where one could go for the UI, @ciphrd encouraging the use of modern web framework like Vue or React.

What I took from these :

Ideally the user wouldn’t have to interact with the default params panel.
Put the collector’s experience and the minting itself front and center.
Wide demographic target, so simple and intuitive interactions
Openness / democratization which seems a core value at fx(hash)
Some pedagogic component

Some of these are from the prompt, some from the optional goals.

Inspirations

`Tears of the Kingdom`

A video game I played not long before the event, in which you have a system allowing the player to stick various objects together, supported by a very controlled/safe physics simulation. Some objects have special effects (motors, wheels, springs, etc.) and often have interesting interactions with each other and the environment. Most of all, it’s packaged in a easy-to-use interface and the things you build behave generally as you would expect them to, and often make you feel clever.

`Universal Rayhatcher` by @Piter Pasma

A very cool project in which you can draw basically anything as long as you can write the mathematical function than defines it in less than 512 characters of code. A very collector focused project where a good chunk of the community participated. Lots of creativity, very stimulating, if a bit technical and obviously requiring some notions of math/programing. The project was a big success, and gave me the proof I maybe needed to go for a collection where I wouldn’t control much, leaving nearly everything to collectors.

`The Incredible Machine`

Another game, in which the player solves puzzles by designing increasingly complex Rube Goldberg devices, supported by a deterministic physics engine. I hadn’t thought about that game in years, but while brainstorming for the event, it came to me, and I remembered the fun of building these machines and then watching them animate. The title is a reference to this game, and the general gameplay loop is the same.

Basic concept

From this analysis and the context/mood at the time, it’s pretty easy to imagine the process, but what I knew was :

Complex emergent behavior from (outwardly) simple elements. This would manifest by the use of a physics simulation on one hand, and interdependent systems on the other.
2D for simplicity. 3D controls are way harder, and people are way less familiar with them.
Easy/compact serialization : params are written on the blockchain, each byte costs money, so they need to be as small as possible. Hence a fixed grid and ideally mostly discrete values.
I wanted to put any collector in the shoes of a generative artist, have them required to go through similar processes as I do, like finding beauty in chaotic and often hard to control systems, balancing performances versus complexity, or simply choosing colors. Trade-offs , impactful choices, but also experimentation and that rewarding feeling of achievement.
And following that, a healthy dose of game design: friction where appropriate, hidden information, surprises, rewards.
Ideally give the collectors the feeling of building the collection together, as a group.

And that was enough to start prototyping. I also kept in mind the deadline and already knew at the time that it would be a tight fit, considering there were still some big technical unknowns, so …

Feasibility check

Update to params

First to test was the update to the fx(params) system and the code-driven params. My latest released project before this one was my first params test, Beneath a Sick Sky (still a few to mint as I write these lines btw) for which I remember thinking that the only thing lacking was real-time update.

I work nearly exclusively in real-time (i.e. ideally the artwork should render in less time than the refresh rate of your screen) and as such, code-driven params make perfect sense.

I took a session to test these update and integrate them in my usual code structure.

Setting up a project for #fxhackathon, looking at the updates to Params, and going for the obvious thing to test : user drawing as a param!
Now to find a good idea …#fxhash #generativeart pic.twitter.com/VeETEPgBV4
— Christophe Choffel (@ULuIQ12) June 25, 2023

No integration problems there, only maybe some questions about params maximum size. I had the 512 characters from Universal rayhatcher in mind and didn’t even consider compressing them at the time.

Deterministic physics

A core rule at fx(hash) is that artworks should be deterministic. It’s one of the most common mistake creators make when first coming to the platform because they’re not necessarily used to working with seeded random functions. Determinism troubles are also often due to floating point number precision : memory being a finite resource, numbers are stored in a fixed number of bits, but you cannot represent all number that way, you have to simplify ( how many decimals of PI do you really need ? ).

Physics simulations are bound by these precision concerns and often are NOT deterministic, meaning that more often than not, it’s perfectly fine to allow some variance. But in our case, we need result to be reproducible.

With these concerns in mind, and not willing to develop my own physics implementation not knowing precisely what I would need, I looked for a free/open source solution and found the Rapier physics engine which promised cross-platform deterministic physics.

After some test, turns out their claims were true as far as I could tell, so I went with it and have not been let down. Every time determinism was broken it was my own fault 🙂

User Interface Layer

I usually put some user interface in my projects, but use either some very lightweight UI kit or do my own components in the Context3D ( same as the rest of the artwork ), which is manageable as long as it stay simple.

For this project though, we’re all about the interface. As we’re on the web, in a browser, the most sensible solution is to use the DOM, that is what you see most of time in your browser : buttons, text inputs, dropdowns, etc. controlled by some JavaScript. In modern websites, you would usually manage/design those with a framework, and I knew I would need one too, and sooner than later.

Vue.js was recommended during the talk with @ciphrd, and React was also mentioned ( fx(hash) runs on React for instance).

I quickly tried to setup a project with Vue, but had trouble setting up all my tools. Then I tried with React and manage to have a functioning setup with webpack, the fx(hash) boilerplate, my code, NPM, Typescript and JSX / TSX support (these are all tools that allow me to reduce errors, have some nice auto-complete in my code editor, easy external code import , and easy project exports). Added Material UI and Material Icons on top of that ( a React components library, and an icon font ) for a good basis covering most of what I would need.

Prototyping

With all these concerns out of the way, I had to first test if the concept had any merit or if I had to go back to the drawing board.

A few general paradigms were quickly established

The 7×7 grid. 5×5 felt to small, 9×9 was already a bit overwhelming. I very quickly ended on this value for the design space.
The data model, consisting of these 49 tiles, each having a type (Empty by default). Each tile would hold the various configuration data that would translate to the main param. The string of 512 characters I had in mind would fit rather well with around 10 bytes per tile.
The big switch between the Design Mode and the Simulation Mode which mapped well the the minting / standalone modes on fx(hash). The most important is that the switch had to be fast, to allow for fast test/iteration while designing.
The change of scale between the two modes
Wrapping behavior for the contraptions. I tried to have collisions with the frame, but the contraptions would just get stuck in corners.
The separation between motor blocks and drawing blocks, and then came the modificator blocks that influence the behavior of their orthogonal neighbors.

Here’s what it looked like, with already most of the concept implemented one week after the event’s announcement :

It sure is … generative.
Will keep digging! ⛏️#wip #fxhackathon #generativeart #fxhash pic.twitter.com/Gmw0DUGLIC
— Christophe Choffel (@ULuIQ12) July 3, 2023

I then focused a bit on the “art” part to at least to do a general performance check with regards to the number of vertices . I knew it would be pretty abstract, with lines and particles. The particles would be represented as quad, that is 4 vertices defining 2 triangle faces, and the lines would be long strips of those quads, sharing vertices.

Here’s an early implementation, with also the concept of modificator blocks added :

It's a bit hard to explain what's going on in a tweet, but the idea is that the construction blocks have interesting interactions when adjacent, here a "Sine" block next to a motor changes its force, and next to a brush changes its width#wip #fxhackathon #fxhash #generativeart pic.twitter.com/8zUJ8pZMrI
— Christophe Choffel (@ULuIQ12) July 6, 2023

Note that in this example, the particles are composed of two quad, one for the color part and one for the outline, to allow for some merging effect. This got abandoned mostly for performance reasons, as they’re twice as costly, and I decided I’d rather have more.

That’s it, done! Everything is there. Just have to polish !

Production and iterative design

As a mirror to what I had in mind for the experience itself and with the basic concepts established previously in mind, the rest of the project evolved by iteration, by adding features progressively and testing them extensively, along four main axis :

The art part : what does the artwork itself looks like , but also what the building part / UI looks like. Is there a theme ? I had no clear idea at that point.
Minting UI design and implementation which I knew would take at least 50% of the project, and for this event specifically, at least as important if not more than the artwork itself, artwork mostly in the hands of the collectors anyway.
UX / Game design : What do I want the collectors to experience ?
The code itself, with its structure, but also the hacks and trade-offs.

The art part

How do you design the look of a collection that you put nearly entirely in the hands of collectors ? The most successful example in this case on the platform is Universal Rayhatcher, where the look comes from the hatching.

It came very progressively, but I ended reusing more and more of the tools I used for my previous collection Clutch :

I really like what I did for this collection. You can read more about it this fx(text) with many technical details very similar to what I ended up doing for the look of this one especially regarding shaders, which is where most of the action happens.

Distressed contours, a good helping of grain, transparency play with holes in the layers evocative of halftone patterns, and only a handful of colors, with the occasional gradient. And a theme, which may or may not be mostly in my head, but it’s there 🙂

In the project, collectors can choose a color palette among a selection which also came about all along production as I tend to add palettes as the fun side activity when I get tired doing the more boring parts. I tried to put a large and diverse selection in there to cater to various tastes. When I started integrating these palettes, I picked the first ones from @Studio Yorktown Color Town Hall and tow of those ended up in the final selection. Many also come from my previous project, especially Clutch.

Different palettes, same seed and design

The blueprint theme came after my first idea which was a very lean bright and colorful UI, more in line with what you see in modern websites, and synergized well with Material UI, which you can see in this video at 0:07 :

– Determinism looks now OK
– High-res exports OK
– More tooltips!
– Palette concept is taking form, with classic 5 colors.

I want to design more blocks, but I have to get to the onboarding/tutorial/help beforehand. Life's hard.#wip #fxhackathon #fxhash #generativeart pic.twitter.com/ajNcbN53gw
— Christophe Choffel (@ULuIQ12) July 19, 2023

After working with it for a couple of days, I found it a bit too neutral and I wanted something a tad more playful, but still serious. I gradually added the blueprint esthetic to convey and reinforce the ideas of design, creativity, drawing and precision. Mostly by the use of the white and blue, lines and dashes but also that chalky graininess.

I lost the use of colors though which can be very helpful as far as UI design goes and had to adapt.

I found a couple of more thematic google fonts for my textual needs :

Cabin Sketch for titling
Edu SA Beginner for everything else

The title “The Incredible Contraption” also got fixed, in reference to the old game The Incredible Machine.

Final UI look, with DOM and context3d layers

You can note the use of drop shadows which is not very thematic, but I found that it lacked a bit of contrast without it.

UI/React

I previously had barely any experience with React so this was a good occasion to take another look at it, and I’m happy I did because it helped tremendously throughout the project as I could quickly implement my UI needs. Both React and Material UI come with comprehensive documentation and examples, so it’s easy to jump in.

That’s not to say it doesn’t have its quirks and jargon that can be a bit confusing at first. I made quite a few mistakes, my implementation is still very much a patchwork of experiments, but it works and it doesn’t send me warnings anymore.

UX/Game design

The main notion I want to communicate with this project is the joy in the act of creation and specifically linked to the chaotic aspect of generative art. You create art, yet you’re at the mercy of the random number generator, and it is beautiful. I wanted to talk to a very wide range of people, from someone who’s never heard of generative art to the artist themselves, and in any age group (ok, maybe not toddlers). I tried to design a toy that is intuitive, and usable even if the user doesn’t read any of the text I wrote except for the “Skip tutorial” button. I used a lot of icons most provided by Material Font, many familiar or very descriptive in themselves. Then the user can just use the simple drag and drop interface and click the big button, and understand how it works. There are tooltips for all the buttons, descriptions, and a comprehensive help file to help clear confusion. I didn’t want any friction in the UI itself.

I ended up with slightly more things on screen that I thought, because there’s other design intentions.

I also wanted to communicate the feel of iterative design, the exploration of a space, the tweaking, and the testing. That’s were the configuration menu for the blocks and all the other little menus get involved. It’s another kind of experience/user, and I hope many will enjoy, but I know is already a big ask, with more time required and way more interactions. You can, if you tweak enough, get a very precise control of what is drawn in the simulation.

The main resource when designing is 2d space : on one hand the grid where you put your blocks, and the other the frame where the contraptions draw.

The grid is fairly large with its 49 spaces, but the adjacency rules and the way positions in the grid influence the physics throw wrenches in your well oiled machine, so you need to space things out, and then you run out of space.

The frame for the artwork and the way it interacts with the contraptions ( wrapping ) and the paint (wrapping or colliding ) makes for an interesting object that you have to take in account when you tune the movement of the contraptions.

I mused for a time to add a “power” or “paint” limited resource, that would have a nice correlation with my vertices budget, but it never gelled into anything more concrete than the progress bar at the bottom of the screen.

The params system and how I imagined encoding all the configuration of the contraption in small integer influenced the fuzziness of the controls for the blocks : all floating point values are standardized in discrete values from 0 to 9 and there are no numbers displayed anywhere, only vague text approximations. I spent a lot of time fine tuning the different values and ranges so that they produce interesting and understandable results while keeping the numbers hidden from the user.

I think it synergizes well with the fact that the end result of the exercise is abstract art and often unpredictable due to the nature of the physics simulation.

There’s something to say about the abstract aspect and the fact that I ask people to create some. The output don’t have to mean anything, but I hope people will manage to create artwork that at least speak to them and that they’ll be proud to display in the overall gallery, and hopefully share their experience with others. It’s one of the reasons I added the customizable title that’s displayed on the blue print and on the overlay in simulation mode.

Code design ?

In large part because I’m working alone on these projects, I tend to write some pretty freeform and hack-y code , and it’s usually perfectly fine as these project are rarely massive and also rarely fully planed.

Here it’s mostly that but I still took the time after prototyping to design/refactor some systems for the grid and the blocks ( called Modules in the the code ), and how they’re represented in the simulation.

We have 49 blocks, which can be empty or have configuration. Configurations are a series of parameters that depends on the type on the type of block. This is the data you see when you look at the blueprint and open the configuration panels for the blocks ( except some parameters are intentionally masked). When the simulation starts, various types of behaviors are create according to this data. The simplest block “Weight” for instance has only one parameter “Mass” and doesn’t do anything other than being created in the physics layer. More complex blocks like the particle spray interact each frame with the physics simulation, create graphic objects, must report how much paint it has left, etc. All in all fairly simple, but thanks to the time I took to standardize here, I could add new blocks and configuration parameters as well as implementing behaviors quickly.

A good chunk of the code is the stuff I do for all project, that you would think I have a good codebase for that I can reuse, but usually end up I customizing each project and try to solve similar problems in a different ways that take me a lot of time :

fx(hash) integration : first I had to rework the params integration to accommodate the updates. That was planned. But I seem to always have trouble with the thumbnail capture (which you can only really test by starting the project minting process in the website) which are entirely self made and that I end up sorting out, but are time consuming.
Resizing / scaling : Seems each project I do follows different rules. Here with the interactions with the DOM and the dynamic zoom, I wrote some rules that work but are really awful code wise, and the whole thing took way longer than it should have.
Performances : always a concern with real-time projects. Ideally I need desktop screen of a reasonable size with a dedicated GPU to back it, but I know that’s far from the most common use case. I can often degrade the quality to gain some performance, but if I do that, is the art still deterministic ? In this project specifically you can easily overload the GPU and/or the CPU, but I kind of circumvented the problem by putting it in the hand of the collectors and justifying it the name of creator freedom. Maybe I’m thinking too much about it because my next release will be a monster on that front.

I put the code on GitHub for anyone to see here. It is what it is : the end result of this iterative process, often messy, prone to use shortcuts and hacks, but also I hope decently readable due the the typing and clear file/variable names.

No tweet found for this url

Things that didn’t make the cut

Sound design : I think the experience would benefit greatly from some sounds, but I feel it’s a big ask for users in the context of fx(hash) to have sounds if it’s not the main attraction. Plus I never really looked into what you can do with browsers/JavaScript and that would have been another “unknown”.
Accessibility and language options: Another thing that would have been nice, but I admit I didn’t really have time to dig into.
Real mobile support : the experience runs on recent phones, and the UI works in vertical, but it’s broken when horizontal. The components are mostly responsive enough that you can probably do the all thing on a phone, but I didn’t test it enough and that’s why a really recommend the desktop use with a mouse.
A more involved onboarding, with a real step by step tutorial. Plus some sort of generative mix between Clippy and Professor Oak to guide you through it. I’m still obsessed with generative humanoids/creatures.
Group selection on the grid, with group move. I realized too late that it could be a need.

Parting words

I’m quite satisfied with where the project ended up. It could have been bigger, but I feel that it would have required working with a team. It’s something I regretted for most of the project : I find all these UI/UX concerns are way better handled when you can bounce ideas around in a team, and the code would have been way nicer if I had to share it with say a React specialist and an illustrator.

I also mused with the idea of organizing some play tests with some of my collectors beforehand. I decided against it for I didn’t really have the time to organize it properly, but it certainly would have been useful.

I hope the community will be as enthusiastic as I am for the project. Even though the future for this kind of projects seems murky (this goes way beyond the usual scope of a gen art project), I really enjoyed developing it. I love UX/game design as a subject, and you can see if you pay attention that even in my previous projects on the platform, most have some sort of creative space for the end-user.

I created this project with the fx(hash) community in mind. Even though I have a very para-social relationship with it and it mostly happens in my head, I know it’s composed of open minded people with a genuine interest in the medium (generative art and its many forms), people who have given me plenty of validation over my time on the platform. This project is for all of you.

The Incredible Contraption is published !

Minting open Wednesday if nothing breaks by then!
Here's a fun trailer to get you pumped :

Project page link below ⬇️#fxhash #fxhackathon #generativeart #creativecoding pic.twitter.com/0qN1hu7yZT
— Christophe Choffel (@ULuIQ12) August 21, 2023

The Incredible Contraption

Project conception and development as an entry for #fxhackathon2023

More content