During 2007, I’ve been doing some very interesting work with “visual” programming environments as part of my digital interface design degree I’m taking at Senac São Paulo. In the first semester of that year I had my first contact with Max/MSP, and then jumped to Max/Jitter. It was the first time I’ve that kind of “programming” platform, and despite some initial frustration, it turned out to be pretty in in the end. My main project that semester was an image transformation presentation that distorted camera input and changed its color to try and create abstract Rorschach-like ink blots – I called it Rorschachoscope. It was very crude, but an interesting development nonetheless.

The following semester I continued my research on this area, but using another, more powerful (and free) image synthesis platform, called vvvv. That program borrows a bit from the standard Max/MSP approach to visual programming, but does things a bit differently in some important aspects. It was more or less love at first sight, and I really enjoyed using the program. With it, I not only learned a lot more about image synthesis (through Pixel Shaders), but also created a new version of the same project I built the previous semester, calling the new one Rorschachoscope 2. You can see it in action here.

More importantly, after using those programs – and also testing Puredata and Processing, as well as unusual input devices – I got a lot of ideas in my head. While the field is already well served with many powerful platforms for real time multimedia presentations – vvvv and Processing being my favorites – I started to think how nice would be to develop an environment like that in Adobe Flash (which is the technology I work with in my commercial work). The bad news was, it wouldn’t compare with what much of what the existing platforms can do: since Flash is an environment with a much higher level of management (which is a bad thing), it is not able to exploit the machine as much as a native application can.

But on the other hand, it could work pretty well in terms of distribution – one wouldn’t even need to install the program, since it can run from the browser on Windows, Macintosh, and Linux systems, making it an online application. An additional downloadable, standalone version could be made available with Air, Adobe’s standalone cross-platform application environment. I really liked that idea, and let it bounce for a few months in my head while I finished that last semester of 2007.

But this is not just a story of an interesting idea. It’s also the story of a difficult decision.

The year of 2008 would be the last year in my bachelor degree — 4 years of an interface design course. In that last year, students must create one big project that encompasses what they’ve learned, and what they seek to learn and work with in the future.

When I started my degree back in 2005, I already had my last project decided: it would be a markup language for the definition of graphical user interfaces for games – I call it XUG. It is something pretty complex, pretty serious, and from my point of view, a great solution to a common problem.

While I haven’t done much of the real work in the subject due to lack of time (and also because we’re supposed to work with just in the last year of college), I’ve spent a good part of the free time I had in those first three years of my degree doing research on the subject. I’ve decided on various aspects of my approach to graphical user interface declaration, researching with many alternatives such as CSS, HTML, XUL, XAML and others in the process, and I believe I was into something pretty nice. I’ve built a list of preliminary keywords used by the markup language, started the language documentation, created the definition of modules of implementation, and gathered a huge number of reference documents and papers. And while the project goal was the definition of the markup language, I even developed some test renderers using ActionScript 3 and C# (running under XNA) so I could test the new units I created for the standard and how well they performed on near real-world situations of interface development.

In the end of 2007, however, I started to realize that project was too big. Not that much terms of development – I was confident I could do it – but that it was so complex that not many people would even understand it, or its purpose. It was too specific, and too technical; even teachers I talked to had trouble understanding the concept. One would need a lot of knowledge of both game development and interface development to understand why something like that was needed in the first place.

At the same time, the revelation I had working on graphical programming languages such as Max and vvvv led me to a fun, new way to create multimedia presentations. All of a sudden, image and audio synthesis and analysis became something I was deeply in love with. Also important was the fact that it had a lot more to do with my graduation course than XUG, so it would fit well into my last degree work.

It was just a few days before 2008 started that I decided to push that idea of a visual programming environment in Flash ahead and put XUG on hold. This was not an easy decision – I’ve already spent 3 years thinking on every aspects of the development of XUG, and a lot of the solutions I’ve managed to come up with really made me pleased. It also meant a personal defeat of sorts for me, as if it meant I couldn’t keep my mind on the same thing for too long.

However, I trust this was for the better. The main idea in Fnk is fun, it fits a lot better into the definition of my degree, and it’s even easier for people to understand once they see it working. More importantly, I’m putting the same passion I’ve invested in XUG for these past three years in Fnk, and I’m confident I can do something pretty cool in the end.