Posts

Autoplaying video previews for the web

If a picture is worth a thousand words, and a video is made of several pictures in a sequence, then a video is worth several thousand words.
A great way to show a preview of online content is by using video thumbnails – an ambiguous term that can both refer to a static image, usually depicting one frame of the video (also known as poster image), or, more interestingly, a video preview that’s a smaller and shorter version of the referenced content.

If you arrived to this page through the homepage of my website, you might have noticed that every post on it features an animated thumbnail. It was a natural choice since most of my work on display is some form of animated media, or at least accompanied by video assets.
For a long time my thumbnails have been simple animated GIFs – one of those ancient formats that managed to maintain popularity for longer than expected – but there now is a good selection of modern, more efficient formats to display motion content on the web.

WebM for the win

I researched a few options: animated PNG (APNG), WebM, animated WebP, MP4. After evaluating the pros and cons of each container, I finally settled for WebM
because of the smallest file size (for a quality that’s visually similar to other solutions tested), a wider browser support, and because – I’m betting my money on it – it’s a future-proof file format. (I’m joking. I never bet any money.)
WebM is an open-source container format, while the actual video encoding format I am using is VP9, which has very good browser support even across the mobile and iOS landscape.

I would have loved to lazily set all videos to play on load and in a loop, but the state of the technology told me we we’re not there yet.
Having many looping videos playing simultaneously is resource-intensive. For this reason if on a page there are multiple videos with the autoplay attribute, most devices will let only the first one play automatically.
So I had to come up with my own method to dynamically control which video is playing, based on how far the user has scrolled the page. I do this in JavaScript. An additional benefit of controlling video with scripting is the ability to only load a resource as the user is about to display it (which saves my bandwitdh and users’ metered data). There’s also room for a fallback poster.

Generating WebM files

My editing video software can output WebM, but in case yours doesn’t, you can use a PNG sequence as intermediary format, or an MP4 video with lossless or near-lossless compression settings. Then you’d batch transcode your source videos with the open-source tool ffmpeg and the following command:

for f in *.mp4; do
echo "Processing file $f."
ffmpeg -i "$f" -c:v libvpx-vp9 -b:v 500K -pass 1 -an -f null /dev/null && \
ffmpeg -i "$f" -c:v libvpx-vp9 -b:v 500K -pass 2 -an "$f".webm
done

This is what I’d use instead to convert a PNG image sequence to WebM:

ffmpeg -i '%.png' -pix_fmt yuv420p -r 24 -c:v libvpx-vp9 -b:v 500K -pass 1 -an -f null /dev/null && ffmpeg -i '%.png' -pix_fmt yuv420p -r 24 -c:v libvpx-vp9 -b:v 500K -pass 2 -an output.webm

Note the need to perform two encoding passes, and the use of the flag “an” (audio=none) to suppress any audio channel.

The JavaScript code

This is the code that does all the magic. Just include it on a page with video elements and it will automatically control their playback. The first video in the DOM will play immediately. As new ones become fully visible and start playing, one at a time, previous ones will stop.

var videos
var currentVideo

// Callback function for the window scroll event (called through debounce() to
// limit the frequency of calls)
function onScroll(){
    let _selectedVideo = videos[0]
    let _last = -1
    for (const [_index, _video] of videos.entries()) {
        if(isVisible(_video))
            loadOnce(_video)
        if(isVisible(_video, true)){ // if the second parameter is true, a video needs to be fully visible for the function to return true
            if(_index>_last){
                _last = _index
                _selectedVideo = _video
            }
        }
    }
    if(currentVideo != _selectedVideo){
        currentVideo = _selectedVideo
        playOneVideo(_selectedVideo)
    }
}

// Lazy-load a video 
function loadOnce(video) {
    if(video && video.dataset.source){
        video.setAttribute("src",video.dataset.source)
        video.load()
        delete video.dataset.source
    }
}

// Play a video after pausing all other videos
function playOneVideo(video) {
    if(video && video.readyState === 4){
        // Pause all videos first
        for (let _video of videos) {
            _video.pause()
        }
        video.play()
    }
}

// Attach to the window as soon as the DOM is fully loaded
window.addEventListener('load',
    function() {
        videos = document.querySelectorAll('video')

        // Play the first video
        if(videos.length>0){
            currentVideo = videos[0]
            loadOnce(currentVideo)
            currentVideo.oncanplay = function(event) {
                playOneVideo(event.target)
            }
        }

        window.addEventListener('scroll', debounce(onScroll))
    }, false)

// Check if an element is visible in the viewport
function isVisible(elem, fullyVisible) {
    let bounds = elem.getBoundingClientRect()
    let windowHeight = document.documentElement.clientHeight
    let isTopVisible = bounds.top > 0 && bounds.top < windowHeight
    let isBottomVisible = bounds.bottom < windowHeight && bounds.bottom > 0
    if(fullyVisible) {
        return isTopVisible && isBottomVisible
    }
    return isTopVisible || isBottomVisible
}

// Limit the amount of calls to a callback function.
// From https://gist.github.com/Sidd27/daa8c600694ee99b62daabcba0af85cb#file-debounce-js
function debounce(func, wait, immediate) {
	var timeout
	return function() {
		var context = this, args = arguments
		var later = function() {
			timeout = null
			if (!immediate) func.apply(context, args)
		}
		var callNow = immediate && !timeout
		clearTimeout(timeout)
		timeout = setTimeout(later, wait)
		if (callNow) func.apply(context, args)
	}
}

That’s it! Now everything moves, but only when we need it to.

Sources:
https://trac.ffmpeg.org/wiki/Encode/VP9
https://corydowdy.com/blog/apng-vs-webp-vs-gif

Portami

Portable USB Midi synth with onboard speaker and audio out

Portami

See it in action: YouTube video

Portami is a portable, battery-powered polyphonic synth built around a SAMD21 microcontroller. It’s designed to be a companion to small Midi keyboards.

The software is a fork of Marcel Licence’s excellent code.

My implementation focuses on USB Midi communication (Midi-in through the onboard USB-C port) and features a push button to cycle between wave types (sine, saw, square, pulse, triangle, and noise). An RGB led inside the button changes color according to the current wave type.

It should work out-of-the-box with basic keyboard controllers, like the AKAI LPK25 in the picture. Knobs can be mapped to change synth parameters like modulation speed or pitch bend range. I tested it with an AKAI MPK Mini Play, and a keyboard mapping for this specific controller is provided. There also is a debugging tool (SAMD21synthDebugger) that can be used to find the address of knobs and sliders of more controllers.

Other than the onboard speaker driven by an LM386 amplifier, Portami has a 3.5″ mini-jack port to plug a pair of headphones. The audio socket has a switching mechanism, disconnecting the speaker when in use.

The whole setup is powered by a 3.7V, 1100mAh li-ion battery, recharged and protected by a TP4056 USB-C charger module. A regulator boosts voltage to 5V, providing power to the Midi keyboard (you might want to check current requirements for your specific setup, or power the keyboard externally).

The word portami in Italian means “take me” (to a location). As in “portami con te” (take me with you).

Source code available at github.com/TuriSc/portami

Voice changer toy

Lo-fi sampler based on the ISD1820 IC and module.

Voice recorder with pitch/speed modulation.
Press one button to start recording the module’s onboard microphone. The other button plays back the sample.
Turn the potentiometer to increase or decrease pitch and speed. You can sound like a chipmunk or a monster in a cave!
Recordings can be about 10″ long. Powered by a rechargeable lithium-ion 550mAh battery.

Internationalized ESP8266 Weather Station

A device that acts as a weather station, displaying real-time weather and astronomy data, forecasts and room temperature.

The original design and code of this project has to be credited to ThingPulse.
I reworked an existing fork of that code, the German version by Mario Ammerschuber.
Mario’s optimizations included, among other things, a captive portal Wifi manager, which is great to avoid hardcoding a Wifi name and password and therefore making the device more portable.

My code has been tested on two different development boards (A NodeMCU and a D1 mini)

My contribution is focused on internationalization: I created the Italian version, and speakers of a different language can easily localize all the displayed text by editing a single list of strings.
I also set up the weather station to use a push-button instead of touchscreen controls, and added a DS18B20 Temperature sensor to display internal temperature alongside the weather data pulled live from OpenWeatherMap.

If you want to build one, you can find the source code and more development notes here: github.com/TuriSc/esp8266-weather-station-color-i18n

Grimmboy – Arduino-controlled kids’ audio player

My toddler niece is too young to read but always in for a good story. So I created for her a portable audio player of children stories.
It’s got a cool retro vibe for the enclosure and it’s very easy to operate: you just choose one of the many ‘tapes’ from its drawer and place it on top of the player to start playing that audiobook track.

It’s multilingual! I managed to find audio recordings of the same story in multiple languages, so for most stories there are four different tapes, one for each language. I believe this could be a good language learning device.

This project was inspired by the open source project Tonuino, from which I also borrowed the idea of using NFC tags to trigger an Arduino-controlled mp3 player. My player has fewer functions but is much easier to set up, is battery-powered, and uses a custom data format for the tags so they can be configured using a smartphone.

Inside the enclosure, the Grimmboy circuit uses an mp3 player chip controlled by an Arduino Nano that’s also connected to an RFID reader.
Each tape has an NFC tag hidden inside, while all the audio tracks are stored on a microSD card.
On the top side of Grimmboy there’s a pause/unpause button, an LED that blinks when no track is playing, and a volume knob.

The lithium-ion battery is recharged using a common USB-C smartphone charger, and is protected by a circuit that prevents overcharging and overdischarging.
Even though the mp3 player supports stereo playback, for energy saving reasons I chose to make it mono and include only one speaker.

The cassette tapes contain a minuscule radio chip (Ntag213, aka MiFare Ultralight C), writable using a common tag editing app for Android. However, since I created over a hundred cards I automated the job and coded a batch-writing tool (whose source code is available, see link below).
Each tag is sandwiched between two printed pieces of paper and then laminated into a microcassette-card. Side A has the story title, cover artwork, and a language flag; on side B I wrote the running time and the tag ID (which I kept note of so that if a tag gets lost I can recreate it).

One function I did not implement is automatic poweroff. It would require a power latching circuit or a more modern microcontroller but I’ll leave that to a future version.

If you want to build your own Grimmboy you can find the source code and more development notes here: github.com/TuriSc/grimmboy

This project was featured on the official Arduino blog and on Hackaday.

▶ Things that Move

– So, what do you do for a living?
– I move things.
– Like what? Do you move furniture? Walk animals, drive cars? Do you juggle balls, rotate stars, fly paper planes?
– Yes.

A collection of my latest CGI experiments, with examples of physics simulation, material design, motion tracking and match-lighting. They’re all personal projects, as I did not include commissions or commercial work.
The music track was in part performed on a new digital/analog hybrid synthesizer I’m putting together.

TS-TAP4 – tactile Arduino drum machine with looper

As a bass player sometimes I want a backing drum track for music practice, so I built myself this little drum sequencer, a unique touch-sensitive drum machine.
I just need to tap a rhythm on its touch sensors and it will start looping my sequence right away.

This project is based on an Arduino Nano/Arduino Uno. The sound samples are unsigned 8 bit 8000 Hz WAV files that have been converted to header files in order to be compiled with the rest of the code.
The touch interface works via capacitive sensing, there’s an Arduino library just for that. One of the drawbacks of using capacitive touch sensors with a battery-powered circuit is that grounding is unreliable. For this reason I surrounded the device with two rails of conductive copper tape, so that holding the device will naturally provide some grounding through the user’s hands.

An LM386 power amplifier in a gain=200 configuration is used to drive the 8Ω speaker.
The whole circuit is powered by a single rechargeable 18650 cell connected to a TP4056 charger module with USB input, which delivers a steady output at 5V.
I fitted everything inside a wooden trinket box I bought specifically for this purpose, as it looked like a drum to begin with. And I love when wood and electronics get together.

Download the source code for this project: github.com/TuriSc/TS-TAP4

TS-DET1 – micro:bit keypad synth with detuning

It feels rewarding to use a device by leveraging its unique traits, and the BBC micro:bit has two features that are rarely found onboard other microcontrollers: a LED matrix and gyro sensor.
So, I decided to design a circuit that detects and displays tilting, and uses it to alter the pitch of the notes being played.
The result is a quirky musical toy, surprisingly expressive and with a personality of its own.

The input part of the instrument is a 5×5 matrix of tactile push buttons, each one paired with a diode to prevent ghosting (ghosting as in false input. Diodes won’t solve unrequited love issues.)
Some of the pins used as key inputs are shared with the LED matrix, requiring to continuously switch the display off and back on, which results in a noticeable flicker.
I connected the audio signal output of the micro:bit to a quad op-amp to drive two small speakers. If you’re building your own TS-DET1, you could use a single 8Ω speaker and a power op-amp, like the LM386.

I did some research and I decided to add a passive low pass filter before the amplification stage. It’s a way to soften the harshness of the pure square waves, but I also did it because I know that children are more sensitive to high frequencies than adults, and do not like beeps that are too high-pitched.

Comparison of two recordings, one with and one without the low pass filter

The whole circuit is powered by two AAA batteries and is fitted into a plastic case for playing cards.

The prototype in these pictures is now in the small hands of its new owner, and I do not plan to create new iterations. However, if I were to do it again, I would try to skip micropython and micro:bit’s music module, trying to use C instead to set my own timers to control the PWM and produce the different notes.
Also, the two unused tilt axes could be used to control volume and octave shift.

Download the source code for this project: github.com/TuriSc/TS-DET1

LED dice – LED runner with LDR and sound

LED runners are circuits where LEDs turn on one at a time in a sequence.
I made one with a twist, turning it into an electronic dice: you press the button to roll and release it to get a number.
There’s also a speaker that emits a ticking sound and a light dependent resistor to control the speed of the run. It’s very slow in a dark room, and extremely fast in the sunlight.

The signal originates from a 555 timer, whose output triggers a CD4017 decade counter (and the speaker, in parallel).
As an enclosure I used a wooden domino box, and the little animal figures you see in the pictures are erasers, just for decoration.
Rather than from one to six, the possible outcomes of my circuit range from zero to five. If you are building this and want a 6th LED, just connect it to the counter’s Q5 pin, and leave Q6 connected to RESET.
The LEDs can share a single 470k resistor since they’re lit only one at a time.

For a future version of this project I would use a capacitor to ease out the roll after the push button is released.

Lost in the Woulds

I don’t play video games anymore, but I’ve always been fascinated by their hybrid nature, crossing artistic expression and a constant push for technical evolution.
When I was a kid and the Commodore 64 was popular, game making had just become accessible to everyone.
Now that the video game industry is larger than all the other entertainment industries combined, games contain universes, and spending time making a small game might feel pointless. However, these days ‘indie’ game development is thriving, its community is very active, and the resources required do make a game by yourself are freely available online.
I can make things move on a screen, tell the computer what to do, make it beep.
And I had a story to tell. A journey that’s emotional, profound, dramatic, and darkly mysterious.

Continue reading Lost in the Woulds

Picbag

I worked on the UX/UI and brand identity for Picbag, a visual organization app at prototype stage.

Picbag lets users collect inspiration, intelligently tag and sort photos and image snippets, and create moodboards they can share.

Continue reading Picbag

Taormina Jazz Festival

Every summer since 2010 my family and I, together with the cultural association Taormina Jazz, have been organizing the Taormina Jazz Festival.

Of the artists presented, some play traditional jazz, while others dedicated their musical research to more experimental forms.
We’ve had David Kikoski, Ed Howard and Nasheet Waits,
Kurt Elling, Charlie Hunter and Derrek Phillips, Kenny Werner, The Bad Plus, and many others.

I curated part of PR and the brand identity of the festival and produced several posters, program booklets and promotional videos.

Material design illustrations

I designed a large set of illustrations for the CIPD Profession Map.

This project was the perfect opportunity to represent the many diversities of inclusive workplaces, while challenging common stereotypes regarding gender, ethnicity, age, physical abilities.

We went beyond the avoidance of preconceptions to appeal to a global audience – it was a conscious use of illustration as an unbiased brand language to speak for the professional values promoted by the campaign.

The people I was asked to draw are not just neutral models. They are active characters, interacting with each other, often displaying a hierarchy or relationship of power.

Procedural Circular Animations

I always found generative art mesmerizing, so I experimented with a few JavaScript rendering libraries to create something of my own.

Click on the images to launch the real-time animations

The first one is created with the impressive rendering library PIXI.js. The other three are drawn with Two.js, which defaults to a SVG output.

Earboard – Web App

Earboard is an ear trainer for musicians whose purpose is to generate note intervals to play along with. Its layout is modeled on the fretboards and standard tuning of stringed instruments like guitar and bass.

See it in action here: Earboard

It’s a responsive JavaScript web app. Source code

Six Capitals

Illustrations of the six capitals observed by Integrated Reporting: Financial, Manufactured, Intellectual, Human, Social and relationship, Natural.
The report they appear in is part of the Valuing your Talent research conducted by the CIPD and CIMA.

▶ LMO – Motion Graphics

Art direction and animation of a teaser video for an upcoming survey report by the CIPD, which inspired the space observation metaphor.
Animated in After Effects with a lot of custom expressions.

Health and Well-being – Social Media Graphics

Art direction, illustration and animation of a motion graphics video and related engagement media for a research report by CIPD on how HR can develop organisations with well-being at their core for happier, healthier, sustainable business.

▶ Valuing your Talent – Motion Graphics

Art direction, illustration and animation of the teaser video and engagement media for Valuing your Talent – a collaborative, industry-led movement helping organizations realize the full potential of their workforce through understanding and measuring the impact and contribution of people to business performance.
Motion graphics. Part of a larger series of animated creatives.

HTML5 interactive banners

I worked a few months for Tag Worldwide producing interactive and animated HTML5 banners for Nike, Sony, Jameson Irish Whiskey, HTC, Argos, Boots, Amazon.
Flash was cool, but the 2000s are over.

AXE/Lynx ads

I designed and animated a series of rich media creatives for AXE/Lynx, for digital agency AddictivePixel at BBH, London.