A visitor walks beside a media wall and the image shifts nearby. The connection can feel simple, even when the sensing system is working through changing light, people crossing in front of one another and an oblique view of the space.
The design begins with what the artwork needs to respond to. Position, direction, proximity and time spent in a zone describe different events. Each asks something different of the sensing system.
From a detection to a movement
A detection describes an observation at a particular moment. Tracking connects observations over time, so the system can follow movement and maintain continuity through brief interruptions.
If a tracked position jumps, the image may jump with it. Filtering can reduce small fluctuations, but it also affects how quickly a change reaches the artwork. A slow fluid field may accept a more gradual signal than a direct gesture.
The required balance follows the visual behaviour. It should be established before choosing processing settings or promising a particular response time.
Let the site inform the sensing
Camera position and field of view determine what can be observed. A low viewpoint may be blocked by visitors; an angled view changes the relationship between movement in the room and movement in the image.
Light, reflective materials, circulation and crowd density also affect the scene. A useful study identifies the relevant zones, likely paths and available mounting positions, then tests representative conditions. The resulting evidence defines the operating range for that installation.
Design what happens when input is uncertain
People overlap and observations become incomplete. The artwork needs a deliberate response when confidence changes. Depending on its character, an influence might soften, hold briefly or ease back into an autonomous state.
Combining sensors can provide complementary evidence where the brief warrants it. Each input should have a defined role, with a clear behaviour if that input disappears. More sensing brings additional calibration and integration work, so its purpose needs to be specific.
Nespresso New York
For the flagship’s generative wall, Refractiv developed a sensing layer combining computer vision and multiple LiDARs. Movement and proximity become forces within a coffee-and-cream fluid simulation.
The artwork continues moving on its own. A nearby visitor gains local influence, producing ripples, trails and wakes that merge back into the wider composition.
Refractiv’s scope included the interactive system, hardware specification, implementation and calibration. Commersive Solutions commissioned the work and led the project and AV integration. Calibration connected the sensing coordinates to the installed wall and store circulation.
Define the useful signal
Before choosing a sensor, define the event the work needs to understand, how quickly it should respond and what happens when observations are incomplete. Those decisions give hardware selection and testing a concrete purpose.
For a sensing study, share the intended response and what is known about the site. Contact Refractiv to discuss the brief.