Ben Falchuk Selected designs and products*

Ben's personal page

*Best viewed on desktop. More info is on my landing page, while my peer-reviewed publications, US patents, and professional profile are also elsewhere. This is an ever-changing set, selected from a very broad range of design and coding work (personal and professional work is intermixed). Each project is briefly summarized (reach out for details). Some sensitive/new work is not shown.

Virtual Reality Underpinnings and Software

I am a graduate of the Udacity/Google VR Nanodegree program which provides comprehensive, practical, and technical underpinnings related to VR software, design, and 360 degree video (samples below).
I have worked with technology partners to develop new Intellectual Properties related to VR underpinnings. These techniques address open challenges in the underpinnings of social VR such as privacy and navigation.
Results: Several US patents pending relating to VR support systems, software, and techniques that provide new modes of situation awareness and privacy within social VR environments. You can see the abstract of our peer-reviewed paper here.
Results: VR coursework results in a series of small to medium sized applications from the ground up, emphasizing various important aspects of interactive VR such as scale, gameplay, and navigation. Implemented on Unity3d C# platform for Google Cardboard (the concepts are broadly applicable).

Various apps built for VR including a "crowded room" simulator that encloses you in a small messy space.

Novel Interaction Technique for Mobile Video Skimming

Description: A suite of designs, algorithms, and human-computer interaction techniques to dramatically improve the UX of mobile video by enabling a skimming phase to precede play-back. By splicing and displaying video content in high-density views (over its entire range, or any temporal segment) the false-positive rate decreases and bandwidth is generally saved.
Result: The tools I designed spliced videos into frames and mapped them atop 2D and spin-able 3D shapes to provide a means or visual search of content and scenes. This technology resulted in 3 US patents, several peer-reviewed publications at esteemed events (e.g., Eurographics), internal R&D, coverage in New Scientist, and a licensing agreement with a NYC-area startup.
Tech: Android, Blackberry, Java, Tomcat, OpenCV.

Skimming video content before streaming it allows for recognition of key scenes and content.

Further illustration of skimming phase using 2d lattices and a 3d shape over which video keyframes were mapped in temporal order.

Sketching, Wire-Framing, Prototyping

In my experiences sketching, wire-framing and prototyping are often an early part of product development. I consider these phases important for establishing conceptual clarity and I am very comfortable with them. Sometimes it plays a minor role only in brief brainstorming meetings while at other times the sketches evolve over more iterations. I have used special-purpose tools like Figma for wire-framing but presently make do mostly with paper and pen, Procreate, PowerPoint, Photoshop, or Visual Studio Code.

Sketches and wireframes (misc.)

Human-AI Collaboration in Video Content Search (via Pose Estimation)

Description: Project Donatello was a unique approach to building search cues for mobile video analytics, making use of pose skeletons. The results were aimed at alleviating some of the challenges of video content search and at reducing video search false-positives, issues that remain even today.
Result: We bundled a computer vision (CV) library into our video segmentation components allowing the tool to highlight body poses extracted from each frame. This work resulted in new understandings of use cases, effectiveness, and technical aspects of delivering machine-learned, pose-extracted, video-skimming. We also focused on the design of a compact, minimalistic, and aesthetically pleasing user interface. Several US Patents and peer-reviewed publications also resulted from the work. You can find one such peer-reviewed paper abstract here.
Tech: We used a frame layout algorithm that fused components of the video segment onto a 3D shape on the UI. Spheres were settled upon, though cuboids were tested. Human pose extraction was achieved using Open Computer Vision (OpenCV) and adapting and advancing work done by Ferrari et al (Oxford Univ.) that employed machine learning to better extract poses from uncontrolled video content.

Leveraging computed body pose estimation ("skeletons") within a novel video skimming application.

Situation Awareness and Visualization

Description: Situation awareness is a cross-cutting need across many US government agencies. In turn, it demands the careful design and implementation of visualization techniques that adapt to the needs of the user while remaining performant and showing "the right information at the right time". I have employed 2D and 3D spatial metaphors, multi-resolution views, dashboards, and other techniques to good effect in products for several agencies. Usages include a) monitoring and visualizing massive DDoS attacks in order to train maneuvering algorithms, b) visualizing webs of strategic undersea cables worldwide, and c) displaying tactical networks in ways that help analysts gain insights.
Result: My R&D and product development in this realm has resulted in significant continued funding with agencies (by delivering innovative tools that meet and exceed requirements), new customers, publications, and internal R&D. Note that some works cannot be disclosed here. You can find a press release here.
Tech: Unity, Google Maps, d3, JavaScript, JQuery, Eclipse, Bootstrap, Google Web Toolkit, SVN, Git, many more.

Geospatial awareness is at times important when physical assets are managed and controlled. Often, supplying the right amount of navigation and search capability is key.

Shifting perspectives can provide value to analysts - here the various logical layers of a network (e.g., VPN, CDN, Link layer) are visually separated into navigable 3d levels.

A part of a sophisticated application designed to monitor and control a test environment which can launch massively scaled DDoS attacks on virtual networks.

Design of summarizations and high-level charts (leading into details with click-through) provides customer/analysts the control they are seeking.

Web Analytics - Designs and Implementations

Description: Redesigned and built our Smart Grid (energy) analytics product with a clean, modern, responsive design. Addressed the many challenges we faced revolving around how to efficiently enable critical insights using compelling graphics and analytics, without getting bogged down in accessing the massive data-sets of almost a billion entities.
This work resulted in DoE funding, US patents, a peer-reviewed publication, internal R&D, and trials at Utilities across the USA.
Tech: Bootstrap, Google maps, KML, d3.js, jQuery, Google Charts, Cesium, Git, VirtualBox, Linux, Django, JavaScript.

Product screens from SecureSmart which Falchuk redesigned from a scattered ad hoc implementation to a consistently simple and clean design using Bootstrap CSS.

Miscellaneous screens of large scale, complex, web apps for analytics and an example of how the ability to shift from 2D to 3D views provides value to analysts to, for example, visualize coverage of mesh network probe devices on various terrains.

Interactive Data Visualization of "Filler Words" in Spoken Word Podcasts

Description: Ever noticed how people sometimes use the words "umm" and "like" while they are speaking? These are called "filler" words. I designed and built a light-hearted dashboard, attempting to get to the bottom of how the word "like" is used in various popular spoken word podcasts.
Result: An interactive dashboard that shows useful and vibrant data visualizations that tell the story of how the word "like" was used across 4-6 episodes of popular podcasts (such as NPR's Fresh Air). You can view it on Amazon Cloudfront.
Tech: Chart.js, JQuery, JavaScript, and more.

Views from analytics dashboard in which a light-hearted - but critical - eye was pointed at how much (or little) the filler word "like" was used in spoken word podcasts.

Interactive Webapp to Teach Tennis Scoring

Description: When my daughter was learning how to play tennis we found that there were not many simple ways for kids to learn the quirky scoring system (e.g., "love", 15, 30, etc.) I tasked myself to design and build a responsive lighthearted webapp to fill this need.
Result: This work was a design challenge hoping to achieve a design that was at once minimal, user-centric, visual and explanatory. It didn't succeed 100%, but it's not bad and I learned quite a bit.
Tech: CSS and JavaScript was used in an effort to make these screens maximally responsive - from iPod's up to large smartphones, tablets, laptops. You can see the experiment on Google Cloud Platform to judge for yourself how well it worked.

Responsive and interactive webapp to teach the quirky way in which tennis games are scored.

Visual Archive for Geo-Referenced Architectural Sketches

Description: A community project called Flash Sketch Mob - in which everyday artists sketch the Main Streets of the town - has had a couple of successful iterations in my own village. I designed and built a webapp to feature the results such that each sketch seems co-located with its actual (architectural) subject.
Result: This work - a collaboration with an activist/artist Bill Batson - resulted in a bunch of local media coverage as well as happy artists, young and old. The website design is simple and pragmatic and allows the artworks be the stars of the show. Small touches (like pin-drops) give it a slightly whimsical feeling, matching the spirit of the event. You can see the webapp here, and one of the press releases here.
Tech: Bootstrap, Google Maps, Leaflet, JavaScript, jQuery, Photoshop, Eclipse, Google App Engine.

Screen from the "Flash Sketch Mob" site in which pins represent user sketches and allow for drilldown and exploration.

Interaction Technique for On-the-Move Media Sharing Policies

Description: Whimsical - but functional - system in support of ad hoc spatial sharing of media. This work was inspired by the (now long defunct) Microsoft Zune model of playlist sharing in which the list of your favorite songs could be shared with other nearby Zune users.
Result: In our twist on this work we designed a system for editing a visual, personal, media-sharing policy to constrain the spatial conditions under which your media will be shared with nearby devices while on-the-go. The work proved both provocative and valid. We used the visual metaphor of a radar screen (with color coding) to help users understand the geometry of movement and their policy. The work resulted in internal R&D funding, and a peer-reviewed research paper at a conference on ambient media. The notion of ad hoc media sharing is nowadays embodied by technologies such as AirPlay.
Tech: Java, Windows phones, mathematical simulation software, bash scripting.

A visual editor for on-the-move media sharing; although the platform is obsolete the principles are valid.

Web Dioramas and Storytelling

Description: I am interested in how web technologies can create the feeling of immersion in various useful ways. I have experimented with web technologies that - while slightly flatter than VR - nonetheless add new life to otherwise static imagery.
Result: One quick experiment resulted in a set of web dioramas featuring my photo-illustrations, a parallax effect, text snippets from Shakespeare sonnets, and creative layering to create something more than the sum of its parts - almost poetic. Find it on Google Cloud at https://parallaxscenes.appspot.com
Result: Another small experiment resulted in a responsive website that conveys some back-story to an iconic photograph: Eisenstaedt's "V-J Day Kiss" Life magazine cover from 1945. It uses a minimalist web design and interaction and shows users a familiar image in a new way. Experience it yourself on Google Cloud at https://entervjday.appspot.com
Tech: Javascript, HTML, JQuery, parallax, Photoshop, Ken Burns effect, type ahead effect, more.

Interactive web vignettes.

Virtual Reality (VR) Bird Garden

Description: My neighbor Gene has had a distinguished life as an outdoorsman and bird watcher. His watercolor paintings are so vibrant and accurate that I wanted to situate some of them in something more than a simple lightbox.
Result: A WebVR site that immerses Gene's watercolors into a 3D cloister scene with waypoints allowing users to move about the tranquil space. Sounds and visual effects make the experience quite Zen-like, and Gene's paintings appear therein, nestled into various parts of the quiet scene. See it on Amazon S3.
Tech: JavaScript, WebVR, Three.js, Unity3D, Photoshop, CSS3, jQuery

Views from within the immersive bird garden.

3D Interactive Tweet-storm Visualization

Description: The President of the United States (POTUS) tweets a lot! But what are his main topics and when did he tweet them? This whimsical, web-based, interactive, data visualization focuses on providing such an analysis.
Results: a situated, novel, cartoon-like, data visualization that shows information in a statistically correct (and fun) manner. The site creates a 3D scene in which the POTUS tweets from any timeframe can be "replayed" using the interactive user interface. In the scene, tweets are represented as blue orbs that tumble down Mount Rushmore in direct proportion to the actual numbers of tweets made in the chosen time-frame. Tweet counts and common words and terms in the POTUS tweets are called out using simple graphics and charts. See it on Amazon S3.
Tech: WebGL, Unity 3D, Shaders, Three.js, jQuery, JavaScript, CSS, Photoshop, JSON, data visualization, many more..

Exemplary moments during tweet-storm visualization.

Interactive Dashboard for US Power Utilities

Description: Power Utilities face the daunting task of managing critical infrastructure that is constantly under threat from cyberattackers and misconfigurations. This work was a serious interactive tool to allow Utility personnel to envision and manage their Operational assets, such as Relays, Breakers, Substations, and High voltage lines. Designed for - and tested at - DARPA RADICS Exercises at the Plum Island testing facility.
Results: Starting with the power one-line diagram with which Utility personnel are familiar, designed and built a series of data-driven views into the assets from many perspectives, such as: correlation of power anomalies to equipment, comparing telemetry between times and devices, and viewing suspicious cyber emissions. Provided an aggregated summarized view of statuses from many additional backend services, each with a special capability of its own in the Utility setting. This tool succeed well at Exercises and is packaged as a commercial offering to US Utilities. A press release can be found here.
Tech: JavaScript, HTML, JSON, AJAX, Python, MySql, WebSockets

Distilling very complex data into meaningful views for Utility operators.

Rock God Pose - Fun Webapp Built on TensorFlow & PoseNet

Description: A fun web experiment in which you are challenged to strike a pose to match that of several famous "rock gods" of years past like Bowie and Mick Jagger. This app focuses on creating a usable and fun interaction with the webcam and leverages machine learning to extract, label, and match poses.
Results: A vibrant web app that is both fun and challenging. If you love classic rock you will probably smile when you see Rock God Pose. See it on Amazon S3.
Tech: Bootstrap, HTML5, CSS, JavaScript, TensorFlow, ML5, PoseNet

Exemplary screens from Rock God Pose in which the user must strike various rock star-like poses.

System-of-Systems Security Front-End with Human-readable Evidence

Description: How can AI and user interface come together to provide insights to human users at just the right level? In this product built for the DARPA ConSec program a backend neural network and optimizer compute optimal configurations for enclaves, given user objectives and constraints. Information about these derived configurations must be shown efficiently and at a level no higher than the skillset of the targeted technicians.
Results: Designed a human-AI application to serve as a proxy between the human and AI backend, to gather user objectives, and to allow configuration space exploration, trade-off analysis and what-if's (i.e., How does changing setting B to value X affect security?). Information and evidence for optimal networks is presented in a series of clear steps and panels, with no more complexity than, say, a banking dashboard. This is important since the configuration space is incredibly large. A variety of clear and simple techniques are used, such as bar-charts, visualization of textual differences, and the small multiples technique, all in the name of allowing rapid comparison between candidate configurations and evidence. A press release can be found here.
Tech: Flask, MySQL, HTTP, JSON, PyTorch, JavaScript, CSS, Chart.js, Bootstrap, IBM CPLEX

Screens from the human interface that sits between an AI-based backend and a network administrator.

"Neo Vs The Mask" - A Hand Tracking Game

Description: How can a webcam based hand-tracking capability be transformed into an engaging web game with a fun user experience? This experiment set out to find out.
Results: Designed and implemented a simple interactive webapp that challenges users to try to dispel bullets heading for Neo by controlling Neo's hand with their own. This experiment is quirky but nonetheless surmounted many mini-hurdles to come together as a whole. It features a distinct visual aesthetic that helps create a Matrix-like mood. Play "Neo Vs The Mask" now on Azure Cloud.
Tech: HTML, CSS, JavaScript, Hand Tracking, Tensorflow.js, AI, physics engine, webcam, Photoshop.

Gameplay from "Neo Vs The Mask"

Development Kit for High Performance Computing

Screens from the Peraton Labs "FLEET" HPC software offering

Description: DARPA FastNIC targeted orders of magnitude improvements in High Performance Computing (HPC) applications (such as deep learning) through advances in configurations of storage resources and optical NIC’s. Falchuk contributed to the Simulation (SIM) sub-team which focused on the modeling, specification, execution, and true-ups of simulated HPC program loads, with a goal of predicting time and energy usage.
Results: an IDE for HPC engineers in support of common tasks: (1) Task Graph Viewer and editor – a tool to visualize the HPC flow being simulated; it expands a JSON format into an interactive directed graph with point-and-click capabilities. (2) Target Platform Viewer – a tool to help users view, express, and edit the simulated HPC platform of their choice, using the visual metaphors of nodes, edges and containers. (3) Simulation monitor – tracks the progress of running simulations and visualizes metrics. (4) GUI-based HPC data experiment editor/creator – a tool to support creation of HPC models through simple point and click interactions with a GUI that stewards access to a constraints DB library from which experiments are built. (5) HPC planned task playout tool - highlights HPC resources and tasks as they run in simulated execution, and also time and energy usage.
Tech: Python, JavaScript, Chart.js, Cytoscape.js, web workers and broadcast, Flask, HTTP, JSON, MySQL DB, Mongo DB, shell scripts, GitLab.

Human-AI Collaborative Identification of Circuit Purpose

Integration with a wide array of AI functionalities such as extractive summarization, Large language models (chat), circuit part detection and labelling, as well as 3d graphics to convey circuit functionalities with reference to a vast set of organized categories (or "experts").

Description: This product addresses the challenges posed by the DARPA REPO call, in which human non-SME Analysts are generally overwhelmed by the task of determining the purpose of scanned electronic components and circuits; E.g., "is this a circuit generally found in radar systems, GPS, or in weapons?" An AI system is desired which ingests, processes, and provides the human with the right kind of tools and insights to make important decisions about circuit classification and next steps.
Results: Designed and implemented a suite of visual tools intended for use in an interactive session with an Analyst. Integrates with scores of backend AI functionalities (built by our dev-team), shielding the Analyst from AI complexity but providing simple and clear access to the power of trained Neural Networks, AI-based image segmentation, image labelling, text summarization, and LLM chats, all for the purpose of circuit "purpose detection". Includes the ability to classify text (found in the "mystery" component diagrams) into branches of a large hierarchical tree of Human Experts and teams, with 3d visualizations of same. Software delivered to DARPA for potential transition to real-world use cases (likely classified).
Tech: Python, Javascript, Three.js, Docker, Docker compose, Jupyter Notebooks, Machine learning, Neural networks, Siamese networks, graphs, HTTP, JSON, shell scripts, GitLab.

..that's all for now.

Back to Ben's personal page

Email Ben