Jacob D. Pennock

Engineer, Experience Designer, Futurist, Game Developer

Interests include emerging human computer interaction techniques, close range hand tracking, computer graphics, gesture and computer vision based interaction systems, gestural gaming systems.


Lead Systems Engineer

Experimental Design, Los Angeles, CA

Apr 2019 - Present

Currently leading the development efforts of Alex McDowell's holistic world building, production design, and interactive experience studio.

Lead Developer

Sunken Places, Los Angeles, CA

Aug 2018 - Present

During my nights and weekends I help this small studio try and finish their Kickstarter funded VR title Classroom Aquatic.

Senior Developer – Unity AR/VR

[email protected], San Mateo, CA

Nov 2017 - Jun 2018

I was contracted through DigitalFish to work inside Facebook building 8 on the Unity SDK for an unannounced AR platform.

Senior Manager Interactive Systems

BYTON, Santa Clara, CA

Mar 2017 - June 2017

I managed and built the test bed and prototyping framework for this car manufacturer's infotainment suite. As well as designed and prototyped much of their gestural interaction paradigm.

Studio Head

Livid Interactive, San Francisco CA

June 2014 - Feb 2017

Livid was formed as a joint venture between myself and Helios interactive partially funded by myself and Intel with the goal of producing consumer games for PC and consoles. Our first tile Head of the Order saw an arcade release as an Intel promotion inside 60 Best Buy locations across the US. Our second title Shatter Might was canceled after about 12 months of development.

Senior Creative Developer

Helios Interactive Technologies, San Francisco CA

Jan 2013 - Feb 2017

Helios Interactive specializes in creating interactive art installations and interactive marketing experiences for major brands.

Throughout my employment at Helios I was part of the development and launch of over 20 games and interactive experiences. The majority of which I was the principal developer and or designer. On others I filled many roles including Tech Director, Game Designer, Producer, and Out Sourcing manger.

At Helios I've produced everything from large scale permanent installations to short term event based attractions. I've created content for a large array of technologies including; close and long range gesture systems, large touch walls to small mobile games, facetracking and augmented reality, giant 4k video walls and crazy multi-projector setups.

Lead Developer

Unicorn Forest Games, Oakland CA

Aug 2011 - Jan 2013

Unicorn Forest was the first game studio my ex-wife and I founded and ran together. We provided a wide range of freelance game development services to other development houses and have helped other game studios realize their vision. Particularly in the area of visual fx design, but also, art, animation, UI and character design, as well as general game play programming and development.


Unity Game development and design

I'm a unity expert. It has been my main professional development and content creation platform for almost 7 years now. I know many of it's subsystems and api calls like the back of my hand. I'm very active in its developer community.

Gesture, Touch, and Motion experience

I have worked extensively with many of the newly emerging human computer interaction devices and their SDKs

In the long range gesture space I have experience with the Kinect, both the microsoft SDK and primesense openNI. I've also worked with less common sensors like the Panasonic D-IMager and custom tracking SDKs

In the close range gesture space I have worked with the DS-325 and the softkinetic iisu platform for a number of applications. I've also worked with the Creative Senz3d camera and the Intel perceptual computing platform, as well as the leap motion device and it's SDK.

In the touch space I have built a number of large scale touch apps that receive TUIO, as well as have worked on custom trackers that output TUIO. I've also worked with iOS and android mobile and their respective touch apis.

I've also worked with the Sony Move and Nintendo wii game controllers.

Research programming expertise

Using C/C++/C# for the research and development of advanced real-time computer vision applications associated with, image and video processing, object tracking, gesture recognition, and real time patten matching.

Specific skills

Primarily develop in C# and C/C++.

Strong object-oriented design and programming skills.

Strong entity component system design and programming skills.

Strong understanding of HCI and usability issues in software experience design.

Comfortable using Photoshop and other graphic design software, with very capable UI design skills.

Awards and Publicity

Best game, Intel Perceptual Computing challenge

$75K cash prize and $150K investment


International Game Developers Association

2011 GDC scholar and 2011 E3 scholar.

First individual to receive both of these very selective awards.


Engadget, PCMAG, and Playstation Lifestyle.


Some of my gesture recognition work with the Playstation Move controller has been featured on a number of major tech blogs, including: engadget and PCMAG. Playstation LifeStyle was kind enough publish a full interview with me on the topic.

School related


Winner of the 2010 ECU Annual Research & Creative Achievement Week.
Dean’s List 2008
IHFA Scholarship 2004


BS computer science

East Carolina University

2004 - 2008

BS mathematics

East Carolina University

2004 - 2008

MS computer science (Unfinished)

East Carolina University

2010 - 2011

I completed all required coursework for this masters degree,
but left the university to pursue entrepreneurship before completing other requirements.
GPA at time of leaving 3.55


Multitouch gesturing systems &
Multitouch system development

East Carolina University, Greenville NC

2006 - 2010

Developed a new unistroke gesture recognition algorithm which has improved accuracy over many well established recognition algorithms.

Developed an extension of formal language theory which focuses on describing languages whose input come in the form of several concurrent streams of information. This allowed many well studied text pattern matching techniques to be applied to multi-touch gestural input.

Designed and built a FTIR and projector based multi-touch surface. Helped develop some of the open source libraries used in building multi-touch screens, including tbeta and Community Core Vision.

Efficient routing protocols for
geographic mesh networks

Auburn University, Auburn AL


Developed a new geographical routing protocol that averaged roughly twice the efficiency of the leading protocol at that time, but required no additional computational or memory costs.

Feature abstraction and curve fitting
for offline handwriting recognition

East Carolina University, Greenville NC


Designed several image processing algorithms, and assisted a team of graduate students in implementing them.

Divergences in Quark and Neutron Matter
in extremely hot and dense environments

East Carolina University, Greenville NC


Assisted in studying gravitational wave models to make predictions about differences between quark and neutron stars


A Survey of Input Sensing and Processing
Techniques for Multi-Touch Systems

J.D.Pennock and M.H.N.Tabrizi
In Proceedings of the 2008 International Conference on Computer Design
WORLDCOMP’08 ISBN:1-60132-056-6


Pending Patents

Gesture and motion based
control of user interfaces

Wolfram Luncher, Jacob Pennock, Eric Veit
U.S. Patent Application No. 15/696,022

Filed: 5 September 2017