Jacob D. Pennock

Creative Developer and Game Designer

Interests include emerging human computer interaction techniques, close range hand tracking, computer graphics, gesture and computer vision based interaction systems, gestural gaming systems.

Positions
Currently
Held

Senior Creative Developer

Helios Interactive Technologies, San Francisco CA

Jan 2013 - Present

Helios Interactive is an experience design studio located in the heart of San Francisco. It specializes in creating interactive art installations and interactive marketing experiences for major brands.

At Helios I've produced everything from large scale semi-permanent installations to short term event based attractions. I've created content for a large array of technologies including; close and long range gesture systems, large touch walls to small mobile apps, facetracking and augmented reality, giant 4k video walls and crazy multi-projector setups.

Chief Game Designer / Lead Developer

Unicorn Forest Games, Oakland CA

Aug 2011 - Present

Unicorn Forest is a passionate side project game studio that I founded with my wife Melissa. Its where we work on all the independent and experimental game design ideas we have.

We also provide a wide range of freelance game development services to other development houses and have helped other game studios realize their vision. Particularly in the area of visual fx design, but also, art, animation, UI and character design, as well as just general game play programming and development.

Recent
Projects

Head of the Order

Head of the Order

At Unicorn Forest, for the Intel Perceptual Computing Challenge

I developed one of the most advanced close range gestural experiences to date. In this tech demo feel like a wizard and cast increasing elaborate spells at your opponents. Click the image to the left to check out the video. Its pretty cool.

Head of the Order

Speed and Agility Challenge

At Helios, for the 2014 Super Bowl XLVIII Host Committee

For the run-up to the Super Bowl, the Host Committee was looking to create an engaging, interactive experience that would help recreate a real ‘game’ experience for their "Join The Huddle Tour".This arcade game is touring the northeast, visiting over 40 locations in the four months leading up to the Super Bowl.

Recent
Clients

Technical
Skills

Gesture, Touch, and Motion experience

I have worked extensively with many of the newly emerging human computer interaction devices and their SDKs


In the long range gesture space I have experience with the Kinect, both the microsoft SDK and primesense openNI. I've also worked with less common sensors like the Panasonic D-IMager and custom tracking SDKs

In the close range gesture space I have worked with the DS-325 and the softkinetic iisu platform for a number of applications. I've also worked with the Creative Senz3d camera and the Intel perceptual computing platform, as well as the leap motion device and it's SDK.

In the touch space I have built a number of large scale touch apps that receive TUIO, as well as have worked on custom trackers that output TUIO. I've also worked with iOS and android mobile and their respective touch apis.

I've also worked with the Sony Move and Nintendo wii game controllers.

Unity Game development and experience design

I love the Unity Game engine. It has been my main professional development and content creation platform for almost 4 years now. I know many of it's subsystems and api calls like the back of my hand. I'm very active in its developer community and on its private beta developer list.

Research programming expertise

Using C/C++/C# for the research and development of advanced real-time computer vision applications associated with, image and video processing, object tracking, gesture recognition, and real time patten matching.

Mathematical expertise

Algorithm development, automata, formal language theory, abstract machines.

Specific skills

Primarily develop in C# and C/C++, but have also built applications in Javascript, Actionscript, Haskell, Prolog, and Java. My expertise in language theory allows me to pick up and build in new languages, of any paradigm, very quickly. I tend to prefer working with an object-oriented approach in a strongly typed language.

Strong object-oriented design and programming skills.

Strong entity component system design and programming skills.

Experience with SDL/OpenGLES game engine design for Nokia arm devices.

Strong understanding of HCI and usability issues in software experience design.

Comfortable using Photoshop and other graphic design software, with very capable UI design skills.

Awards and Publicity

International Game Developers Association

2011 GDC scholar and 2011 E3 scholar.

First individual to receive both of these very selective awards.

2011

Engadget, PCMAG, and Playstation Lifestyle.

2011

Some of my gesture recognition work with the Playstation Move controller has been featured on a number of major tech blogs, including: engadget and PCMAG. Playstation LifeStyle was kind enough publish a full interview with me on the topic.

School related

2004-2010

Winner of the 2010 ECU Annual Research & Creative Achievement Week.
Dean’s List 2008
IHFA Scholarship 2004

Education

BS computer science

East Carolina University

2004 - 2008

BS mathematics

East Carolina University

2004 - 2008

MS computer science (Unfinished)

East Carolina University

2010 - 2011

I completed all required coursework for this masters degree,
but left the university to pursue entrepreneurship before completing other requirements.
GPA at time of leaving 3.55

Research
Experience

Multitouch gesturing systems &
Multitouch system development

East Carolina University, Greenville NC

2006 - 2010

Developed a new unistroke gesture recognition algorithm which has improved accuracy over many well established recognition algorithms.

Developed an extension of formal language theory which focuses on describing languages whose input come in the form of several concurrent streams of information. This allowed many well studied text pattern matching techniques to be applied to multi-touch gestural input.

Designed and built a FTIR and projector based multi-touch surface. Helped develop some of the open source libraries used in building multi-touch screens. Including touchlib and Community Core Vision.

Efficient routing protocols for
geographic mesh networks

Auburn University, Auburn AL

2006

Developed a new geographical routing protocol that averaged roughly twice the efficiency of the leading protocol at that time, but required no additional computational or memory costs.

Feature abstraction and curve fitting
for offline handwriting recognition

East Carolina University, Greenville NC

2005-2006

Designed several image processing algorithms, and assisted a team of graduate students in implementing them.

Divergences in Quark and Neutron Matter
in extremely hot and dense environments

East Carolina University, Greenville NC

2004-2005

Assisted in studying gravitational wave models to make predictions about differences between quark and neutron stars

Additional
Attributes

Myers-Briggs personality type

INTP

Hobbies

I enjoy solving problems, and am especially talented at finding solutions within limited environments or given interesting constraints.

I founded a university Game Development club.

I am a self motivated learner, and have a great love for learning.

I spend my spare time contributing to the open source community, building small avr based electronic projects, keeping up with tech trends, and gaming.

Languages

English (native)
French (pretty rusty at the moment)

Publications

A Survey of Input Sensing and Processing
Techniques for Multi-Touch Systems

J.D.Pennock and M.H.N.Tabrizi
In Proceedings of the 2008 International Conference on Computer Design
WORLDCOMP’08 ISBN:1-60132-056-6

2008