GECCO ’17 Proceedings of the Genetic and Evolutionary Computation Conference Companion
http://dl.acm.org/citation.cfm?id=3084216

Abstract: “Genetic Programming, a kind of evolutionary computation and machine learning algorithm, is shown to benefit significantly from the application of vectorized data and the TensorFlow numerical computation library on both CPU and GPU architectures. The open source, Python Karoo GP is employed for a series of 190 tests across 6 platforms, with real-world datasets ranging from 18 to 5.5M data points. This body of tests demonstrates that datasets measured in tens and hundreds of data points see 2-15x improvement when moving from the scalar/SymPy configuration to the vector/TensorFlow configuration, with a single core performing on par or better than multiple CPU cores and CPUs. A dataset composed of 90,000 data points demonstrates a single vector/TensorFlow CPU core performing 875x better than 40 scalar/Sympy CPU cores. And a dataset containing 5.5M data points sees GPU configurations out-performing CPU configurations on average by 1.3x.”

My first 1st-author paper is published! Thank you Lee, Eddie, Marco, Arun, and Iuri for your input, support, and collaboration. –kai

Research has its moments of upset, intrigue, and thrill.

But for this past six months these moments are lost to the effort of writing.

Proposal followed by proposal, I am a salesman with a briefcase full of ideas. Some new. Some old. Some revised. Some bold. If my visions for a better tomorrow are a good match to your funding of today, then we will enter into a partnership in which I am paid to investigate, experiment, and to play.

“Some stories only make sense in retrospect, the looking back giving foundation to where we now stand. This is the first of what will hopefully be a series of essays to describe the path from a Good Sam’s campground in 2011 to in some way, helping develop the first community on Mars.” –kai

It starts long ago, beneath a stair case in the basement of our family home in Columbus, Nebraska. Friend Jason Zach and I covered the underside of the stairs with plywood, cardboard, a dead monochrome CRT, and myriad electronic components, wires that stimulated Radio Shack switches, piezoelectric sirens, and LEDs and wires that went nowhere. In that spacecraft, we journeyed across the galaxy, venturing to the shores of distant planets whose inhabitants had never before seen humans. Jason was an expert marksman, never afraid to attack. I was keenly interested in obtaining samples, studying the cultures, and welcomed Jason to cover my back.

Many years later, while camped at a Good Sam’s, in Seabrook, New Hampshire on August 2011, I returned to that child-like sense of belonging to a distant place and time. As described, I believed I gained some insight as to how isolated communities might evolve on space stations, Mars and asteroid outposts, even among the stars.

Later that same year, I returned to Holden Village, an isolated village in the Cascades of Washington State. In those months late in the year, the retreat of summer saw the last of the guests depart down the sixteen miles to Lake Chelan. Those of us who remained, counted by dozens, shifted our daily routine from that of a more finite task to general support of the village. Files had to be stoked in order to heat the buildings, snow shoveled, and the water driving the hydro-electric generator kept from freezing, else the electricity would fail.

In those crisp, cold, mostly dark winter days that followed, Holden was a true Village. While a hierarchy of command remained, we became more egalitarian, sharing in the responsibilities of maintenance, even survival should a heavy snow storm bury the pathways and building exits or make impossible a medical evacuation. It was then that my interest in village (communal) living was again stimulated, and the journey to Mars re-ignited.

For five months in 2012 I worked as a photo journalist and documentary filmmaker in Palestine, where a sense of isolation from the world was applied not a mountain village, but the confines of geopolitical boundary that has the power to contain people from birth to death. I witnessed first-hand how the skilled craftsmen and capable artisans were the backbone of an economy of trade and negotiation in place of the familiar currencies of exchange. I learned how much individuals depend upon each other, especially in the challenging times.

I was building a sense of what it meant to live with the challenge of an isolated environment.

On an isolated ranch in the Rocky Mountains of Colorado in 2013, I lived for six months—up to six weeks without face-to-face contact with another human being. In those months I gained from the challenge and ultimate reward of true isolation; a chance to discover who I am without the influence of others, without opportunity to attribute my success nor place blame on the actions of others.

In 2014 I joined MarsCrew134 at the Mars Desert Research Station (MDRS) as the seventh member of an isolated, Mars analog crew. We lived for two weeks in the confines of a simulated Mars lander, a two-story vehicle just large enough to contain individual sleeping quarters, two airlocks, kitchen and crew commons, toilet and shower, lab, and minimal storage. We departed the structure only while wearing a spacesuit, the visor scratched and needing replacement; the radios dodgy at best. The crew came from six countries, representing seven nationalities and more than a dozen languages spoken. It was not always easy, and at times far from fun, but we made the best of those two weeks, focused on our research, data collection, and surviving the simulation. We came away friends for a life-time, even now traveling far to see each other again.

It was then that I became invested in a study of village life. In part because I realized that is where I felt most at home; in part because at least for the first generation, that is how humans will once again live when we finally place boots on Mars.

This week I submitted a proposal to the Interplanetary Initiative at Arizona State University’s School of Earth and Space Exploration (SESE) for the research and development of a mathematical model of a scalable, isolated model of an off-world community (SIMOC).

Now we wait …

If you are coming to OSX from Ubuntu, the installation of applications can go smoothly, as Apple intends, or leave you beyond the threshold of frustration, in a dismal state of frustration, rage, and despair. The instructions at the TensorFlow website are correct, if everything goes as planned. But as noted, “Since a native pip installation is not walled-off, the pip installation might interfere with or be influenced by other Python-based installations on your system,” for me, this became a real problem.

I tried pip, then virtualenv, then Anaconda only to find that none of these worked, for various reasons I won’t go into here. The key issue with pip was that the ‘Collecting …’ stalled, it would hang there, for minutes, even hours (as I waited, and waited, just to make certain). When I canceled the process (CTRL-C), I noted a lockfile in the process output.

It was not until my associate Iurii M. had time to assist me, that he discovered a work-around, using pip with the ‘–no-cache-dir’ extension in order to force it to bypass the locked file (which we never did locate). Then, the collection just worked, as it should, and for the most part, it was installed and running in roughly 30 minutes, including the time required to open an NVIDA developer account and obtain the CUDNN license.

So, here’s how the installation of TensorFlow on OSX goes …

1. Install Homebrew
   $ brew update
   $ brew upgrade
   $ brew doctor
   $ brew install python

   * always run brew and pip from user session, not root

2. Upgrade pip install:
   $ pip install -U pip
    
3. Install TensorFlow
   * use pip with cach disabled (to keep terminal from stalling):

   $ pip install –no-cache-dir -I tensorflow-gpu

4. Install CUDA

   $ brew install Caskroom/cask/cuda

5. Install CUDNN (from NVIDIA)
   https://developer.nvidia.com/cudnn

   $ Download cuDNN v5 (May 27, 2016), for CUDA 8.0

6. Unzip & Install
   1. select location for unzip
   2. move items proper location:
      mv -v lib/libcudnn* /usr/local/cuda/lib
      mv -v include/cudnn.h /usr/local/cuda/include

    

7. Test TensforFlow
   $ python
   $ import tensor flow

Yesterday I received a revision to Karoo GP which now includes the Python machine learning library TensorFlow. The 10,000 row dataset which consumed 48 hours for 30 generations of evolution on a powerhouse 40 core motherboard now runs in less than 4 minutes on a single GPU card.

30 lines of code revised, and Karoo enjoys a 720x improvement in performance.

I am blown away.

The updated version of Karoo will be released to github with the close of December, after the contract developer and I complete a suite of tests and the code is prepared for release.

In June, July, and August I was almost daily engaged in the application of evolutionary computation to glitch classification at LIGO. I worked extensively with Marco Cavaglia and his students Hunter, Luciano, and Kentaro for this effort.

We wrestled with the data, trying to find new ways to extract features which provide stronger correlations. We made progress, got lucky a few times, but more often than not hit dead ends which forced us to circle back to the start.

The joy of this arduous process is complex, for it entails both a passion for success and failure, the two faces to discovery. My professor too often said, “Research is hard” as a badge of honour, a mark of the fearless and brave and dedicated. Yet he failed to say “Research is rewarding!” As a recent New Scientist article presented, it is the people we work with that make most jobs tolerable. I am fortunate to have the best of both–an incredibly engaging challenge conducted with incredibly engaging people.

I have the joy of working with some of the brightest and the best, the funniest, the most seasoned and the most juvenile. We laugh far more than we do argue, yet we celebrate only long enough to realise our mistake and then dive back into another seemingly endless, dark tunnel. The phone calls, the TeamSpeak meetings, the hundreds of emails that keep us going. For with each communication we are challenged to prove our findings, we are challenged to be better at our job than we were before. No one ever says, “That is good enough.” Always, the challenge is for more. Higher accuracy. A stronger correlation. An improved better dataset. Better writing, presentation, and publication.

This week I will officially engage as a “Visiting Scientist” at the Embry-Riddle Aeronautical University in Prescott, Arizona where I am working under Dr. Michele Zanolin and with Marek Szczepanczyk, PhD candidate and chair of the supernovae group at LIGO.

Together, we are applying evolutionary computation, genetic programming in particular, to the classification of Coherent Wave Bursts (CWB) in LIGO data. While we have just begun, only a few data runs in our shared experience, we know the work will be long, challenging, and more likely to fail than succeed. But it is the people with whom I am working which compels me, as much as the prospect of success. If I can play a small part in the team which may, some day, detect supernovae using gravitational wave astronomy, then that small part will be an honour indeed.

My revised MSc thesis was submitted today. More than 120 hours effort across six weeks. There is no end, for there is always room for improvement, always more content to generate. Ultimately, you just stop and turn it in. Challenging, educational, and rewarding. Now, I wait …

Six months ago I submitted my thesis.

I waited for six weeks, to learn of my fate.

I waited for six more weeks to learn the meaning of the feedback given.

I then busied myself with conferences, workshops, research, programming, and travel.

Now, with just a few weeks to go, my thesis waits for me.

An intense application of feature engineering, genetic programming, and proposed ensemble learning applied to LIGO glitch classification leads to an email exchange in which an imaginary beach harbors a pile of shells assembled by a greedy fitness function. The lines are drawn in the sand. The decision tree vies for position while the binary classifier stands its ground.

The conch? Well, it was an outlier whose incorrect classification initiated a war amongst otherwise solid researchers, upstanding men and boys. I fear Sam-n-Eric are endangered once more …