Wednesday, April 23, 2014

Reflections on Scientific Programming

I am really starting to associate with the curse, "May you live in interesting times." Having spent more than a decade of my life in science, particularly supporting genetics labs' principal investigators (PIs) with computer programming, I have seen major changes. This blog entry was instigated by a thought about upgrading my R language idioms in light of many social media stories about Hadley Wickham's prodigious output of R-related packages that augment and in some cases replace base R language constructs. At the moment I am trying to finish a study that involves simulated as well as real data. As is typical of scientific programming, I know what I really need to do about the time I finish generating data from analytical experiments and plotting the results-- time to publish. From there we move on to the next study. Rarely do we find the time to "sharpen our saws" when we want to starting sawing down a new problem and start chopping it up. I do not put the blame on PIs, who trust their programmers and technicians to do the right thing. The problem, like so many in our society, is one of information. As science becomes more transparent, this issue will arise again and again. Transparency in programming is difficult, even when all sources of information are available. How many projects are actually based on reuse and not cut-and-pasting efforts (code surgery). Code reuse should probably be a subfield of archaeology. Tools and system are being created to address this issue, but I do not have a great deal of optimism. The systems and their solutions can be just as complex as those they are trying to address. Maybe I'm just getting old, but my opinion is that we need to slow the fuck down. A topic for a different blog post: What are we getting for all this frenetic effort? I have had the feeling for some time that a great deal of science will be exposed as (mostly innocent or accidental) fraud. There is too much previous work to check, while output increases exponentially. I don't want to end with pessimism, so I will say that I do believe most of science is done by responsible and ethical people. The perception in many cases is that science is a cut-and-dried field of exacting work. My experience is that it is mostly trial-and-error supported by statistics. Again, this is just misinformation about what science attempts to do. I love it, but I do have serious doubts about its efficacy versus common perception.

Thursday, January 26, 2012

Please don’t learn to code

Check out this article from DZone where Programming with a capital P is mentioned:

Please don’t learn to code

Friday, November 11, 2011

"P"erl versus "P"ython

Capital "P" programmers everywhere are making the switch from Perl to Python for scripting. Like many, I've resisted the change. I mean, c'mon, you cannot dictate how I structure my source code. Free form is dangerous but allows the flexibility to be elegant or not. HA! Moving into a new job, I thought it might be a good time to give Python a true chance to live in my research programming environment. I fell in love!!! Once you get past the initial anti-authoritarian, rebellious feelings, it starts to pay off. Not having to think about formatting reminds me of how much I like LaTeX for different but similar reasons. LaTeX does not assume you have the skills of a typesetter, as Word and others do. I just type the content, tag the special bits like "this is a section title" and let the "compiler" do it's thing. Python starts to feel the same way after a while. Why do I need to type a lot of extra characters (braces for blocks, semicolons at the end of the lines, etc) to do simple things like conditional tests and loops? I admit I have not become fully "pythonic" yet (there are many, many cool features in the language beyond straight procedural programs) , but I feel no need to go back to Perl, nor do I feel the need to slam it either. Like MS-DOS, it does work, it has tons of users and it helped me get experience and better appreciate good design. One big selling point for Perl has always been CPAN, the comprehensive perl archive of modules to do everything under the sun. If one needs an algorithm, it's probably already been done in a Perl module on CPAN. It might not the greatest implementation, but good enough to get started. However, Python is catching up VERY quickly, and in fact seems to have more serious attention from professionals. One reason might be that Python code is READABLE, a quality not seen so much in Perl unless effort is taken by the programmer to make it that way. Python code has forced structure that works both functionally and aesthetically. This has many advantages, the biggest one being readability. Perl is notorious for being a write-only language, where the author is the only person who can grok the code! While a language is largely irrelevant when it comes down to it, I feel like Python, when a choice is possible, is my default scripting language.

Thursday, January 14, 2010

Reflections on 10,000 Hours

I am reading a great book by Malcolm Gladwell called Outliers. Upon reading about the 10,000-hour rule, I began to see the connection with capital P programmers. We are not born this way. While we may have an aptitude for certain abilities, most of what it takes is just plain old practice. I spent the years from 1987 to 2000 doing nothing but programming in every available moment I had. I did not have a social life to speak of, only my connection to the pure logic and meritocracy of the computer universe. Most of my friends and relatives were busy maintaining familial obligations while I immersed myself in the computer world-- from transistors to the foundations of computer science itself. Since then, I have been less extreme but nonetheless focused and consequently isolated. Is this the price to pay for "success", however we measure it? I guess I'll thinkk about this more as I read on.

Monday, September 29, 2008

Why Programmers Love Linux, Or How I Learned to Stop Worrying and Love Text Files

Why is it that Programmers love Linux? Is it the power of the command line? The responsiveness of the systems compared to a bloated Windows interface? The programming tools that are standard on a Linux system? I submit the power of Linux is in large part due to the UNIX philosophy that "everything is a text file". The standard Linux shell on most systems is ready to work with text files immediately. Utilities such a 'wc', 'sed' and 'awk" are workhorse tools I use daily. Much of these powerful utilities' features has been absorbed by and as a consequence made more internally consistent by the Perl scripting language. For larger applications, Perl is great, but the tools available from the command line are there for immediate use and bypass the edit-run-debug cycle of scripting.

I will futher pursue the topic of text file manipulation in later posts, as this is a fundamental part of what Programmers do.

Thursday, September 11, 2008

BASIC, Considered Harmful?


It is almost a prerequisite of Capital "P" Programmerdom to have cut your teeth on BASIC. I started with the Radio Shack Color Computer BASIC and gradually made my was to BASICA when IBM PC-compatible machines started to take over. I spent many days typing in listings and running them to my constant amazement. Additionally, hours of fun could be had at the local department store, typing in crazy loops and other likewise routines, only to find the clerk bewildered at what happened and immediately restart the machine. Of course, in those days, a restart/reboot took only a few seconds, since most of these earlier computer did not have an operating system! 

The first BASIC I used on an operating system was GWBASIC on Microsoft DOS 3.x. From GWBASIC I followed BASIC through Visual BASIC (VB) for DOS and ultimately Windows. In fact, VB helped put me through my bachelor's degree in computer science by proving
to be a quick and quite useful way to develop Windows applications. I used VB in some truly unique situations, including a GPIB sensor/instrumentation/logging system and case where 
the application had no user interface at all! 

In any event, I learned the fundamentals of programming by hacking out loops and subroutines in the first consumer/hobbyist-available programming language. Later, in my engineering experience, BASIC showed itself many times: automation and control systems, instrumentation, specialty devices (eg: bar code scanners), etc, etc, etc. Today, there is some rabble about BASIC being a poor choice as a learning language. While I would not support the teaching of computer science using BASIC as the implementation language, BASIC does live on and is a fine language to solve computing problems. Just recently, I was turned on to an implementation of BASIC that carries on the tradition. I could go on and on about BASIC.

Conclusion: BASIC is a fine computer language, and I will defend it as a practical programming language and a perfectly acceptable way to learn and experience a little bit of what it's like to be "P".

Wednesday, August 27, 2008

In the time of chimpanzees I was a monkey

I have been called an anachronism, with my multiple xterms, emacs and a command-line compiler. Can one really say their productivity is enhanced by IDEs and GUI tools? Not that I don't use these tools on occasion, but my ability to work largely without a mouse enhances my programming experience. I prefer to tell the computer what to do versus asking-- an imperative versus interrogative interface to the capabilities of the system. A now even more dated book on the subject is Neal Stephenson's In the Beginning Was the Command Line.