My programming lecturer used to say "programming is the process of breaking a problem down so a high speed idiot can understand it, if you don't understand the problem then you can't explain it".
He also said that a programmers most useful tool was a pencil and a 5mm square A4 pad.
After years of solving thorny problems by getting away from the screen and scribbling in a pad I can't say he was wrong.
Yes I learned how to program on paper. It was when computer access was expensive. We learned how to write code on paper and then go over each line with a table for variable values as we run over the code.
I learned BASIC on punch cards using an IBM Mainframe for the junior high school math computer club, and it got changed to the math contest club after they couldn't afford computer time anymore.
In modern times there are IDEs to help people, but most of my best code I used to write on paper first. Visual Studio kind of ruined that as I wrote code in their IDE and painted forms for GUI access.
Do you still do any programming on paper? I've noticed I've occasionally felt like I've relied too much on my tools like my IDE or Resharper. I've been thinking I should get in the habit of doing some programming using just a simply text editor or pencil/paper to prevent that.
If you feel that, you certainly should. Going after simpler tools may give you
somewhat different experience of programming, but for pen and paper, it's
totally different. It has much friction along most of the axes (you can't
cut'n'paste or easily rename your variables, and it's difficult to pre-plan
the length of a block of instructions), but along some, the friction is
non-existent (invalid and ad-hoc created syntax, not defined functions,
ellipses for obvious arguments and statements). Combine all that and your mind
starts thinking in a different way, focusing on different things than it would
with a computer editor.
I find it very beneficial in the early stage of planning a program (or
a subsystem of program).
He was an insanely smart guy, Type 2 Unixbeard (the skinny one with the pony tail rather than Type 1, the portly one with the T-shirt of a bankrupt tech company).
Back in the 80's he worked on the software that ran/runs digital switching for (iirc) BT, high scale, high reliability stuff in C.
One time, in university I did not have access to a computer for a couple of days and had to write code for a b-tree in C++. I designed and wrote the code fully on paper and when I got access to a computer, I typed it in and it worked!
My statement was a little tongue-in-cheek but that was the first time I felt the raw power of my mind. I still feel it now, but it doesn't have as much of an impact as it did then. It's probably what everyone feels when they are in the zone.
So now, I will walk around doing nothing for 4 to 5 hours to let my brain work on a problem, then type it in and have it work in 10 minutes. I do this same thing with design, not just code. I've noticed that coding has become a lot easier as a result.
I still get stuck when I'm dealing with other people's bugs though, especially with third party libraries.
One I've noticed is that being in my head so much makes it more difficult to explain things to other people and I think this hurts me.
I keep my notebooks for many years and occasionally have fun looking back at old ones and trying to understand what it was that I was struggling with at the time. Usually I can't so somehow these doodles were extremely helpful while trying to pin down some problem long enough to be able to solve it but they definitely don't serve their intended goal as long term documentation for me. I probably should simply discard them.
I'm a pens-and-notebooks power user, and I know exactly what you're talking about.
The trick is, whenever you're writing anything, add notes for your future self to make sense of whatever you're doing. It's like adding comments to explain your code.
It's a little extra effort and bookkeeping but it makes old content so much more valuable!
That's interesting. I do the reverse. I have a trail of hundreds of mind maps and lists on paper, some of them probably recycled by now. I've always wondered if it would have been better to save them and review. I think I've lost a lot of work on some problems/ideas I wanted to explore that way. Probably forgotten a good number of problems I was interested in to start with. Mostly personal though - professionally, my desk piles up until I'm forced to review the usefulness of each piece of paper (still, the 'where's that sheet gone' dance is common).
Interestingly, it seems based on my limited understanding of many AI and machine learning techniques that these techniques don't really follow the rule. If you train an artificial neural network to recognize certain photographs, you don't really understand what is going on. You know how the artificial neurons work, and how the training process works to apply weights to various connections between neurons, but you don't really know in precise terms what is going on any more than you do with a biological neural network.
I don't know what the implications of this are, or even if my simplistic understanding is accurate, but it does seem interesting.
Well, one implication is quite obvious. Using ML-techniques you are sometimes able to solve a problem even if you don't have a precise idea of how you would do it. I feel that's part of the point, really.
[Edit: Elaboration]
In ML, you effectively transform your problem from "What piece of code will solve my problem?" to "What piece of data-driven code might solve my problem, and how do I acquire the right set of data?". Also, some ML-algorithms are designed specifically to enable you to analyze their "reasoning" about the problem after they have been trained.
This might be a strange question, but what does 'scribbling on a pad' actually entail? Do you write the actual code, or is it usually more about drawing out the flow of program?
For me it's both. You're trying to visualize the problem, its parts and how they interact, and you may write down some code for specific sub problems because it just popped into your head and you don't want to context switch.
There's a limited amount of things you can keep in your head at once, and the paper is somewhat of a navigational aid and rapid access memory. Paper also offers you a lot of flexibility in regards to laying out things fast and sketching things, which computers still don't have quite down yet. If I try to layout my thoughts in emacs or photoshop a lot of my processing power is taken up by the act of using the tool ("ugh, was it M-x rect or M-x rectangular-region"/"Let me get the pen tool, and the rectangle tool, and where did the tool tray go?").
Scribbling on a pad doesn't just apply to coding though, it's a useful technique for a lot of areas that have thorny problems that exceed your local memory capacity.
This is good encouragement for me to try using pen and paper again. The problem is that every time I do, I get frustrated by how slow it is for me, my hands get cramped, and I miss the backspace key. But perhaps I just need to keep at it for a bit, because I do see (and have experienced, back in college) the advantages.
The backspace key equivalent of pen and paper is crossing things out/erasing them. If there's a line of writing I don't like, I'll cross it out and start that piece over.
If you want to get faster and better at sketching diagrams I'd recommend you practice some of the basic techniques that say concept artists use. Straight lines from the elbow with little wrist movement and a little practice, and you'll be drawing things in no time in no time. Unless I need an incredibly straight line, I usually forgo rulers at this point.
If your handwriting has slowed down, or you're cramping, you might want to give that a little bit of practice as well. No need for the cleanest Suetterlin, but getting posture right and yourself up to speed might be worth the time investment. I've been playing around with the palmer method recently, it seems quite nice for this purpose.
Then there's tools. You don't want to be writing with cheap ballpoint pens on rough paper. The lines don't come out properly, which will make you press harder, which leads to fatigue. My current favourite pens are the Uniball eye-fine which is my go to for writing, as its a very smooth disposable ink pen. I've got one of those in front of me with a Kum ergonomic grip. Fountain pens are also very nice, and an entry level Lamy, Pelikan or Pilot will serve you nicely, but they're more suited for linear writing as they may smudge.
Pencils are nice for sketching, but you have to sharpen them, which is annoying. For regular use I use a Faber-Castell Grip 1345 0.5mm mechanical pencil, which is still affordable but doesn't have many of the issues that cheaper mechanical pencils suffer from (broken leads, leads going back in because of insufficient internal grip, etc.). The Faber-Castel leads are also very nice and sturdy and make decent lines even at low pressures.
Paper isn't that variable, as long as you don't totally cheap out on it. I'm particular to .5cm grid paper, but there's circumstances when plain white is nicer to work on.
It's a bit much of a list, but I think a lot about stationery. You don't need all of these, and if you find other tools that do the job for you, then by all means go for them, these are just my personal favourites. One pen and a block of paper is more than sufficient.
But I do think coding and thinking through problems on paper is a valuable skill. One neat thing it allows you to do is to go to a different place than the one you type things at to go think deeply about problems, because it's a lot more portable than a laptop. If like me, you haven't quite figured out how pseudo-code works and still write straight up source code, it also gives you a reason to memorize reserved words which I find does make things quicker ("was it import or include?"). And finally, your IDE won't distract you by having the gumption to tell you that "hey, you aren't using this variable, you lugnut, lookie look, this import is BROKEN! Aint you got any IntelliSense in you?" when you really want to be thinking about more conceptual things and not tiny implementation details. Sometimes you do do your best work when you're away from where you do your regular work.
Great advice here. Regarding beat drawing of diagrams with lines drawn from the elbow, there is a great set of online lessons that teach this through exercises. I totally recommend it:
> But I do think coding and thinking through problems on paper is a valuable skill. One neat thing it allows you to do is to go to a different place than the one you type things at to go think deeply about problems, because it's a lot more portable than a laptop.
Drawing the problem is also useful because it focusses
different parts of your brain on the problem. If the problem was easy to solve with just the more linear textual parts, you probably could just have sat down and coded it.
But if that's not enough, engaging the visual and spacial parts can help you understand the solution without being able to verbalise it (ie code it) yet.
The Palmer method, that is what I learned back in the mid-nineties in elementary school. I didn't know it had a proper name other than 'cursive writing'. Neat!
Edit: Err wait, maybe not. It looks the same but so does the Zaner-Bloser form. Looking at some more methods, apparently I don't understand how it all works.
Drawing representations of data structures and algorithms for me. Imagine drawing a linked list. Probably a box with a pointer to another box. Code looks nothing like that, but when you have the idea on paper its a lot easier (for me) to work out what the code should be.
> Underlying our approach to this subject is our conviction that "computer science" is not a science and that its significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology the study of the structure of knowledge from an imperative point of view, as opposed to the more declarative point of view taken by classical mathematical subjects. Mathematics provides a framework for dealing precisely with notions of "what is." Computation provides a framework for dealing precisely with notions of "how to."
From SICP preface, which is the quote that actually matters, this is inspired by Minsky's quote. I hate that the quote that went public were all the other less inspired quotes from SICP.
The purpose is completely different! Here's one of several important reasons.
You wouldn't survive doing math with longer variable names. Imagine yourself writing "time" every time you need to write it in a formula in the course of a working on a problem. Just expand the following expression (time + 2)(2time - 3)(4time + 5) by hand, writing "time" every time. Or imagine studying vector calculus and writing out the formulas with the long names like "grad", "curl", etc. That kind of notation was replaces with the nabla symbol and overloaded notation for a very good reason.
Also, most derivations, for simple formulas even, go through larger complexities before they converge to something slick. Take a look at the derivation of the Navier-Stokes equations [1] and see if you could have done the same thing with longer variable, function and operator names.
So yea, it all boils down to repetition and quantity.
I think it's largely because the long-standing mathematical tradition comes from an era where those formulas always had plenty of context, be it that of a topic-specific class/lecture or a paper that is published in a journal with a particular focus. Once you know what field of mathematics you're dealing with, there are strong conventions for what variables mean what; that's one of the things that has let to mathematics drawing from so many alphabets. It's only been with relatively recent advances in information technology that the highly technical details of many specialties have been aggregated closely enough for the context to be lost.
Also, long variable names would greatly obscure the overall structure of many formulas, which could be an even bigger problem.
I think that the short variable names in math come from the time when these things had to be written out in long hand on blackboards. You ever see old sci-fi movies, like "The Day The Earth Stood Still" where professors are working out ideas on huge blackboards? Having long variable names wouldn't fit on the blackboard and would be slow and hurt your hands.
The lack of an efficient IME for verbose math input on a computer certainly accounts for why math is still terse when done in real-time such as when extemporaneous lectures or when working something out. But I don't think it fully explains why the convention persists even in published works that took months or years to prepare; there's got to be a significant effect for the readers and not just the writers. If a longhand style of presentation were as clearly superior for mathematics as for software source code, one probably would have caught on by now.
It's probably just the weight of tradition from the longhand days that leads people to use the short variable names, not that it's somehow better, even on a computer.
All it would probably take would be for someone to start a math journal that required a minimum of three letter variable names and everyone would switch over.
And then they run out of Latin letters so they switch to Greek. Then they run out of Greek letters so they switch to Hebrew...
You have to remember that math is written for the sake of mathematicians' brains, not for the sake of dumb machines which accept only linear strings of a limited character set through a teletype. A mathematician can look at a formula and subconsciously break it down into constituent parts based on its shape. Multi-letter (word? phrase?) variable names would only serve to add unnecessary clutter and obscure the shape of the formula. Meaningfulness of names is a secondary concern, if it is a concern at all; meaning can be established in the text surrounding the formulas or equations; and if things indeed get too complex for a set of a few formulas, the option is always there to switch representations -- to computer code or pseudocode.
Also, mathematics' naming conventions are quite strict.
The nabla symbol is almost-exclusively used for vector-calculus operations. The letters t, s, u, v are almost-always used for parametrizations. The letters i, j, k, n, m are usually used as indices of summations. We can go on.
There are also Beth numbers (https://en.wikipedia.org/wiki/Beth_number), closely related to Aleph numbers, but as far as I know that is the only place hebrew letters show up in mathematics.
Your comment just made me realise why I hate any hackerrank "challenges" I have looked at. (Skyscanner uses them for interview tests, I have been close to applying twice, but got offered other jobs before I got around to completing them).
I have decided to try a warm up exercises a couple of times before applying but the way they word the questions is so unmeaningful: 'n' lines with y variables, plus spending 20 minutes trying to work out how to read their input. No debugger and not easy way to cut and paste, as its all in a web form.
Fuck that, I am a software engineer, I usually read data from databases and name my variables properly and use tools to help me understand the problem, not hinder myself.
All right, so nowadays software engineers cannot read specifications, cannot program and correct their mistake without a debugger, cannot do anything without an IDE, cannot do anything without copy-pasting (from SO) and read their data from a database.
I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program, and most variables don't have a fixed meaning.
Can't really think of one that could be improved by writing it as pseudo code.
> I don't think that's possible for most of them. Quite a few formulas aren't equivalent to any one program,
Even every proof can be expressed as a program, see Curry–Howard correspondence [1].
> and most variables don't have a fixed meaning.
I think this is kind of the point of the original comment.
For a physicist PV=nRT is concise and clear but for everyone else?
The letters P, V, n, R, T can have many meanings but in the context
of this formula their meaning is fixed. Why not write:
pressure * volume = AVOGADRO * BOLTZMAN * temperature
> Can't really think of one that could be improved by writing it as pseudo code.
See example above, but I think it strongly depends on what one is familiar with.
>Even every proof can be expressed as a program, see Curry–Howard correspondence
Well, that is true, but that's still pretty far removed from any common programming language. And readability doesn't usually improve.
>For a physicist PV=nRT is concise and clear but for everyone else?
You've somewhat got a point there, physics would be a lot easier if all formulas and variables were actually defined when needed. But symbolic notation is still a necessity, simply because symbolic manipulation is really powerful, and it's a pain to have to write 'BOLTZMAN_CONSTANT' every single time, instead of just k_B.
Also, your notation might work for simple equations like the ideal gas equation, but an equation that's only slightly more complicated, the van der Waals equation, would become quite hideous:
There are a lot of conventions in physics and math and they come to the rescue. In the CONTEXT of gases, it's easy to realize that V must be volume, T must be temperature, etc. The formula PV = nRT is for physicists, not for "anyone else".
It is a little tough to slog through now, though I'm sure it would've been pretty enthralling in '67. I liked this from the conclusion quite a bit, though:
A computer is like a violin. You can imagine a novice trying first a phonograph and then a violin. The latter, he says, sounds terrible. That is the argument we have heard from our humanists and most of our computer scientists. Computer programs are good, they say, for particular purposes, but they aren't flexible. Neither is a violin, or a typewriter, until you learn how to use it.
I also liked what seems to be a pretty central point of the article (which I admittedly only skimmed): just because some medium (e.g. computer programming) has very precise requirements (e.g. in terms of syntax) for the forms expressions can take on, does not imply that the content of what can be expressed is similarly constrained.
I disagree. The content and expression become constrained because a description of the problem domain has to be forced to fit inside them. Usually you can only make this happen with a drastically over-simplified model.
One of the tragedies of AI and CS is that it's easy to delude yourself that you've understood something because you can make a toy model of it - when in fact non-toy models that are comprehensive enough to be useful to a domain expert tend to be sprawling, irregular, complicated, and not very coherent.
The problem leads to a kind of paradox: Most human behaviour can't be simulated with an algorithm that's simple enough for humans to understand.
Violins turn out to be a good example. It's trivially easy to sample a violin sound. But sampling is nowhere close to being able to model the musical data bandwidth in a performance by a real string quartet.
Tacit knowledge is a useful concept not mentioned by Minsky. I don't think programming is really a good way of expressing a "poorly understood and sloppily formulated idea" of this type: a third party's visual experience and verbal description of a skillful but complex human action such as shearing a sheep.
The old philosophical distinction between the discursive and the intuitive is key here. Can you systematically write a program which exactly replicates your intuitive responses to some set of possible situations? I'd say it's difficult or impossible, because of the complexity of the brain and our lack of perfect introspection into what goes on in our minds.
Not sure Minsky really has a lot to offer a contemporary AGI readership. His work feels more than a little mechanistic.
Maybe you're familiar with Dreyfus's critique of "cognitivism" in AI, as explained in his book What Computers Can't Do. His recent article "Why Heideggerian AI Failed" is also relevant... and his book on Heidegger, Being-in-the-World, which has some philosophical background for his anti-cognitivism.
Recently I noticed a parallel between Antoine de Saint-Exupery's comments on technology and Heidegger's notions of presentness-at-hand and readiness-to-hand. It's in his essay "The Tool", which also includes the famous line about perfection being achieved when there is "nothing left to take away."
Saint-Exupery has a profoundly romantic attitude to technology: I'm not sure about the order of publication but it is a fact that Heidegger applauded Saint-Exupery's Petit Prince, and apparently Sartre felt the two men's philosophies were related.
(Apparently the controls of the Lockheed Lightning, the aircraft type in which Saint-Exupery died, were too complex for his taste, turning the pilot into an "accountant" rather than receding from conscious attention, as the primitive instruments of earlier, simpler planes had. Saint-Exupery clearly saw the purpose of "the tool" as facilitating his experience (and responding to his personal "authority") as a free spirit of the air, without much thought given to the constraining military command hierarchy or the pilot's job of killing.)
Thanks for the link! I didn't know that was the context for that famous quote, and it was a nice read. It seems that we so seldom get to experience that kind of perfection in software, which is always tangled up and half nonsense. It really is a wonder that anything works.
To me, the ideal of "abstraction" in software has to do with this search for unity, on a conceptual level. Unix with its files and processes seems to come pretty close to being this kind of dependable, understandable, cohesive abstraction... which is why I love shell scripting so much. Sure, it's not the most beautiful language, but the basic building blocks make sense in some almost timeless way.
Digressing further, I'd really love to work on a cleaner shell language. Like, Perl is an exploration of what happens if you write your scripts in a souped-up awk, but what I really want is a bash for us new kids, with good support for nested data structures, functional patterns, some more clever ways of representing background processes and FIFOs, modules and packages, etc etc.
I forget how I got from Heidegger to FIFOs, but yeah, a ready-to-hand shell that works well with common tasks like serving HTTP and building JSON and stuff. I love the Unix philosophy but people aren't really learning shell these days... something like it seems like a way forward... composing simple programs... ah!
"We must not fall into anthropomorphic ways of thinking about machines; they do only what their programs say; they can't be original or creative." In debunking this fallacy, I would have thought that the "Computers Are More Flexible Than You Think" argument wold only need to go halfway, and "Brains Are Less Flexible Than We Think" could take us the rest of the way there. Of course, in 1967 I imagine the support for the second argument was much thinner than it is now.
The beauty is in that a weighted graph of lambdas could be build out of conses even back then. The idea that code is data (and it could be generated by a program itself) was already there, but they have dropped this right intuition and turned back too early, and the dark ages of Java and packer's mindset followed.
He also said that a programmers most useful tool was a pencil and a 5mm square A4 pad.
After years of solving thorny problems by getting away from the screen and scribbling in a pad I can't say he was wrong.