One of my team asked me today for book recommendations on software design and architecture, and I thought about it, and then I realised, well, I haven't really read any. Which sort of struck me as surprising, because I've got a head half full of information and ideas about how to do it (the other half being full of perl ... probably ...) so it's got to have come from somewhere.
Now, admittedly, I've been programming since 1988, first for a hobby and then for money and as a hobby, so I've probably forgotten how I learned a lot of things, not to mention a fair few of the things themselves. Hopefully not important ones, but you never know.
So, I think at the the highest, vaguest level it comes down to: reading words, reading code, and making and discovering mistakes in code.
I've read a lot of words about design, even if I've never read them in dead tree form. I found the original wiki, the Portland Pattern Repository, absolutely fascinating. I also tend to google around when I hear people discussing a pattern - I've probably read at least twenty definitions of Model/View/Controller and associated similar patterns for UI code, including some stuff written by the original smalltalk implementors. I've lost count of the number of blog posts I've read, from insightful to idiotic (you can learn as much from people misunderstanding design as from people understanding it because figuring out how they got into a fallacial viewpoint can help you catch yourself doing the same thing).
I've read The Art Of Unix Programming and other similar things; Structure and Interpretation of Computer Programs is well worth it, but equally so are documents about how to use things written by their designers. The pragprog Programming Erlang book was interesting and gave me more of an insight into why erlang's such an effective platform for the types of application it focuses on. Thinking Forth was a fascinating look at the mindset involved in produces very tight, efficient systems in a stack based language. Paul Graham's article on the roots of lisp has a wow moment at the end when you see the metacircular evaluator assembled from a tiny set of primitives, and Higher Order Perl is a fantastic insight not only into functional design but into the way Mark Jason Dominus reasons about the design process.
I've also read around the culture and concepts of UNIX itself and the early programming done for it and other systems - the Jargon File, the Macintosh Stories and others. I think I googled (some time in 2001, I forget the good links, sorry) basically for early guides to unix and discussions of unix culture. I found interesting little programs I'd never heard of (comm for a start) and gained an insight into how these people were thinking when they designed and built the 70s-era software that laid the groundwork for so much of what we do today. I've gone through the xerox PARC archives (parcftp.xerox.com seems to be down right now but there'll be mirrors if you look) reading about the early days of object orientation, the LOOPS system that later lead to CLOS and so on. I've read a bunch of papers from smalltalkers about how they designed various object systems, and I've read minimal object systems in lisp (and even wrote a tiny one myself, r2-oo.scm).
I've read a lot of source code too - I used to entertain myself by going through linux distro and *BSD package lists to see what perl modules they'd packaged, skim the docs of the ones that looked interesting and then skim the source of the ones that passed that test. I've also done a zgrep on the 02packages.details.txt.gz in my .cpan for huge namespaces like XML:: and applied the same tests. Modules by crazy people such as Audrey Tang or her previous incarnation, Autrijus, Ingy dot Net and Simon Cozens have proved edifying, inspiring, and sanity-defying in equal measure too.
The most important thing I think I've done though is that when I encounter a library design in the wild, or when I'm attempting to create one, I start inventing use cases in my head, as strange and complex as I can think of, and then try and apply them to the design available. If the answer is "it will snap if you do that", I try and work out how to make it bend instead - and 98% of the time I never encounter that need and never write the code, but in thinking through the potential problems I gain a much better understanding of how something has been built and why it's been built that way, or of how to build something similar in the future if I ever do need it.
This sort of thing is, I think, known as "directed practice", but I fail to find a useful citation for that. The point is to always be attempting to solve problems just at or beyond the limits of your ability; succeed or fail, you'll expand your mind substantially that way. And since you're doing it in your head, you can do it on the bus, or waiting for somebody to arrive in a pub or restuarant, or because you can't sleep, or ...
Of course, the other really good way to learn about design is to create libraries and put them on CPAN and then realise just how bad you are at it when they try and do perfectly sensible things with them and your code snaps. Going back over your old designs and asking "what did I get wrong and how could I have done this differently?" is usually good as well - I think a rule of thumb is that if you don't hate a good percentage of your own designs of six months to a year ago, you aren't learning fast enough.
None of which says that any of these techniques are the right, the only or even necessarily a good way of learning this stuff; but it's the ways that worked for me at least in so far as I remember and I'm sure there's at least some things in there worth considering (and I suspect that I've missed a few wonderful things I recommended to other people over the past couple years, but perhaps they'll come and kick me and remind me of them).
I'd also be interested to hear how other Iron Mongers learned what they know about design and architecture - so if you blog about this please drop me an email so I can add your post as a reference at the bottom of here.
Here's to learning!
-- mst, out.