an obession with first principles

Versioning and Releasing

Posted: Thursday Jan 6th | Author: JohnO | Filed under: Leadership, Management, Programming | View Comments

OK, I wrote about this a while ago. I still see lots of people wondering about just how to do this. The local Boston django group has about four people who submitted ideas around project management and deployment.

In the abstract you really need three things: 1) a distributed version control system (I prefer git), 2) a database migration system (in the rails world, rake, in the django world south, in the php world you can use manual sql files for both up/down migrations with a simple DB table keeping track – it is essentially what rake and south do, only without an ORM), and 3) a system that gets you onto all of your servers to run the necessary commands/scripts (again, in the rails world capistrano, in the django world fabric, in the php world make some shell scripts).

I have found it easiest to create feature branches in git for new features. Having your main production server running off of master. To write new features you just create a branch off master. If these features need qa, or user acceptance testing, you can create a point release branch to push to those servers only the features which you need at any time (or if it is not that busy, just push the feature branch).

(Aside: I always work in a local branch. So if I am working on master features. I create a master_local. If I am working on feature_a which is a remote branch, I create feature_a_local. This enables you to rebase upstream changes from master/feature_a without messing up the distributed history. It also prevents the useless commits from when pull causes a merge.)

I am against a release branch. I am fine with QA/user acceptance testing branches. This is the difference. If the QA branch gets messed up – fine, create another one. If the branch for the client’s preview gets messed up – create another one. (If you’re squashing your commits it makes it even easier to keep track of things). But once something is ready for release you just don’t want to touch it. Only things you are absolutely positively sure of get on master. Nothing else. That is an easy rule to enforce. Master is production. You mess with master you’re in trouble. With point release branches you get the endless wondering of “wait, which version is production, and which is QA, and which is the clients?”. It is fine that a QA or client branch get messed up because someone merged to the wrong point branch. But when you have master checked out and you type `git merge` you better be doing the right thing. It is a lot harder to confuse `master` with `` vs ``.

Some people worry about reverting. With git it is easy to revert. If you are pushing lots of things you should be squashing your releases down to one commit (that is, when you merge your feature branch into master, squash it first). So if your release goes poorly you can migrate off whatever you need (or don’t), and then git checkout the patch before the release.

Some people worry about atomicity. Git is built from a filesystem perspective so there is no lag time where any files are out of version with one another. In any case, once you’re making DB changes in addition to your code pushes – any atomicity (in milliseconds) is ruined since the DB takes seconds (in the case of large indexes, perhaps even minutes) to get up to date. If you’re worried about atomicity stop the webserver gracefully. If you’re worried about down-time – you shouldn’t be. If you gracefully take apache down, and seconds later bring it back up it is unlikely any user will ever notice. If you’re worried about it – release when your system is least in use. (In the case of rails and django you have to stop the webserver when releasing to ensure that your updated source files get re-compiled.)

And I will add one more tip for client caching. Users ought to cache this content aggressively. As a developer it is your job to deliver the new application code to them. You will want to add a version to the querystring to all the CSS and JS that you send out in your application. So when you make a change to a statically served file, you change the version on that querystring (in a settings file) so that no user has this cached.

All in all I really don’t see the huge headache in this area. I hope that we (Active Frequency) can put together a presentation on the specifics of our django deployment process (fabric, south, git/mercurial). (We also use virtualenv to sandbox our pypi and django version dependencies.)


Posted: Thursday Jul 29th | Author: JohnO | Filed under: Anthropology, In the News, Leadership, Programming | View Comments

I’m floored over this article for two reasons. First, Jason of 37 Signals tweeted it. This shows some element of critical thinking and acceptance of the limitations of his advice. Second, the article itself (which is the more important point) posits a fantastic distinction that we need to pay attention to. I will be paying attention to it, at least.

What people will pay for a class of good is a big deal. Forgetting which class you’re put in is a giant mistake. The barrier for entry into social spaces is nothing. It costs nothing for friends to get together and hang out at someone’s place. You can’t be more expensive than that. When it comes to work, however, people are more than willing to pay for something that will make them money. Saving money is the same as making money.

Never forget this rudimentary lesson of pricing. It makes me think about what kind of product I would be inclined to put out there. One where people will pay me directly, or one where I will be indirectly paid. The formula for any indirect payment seems much more vague and hand-wavy. The formula for direct payment is very straight forward. It is a harsher reality, because if you’re not good enough, you’re not getting paid. I like that. If you have a social media service, you might have lots of registered users. You might have some traffic. You might get paid. You might not. But you’ll never know if you’re actually succeeding at your task. Meaningful metrics are much harder to obtain for you.

Be Wrong

Posted: Monday Jun 21st | Author: JohnO | Filed under: Leadership | View Comments

I want to be successful. We all want to be successful. We have to be wrong. Take this on for size:

I had this experience a couple of years ago where I got to sit in on the editorial meeting at the Onion. Every Monday they have to come up with like 17 or 18 headlines, and to do that, they generate 600 headlines per week. I feel like that’s why it’s good: because they are willing to be wrong 583 times to be right 17.
Ira Glass

The Onion is a hands down success. Everyone recognizes their brilliance. They are wrong 583 times per week. And they know it. They relish in it. Being means they can be right. I have to remember that being wrong is part of being successful. If you are never wrong – how do you know you are right?

300 words on Education

Posted: Monday Jun 14th | Author: JohnO | Filed under: Leadership, Philosophising | View Comments

There is a 300-words-a-day project in which some great writers are participating. One on education caught my eye this morning:

As I was reading the paragraph, I could not help but notice the parallels between the state of education in Byzantium circa 775 and the US today. How many of our civil servants are truly wise, capable of understanding the law and applying it, understanding history’s role in shaping where we are today, and able to govern effectively? Furthermore, study after study shows that our population is getting less intelligent with every generation. What value does our education system put on philosophy, language, literature, the arts, and preserving the American culture? Scott Barstow

Now, I couldn’t care less about American culture. In my opinion, there are only a few things about it that are redeeming, the rest is mostly garbage. I do, however, lament his point about the education system. It is incredibly likely that I will not attempt to pursue (and therefore somewhat likely that finishing the MTS is a priority) a PhD because there is no where to use it.

On Education

Posted: Thursday May 13th | Author: JohnO | Filed under: Leadership, Philosophising | View Comments

Seth Godin, as a marketer, talks about education. Education has always confused me. They told me I went to one of the best schools on Long Island. I graduated in the top 25% of my class. I slept through two of my senior year AP classes. I’m not that smart. I realize there are many people much, much smarter than I.

The commoditization of education, as Godin points out, is a massive problem. Those people who, of course, seek to consume commodities (the helicopters) they seek out what they know.

There are many in my generation, and in the next it seems, who are going around this trend. They appear to be happier. This is a present state of being, who knows if it will last. The massive difference is that they, for the most part, are not building wealth (not money, mind you) in where they are. Some, particularly the entrepreneurs, are building wealth. More than a few continue to go on (myself included) getting graduate degrees. Finding the amazing professors who can teach you is certainly the right way to go about it.

It seems to me that the answers, tentatively, is to find someone who can teach you. Whether it is a professor, a professional, whoever. Find someone who has accomplished what you want to. Get to know them. Get interested. Ask them questions. If they are passionate, and you are passionate – they will help you along. That is how you learn. Through people.


Posted: Tuesday Feb 16th | Author: JohnO | Filed under: Leadership | View Comments

It is amazing, yet true: “The rest have allowed themselves to be brainwashed into compliance.” We believe that we “communicate well” on a piece of paper, a resume. We believe that our resume actually tells the story of our work. Godin tells us “You are not your resume, you are your work”.

The key lies in making that work extremely visible in many different ways. Not just one static piece of paper. Don’t play the game that everyone else is playing – the odds are stacked against you. Make your own game, your own space, where no one could ever best you. Then people will see your work. The principle is simple, executing on it is very hard.


Posted: Wednesday Jan 20th | Author: JohnO | Filed under: Leadership, Management | View Comments

As companies expand, the people within them start to specialize. At such a point, some managers will conclude that they have a “keep everyone on the same page” problem. But often what they actually have is a “stop people from meddling when there are already enough smart people working on something” problem.

And on every project, assign one person to make sure that communication happens — but only the right communication. Otherwise the team will just start having long meetings with everyone there and, frankly, people will socialize, and bloviate, and speechify, and argue about things they don’t really care about just to hear their own voices.

Ah, Academics

Posted: Monday Oct 19th | Author: JohnO | Filed under: In the News, Leadership | View Comments

Lessig has to further explain what he was going on about in his last piece. Much like most people misunderstood Archbishop of Canterbury Rowan William’s talk about the place of Sharia law within England, I feel that others have done the same to Lessig.

To one degree, it comes down to the difference between academic argument – and don’t forget Lessig was, or still is, a law professor – and the sort of argument you’ll find on Leno, Letterman, or even O’Reilly. No amount of framing, nuance, or subtlety seems to insulate oneself from being entirely misunderstood and caricatured.

It really makes me wonder, as I certainly approach things with the academic method, how to effectively communicate outside the academy. Where effective doesn’t mean getting slammed from both the right, left, top, and bottom, because you might not be an ideologue, or whatever reason. I imagine it requires some sweetness of the tongue that is rather unbecoming an academic. If we all had it, we might as well have been calculating CEO’s I guess.

On Clients

Posted: Monday Sep 14th | Author: JohnO | Filed under: Leadership, Management | View Comments

Do not ask your clients what they want. Tell them, and sell them, what they want. If you know your clients and your market – they will buy it. To ask them what they want is to refuse to do your job as a product manager. You are asking your customers to do your job for them. And they are not going to spend forty hours a week at it. That is all I have to say about clients.

Team Behavior

Posted: Monday May 18th | Author: JohnO | Filed under: Leadership, Philosophising, Power | View Comments

For nearly all of my life I’ve been involved in organized sports. I’ve played baseball, basketball, and ultimate in more formal settings, and soccer and football in informal ones. And now, in my short life, I’ve been fortunate to work for several, diverse, employers. Being the observant fool that I am, I cannot help but make a big correlation between the behavior of sports and management teams.


Whoever takes a position of leadership, de jure or de facto gets imitated. If they are nit-picky, verbose, and overly analytical your entire team will become that. If they are engaged, calculated, and careful how they speak your entire team will become that. The same goes for companies. The most influential culture gets created by the leaders. If they work with one another in personal attacks and silly arguments – that will be your company culture. If they work with ad-hoc, loose, and overlapping responsibilities – that will be your company culture. If there are clear, and refined boundaries for ownership, responsibility, in the management team the rest of your company will imitate it.

This works for both better and worse. It is why ‘groupthink’ can be such a systemic problem. The worldview in which decisions are made is never re-assessed. New data is simply slammed into the old view, resulting in poor decisions and frustration from everyone.

An Example

This past weekend we played at Bellcrack in Philadelphia. We played extremely well while losing. We were outmatched by many teams, but we kept our composure and made good decisions. But while we were winning some players started getting excited and very vocal. Once something went wrong, they took a leadership position by speaking out, and speaking out in areas they don’t know very much about. It really turned the team dynamic sour, and turned into arguments about fine points that just did not matter on the field one bit. People started imitating the vocal ones by being vocal themselves and taking the bait entering into arguments. Not because they were leaders, not because they were excellent players, only because they appeared to be leaders by being vocal. So the entire team got analytical and snippy, instead of letting the actual captain set the tone. Unfortunately our captain ended up getting sucked into the arguments as well and we would play worse and worse.