Blog Archives

Small Projects, Big Projects

The most valuable skill for the average software engineer doesn’t get taught in the typical CS curriculum. It’s also almost impossible to teach it to yourself. Nearly every major programming outfit spends a significant amount of time trying to instill it in their employees, and they still often wind up failing to do so. Many programmers don’t realize they need this skill, and look down on those who have it and put it to good use.

You’re probably wondering what I’m talking about. The skill that I’m referring to is the ability to write code in such a way as to make it maintainable and amenable to large-scale collaboration.

I’m going to refer to code written with this skill in mind as “big-project code,” because it’s the kind of code that you need to write to make the most valuable contributions to a project of any significant size (as measured in people, not lines of code). In contrast, code that isn’t written with this skill in mind (or knowingly takes a pass on it) is “small-project code.”

Small-project code is not inherently bad code. There are tons of incredibly useful and clever programs out there written as small-project code. I’ve written many myself. No, the key distinction here is how easy it is for others to collaborate with you on your project.

There are many aspects of code that can vary between small-project and big-project. One example is its mental state requirement: how much of the code you have to hold in your head to be able to reason about its behavior? A program written in a small-project style will often need you to be familiar with a significant portion of its code to reason about what a particular handful of lines will do. Big-project code, on the other hand, will attempt to minimize the necessary mental state involved in figuring out what a particular function’s code does. Consistent naming, more verbose comments/docstrings, and clear object interfaces are some of the factors that play into big-project style.

Another aspect is what you might call walk-in tolerance: how simple is it for someone who has never worked with your code to start making useful changes to it? The lower the mental state requirement, the higher the walk-in tolerance, but other factors come into play as well. Unit tests help boost developer confidence (even for seasoned contributors) by making it easier to notice if new code breaks existing functionality. Style guides prevent bickering over personal preferences and lead to more consistently readable code. Good logging makes it easier to diagnose and debug problems.

The big-project mentality can also be applied at a higher level than the code itself. For example, the concept of a service-oriented architecture is inherently a big-project ideology. It uses well-defined APIs to create a modular environment in which a given developer can focus on a particular service and not have to put much thought into the specific implementations of other services. In doing so, it trades some up-front development time (good APIs take effort to design and implement) in return for a long-term payout (as the overall project grows, the amount of developer time saved not keeping mental state on other services increases drastically).

For the typical hobbyist project or college CS classwork, small-project code is fine – even encouraged. After all, the thing you’re hacking on probably isn’t going to wind up with tens, hundreds, or even thousands of developers contributing to it. Big-project code is a long-term investment, and it only makes sense to make that investment in projects where it will pay off.

For the average software engineer, however, their day job isn’t a hobbyist project. It tends to involve a project with a significant number of developers working together to produce a product that is hopefully greater than the sum of its parts. This is what big-project code is about. It’s about making not only your own job, but the job of others, easier. It’s about spending the time greasing the gears so that when it’s time to get things done, you’re not losing some of your energy to grind.

You generally don’t tend to truly appreciate it (or even get a good grasp on what it is) until you’ve actually spent time working on such a project. Small-project code often feels more “fun” because you’re not spending as much time and effort on long-term investments. It’s easy to shy away from ever working on big-project code because the barrier to entry seems so high – but it can be worth it. In the long run, collaboration often leads to results you never even imagined on your own.

So my advice to you is this:

  • If you’re already a software engineer, try to put some extra effort into figuring out what parts of your work are worth investing in for the future, and how you can approach them in more of a big-project way.
  • If you’re a student, but looking to eventually get a job as a software engineer, consider trying to get some experience with big-project code. One great way to do this is to contribute to an open-source project with a significant developer community. Not only will it give you experience with big-project code, it also stands out on a résumé. Not sure how to get started? Check out OpenHatch.
  • If you’re a CS teacher, consider trying to find a way to work a large collaborative project into your curriculum. An example might be an overarching project for a class with modular services designed and developed by smaller teams of students. Most college CS programs today focus heavily on the theoretical side of computer science and give little in the way of practical programming experience.