In-Depth
Contested Code
How TopCoder uses a contest format to build high-quality software for clients.
- By Peter Varhol
- 12/01/2007
By Peter Varhol
In the seminal paper entitled "The Cathedral and the Bazaar," software expert Eric Raymond drew a sharp distinction between the two fundamental processes used to develop software.
In the cathedral, development is limited to a small group of developers who work for the same company. The code tends to be closely held among these developers, and all planning, design, development and tests are confined to this group. In the bazaar, code is distributed out to all and development contributions come from all. This is the classic distributed open source model, where interested programmers take existing source code, understand how it works, and offer enhancements and bug fixes back to the source code base.
Raymond concludes that the bazaar represents a superior development process, based mainly on the observation that greater numbers of developers sharing ideas and code ensures that the best concepts emerge. That's not to say that there aren't advantages in the cathedral approach. For one thing, the closed model has a definition of what must be built and tested in order to satisfy a requirement. The bazaar, with its far-flung dev community working off a loose project plan, is far more chaotic.
This is where TopCoder Inc. comes in. You may have heard of TopCoder, or even participated in one of its software design and development competitions. What you may not know is that TopCoder is also a commercial software company that bids on outsourcing projects from enterprises and government entities.
Community and Cloister
What's the connection between the two activities? TopCoder employs a unique development model that uses contests to bring talented developers into projects, which are shaped and guided by project managers who know exactly what the company wants to develop.
The result? A new approach to custom software development that combines an active and highly competent community with a business model geared toward delivering high-quality software to client organizations very quickly.
Like many custom dev shops, TopCoder contracts with client companies to design and build tailor-made software. To build the software, TopCoder provides a library of software components (both .NET and Java) that are used as building blocks for applications.
The real trick is transforming a software development contest into a process for crafting building blocks for custom software development. TopCoder uses a technique it calls Component-Based Development (CBD). The company's architects have specified a number of reusable components that can be useful across different apps. According to TopCoder Director of Software Operations Tony Jefts, many components are implementations of established design patterns.
The key to the TopCoder model is that the contests represent an endpoint rather than a beginning. Each contest results in a discrete work product (usually a design, a component or an app). Contests span several levels of the software dev cycle. First are contests to design and build individual components. In many cases, these are components that TopCoder architects deem worthwhile for future products. In others, they're needed for specific projects today, but they'll typically be designed and written in such a way that they're useful in the future.
There are also separate contests for design and build. Typically, a TopCoder architect writes requirements and a detailed functional specification for a component and puts it out for design. During a design competition, participants produce a design, including a detailed description with UML diagrams containing activity diagrams and class diagrams.
A design competition typically lasts about a week. A team of reviewers examines the design and selects a winner based on set criteria. Once a winning design is chosen, the designer may be asked to fix some specific issues in the design. The finalized design is usually then sent out into another competition to build it. In this competition, the winner of the design competition becomes a resource for the development contestants through an online forum. The contestants can ask the designer to explain or clarify parts of the design to make sure what is built accurately reflects the design. The build competitions can take anywhere between one and two weeks.
Last, in response to a customer's application need, TopCoder offers assembly contests. These involve integrating existing components into an application specified in the initial interactions between the customer and TopCoder. A TopCoder systems analyst can spend several weeks with the customer, precisely defining the deliverables. If additional components are needed as a result of this definition, they are built in additional components competitions. Once the required components exist, TopCoder runs the assembly competition to assemble those components into a working application.
Assembly consists of piecing components together and configuring them to run on a particular hardware, operating system and middleware platform. Components generally have interfaces that make them relatively straightforward to work together, so the bulk of this work is in designing database structures, configuring Web servers and executing the flow of the app.
A key to each of these competitions is scope. Each contest project has to be structured to fit into the standard timeframe expected by the participants, typically one to two weeks. TopCoder pays more in prize money for projects it deems more complex, although it still has to scope them properly. Participants won't sign up for a competition that they believe requires too much effort for too little potential return. TopCoder recognizes that, too, and has been known to pull back a project to either redefine it or to increase the prize money.
TopCoder typically has several competitions going on at any one time. Community members have been known to participate in several at one time. "I've sometimes done five or six in a week," says Piotr Paweska, one of the leading designer contestants. "You basically don't sleep during that time."
Most community members specialize in one aspect of the competition. Paweska, for example, focuses primarily on design. He started with Microsoft designs, switched over to Java for several years, and is in the process of switching back to primarily Microsoft and .NET. "Visual Studio is a great environment to work in," he explains. "Microsoft really knows how to make the development process productive."
Prototypes are an important part of design competitions, and Paweska uses Visual Studio to build prototypes of his .NET designs. Prototypes provide information that development contestants can use in building components. From the designer's standpoint, prototypes lessen the need to answer questions during the development phase.
The Community Delivers
What came first, the client or the community? In TopCoder's case, it was the community. The company invested its initial efforts into finding and encouraging a community of developers around the world to build the very best software in response to exacting needs and standards.
Justin Gasper is a typical TopCoder community member. He specializes in designing .NET components, working extensively with Windows Communication Foundation (WCF) and Silverlight. In fact, Gasper has been participating in TopCoder competitions as his full time job for well over a year. "TopCoder lets me learn new things, and I actually make more in the contests than I did as a full-time developer, even though I work less hours," he says.
Few of us have a good idea of how to build a dispersed yet highly effective developer community. In many cases, the community just seems to develop if the project is interesting and explained well. It also helps if the project leaders are able to articulate a clear need and vision for the project.
With TopCoder, the hook is the contest. Participants register for the contest and are able to see requirements and design documentation, depending on the nature of the program. There are also organized competitions, including several university competitions.
What kind of customers turn to a contest to outsource development? AOL LLC, for one, which first engaged TopCoder on a low-priority project called the Extensible Messaging and Presence Protocol (XMPP) for its instant messenger (AIM) client. The company planned to execute the project in phases, according to Technical Director Nic Perez, but TopCoder asked for the entire project in a single phase. This increased the cost of the proposed initial phase but reduced the cost of the entire project.
Perez spent several weeks working with a TopCoder analyst to define the project, an effort that helped yield a better end product for AOL. According to Perez, the quality of the typical TopCoder project is better than the average for its projects as a whole. The price is "very competitive," and the schedule is aggressive.
The TopCoder approach isn't for every project, Perez notes. "If you're not sure what you want, TopCoder isn't the company to use. The definition phase will take far too long."
More generally, the approach is clearly not an agile-style methodology, which tends to emphasize iterative development, fast turnaround of incremental new features and extensive end-user involvement. The planning stage is front-loaded and opportunities for in-course changes are limited.
Companies like AOL have found they can use the TopCoder community to help explore possibilities. AOL is currently building up its own library of components, which the company plans to make available to partners to create a useful development platform. It also goes to TopCoder participants to get feedback on future projects.
"We could hire a market research company to conduct in-depth interviews for $200,000," says Perez. "Instead, we ask the TopCoder community for their best ideas, and pay $6,000 in prize money for the best ones. And this is exactly the audience we want to reach with new products."
[click image for larger view] |
| Many coders come from outside of the United States, where contest winnings can be substantially larger than the pay for typical programming work. |
Ultimately, TopCoder's success is built on careful planning and definition. It's ironic that this reflects how software development is taught in colleges-painstakingly collecting and understanding requirements and focusing primarily on specification and design. With a good understanding of what the application is supposed to do-and a precise design-coding is supposed to be almost a formality.
In the case of TopCoder, it's actually a contest. But so is the design process and the application assembly. What really makes all this work is a worldwide community of highly competitive software designers and developers, who find a challenge in producing the best software components quickly and getting rewarded for doing so.