• It works, and continues to work — with five-9 availability — at the enormous scale required by carriers
• It solves a real world problem
• It contains sustainable differentiation
• It has no “single point of failure” (i.e. single external dependency) on which is relies to deliver value
• Avoids “specials” — one off development for a single customer
• Reflects input from customers and customer advisory boards
• (If a platform) provides robust, comprehensive SDKs
• Easy to install, particularly for PoCs
• Easy to configure, customize, and integrate
• Easy to update

I was thinking today of the ideal product management team structure, building on my assertion that the product management primary role is to define successful products. I tend to agree with the “Triad” structure promoted by the Pragmatic Marketing folks, in that this structure provides for a manager of each of three important elements of successful product definition: a person to guide overall strategy (in my mind this is the leader of the team), a product manager to define requirements, and a product marketer to tell the world about the product.

So far so good, but I think there’s an important role missing within the triad; namely, a person responsible for ensuring the usability and overall look and feel of the product — a product designer. This person is responsible for defining the experience of using the product, for building prototypes, for conducting usability tests, and for acting as a counterweight to the person defining the requirements. This last is particularly important as it sets up a healthy discussion within the product team, testing whether important requirements are missing (“if we don’t have a breadcrumb, the user gets hopelessly lost”), or whether existing requirements are really high priority from a user perspective (“23 options for report generation will only confuse the user, since only three types ever get used”) .

Among the most consistent experiences I’ve had over the course of my career is an organization’s consideration of user experience as a second, third, or last priority. The “hard part” is seen as the development of the features themselves, with user design seen as a trivial exercise to be addressed sometime later. Incredibly, I’ve seen a company ignore user design even after its enterprise customers refused to purchase a product because it was too hard to use.

I’ve got a theory that this attitude is a symptom of “technical bias” prevalent in high tech, which posits that writing code is hard and important, while everything else — interface design, for example, or documentation — is easier and less important. People that write code are smart, and everyone else is less so. Engineering offers real ROI, and other functions don’t contribute equal value for the money.

It’s certainly true that writing code (at least writing it well) is hard, that it takes a certain cognitive acumen to do it, and that an organization will fail without it. But it’s also true that a development team’s effort is worth nothing if people don’t buy it. Seriously, have you seen people try use a product with an interface designed by engineers? “Wow, check out them elegantly coded algorithms” just isn’t typical feedback.

• First, I believe that the requirements definition process should be collaborative, open to everyone, and written by beginning at a high level overview and moving to the nuts and bolts details through a series of iterations. Dare I say this is a kind of Agile style of product definition?
• A product steering committee should exist (sometimes called a product council). In yesterday’s post I was referring to the product steering committee for the first, high level, review of the product requirements. The PSC is made up of senior stakeholders in the organization and is responsible for the overall strategic direction of the product (it’s also a key component of my philosophy of collaboration).
• Requirements should be defined using many different techniques working together. Detailed prototypes (working is better than paper), essay-style descriptions, use cases, and traditional “the product shall…” requirements should all be used. Depending on the product, one technique will probably be emphasized over the others.
• Requirements should be maintained in an online format available for anyone to see. There are lots of ways to do this (commercial software, wikis, trackers, blogs, etc.). I have my favorite (alluded to ad nauseam elsewhere).
• Requirement definition has a finite time line. You can’t succeed by arguing endlessly about a requirement, and it is very easy to fall into this trap (By the way, did you ever notice that the first 10% of requirements in a review get 90% of the time, with the remaining requirements getting rushed through as people get tired? Put really important requirements toward the end if you want to improve the odds of avoiding an argument about them) . They must be a well understood deadline, after which requirements are “frozen.” “Frozen” means that any changes to the requirement must be conducted through a change control process.
• Finally, and this isn’t mentioned or implied in my previous post, but it’s important: the test cases built for QA must map to the requirements. Product management is responsible for making sure that this happens (because, believe me, no one else will bother). If a feature, failure scenario, or edge case behavior isn’t part of the test case, there is high risk that it won’t be in the product, either.

For the sake of this discussion, however, imagine that you already have solved these problems. You have, whether by flash of inspiration or painful research, conceptualized exactly the product that meets all your business and customer needs. The problem now is how to effectively communicate this product so that engineers can build it.

There is a catch-22 to writing product requirements documents. On the one hand, the document must be detailed enough to define all the features and functionality you are looking for, all inherent (but non-user facing) capabilities of the product (availability, scalability, redundancy, etc.), corner cases, failure scenarios, installation procedures, etc.

On the other hand, people need to read and understand what you are asking them to build. And the more detailed a document you write, the less likely it is anyone will bother to carefully read it. This results in painful meetings in which the document is projected onto a wall to be read and discussed in real time, lots of misunderstandings, increased functional bugs, and lots of wasted time.

For example, the first product requirements documents I wrote was a carefully researched document with more than 200 pages of entries that looked like this (taken verbatim from my effort):

ID: 379
Priority: 1
Requirement: The EMS shall support ITU X.733 based fault processing
Additional Details: The ITU X.733 specification (see appendix D) defines the core approach to fault management and specifies the classes and probable causes of faults.

Not only is this just one of many, many requirements, but understanding the requirement forces the reader to research the ITU X.733 specification. I worked long and hard on this requirements document, and I naturally assumed that everyone else in the organization will have worked just as long and hard to read it. Entering that first PRD review meeting I expected some short discussion about the requirements, with frequent exclamations about how good a job I had done. In fact, no one had even read the document, and the meeting became a good lesson in product management.

Thus the catch-22: requirements need to be specified in detail to enable the product to be developed correctly, but a detailed requirements document isn’t effective communication — it doesn’t foster shared understanding.

So how to solve this catch-22? All stakeholders must understand and accept the product to be developed, but the painful details are important — the ITU X.733 specification really did matter to my product!

The first order of business is to make it easy for people to find and read the requirements, which is why I advocate writing them in an on-line tool (I’ve discussed the tool I use in in a previous post).

Then, I try to follow the following steps (acknowledging that it’s a little more messy in practice). In a nutshell, I use a iterative series of increasingly detailed requirements and meet with stakeholders at each step:

1. I organize the product requirements into high level themes and build a short document (actually a page in my online requirements tracker) that lists these themes and provides a paragraph of verbiage explaining each one. The themes usually include high level sketches of use case (just defining a user and the user’s goal related to the feature)
2. I email the link to this page to the product team and organize a meeting with team leaders to review the page and discuss the themes.
3. I iterate the requirements by expanding on the user/goal lists with use cases for each theme. If the product is web-based, I’ll build prototypes of the functionality to complement the use cases. Again,all this work is available on line.
4. I email the link(s) of my updated requirements to the stakeholders, and schedule meetings to review the use cases. Depending on the number and complexity of the themes, I might schedule a different meeting for each theme. During these meetings we:
   • Discuss the use cases and identify corner cases
   • Brainstorm failure scenarios that need to be defined
   • Identify areas that need specific definition (for example, if a use case involves the generation of a report, a requirement must be written that defines the content and structure of the report).
5. I iterate the requirements by filling out the details per the discussion, defining the failure cases, corner cases, and additional requirements
6. I call a final meetings to review and “freeze” the requirements, which places them in change control.

I know, that’s a lot of meetings. The thing is, this method almost always results in fewer and more productive meetings than would otherwise be the case. Too, they are necessary to ensure everyone reads and understands the requirements.

Following this process has many benefits, the most important of which is that it maintains a common understanding of how the product is evolving across the product team. All stakeholders (including executives and field personnel) have at least a high level understanding of the product direction. Engineers have an early view into the product and can start preparing and designing. Perhaps most important, little time is wasted while everyone waits for the perfectly detailed comprehensive requirements document.

These activities, designed to lead to product success and eventual early retirement, did not at all correlate to my actual experience. Product managers live a very non-serial life. Every activity is urgently needed at once, and there’s never enough time to do any particular activity as well as I would like. This was irritating.

If I had a better grip on best practices, I thought, or had a deeper understanding of technology, things would get less messy. I was partly right. I did need a better understanding of best practices, and my efforts to understand various technologies have paid dividends (if not royalties, as in the case of a book I wrote about the mobile internet). Sadly, the messiness continued.

In the fullness of time, however, I came to accept two fundamental truths: First, making software is hard, and you just can’t fit the process into a neat series of boxes — activities will bleed together. Second (really a corollary to the first), trying to achieve perfection in any one stage of a process will cause neglect of other stages, and is thus an enemy of success.

This means that a good strategy lies in ensuring product management activities complement one another instead of competing for time and bandwidth, and that each activity gets enough attention to achieve success.

The best single way to do achieve these goals is to have an open information policy. I ask my team to post everything (work related) they do in a wiki created for just for the product team, and I encourage everyone in the company to do the same. Product requirements are written using an online tool that everyone can access; the entire requirements-writing process is available for all to see (and comment on) in its painful glory.

In this way, my team has the benefit of everyone’s thoughts and input throughout the entire process. The engineering team is better prepared for product requirements, and can get a jump start on design and feedback. The sales team can see product direction and can provide suggestions related to their geography or to a customer. Since everything is online and updated in real time, material is immediately available to anyone who deems it “good enough” for their purpose.

Given two equally smart people, it isn’t true that the one with the MS is automatically the better product manager than the one with the MA. For one thing, and this doesn’t color my opinion at all, I don’t have an engineering background. I have a masters in Journalism and my first career was as a comedian (now retired, and no longer funny).

The notion that an engineering background is the best foundation for product management has two drivers. First, it seems to provide proof of technical chops. I suppose this is true. Thing is, though, all software efforts are unique. A background developing a software product isn’t necessarily relevant to the management of a different product. I’m more interested in experience successfully managing products that depend on different technologies than a background writing code.

Second, there’s the notion that an engineer will be more successful working with the developers that will ultimately build the product. I dunno. Claiming that a person will be more successful brokering relationships by dint of an engineering background seems a stretch to me. Besides, a product manager has to successfully form relationships with many different stakeholders. It’s not enough to be “respected” by engineering. You also have to work well with the sales organization, the marketing organization, the operations folks, and, most of all, the customer.

To me, there’s no specific background or degree that’s better or worse for a product manager. What matters is intelligence (both academic and emotional are equally important). Proven technical acumen — not a degree, but a demonstrated record of successful management of software products. A strong, palpable passion for creating products — show me a person that talks with excitement and passion about previous projects. Communications acumen — an ability to speak well in front of others, to effectively present, and to write well. Demonstrated leadership — I’m leery of candidates that break out the old saw about how the most difficult part of product management it is to “be responsible for the behavior of people over which you have no authority.”

To boil it down: hire people that are smarter than you, and that aren’t assholes.

In Silicon Valley, reqs almost always ask for “strong communication skills” and “solid writing ability,” but in the end its the acumen for technology that matters. What people ask about is whether the person is sufficiently “technical.” So how technical is technical enough?

The answer is that it depends. A product manger’s technical depth should be sufficient to enable them to innovate, to think creatively about how the technology could be used to solve customer’s problems.

Sometimes, this don’t mean a person has to be particularly technical. Web technology, for example, is mature enough for people to speak about it abstractly; to talk about features and functions without knowing or caring how it happens. A product manager for an MPLS router deployed in 10GigE networks, on the other hand, needs solid geek credentials.

A good way to gauge if a person is technical enough for a particular product management role is to ask about how they have innovated in the past. For example, in the context of your particular product, ask the candidate to “give me a couple examples where you were creative in defining product features to solve a customer’s problem, or tackle a market opportunity.”

Not only will this help you measure if the person has the appropriate technical depth, but whether they are too technical. After all, someone with a background developing compression algorithms might not be the right choice to manage a social networking site for pre-teens.

A product manager is measured by the success of the product in the market. An engineer is measured by the extent to which the developed product aligns with the requirements developed by the product manager. A product manager reporting into engineering is set up for failure: product requirements will be developed with an eye toward helping the engineering department succeed, not success in the market.

To be fair, there is one benefit of placing product management in engineering: When the company goes out of business and everyone is saying their goodbyes in the parking lot, people will feel a little better knowing that they consistently developed a product exactly according to spec.

Cam Studio is simple, but incredibly cool: it records the audio and video on your computer screen, and delivers the result it in streaming flash videos that anyone can watch. Create a video of yourself demonstrating your product (speaking into a microphone attached to the computer) and send the video to anyone on your team that might give a similar presentation.

You can do the same for anything that you might otherwise present in person; PowerPoint charts, for example. You can make a video of yourself explaining specification sheets or architecture diagrams. Anything at all. Easy to make, easy to send, easy to watch, and free.

To review. The Waterfall method is a software development process in which a product is created in a series of discrete stages. For example, a set of software requirement are written, then the product is designed, then built, then tested, then installed and maintained.

On paper, it’s rational. This rationality is key to its continued popularity: the many milestones and associated deliverables (requirements documents, design documents, test plans etc.) make people comfortable. There is a clear road ahead, a discrete series of steps toward success, all of which can be measured.

Unfortunately, development just doesn’t work that way. Software is hard, and it’s messy. For example:

• Requirements are incomplete when design and development begins. Feature requirements written at the beginning of a project will always be incomplete because the design process will dig up a many “derived” requirements that were not initially obvious. It’s not possible to think through every iteration, use case, and failure cases of a complex software product in the abstract — things become clear with time.
• Development estimates are wrong. Software estimates (i.e. the amount of time a project will take) are typically performed at the beginning of a software life cycle. Unfortunately, this is when the project is least well understood. Designs aren’t fully worked out. Complexity isn’t fully appreciated. Optimistic assessments are made relating to the eventual quality of the code and the amount of time that will be required for testing and debugging. A corollary to this rule is that once an initial estimate is made, it’s rarely updated as the project wears on.
• Requirements change. Customers change their minds. They add new requirements, they modify old requirements. Often they don’t really know what they want, don’t know how to prioritize, and don’t fully appreciate the impact a change can have on the development process. These requirements changes require time to document, design, develop and test, and out of this process even more derived requirements emerge.

Truth is, I don’t believe the Waterfall method really exists. I’d bet that no software project of any meaningful size has ever been developed in discrete stages. A good product manager understands this, and implements process to accommodate randomness, to anticipate chaos. This flexibility anticipates incomplete requirements, poor development estimates, and changing feature sets.

I don’t mean that a product manager should attempt to overturn process within a company committed to the Waterfall method (by insisting that an Agile process be adopted, for example, or a rational unified process). That’s not usually realistic or possible. You can borrow techniques from other methods, however, and integrate those techniques into the existing process.

For example, I’ve had success implementing a daily “semi-Scrum” meeting with all the stakeholders working within a Waterfall-style project. The meeting fostered a common understanding of every detail affecting the project and forced an action toward resolution. It doesn’t matter what specific methods you use, as long as you can answer the following questions in detail:

• How do we decide what to do with a change request? That is, how do we determine whether or not a change request (a new feature, or a altered requirement) meets business goals and therefore should be integrated into the project, deferred, or discarded?
• How do we ensure that development estimates are accurate?
• How do we manage potential delays in the project, or make decisions in terms of trading quality, feature depth, or delivery time.

Incidentally, understanding these issues is a basic measure of an experienced product (or project) manager. It’s amazing how many people I’ve interviewed that present themselves as experienced and savvy, and can about deliverables, document templates, and milestones, but are unable to meaningfully discuss how to manage the problem of unexpected change.