Traditional management and software projects simply don't work well, and sometimes, not at all. The fault is often ascribed to recalcitrant programmers, but at best, that is only a partial answer. The real answers are more complex.
There are three parts to good software project management (SPM). They are motivation, management structure, and communications.
Management structure that comes from traditional organizations has been slowly changing as the critical value of good information has become obvious to the executive levels. A CIO (Chief Information Officer) is no longer a novelty, but the structure below this level has rarely varied from the traditional hierarchical form of a corporation.
This creates a clash with people who work in the IT industry because of the non-clerical nature of the work. Everything in IT, with rare exceptions, requires a level of knowledge beyond everyday actions and understanding. Training, both education and OJT, is required for effective performance in the IT industry. Most people who enter the industry are bright and ambitious, though many lack social skills and awareness.
Programmers are labeled 'Hard to manage' because they are motivated by different incentives from a lot of people. New challenges and learning are part of the typical motivation of programmers, and if they don't get that, often leave for apparently greener pastures elsewhere.
Even if those elements are supplied, there are still major obstacles to programmer motivation and retention. These obstacles fall into the category of having to fight the corporate system to get work done. From a programmers point of view, the corporation structure is part of the problem. Corporations usually just say the programmers don't understand corporate culture and dismiss the complaints as invalid.
The corporations cannot see that from a SPM view, the programmers are right. The nature of software projects is that they affect, directly and indirectly, the whole corporation even if it is only a single application. Hidden in a software project are a set of assumptions about the environment, including corporate policies.
Corporations change policies based mainly on how to get the human side of the policies implemented. The problem here is twofold - first recognizing that policies can impact software (and vice versa), and second, finding and changing the software to reflect the new policies.
Few programmers think about corporate policies when designing software, or what impacts policy changes might have. When that change happens, urgent demands for modification of software result, disrupting current work and causing side effects I'll discuss later. The real source of the problems are the hidden assumptions on both sides.
Corporations assume that policy changes either don't affect software or only require minor changes. Program designers usually don't list assumptions or policies as part of the program design process, and proceed to lock those unstated assumptions into code. Not surprisingly, changing assumptions cause major rework because the design is often affected.
A programming manager has a job similar in concept to a one man band. Everything in the programming area is his or her responsibility, yet rarely does anyone have an adequate background for that broad responsibility. Typically managers are either former programmers, or managers with limited technical background. Neither background is adequate preparation for managing programmers and projects together.
The main missing element is project management, as in engineering projects. The concepts of critical paths, task estimation and dependencies are barely understood in most programming groups. Planning projects can be as little as 30 minutes spent in guessing how much time each module will take to write, with no consideration of dependencies, testing, documentation or training. I've seen that happen.
Given the lack of project management experience, missed deadlines and poor software should come as little surprise. But time after time, software is delivered late, over budget, incomplete and buggy. Sometimes the problem starts with management saying "We need this in nine months. Can you do it?" It's all too human to respond "Yes" even when the details have not been thought about, never mind subject to analysis.
To truly solve this problem, we need engineering style project managers. People whose primary responsibility is to make sure nothing is overlooked in the plan, deal with problems, and accurately track progress. These project managers should report at the same level as the programming manager. They should be the primary contact with the main client and financial backer for any project. They must have final say over project plans and schedules.
This is a large order, and people with those credentials earn high salaries with good reason. Employing a real project manager means a much higher quality and success level with fewer overruns. The more complex the project, the bigger the need for a good project manager.
Project managers deal with the technical part of the job. They don't replace the programming manager. Programming managers will remain with administrative authority and organizational & staffing jobs. This is still a full time job, but now it can be handled by mere mortals.
Programmers come and go, projects start, develop and finish. Rarely are the two in synchronization. The programming manager deals with the staff and organizational continuity while the project manager deals with project continuity.
The programming group should be divided into teams of two to four people. The team members need to be comfortable working together and should share a common area, ideally a private office. Each team has a team leader who is responsible for mentoring and keeping programmers on track. The team leader also writes reports for programming and project managers and meets with them on a regular basis.
A team may be dedicated to one project, or more typically, working on more than one. Teams usually share an area of expertise such as database, and they can discuss options and get help right within the team. This tight communications helps if a member is sick or leaves. Since the leader is on top of the status, new members can be brought up to speed quickly.
A team offers continuity in the technical areas, which reduces the impact of turnover. Teams also foster learning and group support, which should reduce the need for programmers to move elsewhere.
Summing up, a programming group consists of a programming manager, one or more project managers, and several teams of programmers, created around areas of technical speciality.
Even with all of the above tools and structures in place, there is one more necessary element. That is good communications.
What do I mean by 'good' communications? Primarily two things - clear written documents that state the project requirements, and close communications between the client and the people implementing the project. This is the realm of the project manager.
The problem with oral communications is that it is unreliable and subject to different interpretations. If you have ever played "Telephone," a game where one person whispers a sentence to the next, you know this effect. Once a sentence has gone through three or four people, the result is nothing like the original. The classic cartoon "What the customer wanted" is another take on this problem.
The only way to avoid this problem is to develop the specification in writing and get it signed off by client and project manager. While this will take time and effort, it will result in time, money and aggravation saved by not having to change and redo things that were unclear or unspecified.
Exactly what should be contained in that document, and how it should be used, is the subject of the four part BSOF series. Here is a summary of each part.
There are three main problems in developing custom software:
The essential tools for quality software are clear communications and good organization. Most of the problems that arise in a project are caused by misunderstood or missing information as the result of faulty communications. This occurs in each step of the project and studies have shown that the earlier in the project that an error happens, the more costly the cure.
As Brooks has said "The sooner you start, the longer it takes". _The Mythical Man-Month_by Brooks should be required reading for any quality conscious software development. He wrote the best early advice on what to avoid, but advice on how to properly develop software has been harder find.
Building software is analogous to building a house: It starts with determining what the customer can afford (budget), wants (requirements) and proceeds to details (specification) and then to construction (design & implementation).
After the building is done, it is inspected (quality assurance) and any problems (bugs) are fixed. Finally it is delivered - late and over budget happen here too, for much the same reasons as software projects.
The Project document is like a contract between the architect and the future homeowner. It is basically a management document, dealing with financial and authority issues, and procedures to resolve problems. This document should identify the customer, the financial supporter, the project leader, all of the affected groups and a definition of the customer's goal. Current costs should be identified and the planned cost and schedule laid out, understanding that once the RSD is done, both may have to be revised.
The problem you face in writing specifications is that it doesn't seem to have an obvious payoff. It is easy to write up something that covers the ground but misses important points. The information below examines the eight points of specification and gives background for how and why they should be entered.
Requirements and Specifications form the basis for the Design Document(DD). This is the point where the RSD process starts to pay off because the DD is the critical first step in creating a reliable, long lived application. Any information that is missing or incomplete when the DD is created can have a huge impact on the cost and success of the whole project.
Studies have confirmed that the most expensive errors to fix are those which are missing or wrong in the specifications. The reason for this fact will become clearer when you understand how entropy affects program costs, and how the DD enables you to keep entropy from destroying systems as they age.
You've no doubt seen the phrase "Quality cannot be tested in". Bugs can be eradicated, but the essence of quality is knowing what you are doing every step of the way, and communicating information in a clear and unambiguous way.
Quality software requires a process to insure all the bases are covered. Delivering on that promise depends on the final step in the Parallel Development Process, the QA step. QA involves testing but is very different from the testing programmers usually do.
QA is testing done according to a strict script that has been developed independently from the programming and from the exact same source of information, the Formal Requirements Document (FRD). This process insures an independent verification of the program's function, sometimes called Verification and Validation.
All content on this site is Copyright 2001 - 2003 by Bill Nicholls