How do you evaluate new technology?

1. What are your requirements? What is most important and where can you compromise? Spend some time thinking about this and talking with other stakeholders. Most projects like the one you're embarking on fail or fail to meet expectations not because the wrong technology was selected, but because of a misalignment between what you need and the solution you selected.

2. Create a simple spreadsheet that lists these key requirements along the left column, and each of the solutions you'll be looking at along the top row. Then rate these solutions based on your requirements. There are lots of web sites that list solutions in any category, including this one. Just do a search on "SCM comparisons" or "version control technology," for example. Find a comprehensive list and don't just look at the first 5 solutions listed. There are some terrific niche players out there whose products could align perfectly with how your team likes to operate.

3. Be disciplined in filling out the spreadsheet. If Product A has lots of cool bells and whistles that would be fun to have but doesn't meet certain core requirements, you've got to be willing to let it go. See my comment in #1.

4. In my experience, you'll find you've eliminated all but a couple of solutions. This is when you start looking at Total Cost of Ownership (TCO) and Return on Investment (ROI). TCO is essential if there's any kind of disconnect between what the executive who is going to sign the check thinks a solution is going to cost, and its true costs, and this is very often the case. The second biggest reason many projects fail to meet expectations is because they cost more than the execs were led to believe, and they get cut before they realize their full potential. I have a blog on this topic that lists all of the items you need to consider when building a TCO; it's at http://www.InterimTechExec.com/blog in the Business Case category. How do you compile the numbers? You ask your IT department and your vendors a lot of questions, and it means really knowing your requirements in order for them to be even reasonably accurate.

5. ROI is needed when the business case for new technology isn't apparent and may not be worth the investment. In the case of product development tools, if you're talking software and if the problem is really SCM policy, i.e., not everyone is using the solution in the same manner, then no new technology is going to fix it. The team has to get together and reach agreement about policy. I have a white paper about this I can share with you, email me at sbrown@InterimTechExec.com if you're interested. An ROI shows how much you have to spend, how much you can expect to gain (in productivity, time to market, or whatever your company values here), and how long it will take to meet that break even point.

I hope this helps to get you started. Please don't hesitate to contact me if I can help you further.

Susan Penny Brown
Interim Technical Management, Inc.