On your mark, get set, compute
Seattle Times technology reporter
There are no checkered flags, no exhaust fumes and no shirtless fans holding cans of Budweiser at the races where Microsoft proves the mettle of its industrial-strength software — only a roomful of humming computers.
But the Redmond company and its rivals treat these competitions, called benchmark tests, just as seriously as Ford does stock-car races.
"If you're in, you're in to win,'' said Paul Flessner, the senior vice president in charge of Microsoft's enterprise-server business.
So far, Microsoft has won most of the categories it has entered, helping the company prove that its software is powerful enough to run network servers as well as personal computers. With the prize being a bigger share of the $60 billion server market, Microsoft aims to use benchmarks in the coming year to show that its software can run larger and larger networks — and perhaps win customers from rivals such as Oracle, IBM and Sun Microsystems.
Benchmark tests simulate a computing situation, such as a bank running ATM machines or a Web site selling books, then measure how fast the systems can process data requests. They also factor in the price of the system and rank performance in terms of the cost per inquiry processed.
Working with computer makers such as Dell, Compaq and Unisys, Microsoft has built systems that dominate some benchmark categories run by the Transaction Processing Performance Council (TPC), an independent testing organization based in San Francisco.
A typical test, usually at a company's campus, may involve $10 million worth of computers and software put through their paces under the eye of a TPC auditor.
In the test simulating an online bookstore, a system must simultaneously process orders, track inventory and support an online catalog being perused by numerous shoppers.
To prepare, Flessner may have 10 engineers spend four or five months on a benchmark project.
Many experts scoff that the tests are as unrealistic as eyeing racing cars not available on the showroom floor. But customers check the numbers the way car buyers peruse road tests to see how fast different models go, even though they may never push their machines as hard.
Benchmarks also help sales staffs differentiate their products, and they're another way for ultra-competitive companies run by lead-footed tycoons to challenge each other on neutral turf.
"You can think of benchmarks as an Indy car race — they're done in a specialized, laboratory-type setting," said Jean Bozman, an analyst at IDC, a Framingham, Mass., research company. She follows the market for computers and software aimed at large companies or institutions, known as the enterprise market in the trade.
Bozman said the results may be useful when shopping for a system, but big customers are likely to test the systems themselves.
Microsoft executives also say benchmarks are just one selling point for the database and server software that have helped the company evolve from a desktop software business into a leading provider of business network software, as well.
Importance of Windows 2000
A crucial turning point came in 2000 when the company introduced its Windows 2000 operating system, a server version to run business networks and an advanced server version for more demanding applications.
At the time Microsoft was still working to prove that Windows was powerful enough to do tasks typically handled by costly "big iron" mainframe computers.
Using clusters of linked computers running Windows and its SQL database software, the company outperformed Oracle on a TPC benchmark simulating a warehouse-management system.
Results of one test were done just days before Microsoft Chairman Bill Gates launched Windows 2000 at a conference in San Francisco. The night before the launch, Flessner and his team stayed up all night running another test that resulted in even better performance and had the results certified by a TPC auditor at 4 a.m.
The first test showed that an eight-processor system could handle 150,000 transactions per minute, but the second one showed that the software could run a 12-processor system doing more than 227,000 transactions per minute.
"That's enough to handle all the e-commerce done on the Web last year in two days, and so there's a lot of headroom in that performance capability," Gates said in a speech that featured legendary rock guitarist Carlos Santana.
The tidbit barely made it into the speech. As Gates and Santana were preparing, a haggard Flessner rushed in with the results but could not get backstage to Gates. "I got blocked by Santana's security," he said.
Eventually they relented and the show went on.
A long path
It took a decade for Microsoft to establish itself in the enterprise market but, with PC sales waning, the effort is paying off and providing a crucial revenue boost.
It also guarantees that Microsoft won't become irrelevant, as computing moves further from the desktop to Internet-connected networks.
Enterprise sales grew from 16 percent of the company's total revenue in its 1998 fiscal year to 19 percent in fiscal 2001. Year-to-year sales rose 18 percent from 2000 to 2001, compared with 3 percent growth in PC-application sales and 15 percent in Windows platforms.
Meanwhile, Microsoft's share of the worldwide server market — where it competes with Unix, Linux, NetWare and other platforms — grew from 40 percent to 54 percent, when counting unit sales, from 1998 to 2000, IDC said.
Microsoft is on track to own the enterprise market just as it does the PC software market, said Rob Enderle, an analyst with the Giga Information Group based in San Jose.
"It's never been a question of if, it's just a question of when," he said. "Given their progress, it's not only probable, it's more than likely they'll be the dominant provider, even at the high end."
The company approached the enterprise market the way it did PC software in the 1980s — by introducing low-cost, high-volume products into a market dominated by expensive products offered by more established companies.
That strategy was overshadowed in the dot-com boom years, when cash was easy and companies were shopping for Ferraris instead of Chevrolets, but it resonates now that chief information officers are focusing more on value than top-end performance.
"I think it's a real selling point in this climate,'' said Jim Moore, an analyst at Deutsche Banc Alex. Brown in San Francisco. "There's no question that as they get more credibility in the back office, at the same time the CIOs are trying to save money, that business should remain very strong for the company."
Price "may get you in the game, but no one buys you just because you're the lowest-cost option,'' said Barry Goffe, group product manager of the .NET enterprise solutions group at Microsoft.
Goffe and Flessner said companies weigh purchase price with the cost of operating a system and the system's capabilities and flexibility.
"You can be free, nobody cares," Flessner said. "You've got to have all the proof points."
Different kind of competition
Even though he's a speed demon on the TPC benchmark circuit, Flessner doesn't share Gates' penchant for fast Porsches. He drives a 1993 Ford Ranger pickup.
Flessner is a laid-back, 42-year-old Midwesterner who spends free time tinkering in his Sammamish woodshop and helping a son build computers from scratch.
Still, Flessner is intense at work, where Microsoft is trying to build bigger, faster setups to compete in the market for huge systems dominated by Oracle.
Oracle, NCR, IBM and Sun Microsystems have a lock on TPC benchmarks for huge databases 1,000 gigabytes and larger.
On Jan. 17 Oracle posted a record in this category with a $5.4 million, 72-processor Sun server setup.
"With this latest result, Oracle continues to strengthen its lead in benchmark performance, leaving IBM DB2 and Microsoft further behind," the company said. "Microsoft has yet to publish a benchmark beyond the 300 gigabyte scale factor."
But Microsoft is expected to become a contender when its servers can run on a new generation of computers based on a powerful new Intel processor being released later this year.
Microsoft dominates tests for smaller, 100 gigabyte databases, when the systems are measured by their performance relative to price, but the top slot for systems regardless of price was nabbed last spring by a Linux-based system.
Flessner said the competition also motivates employees. That's especially useful in a business where it may take a long time to score a "win" over competitors.
"At first people just didn't believe benchmarks. They thought it's just a stunt,'' he said. "But now they know it's real."
Brier Dudley can be reached at 206-515-5687 or firstname.lastname@example.org.