By Patrick Moorhead, Contributor
Benchmarks are used in every market as one way to show why one company’s widget is better than another company’s widget. Whether it’s MPG on a car, energy ratings on appliances, or a Wine Spectator rating, it’s a benchmark. The high-tech industry loves benchmarks, too, and there is an industry full of companies and organizations that do nothing but develop and distribute benchmarks. With high-tech benchmarks come controversy, because the stakes are high as all things equal, the widget with the highest benchmark scores will typically receive more money. The recent spat about Intel versus ARM in smartphones illustrates what is wrong with the “system”. I want to explore this and offer some suggestions to help fix what is broken. The first thing I want to highlight is that getting benchmarks “right” is important to the buyer. If a buyer makes a choice based on a benchmark and either the benchmark isn’t representative of a comparative user experience or if the benchmark has been manipulated, the buyer has been misled. The first case would be like a buyer buying a racing boat based on the number cup holders and the second would be an auto MPG test during 100 mph tail winds. The smartphone benchmark blow-up has accusations of both and reminds me a bit of a Big Brother episode. …read more
Source: FULL ARTICLE at Forbes Latest
