the Hidden Realities of Computer Industry in Japan Japanese

7.1 The Effectiveness of Tests


Sometimes ago, I had a chance to see a test of a certain program whether its performance accurately met to an international standard's requirement. Some foreign examiners came and tested the program. There were enormous items to be tested, and they inspected each item to every hole and corner, even to a pinhole.

There are some interesting matters for me. First, the semantics of the standard itself is not so clear. Next, the examiners had some ambiguous points in reading the standard. And, last, the program prepared for the test had some errors, which the examiners were correcting in between whiles of the test. Having come to such pass, it was hard to tell which tested which.

I had used a program product distributed by a famous company which all of you readers should have heard its name. It was when we were developing a considerably large program utilizing the product.

We had begun with provisional edition of the product, which had been distributed to the parties concerned before its commercial release. While continuing the development, we had been announced that the product then carried out a commercial release and no longer provisional edition. Because the authoritative commercial release came out, we had not to supply with our program based on the provisional edition, but had to make it run on the new commercial release. So we had replaced the old provisional edition with the new one.

Then, how was it, our program had stalled. Replacing provisional edition with commercial version causes this; it is obvious, but we also had to make it clear that the actual problem lay whether on our developping program or on the commercial version product. Inspecting with a special tool (a program), we had found that the provisional edition acted correctly but the commercial release went wrong.

This has one more funny episode. As we had duly purchased the product and had signed up a support contract, we fairly could complain when we had troubles with the product. Therefore we faxed a information that clearly showed where and how the product went wrong to support section. Whereas the cause had become completely clear, we had no our own way to modify the product. Therefore we had added the following expressly, "Because this is a quite basic error, you ordinarily would remove such kind of error before releasing your product to market, I suppose."

Nevertheless they had been slow to reply. It had come the designated day that they had promised us to make an answer for. We were waiting for the answer, while suspecting that we could not get it on the day, and it went so. Thus I faxed a message again soon after midnight and then left office for home at once.

The next day, I was surprised at my office. The reply had come past 1 A.M. Really working, they had been.

Furthermore, there was one more incredible matter. They asked us to continue the investigation of the errors which we had pointed out. They said, as having no tool, they could not inspect details of the errors.

It's no joke. We regarded ourselves as an official "customer" who spent money for their product. By whatever reason, why did a customer have to support sales side technically? Silly gag.

This product has a well-known company's brand. In fact, I already knew that the company just only sold it and owed another company for its technical side. Its chief technical engineer was one of my computer engineer companions, and there also had been an offer to me to work at programming of the product. Therefore, I had known everything. To say, I was just kidding on it as it was funny.

The circumstance how that error creep into the product was as follows. The development company had supplied the properly working version to the vendor company. The problem is not due to the developper side, as its provisional version had also worked properly. When the program released commercially, the vendor mistook control of its revision. They had corrected(?) correct revision to errornous one, then sold it.

Furthermore, why that error was detected was interesting. It was a kind of error that could not be detected so far as using a program piece prepared for the test. The error was detected because we used the product as a part of large program and we processed enormous amount of data on it. In other word, what really made the symptom of the product be developed was the program used in business condition, not such program piece for the test.

It is truly difficult to find errors. It is much difficult to find out errors than correcting them. Finding an error can carry us more than halfway to solving the problem. Correcting work is much easier than that of finding. To be awarded prizes or the likes are those who find out errors.

There is a free software made in the United States that you can get an award if you point out an error on. The author himself offers to give the prize. Certainly, 1 dollar for the first, and it will be doubled for each new error found. I, a coward, have no courage to bet on such extremely dangerous game.

This software is now very famous, and now is used everywhere in the world. Though it is distributed worldwide for free and is used universally, inclusive of Japan, we have not heard that the developper himself went bankrupt. It is Professor Knuth of Standford University; an ultra-famous person whose name is known to every computer engineer. He is especially celebrated for that he hardly ever makes mistakes.

Imagine any of those various software companies give prize, or refund, when any errors are reported on their product, as Professor Knuth does so. Such that must have 99% of software companies go bankrupt. But I want them to think of like this at somewhere in their mind.


Copyright 1996, 2000 Hirofumi Fujiwara. Translated by Y.K.
No reproduction or republication without written permission.

the Hidden Realities of Computer Industry in Japan Japanese