Industries subject to high turnover and employing a large number of people are increasingly turning to big number crunching or big data to assist them in making hiring decisions.

Interestingly, such big data analysis can turn out some far-from-obvious conclusions. For example, a recent article in The Economist quotes Evolv, a company which monitors recruitment and workplace data. Evolv's analysis of some 3 million data points from more than 30,000 employees found that people who filled out their employment application on a browser that did not come with the machine proved to be better employees, both in terms of performance and retention. The "why" behind this finding is unclear. Perhaps these employees displayed better initiative, higher literacy rates – and thus a greater ability to learn – and better computing skills, all crucial in the modern workplace.

Regardless of the "why," the big data number crunching produced results which proved useful for the process of hiring staff. In an industry where achieving a high retention rate is a major factor, the big data shows that those who live close to work and therefore have a shorter commute are more likely to stay with the company. In other cases, the big data analysis shows that common hiring assumptions and rules are wrong. For example, many firms automatically eliminate applicants with a criminal record; yet for some occupations, the data shows that there is no correlation which would support this traditional policy.

Computers using algorithms and questions grounded in psychological and other sciences have long been deployed as a mechanism to screen candidates. For higher level positions, these systems are used in conjunction with an intensive interview process. For example, as CEO of a private equity portfolio company, I have been required to take a battery of tests designed to measure drive, innovativeness, persuasion, task vs. people orientation, resilience and much more.

Today's students, the employees of the future, are, of course, used to such large scale screening, for example by standardized tests such as the GMAT and SAT. The growing use of big data, however, suggests that employment choices may also be based upon more circumstantial information, a dataset and formula derived from large volumes of data gathered from you as you interact with the World Wide Web and especially social media. These social "fingerprints" are likely to be revealing and provide new insights which will likely result in their inclusion in enhanced talent selection strategies. They may also challenge many of our traditional talent selection rules and ultimately show them to be largely groundless.

In the world of algorithmic hiring, however, we should keep several caveats in mind. While these analytics are proving to be increasingly useful in hiring decisions, it is important to remember that they are only as good as the human who designs them and mistakes can be made. And, when dealing with such big numbers, any mistake can be both big and costly. Also, any use of computers to gather information on people has to be balanced with concerns about privacy and related legal issues.

On a wider scale, the growing use of big data suggests that machines are increasingly undertaking levels of "intelligence" which were previously considered impossible. For example, as an avid chess player and coach, I constantly heard people proclaim that a computer would never defeat a human chess grandmaster. Not only has this happened, but IBM's Deep Blue computer also defeated then world chess champion, Gary Kasparov. Note, however, that this does not mean the computers involved thought in the same way as a human; rather, the power of the algorithm and massive number crunching capacity enabled the computers to achieve a better result.

Looking to the future, as computers increasingly do what many humans have traditionally done and machines learn to adapt and learn, we may eventually reach an age predicted by many where we must reconceptualize our Industrial Age notion of what we mean by "work." This in turn will present many challenges, but optimistically may be the beginning of a new age of individual creativity.

For the moment, the clear message for us mere mortals seeking to survive, let alone thrive, in this new work environment, is that we must keep learning and gravitate towards work that is unlikely to be emulated by computers.

The author is a columnist with China.org.cn. For more information please visit: http://www.ccgp-fushun.com/opinion/eugeneclark.htm

Opinion articles reflect the views of their authors, not necessarily those of China.org.cn.