As a manager, imagine having access to more accurate predictions about essential parameters such as revenue, sales, employee retention and attraction, product performance, or customer satisfaction.

In addition, imagine being able to harvest this knowledge from your employees directly and at a low cost. Now, as an employee, imagine that management makes decisions based on your weekly predictions and insights.

The use of crowd predictions contains new methods and tools for leaders of businesses, public organizations, and NGOs. If you have not yet heard of these concepts, you are not alone. In Denmark, only a handful of leading companies have implemented crowd predictions. When looking at the US, the number is considerably higher with companies, including giants such as Google, Ford Motor, Eli Lilly, and IBM. These companies have all tapped into the potential of collective intelligence. Among other things, Google has used employees to predict how many new users Gmail will receive.

The former Collective Intelligence Unit (CIU) at the Copenhagen Business School (now the Collective Intelligence Group at the IT University of Copenhagen) conducts research on collective intelligence in organizations and societies. The research group is led by our co-founder of Mindpool, Carina Antonia Hallin.

What research projects that test collective intelligence can do for companies

Throughout her academic career, Carina has conducted a number of real-life field experimental studies and cases with both organizations and public institutions. One of the studies from July 2017 to August 2019 conducted by the research group was the project named “Crowd predictions from the frontline,” supported by the Danish Industry Foundation. During the project, the unit collaborated with Lego Group and Radiometer to test the potential of crowd predictions as a new proactive decision-making tool.

Employees at the frontline of organizations possess a collectively and managerially usable intelligence, as they are the first to detect ongoing changes in operations and the market environment. 

The project aimed to use this digitally collected knowledge to predict operational challenges and strategic opportunities, improving management’s ability to act more proactively. More specifically, the testing of crowd predictions was done using crowd prediction technologies; In a contract with the CBS Legal department, it was agreed that Mindpool could supply the prediction software for the research project for free.

The basic idea was to collect collective predictions from different employee groups, measured over time to assess whether it produces unbiased and accurate information as a reasonable basis for management’s decisions. In this way, such software can be used as a useful management tool.

Key takeaways from the study

After conducting the research, Carina Antonia Hallin and her research team published the high-level and preliminary results of the study in a public research report.

The research studies at the Lego Group and Radiometer consisted of four phases: interviews, ideation, filtering, and predictions. From this, the research group was able to extract step-by-step levels of knowledge that helped validate collective intelligence and crowd predictions as a method.

In the Lego Group, the researchers tested Hypothesis 1a that frontline employees across brand stores, even though they have less experience in evaluating product quality compared to the product quality experts at the headquarter, still have an advantage as predictors resulting from their accumulated daily tacit knowledge. They interact directly with customers to a greater degree than management does. This extraordinary knowledge means that their predictions on an aggregated level are just as accurate as quality experts in Billund.

For the Radiometer study, the case report describes the testing of two hypotheses. Hypothesis 2a tests that within groups of employees in the organization, some sub-groups can better predict fluctuations than other groups. The reason for this assumption is that some groups acquire more tacit knowledge than others, as they are more in touch with the respective conditions that they should predict. Hypothesis 2b tests that employees are better at predicting fluctuations in the organization with which they are often in contact with their daily work than those fluctuations that are outside of the organization.

But how did the results unfold after the research group tested the hypothesis mentioned above?

The following figures from the research report describe Dinosaur Fossil and Batmobile Product's prediction accuracy from 1989. The subgroups are i) the product quality experts at LEGO Group’s head office in Billund, and ii) frontline employees in Lego Group’s brand stores in London and Slough, UK. A more detailed account of the results can be read on the Industry Foundation’s webpage here.

In the Lego Group case, the research group analyzed customer satisfaction incidents as customer complaints per 1 million product boxes sold with a prediction horizon of 1,3, 6, and 12 months. In this case, the researchers found that frontline employees in the brand stores in the UK predict as accurately as the experts (the product specialist group at the Lego headquarter group in Billund) concerning customer incidents. See Figure 1 and 2 below.

For the Radiometer study, the following figure presents a selection of results. The research group tests the margin of error in the predictions about the current performance targets, which in these examples relate to the comparison of two prediction variables and comparison across groups in the organization.

Based on Radiometer's preliminary results, the research group finds that predictions about the variable «voluntary resignation from position» was better predicted than new hires. This agrees well with the assumption that the frontline gains more significant insights into variables close to them than conditions beyond their control, such as knowledge about new employees.

Besides, the research group finds significant differences in predictive power across groups. That is, some subgroups (branches) are better at predicting than other groups.

A more detailed account of the results can be read on the Industry Foundation’s webpage here.

In future analyses of the dataset, the research group will look further into how specific business and individual factors may explain organizations' predictive power.

These analyses will continuously occur in different groups the coming year, while the research group awaits to receive more performance data for the future prediction horizons, 6 and 12 months forward.

We are looking forward to reading about the upcoming and more detailed results of how collective intelligence can become a useful tool for organizations.

You've just read a blog post from Mindpool - a platform that helps you harness the collective intelligence of employees. Mindpool taps into the knowledge of employees to provide actionable predictions and curated insights. Mindpool is rooted in decades of research in collective intelligence. Read more from our resource universe or contact us here if you'd like to tap into your employees' valuable knowledge.