BCG consultants using AI closed most of the gap with trained data scientists

What happened

BCG ran an experiment comparing 480 consultants against 44 trained data scientists on tasks involving Python, data cleaning, predictive modeling, and statistical analysis. On the coding task, consultants using generative AI scored roughly 50 percentage points higher than consultants without it - closing most of the gap with the data-scientist benchmark.

But on the predictive-modeling task, AI use could lead to over-reliance on the model and worse outcomes. The takeaway: AI can expand someone's range of competence, but only when they understand enough to evaluate what the AI produced.

The business problem

Giving employees AI access without the judgment to evaluate its output can produce worse results than not using it at all.

Why it worked

• AI expands capability most on well-defined, checkable tasks
• It can quietly produce worse outcomes on tasks requiring judgment, if over-relied on
• Pair tool access with enough domain training to evaluate the output
• Not every task benefits equally - test before scaling

Where this points