Engineering education is good preparation for a lot of things

I am a big fan of engineering education.  I think studying engineering for four years is intellectually challenging, exciting, and (when it is done right) fun.  An engineering education is good preparation for careers in many fields and (again, when it is done right) good preparation for citizenship.

So you can imagine my reaction to a click-bait headline like, "Does Engineering Education Breed Terrorists?", which appeared recently in the Chronicle of Higher Education.

The article presents results from a new book, Engineers of Jihad: The Curious Connection Between Violent Extremism and Education, by Diego Gambetta and Steffen Hertog.  If you don't want to read the book, it looks like you can get most of the results from this working paper, which is free to download.

I have not read the book, but I will summarize results from the paper.

Engineers are over-represented

Gambetta and Hertog compiled a list of 404 "members of violent Islamist groups".  Of those, they found educational information on 284, and found that 196 had some higher education.  For 178 of those cases, they were able to determine the subject of study, and of those, the most common fields were engineering (78), Islamic studies (34), medicine (14), and economics/business (12).

So the representation of engineers among this group is 78/178, or 44%.  Is that more than we should expect?  To answer that, the ideal reference group would be a random sample of males from the same countries, with the same distribution of ages, selecting people with some higher education.

It is not easy to assemble a reference sample like that, but Gambetta and Hertog approximate it with data from several sources.  Their Table 4 summarizes the results, broken down by country:

In every country, the fraction of engineers in the sample of violent extremists is higher than the fraction in the reference group (with the possible exceptions of Singapore and Saudi Arabia).

As always in studies like this, there are methodological issues that could be improved, but taking the results at face value, it looks like people who studied engineering are over-represented among violent extremists by a substantial factor -- Gambetta and Hertog put it at "two to four times the size we would expect [by random selection]".

Why engineers?

In the second part of the paper, Gambetta and Hertog turn to the question of causation; as they formulate it, "we will try to explain only why engineers became more radicalized than people with other degrees".

They consider four hypotheses:

1) "Random appearance of engineers as first movers, followed by diffusion through their network", which means that if the first members of these groups were engineers by chance, and if they recruited primarily through their social networks, and if engineers are over-represented in those networks, the observed over-representation might be due to historical accident.

2) "Selection of engineers because of their technical skills"; that is, engineers might be over-represented because extremist groups recruit them specifically.

3) "Engineers' peculiar cognitive traits and dispositions"; which includes both selection and self-selection; that is, extremist groups might recruit people with the traits of engineers, and engineers might be more likely to have traits that make them more likely to join extremist groups.  [I repeat "more likely" as a reminder that under this hypothesis, not all engineers have the traits, and not all engineers with the traits join extremist groups.]

4) "The special social difficulties faced by engineers in Islamic countries"; which is the hypothesis that (a) engineering is a prestigious field of study, (b) graduates expect to achieve high status, (c) in many Islamic countries, their expectations are frustrated by lack of economic opportunity, and (d) the resulting "frustrated expectations and relative deprivation" explain why engineering graduates are more likely to become radicalized.

Their conclusion is that 1 and 2 "do not survive close scrutiny" and that the observed over-representation is best explained by the interaction of 3 and 4.

My conclusions

Here is my Bayesian interpretation of these hypotheses and the arguments Gambetta and Hertog present:

1) Initially, I didn't find this hypothesis particularly plausible.  The paper presents weak evidence against it, so I find it less plausible now.

2) I find it very plausible that engineers would be recruited by extremist groups, not only for specific technical skills but also for their general ability to analyze systems, debug problems, design solutions, and work on teams to realize their designs.  And once recruited, I would expect these skills to make them successful in exactly the ways that would cause them to appear in the selected sample of known extremists.  The paper provides only addresses a narrow version of this hypothesis (specific technical skills), and provides only weak evidence against it, so I still find it highly plausible.

3) Initially I found it plausible that there are character traits that make a person more likely to pursue engineering and more likely to join and extremist group.  This hypothesis could explain the observed over-representation even if both of those connections are weak; that is, even if few of the people who have the traits pursue engineering and even fewer join extremist groups.  The paper presents some strong evidence for this hypothesis, so I find it quite plausible.

4) My initial reaction to this hypothesis is that the causal chain sounds fragile.  And even if true, it is so hard to test, it verges on being unfalsifiable.  But Gambetta and Hertog were able to do more work with it than I expected, so I grudgingly upgrade it to "possible".

In summary, my interpretation of the results differs slightly from the authors': I think (2) and (3) are more plausible than (3) and (4).

But overall I think the paper is solid: it starts with an intriguing observation, makes good use of data to refine the question and to evaluate alternative hypotheses, and does a reasonable job of interpreting the results.

So at this point, you might be wondering "What about engineering education?  Does it breed terrorists, like the headline said?"

Good question!  But Gambetta and Hertog have nothing to say about it.  They did not consider this hypothesis, and they presented no evidence in favor or against.  In fact, the phrase "engineering education" does not appear in their 90-page paper except in the title of a paper they cite.

Does bad journalism breed click bait?

So where did that catchy headline come from?  From the same place all click bait comes from: the intersection of dwindling newspaper subscriptions and eroding journalistic integrity.

In this case, the author of the article, Dan Berrett, might not be the guilty party.  He does a good job of summarizing Gambetta and Hertog's research.  He presents summaries of a related paper on the relationship between education and extremism, and a related book.  He points out some limitations of the study design.  So far, so good.

The second half of the article is weaker, where Berrett reports speculation from researchers farther and farther from the topic.

For example, Donna M. Riley at Virginia Tech suggests that traditional [engineering] programs "reinforce positivist patterns of thinking" because "general-education courses [...] are increasingly being narrowed" and "engineers spend almost all their time with the same set of epistemological rules."

Berrett also quotes a 2014 study of American students in four engineering programs (MIT, UMass Amherst, Olin, and Smith), which concludes, "Engineering education, fosters a culture of disengagement that defines public welfare concerns as tangential to what it means to practice engineering."

But even if these claims are true, they are about American engineering programs in 2010s.  They are unlikely to explain the radicalization of the extremists in the study, who were educated in Islamic countries in the "mid 70s to late 90s".

At this point there is no reason to think that engineering education contributed to the radicalization of the extremists in the sample.  Gambetta and Hertog don't consider the hypothesis, and present no evidence that supports it.  To their credit, they "wouldn’t presume to make curricular recommendations to [engineering] educators."

In summary, Gambetta and Hertog provide several credible explanations, supported by evidence, that could explain the observed over-representation.  Speculating on other causes, assuming they are true—and then providing prescriptions for American engineering education—is gratuitous.  And suggesting in a headline that engineering education causes terrorism is hack journalism.

We have plenty of problems

Traditional engineering programs are broken in a lot of ways.  That's why Olin College was founded, and why I wanted to work here.  I am passionate about engineering education and excited about the opportunities to make it better.  Furthermore, some of the places we have the most room for improvement are exactly the ones highlighted in Derrett's article.

Yes, the engineering curriculum should "emphasize human needs, contexts, and interactions", as the
executive director of the American Society for Engineering Education (ASEE) suggests.  As he explains, "We want engineers to understand what motivates humans, because that’s how you satisfy human needs.  You have to understand people as people."

And yes, we want engineers "who recognize needs, design solutions and engage in creative enterprises for the good of the world".  That's why we put it in Olin College's mission statement.

We may not be doing these things very well, yet.  We are doing our best to get better.

In the meantime, it is premature to add "preventing terrorism" to our list of goals.  We don't know the causes of terrorism, we have no reason to think engineering education is one of them, and even if we did, I'm not sure we would know what to do about it.

Debating changes in engineering education is hard enough.  Bringing terrorism into it—without evidence or reason—doesn't help.

The retreat from religion is accelerating

Updated results from the CIRP Freshman Survey were published last week, which means it's time for me to update to my annual series on the increasing number of college students with no religious affiliation (the "nones").

Since I started following this trend it 2008, I have called it "one of the most under-reported stories of the decade". I'm not sure if that's true any more; there have been more stories recently about secularization in the U.S., and the general disengagement of Millennials from organized religion.

But one part of the story continues to surprise me: the speed of the transformation is remarkable, averaging almost 1 percentage point per year for the last 20 years, and there is increasing evidence that it is accelerating.

30% have no religion

Among first-year college students in Fall 2015, the fraction reporting no religious affiliation reached an all-time high of 29.6%, up more than 2 percentage points from 27.5% last year.

And the fraction reporting that they never attended a religious service also reached an all-time high at 30.5%, up from 29.3% last year.

These reports are based on survey results from the Cooperative Institutional Research Program (CIRP) of the Higher Education Research Insitute (HERI).  In 2015, more than 141,000 full-time, first-time college students at 199 colleges and universities completed the CIRP Freshman Survey, which includes questions about students’ backgrounds, activities, and attitudes.

In one question, students select their “current religious preference,” from a choice of seventeen common religions, “Other religion,” "Atheist", "Agnostic", or “None.”  The options "Atheist" and "Agnostic" are new this year.  For consistency with previous years, I compare the "Nones" from previous years with the sum of "None", "Atheist" and "Agnostic" this year.

Another question asks students how often they “attended a religious service” in the last year. The choices are “Frequently,” “Occasionally,” and “Not at all.” Students are instructed to select “Occasionally” if they attended one or more times.

The following figure shows the fraction of Nones over more than 40 years of the survey.

The blue line shows actual data through 2014; the red line shows a quadratic fit to the data.  The dark gray region shows a 90% confidence interval, which represents uncertainty about the parameters of the model.  The light gray region shows a 90% predictive interval, which represents uncertainty about the predicted value.

The blue square shows the actual data point for 2015.  For the first time since I starting running this analysis, the new data point falls outside the predicted interval.

30% have not attended a religious service

Here is the corresponding plot for attendance at religious services.

The new data point for 2015, at 30.5%, falls in the predicted range, although somewhat ahead of the long term trend.

The trend is accelerating

The quadratic model suggests that the trend is accelerating, but for a clearer test, I plotted annual changes from 1985 to the present:

Visually, it looks like the annual changes are getting bigger, and the slope of the fitted line is about 0.4 percentage points per decade.  The p-value of that slope is 3.6%, which is considered statistically significant, but barely.  You can interpret that as you like.

The gender gap persists

As always, more males than females report no religious preference.

 The gender gap closed slightly this year, but the long term trend suggests it is still growing.

Predictions for 2016

Using the new 2015 data, we can run the same analysis to generate predictions for 2016.  Here is the revised plot for "Nones":

This year's measurement is ahead of the long-term trend, so next year's is predicted to regress to 28.6% (down 1.0%).

And here is the prediction for "No attendance":

Again, because this year's value is ahead of the long term trend, the center of the predictive distribution is lower, at 29.8% (down 0.7%).

I'll be back next year to check on these predictions.

Comments

1) The addition of the two new categories, "Atheist" and "Agnostic" is new this year.  Of the 29.6% I am treating as "no religion", 8.3% reported as agnostic, 5.9% as atheist, and 15.4% as none.  It's hard to say what effect the change in the question had on the results.

2) The number of schools and the number of students participating in the Freshman Survey has been falling for several years.  I wonder if the mix of schools represented in the survey is changing over time, and what effect this might have on the trends I am watching.  The percentage of "Nones" is different across different kinds of institutions (colleges, universities, public, private, etc.)  If participation rates are changing among these groups, that would affect the results.

3) Obviously college students are not representative of the general population.  Data from other sources indicate that the same trends are happening in the general population, but I haven't been able to make a quantitative comparison between college students and others.  Data from other sources also indicate that college graduates are slightly more likely to attend religious services, and to report a religious preference, than the general population.

Data Source
Eagan, K., Stolzenberg, E. B., Ramirez, J. J., Aragon, M. C., Suchard, M. R., & Hurtado, S. (2014). The American freshman: National norms fall 2014. Los Angeles: Higher Education Research Institute, UCLA.

This and all previous reports are available from the HERI publications page.

A quick Bayes problem in IPython

I've been busy with other projects, so I haven't posted anything new for a while.  But a reader posted a nice Bayes's theorem problem, which I am using as an excuse to experiment with embedding an IPython/Jupyter notebook in a blog, following these instructions. Let's see how it goes...

The following problem was submitted to my blog, Probably Overthinking It, by a user named Amit, who wrote:

The following data is about a poll that occurred in 3 states. In state1, 50% of voters support Party1, in state2, 60% of the voters support Party1, and in state3, 35% of the voters support Party1. Of the total population of the three states, 40% live in state1, 25% live in state2, and 35% live in state3. Given that a voter supports Party1, what is the probability that he lives in state2?

My solution follows. First I'll create a suite to represent our prior knowledge. If we know nothing about a voter, we would use the relative populations of the states to guess where they are from.

In [2]:

prior = thinkbayes2.Suite({'State 1': 0.4, 'State 2': 0.25, 'State 3': 0.35})
prior.Print()

State 1 0.4
State 2 0.25
State 3 0.35

Now if we know a voter supports Party 1, we can use that as data to update our belief. The following dictionary contains the likelihood of the data (supporting Party 1) under each hypothesis (which state the voter is from).

In [3]:

likelihood = {'State 1': 0.5, 'State 2': 0.60, 'State 3': 0.35}

To make the posterior distribution, I'll start with a copy of the prior.

The update consists of looping through the hypotheses and multiplying the prior probability of each hypothesis, hypo, by the likelihood of the data if hypo is true.

The result is a map from hypotheses to posterior likelihoods, but they are not probabilities yet because they are not normalized.

In [4]:

posterior = prior.Copy()
for hypo in posterior:
    posterior[hypo] *= likelihood[hypo]
posterior.Print()

State 1 0.2
State 2 0.15
State 3 0.1225

Normalizing the posterior distribution returns the total likelihood of the data, which is the normalizing constant.

In [5]:

posterior.Normalize()

Out[5]:

0.4725

Now the posterior is a proper distribution:

In [6]:

posterior.Print()

State 1 0.42328042328
State 2 0.31746031746
State 3 0.259259259259

And the probability that the voter is from State 2 is about 32%.

In [ ]: