Teaching Executives to See Social Capital
An ROI Study of a Four Year Executive Education Custom Program at Chicago GSB

This article is an abridged version of "Teaching Executives to See Social Capital" by Prof. Ronald S Burt and Don Ronchi
First published: November 2005
The full version of the article can be read at: http://gsbwww.uchicago.edu/fac/ronald.burt/research/TESSC.pdf

Abridgement by Roderick A Millar, Editor of IEDP, March 2006

The business education world has been challenged to answer one central critical question for many years - is there any evidence of value? Statistical appraisal of MBA's, for example, has not proven that "possessing an MBA degree nor grades earned in courses correlate with career success".

In an attempt to redress this empirical gap Ronald S. Burt, the Hobart W. Williams Professor of Sociology and Strategy at the University of Chicago Graduate School of Business, and Don Ronchi, the then Vice President of Raytheon Six Sigma and Chief Learning Officer of Raytheon Company and currently adjunct professor at Chicago GSB followed a group of Raytheon executives participating in a custom executive education program at Chicago GSB over a period of years and measured their post-program career development against a control group of similar Raytheon executives who did not attend the class.

The program that was followed focused on teaching executives how to effectively use "social-networks" to enhance their managerial performance. The course content and focus clearly has an impact on the outcome, Burt and Ronchi emphasize that other programs focused on social capital content could provide similar results. What is unique here are the panel data collected that demonstrates the programs effectiveness.

The Topic - Social Networks and Social Capital.

The program taught to the Raytheon executives centred on helping them understand "the network structure of social capital". Social capital can be defined as the value accumulated through the degree to which a community or society collaborates and cooperates (through such mechanisms as networks, shared trust, norms and values) to achieve mutual benefits.

One of the opening acts for the twenty-first century was venture capital discovering social networks. Tens of millions of dollars were invested in software companies purporting to make people better off through network services providing lucrative or emotionally rewarding relationships. Companies sprang up one after another like spring flowers, drawing comment from establishment media such as Fortune and the Wall Street Journal, which in turn sent technology gossip all a-blog. SelectMinds and Classmates Online catered for college and high school alumni. Data-rich Craigslist provided love and business contacts all in one screen. Targeting business, companies like LinkedIn, Tacit Knowledge Systems, and Spoke Software focused on facilitating new connections between professionals.

There is a common-sense business case to be made for these companies. More lucrative and emotionally rewarding relationships must be a good thing. Certainly, "bowling alone" is not good. Motivational anecdotes posted on company websites describe people who used the company's service and are better off today.

The study of social networks and the bridges of communication between clusters of people have been conducted for many years. In 1912 Schumpeter examined how entrepreneurial "leaders" bring together elements from separate production spheres in which people live by routines; other studies include Milgram (1967) on the "small world" phenomenon in which people at great geographic remove can communicate with one another through surprisingly few intermediaries because of bridges between social worlds and March (1991) on organizations "exploring" for new opportunities versus "exploiting" known revenue streams. A theme in this work is that behavior, opinion, and information broadly conceived, are more homogeneous within than between groups. People focus on activities inside their own group, which creates holes in the information flow between groups, or more simply, structural holes.

Figure 1 indicates two different networks, those of Robert and James. While they both have the same number of contacts, Robert's contacts are not connected with one another while James's network is densely connected with one another.

Figure 1

Robert is better positioned for the social capital of brokerage. Given greater homogeneity within than between groups, people whose networks bridge the structural holes between groups have earlier access to a broader diversity of information and have experience in translating information across groups. They have a vision advantage in detecting and developing good ideas. They are able to see early, see more broadly, and translate information across groups. Like over-the-horizon radar in an airplane, or an MRI in a medical procedure, brokerage across the structural holes between groups provides a vision of options otherwise unseen. There is abundant and accumulating empirical evidence of the returns to brokerage - in terms of more positive performance evaluations, faster promotions, higher compensation, and more successful teams. The social capital of brokerage is the theoretical foundation for training programs intended to enhance leadership skills in building bridges across organization silos.

James is better positioned for the social capital of closure. Where mutual contacts close the network around bridge relations, reputation pressures encourage the trust and collaboration needed to deliver the value of bridges, creating a social capital advantage defined in terms of closure across structural holes, especially at extreme levels of closure coordinating across extensive bridge relations as in a skunkworks or crisis team. Illustrated by James in Figure 1, the social capital of closure is about the efficiency advantages of driving variation out of group behavior or opinion.

Where Robert is positioned to benefit from differences between people who vary in their behavior and opinion, James is positioned to integrate the work of people who have much in common. Where James is positioned to drive variation out of group B, Robert is positioned to introduce into group B variation from the other groups A and C with which he is familiar. The social capital of closure is the theoretical foundation for training programs (e.g., Six Sigma and Lean Production) intended to enhance skills in process efficiency.

The two forms of social capital are twice complementary. The greatest returns to informal organization occur when both forms are present; when closure within a group occurs with brokerage beyond the group.

Second, brokerage and closure are complements in providing a cure for the other's failure mode. Unrestrained brokerage can create organization chaos, manifest in errors such as resources allocated to conflicting goals and units in the same organization competing against one another. Closure's reputation mechanism brings people back into alignment. It is no accident that reputation metrics provided by multi-point (360º) evaluations were adopted in so many large companies at about the same time that companies were removing layers of bureaucracy, which made them more dependent on informal brokerage to integrate operations across groups. On the other hand, closure's reputation mechanism can create groupthink and rigidity. Stories echoing within the closed network amplify opinion to positive and negative extremes, making the existing structure resistant to change, deepening the structural holes that segregate groups, again especially at extreme levels of closure. Brokerage has the potential to crack closure-induced arthritis with selection and synthesis among conflicting alternatives.

A Business Leadership Program (BLP)

As venture capitalists were discovering the benefits of social networks, Raytheon Company, one of America's leading electronics firms, was attacking the problem of how to coordinate across the organization silos of its acquired companies and its many product programs.

In the wake of industry consolidation, operations were balkanized in the sense that programs that would do well to coordinate with one another were segregated in distant organization silos. The task was to preserve efficiency and tacit knowledge within the silos, while harvesting the value of integrative work across the silos.

Ron Burt and Don Ronchi created Raytheon's senior leadership development program, titled the "Business Leadership Program" (BLP), to teach director and vice-president executives to see the social capital in how they organize to create value. The program first ran in the final quarter of 2001 and ran with 11 different cohorts, the last one taking place in 2004.

Relative to a traditional multi-week senior executive program, the BLP is a quick surgical strike that combines academic work with project applications to ongoing company operations. A cohort of two to four dozen executives spends five days at the University of Chicago GSB.

The program begins with an academic introduction to the vision mechanism by which growth occurs when people build network bridges across the structural holes in an organization, and the reputation mechanism by which trust and efficiency occurs when people close the network around a group. Written and video cases are combined with research evidence to communicate the network mechanisms and highlight their failure modes, the agency problems that can occur with brokerage and the groupthink, stereotyping, and rigidity that can occur with closure. The network mechanisms are then used to discuss traditional questions of business and corporate strategy about market behavior, where to position programs in an organization, the choice between tight or loose couplings between programs, and how to effectively connect with customer markets. Again, written and video cases are combined with research evidence to drive home the content. The fourth and fifth days shift to application and active learning. There is a transition morning on Raytheon strategy in light of the principles discussed, followed by a day and a half in project teams applying the principles to practical issues in Raytheon.

Six weeks later, the cohort assembles for a "mid-course" day at corporate headquarters to discuss project progress and have a two-hour workshop with the CEO on Raytheon strategy and leadership.

Six weeks after that, the cohort assembles for a "graduation" day at headquarters. They debrief one another and the CEO on their final proposal, and their progress on a plan to transfer the BLP project into the company for development or implementation.

The BLP has shifted with strategic initiatives in Raytheon Company, and morphed in response to the interests and style of a new CEO, but two goals have remained consistent: (1) Improve Director and Vice-President alignment with Company strategy and emerging strategic initiatives. (2) Enhance market-driven collaborative connections among senior people across the Company. It was hoped that fulfilling these goals would enhance the ability of senior leaders to identify and effectively act on strategic opportunities.

Program Evaluation

Lower-level training has concrete outcomes, where the training is intended to make a known process more efficient - reduce the number of defective parts, increase the number of units shipped, or reduce the number of customer complaints. The goals of executive education programs are less concrete, less about refining what exists than creating what does not yet exist.

Donald Kirkpatrick proposed in 1959 a four-level typology of training evaluations that continues to be the most popular framework for guiding evaluations. The gist of the typology is:

• Level One: reaction (Did participants like the program?)
• Level Two: learning (Did participants change their attitude, knowledge, or skills?)
• Level Three: transfer (Did participants change their behavior?)
• Level Four: results (Did participants become more productive?)

Level one evaluation refers to the "smilesheets" distributed at the conclusion of training programs. With respect to the BLP, the combination of advanced research, practical application to one's own work, and the opportunity to think through strategy problems with the CEO was well received by participants. Opinion of the faculty was tightly clustered on a five-point scale around a 4.94 mean. Although participants were well along in their careers (48 years of age, on average), most said they had learned the maximum "quite a bit" that would help them be a better leader. An even larger majority would recommend the program "without reservation" to an able colleague.

Level two and three evaluations of the BLP refer to cognitive and behavioral change in participants. The BLP is an executive program leveraged on the domain competence of the participants. The desired program outcome is not senior leaders conforming to a prescribed profile of beliefs or behaviors. It is senior leaders growing the business by more ably identifying and effectively acting on strategic opportunities. Opportunity detection, and how participants act upon their opportunities, is up to the vision and drive of participants. Burt and Ronchi decision to move past levels two and three, directly to level four, was informed by their experience teaching network analysis. They knew that people can be taught to see network structure more accurately. Given that ability, the BLP focused on the value produced.

Nevertheless, in October of 2005, four years after the first BLP cohort and four months after the most recent, the 344 program graduates still with the company were sent an email asking about their BLP project and whether they had developed consequential relationships from the BLP. Responses came back from 59% of the graduates. Most graduates felt that the BLP had resulted in consequential relationships (90%). Beyond the "yes" responses, illustrative comments included the following:

"Many relationships were established. In particular, the close relationships with my project teammates as well as the other class members. The project also gave us the opportunity to meet and talk to many other Raytheon employees I would have never met as part of my normal assignments."

"It was a great group of people and it is a pleasure to continue those relationships four years later."

"Not just my project team members, but others in the class. This has proven to be the most beneficial aspect of the class from my perspective. If I get a call for assistance from a member of my class, it is a high priority for me. I feel my calls for support are also treated as a high priority from class members."

"It broadened my cross-functional network across Raytheon."

"Yes absolutely! Both in my business and with people in other businesses around the company."

And some of the few people who answered "no" seemed to be saying "yes" in a round-about way with illustrative comments such as the following:

"No, but I have contacted members of the BLP when I had similar issues arise in my business to projects that were studied in the cohort."

"I knew many in my BLP course. The course DID help me solidify and enhance some of those relationships."

The comments show that graduates focus on relations with people in their BLP project team, then people who attended the BLP at the same time (with perhaps too little attention to relations across cohorts and into the broader organization), but the comments are consistent in attributing ongoing consequential relationships to attending the BLP.

For Level Four evaluation Raytheon Company's own evaluation metrics were used. Assuming that the annual performance evaluations reflect the quality of an employee's work, Burt and Ronchi asked whether graduation from the BLP was associated with more positive evaluations.

Assuming that people are promoted for merit, are BLP graduates more likely to be promoted?

Assuming that people stay with the Company if they find engaging work, does the program lower the odds of a promising executive leaving the Company?

The Control Group

Using the company's own evaluation metrics highlighted a potential bias, in that promising employees had a higher chance of being invited onto the BLP and therefore BLP graduates were more likely to be those promising employees who are expected to gain promotion. To counter this bias Burt and Ronchi conducted extensive analysis of Raytheon employees to obtain data that indicated the probability or risk of an individual employee being invited to participate in the BLP. The factors analyzed were job rank (VP's and directors had a higher invitation probability), age and age in relation to managerial level (younger senior managers were more likely to be invited), ethnic origins (minority executives had a higher probability of invitation), educational attainment (graduates had a higher probability of invitation, those with six sigma qualifications or multiple specializations had a higher risk of invitation). A successful result in the Senior Leadership Program (SLP) run prior to the BLP was a significant factor leading to an invitation. Other factors were also computed; while the BLP progressed some cultural changes occurred within Raytheon, most notably the arrival of a new CEO (see figure 2), that affected potential participants willingness to be chosen, and also that in the second period of the program the most likely to be invited had already been invited (and either participated or turned down the invitation) however by the third period "new hires" entering the workforce increased the number of those at highest risk for invitation once again. All this analyzed data allowed Burt and Ronchi to profile employees by their "risk of invitation" levels and from this data they were able to create a control group of non-attending BLP employees.

A large Q indicates quarter containing a BLP cohort included in this analysis.
The nine cohorts involved 508 invitations to 361 employees, yeilding 288 participants.

Figure 2 BLP Time Line

The control group for a period contained all of the people invited during the period who did not attend the program. These were people at high risk of participating in the program. The control group was completed by drawing uninvited people according to the probability distributions computed from the analyzed factors outlined above. A sufficient number of people were drawn to have two control-group employees for each program participant at each level of risk - up to the limit of available people in the risk level (e.g., there were not sufficient people in the .9 risk level during any of the periods to double the number of program participants).

Uninvited people were drawn at random in non-zero levels of risk then as a function of risk in the lowest category. Having a control group twice the number of participants was an arbitrary decision intended to provide multiple control observations to offset random sampling for the control groups.

Post Program Participant Performance

Figure 3: BLP effects on Current Senior People

The first bars in Figure 3 show the BLP effect on performance evaluations. The control group (white bars) and the BLP graduates (red bars) are more likely than the general population (black bars) to far exceed expectations (respectively, 41% and 30% versus 15%). At the other extreme, the control group and the BLP graduates are less likely to receive the minimum evaluation of meeting expectations (12% and 16% versus 39% for the grey bar). BLP graduates and their peers in the control group are clearly outstanding within the broader employee population in the top five job ranks, but BLP graduates are significantly ahead of their peers. With respect to far exceeding expectations, BLP graduates are a 31% improvement over their peers in the control group.

The third panel in Figure 3 shows people moving to leadership positions after the company reorganization late in 2002. Bars indicate the percentage of employees promoted to a higher job rank in 2003 or 2004. Given the time order, this is a program correlate rather than a program effect, but the association shows the relative tendency for people in the three categories to be sought for leadership positions. The BLP graduates and their peers in the control group again stood out from the broader employee population: 34% of BLP graduates were promoted and 24% of their peers in the control group were promoted - versus 11% of other people in the top five job ranks. Again, BLP graduates were significantly ahead of their peers in the control group, with 43% higher odds of promotion.

The fourth panel in Figure 3 shows the percentage of employees who decided to leave the Company before the end of the time period. The departures could be for work elsewhere or retirement. About one in ten people leave the Company, but the odds were much lower for BLP graduates. The BLP effect on executive retention was substantial (42%) and statistically significant.

Beginning with the first session in 2001, the faculty sorted program graduates into five categories of participation: stars (led others in successfully engaging the program material), solid (the person's questions and opinions demonstrated comprehension of the program material), good (able but undistinguished participation), weak (the person's questions and opinions demonstrated that they had trouble thinking strategically), and mute (the person neither asked questions nor offered opinion). By the middle of 2004, the 288 program graduates are distributed across the categories as follows: 16% stars, 13% solid, 28% good, 26% weak, and 17% mute.

Figure 4: BLP effects on Current Senior People, Distinguishing Active Participants

The five categories reduce to two when we analyze participation categories with respect to post-program evaluations, promotions, and departures. Stars and solid participants are similar to one another in doing well after the program. These are the "active" participants in Figure 4. Good, weak, and mute participants are similar in showing minor program benefits. These are the "other" participants in Figure 4.

Figure 4 shows program benefits concentrated in the "active" participants. This point has two parts. First, the "active" participants get more benefit from the program. They are much less likely to receive a "meets" evaluation. They are much more likely to be promoted, and much less likely to leave the Company. Second, the less active participants show no more program benefit than the control group, people who never attended the program. This is apparent in Figure 4 from the pink bars being closer in height to the white bars (control group) than the red bars ("active" participants).

Further analysis of the participants and their project teams and feedback from the 2005 email questionnaire referred to earlier showed that passive participants, especially those who did not maintain any post-program contact with other program participants were the most negative about the benefits of the BLP. Of the 29 respondents who were more negative about the benefits than the rest of their teams, 72% were identified as "passive" participants. Conversely, of the 16 participants who were more positive than teammates, 63% were "active" participants.

These positive comments do not contradict a negative team outcome. Rather, the comments reveal a respondent finding an opportunity outside the team to generate benefit from the team project.

The importance of active participation raises the question of individual differences. Active participation could be a proxy for some unmeasured quality that distinguishes superior employees. If we knew what made a program participant "active," and could measure "active" in the control group, it could turn out that "active" people in the control group are just as likely as "active" program graduates to do well in the company. The employees more at risk of invitation were in more senior job ranks, better paid than their peers, receiving higher job evaluations, et cetera. It turns out that "active" participants were at higher risk of invitation to the program. The average risk-of-invitation score for "active" participants is .37 versus .21 for the "other" participants.

In other words, individual differences before the program are correlated with differences in activity during the program. However, when we hold constant the individual differences captured by the summary risk-of-invitation measure, the effects visually apparent in Figure 4 remain statistically significant.

How long before program effects are apparent? Figure 5 shows promotions by when they occurred, before and after the program. Test statistics at the bottom of Figure 5 show that the promotion-chances of the program graduates are indistinguishable from the control group in the year before attending program, and during the year they attend the program.

Figure 5: Time-dependent BLP Effect on Promotion

One year after the program, graduates on average are indistinguishable from the control group, however, those who had been "active" participants were more likely than the control group to be promoted and were well ahead of the "other" program participants, the people who were inactive during the program.

Tests were also run for associations with promotion indicated by people remaining at the same job rank but working for a new boss. There were no statistically significant differences from the control group before or after the program. Further tests were run for associations with promotion indicated by people remaining at the same job rank but doing their work in a new city. Again, there were no statistically significant differences from the control group before or after the program.

Two years later, program graduates on average were more likely than the control group to be promoted, but more striking was the 173% higher odds of promotion for graduates who had been "active" participants.

Burt and Ronchi concluded that, holding constant individual differences coming into the program, the program enhanced the promotion-potential of participants, especially those who were "active" participants.

Conclusion

Performance improves for executives educated in the network structure of social capital. Panel data on executives in a large organization - some of whom had participated in a Business Leadership Program (BLP) grounded in principles of social capital - gave a rare opportunity to estimate social-capital education effects. The effects turned out to be substantial relative to a control group of untrained, but otherwise comparable peers:

Program graduates are 36% to 42% more likely to receive top performance evaluations, 43% to 72% more likely to be promoted, and 42% to 74% more likely to be retained by the Company.

These are noteworthy performance improvements, but their statistical significance is less striking. Effects are just over twice their standard error. That is statistically significant, but not by a large margin. In other words, performance did not improve equally for all graduates. In fact, active participation turned out to be critical to program effects. The subsequent careers of executives who were quiet spectators in the program cannot be distinguished from the careers of people in the control group, people who never attended the program.

What can be concluded here is that the program has substantial positive effects for the participants and the Company. Executive performance is significantly higher for BLP graduates relative to high-performing peers at equal risk of attending the program.