The Development of Innovation Skills through Project Based Learning

By Douglas J. Asbjornsen | September 13, 2015
0 Comment

Summary: Innovation is valued worldwide. Some would argue it has been and continues to be one of the strengths of the United States. Both technical and social innovators are sought after by businesses and other organizations, seeking to invent new products, cure diseases, develop new processes, etc. But how does one become an innovator? How do we as individuals acquire the requisite knowledge, skills, and attitudes that are critical for innovation? Are these directly and/or indirectly taught? If directly taught, are there strategies and methodologies that are more efficient and effective in teaching innovation? Can we measure the learning? Project-based and Problem-based Learning, based upon Constructivist theory, are teaching/learning strategies that may be both effective and efficient in helping students become better innovators. This review asks a number of questions, attempts to answer them, and does a review of the history and empirical research on Project-Based Learning/Problem-Based Learning (PBL). Originally, special emphasis of this study focused on the impact Project-based Learning has on 1) Creativity, 2) Self-Efficacy, 3) Energy, 4) Risk-propensity, and 5) Leadership. After no connections were found due to lack of research focusing on PBL and these constructs, the focus turned to the impact PBL has on overall academic performance.
Keywords: innovation, project-based learning, problem-based learning

Резюме (Дуглас Й. Асбьорнсен: Развитие способности к инновациям на основе проектного обучения): Инновации получают высокую оценку по всему миру. Некоторые утверждают, что инновации были и остаются сильной стороной Соединенных Штатов. Предприятия и организации ведут поиск инноваторов как в технической, так и в социальной области, с целью разработки новых товаров, лечения болезней, развития новых методов и т. д. Но как вообще становятся инноваторами? Как можем мы как индивидуумы получить необходимые для инновации знания, умения и навыки? Получают ли их прямо и/или косвенно при обучении? Если им обучают прямо, то существуют ли тогда стратегии и методы, которые ведут к развитию способности к инновациям более эффективным и результативным путем? Можем ли мы измерить качество обучения? Проектное и проблемно-ориентированное обучение, основываясь на конструктивистских теориях, состоит в стратегиях обучения и учения, которые могут быть как эффективными, так и результативными при поддержке обучающихся в их стремлении стать более успешными инноваторами.
Автор данной статьи поднимает ряд вопросов, пытается найти ответы на них, и представляет обзор истории и эмпирического исследования проектного обучения (Проектное обучение; ПО). Первоначально данное исследование концентрировалось на влиянии проектного обучения на 1) креативность, 2) самоэффективность, 3) энергию, 4) склонность к риску и 5) управлению. После того, как в недостатке исследований не было найдено связей с проектным обучением и его концепцией, основы влияния проектного обучения были направлены на общие академические достижения.
Ключевые слова: инновация, проектно-ориентированное обучение, проблемно-ориентированное обучение

Zusammenfassung (Douglas J. Asbjornsen: Die Entwicklung der Innovationsfähigkeit durch Projektbasiertes Lernen): Innovation wird weltweit geschätzt. Einige würden argumentieren, sie war und bleibt eine der Stärken der Vereinigten Staaten. Sowohl technische als auch soziale Innovatoren werden von Unternehmen und anderen Organisationen gesucht, um neue Produkte zu erfinden, Krankheiten zu heilen, neue Verfahren zu entwickeln, etc. Aber wie wird jemand zum Innovator? Wie können wir als Individuen die für eine Innovation erforderlichen Kenntnisse, Fähigkeiten und Einstellungen erwerben? Bekommt man diese direkt und/oder indirekt beigebracht? Wenn sie direkt gelehrt werden, gibt es dann Strategien und Methoden, die effizienter und effektiver zu innovativer Befähigung führen? Können wir das Lernen messen? Projektbezogenes und problemorientiertes Lernen, basierend auf konstruktivistischen Theorien, sind Lehr-/Lernstrategien, die sowohl effektiv als auch effizient sein können bei der Unterstützung der Schüler, bessere Innovatoren zu werden.
Dieser Artikel wirft eine Reihe von Fragen auf, versucht sie zu beantworten und gibt einen Überblick über die Geschichte und die empirische Forschung zum projektbasierten Lernen (Problem-Based Learning; PBL). Ursprünglich konzentrierte sich diese Studie auf die Auswirkungen projektbasierten Lernens auf 1) Kreativität, 2) Selbstwirksamkeit, 3) Energie, 4) Risikoneigung und 5) Führung. Nachdem aus Mangel an Forschungen keine Verbindungen zum Schwerpunkt PBL und zu diesen Konstrukten gefunden wurden, richtete sich der Fokus auf die Auswirkungen des PBL auf die allgemeine akademische Leistung.

Schlüsselwörter: Innovation, projektorientiertes Lernen, problemorientiertes Lernen

Introduction

How do we grow innovators? What can we do to create the environment for both children and adults where innovation skills and attitudes are not only encouraged, but also developed and nurtured? These questions were sparked by my recent reading of the Global Achievement Gap (Wagner, 2008) and Creating Innovators: The Making of Young People Who will Change the World (Wagner, 2012). My desire to find answers was fueled even further by recent visits to nine of the twelve High Tech High Schools in San Diego, California and to Riverpoint Academy, Mead School District, near Spokane, Washington.

At both High Tech High and Riverpoint Academy, I personally observed the best teaching and learning I have seen in public secondary education. Both students and teachers were heavily engaged. Students reported to me that they loved their schools, were excited about learning, and many of them stated the format (Project-Based Learning) stretched them in their overall learning, confidence, and their communication/collaboration skills, especially public speaking. A number of them talked about how they were shy in the past, but how much confidence they have gained through their school experience.

As an educator, I am interested in all students gaining the knowledge, skills, and attitudes that will help them be successful in life and contribute to society. My hope is that they would take their knowledge and apply the higher level thinking skills of analysis, synthesis, and application to solve problems, invent knew products and processes, and contribute positively to our society and the world. This takes not only knowing facts, but also possessing multiple “soft skills” to create ideas, and turn those ideas into innovations. But what are those skills? What is our school system doing to promote or impede the development of them?

I believe the skills and characteristics of innovation can be identified, taught, learned and assessed. In other words, we can (and do) create innovators. An innovator is a person with a solid knowledge base to innovate from, possesses the requisite “soft skills,” and has the self-efficacy to move an idea into an innovation. Although these innovation skills and characteristics must have been taught and learned through many methods in the past, are there most effective and efficient approaches to teaching and learning these? Project-Based and Problem-Based Learning (PBL) may be one of these strategies. In this paper, I will explore the elements of innovation, define Project-Based/Project-Based Learning, and review some of the empirical research on the efficacy of Project-Based Learning in the development of content knowledge, skills, and attitudes, with special emphasis on those related to innovation.

Hypothesis: Project-Based Learning has a positive, significant effect on the knowledge skills, and attitudes associated with innovation.

Methodology

This hypothesis to support (or not) requires the answers to a number of questions:

  1. What is innovation? How does it differ from creativity?
  2. What are the characteristics of innovators? What is fundamental in the knowledge, skills, and attitudes innovators possess?
  3. What of these can be taught and learned in school?
  4. If they can be taught and learned, can we measure them?
  5. Can they be assessed for individual gains as well as the efficiency and effectiveness of teaching methodology and curriculum?
  6. Do the assessment tools exist today to answer these questions?
  7. If so, are they valid and reliable?

If all of the aforementioned are answered in the affirmative, we can ask the question:

  1. Can PBL contribute to the acquisition of the knowledge, skills, and attitudes inherent in innovators? Is there empirical data to support this?

Foundational to answering this question is to define Project-Based and Problem-Based Learning and to set it in some historical context.

Project-Based and Problem-Based Learning

According to John Larmer, Editor in Chief at the Buck Institute for Education (BIE), “The term ‘project learning’ derives from the work of John Dewey and dates back to William Kilpatrick, who first used the term in 1918” (Larmer, 2014, p. 1). In “The Project Method”: Child-Centeredness in Progressive Education, Kilpatrick outlined the theory of “Wholehearted purposeful activity,” asking teachers to “position each child at the center of the learning process by focusing activities around the interests of the pupil” (History Matters, p. 1). In his writing he states,

“It is the purposeful act with the emphasis on the word purpose that I myself apply the term project.” He goes on to state, “As the purposeful act is thus the typical unit of the worthy life in a democratic society, so also should it be made the typical unit of school procedure. We of America have for years increasingly desired that education be considered life itself and not as a mere preparation for later living.” In his conclusion, he states, “Under proper guidance purpose means efficiency, not only in reaching the projected end of the activity immediately at hand, but even more in securing from the activity the learning which it potentially contains. Learning of all kinds and in its all desirable ramifications best proceeds in proportion to as wholeheartedness of purpose is present” (Kilpatrick, 1918, as cited in History Matters).

Dewey’s comments on his constructivist theory are foundational to PBL and are as applicable today as they were when he wrote them in his many writings. Pragmatically, he asks,

How many students, for example, were rendered callous to ideas, and how many lost the impetus to learn because of the way in which learning was experienced by them? How many acquired special skills by means of automatic drill so that their new judgment and capacity to act intelligently in new situations was limited? How many came to associate the learning process with ennui and boredom? (Dewey, 1938, pp. 26-27)

To counter this boredom and lack of relevance, and to further lay the foundation for PBL, Dewey states,

There is, I think, no point in the philosophy of progressive education which is sounder than the emphasis on the importance of the participation of the learner in the formation of the purposes which direct his activities in the learning process, just as there is no defect in traditional education greater than its failure to secure the active co-operation of the pupil in construction of the purposes involved in his studying. (Dewey, 1938, p. 67)

Barrows gives an overview of Problem-Based Learning, giving background on why it was adopted by the medical profession in the 1970s (Barrows, 1996). Although as stated, the theory is attributed to Dewey and Kilpatrick (and arguably, components of it by many educators/philosophers before), the medical profession was the early adopter, at least on the scale that has resulted in the earliest research on the efficacy of Problem-Based Learning. Barrows states, “The McMaster University Faculty of Health Sciences established a new medical school with an innovative educational approach to be used throughout its entire three year curriculum, an approach now known as problem-based learning. It graduated its first class in 1972” (Barrows, 1996, p. 3). He goes on to state, “By the early 1980s, medical schools with conventional curricula began to develop alternative, parallel problem-based curricula for a subset for their students. The leader in this trend was the Primary Care Curriculum at the University of New Mexico. Later on, other schools took on the more arduous task of converting their entire curriculum to problem-based learning” (Barrows, 1996, p. 1). Schools mentioned include the University of Hawaii, Harvard, and the University of Sherbrooke in Canada (Barrows, 1996). Ultimately, PBL gained momentum, and as Barrows states, “Now countless medical schools in the United States have developed or are developing problem-based curricula in courses, alternative curricula, or as an entire curriculum revision” (Barrows, 1996, p. 4).”

Barrows attributes the dissatisfaction with medical education as the catalyst and motivation to adopt PBL. At McMaster, “students were disenchanted and bored with their medical education because they were saturated by the vast amounts of information they had to absorb, much of which was perceived to have little relevance to medical practice” (Barrows, 1996, p. 4).

Definitions

In defining Project-Based and Project-Based Learning, Lamar states, “At BIE, we see project-based learning as a broad category which, as long as there is an extended “project” at the heart of it, could take several forms or be a combination of:

  • Designing and/or creating a tangible product, performance, or event
  • Solving a real-world problem (May be simulated or fully authentic)
  • Investigating a topic or issue to develop an answer to an open-ended question (Larmer, 2014, p. 2).

He goes on to say, “So according to our “big tent” model of PBL, some of the newer “X-BLs” – problem-, challenge-and design based – are basically modern versions of the same concept. They feature, to varying degrees, all of BIE’s 8 Essential Elements of PBL” (Larmer, 2014, p. 2).

For research purposes, I found this definition too loose and all encompassing-not being specific enough. When does a project or problem meet the criteria to actually qualify as PBL? Much like Cooperative Learning, when does one move from working in groups to true Cooperative Learning? Of course, it is fair to say all project or group activities can be plotted on a continuum, but there needs to be a threshold when the activity “qualifies” so empirical research has some level of validity and reliability.

Barrows found that as more schools utilized PBL, it became harder to define. “All these approaches to problem-based learning represent such a wide variety of methods that now the term has far less precision than might be assumed” (Barrows, 1996, p. 5). He points out that, “a core model or basic definition with which others can be compared is needed” (Barrows, 1996, p. 5). He goes on to state, “the original method developed at McMaster works well as a model” (Barrows, 1996, p. 5).

The characteristics are:

  • Learning is student centered
  • Learning occurs in small groups
  • Teachers are facilitators or guides
  • Problems form the organizing focus and stimulus for learning
  • Problems are a vehicle for the development of clinical problem-solving skills.
  • New information is acquired through self-directed learning (Barrows, 1996, pp. 5-6)

In the same article, Barrows lists the educational objectives of a medical problem-based curriculum. They are:

  • The acquisition of an integrated knowledge base
  • The acquisition of a knowledge base structured around the cues presented by patient problems
  • The acquisition of a knowledge base enmeshed with problem solving processes used in clinical medicine. The development of an effective and efficient clinical problem-solving process
  • The development of effective self-directed learning skills. The development of team skills (Barrows, 1996, pp. 6-7)

The first solid working definition I came across for K-12 education was presented by John W. Thomas in his paper, A Review of Research on Project-Based Learning (Thomas, 2000). Thomas acknowledges the challenges of defining Project Based Learning and the subsequent impact it has on research. He states,

This diversity of defining features coupled with the lack of a universally accepted model or theory of Project-Based Learning has resulted in a great variety of PBL research and development activities. This variety presents some problems for a research review. First, as Tretten and Zachariou (1997) report in their observation report on Project-Based Learning in multiple classrooms, the variety of practices under the banner of PBL makes it difficult to assess what is and what is not PBL, and whether what you are observing is a “real project.” For example, should a design in which project materials are “packaged” or in which student roles are scripted in advance be considered examples of Project-Based learning? Are there particular features that must be present or absent in order for an instructional activity to be considered PBL? Second, differences between instances of PBL may outweigh their similarities, making it difficult to construct generalizations, across different PBL models, about such questions as the effectiveness of Project Based Learning. Third, there are similarities between models referred to as Project-Based Learning and models referred to other labels, for example, “intentional learning” (Scardamilia & Bereiter, 1991), design experiments,” (Brown, 1992) and “Problem-based Learning” (Gallagher, Stepien, & Rosenthal, 1992). Should these other models be considered part of the PBL literature, and if so, on what basis? (Thomas, 2000, p. 2)

I included this extended quote, as I believe Thomas identifies the basic issue and articulated so well the challenges and the questions that need to be asked. I also keep coming up with the same concerns in my initial review of research on the subject. In my search, I have asked some of these additional questions: Who owns PBL? I believe it has been developed and refined through an evolutionary process since the time of Dewey and Kilpatrick. They laid the foundation of Progressive Education, but to my knowledge, did not define what is and isn’t PBL. If no one lays claim to inventing or owning it, who should define it? Fortunately, Thomas (2000) sees a pragmatic need and does present some reasonable criteria to base his review on. This criteria has been sited in multiple studies I reviewed since his was published.

Thomas states, “To capture the uniqueness of Project-Based Learning and to provoke a way of screening out non-examples from this review, the following set of criteria are offered. These criteria do not constitute a definition of PBL, but rather are designed to answer the question, “what must a project have in order to be considered an instance of PBL?” (Thomas, 2000, p. 3).

Although Thomas elaborates with greater detail on each, he lists five criteria that was used in his paper and was exceptionally helpful in my review. They are:

  • PBL projects are central, not peripheral to the curriculum
  • PBL projects are focused on questions or problems that “drive” students to encounter (and struggle with) the central concepts and principles of a discipline.
  • Projects involve students in a constructive investigation
  • Projects are student driven to some significant degree
  • Projects are realistic, not school like (Thomas, 2000, pp. 3-4))

Based upon these criteria, Thomas does include in his review “research related articles on “project-based learning,” “problem-based learning,” “expeditionary learning,” and “project based instruction” that conform to the criteria above” (Thomas, 2000, p. 4).

In an Overview of Problem-based Learning: Definitions and Distinctions, an article from the Interdisciplinary Journal of Problem-based Learning, lists and describes ten characteristics of Problem-based Learning (Savery, 2006, pp. 12-14). They are:

  • Students must have the responsibility for their own learning
  • The problem simulations used in problem-based learning must be ill-structured and allow for free inquiry
  • Learning should be integrated from a wide range of disciplines or subjects

Collaboration is essential

  • What students learn during their self-directed learning must be applied to the problem with reanalysis and resolution
  • A closing analysis of what has been learned from work with the problem and a discussion of what concepts and principles have been learned are essential
  • Self and peer assessment should be carried out at the completion of each problem and at the end of every curricular unit
  • The activities carried out in problem-based learning must be those valued in the real world
  • Student examinations must measure student progress towards the goals of problem-based learning
  • Problem-based learning must be the pedagogical base in the curriculum and not part of a didactic curriculum

Although all ten above are important, I would like to highlight two in particular. Problems must be ill-structured or they are not really problems. Along with this, PBL (Project or Problem-based Learning) ideally is the base or put a different way, is the curriculum, not just a unit or an addition to it. If not, projects often are more activities and are not properly structured.

What is innovation? How does it differ from creativity?

Although often used interchangeably, there are some subtle differences between the two terms, which requires innovation to have additional skill sets beyond creativity. For research purposes, both are challenging to define. For example, Batey states, “It may be argued that the primary issue to hamper creativity research centers around the lack of a clear and widely accepted definition for creativity, which, in turn, has impeded efforts to measure the construct” (Batey, 2012, p. 55). He goes on to state, “Most researchers agree that creativity may be defined with regards to the terms new and useful” (Batey, 2012, p. 56)

Searching the Internet, especially on sites relating to innovation in business, there are some trends. By not coming from journals and not used for empirical research, these thoughts on creativity are limited. However, in absence of a widely accepted definition for creativity in the literature, some ways of looking at creativity and innovation are: “Creativity refers to the ability to come up with new ideas, the ability to think widely, to have a free mind and approach matters in a new way. Whereas innovation is the ability to confine the creative ideas and make them turn into reality so as to achieve successful performance” (Link, 2013, p. 1).

“Creativity: The process of generating ideas-divergent thinking. Innovation: The sifting, refining and implementation of ideas-convergent thinking-putting ideas into action” (Link, 2013, p. 1).

“Creativity is by far much different than innovation where creativity is the capability or act of conceiving something original while innovation is the implementation of something new. When people come up with new ideas, this is a display of creativity but there is no innovation until you take the risk of implementing it” (Link, 2013, p. 1).

“The difference between creativity and innovation is simply that creativity refers to the ability to generate new ideas while innovation is the ability to turn new ideas into reality” (Link, 2013, p. 1).

Creativity is subjective, making it hard to measure, as our creative friends assert. Innovation on the other hand, is completely measurable. Innovation is about introducing change into relatively stable systems. It’s also concerned with the work required to make an idea viable. By identifying an unrecognized and unmet need, an organization can use innovation to apply its creative resources to design an appropriate solution and reap a return on its investment (Marshall, 2013, p. 1).

If one uses the concepts above to loosely define creativity and innovation, creativity is a sub-set of innovation. In other words, in the extreme, one can be creative with new ideas, but never applied. To be creative, there is no requirement to put these creative thoughts to work or even be outwardly expressed through art, literature, etc. Thoughts and ideas can be highly creative, but never shared. Innovation on the other hand, requires a number of skill sets, depending upon the complexity of the task of bringing the creative idea to life. In summary, it is a fair argument that one can be creative, but not innovative. However, one cannot be innovative, without being at least somewhat creative. Of course, this statement is dependent upon one’s working definition of creativity.

What are the characteristics of innovators? What is fundamental in the knowledge, skills, and attitude innovators possess?

An all-inclusive list of the characteristics or attributes of innovators is illusive. Like the construct of creativity, one can recognize it when seen and/or experienced, but to list the all the attributes of an innovator, one risks leaving out essential elements. One reason for this is there are different types of innovators. In two broad categories there are STEM innovators and social innovators (Wagner, 2012). Are the requisite skills and characteristics the same? There is probably great overlap, but there could be differences. If so, what are they? Certainly, the STEM innovators must have a base knowledge of their discipline. Core knowledge in science and math is critical for this type of innovation. One cannot connect the dots in a unique way without knowing something about the dots. However, are there other key elements? Possibly.

In my review, I found two sources that may be helpful in identifying at least some of the characteristics of innovators. Supported by the United Kingdom’s National Endowment for Science, Technology and the Arts (NESTA), Chell and Athayde at Kingston University developed the Youth Innovation Skills Measurement Tool. “The tool measures five generic skills that underpin innovative behaviour and form a set of attributes clearly linked to the innovation process” (Chell & Athayde, 2009, p. 3). They are, 1) Creativity (imagination, connecting ideas, tackling and solving problems, curiosity), 2) Self Efficacy (self belief, self assurance, self awareness, feelings of empowerment, social confidence), 3) Energy (drive, enthusiasm, motivation, hard work, persistence and commitment), 4) Risk-propensity (a combination of risk tolerance and the ability to take calculated risks), and 5) Leadership (vision and the ability to mobilize commitment) (Chell & Athayde, 2009). “The skills were identified through a literature review and through testing concepts with separate focus groups of young people and teachers from different disciplines in schools and colleges in Greater London and Hampshire” (Chell and Athayde, 2009, p. 3).

In revisiting Wagner’s Creating Innovators (Wagner, 2012) he identifies “seven survival skills.” They are, 1) Critical Thinking and problem solving, 2) Collaboration across networks and leading by influence, 3) Agility and adaptability, 4) Initiative and entrepreneurship, 5) Accessing and analyzing information, 6) Effective oral and written communication, 7) Curiosity and imagination. He goes on to state in his book, “However, the list doesn’t touch on some of the qualities of innovators that I now understand as essential-such as perseverance, a willingness to experiment, take calculated risks, and tolerate failure, and the capacity for “design thinking,” in addition to critical thinking” (Wagner, 2012, p. 12).

Wagner states, “IDEO’s concept of “design thinking” is widely regarded as a way of viewing the world that is fundamental to any process of innovation” (Wagner, 2012, p. 13) He goes on to site Tim Brown’s article from the Harvard Business Review, identifying five characteristics of design thinkers. They are, 1) Empathy, 2) Integrative thinking, 3) Optimism, 4) Experimentalism, and 5) Collaboration (Wagner, 2012, p. 13).

Although there are differences in each the aforementioned attributes of Chell and Wagner’s work, there is a great deal of overlap and some are sub-components of broader categories. For example, “Leadership” can be an extremely broad construct and encompasses “collaboration across networks and leading by influence” as well as a number of others including “empathy” and can even include “effective oral and written communication.” As the five attributes identified in the Youth Innovation Skills Measurement Tool can encompass all of the above to some degree, I will use these for a review of the research.

What of these can be taught and learned in school?

Although the construct of innovation (by definition of the term construct) cannot be directly measured, all of its attributes or elements individually can be taught, learned, and measured (at least, in theory). Some are easier than others. Some may be more dependent upon personality such as Risk-propensity. Others could be impacted by health issues such as Energy. Certainly, a student may have natural aptitudes for some or all of these attributes, just as some students are naturally gifted as artists, musicians, etc. However, if one can identify the characteristics of innovators to some level of fidelity, one can deliberately and intentionally teach the knowledge and skills associated with innovative behavior. One can also impact attitudes through direct teaching and modeling of values and priorities, although this can be complex due to multiple variables.

If they can be taught and learned, can we measure them? Can they be assessed for individual gains as well as the efficiency and effectiveness of teaching methodology and curriculum? Do the assessment tools exist today to answer these questions? If so, are they valid and reliable?

The answers to these questions are complicated. Yes, we can measure all of the aforementioned attributes individually, especially if they are truly elements. If the proper assessment tool is used and set up correctly, we can also evaluate the methodology and curriculum. Some attributes are easier to measure than others, as many can be relatively hard to define. For example, as stated previously, “Creativity” can be difficult to find an agreed upon definition. In The Measurement of Creativity: From Definitional Consensus to the Introduction of a New Heuristic Framework, the author states, “The cultural value placed upon creativity in the arts, sciences, technology, and political endeavor is immense … Yet, despite the undeniable importance of creativity, it is infrequently studied in comparison to other similar constructs like intelligence or personality” (Batey, 2012, p. 1). As mentioned earlier, a significant factor and a major reason for this, it is difficulty to define. It is fair to say, that the harder a construct is to define, the more difficult it is to identify and measure its elements.

A measurement that assesses the knowledge, skills and attitudes attributed to innovative behavior in a holistic way (taking into account all the listed attributes) may exist, but at the time of this writing, I have not discovered it. Initially, I was encouraged when I found the Youth Innovation Skills Measurement Tool. After contacting Dr. Elizabeth Chell, one of the developers (personal communication, July 22, 2014), I acquired a link to the tool. In reviewing and personally taking it, I found it to be a 38 item, on-line questionnaire, utilizing a seven point Likert scale. As the responses are self-reported, the tool has weaknesses that are inherent to any type of self-reporting instrument. At best, it can touch on attitudes and self-perception associated with innovative behavior, but does not actually measure the development of the knowledge and skills identified with innovation.

I found one doctoral dissertation utilized the Kirton Adaption-Innovation Inventory with middle school students to measure creativity (Selby, Treffinger, Isaksen, and Powers, 2011). This tool was designed for corporate use and may have minimal value in K-12 settings. I will continue to do research on this tool as it may have been used in additional K-12 studies. However, at the time of this writing, I have not found any.

Can Project Based Learning contribute to the acquisition of the knowledge, skills, and attitudes inherent in innovators? Is there empirical data to support this?

Review of Literature

I found the research investigating Project and Problem-Based Learning is varied and to my surprise, relatively weak in quantity and quality, at least in regards to K-12 education. I state this not to be critical, but recognizing the number of variables and challenges that come into play. As stated in one meta-analysis, “Enthusiasm for Problem-based learning (PBL) is widespread, yet there exists little rigorous experimental evidence of its effectiveness, especially in K-12 populations” (Wirkala & Kuhn, 2012, p. 1157). In my query, I found the vast majority of the research comes from the medical profession, limiting at least to some degree, the value to K-12 education. However, as with any quality research, some things can be gleaned from it.

In “When is PBL More Effective? A Meta-synthesis of Meta-analyses Comparing PBL to Conventional Classrooms,” the researchers report in their Results and Discussion section, “For the Knowledge assessment category, measures of short-term knowledge acquisition and retention returned mixed results, but tended to favor traditional learning approaches” (Strobel & Barneveld, 2009, p. 53). However, they also reported, “Long term knowledge retention favored PBL” (Strobel & Barneveld, 2009, p. 54). Along with this, they go on to state, “Overall, students and staff indicated greater satisfaction with the PBL approach to learning” (Strobel & Barneveld, 2009, p. 54).

As expected, one can find research on both sides of the effectiveness of PBL in fields outside of medicine including K-12 education. In A Problem Based Learning Meta Analysis: Differences Across Problem Types, Implementation Types, Disciplines, and Assessment Levels, the researchers looked at “47 outcomes outside the fields of medical education and allied health” (Walker & Leary, 2009, p. 24). They report in their Conclusion section, “Early findings pointed to concept or content knowledge differences favoring lecture” (Walker& Leary, 2009, p. 24). In the same section, they state, “As initially posted by Barrows (1986), problem type does appear to play a role in the effects of PBL” (Walker & Leary, 2009, p. 25). They go on to state, “While much more needs to be known about which PBL methods were employed before confident assertions can be made, closed loop problem based learning appears to improve student learning outcomes (dw = 0.54)” (Walker & Leary, 2009, p. 25). The effect size they are reporting is from Jonassen’s work (Jonassen, 2000).

As stated above, there is no lack of enthusiasm for PBL in the literature, especially by proponents connected with organizations promoting PBL. For example, Bell in Project-Based Learning for the 21st Century: Skills for the Future states,

“Standardized testing is one measure of achievement. Each state has its own standardized measure of academic competency. Each standardized test only measures the specific content knowledge it is designed to test. In measuring basic academic subject proficiency, standardized testing shows that students engaged in PBL outscore their traditionally educated peers (Geier et al. 2008)” (Bell, 2010, pp. 39-40).

In reviewing the original research, I found Geier did not make this claim. Instead, they stated in the discussion section, “We do not claim nor do our data support a conclusion that inquiry science units alone will enhance achievement. Rather the results indicate that an effort incorporating and aligning the best practices in curriculum, professional development, and learning technology in the context of a systemic reform can achieve substantive results on politically important measures” (Geier, et al. 2008, p. 934).

Although Bell misrepresented the conclusion reported by the research team, the claim did lead me to some enlightening research showing positive results for urban youth in Detroit. Taken from the abstract,

The effort was one component of a systemic reform effort in the Detroit Public Schools, and was centered on highly specified and developed project-based inquiry science units supported by aligned professional development and learning technologies. Two cohorts of 7th and 8th graders that participated in the project units are compared with the remainder of the district population, using results from high stakes state standardized test in science. Both the initial and scaled up cohorts show increases in science content understanding and process skills over their peers, and significantly higher pass rates on the statewide test. (Geier, et al, 2008, p. 922)

This was an encouraging study in that it showed an intervention that produced significant gains for students, especially since the student body of the Detroit School District is comprised of 91% African American and 5% Latino students. The study went three years, ending with the 2000-2001 school year. Approximately 5,000 students were involved along with thirty-seven teachers in 18 schools. The Michigan Educational Assessment Program (MEAP), a statewide standardized assessment was used. The test is aligned with the state objectives for science achievement. A treatment sample of 760 students in Cohort 1 and 1,043 in Cohort II, was compared with a group of 8,900 and 8,662 respectively. The treatment was participation in at least one of the three project-based inquiry science units. Some students participated in two and some participated in all three (Geier, et al, 2008). The results were impressive.

In Cohort I, students who completed at least one LeTus (The Center for Learning Technologies in Urban Schools) unit during 7th or 8th grade significantly outperformed their DPS (Detroit Public Schools) peers on their overall MEAP Science score. Moreover, the difference was not confined to one area of the test. Higher scores were achieved in all three science content areas (Earth, physical, and life science) and both science process skill groups (constructing and reflecting) measured by the science MEAP. (Geier, et al, 2008, p.930)

The standardized effect size for Cohort I was .44 and Cohort II was .37 (Geier, et al, 2008).

According to Hattie, “For any particular intervention to be considered worthwhile, it needs to show an improvement in student learning of at least an average gain-that is an effect size of at least 0.40. The d = 0.40 is what I referred to in Visible Learning as the hinge-point (or h point) for what is and what is not effective” (Hattie, 2012, p. 3). Given the size and length of the study and the effort the researchers put forth to reduce bias, etc., these effect sizes are encouraging.

I parked on this research as it was large scale and appeared to be well done in both design and in the integrity of reporting the results. Not all of the research I reviewed was of this quality. In fact, the majority was not. I also was pleased in that it gave hope for a very challenging, urban school district serving a huge minority population. However, it would be misleading to claim that problem-based learning alone was the only variable contributing to these gains. The authors repeatedly stressed the alignment component of their efforts. In referring to their partnership of the University of Michigan and Detroit Public Schools, “We refer to an effort where standards, policy, curriculum, instruction, professional development, assessment, and learning technologies are coherent and integrated as highly aligned” (Geier, et al, 2008, p. 924).

Along with proponents of PBL making misleading statements and promising and encouraging research utilizing PBL units (along with other variables), the literature also has its detractors. For example, a fairly hostile paper towards inquiry-based teaching and learning was presented in the Educational Psychologist. In their paper, Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching (Kirschner, Sweller, & Clark, 2006), the authors state in their conclusion,

After a half century of advocacy associated with instruction using minimal guidance, it appears that there is no body of research supporting the technique. In so far as there is any evidence in controlled studies, it almost uniformly supports direct, strong instructional guidance rather than constructivist-based minimal guidance during the instruction of novice to intermediate learners. (Kirschner, Sweller, & Clark, 2006)

The authors were critical of constructivist learning, especially for novice learners and presented their arguments from a number of angles. In particular, they utilized learning research on long-term memory, including cognitive load theory.

In a follow up paper printed at a later date in the same journal, Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006), the authors point out that “Kirschner et al. have mistakenly conflated PBL and IL with discovery learning” (Hmel-Silver, Duncan, & Chinn, 2007, p. 99). In their conclusion, the authors state,

Even in the limited review of research on PBL and IL (Inquiry Learning), it is clear that the claim that PBL and IL “does not work” is not well supported, and, in fact, there is support for the alternative. But we would argue that “does it work?” is the wrong question. The more important questions to ask are under what circumstances do these guided inquiry approaches work, what are the kinds of outcomes for which they are effective, what kinds of valued practices do they promote, and what kinds of support and scaffolding are needed for different populations and learning goals. The questions that we should be asking are complex as is the evidence that might address them. It requires not only learning content but also learning “softer skills” (Bereiter & Scardamalia, 2006) such as epistemic practices, self-directed learning, and collaboration that are not measured on achievement tests but are important for being lifelong learners and citizens in a knowledge society. (Hmel-Silver, Duncan, & Chinn, 2007, p. 105)

As mentioned earlier, the quality of the research on PBL has come into question, at least in the validity and reliability of the instruments that have been used. (Belland, French, & Ertmer, 2009), did a review of 33 empirical studies on Problem-Based Learning, focusing on target outcomes of deep content learning, problem solving ability, and self directed learning. Results indicated that, “few studies included 1) theoretical frameworks for the assessed variables and constructs, 2) rationales for how chosen assessments matched the constructs measured, or 3) other information required for readers to assess the validity of author’s interpretations” (Belland, French, & Ertmer, 2009, p. 59). I did not perceive the authors were overly critical of the research itself, but emphasized that researchers be clearer on their rationale for selection of instruments. The authors states, “Rather, the solution is to report better on the selection, use, and psychometric properties of the measures. Such information should lead to researchers realizing the shortcomings of measures and seeing the need to improve these measures for future research” (Belland, French, & Ertmer, 2009, p. 80). The authors acknowledged in their Limitations section that “the majority of research reviewed here was in the area of medical or allied education” (Belland, French, & Ertmer, 2009, p. 81). Again, even though they were interested in looking at K-12 research, as of 2009, they were limited in what they could find.

Conclusion

Through the course of this study, I find I am not able to support my hypothesis: Project-Based Learning has a positive, significant effect on the knowledge, skills, and attitudes associated with innovation. This conclusion is in no way stating that it does not. Nor is it stating that this is not an important area of future research to explore. It is stating that given the time constraints and limited resources for this project, I have not been able to adequately answer the foundational questions I proposed in the methodology section (even with unlimited resources, this would still be a challenging task). For empirical research to be both valid and reliable, one must have consensus on definitions and on the instruments that accurately measure the outcome being tested. The definitions for the constructs of innovation and creativity are illusive. Like many things, you know it when you see it. To measure them holistically is even more difficult. Depending on one’s definition, potentially impossible. The only instrument I could find in my research that was applicable to K-12, only measured self-reported attitudes and self-perception. Although somewhat helpful, I found the instrument disappointing and of limited use as a measurement of innovative skills when I actually explored it. However, it was helpful in identifying some possible elements of innovation. As a result, the tool gave some framework to start from and may be a base for some future research.

After discovering the aforementioned challenges, I refocused my review of PBL research on a broader question: does it work? I abandoned my initial query to find PBL research that tied into the innovation elements I have previously described. I did this for two reasons. First, I realized I needed a broader base of knowledge on what the research says about PBL to move forward with possible connections. Second, pragmatically I found no research in my first review of the literature that even looked directly at the development of “soft skills” through PBL.

To the question, does Project-based Learning work? The answer to this question is yes, no, and most of all, it depends. The reason for the diversity of the answers is I was asking the wrong question. My future questions must be much more specific to get any meaningful answers. As so well articulated by Hmel-Silver, Duncan, & Chinn, the better questions to ask are “under what circumstances do these guided inquiry approaches work, what are the kinds of outcomes for which they are effective, what kinds of valued practices do they promote, and what kinds of support and scaffolding are needed for different populations and learning goals” (Hmel-Silver, Duncan, & Chinn, 2007, p. 105).

Focusing strictly on PBL, the vast amount of research I initially found was from the medical profession, especially in the training of physicians. I saw trends where direct instruction in acquiring knowledge was more effective, while components of application and enjoyment of the learning process favored PBL.

In the K-12 research, essentially all of it focused on academic achievement. Again, there are so many variables, including how one is defining PBL, the research can lean both ways. Much of the research that was in support of PBL came out of BIE, a strong proponent of PBL. Because of a strong bias to advocate for its position, one needs to be leery of any organization with a vested interest in funding or even reporting research, or at least be aware of the potential for conflict of interest issues. That is not to say that the BIE research lacks integrity, but one should at least be aware of the interest they have in promoting PBL.

Along with the lack of K-12 research and challenges with definitions (does the unit or curriculum meet the criteria to be identified as PBL?), the vast majority of school research utilizes relatively short units of PBL. Ideally, Project-based learning is not just part of the curriculum, it is the curriculum. It is taught by teachers with extensive training in the development of robust, well thought out projects or problems, which are designed to build content knowledge and various “soft skills” such as problem solving and collaboration. With this said, PBL takes a great deal of knowledge and skill, along with a significant amount of time and dedication to implement effectively. In many ways, traditional approaches can be easier. Dewey laid this out in stating, “It is, accordingly, a much more difficult task to work out the kinds of materials, of methods, and of social relationships that are appropriate to the new education than in the case with traditional education.” (Dewey, 1938, p. 29). As I stated in High Tech High (Asbjornsen, 2014), “This model is extremely hard work for teachers and support staff. There must be structure underneath the “looseness.” There clearly are high expectations that require a huge amount of planning and focus to effectively implement, including the value of and skill sets around collaboration. The traditional preparation of teachers would need to be modified for this to be successful on a larger scale.” I would further state that Detroit Public Schools’ systemic alignment that was earlier highlighted is key. PBL, without the underlying support structure and alignment may be doomed for failure. Again, it is very hard work and requires significant knowledge and skills to make it effective.

In looking at future research, another problem arises. That is, to truly evaluate PBL as the curriculum (not as a unit), one is often forced to deal with skewed populations that immediately introduce bias into the study. Although they may exist, as my review of the literature was far from exhaustive on this subject, almost all K-12 schools using a “pure” form of Project Based Learning as described by Thomas (2000) are unique in some way or another. Even in schools like High Tech High in San Diego that use a lottery system for admission and have shown phenomenal results with low income and minority students. According to their website, 98% of High Tech High graduates have gone on to college (High Tech High, 2014). However, it is fair to say the school system has a special population. These students are self-selecting by applying. They may not be testing or interviewing to get in, but they almost all know the teaching model used and expectations before they apply and submit their name for the lottery. All have goals of not just going to college, but the majority desiring to be accepted into the University of California system. Consequently, the student body at High Tech High is not composed of the average San Diego student. One in five have already won the lottery. Most feel very fortunate to be “in” and are highly motivated to stay there. With this, it is hard to find a control group using traditional methods that truly would be comparable, showing one or the other to have a significant effect.

As with all research, especially in the social sciences, additional research is always needed as more questions are developed and different angles beg to be explored. In PBL, I believe the research should be expanded. In his Meta-analysis, Thomas states,

With a few exceptions, much of the research reported above incorporates only one or two indices of learning to measure PBL effectiveness, typically academic achievement and conceptual understanding. Elsewhere, some of the newer constructivist models of learning have proposed that evaluations of student learning be conducted using multiple indices, supplementing measures of understanding (application, explanation, concept mapping) with those of collaboration, metacognitive ability, communication, and problem solving (Thomas, 2000, p. 37).

I totally concur. Even though his report is dated, I found the aforementioned still true. Although, certainly my review of the research was not comprehensive, I found very little reported in these areas. But even if the research on PBL improves and expands and many of the concerns and challenges in this area are addressed and remedied, what if it is found there is no significant positive effect on academic achievement? What if traditional methods prove over time to be slightly more effective and efficient than PBL in the development of content knowledge? The question still exists, what impact does PBL have on the enjoyment, love, and passion of learning? What about the “soft skills” associated with innovation? The examination of this is still relevant. In fact, this is where I think the real potential exists in developing the skills and attitudes associated with creativity, self-efficacy, energy, risk propensity, and leadership. Again, although hard to define and measure, future research should be conducted to explore these relationships.

In conclusion, this study required the refocus of efforts in the middle of the research. The discovery that the quest to relate innovation skills to PBL is immense and is well beyond the scope of this paper was illuminating. In addition to this, although the review of research in PBL involved the reading of a number of articles and journals, relatively few are highlighted here. There are only enough to give a sampling of some of the perspectives, conclusions, and challenges that exist in examining PBL as an effective strategy to improve academic performance, let alone its impact on “soft skills” that may contribute to innovation. The research collected and reviewed lays a foundation for my own future in PBL and innovation.

References

  • Asbjornsen, D. J. (2014): High tech high (Unpublished paper), Seattle Pacific University.
  • Asbjornsen, D. J. (2014): What I believe about public education (Unpublished paper), Seattle Pacific University.
  • Barrows, H. S., Myers, A., Williams, R.G., & Moticka, E. J. (1986): Large group problem-based learning: A possible solution for the ‘2 sigma problem.’ In: Medical Teacher, 8(4), pp. 325-331.
  • Barrows, H. S. (1996): Problem-based learning in medicine and beyond: A brief overview. In: New directions for Teaching and Learning, no. 68. San Francisco: Jossey-Bass, pp. 3-11.
  • Batey, M. (2012): The Measurement of Creativity: From definitional consensus to the introduction of a new heuristic framework. In: Creativity Research Journal, 24(1), pp. 55-65. http://dx.doi.org/10.1080/10400419.2012.649181
  • Bell, S. (2010): Project-based learning for the 21st century: Skills for the future. The Clearing House, 83, pp. 39-43. Retrieved from http://www.tandfonline.com/doi/abs/10.1080/00098650903505415#.U9mrhlaAdBV
  • Belland, B. R., French, B. F., & Ertmer, P. A. (2009): Validity and problem-based learning research: A review of instruments used to assess intended learning outcomes. In: Interdisciplinary Journal of Problem-Based Learning, 3 (1). http://dx.doi.org/10.7771/1541-5015.1059
  • Bereiter, C., & Scardamalia, M. (2006): Education for the knowledge age: Design-centered models of teaching and instruction. In: P. A. Alexander & P. H. Winne (Eds.): Handbook of educational psychology (2nd ed., pp. 695-713). Mahwah, NJ: Erlbaum
  • Brown, A. L. (1992): Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. In: Journal of Learning Sciences, 2, 141-178.
  • Chell, E. & Athayde, R. (2009): Development of the youth innovation skills measurement tool: The identification and measurement of innovative characteristics of young people. Retrieved from http://eprints.kingston.ac.uk/5985/2/Chell-E-5985.pdf
  • Dewey, J (1938): Experience and education. New York, NY: Simon & Schuster.
  • Geier, R., Blumenfeld, P. C., Marx, R. W., Krajcik, J. S., Soloway, E. & Clay-Chambers, J. (2008): Standardized test outcomes for students engaged in inquiry-based science curricula in the context of urban reform. In: Journal of Research in Science Teaching, 45(8), pp. 922-939.
  • Gallagher, S. A., Stepien, W. J., & Rosenthal, H. (1992): The effects of problem-based learning on problem solving. In: Gifted Child Quarterly, 36, pp. 195-200.
  • Hattie, J. A. C. (2012): Visible learning for teachers: Maximizing impact on learning. London: Routledge.
  • High Tech High Results Website (n.d.). Retrieved from http://www.hightechhigh.org/about/results.php
  • History Matters (n.d.). The project method: Child-centeredness in progressive education. Retrieved from http://historymatters.gmu.edu/d/4954/
  • Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007): Scaffolding and achievement in problem-based and inquiry learning: a response to Kirschner, Sweller, and Clark (2006). In: Educational Psychologist, 42(2), 99-107.
  • Jonassen, S. D. (2000): Toward a design theory of problem solving. In: Educational Technology Research and Development, 48(4), pp. 63-85.
  • Kirschner, P. A., Sweller, J. & Clark, R.E. (2006): Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. In: Educational Psychologist, 41, pp. 75-86. Retrieved from http://dx.doi.org/10.1207/s15326985ep4102_1
  • Larmer, J. (2014): Project-Based learning vs. Problem-Based Learning vs. X-BL. Retrieved from http://www.edutopia.org/blog/pbl-vs-pbl-vs-xbl-john-larmer
  • Link, J. (2013, March): Idea-Links. Retrieved from http://thenewcreativity.com/creativity-vs-innovation-yes-there-is-a-difference/
  • Marshall, D. (2103, April). Business Insider. Retrieved from http://www.businessinsider.com/difference-between-creativity-and-innovation-2013-4
  • Savery, J. R. (2006): Overview of problem-based learning: Definitions and distinctions. In: The Interdisciplinary Journal of Problem-based Learning, 1(1), pp. 9-20.
  • Scardamilia, M. & Bereiter, C. (1991): Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. In: Journal of the Learning Sciences, 1, pp. 37-68.
  • Selby, E. C., Treffinger, D. J., Isaksen, S. G., & Powers, S. V. (2011): Use of the Kitron adoption-innovation inventory with middle school students. In: The Journal of Creative Behavior, 27(4), pp. 223-235.
  • Strobel &Van Barneveld (2009): When is pbl more effective? A meta-synthesis of meta-analyses comparing pbl to conventional classrooms. Interdisciplinary Journal of Problem Based Learning, 3(1), pp. 44-58.
  • Thomas, J. W. (2000). A review of research on project-based learning. The Autodesk Foundation. Retrieved from http://w.newtechnetwork.org/sites/default/files/news/pbl_research2.pdf
  • Tretten, R. & and Zachariou, P. (1997): Learning about project based learning: self assessment preliminary report of results. The Autodesk Foundation.
  • Walker, A., & Leary, H. (2009): A problem based learning meta analysis: Difference across problem types, implementation types, disciplines, and assessment levels. In: International Journal of Problem-based learning, 3, pp. 12-43. http:/dx.doi.org/10.7771/1541-5015.1061
  • Wirkala, C., & Kuhn, D. (2011): Problem-based learning in K-12 education: Is it effective and how does it achieve its effects? In: American Educational Research Journal, 48, pp. 1157-1186. http://dx.doi.org/10.3102/0002831211419491
  • Wagner, T. (2008): The global achievement gap. New York, NY: Basic Books.
  • Wagner, T. 2012): Creating innovators. New York, NY; Simon and Schuster.

About the Author

Douglas J. Asbjornsen: Doctoral student, Seattle Pacific University, Seattle (USA), former public school teacher, principal at all levels, and school superintendent, contact: asbjod@spu.edu

Category: