March 6, 2010
eCoaching Tip 75 Designing for Evaluating and Creating Levels of Bloom’s Taxonomy
This ecoaching tip is the first tip designed as a follow-on from a SLPA webinar, in this case, the webinar on 2/23/10 that focused on the two higher levels of Bloom’s taxonomy, evaluating and creating.
Kudos to all the faculty who participated and shared their practices and teaching experiences. Your descriptions of higher-level experiences such as case studies and critiques provided valuable contexts for higher-level experiences. A special thanks to Kathy Hollywood who teaches MLLS 717 – Leading people and Managing Relationships — for sharing her evaluating assignment on critiquing the effectiveness of a recent manager and also for sending her favorite reference on critical thinking –Thinking critically about critical thinkingby Mandernach (2006).
This tip shares a sampling of faculty insights and comments from the webinar as well as some of the key ideas about the character and benefits of evaluating and creating experiences. We hope that you are tempted to review the archive and check out the full archive and the resources in the webinar area.
The Webinar Question – Evaluating and Creating Experiences
The question that we focused on answering in the webinar was “How can I design learning experiences that encourage the “use” of knowledge as well as the “acquisition” of knowledge?”
Designing learning experiences for learners to use knowledge in evaluating and creating experiences is always somewhat of a challenge. Yet these are the levels that most directly engage and absorb your learners. These are the levels that can reach the standard of significant learning that integrate people, ideas and realms of life as encouraged by Fink (2004). These more demanding cognitive levels require learners to integrate knowledge from readings, discussions and their existing knowledge and use the knowledge in checking, judging, generating, planning and creating, all higher level cross-functional skills.
- K. Cunningham who teaches one of the intro leadership intro courses and who favors the use of case studies for higher level learning shared his observation that learners get most engaged when they make connections between the content knowledge and some of their own life experiences. In doing so, Cunningham feels that students are creating something new. The excitement for learners that results from “making new meaning” was confirmed by many others.
Refresh of Bloom’s Taxonomy
Here is a quick refresh on Bloom’s Taxonomy. As you may recall, Bloom’s taxonomy was a large-scale 8-year team effort. Launched at a meeting of the American Psychology Association in 1948, the effort resulted in a publication that focused on one of three domains of knowledge: the cognitive or knowledge domain. The taxonomy did not address the affective (feeling and emotion) or the psychomotor domains. Those taxonomies came out respectively in 1964 (Krathwohl, Bloom, Maisie) and 1972 (Simpson).
During the 1990’s, a student of Bloom’s by the name of Lorin Anderson, chaired a new group who were tasked with updating the taxonomy, hoping to add relevance for 21st century students and teachers. This revision includes several seemingly minor yet actually quite significant changes. Here is the original and the revised taxonomy.
As you can see there are two significant changes. First, the cognitive levels are now described with verbs (e.g., application to applying) and secondly, the highest level is creating and evaluating is second highest.
To sum up the revisions the focus in the revised taxonomy is on the steps in acquiringknowledge and then usingthe knowledge.
As you are building your syllabus and designing your learning experiences, a good question to ask yourself is, “How do you envision your students using the knowledge they are hopefully acquiring from the combination of resources and your expertise?”
Evaluating and Creating
Let’s now take a closer look at evaluating and creating experiences. Here is an easy way of differentiating them. The focus of evaluating experiences is on a process, product, or design developed by someone else, including other learners and the focus of creating experiences is on a process, product or design developed or being lived by the learner or learner group.
Definition of Evaluating
Just what is the cognitive process involved in evaluating? Evaluating is “making judgments based on criteria and standards.” For evaluating experiences, the criteria or standards can be identified or selected by the student; in other cases, the criteria, such as professional standards of research or practice, might be provided to students.
For historians there might be standards or criteria for assembling and interpreting information; for scientists, learners might use the accepted standards, criteria for experiments; and in business learners might be expected to apply standards and practices for pricing strategies.
The two cognitive processes most often used in evaluating include checking and judging. Examples might include experiences where learners are expected to detect inconsistencies or fallacies, determine if a process has internal consistency, or if a process is effective. Examples of judging or critiquing experiences might be where learners judge the merits of a product or operation based on specified criteria or judge the appropriateness of a procedure or solution to a problem (Mayer, 2002). The essential element of evaluating experiences is the use of standards or criteria. Fortunately these standard or criteria often map to the core concepts or essential learner outcomes.
More Examples of Evaluating Experiences
More examples of checking experience might include these types of activities:
- Check consistency of a leader’s behavior with stated beliefs
- Identify logical flaws; affirm logical consistencies of a scientist’s conclusion following from data and research
- Check internal consistencies of reports, project plans, marketing plans, mathematical solutions, plot lines, crisis responses, etc.
Additional examples of critiquing experiences include:
- Critique web sites based on criteria, such as design, communication, inclusivity
- Critique CEO’s responsiveness, handling of crises
- Critique plans, solutions, approaches to challenges such as change, global business, local community development
- Critique podcasts or blogs on effectiveness of communications, suitability for various audiences
Example of Evaluating Assignment
Here is an example of an evaluating experience from Kathy Hollywood who joined us from her home in Prescott, AZ. Kathy described one of the activities from her MLLS 717 course — Leading People and Managing Relationships. Kathy described the evaluating activity, the instructions and the feedback and assessment strategy. The rubric for this activity is on slide 16 of the webinar archive. This assignment asked learners to evaluate one of their more recent managers on questions such as effectiveness of leadership style, communication skills, managing relationships, teaming, implementing strategic actions, political behaviors, modeling organizational behaviors, articulating organizational mission and vision, and lastly, valuing workers in a three-page essay. Part of the instructions directed learners to provide examples of the manager’s behaviors when you are assessing his/her effectiveness in those functions. Kathy emphasized that her learners often need “common ground” before attempting this evaluating assignment and that she establishes that common ground and shared experiences from the readings and discussions in the first three-four weeks of the course. In particular Kathy noted that learners need to learn how to suspend judgment and become objective in applying standards of effectiveness.
Barb Vittitoe affirmed her experience at how students need to establish common ground for decision-making and problem-solving experiences. She does this by establishing a familiarity with mental models and learners examining their own and the mental models of others. These experiences on mental models also help build community as learners get familiar with the mental models of their fellow learners.
Three Cognitive Processes within Creating
Creating experiences include three distinct cognitive processes: generating, planning and producing. The generatingphase is a divergent phase in which the learners are hypothesizing and considering alternative solutions and strategies. This is the phase in which learners attempt to understand the task and generate possibilities. In social science experiences, learners might generate useful solutions for social problems.
The second phase, planning, is a convergent phase in which learners devise or select a solution method and prepare a plan of action. In mathematics, for example, learners might list the steps in solving known problems, or the steps in approaching unknown problems.
The third phase of creating is the actual producing or constructing phase. In this phase learners implement or execute their plan. For example, in a leadership course, learners might complete their personal philosophy for leadership.
More Examples of Creating Projects
Many course projects follow a traditional pattern of either a paper or a report of some type. Today’s students might enjoy branching out from these traditional projects to create radio, television or Internet news shows, interviews, short presentations such as webinars, or lasting contributions to Wikipedia or course resource databases.
A course on old radio dramas at Duke University is very popular as students get very engaged in all the processes of producing a radio drama show. After listening to the classic radio dramas, students do their own productions of the shows, which are then made available on The mp3ater Project Web site. http://cit.duke.edu/ideas/projects/2006/08/01/ipods-aid-dramatic-imagination/
Other evaluating and creating assignments mentioned by faculty include serving as an IT consultant (Rich Taylor) and creating their own web business (Pat Phillips).
Creating Projects for Wikipedia
Other creating course projects have focused on discipline-specific contributions to wikipedia. For example there is a criminal justice section of wikipedia developed by students at North Carolina — http://en.wikipedia.org/wiki/Criminal_justice. In a cognitive psychophysiology course at the University of Illinois students are creating or editing pages covering major event-related brain potential (ERP) components used to study neural and cognitive functioning. And one of the courses at Duquesne, Shaping the Modern World has a wikipedia project going on this spring.http://en.wikipedia.org/wiki/Wikipedia:School_and_university_projects/Shaping_the_Modern_World_SP2010
Here is more detail on how wikipedia projects work.
Wikipedia – Boston College Biology (Fall 2009)
As a part of the BI481, Introduction to Neuroscience course at Boston College, students were assigned the task of writing several articles in Wikipedia on neuroscience. The Society for Neuroscience had recently started an initiative to update and improve incomplete neuroscience related articles on Wikipedia. With this writing project, students had an opportunity to learn and contribute to this initiative.
There were approximately 20 groups of 3 students each. Each student had their own separate Wikipedia account, and each group proposed, wrote, edited, and maintained a new article or significantly expanded upon an existing incomplete article. Learners were expected to expand their article to the level as close to Featured Article as they could. See more about featured articles just below. One of the course instructors was in charge of introducing students to Wikipedia and ensuring that they and the project were working within the bounds of Wikipedia guidelines. Course page: Wikipedia:School and university projects/User:NeuroJoe/Fall 2009.
Featured Article Status in Wikipedia
A featured article exemplifies the very best work of editors and is distinguished by professional standards of writing, presentation, and sourcing. In addition to meeting the requirements for all Wikipedia articles, it has the following attributes.
- Well-written – engaging, professional standards
- Comprehensive
- Well-researched
- Neutral
- Stable
- Concise lead section
- Appropriate structure
- Consistent citations
- Images comply with image policy use
A featured article is distinquished by this icon. More about featured article status and the submission and review processes is at http://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria
Rubrics
Just as planning and designing these higher-level experiences can be a challenge, assessing these experiences can require more sophisticated assessment rubrics. Here are three rubrics that you may find useful from a website that mapsthe levels of Bloom’s taxonomy to the many and varied information and communication technology (ICT) tools that we have available.
- Student podcasting projects
o http://edorigami.wikispaces.com/file/view/Publishing+-podcasting+rubric.pdf
- Validating information
o http://edorigami.wikispaces.com/file/view/validating+rubric.pdf
- Digital publishing, including blog entries, wikipedia, etc.
o http://edorigami.wikispaces.com/file/view/digital+publishing+rubric.pdf
Conclusion
Remember, the archive for the Bloom’s Evaluating and Creating webinar is at the organizational link for SLPA faculty webinars. Watch for surveys, announcements and invites about the March webinar on Types and Uses of Feedback – Guiding, Mentoring and Communicating with Learners.
Selected References
Anderson, L.W., & Krathwohl (Eds.). (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.
Bloom, B. S. (Ed.). (1956). Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain. Addison Wesley Publishing Company.
Boettcher, J. V. E-Coaching Tip 64 (#3 Spring 2009) Three Best Practices in Assessment. www.designingforlearning.info/services/writing/ecoach/tips/tip64.htm
Fink, L. D. (2003). Creating significant learning experiences: An Integrated approach to designing college courses. Jossey-Bass.
Forehand, M. Bloom’s Taxonomy. In Orey, M.(Ed.). (2001). Emerging perspectives on learning, teaching, and technology. Retrieved March 2, 2010 from http://projects.coe.uga.edu/epltt/index.php?title=Bloom%27s_Taxonomy
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212-218.www.unco.edu/CETL/sir/stating_outcome/documents/Krathwohl.pdf
Mandernach, B. J. (2006). Thinking critically about critical thinking: Integrating online tools to promote critical thinking. InSight: A Journal of Scholarly Teaching, 1(1), 41-50. Retrieved March 3, 2010 from http://www.insightjournal.net/Volume1/Thinking%20Critically%20about%20Critical%20Thinking%20Integrating%20Online%20Tools%20to%20Promote%20Critical%20Thinking.pdf
Meyer, R. E. (2002). Rote versus meaningful learning. Theory into Practice41(4), 226 – 232. (Same URL as Krathwohl)
Note: These E-coaching tips were initially developed for faculty in the School of Leadership & Professional Advancement at Duquesne University in Pittsburgh, PA. This library of tips has been organized and updated through 2016 in the second edition of the book, The Online Teaching Survival Guide: Simple and Practical Pedagogical Tips coauthored with Rita Marie Conrad. Judith can be reached judith followed by designingforlearning.org.