class: center, middle, inverse, title-slide # Designing and Writing Exercises ### Toby Hodges ### 8/5/2021 --- # About The Author **Dr Toby Hodges** Curriculum Team Lead at The Carpentries The Carpentries is a US-based Not-For-Profit organisation building global capacity in essential data and computational skills for research. Check out our community's lessons in [The Carpentries Incubator](https://carpentries.org/community-lessons/). --- # About This Presentation The aim of this presentation is to get you thinking about how to design good exercises that are appropriate to your audience and topic, and to show you how to format those exercises within The Carpentries lesson template. --- # The Story So Far... This presentation is most relevant if: * you have a clear idea of target audience for your lesson * your lesson already has defined learning objectives, describing the skills that learners should leave with --- # Designing Exercises 1 The goal is **to design (at least) one exercise to assess learners' progress towards each defined objective**. These exercises can be used for _formative assessment_ during the workshop: assessments that tell the instructors (and the learners) whether the learners have learned the things the lesson is designed to teach them, allowing the instructors to adjust their teaching based on the result. --- # Designing Exercises 2 For example, if your lesson has an objective, > Learn how to add additional gene annotations to the object. an appropriate exercise for formative assessment could be <blockquote> <p>Fill in the blanks in the code block below, to add the <code>sample_identifier</code> vector as a new annotation to the SummarizedExperiment object, <code>se</code>.</p> <p><code>sample_identfier <- 1:24</code></p> <p><code>se <- addAnnotation(x=___,</code></p> <p><code> sampleAnnotationType="sample_id",</code></p> <p><code> sampleAnnotation=___)</code></p> </blockquote> --- # Designing Exercises 3 * exercises should help you and the learner assess whether they have learned what you have just taught them * they should be specific to their target learning objective i.e. they should not test more than one skill/concept at once * ideally, incorrect answers should have _diagnostic power_ i.e. should help you identify what the learner has misunderstood/missed * exercises should be regularly spaced throughout the material * aim for at least one every ~15 minutes --- # Exercise formats There are many suitable formats for exercises. - Code & run - Multiple choice questions - Fill in the blanks - Minimal fix - Theme & variations - Labeling a diagram - Parsons problem Find more in Greg Wilson's [Teaching Tech Together](http://teachtogether.tech/#s:exercises). The following examples are taken from that chapter. --- # Exercise formats: Multiple choice questions > In what order do operations occur when the computer evaluates the expression `price = addTaxes(cost - discount)`? > > 1. subtraction, function call, assignment > 2. function call, subtraction, assignment > 3. function call, then assignment and subtraction simultaneously > 4. none of the above - each incorrect answer should be a _plausible distractor_ --- # Exercise formats: Code & run > The variable `picture` contains a full-color image read from a file. Using one function, create a black and white version of the image and assign it to a new variable called `monochrome`. - be careful! it's easy to make code & run exercises too difficult for novices - you can combine with multiple choice question to give answer options --- # Exercise formats: Minimal fix <blockquote> <p>This function is supposed to test whether a number lies within a range. Make one small change so that it actually does so.</p> <p><code>def inside(point, lower, higher):</code></p> <p><code> if (point <= lower):</code></p> <p><code> return false</code></p> <p><code> elif (point <= higher):</code></p> <p><code> return false</code></p> <p><code> else:</code></p> <p><code> return true</code></p> </blockquote> - helps learners to develop debugging skills. --- # Exercise formats: Theme & variations <blockquote> <p>Change the inner loop in the function below so that it fills the upper left triangle of an image with a specified color.</p> <p><code>function fillTriangle(picture, color) is</code></p> <p><code> for x := 1 to picture.width do</code></p> <p><code> for y := 1 to picture.height do</code></p> <p><code> picture[x, y] = color</code></p> <p><code> end</code></p> <p><code> end</code></p> <p><code>end</code></p> </blockquote> - helps learners develop a key skill: adapting existing code for a new purpose --- # Exercise formats: Labeling a diagram > Figure 1 shows how a small fragment of HTML is represented in memory. Put the labels 1–9 on the elements of the tree to show the order in which they are reached in a depth-first traversal. - an effective way of assessing a learner's mental model of a (sub)topic - probably better suited to a physical classroom than online/self-directed learning --- # Exercise formats: Parsons problem <blockquote> <p>Rearrange and indent these lines to sum the positive values in a list. (You will need to add colons in appropriate places as well.)</p> <p><code>total = 0</code></p> <p><code>if v > 0</code></p> <p><code>total += v</code></p> <p><code>for v in values</code></p> </blockquote> - particularly well-suited for testing understanding of control flow --- # Exercise formats Different types of exercise are suited to assess different skills and experience levels. E.g. "Fill in the blanks" exercises are a good fit for novices, because they allow the learner to focus only on the specific concept/syntax you are assessing. "Minimal fix" exercises are less well-suited to novices: they require the learner to trace execution of a program, identify problems in it, consider one or more ways to fix it and which one is the "minimal" fix, and then implement that fix correctly. --- # Designing exercises: recommendations 1. Try to align exercises with specific learning objectives in the lesson/section 1. Use a variety of formats 1. Provide plenty of scaffold for novices 1. Always include a solution --- # Exercises in The Carpentries lesson template Exercise blocks in a Carpentries lesson are created as special blockquotes in the source (R)Markdown file, followed by a `{: .challenge }` tag assigning the CSS class. --- # Exercises in The Carpentries lesson template https://carpentries.github.io/lesson-example/05-rmarkdown-example/ --- # Exercises in The Carpentries lesson template Watch out for: 1. The level of blockquote - solution boxes are two blockquotes deep. 2. The level of the exercise heading - this should be `h2` i.e. `##` in Markdown. --- # Resources and support - Lesson template documentation: https://carpentries.github.io/lesson-example/ - Source code of that site: https://github.com/carpentries/lesson-example/ - Questions and bug reports to https://github.com/carpentries/styles/issues Contact me: [tobyhodges@carpentries.org](mailto:tobyhodges@carpentries.org)