Developing an evaluation plan

Online learners are accessing their classrooms from across the globe using technology tools that are constantly evolving. Keeping pace with technology developments may seem intimidating, but there are many resources available to guide your choice of technology tools, and to support their integration into your online teaching.

In support of student learning, you want to start by determining what you want your students to do, and then explore the possibilities that technology tools can enable. In the next activity, take a look at how Matt Bower and colleagues (2009) have mapped different technology tools to student learning outcomes.

In support of student learning, you want to start by determining what you want your students to do, and then explore the possibilities that technology tools can enable. In the following tables, consider how Matt Bower and colleagues (2009) have mapped different technology tools to student learning outcomes.

In the following activity, you will be presented with a table demonstrating how different technology tools might be used to support different levels of cognitive process in relation to different sorts of subject matter. Click on each of the tabs at the top to reveal some examples of tools supporting that particular cognitive process.
Do this icon

Foundations

Bower, M., Hedberg, J. & Kuswara, A. (2009) 'Conceptualising Web 2.0...

Key terms

Blog: A web journal where people can post entries, and comment on existing...

Do this icon

Remember


Subject matter type Example technology tool(s)

Factual
knowledge

Microblogging – document and share new items of factual knowledge with a group as they come to hand.

Conceptual knowledge

Wikis – identify the main concepts relevant to the topic on the wiki.

Image creation – draw an image to represent a concept or set of concepts.

Podcasting – listen to a podcast of a lecture and attempt to recall the main concepts.

Procedural knowledge

Video – watch a video of a process and recall the key stages.

Podcasting – create a podcast describing a process that has been observed.

Metacognitive knowledge

Mindmaps – describe own cognition using a mindmap.


Understand


Subject matter type Example technology tool(s)

Factual
knowledge

Social bookmarking – bookmark with facts relevant to a certain topic.

Podcasting – provide definitions of terms on an audio discussion board.

Conceptual knowledge

Blogs – explain the concepts and issues of a topic as they arise.

Presentation tools – represent and present the knowledge and relationships of a conceptual domain.

Wikis – explain a set of concepts on a wiki.

Mindmaps – draw a mindmap representation of a concept or domain.

Procedural knowledge

Podcasting – describe to your peers on Voxopop the best way to perform a process and then provide constructive feedback to one another.

Digital storytelling – observe an online storyboard and be able to explain the reasons for the processes' sequence of stages.

Metacognitive knowledge

Mindmaps – explain own thinking based on theories of thinking using a mindmap.


Apply


Subject matter type Example technology tool(s)

Factual
knowledge

Image creation – construct an image that represents or describes an item of knowledge.

Conceptual knowledge

Digital storytelling – create a story that exemplifies/applies a concept.

Video – create a video that applies the concepts you have learnt to a concrete situation.

Procedural knowledge

Blogs – create a portfolio explaining stages of a products development.

Desktop recording – create a desktop recording that demonstrates how to perform an IT process.

Video – create a video that demonstrates the application of a kinaesthetic process.

Metacognitive knowledge

Blogs – explain how own approaches to learning changes as the subject progresses and as a result of reflecting on learning own processes.


Analyse


Subject matter type Example technology tool(s)

Factual
knowledge

Wikis – analyse the definitions provided by peers and provide them with constructive comments on how to improve.

Conceptual knowledge

Wikis – construct/adjust a knowledge network so that it appropriately interrelates concepts.

Podcasts – collaboratively analyse an image or artefact using Voicethread.

Procedural knowledge

Video – analyse the way in which peers/self performs a process by posting comments on the video page.

Metacognitive knowledge

Blogs – analyse own learning processes throughout a unit of study.


Evaluate


Subject matter type Example technology tool(s)

Factual
knowledge

Social bookmarking – post comments evaluating the quality of factual information saved to the group social bookmarking site.

Blogs – evaluate the factual quality of information on peer blogs and post constructive feedback.

Conceptual knowledge

Wiki – evaluate the quality of peer conceptual explanations and make alterations/suggestions as appropriate.

Blog – evaluate the conceptual quality of peers based on their blog postings and provide them with constructive feedback.

Procedural knowledge

Blogs – evaluate the production process that peers have described and post constructive feedback.

Desktop recording – evaluate the efficiency of peer/self IT process.

Video – evaluate performance of a kinaesthetic process and provide constructive feedback.

Metacognitive knowledge

Blogs – evaluate the degree to which own learning processes improve as a result of self-reflection.


Create


Subject matter type Example technology tool(s)

Factual
knowledge

Image creation – use a collaborative whiteboarding tool to create new definitions for an area of innovation being considered.

Conceptual knowledge

Shared document creation – collaboratively construct a report/campaign that addresses the key issues of a topic of study.

Mindmaps – demonstrate a new conceptual understanding or innovation using a mindmap.

Procedural knowledge

Image creation – draw a flowchart to explain a new process.

Metacognitive knowledge

Mindmaps – suggest more efficient ways of thinking as a mindmap.

Source: Bower, M. et al (2009). Used with permission.

Having considered the way in which use of these tools has been aligned with student learning outcomes, let's now think about developing an evaluation plan to further assist your selection of technology tools for teaching and learning. Even before you know what's out there, setting up parameters for review will help guide your review and selection process.

Keep these five steps in mind as you progress through this course. They will help you formulate the questions you need to ask as you work with existing tools and integrate new tools and approaches into your online teaching.

In the following activity, click on each step to find out more about planning your selection of appropriate technology tools for your online teaching.

The following paragraphs will present five steps to consider when planning your selection of appropriate technology tools for your online teaching.

Useful links

These sites provide easy access to a variety of resources related to...

Did you know?

You may find the 'SECTIONS' framework (Bates and Poole, 2003)...

Step 1 of 5: Identify the purpose

Detail exactly what it is that you want your students to do, and what specific technology tool or approach will enable them to do it. Will you be using the technology tool to deliver content, develop interaction, or perform assessment?


Step 2 of 5: Consider your options

Identify the tools that are available within your LMS/VLE, and other tools that are used and supported at your institution. What support is available for you and for students? Think about the benefits and drawbacks of using technology tools not supported by your institution.

There may be policy or governance issues related to technology use, so reach out to the relevant teaching and learning support unit in your institution for usage guidelines.

Check with your colleagues to see how they have worked with this or similar technology tools.


Step 3 of 5: Review your objectives

Review your course and programme learning objectives and reflect on how this technology tool will support desired outcomes and student performance, and how it might undermine them.


Step 4 of 5: Assess the possibilities

Check for research and other information/documentation that will support the successful integration of this technology tool into your teaching and warn you of any pitfalls. Consider running a pilot with a small group of students to evaluate and then report on your adoption and approach.


Step 5 of 5: Select and implement

Select and implement this technology tool in your online teaching. Establish clear guidelines for use and provide students with the resources they need – such as documentation or training – allowing time for them to master using the technology tool at an acceptable level.

Portfolio icon

The next activity will further guide you through the evaluation process, and help you better understand what to consider before using existing technology tools, and/or integrating new technology tools into your teaching. Again, these are questions to keep in mind as you progress through this course and as and when you come to select new technology tools.

Take some time now to work through the questions, keeping in mind either a tool you are already using in your online teaching or a tool you have been thinking of using. When you have completed the activity, you may wish to copy your responses into your Teaching Online portfolio for future reference.

The next exercise will further guide you through the evaluation process, and help you better understand what to consider before using existing technology tools, and integrating new technology tools into your teaching. You may wish to copy your responses into your Teaching Online portfolio for future reference.

In the following activity, consider each question, make a note of your thoughts in the space provided and then click 'View feedback' to see our tips. Use the 'Next' button to move to the next question.
In the case of each of the following questions, consider your response and then continue on to reflect on our tips. You may wish to copy your responses into your Teaching Online portfolio for future reference.

Portfolio activity

Consider the following reflective questions, applying them to three to...

PROGRAMME | Teaching Online
COURSE | Using technology tools for teaching online
UNIT | 1 : Introduction
PAGE TITLE | Evaluating technology tools

Question 1 of 13:

What is your desired learning outcome?

Our tip:

Remember to first determine what it is you want your students to do or learn, then select a tool to support them in accomplishing that goal.


Question 2 of 13:

What technology tool are you considering?

Our tip:

Remember to consider what it is you want your students to accomplish!


Question 3 of 13:

How will this technology tool enhance teaching and learning?

Our tip:

How will this technology tool enhance teaching and learning?


Question 4 of 13:

How accessible is the technology tool for your students?

Our tip:

Consider polling your students for information on how they might access this technology tool.


Question 5 of 13:

How adaptable is this technology tool, or approach, for students with special needs?

Our tip:

Check with your institution on student accessibility and disability policies before you implement a new technology tool.


Question 6 of 13:

What skills are needed for you and/or your students to be able to use this technology tool?

Our tip:

Plan in extra time for you and your students to master a new technology tool.


Question 7 of 13:

What support documentation and/or training are available for academic staff and students?

Our tip:

Documentation and training are important! A simple quick reference guide is a good place to start!


Question 8 of 13:

Who else at your institution is using this technology tool?

Our tip:

If you do not know of anyone at your institution who is using this technology tool, consider reaching out to colleagues at other institutions.


Question 9 of 13:

What are the barriers to using this technology tool at your institution?

Our tip:

Too many barriers to technology adoption may make your endeavour impossible. Being fully informed of any barriers will help you through the adoption process.


Question 10 of 13:

Are there policies in place related to using this technology tool at your institution?

Our tip:

Be sure to check for all related policies! Knowing the boundaries in advance will help you formulate an implementation plan. Check your department or staff handbook for related policies, or reach out to your colleagues in IT, the library, student services, and the registrar's office.


Question 11 of 13:

What research supports the integration of this technology tool into online teaching?

Our tip:

If you have trouble finding resources, refer back to the resource guide at the end of this course.


Question 12 of 13:

How will you measure the successful implementation of this technology tool?

Our tip:

Always tie your technology initiatives to measurable learning objectives. That will help you determine what a successful implementation would look like.


Question 13 of 13:

What back-up plan can you put in place if this technology tool fails?

Our tip:

Always have a back-up plan! Consider reaching out to your students for ideas to implement should all else fail!

It can take time for you and your students to master new technology tools, so be sure to provide the time and support resources necessary for a successful implementation. Try your best to record as much information as possible along the way, and remember to reach out to your students and peers for input and feedback on their preferences and experiences with technology for online teaching.


Foundations

Key terms

Blog: A web journal where people can post entries, and comment on existing posts.

Desktop recording: Recording a video of your screen – of whatever you are doing on your desktop. This is useful when showing others how to use a new program.

Digital storytelling: Using a variety of computer-based media such as video, images, audio, or text to tell a story.

Microblog: A web-based service in which users are only able to post short entries, which often contain links to associated resources.

Mindmap: A diagram that visually represents information or ideas, usually arranged around a core concept.

Podcast: A multimedia digital file (often audio) available on the internet for downloading to a portable media player, computer or similar device.

Presentation tools: Programs or websites that allow you to create a graphic presentation with visual aids (e.g. PowerPoint, Prezi).

Social bookmarking: Storing or sharing a website link. Bookmarks are stored on the internet and can be accessed from any computer.

Wiki: A collaborative website where people can add, edit, and delete shared content. Typically, wikis allow you to track changes and upload supplementary resources.

Useful links

Did you know?

You may find the 'SECTIONS' framework (Bates and Poole, 2003) helpful in evaluating the possibilities offered by a particular technology tool. This framework highlights eight areas of consideration when selecting a new technology tool for teaching and learning:

  • Students
  • Ease of use
  • Costs
  • Teaching and learning
  • Interactivity
  • Organisation
  • Novelty
  • Speed.

Find out more about the framework by visiting the following links:

Portfolio

Duration: 45 minutes

Consider the following reflective questions, applying them to three to five technology tools that you are considering integrating into your teaching:

  1. What desired course learning component would you like to develop/deliver?
  2. How would you develop/deliver this in a face-to-face environment?
  3. How will you develop/deliver this in a blended or fully online course?
  4. What constraints are you most worried about getting in the way?

Download the attached template, which contains a worked example, to record your thoughts, or use the relevant page of your Teaching Online portfolio.