E-portfolios: complexity and resource commitment

Yesterday, I gave a short presentation on e-portfolios to the Information Resources Committee at our University. Assessment is a growing concern here as with all other accredited educational institutions, and we are mightily trying to figure out just what we want to accomplish as well as what the best tool might be for the task. By the time I finished the presentation, I was uneasy about where we went in the discussion.  After 6 hours of working on this blog entry, I wound up in a very different conceptual place than where I started.

Certainly there are three fairly standard types of portfolios: Developmental, Assessment, and Presentation/Showcase. When I am engaged in conversations about portfolios with colleagues, I often hear that the “goal” of portfolio use revolves around generating assessment data, both at the course level and at the program level. Interestingly enough, when the same people start describing what a portfolio should look like, the description references showcase/presentation portfolios. Maybe people get caught up in the presentation of the portfolio artifacts without sufficient attention to the structure, purpose, or type of data that will be collected.

I think it unlikely that you could ever force a successful hybrid portfolio design, for example, between a presentation portfolio (highly student driven, light on reflection, light on structure, and capturing the “end zone” of the student educational experience) and an developmental portfolio (highly institutionally driven, reduced flexibility with respect to student selected artifacts, heavy on reflection, encompassing a longer period of time).

I have concluded that two important boundaries–allowable complexity and maximum resource commitment–must be determined fairly early in the process. A simple process is more likely to be adopted and used. A complex process has to overcome its own inertia as well as generating additional resource requirements (primarily in training/effort costs). Complexity can be measured with a number of indicators. These seem most obvious:

DIAGRAM 1

A second perspective is resource commitment. Portfolios with more artifacts, more evaluations, over a longer period of time are (of necessity) more resource intensive than those that focus on artifacts collected over a short period of time and covering a narrow range of skills.

Digram 2

Reporting mechanisms and number of evaluator interactions, in my experience, can be make-or-break items. If you have more than 10 portfolios to evaluate, you have to start thinking about the amount of time it takes to actually generate any useful and meaningful data. If the ratings and results must be entered and tabulated manually (worst-case on paper, slightly-but-not-much-better-case in a spreadsheet file) the amount of time required for collection and data entry becomes prohibitive. In addition, options for using the data in any other context or sharing it with any other office or program are much reduced.

So this leaves me thinking several things.

1) I have been confusing assessment systems with portfolios. (see Trey Batson and The E-Portfolio Hijacked.) We might be far better off investigating how to use outcomes in whichever Learning Management System is used on campus or creating/purchasing a web-based data collection tool to collect and aggregate performance information. This has the advantage of narrowing the focus and decreasing the effort required to generate, collect, and aggregate useful and meaningful data.

2) Personal Learning Environments, where students create a dynamic personal record of their journey toward enlightenment might be a better solution. Graham Attwell writes about this fairly often. In particular, I enjoyed this presentation on slideshare. But I cannot honestly say that I think most students will embrace this concept and become motivated enough (or technologically skilled enough) to create such a system/record of knowledge.  Nor do I think most faculty at my institution will be able to help them do this.

3) Traditional portfolios are meant to encourage students to think critically about who they are, what they’ve learned, and how they have changed as a result. It seems to me that a showcase/presentation type portfolio can do this fairly well. University students generally create these types of portfolios as they approach matriculation. Maybe my colleagues and I have a bias toward this type of portfolio because it represents an opportunity to get students to impose some coherence on what may be a fragmented curriculum, transferred credits collected from a number of institutions, taught by faculty members who never draw explicit connections to courses taken before or after. (see the Greater Expectations Report for a thorough discussion of the challenges facing Higher Education in the United States.)

4) Finally, I have to confess that I have long been content with a superficial approach to portfolios, and even being distracted by an assessment system masquerading as a portfolio. I am a fraud! I admit it.

Fortunately, I can change.

Leave a Reply

Your email address will not be published. Required fields are marked *