Iceland Academy of Arts

A Student Guide to Assessment

The Purpose of Assessment

You probably know what you think about assessment, but have you ever asked yourself what it is for? What is the ultimate purpose of examinations, evaluations and grades? Perhaps we avoid asking this question because life is too hectic, or we see assessment as a necessary 'fact of life' because we do not have authority to challenge the status quo. After all, what would happen if we disagreed with the way that things are done in virtually all schools and universities? Would anyone allow us to change the system? What do you think? Whether you are someone who sees it as a necessary, but irrelevant, number-crunching exercise, or a competitive person who aims for top marks, you should think about what you want from an assessment process. Is assessment a tiresome habit that is past its 'sell-by date' or, simply, a 'law of nature' that cannot be changed or challenged? We are not offering you total revolution. We are not even asking for clear answers to these questions, but we believe you should think about them.

Connecting Learning With Assessment

We want to support your long-term growth as an individual citizen, and as an effective professional. For you, it will entail learning how to manage the many levels of complexity that you will encounter in your professional life. As this is a lifelong task it will mean learning how to learn. It will include learning how to set your own agenda and to deal with other people. It will mean learning how to present, describe and explain your designs to people who may think in a way that seems alien to how you think. But how can we assess this very elaborate, unpredictable process? To be an effective designer you need to be a human being with a good understanding of yourself. Unless you are in touch with how you are thinking and feel, how can you empathise with, others? On the MA Design program at Icelandic Academy of Arts we see the assessment as an important framework for supporting (and mirroring) the learning process. How might assessment support you through the course? How will it affect how you learn? If we had no formal assessment system, how would you judge whether the course has been helpful to you? Will your grades help you to get the kind of work you want, say in ten, or twenty, years time?

Can Assessment Be Both Flexible & Fair?

In reality, of course, practical assessment systems are unlikely to be as simple and clear as we would like them to be. One reason for this is that we may want them to achieve contradictory things at once. Most, for example, represent an uneasy compromise between fairness and flexibility. Which do you believe to be more important? Can we have both at the same time?

Calculator

It is hard to think of a fairer assessment system than a timed, 'box-ticking' examination with one-answer questions. But this seems entirely inappropriate for designers. It will not support your unique interests and needs, or, even, your long-term employability. This is why you should think, not only about whether the assessment system will acknowledge your merits adequately but, also, whether it will encourage you in issues of real importance. It's always nice to be top of the class, but if you can't understand the assessment system, or you disapprove of its criteria, this will be of very little value to you.

How Many Dimensions Are There in Assessment?

Let us start by thinking about the mechanism of the 'tick-box' example, and what it achieves. We might visualise it as a single relationship (the green connector) between two things (i.e. A and B )

A. What the examiner believes to be the 'right' answer to a test question
B. What the student believes to be, and selects as, the right answer.

Although we described tick-box assessment as being, potentially, very 'fair', if it can be described using only one 'dimension' (or relation) this seems unhelpful for our purpose. It is hard to imagine a design context in which there is only one 'right' answer. In assessing only one relation, 'tick-box' examinations have the advantage of fairness and clarity, but this is at the expense of the many other relations that create the complex 'reality' in which you work. Design is a multi-dimensional process that involves you and the world. But this is more complex than it might seem. Even the simple word 'you' refers to a highly complex, multi-dimensional entity that changes according to how you are thinking and feeling, what you are doing, and who you are talking to at the time. When you design something in the professional world, while you may not notice you are doing it, you are evaluating, and managing, a whole set of relations that make up the whole situation. Here are just a few of these relations for you to think about and to add to:

Relations between you and your ideas/designs/materials, etc.
Relations between yourself and your client/brief/constraints
Relations between you and the greater context (e.g. ecological/economic/ethical background etc.)

Can We Agree on What is Important?

We offered you a place on this masters program because we believe you have the potential to manage many 'dimensions' (or 'relations') at the same time. Obviously, while nobody tests design students using tick-box questions and answers, there seem to be too many factors to take into account if we are to devise an assessment process that is sufficiently fair and complex. Inevitably, human judgement must play an important role in the evaluation process. But maybe we can improve this balance by looking more at the relations between the things that you manage, and less at the things themselves. Put yourself in the University's shoes, for a moment. If you were assessing yourself, what would you emphasise?

1. Your talents and flair as a designer?
2. The (perceived) quality of work you submit?
3. Your ability to please clients, customers or users, etc.?
4. Your ability to follow detailed academic procedures (e.g. project briefs)?
5. Your professional potential by the end of the course?
6. Your potential at the (eventual) peak of your career?
7. Your commitment and hard work?
8. Other criteria...(write down what you think is important).

Evaluating Relations, Rather Than Things

One of our primary aims is to encourage you to prepare you for your long-term (rather than your short-term) success. This means giving you more responsibility for managing your own criteria. Unless you can make judgements about your own values and actions you will have no effective basis for making judgements on behalf of others. And, unless you can do this as though you are (metaphorically) standing in someone else's shoes, how can operate effectively as a designer? Our assessment system is intended to invites you to become adept at doing these things. It helps you to think about who you want to be, so that you are better able to survive, and help others, in the so-called 'real world'.

Acknowledging More Dimensions

We are developing an assessment system that seeks to be fair, open and flexible. It should be explicit enough to enable everyone to understand it. It should be holistic enough to enable each learner to be able to adapt it to their particular interests and needs. Assessment would largely mean checking how well each learner has managed all of the criteria that they agreed to use. This approach is unusual, as it focuses on the relations within learning, rather than judging the specific 'products' or 'outcomes' (e.g. designs) in themselves. This is to encourage you to manage your whole learning, designing and making process in an open, self-critical and self-reflexive way. It should also help you to reconcile learning with assessment in the specific context of your long-term career aims.

Four Players Means Six Relations

A useful way to see the design process is to simplify it as four entities (in no particular order):

A. You (the learner/designer/author/etc.)
B. Recipient (the client/user/reader/stakeholder/etc.)
C. Your work (idea/design/essay/system/etc.)
D. The Context (the background/everything that is not included above)

The simple mathematics of having four entities is that there are six possible relations that connect them into a whole system (see above). Focusing on these relations, one by one, makes it easier to manage the whole process in a way that is understandable to you (and to others). Unlike essays and dissertations, this 3D model is non-linear and can be instantly visualised. It can also be modified, when required. For example, the two versions below show how you might use it in different stages, in order to acknowledge possible conflicts that might otherwise go unnoticed. In Stage 1 you might like to model your role as a private individual. This will encourage you to acknowledge what you experience, think, or believe, personally. In Stage 2 you can use what you have learned to inform your position as a professional. Reconciling possible conflicts between the two stages is useful, as it gives you a chance to decide how far you are willing to go against your own principles in order to please your client.


STAGE 1	STAGE 2

This approach assesses the student's evidence for motivation, how she chooses, manages, and learns from the task, methodology, and management criteria, more than the 'finished work' that it produces.
Each student is expected to provide clear evidence for each of these criteria.
We make it clear that the responsibility for providing this evidence belongs with the student.
This approach implicitly encourages staff and students to develop (and to agree?) a methodology with which students manage their development in ways that are visible to all.
This approach therefore relieves examiners from making 'absolute' judgements about the quality of the work.
Instead, the primary task is to assess how well the masters student is managing her own development on a personal, and also on a professional level.
This invites the learner to explore and manage her own interests and career choices.
It also invites the learner to explore and cultivate a sense of ethical responsibility.
Grading is based on how well, or how much each student meets the following criteria:
N.b. Some of the criteria are inward-facing
Other criteria are more outward-facing

Self-knowledge	Did I show that I am self-aware & actively included in this essay?
Reader-sympathy	Did I show that I understand, and can help my reader?

Curiosity	Did I show I am in touch with my interests, questions, and inquiries?
Research Skills	Did I show I am reflective/sceptical/observant/careful/precise/well balanced?

Studentship	Did I show that I attended classes and learned from the course?
Communication	Did I make my proposition tangible/visible to my reader/s?

Professional Aims	Did I include my career aims/practices in the whole presentation?
Ethical Awareness	Did I show I am aware of my responsibility to the Biosphere AND/OR society?

Iceland Academy of the Arts

Eight criteria.

1. Ability to manage things in a self-reflexive way

a) Clearly demonstrating that you have a clear sense of your identities and capabilities
- e.g. personal
- e.g. professional
b) Clearly demonstrating that you know how others see you
c) Clearly demonstrating that you know what you want
d) Clearly demonstrating that you know what you can do
e) Clearly demonstrating that you know your current limitations
f) Clearly demonstrating that you can see positive aspects of your work
g) Clearly demonstrating that you can see negative aspects of your work

1. Managing your learning processes

a) Clearly demonstrating that you are acquiring capabilities beyond those you had when you arrived on the course
b) Should capabilities be proven within the timescale of the course, or are they changes that may become useful, later?
- (Perhaps the criteria should encourage students to be slow achievers who will reach full maturity well beyond the timescale of the course?)
- (Maybe your marking scheme should reward students who attempt ambitious projects (for them) and show how they have learned from their failures?)
- (Maybe your marking scheme should encourage students to think beyond traditional skills and talents.
- (If marks are mainly focused mainly on design flair or competences, some gifted students will choose an easier, 'safer' path on which they learn less than they might).

1. Managing your self-development

Clearly demonstrating that you are refining existing skills as a design-related professional)
Clearly demonstrating that you see/analyse your existing capabilities
Clearly demonstrating that you see your own potential
Clearly demonstrating the ability to see opportunities for developing the above

d) Managing your methodologies
- Clearly demonstrating that you can make a meta-narrative that helps you manage your processes of learning and development.
e) Managing your communication
- Managing (taking responsibility for) the presentation processes in submitting work.
- (Are you showing an outcome or the process by which you achieved it? Which is required?)
f) Managing your client relations
- Clearly demonstrating that you are aware of the professional needs (and/or desires /needs?) of clients, stakeholders (e.g. industrial, social, political, ecological)
g) Managing your self-assessment
- Clearly demonstrating that you can devise appropriate ways to set aims or targets.
- Clearly demonstrating that you can devise the criteria by which to evaluate the success of your projects.

NOTES

Heutagogy
Maturana - able to make a distinction
Lacan's mirror phase - able to see oneself as distinct
Rene Descartes' - able to reason one's existence ("cogito ergo sum")
Socrates - "I know that I don't know"
- Aristotle / Goffman / (rhetoric)
Sun Tzu - know yourself and your enemies
von Foerster - "we think, therefore we are"
combinatory logic
Dobson and others have shown the tendency for our unacknowledged beliefs to cloud our powers of reasoning.
Kahneman, Slovic & Tversky have suggested that humans are 'arational'. This does not mean that we are necessarily 'irrational' or illogical, but that we do not apply rational logic in doing most of the things that we do.
In their laboratory experiments, Kahneman, Slovic & Tversky found that volunteers behaved ineffectively (i.e. did 'non-optimal operations') by doing certain things in conspicuously inappropriate contexts.
Gazzaniga talks of 'confabulation' in the way that sub-agencies of the brain account for contradictory evidence appearing at the conscious level from different routes.

Semir Zeki - but each person's consciousness is built up from many separate mini-consciousnesses at different levels in the brain. Professor Semir Zeki: "Our hypothesis is that there are many many different consciousnesses in the brain". Semir Zeki, co-Director of the Wellcome Department of Cognitive Neurology in University College, London, says: "I think there are two ways in which people have thought about consciousness. "One is that it has nothing to do with the nervous system, it's above it as it were. That's the way philosophers have on the whole thought about it. "The other is the approach of serious scientists who think consciousness is the product of activity in the brain. But they have tended to think of it as a unified thing, a unity." Zeki and his colleagues do not agree. "Our hypothesis is that there are many many different consciousnesses in the brain, which we call micro-consciousnesses, and that these are the results of activities at different stages in the brain. "There are many stages leading to each area of the brain, and each area is specialised for a different kind of perception. One deals in colour, one in lettering, one in faces and so on. "Each one of these can have a conscious correlate. The brain generates all these micro-consciousnesses and then they are bound together into a super consciousness." One bit of evidence for the theory is that when one of the systems is destroyed, for example when the part of the brain responsible for colour perception is destroyed, then the person concerned loses the ability to see in colour but their vision is not otherwise affected. On the other hand, brain damage that affects the whole visual centre but spares the colour centre will result in a patient who is blind, but who can nevertheless visualise colours. If consciousness was a single entity, not an assembly of micro-consciousnesses, then you would expect the whole visual system to be affected by damage. But it isn't. The other bit of evidence was produced by Semir Zeki and his colleague Andreas Bartels themselves. They measured the speed with which normal humans consciously perceive colour, form and motion. They found that humans see colour about eighty thousandths of a second before they see form, and form about the same time before they see motion, even when all three are presented simultaneously. Since each such visual percept is a conscious event, the difference in timing implies that consciousness is made up of separate conscious events. But questions remain to be answered. "We are still left with the puzzle of what it is in the firing of the brain cells the messages they send that results in the experience of consciousness for each system. "That has by no means been solved. But I think the philosophical view of consciousness will have to be modified quite substantially, by thinking about not one unique entity but many, many entities." If the theory is accepted, then in the future, if someone suffers a brain injury, then they may be thought of as having some of their micro-consciousnesses affected but not others, rather than having their overall consciousness affected. Eventually this may affect the way in which some mental illnesses as well as brain damage are thought of, and even the ways in which they are treated. But that is probably many years away.