+ MULTIMODAL PARTICIPATORY DESIGN + RESEARCH PAPER


Prompt: Write a seminar research paper that includes an argument that makes an orginal contribution to the existing scholarship on your subject and formulate a thesis.
Research question: Can designers promote individual communication to create stronger communities using a multimodal approach?
 































Can designers promote individual communication to create stronger communities using a multimodal approach
SHIFTING PERSPECTIVES

A human being is a collection of very delicate receptor organs and an image-generating organ that is equipped with a memory-playback system, where the mind is not singular and seated within the head; it exists throughout our bodies like an interconnected complex system. The work designers create should target this multitude of brains to stimulate people (Hara, 2003). Humans have the ability to communicate and create meaning through the things that we see, touch, hear, and feel–not as singular elements but a combination of those. Meaning-making is individualized, it’s a way for us to interpret and digest information. By empowering individuals to create their own meaning-making through multimodal participatory projects, the outcomes would be more beneficial and effective because it creates a space where everyone can participate.

PARTICIPATE TO CONNECT

Participatory design can be described as the act of transforming participants from audience members to active contributors to help build stronger communities and aggregate user generated content (Armstrong, 2011). “People have been making things – and thinking about the meaning of making things – for a very long time. The power of making, connecting through creating, extends well beyond the online world to all kinds of activities in everyday life” (Gauntlett, 2011). Making is connecting through social interactions, the sharing of information to the world, and how we connect the things we are creating. The act of “making is connecting” carves out a space for participation  – where people can come together to work collaboratively in a shared space through participatory design – whether that is a physical or digital space (Gauntlett, 2011). The core of participatory design is to look at the individual to contribute and requires user content to be completed – the content is then translated into something more meaningful and significant than the initial contribution (Armstrong, 2011). Designers are continually challenging how we interact with and communicate with each other. DIY, Web 2.0 and participatory culture are sources for relevant movements that shifted and challenged the way we disseminate information, communicate and interact with each other. Designers can create a space where individuals express themselves while having the ability to connect with the community using a multimodal approach – an individualized way of meaning-making.

MEANING-MAKING AS INDIVIDUALIZED

Multimodality can be described as a theory of communication that uses more than one mode to express meaning. “Mode is used to refer to a regularised organized set of resources for meaning-making, including image, gaze, gesture, movement, music, speech and sound-effect” (Jewitt & Kress, 2008). All interactions we have are multimodal: conversations, interfaces, and our environments (Norris, 2004) – as humans we are biologically fine tuned to communicate and express ourselves using more than one mode. Traditional approaches of communication relied heavily on print while today “‘image’ is the ‘carrier’ of meaning” (Marchetti and Cullen, 2016) both approaches can be seen as constraining not allowing individual personalization and preferences. Focusing on one mode “[is a] limited and restrictive approach” and there is a need to “expand to new forms” (Marchetti and Cullen, 2016) as a way to engage with people using multimedia technologies. It is hard to create multimodal communication systems that use modes everyone connects with; instead, the designers take on that role and design what they think are the best modes of communication. However, designers should place the decision making in the hands of the individual to create their own communication style.

Although multimodality is not a revolutionary concept, it is continually evolving and in flux when talking about communication in society (Marchetti and Cullen, 2016). “Through technology-induced interaction and connectivity, multimodality provides resources that challenge traditional forms of communication and even language itself” (Marchetti and Cullen, 2016). There have been shifts in our society, and the way we take in information – the transformation from page to screen and the emphasis from text to image– these shifts are influencing our design choices and the way we are trying to communicate those messages. These changes have been occurring before technology came into the picture, but with the rise of Web 2.0 in 1999, which is described as a “particular kind of ethos and approach” and “communal allotment” (Gauntlett, 2011) changes have been seen both digitally and physically. A multimodal approach in participatory design can be one way of expanding to new forms of communication created by the individual; the same way technologies are continually changing to address new advances and human needs.

QUESTIONING THE EXISTING

Existing participatory design projects focus on the meaning-making that is created by the community instead of emphasizing the individual because “to be happy, people need to feel they are involved and not simply under the direction of others” (Barker & Martin, 2011). In most participatory design projects there is a clear objective or at least an understanding of what the individual is contributing. Usually, a set of guidelines is provided. For example, in the case of Aaron Koblin’s Johnny Cash project, people were asked to use a tool that was created on a website to draw images that were then woven together as a tribute to the singer set to his song “Ain't No Grave” (Armstrong, 2011). Although the project has a positive reputation, the tools given to the contributors are restrictive. The targeted participants also become limiting because the tools given only targets people who feel comfortable drawing but does not account for people who would use different modes of expression and communication. Giving individuals a single mode of communication removes the possibility of finding meaning in other forms.


Multimodality has been booming in the educational realm and has positive influences on students. During a study that Marchetti and Cullen conducted, they used a multimodal video lesson, which includes visual, text, movement, and audio or speech. In some cases they offered kinetic typography, which uses a combination of modes. The study was tested on sixty students taking a business language course at a university in Italy. The results showed the individual students’ preferences for learning a new language. The data revealed that the students preferred to communicate face to face, and their second most-preferred method of communication was through different technological tools. Technology has shifted the ways in which we communicate with each other. “Changes in contemporary communications environment simply add urgency to the call to consciously deploy multimodality in learning” (Marchetti and Cullen, 2016). Although this case study is explicitly talking about learning in an educational setting, multimodality can be used to communicate in general – empowering individuals to seek their own language, which can be achieved through participation.

EMOTION//AL CONNECTION

Opening up the platform for everyone to express themselves will lead to a stronger community of people. Research has shown that there is a very strong connection between social participation and happiness (Barker & Martin, 2011). Communication is one of the ways in which we interact in a social environment – it helps us exchange ideas, have meaningful conversations, and feel a sense of belonging. Designers should establish a framework that would allow individuals to contribute to a community using a multimodal approach – where the individual has the ability to pick the modes of communication. Individual participation using a multimodal approach increases social interaction, which can lead to a positive emotional impact on the person (Barker & Martin, 2011).  Allowing the person to contribute and become a part of a larger social community will impact their habitual behaviors and thoughts over the long term. The participation in relation to a multimodal approach includes interaction within the social fabric of the community, input materials, and the individuals themselves.

INTERCONNECTIVITY: MULTIMODALITY AND PARTICIPATORY DESIGN AS AN INDIVIDUALIZED FORM OF COMMUNICATION

By creating a multimodal framework for participatory design it creates a space that is flexible, adaptable and diverse to emphasize the individual and the different ways meaning-making is created. By shifting the emphasis from what is being created to the individual, designers can have a better understanding of the person, thereby creating stronger communities through participation. A participatory project that facilitates multimodal rather than single mode contributions will be more effective and lends itself to address a variety of people's needs. It opens up the platform to create a range of inputs and various combinations of modes for meaning-making. Technology makes it easier to create multimodal frameworks for participatory design, so as designers we should seek the affordances that technology gives us. Designers can create a space where people feel like they can contribute without any restrictions, become part of a community, and increasing their happiness through social connections. By eliminating restrictions around participation it opens up the door to create a space for accessibility, where individuals feel comfortable contributing. Based on research around multimodality and communication, it seems rather important to address the impact this could have on communities and individuals. Each person has his/her own meaning-making language that is constructed using multiple modes, whether that is image and text, text and audio, or video and image. Creating a space that allows for individuals to create their own meaning-making language will foster stronger communities of participation.


WHERE DO WE GO FROM HERE?

Multimodality is a diverse and rich topic that already has seminal texts relating to communication and education. The power behind multimodality, however, lies in the form by which, as a result of multimodality, “we are asked continually to reinterpret, reformat, rework, rethink, and reinvigorate the meanings we find around us” (Hall, 2012). Research around multimodality is open ended because, as humans, we are constantly creating meaning through the things that we interact with–technology will also impact the way meaning is made. While literature regarding multimodality in education is extensive, little is explored with regards to participatory design. Some wider implications to consider is how can the role of technology help create multimodal frameworks for participatory design and create new meaning-making for individuals? Will multimodality coupled with technology become too abstract for people to make meaning with? What are the implications of using technology to create meaning-making?



Works Cited:


Norris, Sigrid. (2004). Analyzing Multimodal Interaction: A Methodological Framework. 10.4324/9780203379493.

Jewitt, C., & Kress, G. (2008). Multimodal literacy. Nueva York (Estados Unidos): Peter Lang.

Gauntlet, David. (2011) Making is Connecting: The social meaning of creativity, from DIY and knitting to Youtube and Web 2.0. Print.

Helen Armstrong and Zvezdana Stojmirovic. (2011) Participate: Designing with User-Generated Content. Print.

Barker, Chris and Martin, Brian (2011) "Participation: The Happiness Connection," Journal of Public Deliberation: Vol. 7 : Iss. 1 , Article 9. Available at: https://www.publicdeliberation.net/jpd/vol7/iss1/art9

Marchetti, Lorena and Peter Cullen. (2016). “A Multimodal Approach in the Classroom for Creative Learning and Teaching.” Psychological and Creative Approaches to Language Teaching. (Page 39-51).

Kurzgesagt – In a Nutshell (2019, February 17) Loneliness [Video File]. Retrieved from https://www.youtube.com/watch?v=n3Xv_g3g-mA

Kress, G. (2010). Multimodality. London: Routledge, https://doi.org/10.4324/9780203970034

Bezemer, J., Kress, G. (2016). Multimodality, Learning and Communication. London: Routledge, https://doi.org/10.4324/9781315687537

Charles J. Forceville. (2011) Multimodality: A Social Semiotic Approach to Contemporary Communication: Gunther Kress, Routledge, London. https://doi.org/10.1016/j.pragma.2011.06.013.

The New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92

Jewitt, C., & Kress, G. (2008). Multimodal literacy. Nueva York (Estados Unidos): Peter Lang.

Jewitt, C., Bezemer, J. J., & O'Halloran, K. L. (2016). Introducing multimodality. London: Routledge.

Donath, J. (2014). The Social Machine: Designs for Living Online. Cambridge (Massachusetts): MIT Press.

Hall, S. (2012). This means this, this means that: A user's guide to semiotics. Second Edition. London: L. King Pub.

Jewitt, C., & Kress, G. (2003). Multimodal Literacy. Bern, Switzerland: Peter Lang US.  Retrieved Apr 30, 2019, from https://www.peterlang.com/view/title/57553

Albers, Peggy. (2007). The Arts, New Literacies, and Multimodality. English Education. 40. 6-20.

Abdullah, Nida (2016) A Visual Archive for Capturing and Coordinating Dialogue in Participatory Design Research. Print.