Legal and postal addresses of the publisher: office 1336, 17 Naberezhnaya Severnoy Dviny, Arkhangelsk, 163002, Russian Federation, Northern (Arctic) Federal University named after M.V. Lomonosov
Phone: (818-2) 28-76-18
E-mail: vestnik_gum@narfu.ru
https://vestnikgum.ru/en/
|
The Formation of the Socio-Semiotic Approach to Multimodal Discourse Analysis: Foreign Experience Review. C. 64-73
|
|
: Linguistics
81̕ 22+811.1/.8
10.37482/2687-1505-V362
The article provides a brief overview of the origination of the foreign socio-semiotic approach to multimodal discourse analysis. Using the methods of diachronic analysis, interdisciplinary synthesis, abstraction and analogy, the author describes the formation and development of complex socio-communicative research technologies in combination with socially constructed cultural resources. Primary attention is paid to the development of multimodal research in social semiotics. The article characterizes M. Halliday’s idea of metafunctionality in relation to language and human social life, which paved the way for the study of the joint functioning of semiotic resources in various discourses to achieve communicative goals. The theory of R. Hodge and G. Kress, which accorded a semiotic status to everything related to humans and their activity, is described. In addition, G. Kress and T. van Leeuwen’s contextual approach to investigating multimodality and M. O’Toole’s visual grammar are presented as universal technologies for the study of semiotic resources. Some features of contextual and grammatical approaches within the framework of systemic functional multimodal discourse analysis are outlined. Particular emphasis is given to the description of R. Scollon’s, S. Wong Scollon’s and S. Norris’ multimodal interactive analysis. The importance of R. Scollon’s idea about discourse as a dynamic social practice and language as a tool for performing actions is highlighted. S. Norris’ attempt to establish a universal unit of multimodal analysis as the smallest unit of pragmatic analysis in communication involving any modality is presented. Visual transcription is described as an effective method of analysing communication mediated by modern digital technologies.
multimodality, multimodal discourse, social semiotics, socio-semiotic approach, multimodal interactive analysis
(pdf, 0.6MB )
- Kress G., Bezemer J. Multimodal Discourse Analysis. Handford M., Gee J.P. (eds.). The Routledge Handbook of Discourse Analysis. London, 2023, pp. 139–155. https://doi.org/10.4324/9781003035244-12
-
O’Halloran K.L., Tan S. Discourse Analysis and Semiotics. Pelkey J., Cobley P. (eds.). Bloomsbury Semiotics. Vol. 4: Semiotic Movements. London, 2023, pp. 231–252. https://doi.org/10.5040/9781350139435.ch-11
-
Norris S. Multimodal Theory and Methodology: For the Analysis of (Inter)action and Identity. London, 2020. 120 p.
-
Kiose M.I. (comp.). Polimodal’nye izmereniya diskursa [Multimodal Dimensions of Discourse]. Moscow, 2022. 448 p.
-
O’Halloran K.L. Multimodal Discourse Analysis. Hyland K., Paltridge B., Wong L.L.C. (eds.). The Bloomsbury Handbook of Discourse Analysis. London, 2021, pp. 249–267.
-
Halliday M.A.K. Language as Social Semiotic: The Social Interpretation of Language and Meaning. London, 1978. 256 p.
-
Hodge R., Kress G. Social Semiotics. Cambridge, 1988. 285 p.
-
Kress G., van Leeuwen T. Reading Images: The Grammar of Visual Design. Abingdon, 2021. 288 р.
-
O’Toole M. The Language of Displayed Art. Rutherford, 1994. 295 p.
-
O’Halloran K.L., Beezer R.A., Farmer D.W. A New Generation of Mathematics Textbook Research and Development. ZDM Math. Educ., 2018, vol. 50, pp. 863–879. https://doi.org/10.1007/s11858-018-0959-8
-
Djonov Е., Tseng C.-I. Harnessing the Potential of Transmedia Narratives for Critical Multimodal Literacy. Crit. Discourse Stud., 2021, vol. 18, no. 3, pp. 349–367. https://doi.org/10.1080/17405904.2020.1804954
-
Caple H., Huan C., Bednarek M. Multimodal News Analysis Across Cultures. Cambridge, 2020. https://doi.org/10.1017/9781108886048
-
Unsworth L. High School Science Infographics: Multimodal Meaning Complexes in Composite Image-Language Ensembles. Pensam. Educ., 2021, vol. 58, no. 2. https://doi.org/10.7764/PEL.58.2.2021.9
-
Lemke J.L. Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text. Martin J.R., Veel R. (eds.). Reading Science: Critical and Functional Perspectives on Discourses of Science. London, 1998, pp. 87–113.
-
Jewitt C. Multimodality and Literacy in School Classrooms. Rev. Res. Educ., 2008, vol. 32, no. 1, pp. 241–267. https://doi.org/10.3102/0091732x07310586
-
Wang Y. On Visual Semantic Algebra (VSA): A Denotational Mathematical Structure for Modeling and Manipulating Visual Objects and Patterns. IJSSCI, 2009, vol. 1, no. 4, pp. 1–16. https://doi.org/10.4018/jssci.2009062501
-
Pierce C.S. Peirce on Signs: Writings on Semiotic by Charles Sanders Peirce. Chapel Hill, 1991. 294 p.
-
Scollon R., Wong Scollon S. Discourses in Place: Language in the Material World. London, 2003. 258 p.
-
Wertsch J.V. Voices of the Mind: Sociocultural Approach to Mediated Action. Cambridge, 1991. 182 p.
-
Scollon R. Mediated Discourse: The Nexus of Practice. London, 2001. 182 p.
-
Wong Scollon S., de Saint-Georges I. Mediated Discourse Analysis. Handford M., Gee J.P. (eds.). The Routledge Handbook of Discourse Analysis. London, 2011. Available at: https://www.researchgate.net/publication/307575422_Mediated_Discourse_Analysis (accessed: 14 April 2024).
-
McNeill D. Hand and Mind: What Gestures Reveal About Thought. Chicago, 1992. 416 p.
-
Geenen J. Introduction: Multimodal (Inter)action Analysis. Multimodal Commun., 2023, vol. 12, no. 1, pp. 1–6. https://doi.org/10.1515/mc-2023-0010
-
Norris S. What Is a Mode? Smell, Olfactory Perception, and the Notion of Mode in Multimodal Mediated Theory. Multimodal Commun., 2013, vol. 2, no. 2, pp. 155–170. https://doi.org/10.1515/mc-2013-0008
-
Norris S. Systematically Working with Multimodal Data: Research Methods in Multimodal Discourse Analysis. Hoboken, 2019. 301 р.
-
Geenen J., Matelau-Doherty T., Norris S. Visual Transcription. A Method to Analyze the Visual and Visualize the Audible in Interaction. Rev. fr. méthodes visuelles, 2021, vol. 5, pp. 1–55.
-
Jones R.H. Towards an Embodied Visual Semiotics: Negotiating the Right to Look. Thurlow C., Dürscheid C., Diémoz F. (eds.). Visualizing Digital Discourse: Interactional, Institutional and Ideological Perspectives. Berlin, 2020, pp. 19–45.
-
Schmidt A., Marx K. Making Let’s Plays Watchable: An Interactional Approach to Gaming Visualizations. Thurlow C., Dürscheid C., Diémoz F. (eds.). Visualizing Digital Discourse: Interactional, Institutional and Ideological Perspectives. Berlin, 2020, pp. 131–151.
-
Cserző D. Intimacy at a Distance: Multimodal Meaning Making in Video Chat Tours. Thurlow C., Dürscheid C., Diémoz F. (eds.). Visualizing Digital Discourse: Interactional, Institutional and Ideological Perspectives. Berlin, 2020, pp. 151–171.
|
Make a Submission
Journal of Medical and Biological
Research
Forest Journal
Arctic and North
|