Scientific concepts are inherently multimodal or multimedia constructs. To use them fluently you need to be able to move seamlessly among their linguistic, mathematical, graphical, and operational (actional) semiotic aspects.
I first studied the role of language as a means of teaching and learning science, applying Michael Halliday’s descriptions of how wordings constitute meanings, linking grammar and word-choice to the semantic relationships across dialogue and text. In Talking Science I dealt with the interplay between language as social action and language as thematic resource, between using language to build relationships among people and using it to create relationships among ideas.
Even then I knew, as a physicist myself, that it wasn’t all about language: equations, graphs, diagrams, and manipulating apparatus matter too. In classrooms, that included gesture and mime, chalkboards and projections. In professional science it includes simulations and visualizations of data. I studied both how both school science and professional science integrated these many modalities of representation.
My work in social semiotics also showed me the importance of sociological and cultural dimensions in science and in science education, including issues of gender/sexuality, ideology, belief systems, and values.