1. Eye-contact: The visual interaction between people by means of looking each other in the eye.
2. Object manipulation: a form of dexterity play or performance in which one or more people physically interact with one or more objects
3. Facial expressions: The use of facial movements to communicate familiar signals
4. Body Movement: motion of all or part of the body
5. Body posture: referrs to efers to the way in which your body is positioned when you are sitting or standing
6. Gesture: Movement of part of the body to express and idea or meaning (especially head or hands).
7. For the purpose of the observation we focused our observations on facial expression, eye-contact, body posture, body movement, object manipulation and gesture
8. According to Sacks, Schegloff and Jefferson (1974), multimodality integrates and intertwines multiple simultaneous sequentialities and temporalities which are the fundamental principles shaping human action.
9. Sales process: Customer's time spent at the stand and whether the customer bought the a product.
10. Where: At the Albert Cuyp Market,"Fresh bakery stand"
11. When: Friday the 30th of November From 9:20 to 10:20 From 12:30 to 13:30 From 15:30 to 16:30
12. Multimodality defined as: ''Language, gesture, gaze, head movements, facial expressions, body posture, body movements, and embodied manipulations of material objects.''(Goodwin, 2000)
13. Settings of the research
14. Definition of multimodality
15. Research question: How does the seller uses multimodality to influence the sales process?
16. During this research, we observed a particular seller at the ''Fresh Bakery Stand'' throughout the three observations; in the morning, at noon and in the evening. During the first observation in the morning, we found out that the seller deployed more multimodality techniques and this attracted more consumers who spent averagely more time at the stand (See Table 1). This resulted in a significantly higher number of customers purchasing the product. Similarly, the noon observation led to comparable findings. The seller used a similar frequency of positive multimodality tools leading to identical times spent and an eventually similar number of customers buying the product (See Table 2). However, the afternoon observation showed a more limited use of multimodality attracting only a few customers to the stand (See Table 3). The interaction was restrained and eventually, not many people chose to buy the products despite the fact the products were still available at the bakery stand. The seller, however, always displayed some indicators of multimodality. Facial expressions, gestures, body movements, and posture were almost always present, although more negatively (e.g. rigid facial expressions, closed body posture and sitting down). As seen in the tables, when the seller made more positive use of multimodality (e.g. smiling, standing straight up, use of eye-contact) her sales were higher than when she displayed more negative multimodality, 8,5 out of 10 were successful sales compared to only 4 out of 10 were successful sales when she used more negative multimodality (See Tables 1, 2 and 3) . It proves that multimodality is an ever-present subconscious phenomenon that is perceived by and reacted upon by other humans. People interpret the perceived multimodality whether it is actively used or not. The interaction, thus, does not depend on whether people make use of multimodality, as it is an always present phenomenon, but on how it is used. We can then conclude that our hypothesis is correct since when the seller used positive multimodality, it shows more interaction with the customers as they stayed longer time resulting in higher probability of sales than when the seller performed lower use of positive multimodality.
17. Gestures
18. Body Movement
19. Body Posture
20. Eye contact
21. Facial expressions
22. Noon: at noon the gesture mostly used was also ''pointing out at products'' (See Table 2)
23. Morning: predominantly the gesture the seller used was ''pointing out at objects'' (See Table 1)
24. Afternoon: in the afternoon the seller used mostly gestures such as ''putting her head down'' or ''placing her hands on the waist''. (See Table 3)
25. Noon: as in the morning, seller maintained a ''long eye contact'' with the customer (See Table 2)
26. Morning: Morning: typically the seller would have ''a long eye contact'' with the customer (See Table 1)
27. Morning: seller's body movement was mostly "lean forward towards customers" for better communication in between. (See Table 1)
28. Noon: seller's body moved mostly in "walking around"or "leaning forward towards customers"(See Table 2)
29. Afternoon: seller was showing body movement of "standing firmly"most of time or less than morning and noon (See Table 3)
30. Afternoon: the seller was mostly ''standing up'' but in the afternoon we can see an increased tendency to ''sit down'' as a body posture (See Table 3)
31. Noon: the seller's predominant body posture was ''standing up'' (See Table 2)
32. Morning: mostly used body posture was ''standing up'' (See Table 1)
33. Afternoon: the seller's facial expressions were marked as ''light smile'' or ''rigid facial expression'' (See Table 3)
34. Noon: similarly, seller was observed to use ''smile'' as a facial expression (See Table 2)
35. Morning: the facial expressions were predominantly marked as ''smile'' (See Table 1)
36. Afternoon: the eye contact of the seller was significantly shorter ''short eye contact'' (See Table 3)
37. Hypothesis: With a high usage of positive multimodality, we expect a higher probability of customers making a purchase.
38. Conclusion
39. How: Observing the multi-modalities used by the seller with 10 costumers and their effects on the sale process. Each researcher wrote observation for each interactions between the seller and the costumer and the time spent at the stand. The multi-modalities observed were: Facial expressions, eye contact, body movement, postures, object manipulation, and gestures.
