Experimental semantics – practical module

elective courses

Classroom

The aim of the practical part of the course is an introduction to research software used in experimental semantics. Students will learn how to use following pieces of software: LimeSurvey – a web-app used to make on-line surveys, Ibex used for conducting on-line psycholinguistics experiments, PsychoPy and PsychoPy Builder – application allowing to run wide range of psychology experiments. The module will consist of two parts. In the first part the students will be introduced to basics of mentioned software , in the second part they will learn how to solve practical problems during the work on their own experimental studies. Emphasis will be put on making use of newest literature on best practices of designing on-line research surveys. Previous experience with mentioned software is not required.

I. Creating questionnaire studies in Limesurvey

At the beggining of the module each of the students will receive account on instance of Limesurvey administered by Pracownia Filozofii Eksperymentalnej (www.kognilab.pl). The advantage of using Limesurvey compared to other similar free to use software (Google Forms, SurveyMonkey, etc.) is its open-source license and much more advanced capabilities, which are comparable with Qualtrics (which is non-free). Additional information about LimeSurvey is available on its official webpage: https://www.limesurvey.org

Lesson 1. „My first survey” - an introduction to the Limesurvey

- overview of Limesurvey’s capabilities

- scturture of a survey (groups, questions)

- presentation and navigation options

- praktycal aim: a student can create simple survey using Limesurvey

Lesson 2. Question types

- presentation of basic types of questions and how to create them

- good practices in designing on-line survey

- practical aim: a student knows what types of questions are supported by Limesurvey and how to chose type of question according to a given research purpose

Lesson 3. Advances capabilities of Limesurvey. Branching.

- presentation of types of randomization availibile in Limesurvey and discussion of purposes of randomization in experimental design

- overview of conditions in Limesurvey micro-tailoring (expression manager)

- practical aim: a student can create a survey using branching and conditional micro-tailoring.

Lesson 4. Preparation of surveys for publication. Import and export of data.

- presentation of examples of crowdsourcing research platforms (mTurk, Clickworker, Prolific) and discussion of typical requirements imposed by them

- setting response limits in surveys (quotas and token-based surveys)

- importing and exporting questions and question groups

- exporting results from/to popular data formats/applications

- archiving of surveys and results

- practical aim: a student can create publication-ready survey and has an ability to export the results to application of choice (Excel, R, etc.)

II. Psycholinguistic experiments in Ibex

Ibex is a simple application used to create and publish on-line psycholinguistic experiments. In contrast to Limesurvey it enables precise measure of response time and self-paced presentation of experimental material. Creating an experiment requires modification of JavaScript templates, but advanced programming skills are not necessary. Additional information about Ibex is available on its official webpage: https://code.google.com/archive/p/webspr/

Lesson 1. Introduction to Ibex. Structure of a survey.

- inner workings of Ibex

- project structure in Ibex (structure of files and directories)

- practical aim: a student can create simple experiment in Ibex

Lesson 2. Question types. Randomization.

- presentation of basic types of questions and how to create them in Ibex

- discussion of different types of randomization and ordering available in Ibex (latin square designs, block designs)

- practical aim: a student can design properly structured experiment (with training items, experimental item and appropriate randomization design)

Lesson 3. Preparation of surveys for publication. Import and export of data.

- customization of survey’s behavior and appearance

- uploading survey to kognilab.pl server

- downloading results, export of data to popular file formats

- practical aim: a student can publish prepared by themselves and download and export results

III. Basics of PsychoPy library and PsychoPy Builder environment

PsychoPy is free and open-source alternative to commercial applications such as Presentation™ or e-Prime™. It is allowing to create and run complex psychology experiments using graphical interface (Builder) or high-level Python library. Additional information about Ibex is available on its official webpage: http://www.psychopy.org/

Lesson 1. Introduction to PsychoPy and PsychoPy Builder

- basic interface of PsychoPy Builder

- structure of experiment in Builder

- practical aim: a student knows basics of PsychoPy Builder, understands how it works and can load example experiment and make a simple modification of its elements

Lesson 2. Creating various types of experimental stimuli in Builder

- creating stimuli in PsychoPy (with emphasis on kinds of stimuli used in experimental semantics research such as textual stimuli (graphical or auditory), images, videos)

- practical aim: a student knows how to chose and create appropriate stimuli for a given experimental design

Lesson 3. Input devices (keyboard, mouse, microphone, other controllers)

- various types of recording participants responses (with emphasis of kinds of input methods used in experimental semantics research such as: choosing an answer by keyboard, mouse or controller, classification tasks, rating scales)

- practical aim: student knows how to chose appropriate method of human-computer interaction for a given research purpose and implement it in PsychoPy

Lesson 4. Preparing experiment. Export of data.

- methods of registering and controlling various parameters of participants’ responses (response times, time contraints)

- exporting data to common formats

- practical aim: a student can prepare publication-ready project of an experiment, collect data and export data to appropriate format in order to conduct further analysis

IV. Group work on students’ research projects

In this part of a course students will work in groups on their own research projects (final project). They will be required to present project of an experiment and chose best tool from tools presented at course. Subjects of classes will be tailored to types of students’ projects. Individual consultations of students groups will be provided.

Limesurvey webpage: https://www.limesurvey.org/

Ibex webpage: https://code.google.com/archive/p/webspr/

PsychoPy webpage: http://www.psychopy.org

„Theoretical” part of the course will be dedicated to good practices of on-line survey design. Below there is an example bibliography. If during the course concrete issues related to students’ projects emerge, we will search for suitable literature.

Tourangeau, R., Couper, M. P., & Conrad, F. (2007). Color, labels, and interpretive heuristics for response scales. Public Opinion Quarterly, 71(1), 91-112.

Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge University Press.

Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2011). Survey methodology (Vol. 561). John Wiley & Sons.

Tourangeau, R., Conrad, F. G., & Couper, M. P. (2013). The science of web surveys. Oxford University Press.

Acquired knowledge:

- student knows how to design and conduct research in the field of experimental semantics

- student knows what kinds of questions Limesurvey offers and how to choose the right kind for the research purpose

- student knows the rules for creating questionnaire surveys

- knows good practices regarding conducting surveys, in particular via the Internet

Acquired skills:

- student designs and prepares a survey using the LimeSurvey platform

- student uses the software for creating Ibex psycholinguistic experiments

- student knows how to prepare a simple psycholinguistic experiment using the PsychoPy package and the PsychoPy Builder program

Acquired social competences:

- student completes a simple research project together with the team, he can plan it and share tasks

- student can plan his work

- student presents results of his work in a comprehensible way

Grading will be done based on weekly homework and final project in following manner:

Homework – 70% of a grade

Group project – 30% of a grade

Number of absences: 3