Interaction Geography Slicer (IGS)

The IGS is an open-source, dynamic visualization tool that provides novel ways to visualize movement, conversation, and audio/video data over space and time. This tool is designed to work with movement data collected automatically through location-based technologies or manually through Mondrian Transcription Software. This page will get you started with how to use the interface, read space-time visualizations produced by this tool, and format your own data to upload and explore with the IGS. We are rapidly developing and expanding IGS features and collaborators in this effort are welcome, please reach out to us or contribute to the GitHub repo.

Interface

The image above is a screenshot from the IGS showing a teacher’s movement during a 56-minute science lesson as a purple path over a floor plan and upward in a space-time cube, where the z-axis is time and the x/y dimensions correspond to the floor plan. Stops are shown as circles on the floor plan where area indicates duration and as thick lines in the space-time view where length indicates duration. Data is from a classroom science lesson from the TIMSS 1999 Video Study (see credits below). This 3D view is one of many possible views within the IGS. The IGS’s interface consists of 4 primary components:

  1. Load Data & Sample Data Menu: where you can explore different sample datasets and import your own data

  2. Canvas: where all data is visualized in 2D or 3D views over a floor plan and a timeline

  3. Interactive Timeline: displays the time range of all imported data and can be dynamically rescaled to selectively study data

  4. Interaction Tabs: where you can selectively study and dynamically interact with data in different ways

How To Use

This demonstration video provides a brief overview of how to read and use the IGS to visually explore different datasets and load your own data. This video uses example data from Michael Jordan’s last shot and from a classroom science lesson to illustrate IGS features and concludes by showing how to load your own data.

Additional Interaction Techniques

The IGS can display data in 2D and 3D views and supports animation and interaction with data in different ways. The image above illustrates several different kinds of supported interaction techniques including the ability to: (1) animate movement paths at different speeds, (2) highlight teachers’ movement based on the lesson plan, (3) filter only stops — one can also vary the amount of time that constitutes a stop or filter only movement — and layer turns of talk when a student asks a question while the teacher is stopped — the IGS supports basic forms of word search to identify when and where words or phrases are spoken, (4) rotate the floor plan, (5) select movement and talk in regions of space by highlighting data on the floor plan — in this case movement and talk is shown at the whiteboard, (6) rescale data — in this case a five minute sequence of interaction at the whiteboard is rescaled, and (7) hover/click to read conversation turns and watch video at moments in space and time.

How To Format Data

To use your own data in the IGS, you need at least a floor plan/image file and one movement file formatted in the ways described below. You can also include conversation, video and code files formatted in the ways described below.

Required Data

Floor plan/Image File (PNG/JPG)

You will need a floor plan image file in PNG or JPG format. A floor plan is a type of drawing that shows you the layout of a space from above. You can create a floor plan image file by hand and take a picture or use software such as Adobe Illustrator or AutoCAD to create a higher quality image file. Figure 1 shows an example floor plan image.

Movement File(s) (CSV)

Creating movement file(s): If you do not have positioning data collected by location-based technology that can be formatted in the file format below, you can use Mondrian Transcription Software to transcribe and create movement files from video in the exact format described below. Make sure the floor plan image file you use to record your data is the same file you import into the IGS.

Movement File Format: Your movement files must be formatted as a CSV file as shown below with columns labeled: “time” with whole number or decimal values corresponding to seconds elapsed from 0 seconds and “x” and “y” with whole number or decimal values corresponding to a pixel position on your floor plan image file. Your file should be sorted in ascending order by time (smallest to largest time values).

Note: the filename of your movement file will become its label in the IGS interface.

Optional Data

Video File (MP4)

Your video must be in a MP4 format. This is the only video format that currently works across all modern web browsers. If you need to convert your video from a different format: For small videos a few minutes in length you can use the free, secure and web-based CloudConvert. For larger videos, use either a video editor or the free, secure and safe 4K Video Downloader. The length of your video in seconds should be roughly the same as the largest time value of your movement file(s).

Conversation File (CSV)

Your single conversation file must be formatted as a CSV file as shown below and can be created using software such as Microsoft Excel. Each row is a single conversation turn spoken by a single speaker with columns labeled: time with whole number or decimal values corresponding to seconds elapsed from 0 seconds indicating when a conversation turn is spoken (you do not need to start at 0 seconds); speaker with text of any length indicating who speaks a conversation turn (you can have as many speakers you like); talk with text of any length indicating the conversation turn spoken. Your file should be sorted in ascending order by time (smallest to largest time values).

Note: To match individual speakers with individual movement paths in the IGS, the FILENAME of your movement CSV file should match the corresponding speaker listed in your conversation table. Conversation file time ranges should be between the starting and ending values of your largest movement file.

Code File(s) (CSV)

Code files allow you to highlight movement and conversation data over space and time in the IGS that occur during periods of time marked in the code file. For example, you can use code files to mark changes in a lesson plan or when someone is holding an artifact. There are two ways to format code files depending on your needs and what approach works best for your data:

Single Code File: Each file is a set of time periods that relate to the same code. The name of your file indicates the name of your code in the IGS. Code files must be formatted as a CSV file as shown below with columns labeled: “start” and “end” with whole number or decimal values corresponding to seconds elapsed from 0. Code files must be sorted in ascending order (from smallest to largest start times). You can have as many single code files as you like allowing you to selectively study single and simultaneously occurring codes in the IGS.

Multi-Code File: This format allows you to create a single file with many different codes. Code files must be formatted as a CSV file as shown below with columns labeled: “code” with text indicating the name of a code that appears in the IGS and “start” and “end” with whole number or decimal values corresponding to seconds elapsed from 0. The file must be sorted in ascending order (from smallest to largest start times). You can only have one multi-code file.

Credits/License

This software is written in JavaScript and p5.js and uses YouTube and Kaltura Video Player APIs and the PapaParse library by Matt Holt for CSV file processing. This software is licensed under the GNU General Public License Version 2.0. See the GNU General Public License included with this software for more details. Classroom discussion example data is used with special permission from Mathematics Teaching and Learning to Teach (MTLT), University of Michigan. (2010). Sean Numbers-Ofala. Classroom science lesson data is made possible by the researchers and teachers who created The Third International Mathematics and Science Study (TIMSS) 1999 Video Study. IGS software was originally developed by Ben Rydal Shapiro at Vanderbilt University as part of his dissertation titled Interaction Geography & the Learning Sciences. Copyright (C) 2018 Ben Rydal Shapiro, and contributors. To reference or read more about this work visit HERE