1: \documentclass[aps,prl,twocolumn]{revtex4}
2: \usepackage{graphics}
3: \addtolength{\topmargin}{0.75in}
4: \addtolength{\textheight}{-0.5in}
5: \setlength{\oddsidemargin}{0in}
6: \setlength{\textwidth}{6.5in}
7: \def\lsp{{\it LSP }}
8: \def\lsps{{\it LSP}s }
9:
10: \begin{document}
11:
12: \pagestyle{empty}
13:
14: % Use the \preprint command to place your local institutional report
15: % number in the upper righthand corner of the title page in preprint mode.
16:
17: \title{Assessing the Impact of Student Learning Style Preferences}
18:
19: \author{Stacey M. Davis}
20: \author{Scott~V.~Franklin}
21: \email[]{svfsps@rit.edu}
22: \homepage[]{http://piggy.rit.edu/franklin/}
23: \affiliation{Dept. of Physics, Rochester Institute of Technology}
24:
25: \date{Aug, 2003}
26:
27: \begin{abstract}
28:
29: Students express a wide range of preferences for learning
30: environments. We are trying to measure the manifestation of learning
31: styles in various learning environments. In particular, we are
32: interested in performance in an environment that disagrees with the
33: expressed learning style preference, paying close attention to social
34: (group vs. individual) and auditory (those who prefer to learn by
35: listening) environments. These are particularly relevant to
36: activity-based curricula which typically emphasize group-work and
37: de-emphasize lectures. Our methods include multiple-choice
38: assessments, individual student interviews, and a study in which we
39: attempt to isolate the learning environment.
40: \end{abstract}
41:
42: % insert suggested PACS numbers in braces on next line
43: \pacs{}
44: % insert suggested keywords - APS authors don't need to do this
45: %\keywords{}
46:
47: \maketitle
48: \begin{center}{\bf Introduction}\end{center}
49:
50: \vskip -0.1in A learning style is a biologically and developmentally imposed set of
51: personal characteristics that make the same teaching (and learning)
52: methods more effective for some and less effective for others
53: \cite{Dunn89}. These include techniques, approaches, and
54: processes\cite{Dunn90}, but also innate physiological factors,
55: experience, habit, and training. Learning styles are consistent with
56: personality types, but there is more to one's learning style than
57: personality. Common learning styles diagnostics range from the
58: Jungian-based Myers-Briggs personality type test\cite{MyersBriggs} to
59: more detailed attempts to discern environmental and physiological
60: effects \cite{Dunn2, Felder}. As these rely on student
61: self-reporting, they suffer greatly from the fact that students often
62: don't know when they learn, let alone how they best learn. Students
63: that claim to learn best by listening often mean that they are most
64: comfortable following a competent lecturer. This comfort often does
65: not correlate with learning (in fact, it sometimes is anti-correlated
66: with learning)\cite{Learning}. As such, it is perhaps more accurate
67: to talk about learning style {\it preferences (LSP)}.
68:
69: To accommodate different \lsps, many research-based physics curricula
70: \cite{WS, ILDS, Tutorials} present information in a variety of
71: representations \cite{Larkin02,Larkin01}. Motion, for example, is
72: described with words, pictures, graphs, and, ultimately, equations.
73: As measured by standard conceptual evaluations \cite{Thornton98},
74: these courses produce learning gains significantly larger than
75: traditional courses. These learning gains are experienced by all
76: segments of the class, with stronger students benefiting the most by
77: the reformed curricula \cite{Beichner}.
78:
79: A study on deaf students \cite{Lang99} found a correlation between
80: learning style preference and course grade, with students who have a
81: more participatory approach to learning earning higher grades. Dunn,
82: et al. \cite{Dunn90} also found that accommodating learning styles
83: could boost student performance by almost one standard deviation.
84: Felder has analyzed \cite{Felder88,Felder02} student performance in
85: introductory engineering classes in the context of {\it LSP}s. He found
86: that extroverts performed almost one full letter grade higher than
87: introverts, and speculated that the cooperative learning benefited the
88: extroverts. He also found a significant gender gap \cite{Felder02} in
89: performance between students who tend to make judgments subjectively
90: and personally (Jungian {\it feelers}), but no gap between those who
91: approach learning more objectively (Jungian {\it thinkers}).
92: Addressing \lsps may begin to remedy the under-performance of women in
93: introductory physics classes \cite{McCulloughProc,McCulloughProc2}.
94:
95: \begin{center}{\bf Multiple-choice \lsp assessments}\end{center}
96:
97: \vskip -0.07in Dunn and Dunn have developed the {\it Productivity
98: Environmental Preference Survey (PEPS)} \cite{Dunn2} which
99: incorporates environmental, perceptual, and sociological preferences.
100: The {\it PEPS} test, a 100-item, 5-point Likert scale, evaluation,
101: breaks from the traditional either/or classification of type, instead
102: reporting a level of compatibility with a particular style.
103: Compatibility with seemingly contradictory styles is possible. For
104: example, an individual may have a high compatibility with a group
105: learning environment as well as an individual environment. Relevant
106: perceptual elements include auditory, tactile and verbal kinesthetic,
107: and visual picture. Preferences for group or individual, tactile or
108: verbal envirnoments might have important ramifications in a
109: group-based introductory physics course.
110:
111: Rundle's {\it Building Excellence (BE)} exam \cite{BE} is similar to
112: the {\it PEPS} test. It is an 111-item questionnaire that uses a
113: 5-point Likert scale. It expands the social dimension to include
114: small teams of 2-3 people, as opposed to just individual or group
115: preferences. In addition, it can be administered online.
116:
117: \begin{center}{\bf Correlating Course Grade with {\it LSP}}\end{center}
118:
119: \vskip -0.05in The {\it Building Excellence} exam was administered to 390 students
120: enrolled in the first quarter of RIT's three-quarter calculus-based
121: introductory physics course. 98 students participated in the fall of
122: 2002 and 292 participated in the winter of 2002-2003. The test was
123: administered on-line, so students could take it at their convenience
124: and it did not detract from class time, although all students that
125: took the test did so within the first 2 weeks of class. We have not
126: investigated whether the classroom activities can influence student
127: response on Learning Style assessments; such a study would be quite
128: interesting. A breakdown of student performance is shown in Table
129: \ref{grade}. The average class grade was the same in the fall
130: quarter, but students in the traditional sections in the winter had a
131: higher average grade (2.84 to 2.43). Our current analysis looks for
132: differences between students in similar environments, so this
133: difference is not a problem. In order to compare performance between
134: students in different environments we compare the deviation from mean
135: section grade. This seems to remove the artifact caused by the
136: different average grade of different sections.
137:
138: \begin{table}
139: \begin{tabular}[c]{||c||c||c||}
140: \hline\hline
141: &Fall 2002 & Winter 2002-3 \\
142: \hline\hline
143: \begin{tabular}{c}
144: \parbox{0.4in}{\begin{center}\vskip 0.2in N \\
145: $<G>$\end{center}}
146: \end{tabular}
147: &
148: \begin{tabular}{c|c}
149: SCALE-UP & Lecture \\
150: \hline
151: 55 & 43 \\
152: 3.44 & 3.48
153: \end{tabular} &
154:
155: \begin{tabular}{c|c}
156: SCALE-UP & Lecture \\
157: \hline
158: 41 & 251\\
159: 2.43 & 2.84
160: \end{tabular} \\
161:
162: \hline\hline
163: \end{tabular}
164: \caption{\label{grade}Average course grades $<G>$ for students who
165: took the {\it Building Excellence} survey.}
166: \end{table}
167:
168: \begin{center}{\bf Social Environment}\end{center}
169:
170: \vskip -0.05in The {\it BE} test gauges compatibility with three
171: different sociological styles, alone/pairs, small groups (3-4
172: students), or in teams (4 or more). Table \ref{group}, combining
173: students from the fall and winter quarters, shows that there was
174: little difference in final class grade in either SCALE-UP or
175: traditional sections. We hypothesize that students mold their
176: environment to match their preferences. Students in traditional
177: classes who prefer group interactions might satisfy this need by
178: formin study groups. Similarly, students in SCALE-UP classes who
179: prefer individual learning might find a niche within their group.
180:
181: \begin{table}
182: \begin{tabular}[c]{||c||c||c||}
183: \hline\hline
184: &SCALE-UP & Traditional \\
185: \hline\hline
186: \begin{tabular}{c}
187: \parbox{0.4in}{\begin{center}\vskip 0.2in \% \\
188: $<G>$\end{center}}
189: \end{tabular}
190: &
191: \begin{tabular}{c|c|c}
192: Alone & Group & Team \\
193: \hline
194: 80 & 53 & 51 \\
195: 2.37 & 2.37 & 2.30
196: \end{tabular} &
197:
198: \begin{tabular}{c|c|c}
199: Alone & Group & Team \\
200: \hline
201: 77 & 55 & 55\\
202: 2.92 & 2.97 & 2.91
203: \end{tabular} \\
204:
205: \hline\hline
206: \end{tabular}
207: \caption{\label{group} Average grade $<G>$ for students expressing
208: compatibility with individual, group, or team environments. No
209: correlation between performance and preference is seen. Students with
210: a strong preference for individual environments do not fare worse in
211: the SCALE-UP environment, where group work is common.}
212: \end{table}
213:
214: \begin{center}{\bf Auditory Learning}\end{center}
215:
216: \vskip -0.05in Of particular interest to many faculty are auditory learners, or those
217: who claim to learn best by listening. Unlike the social dimension,
218: the auditory dimension is exclusive; learners have either high,
219: neutral, or low aptitudes for auditory environments. We looked for a
220: depressed average grade in high-auditory learners in SCALE-UP classes
221: and the converse in traditional classes. As table \ref{aud} shows,
222: however, there is no apparent correlation between auditory preference
223: and grade. There may be some self-selection here, as those with a
224: preference for auditory environments may choose traditional sections
225: over SCALE-UP sections. The data, however, show little benefit from
226: this choice.
227:
228: \begin{table}
229: \begin{tabular}[c]{||c||c||c||}
230: \hline\hline
231: &SCALE-UP & Traditional \\
232: \hline\hline
233: \begin{tabular}{c}
234: \parbox{0.4in}{\begin{center}\vskip 0.2in \% \\
235: $<G>$\end{center}}
236: \end{tabular}
237: &
238: \begin{tabular}{c|c|c}
239: Auditory & Neut. & Low \\
240: \hline
241: 28 & 40 & 31 \\
242: 2.44 & 2.35 & 2.41
243: \end{tabular} &
244:
245: \begin{tabular}{c|c|c}
246: Auditory & Neut. & Low \\
247: \hline
248: 38 & 38 & 23\\
249: 2.99 & 3.03 & 2.69
250: \end{tabular} \\
251:
252: \hline\hline
253: \end{tabular}
254:
255: \caption{\label{aud} Average grade for students expressing a strong,
256: neutral, and low preference for auditory learning. The
257: under-performance of low-auditory learners in traditional settings is
258: not statistically significant ($p=0.1$).}
259: \end{table}
260:
261: Little correlation was found between course grades and any preference
262: as expressed on the {\it Building Excellence} exam. There are several
263: possible explanations for this. The final course grade may be too
264: coarse a measurement of learning to distinguish this effect. Student
265: preferences may not, in fact, align with the environment that best
266: produces learning (consistent with \cite{Learning}). Finally,
267: students may find ways to apply their particular learning styles
268: regardless of course structure.
269:
270: \begin{center}{\bf Student Interviews}\end{center}
271:
272: \vskip -0.05in The ability of students with strongly expressed preferences against
273: group learning appeared to succeed in the seemingly discordant
274: SCALE-UP environment. One student, in particular, had an interesting
275: combination of \lsps and agreed to be interviewed several times
276: throughout the quarter. ``Max's'' {\it BE} scores indicated a low
277: compatibility for learning in small groups, an aversion to auditory
278: learning, and a strong dislike for for authority-driven methods. In
279: class, Max's ostensible participation was very limited, and frequently
280: his partners would turn and talk amongst themselves, leaving Max on
281: the periphery. At the same time, his perceptual \lsp dimensions
282: classified Max as one who is internal and tactile kinesthetic, meaning
283: he learns by verbalizing to himself or to others and needs to be
284: actively doing something. This tactile kinesthetic need may or may
285: not be specific to the task, and Max was often seen doodling, which
286: may have satisfied this need.
287:
288: Max strongly preferred the SCALE-UP classroom to the traditional one
289: (he had dropped out of a previous traditional class), saying
290:
291: \begin{center}\parbox{2.75in}{\small I learn a lot better with hands-on and
292: group activities. As we got into the class, I realized that I
293: understood things a lot better, and I didn't know why. I kind of paid
294: more attention to it and I realized that we were explaining stuff to
295: each other and teaching each other.}
296: \end{center}
297:
298: Max rarely spoke out in class, but saw himself participating in his
299: group although, as noted, his group did not share this view. Max
300: included himself when describing group activities with statements like
301: ``Here's where we are measuring the force...'', ``We're all
302: interacting, doing the same thing...'', or ``We're solving
303: problems...''.
304:
305: Max maintained an above average grade (B) throughout the quarter,
306: falling at the end to a high C. His {\it FMCE} post-test score of 60\%
307: was at the class average (Max did not take the pre-test so no
308: normalized gain can be calculated). Especially when compared with his
309: experience with lecture-based course (he withdrew), Max's story in
310: SCALE-UP can be considered a success despite the extreme mis-match
311: between expressed preference and environment.
312:
313: \begin{center}{\bf Isolating Learning Style Dimensions}\end{center}
314: \vskip -0.05in As many research-based curricula \cite{WS,Tutorials,ILDS} have
315: reported significant learning gains, often attributed in part to the
316: group work, the question of learning styles vis a vis group
317: interactions is important. Specifically, are there students who learn
318: best individually and, if so, how do they fare in group activities? A
319: related question involves the stronger students. A common fear
320: amongst faculty skeptical of group work is that the stronger students
321: in a group will carry along their less capable partners. Work by
322: Beichner \cite{Beichner} and others has shown that in fact stronger
323: students benefit most from the new activities, and a plausible
324: explanation is that the process of explaining ideas to partners
325: actually helps learning (along the idea that one doesn't learn until
326: one teaches). The proof, however, is rather indirect. It is not
327: clear whether the student learning is improved because of the group
328: activities or from the research-based activities all students are
329: asked to perform.
330:
331: {\it Methodology}
332:
333: Student volunteers were solicited and paid to spend two hours working
334: through activities and taking various \lsp assessments. Students were
335: required either to have taken introductory calculus-based physics in
336: the previous 2 years or to be currently enrolled in the course. After
337: a short pre-test, students spent approximately 40 minutes on each of
338: two activities. A post-test concluded the session. In the first
339: hour, half of the students worked on a worksheet in groups of three
340: while the other half worked on the same worksheet alone. In the
341: second hour, the groups switched. To reduce the chance that students
342: would be familiar with the topics, we chose activities involving
343: buoyancy, a topic typically outside the typical introductory physics
344: curriculum. Related activities included hydrostatics, which research
345: has shown students to struggle with. Activities had been developed as
346: part of the {\it Explorations in Physics} \cite{EiP} curriculum and
347: were adapted for this research.
348:
349: The pre-test incorporated those questions from the {\it Building
350: Excellence} survey which probed the social dimension, and assessed
351: student preferences for group or individual activities. Students were
352: also given the Keirsey Temperament Sorter, a 70-item questionnaire, to
353: assess personality types. Pre- and post-content tests were devised
354: and tested on 1st-year physics majors who were not participating in
355: the study. This test confirmed that the topics chosen were at the
356: appropriate level but also commonly misunderstood.
357:
358: Students were randomly divided into groups of 12. Of these, 6
359: students worked on an activity alone, and 6 were split into groups of
360: 3. After working on an activity for 45 minutes, the groups of 12
361: switched lab rooms. Those that first worked individually now worked
362: in a small group, and those who first worked in a small group, now
363: worked individually. The activity guide contained 2-3 self-contained
364: experiments that students could perform with little prior preparation.
365: Students were asked to make predictions, record data, posit
366: explanations and imagine applications for the ideas they develop.
367: Students were asked to record complete answers whether they worked
368: alone or in a group. When they had spent 45 minutes on each of the
369: two topics, students were then given a post-test.
370:
371: {\it Preliminary Results}
372: As with the previous study involving course grades, little correlation
373: between personality types, sociological learning style preference and
374: performance on pre- and post-tests were found. We offer some
375: possible explanations for this lack of correlation, recognizing that
376: there may be many more. Possible explanations include,
377: \begin{itemize}
378: \item \vskip -0.05in the expressed learning style preference may bear little
379: connection with the environment in which the student best learns
380: \item \vskip -0.1in college students may effectively activate other learning
381: resources when placed in a less preferred environment
382: \item \vskip -0.1in activities might need to be refined to fit within the alloted
383: forty-five minutes, or the chosen topics may be inappropriate
384: \item \vskip -0.1in pre- and post-tests are too coarse to measure improvement in
385: student understanding
386: \item \vskip -0.1in 8am on a Saturday morning may be too early to start any study
387: involving college students
388:
389: \end{itemize}
390:
391: \begin{center}{\bf Summary}\end{center}
392: \vskip -0.05in Learning and the educational setting is a very complicated balance of
393: learning styles, teaching styles, personality types, environmental
394: factors, innate physiological and psychological factors, motivation,
395: socioeconomic backgrounds, culture, and numerous other factors that
396: may effect the learner. While common assessments that have been
397: validated for internal consistency do produce some discrimination
398: between different students, there appears to be little significant
399: correlation between learning style preference and performance (as
400: measured by course grade) in different learning environments. This is
401: greatly complicated by the fact that classes, extending over a
402: ten-week quarter, expose students to many different environments. In
403: addition, students possibly seek out-of-class environments that more
404: closely match their preference. (This will be the subject of an
405: upcoming study in which we will ask students about their out-of-class
406: activities and look for correlations with their expressed {\it LSP}.)
407: Attempts to isolate students in a restricted environment do not yet
408: produce discrimination in learning, although we believe this
409: methodology, with significant refinement, shows promise. Finally, by
410: studying individual students with extreme preferences we may gain
411: insight into the manner in which different students learn. Our crude
412: analysis seems to indicate that we are not harming students by placing
413: them in the educational setting that might not best suit their
414: aptitudes.
415:
416:
417: \begin{acknowledgments}
418: SVF is grateful to Teresa Larkin for bringing attention to Learning
419: Style Preference research and assessment tools as well as useful
420: discussions. This work has been supported by the National Science
421: Foundation under Grant No. 0116795.
422:
423: \end{acknowledgments}
424: \vskip -0.2in \bibliography{/home/franklin/Writings/Reference_lists/references}
425: \end{document}
426:
427: