Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id j9DE3kG28960; Thu, 13 Oct 2005 10:03:46 -0400 (EDT) Date: Thu, 13 Oct 2005 10:03:46 -0400 (EDT) Message-Id: <7E0B624DDF68104F92C38648A4D93D8F773069@MAIL.cal.local> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: "Carol Van Duzer" <carol@cal.org> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:1288] RE: FW: [NIFL-TECHNOLOGY:3792] Re: X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit Content-Type: text/plain; Status: O Content-Length: 9772 Lines: 261 This is an important question to consider when choosing (and developing) assessments. I recently asked Dr. Lyle Bachman, a professor at UCLA, about any research on reading test items online vs on paper so I wrote to ask him about the studies he mentioned. His response is below: "Some of the research is essentially inconclusive. Here's the URL for an on-line journal article that one of my students did when she was at UCLA: http://llt.msu.edu/vol5num2/sawaki/default.html She reviewed the literature on paper vs. on-line/screen reading in a number of fields, and found that the results were often not comparable, and that they weren't conclusive one way or the other. The other study I was thinking of was one that two of my students did comparing performance on on-line and P&P reading tests. They looked at differences in performance across web and P&P delivery formats, and also conducted verbal protocol analyses the investigate the strategies test takers used in the two modes. Here's the reference: Vongpumivitch, V. and Xi, X. (2002). Does presentation mode impact test-taking processes and performances on reading? Paper presented at the Language Testing Research Colloquium, Hong Kong. I'm not sure if they've published this anywhere." I hope these are helpful. Both Dr. Sawaki and Dr. Xi are at ETS. You may be ble to contact them there to see if they have any other leads. Carol Carol H. Van Duzer Training Coordinator Center for Adult English Language Acquisition (CAELA) Center for Applied Linguistics 4646 40th St., NW Washington, DC 20016 Tel: 202 362 0700 Fax: 202 363 7204 Email: carol@cal.org Visit our website at www.cal.org/caela -----Original Message----- From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On Behalf Of Ira Yankwitt Sent: Friday, October 07, 2005 9:41 AM To: Multiple recipients of list Subject: [NIFL-ASSESSMENT:1267] RE: FW: [NIFL-TECHNOLOGY:3792] Re: As a former ABE teacher, this discussion about computer-based assessment has raised a question for me: does anyone know of any research on the cognitive or psycho-social differences between reading in a digital environment vs. reading in a more traditional print format? At 09:23 AM 10/7/05 -0400, Howard Dooley wrote: >Interesting discussion and comments. My program also uses the CASAS to >place our ESL students into classes and to assess progress (for funders >and the NRS), and we find that it is both useful and practical, >particularly in our large-scale registrations. We use other measures >and assessments to gather further data that is of value to the >learners, instructors and to make program improvements. In some >classes we use the listening, some the reading; we have begun to pilot >the workplace speaking at worksite literacy classes which are ESOL, and >instructors are reviewing the CASAS functional writing assessment and >comparing it to the REEP to see what would fit best with our program structure and learners. > >I recently spoke with Richard Ackermann, the CASAS staff point person >for CASAS' computerized assessments. CASAS provides several options >for assessing learners, though the multiple-choice tests are the most familiar. >Two of the CASAS assessment series, Life and Work & Employability >Competency, are available as CBT -- which means the test booklet has >been put onto the computer, no change in items. The advantage is >immediate scoring and ease of download into an MIS. > >CASAS is currently working on computer adaptive testing. Richard >provides the following description for us: CASAS is currently pilot >testing computer adaptive testing (CAT) with a large workforce >development program. The workforce program is using CAT in reading, >math and listening. A single administration of a reading, math or >listening CAT is used to both place the student in the appropriate >level of training and serve as the benchmark pre-test for measuring >gain. A principal advantage of CAT is that one test administration >serves as both appraisal and pre-test. A further economy is that the >examinee is administered fewer test items in a CAT than with paper and >pencil. (Because the assessment "adapts" to each individual learner.) >CASAS is planning a widespread release of reading CAT, math CAT and listening CAT in early summer 2006. > >With advances in technology, there will soon be a wide variety of >assessment options available to us. As always, we should view these >advances as expanding our opportunities to match the best method of >assessment to each of our learners. Some of the comments earlier in >the discussion seemed to fear that programs would "force" learners to >take CBT or CAT, when that choice would clearly be inappropriate; just >as it would be inappropriate to give a learner a pencil-and-paper test >if the learner had no experience with reading, multiple-choice testing, >bubble sheets, or pencils. I hope neither happens; and if we suspect >or see it happens, that we can find a way to stop it, by mentoring or educating practitioners. > >Howard > > > > >-----Original Message----- >From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On >Behalf Of Gustav Kocsis >Sent: Thursday, October 06, 2005 9:19 PM >To: Multiple recipients of list >Subject: [NIFL-ASSESSMENT:1265] RE: FW: [NIFL-TECHNOLOGY:3792] Re: >Computer assessment and adult learners > > >We would like to see a good and easy to score paper and pencil test for >our ESL students. We are now using the CASAS and we are finding that >the CASAS tests are not useful to use to asses an ESL student's >language ability. We have to register too many students at one time to >be able to use computer testing such as the BEST PLUS. Many of our >students are not very skilled with computers and a computer based test >might not truly reflect the student's English skills. I know of many >other large programs that would be interested in a valid and reliable >paper and pencil ESL test for NRS reporting. Is there anything else out there that is better than CASAS? > >Gustav A. Kocsis >English as a Second Language Coordinator Adult Basic Education Santa Fe >Community College >6401 Richards Ave >Santa Fe, NM 87508 >505-428-1444 >Gkocsis@sfccnm.edu > > >-----Original Message----- >From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov] On >Behalf Of Marie Cora >Sent: Thursday, October 06, 2005 1:59 PM >To: Multiple recipients of list >Subject: [NIFL-ASSESSMENT:1264] FW: [NIFL-TECHNOLOGY:3792] Re: Computer >assessment and adult learners > > > >-----Original Message----- >From: nifl-technology@nifl.gov [mailto:nifl-technology@nifl.gov] On >Behalf Of ra_duffy@comcast.net >Sent: Thursday, October 06, 2005 1:16 PM >To: Multiple recipients of list >Subject: [NIFL-TECHNOLOGY:3792] Re: Computer assessment and adult >learners > >If the purpose is also to assess their computer skills, I could see >assessing via a computer. However, if a student has virtually no >computer experience, as many of the older immigrants who are in our >program do not have, I can not imagine adding this layer on to what is >really trying to be assessed. > >-- >Ruthann Duffy >ESL Tech Coordinator >Essential Skills Program >Shoreline Community College >16101 Greenwood Ave N. >Seattle, WA 98133 > >Telephone: 206-533-6624 >email: ra_duffy@comcast.net >http://success.shore.ctc.edu/callab > > >> Hi everyone, >> >> I wanted to prod you all to add your thoughts to Nancy's questions >from >> the other day regarding testing via computers in ABE/ESOL. What are >> your thoughts, and what have been your experiences on this? >> >> I have done a little bit of work using the BEST Plus with adult >learners >> - but in that case, it is the teacher/test administrator who actually >is >> using the computer and asking questions of the test-taker. As an >aside, >> the great thing about this type of computer testing is that the >software >> program automatically stops you when the student has reached as far >> as they can go, and it also jumps over questions that it realizes the >> student will have no problem answering. Thus, you avoid the >unnecessary >> and sometimes painful process of forcing students to answer way too >easy >> questions, for example. >> >> But! This is not the situation Nancy is describing here - so do >> folks have thoughts on adult students personally using a computer for >> assessment purposes? I am also sending this reply to the >> NIFL-Technology List to see if folks over there have something to >> contribute to this discussion. >> >> Thanks! >> marie >> >> marie cora >> Moderator, NIFL Assessment Discussion List, and Coordinator/Developer >> LINCS Assessment Special Collection at >> http://literacy.kent.edu/Midwest/assessment/ >> >> marie.cora@hotspurpartners.com >> >> >> >> On Wed 10/5/2005, Nancy Hansen wrote: >> >> I also have a question to those who test using >> > computers: Do you find the skills of the learner who has had next >> > to NO exposure to computers and LITTLE literacy skills are >> > accurately being portrayed via this style testing? >> > >> > I have men and women in our adult literacy program who had never >> > sat down at a computer - much less used a mouse before. I cannot >> > imagine that their extensive life skills would be evaluated >> > appropriately when they hit and miss at the computer - sometimes >> > even striking incorrect keys. >> > >> > Nancy Hansen >> > Sioux Falls Area Literacy Council >> > sfallsliteracy@yahoo.com >> > >> >> > > > > > Ira Yankwitt, Director Professional Development / NYC Regional Adult Education Network Literacy Assistance Center 32 Broadway, 10th Floor NY, NY 10004 (212) 803-3356 iray@lacnyc.org
This archive was generated by hypermail 2b30 : Mon Oct 31 2005 - 09:48:54 EST