Tweet
http://chronicle.com/article/Outsour...With/64954/%20
I guess it is consistent: classes where the students are never in the same room as the teacher - why not having graders never in the same room as either?
Lori Whisenant knows that one way to improve the writing skills of undergraduates is to make them write more. But as each student in her course in business law and ethics at the University of Houston began to crank out—often awkwardly—nearly 5,000 words a semester, it became clear to her that what would really help them was consistent, detailed feedback.
Her seven teaching assistants, some of whom did not have much experience, couldn't deliver. Their workload was staggering: About 1,000 juniors and seniors enroll in the course each year. "Our graders were great," she says, "but they were not experts in providing feedback."
That shortcoming led Ms. Whisenant, director of business law and ethics studies at Houston, to a novel solution last fall. She outsourced assignment grading to a company whose employees are mostly in Asia.
Virtual-TA, a service of a company called EduMetry Inc., took over. The goal of the service is to relieve professors and teaching assistants of a traditional and sometimes tiresome task—and even, the company says, to do it better than TA's can.
Enlarge Photo
Virtual TA
Virtual-TA boasts of the detailed online evaluations that its graders provide.
Virtual TA
Virtual-TA boasts of the detailed online evaluations that its graders provide.
The graders working for EduMetry, based in a Virginia suburb of Washington, are concentrated in India, Singapore, and Malaysia, along with some in the United States and elsewhere. They do their work online and communicate with professors via e-mail. The company advertises that its graders hold advanced degrees and can quickly turn around assignments with sophisticated commentary, because they are not juggling their own course work, too.
The company argues that professors freed from grading papers can spend more time teaching and doing research.
"We tend to drop the ball when it comes to giving rich feedback, and in the end this hurts the student," says Chandru Rajam, who has been a business professor at several universities. "I just thought, "'There's got to be a better way.'" He helped found the privately held EduMetry five years ago and remains on its management staff.
Whether Virtual-TA is that better way remains to be seen. Company officials would not say how many colleges use the service, but Mr. Rajam acknowledges that the concept of anonymous and offshore grading is often difficult for colleges to swallow.
Those that have signed up are a mix of for-profit and nonprofit institutions, many of them business schools, both in the United States and overseas. Professors and administrators say they have been won over by on-the-job performance. "This is what they do for a living," says Ms. Whisenant. "We're working with professionals."
Anonymous Expertise
Virtual-TA's tag line is "Your expert teaching assistants." These graders, also called assessors, have at least master's degrees, the company says, and must pass a writing test, since conveying their thoughts on assignments is an integral part of the job. The company declined to provide The Chronicle with names or degrees of assessors. Mr. Rajam says that the company's focus is on "the process, not the individual," and that professors and institutions have ample opportunity to test the assessors' performance during a trial period, "because the proof is in the pudding."
Assessors are trained in the use of rubrics, or systematic guidelines for evaluating student work, and before they are hired are given sample student assignments to see "how they perform on those," says Ravindra Singh Bangari, EduMetry's vice president of assessment services.
Mr. Bangari, who is based in Bangalore, India, oversees a group of assessors who work from their homes. He says his job is to see that the graders, many of them women with children who are eager to do part-time work, provide results that meet each client's standards and help students improve.
"Training goes on all the time," says Mr. Bangari, whose employees work mostly on assignments from business schools. "We are in constant communication with U.S. faculty."
Such communication, part of a multi-step process, begins early on. Before the work comes rolling in, the assessors receive the rubrics that professors provide, along with syllabi and textbooks. In some instances, the graders will assess a few initial assignments and return them for the professor's approval.
Sometimes professors want changes in the nature of the comments. Ms. Whisenant found those on her students' papers initially "way too formal," she says. "We wanted our feedback to be conversational and more direct. So we sent them examples of how we wanted it done, and they did it."
Professors give final grades to assignments, but the assessors score the papers based on the elements in the rubric and "help students understand where their strengths and weaknesses are," says Tara Sherman, vice president of client services at EduMetry. "Then the professors can give the students the help they need based on the feedback."
Mr. Bangari says that colleges use Virtual-TA's feedback differently, but that he has seen students' work improve the most when professors have returned assignments to students and asked them to redo the work to incorporate the feedback.
The assessors use technology that allows them to embed comments in each document; professors can review the results (and edit them if they choose) before passing assignments back to students. In addition, professors receive a summary of comments from each assignment, designed to show common "trouble spots" among students' answers, among other things. The assessors have no contact with students, and the assignments they grade are stripped of identifying information. Ms. Sherman says most papers are returned in three or four days, which can be key when it comes to how students learn. "You can reinforce certain ideas based on timely feedback," Mr. Rajam says. "Two or three weeks after an assignment is too long."
No Classroom Insight
Critics of outsourced grading, however, say the lack of a personal relationship is a problem.
"An outside grader has no insight into how classroom discussion may have played into what a student wrote in their paper," says Marilyn Valentino, chair of the board of the Conference on College Composition and Communication and a veteran professor of English at Lorain County Community College. "Are they able to say, 'Oh, I understand where that came from' or 'I understand why they thought that, because Mary said that in class'?"
Ms. Valentino also questions whether the money spent on outsourced graders could be better used to help pay for more classroom instructors.
Professors and on-site teaching assistants, she says, are better positioned to learn enough about individual students to adjust their tone to help each one get his or her ideas across on paper. "Sometimes kidding them works, sometimes being strict and straightforward works," Ms. Valentino says. "You have to figure out how to get in that student's mind and motivate them."
Some professors "could be tempted to not even read" the reports about how students responded to various parts of an assignment, she says, because when "someone else is taking care of the grading," that kind of information can become easier to ignore.
Terri Friel, dean of the business school at Roosevelt University, says such worries are common but overstated. In her former post as associate dean of administration at Butler University's business school, she hired EduMetry to help the business school gather assessment data it needed for accreditation — another service the company offers. But Ms. Friel believed that Virtual-TA would not appeal to professors there.
"Faculty have this opinion that grading is their job, ... but then they'll turn right around and give papers to graduate teaching assistants," Ms. Friel says. "What's the difference in grading work online and grading it online from India? India has become known as a very good place to get a good business education, and why not make use of that capability?"
Acceptance has been a little easier at West Hills Community College, in Coalinga, Calif., which turned to Virtual-TA to help some students in its online classes get more feedback than instructors for such classes have typically offered. The service is used for one section each of three online courses—criminal justice, sociology, and basic math. Instructors can use it for three to five assignments of their choice per student. Using Virtual-TA for every assignment would be too costly, says Susan Whitener, associate vice chancellor for educational planning. (The price varies by length and complexity, but Virtual-TA suggests to potential clients that each graded assignment will cost $12 per student. That means outsourcing the grading of six assignments for 20 students in a course would cost $1,440.)
But West Hills' investment, which it wouldn't disclose, has paid off in an unexpected way. The feedback from Virtual-TA seems to make the difference between a student's remaining in an online course and dropping out.
"We definitely have a cost-benefit ratio that's completely in our favor for us to do this," Ms. Whitener says.
Holly Suarez, an online instructor of sociology at West Hills, says retention in her class has improved since she first used Virtual-TA, two years ago, on weekly writing assignments. Before then, "I would probably lose half of my students," says Ms. Suarez, who typically teaches 50 students per class.
Because Virtual-TA provides detailed comments about grammar, organization, and other writing errors in the papers, students have a framework for improvement that some instructors may not be able to provide, she says.
And although Ms. Suarez initially was wary of Virtual-TA—"I thought I was being replaced"—she can now see its advantages, she says. "Students are getting expert advice on how to write better, and I get the chance to really focus on instruction."
At Houston, business majors are now exposed to Virtual-TA both as freshmen and as upperclassmen.
Steven P. Liparulo, associate director at the university's Writing Center, helped give Virtual-TA its entree when the center decided to stop grading writing samples from the nearly 2,000 students each year planning to major in business. The writing evaluation is used to determine if students need extra help. He saw Virtual-TA as a way for the center's tutors to concentrate on working one-on-one with students. "That's just a much better use of their time," he says.
EduMetry's Mr. Rajam hopes that more colleges will see these benefits.
"People need to get past thinking that grading must be done by the people who are teaching," says Mr. Rajam, who is director of assurance of learning at George Washington University's School of Business. "Sometimes people get so caught up in the mousetrap that they forget about the mouse."
Her seven teaching assistants, some of whom did not have much experience, couldn't deliver. Their workload was staggering: About 1,000 juniors and seniors enroll in the course each year. "Our graders were great," she says, "but they were not experts in providing feedback."
That shortcoming led Ms. Whisenant, director of business law and ethics studies at Houston, to a novel solution last fall. She outsourced assignment grading to a company whose employees are mostly in Asia.
Virtual-TA, a service of a company called EduMetry Inc., took over. The goal of the service is to relieve professors and teaching assistants of a traditional and sometimes tiresome task—and even, the company says, to do it better than TA's can.
Enlarge Photo
Virtual TAVirtual-TA boasts of the detailed online evaluations that its graders provide.
Virtual TAVirtual-TA boasts of the detailed online evaluations that its graders provide.
The graders working for EduMetry, based in a Virginia suburb of Washington, are concentrated in India, Singapore, and Malaysia, along with some in the United States and elsewhere. They do their work online and communicate with professors via e-mail. The company advertises that its graders hold advanced degrees and can quickly turn around assignments with sophisticated commentary, because they are not juggling their own course work, too.
The company argues that professors freed from grading papers can spend more time teaching and doing research.
"We tend to drop the ball when it comes to giving rich feedback, and in the end this hurts the student," says Chandru Rajam, who has been a business professor at several universities. "I just thought, "'There's got to be a better way.'" He helped found the privately held EduMetry five years ago and remains on its management staff.
Whether Virtual-TA is that better way remains to be seen. Company officials would not say how many colleges use the service, but Mr. Rajam acknowledges that the concept of anonymous and offshore grading is often difficult for colleges to swallow.
Those that have signed up are a mix of for-profit and nonprofit institutions, many of them business schools, both in the United States and overseas. Professors and administrators say they have been won over by on-the-job performance. "This is what they do for a living," says Ms. Whisenant. "We're working with professionals."
Anonymous Expertise
Virtual-TA's tag line is "Your expert teaching assistants." These graders, also called assessors, have at least master's degrees, the company says, and must pass a writing test, since conveying their thoughts on assignments is an integral part of the job. The company declined to provide The Chronicle with names or degrees of assessors. Mr. Rajam says that the company's focus is on "the process, not the individual," and that professors and institutions have ample opportunity to test the assessors' performance during a trial period, "because the proof is in the pudding."
Assessors are trained in the use of rubrics, or systematic guidelines for evaluating student work, and before they are hired are given sample student assignments to see "how they perform on those," says Ravindra Singh Bangari, EduMetry's vice president of assessment services.
Mr. Bangari, who is based in Bangalore, India, oversees a group of assessors who work from their homes. He says his job is to see that the graders, many of them women with children who are eager to do part-time work, provide results that meet each client's standards and help students improve.
"Training goes on all the time," says Mr. Bangari, whose employees work mostly on assignments from business schools. "We are in constant communication with U.S. faculty."
Such communication, part of a multi-step process, begins early on. Before the work comes rolling in, the assessors receive the rubrics that professors provide, along with syllabi and textbooks. In some instances, the graders will assess a few initial assignments and return them for the professor's approval.
Sometimes professors want changes in the nature of the comments. Ms. Whisenant found those on her students' papers initially "way too formal," she says. "We wanted our feedback to be conversational and more direct. So we sent them examples of how we wanted it done, and they did it."
Professors give final grades to assignments, but the assessors score the papers based on the elements in the rubric and "help students understand where their strengths and weaknesses are," says Tara Sherman, vice president of client services at EduMetry. "Then the professors can give the students the help they need based on the feedback."
Mr. Bangari says that colleges use Virtual-TA's feedback differently, but that he has seen students' work improve the most when professors have returned assignments to students and asked them to redo the work to incorporate the feedback.
The assessors use technology that allows them to embed comments in each document; professors can review the results (and edit them if they choose) before passing assignments back to students. In addition, professors receive a summary of comments from each assignment, designed to show common "trouble spots" among students' answers, among other things. The assessors have no contact with students, and the assignments they grade are stripped of identifying information. Ms. Sherman says most papers are returned in three or four days, which can be key when it comes to how students learn. "You can reinforce certain ideas based on timely feedback," Mr. Rajam says. "Two or three weeks after an assignment is too long."
No Classroom Insight
Critics of outsourced grading, however, say the lack of a personal relationship is a problem.
"An outside grader has no insight into how classroom discussion may have played into what a student wrote in their paper," says Marilyn Valentino, chair of the board of the Conference on College Composition and Communication and a veteran professor of English at Lorain County Community College. "Are they able to say, 'Oh, I understand where that came from' or 'I understand why they thought that, because Mary said that in class'?"
Ms. Valentino also questions whether the money spent on outsourced graders could be better used to help pay for more classroom instructors.
Professors and on-site teaching assistants, she says, are better positioned to learn enough about individual students to adjust their tone to help each one get his or her ideas across on paper. "Sometimes kidding them works, sometimes being strict and straightforward works," Ms. Valentino says. "You have to figure out how to get in that student's mind and motivate them."
Some professors "could be tempted to not even read" the reports about how students responded to various parts of an assignment, she says, because when "someone else is taking care of the grading," that kind of information can become easier to ignore.
Terri Friel, dean of the business school at Roosevelt University, says such worries are common but overstated. In her former post as associate dean of administration at Butler University's business school, she hired EduMetry to help the business school gather assessment data it needed for accreditation — another service the company offers. But Ms. Friel believed that Virtual-TA would not appeal to professors there.
"Faculty have this opinion that grading is their job, ... but then they'll turn right around and give papers to graduate teaching assistants," Ms. Friel says. "What's the difference in grading work online and grading it online from India? India has become known as a very good place to get a good business education, and why not make use of that capability?"
Acceptance has been a little easier at West Hills Community College, in Coalinga, Calif., which turned to Virtual-TA to help some students in its online classes get more feedback than instructors for such classes have typically offered. The service is used for one section each of three online courses—criminal justice, sociology, and basic math. Instructors can use it for three to five assignments of their choice per student. Using Virtual-TA for every assignment would be too costly, says Susan Whitener, associate vice chancellor for educational planning. (The price varies by length and complexity, but Virtual-TA suggests to potential clients that each graded assignment will cost $12 per student. That means outsourcing the grading of six assignments for 20 students in a course would cost $1,440.)
But West Hills' investment, which it wouldn't disclose, has paid off in an unexpected way. The feedback from Virtual-TA seems to make the difference between a student's remaining in an online course and dropping out.
"We definitely have a cost-benefit ratio that's completely in our favor for us to do this," Ms. Whitener says.
Holly Suarez, an online instructor of sociology at West Hills, says retention in her class has improved since she first used Virtual-TA, two years ago, on weekly writing assignments. Before then, "I would probably lose half of my students," says Ms. Suarez, who typically teaches 50 students per class.
Because Virtual-TA provides detailed comments about grammar, organization, and other writing errors in the papers, students have a framework for improvement that some instructors may not be able to provide, she says.
And although Ms. Suarez initially was wary of Virtual-TA—"I thought I was being replaced"—she can now see its advantages, she says. "Students are getting expert advice on how to write better, and I get the chance to really focus on instruction."
At Houston, business majors are now exposed to Virtual-TA both as freshmen and as upperclassmen.
Steven P. Liparulo, associate director at the university's Writing Center, helped give Virtual-TA its entree when the center decided to stop grading writing samples from the nearly 2,000 students each year planning to major in business. The writing evaluation is used to determine if students need extra help. He saw Virtual-TA as a way for the center's tutors to concentrate on working one-on-one with students. "That's just a much better use of their time," he says.
EduMetry's Mr. Rajam hopes that more colleges will see these benefits.
"People need to get past thinking that grading must be done by the people who are teaching," says Mr. Rajam, who is director of assurance of learning at George Washington University's School of Business. "Sometimes people get so caught up in the mousetrap that they forget about the mouse."
.
Comment