Clay Burell's Bookmarks tagged ets →  View Popular

It appears that SAT scores and GPA are giving way to other types of criteria to determine the educational standards of a particular college. What are some of the new methods for assessing how well universities educate their student body?

To date there are nearly 800 schools that have gone SAT optional. In a recent meeting in Seattle the Dean of Admission and Financial Aid at Harvard reported that the SAT is not a great predictor of student success nor should it be the only criteria used to determine who will succeed in college. Many admission committees use writing samples and grade point averages as a better way to determine who should be admitted and who will likely succeed. Assessment of what students are learning in their classes will continue to be a focal point of accrediting agencies and state governments as well as the federal government.

The first challenge for the writer: picking a topic.

Any topic can work — or fail, Muth said.

"It shouldn't be an essay about community service. It should be about a moment of time," he said. "Start writing an essay about John who you met at a homeless shelter who talked to you about his life. Like any piece of good writing, then you're going to make that come alive.

The biggest problem for students, he said, is starting with too wide a focus. "By the time they get to the details, they run out of space," he said. "I'm all for cutting to the chase."

Hilary Brandenburg, who will attend New York University in the fall, wrote about her summer internship at fashion house Liz Claiborne in New York. "I used my experience as a way to frame myself and what I was interested in studying at the schools that I applied to," said Brandenburg, 18, of Washington, D.C.

"I had a lot of different topics I started," she said. "At school we were told to come up with a list of anything we thought would be interesting about ourselves. We went through a lot of workshops and they gave us prompts and then we had to think about ourselves."

She said her internship "was the easiest thing for me to write."

Brandenburg said her biggest frustration was keeping within the word limit. "In the end, it helped me refine what I wanted to say and it became more to the point," she said.

The essay will be scored by experienced and trained high school and college teachers. Each essay will be scored by two people who won't know each other's score. They won't know the student's identity or school either. Each reader will give the essay a score from 1 to 6 (6 is the highest score) based on the following scoring guide.





Please note that the essay images seen by readers for scoring purposes are clearer than the images we can display for students and institutions on our website.





SCORE OF 6




An essay in this category demonstrates clear and consistent mastery, although it may have a few minor errors. A typical essay






• effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking, using clearly appropriate examples, reasons, and other evidence to support its position


• is well organized and clearly focused, demonstrating clear coherence and smooth progression of ideas


• exhibits skillful use of language, using a varied, accurate, and apt vocabulary


• demonstrates meaningful variety in sentence structure


• is free of most errors in grammar, usage, and mechanics

Subgroup Differences in Mathematics, Critical Reading, and Writing

ESL: Students for whom English is a second language increased 5 points in critical reading to 467 and 2 points in mathematics to 523. They scored 469 on the writing section. Although this is 28 points below the mean for all students, it is a smaller gap than these students experienced for critical reading. More ESL students took the SAT this year.

Race/Ethnicity: African American and Mexican American critical reading scores, 434 and 454 respectively, improved over last year by one point each. American Indian mathematics scores improved by 1 point to 494 and Mexican American mathematics scores improved over last year by 2 points to 465. The largest declines in critical reading were seen among White and Other Hispanic students, both down by 5 points to 527 and 458, respectively.

Student Performance on Essay

The essay asked students to respond to a point of view on an issue through an original first-draft format and support a position with reasoning and examples taken from reading, studies, experience, or observations. Two readers scored each essay, each reader providing a score from 1 to 6 for a maximum score of 12. The average essay score was 7.2 out of 12. Reader agreement on essay scores was very high. Nearly 97 percent of readers exactly agreed on scores or differed by only one point. Only 3 percent of essays needed to be resolved by a third reader. The College Board performed analyses on 6,498 essays randomly sampled from the March 2005 through January 2006 SAT administrations. Results demonstrated that longer essays were more likely to receive higher scores; however, the relationship between length and score was only moderate (.62). Eighty-four percent of essay responses reached the second page. Half the essays used the first-person voice. Score differences were slight, with first-person-voice essays averaging a score of 6.9, compared to 7.2 for those not using first-person voice. Only 8 percent of essays were identified as using the typical five-paragraph essay structure. Fifty-three percent of essays used academic examples (historical, literary/art/music, science or technology, and current events or politics) and 52 percent of essays included some type of personal experience as supporting evidence. Some students used both types of examples.

Given that, some have wondered why the SAT essay training and practice test samples typically show high correlations between length and score.

There is a simple explanation for this correlation. The College Board's goal in selecting samples for initial training, and for practice tests, is to find essays at each score point that demonstrate all the criteria of that score point. And one important criterion is development. According to our published Scoring Guide, a "typical" 6 essay "effectively and insightfully develops a point of view on the issue... using clearly appropriate examples, reasons, and other evidence to support its position." These essays tend to be robustly developed, and are among the longest essays in our sample sets. But a "typical" 6 essay also "exhibits skillful use of language," "demonstrates meaningful variety in sentence structure," and "is free of most errors in grammar, usage, and mechanics." Training 6s are usually longer than training essays at other score points, but they are also more skillfully written across the entire range of criteria than essays at other score points.

Compare a training 6 to a training 3. An essay selected as a sample 3 will be "limited in its organization or focus, or may demonstrate some lapses in coherence or progression of ideas"; it will have problems in development, which, in practice, almost always means that it is shorter. But sample 3s also "sometimes use weak vocabulary or inappropriate word choice," "lack variety or demonstrate problems in sentence structure," or "contain an accumulation of errors in grammar, usage, and mechanics." Training 3s are often shorter than training 4s, 5s, and 6s, but they are also problematic in other ways that all readers should recognize.

The College Board selects these typical essays for training because we want teachers and students to be clear about the level of quality that we associate with each score point. In live scoring, of course, we see typical and atypical essays; some shorter essays will still "insightfully and effectively develop a point of view on an issue," while some longer essays will "sometimes use weak vocabulary or inappropriate word choice." But that is the reason the College Board only employs experienced classroom teachers (with a minimum of three years' experience) as scorers. We trust that teachers who have years of experience reading student writing will be able, upon grasping the clear distinctions between typical essays, to perform the trickier task of scoring the atypical essays. Results from the March administration suggest that our confidence is justified.

SAT coaches don't necessarily have the same conscience issues as some teachers. "What we recommend is that you memorize the basic essay formula and a couple of facts about a topic you're interested in, so that you pretty much know what you're going to write before you take the test," says Yale sophomore-to-be Janet Xu, guest editor of the latest edition of the SAT guide "Up Your Score: The Underground Guide to The SAT." "This works because the SAT questions are usually very broad."

Indeed. An SAT coach based in Rochester, N.Y., who works for one of the major test-prep outfits and didn't want to be named, recently took the new test himself to see what he was up against. "I was like, OK, I know all about 'The Scarlet Letter,' I know all about Florence Nightingale, I know all about the tsunami," he says. "The essay question was something like, Does work give life meaning? So, Florence Nightingale, her work was to heal people -- that gives life meaning. The tsunami, the rescue efforts showed the meaning of volunteer work. 'The Scarlet Letter' -- I'm sure there was some work in there somewhere ... OK, the work of retribution gave Hester Prynne's life meaning. If you have examples of anything, you can write any essay," he says. "And you can always use Florence Nightingale, no matter what."

It seems like everybody has a different opinion about how to do well on the SAT essay. Some people say you should write a strict five-paragraph essay, with an introduction, a conclusion, and three specific examples. Some people say you should read well-known books like The Great Gatsby or The Scarlet Letter and refer to them as often as you can. Some people say that the real key is to write as much as humanly possible. Some say you should do all of these at once!

We want students to know that there are no short cuts to success on the SAT essay. The high school and college teachers who will score your essay have seen it all before. These teachers are not going to give high scores to an essay just because it is long, or has five paragraphs, or uses literary examples. The scorers are experts at identifying truly good writing--essays that insightfully develop a point of view with appropriate reasons and examples and use language skillfully.

So what can you do to write a successful SAT essay? Here are some strategies the College Board would like you to consider:

Read the entire assignment. It's all there to help you. Every essay assignment contains a short paragraph about the issue, usually from a specific author or book. Don't ignore this important information in your rush to answer the question. Imagine that you are talking to the author of the paragraph about the issue. What would you say to him or her? Would you argue or agree? What other ideas or examples would you bring up? Answering these questions will help you develop your own point of view.

I have worked scoring essays ETS. Get real. The above essay gets a high score because you should see the rest of them.

I too grade for the ETS in HIstory AP courses. My experience is that the essay portions of these AP exams reward students for jumping though hoops, but not for good writing or critical thinking. The students who do well memorize rubrics. The students who do poorly never had a teacher who gave them the rubric. The best essays rarely received the highest scores.

I *hope* others’ experience has been different.

I’m a high school English teacher. I teach six periods of English a day. I have 150 students. From about the age of 4 until my final year in college, I read 3-7 books per week, sometimes more. I’ve been teaching for six years, and I’m ashamed to say that if I thought hard about it, I could probably make a list of all of the books I’ve read this school year. I don’t have time to read, and frankly, I’ve lost some of my enthusiasm for reading.

That’s something that I, a lifelong bookworm, never thought I’d say, but it’s true. I don’t enjoy reading and I don’t enjoy writing—it’s somehow been totally wrung out of me. I’m currently looking for a way out of teaching English, because I want my old enjoyments back. I’m tired of the pressure of trying to reform years of bad writing habits, facing pressure to get my students to perform well on the state tests, exit exam, SAT, AP, and be prepared for college-level writing. I’m tired of people looking at us and questioning why we are failing as English teachers, when students come to me in the 9th grade as college-prep students and still have trouble with basic homonyms and sentence structure, and what’s worse, have no motivation to improve.

Perelman was also dismayed to learn that the College Board, which administers the SAT, has explicitly said -- on the grounds that it is testing writing, not historical, literary, or scientific knowledge -- that factual references in the essays needn't be correct. If you misstate the date of the War of 1812, for example, the readers won't mark you down.

"To write something that's long and has the appearance of having a sound, detailed argument but really doesn't make much sense: There is a term for that," Perelman says delicately. And that is what is being rewarded."

Perelman cites one of the sample essays the College Board provides for grader-training purposes: a composition that uses "Madame Bovary" as an example of the dangers of secrecy. It's not awful, but it does include sentences such as "If secrecy were eradicated, many problems, such as internal division, but also possibly hate, might also be eliminated." On the 1 to 6 scale, it serves as an example to scorers of a perfect 6.

"By putting writing on the SAT, we believed we'd focus more attention on writing in schools from K through 12 and across the curriculum," she says. But it's the wrong kind of attention, say critics. "You're getting teachers to train students to be bad writers," Perelman says. The specter of the essay, he claims, could have a chilling effect on the learning and teaching of writing in high school, encouraging students only to perfect the art of the formulaic (to say nothing of wordy) five-paragraph "write for the teacher" essay.

"The SAT carries such weight that teachers will feel pressured to teach to the test: to help students prepare for the very specific kind of writing that the SAT asks them to do," says Bob Yagelski, associate professor of English at SUNY Albany and chair of the National Council of Teachers of English Task Force on the SAT and ACT writing tests. In fact, in a recent report, the NCTE charged that the SAT's essay test threatens to "compromise student writers and undermine longstanding efforts to improve writing instruction in the nation's schools."

But at the board’s annual meeting, Wayne Camara, vice president for research and analysis of the College Board, described research that recently found that coaching — even short-term coaching — does have a major impact on those with poor writing skills. For the research, six graduate students were assigned to put together a coaching program by signing up for a bunch of coaching services, and then developing a coaching program based on what they had learned.

A group of college freshmen were then divided in two: One group went through a nine-hour coaching program led by the graduate students and the other students were the control group. The students took the writing test before and after the coaching, and the coaching had a significant impact on those who did poorly the first time around — increasing their scores by an average of 3 points on the 12-point scale used on the essay.

Camara also said that the study had the students write two essays that are of the sort that college freshmen actually write (not the timed essay with prompt that is on the SAT). Coached students who were poor writers at the beginning did better on the actual essays and not just the SAT, Camara said. As a result, he characterized the coaching for the writing test as “not a bad thing,” in contrast to other coaching, which he said is more likely to teach test-taking skills than meaningful knowledge.

Lutz said that the Princeton Review commissioned a third party study on this that won’t be out until next year, but that all the evidence he is seeing from Princeton Review counselors suggests that the new writing test “may be the most coachable portion of the SAT.”

He echoed the views of many college officials who have decided not to use the writing test in saying that its approach is flawed. “It’s a bad way to measure writing skills — a 25-minute essay on an esoteric subject.”