Dr. Kelly's creation of an online textbook and a course portfolio was part of an evolution in his thinking about the structure of the course itself. Over the years, he had become increasingly unhappy with the "coverage model introductory history survey course." Students, Dr. Kelly notes, have grown even more impatient with this format.

In survey after survey that I have conducted over the years, they tell me that the coverage model encourages the memorize-regurgitate-forget model of learning, but that a more focused approach helps them to think more carefully and to arrive at a deeper understanding of the material they are considering. In my surveys this year, fewer than 10% of the students said they preferred a coverage model course to one taught in a more focused manner.

To focus his teaching, Kelly decided to determine what the learning objectives for the course were and how he was going to assess whether or not the students had achieved the desired results. He followed Howard Gardner's advice, that a course should be designed with three concrete objectives in mind: "engage the central problem of the discipline, help students realize that source materials and subject matter do not exist in a pristine vacume, and give them opportunities to come at the same question or evidence from different perspectives."

Rather than approaching the curriculum as a chronology, (Plato to NATO in 14 weeks) Mills identified six essential concepts and broke the course into two-week blocks in which he addressed each of these main themes. Important events were discussed not in terms of what-happened-when, but in terms of how these happenings influenced larger concepts. The main objective was for students to gain skills and knowledge. Kelly's course design privileged understanding over information. He posted a detailed week-by-week chronicle of how the course unfolded.

Dr. Kelly's research led him to some interesting conclusions. He determined that students who access learning resources on the web display a higher level of recursive reading. According to Dr. Kelly, three-fourths of the students in the web sections went back to primary sources. Only one-fourth of students in the course section taught via print went back to materials assigned earlier in the semester. Their final essays bore out this finding, displaying a much lower use of sources assigned earlier in the semester.

Students I interviewed from the web sections said that because the documents they looked at from earlier in the semester were "just a click away," they were much more likely to use them. When I asked if they would have done the same thing with documents supplied in a course pack, all but one demurred, saying that, as one student put it, "having all that paper to sort through" would not be as immediate as a hyperlink. Or, as another student said in her interview, the web "is just easier to use than a book."

He found that the level of recursiveness was directly related to how well the web-based learning resources and assignments were designed.

He also concluded that the web does encourage independent investigation, but not as much as we would like, that the hypermedia revolution signals the doom of conventional history survey course, and historians must begin teaching web literacy. (He addresses this last conclusion in a more recent project, see below.)

Comments
The comment section addressed many interesting issues. I've decided to include excerpts from four authors on the two topics I felt were most important. First, there was a debate over Dr. Kelly's decision to focus on understanding rather than on facts.

Learning "facts" was not stated as a learning objective. One of the most widely touted examples of mediocrity in American public education is the dismal evidence that a significant proportion of people cannot correctly locate the Civil War within 50 years or so. Facts are important as a baseline for use of historical knowledge. -Samuel Thompson

This was refuted by Carolyn Schneider, who reminds us that reference books catalog the facts, but our job is to learn how the think and do.

ultimately, we don't want our students to KNOW as much as we want them to DO--which in our case would be THINKING, ANALYZING, EVALUATING--the "know" part they can always look up, and then that gets us to another "do"--which is research!

Another important point, had to do with the question of teaching vs. technology. Both commentors remind us that teaching is paramount, and solid course design always trumps technology.

I was struck by the extent to which student comments (Small Group Instructional Diagnosis) pertained not so much to web access as to the traditional classroom. Students remarked on the fact that you learned their names, that you delivered interesting lectures, that you encouraged discussions, that your grading focused too much on grammar, etc., etc. In other words, student consciousness of the web--whatever the reason--was certainly less over than my own. And perhaps that means--I should emphasize 'perhaps'--that the more conventional aspects of classroom teaching are still by far the most important thing in teaching, no matter whether the teacher hands them printed or web-accessible texts. -Dan Kaiser

This is especially noteworthy, to me, in your conclusions about technology and hypermedia--which do, as you say, have effects of various kinds but do not in themselves, alone, explain the most important things that do and do not happen in terms of student learning. What you show is that in a sense course design trumps technology. Or rather, technology needs to be seen as an aspect of course design rather than mostly as a different medium of delivery. This sounds obvious but I think it is not the dominant view. -Pat Hutchings

Dr. Kelly's Western Civilization Webography Project was a natural outgrowth of one of the conclusions he came to in his course portfolio research; historians must begin teaching web literacy.

Even the very best students simply do not think very much about whether or not a site is a good source of information. The only test most students impose on the sites they visit is a visual one--if the site appears to be very professional, then the information it contains must be valid.

The Webography project asks student to visit specific sites, rich with primary sources from European History. Students are given a rubric for analyzing the quality of the site. They review the site, assign it a numerical rating and write a brief review. These responses are posted to the database and made public. Dr. Kelly finds that this exercise significantly improves students' online research skills and, subsequently, the quality of their papers.

My on-going assessment of this project, dating back to the spring of 2003 is that very few of my students--meaning only one or two out of 50 in any given semester--turn in papers with poorly chosen websites once they have completed the webography project. In prior semesters as many as half of my students would receive reduced grades on writing assignments as a result of citing poor or misleading information from low quality sites.

In an email, Dr. Kelly told me of his latest efforts: "I have continued my research on the topics raised in my course portfolio, but have not put them into 'print.' Instead, I have funnelled my findings into various endeavors, such as World History Matters and the Western Civilization Webography Project. You'll see when you look at these, that I've veered from the more standard methods of representing my research into developing more practical applications of my findings for teachers and students."