Uninvited Editorial: Let's Prove that Ed Tech Produces Measurable Student Achievement
I would like to see ISTE focus upon what should have been the basic foundation for educational technology; i.e., proving in exactly what areas and with exactly what methods that technology integration outshines every other strategy for producing measurable student achievement.
The logic for this request is simple (although the proof is not).
If other instructional strategies produce better student outcomes than the integration of technology into instruction, then those methods and strategies must be promoted.
In times of budget crunch (and if now wasn't one of those times, it soon would be), teachers and school districts can ill afford to devote time, energy and money on less than stellar student outcomes.
Therefore, we must promote only those (costly) technology solutions that at least equal; and preferably better (beat hands-down), other less expensive and easier to execute and maintain options. (Note: Traditional teacher talk before a collection of bored, disinterested and disengaged students is cheaper than technology integration. This should be an easy standard to beat.)
The logic of focusing upon measurable student outcomes and comparing those outcomes to other strategies is simple, too.
Teachers (other than early adopters, "committed techies", reform junkies, and zealots), school curriculum folks and district administrators will only come on board with technology integration when educational technology proves that we can produce measurable gains in student achievement with solutions that are 1.) easier, 2.) faster, 3.) cheaper and 4.) of higher quality than other methods.
And if the methods that we promote fail to meet those standards, then we are wasting our teachers time and our students time. We can't afford to deflect attention from strategies that could produce better results.
To often, Ed Tech advocates focus upon ideals. Too often we focus upon the future that our students will encounter in the world of work and career, and we focus upon what our students will encounter in life. This focus leads us away from the need to prove, once and for all, that educational technology stands for student achievement...and standds for methods and strategies that out-produce every other method. (If the methods we advocate can't out-produce other methods for ensuring measurable student achievement, then we have no business promoting them.)
ISTE should also take a stand (and take the hit in terms of lucrative vendor booth revenues) by requiring that vendors provide evidence of the instructional effectiveness of the products that they sell. In particular, ISTE should go after (and reject) the hucksters of those inane, mind-numbing, baby-sitting, page-turning software programs that our unenlightened colleagues can "pop students in front of" when the students finish their drill-sheet assignments so that 1.) the students can "do more drill" and 2.) so that the teacher can claim to be integrating technology.
Getting this "junk software" out of the discussion when promoting the measurable student achievement goals of educational technology would go a long way in building credibility for our cause. Future employability and life-long learning goals are nice, but demonstrating measurable student outcomes, right now, gets the attention of "reluctant-to-use-technology" teachers and "eager-to-save-a-coin" administrators.
Summary: Let's focus on student and student achievement, let's make the use of technology transparent (the subject of another editorial) so the use of the technology doesn't call attention to itself, and let's promote only those strategies and methods that produce better measurable outcomes than everything else.
Scientific evidence for reliable, replicable, Ed Tech strategies and methods lacking? Let's get busy and prove the worth of technology integration to hard-nose, real-world teachers and purse-poor, bottom-line administrators.
The logic for this request is simple (although the proof is not).
If other instructional strategies produce better student outcomes than the integration of technology into instruction, then those methods and strategies must be promoted.
In times of budget crunch (and if now wasn't one of those times, it soon would be), teachers and school districts can ill afford to devote time, energy and money on less than stellar student outcomes.
Therefore, we must promote only those (costly) technology solutions that at least equal; and preferably better (beat hands-down), other less expensive and easier to execute and maintain options. (Note: Traditional teacher talk before a collection of bored, disinterested and disengaged students is cheaper than technology integration. This should be an easy standard to beat.)
The logic of focusing upon measurable student outcomes and comparing those outcomes to other strategies is simple, too.
Teachers (other than early adopters, "committed techies", reform junkies, and zealots), school curriculum folks and district administrators will only come on board with technology integration when educational technology proves that we can produce measurable gains in student achievement with solutions that are 1.) easier, 2.) faster, 3.) cheaper and 4.) of higher quality than other methods.
And if the methods that we promote fail to meet those standards, then we are wasting our teachers time and our students time. We can't afford to deflect attention from strategies that could produce better results.
To often, Ed Tech advocates focus upon ideals. Too often we focus upon the future that our students will encounter in the world of work and career, and we focus upon what our students will encounter in life. This focus leads us away from the need to prove, once and for all, that educational technology stands for student achievement...and standds for methods and strategies that out-produce every other method. (If the methods we advocate can't out-produce other methods for ensuring measurable student achievement, then we have no business promoting them.)
ISTE should also take a stand (and take the hit in terms of lucrative vendor booth revenues) by requiring that vendors provide evidence of the instructional effectiveness of the products that they sell. In particular, ISTE should go after (and reject) the hucksters of those inane, mind-numbing, baby-sitting, page-turning software programs that our unenlightened colleagues can "pop students in front of" when the students finish their drill-sheet assignments so that 1.) the students can "do more drill" and 2.) so that the teacher can claim to be integrating technology.
Getting this "junk software" out of the discussion when promoting the measurable student achievement goals of educational technology would go a long way in building credibility for our cause. Future employability and life-long learning goals are nice, but demonstrating measurable student outcomes, right now, gets the attention of "reluctant-to-use-technology" teachers and "eager-to-save-a-coin" administrators.
Summary: Let's focus on student and student achievement, let's make the use of technology transparent (the subject of another editorial) so the use of the technology doesn't call attention to itself, and let's promote only those strategies and methods that produce better measurable outcomes than everything else.
Scientific evidence for reliable, replicable, Ed Tech strategies and methods lacking? Let's get busy and prove the worth of technology integration to hard-nose, real-world teachers and purse-poor, bottom-line administrators.
Tags: achievement, and, integration, measurable, outcomes, promoting, student, technology
Replies to This Discussion
-
Permalink Reply by Sandra Plair on -
Joseph,
Could not agree more with your unsolicited commentary. Our local organization has just instituted an award program for action research to prove or disprove that technology is beneficial to students. Hope someone is listening out there.
-
Permalink Reply by Shafeen Charania on
-
Interesting discussion. It seems that you're assuming every student is the same, that only one teaching technique or tool should prevail, and that technology is a panacea.
Don't get me wrong, I do believe that technology can contribute to transformational learning, but I can't imagine that it solves every learning issue.
Also - ISTE should not exclude innovation from its ranks (your suggestion that only proven approaches be allowed to exhibit precludes new ideas, new approaches, etc. that might have a positive impact; it precludes that very discussion). I don't think you really meant to say that??
IMHO - it would be better to say (as you did) that some technologies should now be "transparent" in the classroom, but we must never exclude innovation. Whether that innovation is technology-based or otherwise, educators should care more about finding the best ways to reach our students, knowing that each one is different.
-
Permalink Reply by Rob Kadel on -
One of the things I love about many Ed Tech tools these days is that they have their own assessments built in. But it's kind of a catch-22. On the one hand, it's possible for most learning management systems to tailor work to each student's needs, based upon how he/she is assessed in quizzes, activities, etc. For lack of a better term, most systems can "learn" what each student needs next. Further, many systems (like Pearson eCollege, CompassLearning, Houghton-Mifflin's formerly-Riverdeep stuff) is designed to give teachers a heads-up view of how students are progressing, thereby allowing teachers to know if students are meeting academic goals and make adjustments where necessary. There's plenty of data out there, if people know how to access and use it correctly.
On the other hand, all of these data systems are products of the LMS's themselves, and that destroys the integrity, you might say, of using such data to determine if the system is effective. It's not an "unbiased third party." Asking CompassLearning to tell me how well I'm performing on CompassLearning is like asking me if my chili should win a chili cook-off. Of course it should! That's why I'm not a judge in the chili cook-off that I enter. There are external judges.
The same process plagues the data produced by today's LMS's. There's lots of data; it's fast; it helps the LMS learn about each student. But it's not an objective measure of how effective the LMS is in changing student achievement. For that, we need other measures -- like statewide standardized tests, but those are only somewhat indicative of the quality of student learning. And they're always delayed. Students learn X, Y, and Z content in October but aren't tested on it until March or April, and practitioners don't see the results until June. Phooey.
In short, I totally agree with your assessment, Joseph. Right now, we just don't have any universally agreed-upon objective measures to determine Ed Tech effectiveness.
-
Permalink Reply by Joseph Chmielewski on -
There can be another hidden assessment problem with packaged "learning systems." These systems require qualified and continual teacher vigilance, but this problem compounds itself when school district place "instructional aides" in charge of managing these lab programs.
The problem?
In these packages, the number of instructional approaches and the number of questions about a specific instructional target are limited. And, each student's individual learning style is assumed to be addressed because the "instruction" contains audio and visual components. But, these systems seldom use alternate instructional strategies besides a single on-screen, page-turning, audio-visual explanation that is replayed.
Unless teachers monitor these systems closely, these systems "learn" that the student always fails at a specific instructional task. What to do? The systems hit a ceiling, then stop wasting the student's time (and server processing cycles and bandwidth) by ignoring that objective.
Supervising teachers would catch this trend, "nip it in the bud," and provide as much individual tutoring as needed before students reach these frustration levels.
But, Lab Attendants seldom search data (for up to several hundred students) to find points of failure...and few Lab Attendances offer remedial tutoring.
The fact that "remedial" students are often funneled into these systems while the better students receive "real teachers" compounds this issue.
And, early research into the use of "drill programs" by Special Education students (Sp. Ed. was the only group with enough funding for the computers, and a low enough computer-to-student ratio to allow students substantial access) showed that more "time with computers" resulted in lower student achievement.
Maybe this inverse relationship between computer time and learning has changed? But, I suspect that the independent variable is the teacher.
If the teacher uses computers and technology for babysitting, for rewards when students "finish their work," or as a "take this kid off my hands (or out of my hair)" tutor; then the learning system, no matter how "research-based" will yield minimal positive, measurable learning outcomes.
However, the issue of measurable outcomes from technology expenditures holds greater import because...
* Decision-makers at all levels of school management want to see countable results generate from costly expenditures
* Decision-makers want to focus budgets on programs, initiatives and strategies that prove themselves to be better than other available options for driving those outcomes
* Proponents of technology integration failed to provide empirical evidence that technology solutions trump all other strategies for specific, desirable student learning outcomes
Technology Project Management centered on phases of infrastructure deployment and the deployment of hardware and software. We took the money and produced technically efficient installations in a cost-effective way...for as far as the money took us.
What we should have done was build a two-pronged project.
* Prong #1: Technology Tools for teachers
* Prong #2: Instructional tools for students
Technology tools for teachers should have followed the simple maxim known by mechanics, technicians and people in the building trades; i.e., "Buy the best tools that you can afford. Period." In the long run, the best tools are cost-effective, last longer, and are reliable under the extreme use of real-world situations.
Instructional tools for students should have been phased in based upon research that shows which technologies beat other strategies and methods for providing measurable students learning. From that research, and from those successes, new instructional technology methods would have evolved in a natural way.
Instead, we have a situation where new technologies are developed and advocates for those technologies try to drag teachers (sometimes kicking and screaming teachers) along with their schemes for solving all of education's woes with a technology magic pill.
The lack of measurable instructional outcomes runs far deeper than the inadequacies of curriculum-package-substitute-for-teachers'-inadequacy software systems.
-
Permalink Reply by cprofitt on -
A few things to consider:
1. How do pencils produce measurable student achievement? (Pencils are technology)
2. How does math courses produce measurable student achievement? (Technology is a subject as well as a tool)
3. How has 'business' used technology to produce measurable results for their shareholders?
---
In some aspect I think that the discussion is looked at from only one angle -- how can technology raise scores... and not as a 'topic' in and of itself.
In yet other aspects technology is seen as that which is not 'integrated' yet. Pencils, blackboards, etc were all technologies. Heck what did they use to teach students before Guttenberg; gasp! Handwritten books? Perhaps technology doesn't change the quality - but it makes it easier to produce the content.
Again, these are just my ramblings... I have never combined them all in to a meaningful thesis.
-
Permalink Reply by Joseph Chmielewski on
-
Excellent observations.
But let's consider a few other things. Easiest first...
(Excuse my taking a few items out of context.)
"1. How do pencils produce measurable student achievement? (Pencils are technology)"
Pencils as Technology:
1.) Pencil may or may not be a technology, but no one cares if a 25¢ pencil sits fallow for months...or years. Not so with computers, software and network infrastructure. The pencil is generally useful three years after it was purchased...again, not so computer technology.
2.) School district budget managers were not lead to believe (at least not in recent history) that the district's purchase of pencils would produce measurable student learning gains
3.) Teachers were not forced to "integrate" pencils into their instruction. When paper and pencils were available, teachers wanted to use them
4.) The use of a pencil is not complicated, yet teacher spend the first three to five years of a child's school career helping them to use it properly. This is called handwriting or penmanship
5.) Pencils require limited maintenance, done by the end user themselves at the back of the room with a sharpener; while computers require unending maintenance by highly paid, technical staff. And these high-tech folks require constant training and upgrading
6.) At the end of the pencil's lifecycle, its stub is tossed in the trash. Computers contain hazardous materials, and must be disposed of properly, often at a cost to the school district
Therefore the entry price and cost recovery associated with a pencil is low while the entry cost of networked computer technology is high and the percent of Return on Investment (ROI) for computer technology is low.
"2. How does [sic] math courses produce measurable student achievement?"
Math courses produce measurable student achievement by testing student competence before (pre-test) and after (post-test). Actually, this is a simple model to employ.
The issue that clouds high-stakes testing is the (unprofessional) misuse of aggregated test scores to judge teacher performance. Since the tests were not normed for the purpose of evaluating teachers, using high-stakes tests to evaluate teachers is unconscionable...supervisory-negligence on-the-cheap.
"Technology is a subject as well as a tool"
"Technology" cannot be a subject in the same way that a point, zero or infinity cannot be an academic subject. In these cases, the subject migh begeometry, calculus, imaginary numbers..or astral physics.
An academic pursuit is "in the use of," "in the fabricating of," "in the care of," "in the biology of," "in the physics of," "in the mathematics, psychology, sociology, history of," etc. of a topic.
A traditional example from the business world. "No one buys a drill because they want a drill. They buy a drill because they want to bore holes...and bore them quickly, effectively, efficiently."
No school district buys technology equipment because they want to show off expensive stuff, but because they want students to learn...to learn more quickly, more effectively, more efficiently,,,and at less cost per student.
"3. How has 'business' used technology to produce measurable results for their shareholders?"
This is called increasing productivity and garnering a Return on Investment (ROI) through expenditure for automation, data access and data mining, client resource management, knowledge management, social engineering, online marketing, etc. Expenditures are made and cost recovery is measured to determine if the expenditure is viable, if the expenditure leads to increased market share, etc.
School districts should operate in a business-like manner, but business models fail to be fully congruent with education in some respects.
For example, increasing market share. Few educators would condone a practice of joint ventures, partnerships, rebates and discounts so that graduates from one school obtain jobs with the area's employers while students from other schools were left unable to compete. While these strategies are acceptable business practices, these strategies fail to embody the societal values we hold for education.
"In some aspect I think that the discussion is looked at from only one angle -- how can technology raise scores... and not as a 'topic' in and of itself."
Almost right. The discussion focuses upon the failure of technology advocates to target valid and viable goals for technology expenditures. Technology advocates failed to "do their homework" to...
1.) Discover what technology, applied to content area learning, trumps all other methods of instruction in that specific area. If other instructional methods and strategies deliver better measurable outcomes than technology applications, then those other strategies should be deployed instead of technology-based strategies
2.) Document the independent variables related to learning outcomes with the employment of technology-based strategies. That is, to prove exactly what technology-based strategies can do and what these strategies can't do, show under what conditions, with what instructional models specific learning targets are achieved, etc.
Many early technology advocates lacked a long-term vision.
If technology advocates had seen "education-first" targets as the only goal for technology expenditures, and if proposals to school district executives had focused upon those measurable outcomes; then today's funding for technology would be assured, School district executive would consider technology a "must-have." Instead, technology expenditures are viewed (rightly) by school district executives as a cost center that exists without proven merit. This makes technology budgets easy to cut.
"Perhaps technology doesn't change the quality - but it makes it easier to produce the content."
Technology makes it easier to mass-produce content.
However, using expensive technology-based equipment for "factory-model" instruction also contributes to the minimal ROI realized by technology expenditures. This is like giving a rifle to a cave dweller, and the cave dweller using the rifle only after the prey is trapped in a net or a pit. Technology implementation models in current use in our schools are often limited by factory-based traditions of instructional delivery.
"...what did they use to teach students before Guttenberg [sic]; gasp! Handwritten books?"
Pre-printing press books were expensive and prized. Most academic learning was auditory and verbal.Only a few people with verbal intelligence and language-based learning styles and wealth or position availed themselves to academic pursuits.
Most skill learning was one-on-one, apprenticeship-based.
Education for large numbers of children is only a goal envisioned in our modern past.
-
Permalink Reply by cprofitt on
-
Good observations as well:
- pencils -
1. They are technology. I agree that computers are more expensive. Though the cost of computing is significantly less than it used to be while being much more integrated in our society.
2. I can not help what school budget leaders believe.
3. Your observation that teachers were not forced to integrate pencils is a bit of a 'false' argument... teachers already had enough training with pencils to understand how to use them. With computers that is not true. If teachers were as comfortable with computers as they were pencils they would not be 'forced' either.
4. It is true that a pencil is not complicated, but as you said several years of training and reinforcement are put in so students (who later become teachers) can be masters. The training also allows them to transfer their knowledge and skill to pens, markers and other 'writing technologies'. With computers students (who later become teachers) do not get enough training or reinforcement with computers to become masters. The training is also vendor specific and students (who later become teachers) have difficulty transferring knowledge from one computer technology to another.
5. Computers only require maintenance by highly trained and immensely intelligent technical staff because the end-user (teachers who used to be students) does not have sufficient training to be a master.
6. Yes, computers cost a great deal to recycle and pencils do not.
- math courses -
Math courses produce measurable student achievement by testing student competence before (pre-test) and after (post-test). Actually, this is a simple model to employ.
Technology courses could do the same. My point, though perhaps not communicated effectively, is that math is a subject area and treated as such. Math is seen as important and its knowledge is measured. One does not measure math solely on its ability to increase English subject area scores or physical education scores. In some instances technology is analogous to math. One can learn Word Processing, Programming, Photo Editing, Robotics, etc and a person's knowledge / competence can be tested.
"Technology" cannot be a subject in the same way that a point, zero or infinity cannot be an academic subject. In these cases, the subject migh begeometry, calculus, imaginary numbers..or astral physics.
Technology is an 'umbrella term' in the same manner math would describe addition, subtraction, multiplication, geometry, calculus, etc.
A traditional example from the business world. "No one buys a drill because they want a drill. They buy a drill because they want to bore holes...and bore them quickly, effectively, efficiently."
No school district buys technology equipment because they want to show off expensive stuff, but because they want students to learn...to learn more quickly, more effectively, more efficiently,,,and at less cost per student.
How does one learn to use a drill? Do they just buy it and know how immediately? How does on know how to pick the right drill bit? The right type of drill? Technology is a much broader set of tools than a drill. Again, my point is that knowing how to use a word processor might be an important skill. Knowing how to program might be an important skill. Knowing how to add memory to a computer might be an important skill. If those things are true then judging 'technology' on how much it raises student achievement in math is not a proper measure.
Technology makes it easier to mass-produce content.
I see technology as being able to differentiate learning in ways that traditional teaching can not unless the ratio of student to teacher is changed.
I see technology (computers and networks) providing the ability for student to more easily access information than they would have been able to prior to the Internet.
I see technology as a means to deliver data to decision makers in a more efficient and dynamic way than paper spread sheets and filing cabinets.
---
Again... the one point I see many miss -- is that technology (in a broad sense) is both a set of tools and a set of subjects. You evaluate a tool on its product. You evaluate a subject on the importance that subject has on helping students become productive members of society.
We do not teach milking a cow in most schools today because the skill is not 'required' or 'highly prized'. Knowing how to do word processing is, in today's world, a required skill. I would not evaluate word processing solely on how much it raises English or social studies scores because it is a required skill.
QUICK LINKS!
Badge
Latest Activity
Come play in the Clouds! Let's talk about the best new free tools on the web. Invigorate your approach to teaching and learning!
This is our forum to create an AWESOME Digital Arts Playground for ISTE 2010.
Making connetions in RL about SL - Educators, Avatars, & Digital Collaborators.
Special Interest Group for Technology Coordinators (SIGTC) Members Attending ISTE 2010
Teacher Librarians and SIGMS Members join together in conversation
Is it possible to participate and not be there? I will be working/volunteering at the digital playground at this time.
Great vid!!!
This is a group for people interested in global education and collaborative projects.
We have noticed a dramatic shift in the way educators are relating to their students or lack of. Have you? We held a conference recently in Houston, Texas and it was amazing to see how far we must go to reach parents, students and community leaders…
We have noticed a dramatic shift in the way educators are relating to their students or lack of. Have you? We held a conference recently in Houston, Texas and it was amazing to see how far we must go to reach parents, students and community leaders.…
About
© 2010 Created by Jennifer Ragan-Fore
