I found this article, Mute the Messenger, through my RSS feed this week. I found the article fascinating. Essentially, it is the tale of standardized testing and what could potentially be the ugly reality of assessment. It is not the shortest of articles, but a great read.

Now, take the article with a bit of skepticism. Still, it is a very powerful article. Yes, many of the points may be simply circumstantial. Yes, there could be a lot of information that is missing. Still.

Let’s take a look at a few of the quotes.

Testing advocates believed that more rigorous curricula and tests would boost student achievement—the “rising tide lifts all boats” theory. But that’s not how it worked out.

This is one of the powerful quotations for me. There is a fundamental belief that making the curriculum and assessments more rigorous would “obviously” led to more learning. Funny thing about learning though, sometimes it is more complex than people want to think.

Texas Education Commissioner Robert Scott, long an advocate of using tests to hold schools accountable, broke from orthodoxy when he called the STAAR test a “perversion of its original intent.”

Yep. Some are starting to realize that just increasing testing isn’t the panacea that some want to think that it is.

Stroup sat down at the witness table and offered the scientific basis behind the widely held suspicion that what the tests measured was not what students have learned but how well students take tests.

…his testimony to the committee broke through the usual assumption that equated standardized testing with high standards. He reframed the debate over accountability by questioning whether the tests were the right tool for the job. The question wasn’t whether to test or not to test, but whether the tests measured what we thought they did.

This points out a profound function of testing that all too many take for granted. What does testing really measure? Yes, we end up with a number at the end of testing. However, what does that number really mean? What do tests really measure? These are crucial important questions.

Stroup argued that the tests were working exactly as designed

Stroup had caught the government using a bathroom scale to measure a student’s height.

The scale wasn’t broken or badly made. The scale was working exactly as designed. It was just the wrong tool for the job. The tests, Stroup said, simply couldn’t measure how much students learned in school.

Here is the crux of the matter. Are we really using the right tools? Are we using assessments correctly? Are we sure that the assessments measure what we think that they measure? I remember times as a principal where the number one question was “what was the topic of the writing” section. Once we knew what the topic was, we were pretty sure (and always right on) about how the students would do on the assessment. Quite frankly, we knew that the topic was really, really important. We knew how well the students could write. Even more importantly, we knew that if the topic was something that the students weren’t interested in, they would not do well on the assessment.

Well, one of the legislators called for Stroup and Pearson to have a debate. That debate would never happen.

…standardized tests have become the pre-eminent yardstick of classroom learning in America, and Pearson is selling the most yardsticks.

Pearson is heavily invested (literally) in assessment. Quite frankly, they are selling the yardsticks.

But, here’s one of the interesting things. Stroup was also teaching kids. He had developed a program that helped students learn math. He knew that the kids were being successful, but that success wasn’t showing up on the statewide standardized tests. He started looking at why.

Stroup knew from his experience teaching impoverished students in inner-city Boston, Mexico City and North Texas that students could improve their mastery of a subject by more than 15 percent in a school year, but the tests couldn’t measure that change. Stroup came to believe that the biggest portion of the test scores that hardly changed—that 72 percent—simply measured test-taking ability. For almost $100 million a year, Texas taxpayers were sold these tests as a gauge of whether schools are doing a good job. Lawmakers were using the wrong tool.

So, he does the research and finds out that what the tests really measure is how well students take the test. His research found that 70% of the test score was “insensitive to instruction”. Essentially, this means that teachers, schools and educators can’t change about 70% of the test results. Pearson called foul. They stated that he had made a mistake. According to Pearson, only 50% of the test is “insensitive to instruction”. That’s right. Pearson admitted that about half of the score that would determine how well teachers were teaching was unchangeable by the teacher. Honestly, teachers are being evaluated by these scores. Jobs, reputations, etc. – all determined by these tests. Yet, here is Pearson admitting that 50% of that score is determined by the student’s ability to take a test. Nothing the teacher or school could do would effect this part of the score.

Stroup concluded that the tests were 72 percent “insensitive to instruction,” a graduate- school way of saying that the tests don’t measure what students learn in the classroom.

After correcting what Pearson interpreted as the mislabeled column, Way wrote, the tests were “only 50 percent” insensitive to instruction.

“teachers account for about 1% to 14% of the variability in test scores,” largely confirming Stroup’s apparently controversial conclusion.

If it’s true that the test measured primarily students’ ability to take a test, then, Stroup reasoned to the House Public Education Committee in June 2012, “it is rational game theory strategy to target the 72 percent.” That means more Pearson worksheets and fewer field trips, more multiple-choice literary analysis and fewer book reports, and weeks devoted to practice tests and less classroom time devoted to learning new things. In other words, logic explained exactly what was going on in Texas’ public schools.

Oh, and the legislator who had called for a debate between Dr. Stroup and Pearson. The debate that never happened. Well, he retired. He is now a lobbyist for Pearson.

Source: http://www.texasobserver.org/walter-stroup-standardized-testing-pearson/

The original concept of the web was to connect documents. Obviously, the web has grown and expanded. Now, there are many, many different places to put “your documents”. Now your “documents” are no frequently longer web pages or documents that you write, but rather things that you post on other’s web sites.

Personal Examples

Lots of people now post their “documents” to places like Facebook, Twitter, LinkedIn, InstaGram, Tumblr, etc. These sites all make it really easy to post information. They all vary in how easy it is to get your data out of their system. They also control where your data lives. Generally, the trade off is the ease of use to post (and network effect of those there) in exchange for your data living in their ecosystem.

Professional Examples

Similar to the personal examples above, there are many professional examples as well. Many people are using some the of the sites above in a professional manner. There are also sites like BlackBoard, Edmodo, Haiku and more. These sites vary from some free access to some very expensive options. Beyond the cost (because usually the cost is born by the school district, not the individual teacher) is the idea of who owns the data. Here the data is the very hard work that a teacher has put into developing that site. Getting that data out can be difficult.

Why does this matter?

I know a teacher who recently switched districts. This is a technology proficient teacher who had created a wide range of classes, activities, resources, etc digitally. However, the district that this teacher was in, had contracted their LMS out. That means that the teacher could not take her data with her. She had some terrific lessons that she had spent time developing and could really use now. However, her data is locked up somewhere else. She can no longer get to it.

There is also a movement called the IndieWeb. The basic concept is that you write a text post, post a picture, post video or some other type of data and then it is connected to whichever other destinations that you want it posted. For example, you could take a picture and post that picture to Instagram, Facebook. This means that even if one of those sites goes away, you still have your picture. Think about how many pictures exist on Facebook only. Maybe that’s fine. But it should be a choice. Here the idea is that you own the data, you post the original and “copies” are distributed around to other sites.

Now the same thing really can be done right now. One can set up a web site, purchase a domain name, install a couple of plug-ins and away you go. This is what I’ve chosen to do via this site. However, it does take some work. Not everyone is going to do that.

It is far easier to sign up for a service like any of those above. Next, hope that there are people there (and especially the right people). Be thoughtful though.

Professionally

This is why I’ve started and use this site. This is one reason that I’m a big believer in Moodle. With this site, I’ve also installed Moodle. I own the data. I can use it however that I want. I’m not dependent upon others. That also means that I need to maintain the site. *It’s really not that hard to maintain.

No matter what you choose, at least make the choice thoughtfully. Do you want to own the data? Do you care if the data goes away? Just how important is the data?

Cheers.

I’m currently enjoying the Traverse City Film Festival. One of the movies that I chose was The Internet’s Own Boy:The Story of Aaron Schwartz. See below for a movie recap and my thoughts.

Movie Recap

For those of you who are familiar with the story, Aaron was a gifted child who learned to program. At the age of 14, he helped write the standard of RSS (RSS is a fundamental standard for the web, even though most people don’t know what it is, they use it). Note that he was involved in writing the standard. These are the guiding principles of RSS. This is one example of the deep thoughts that he had, and communicated.

He was also one of the founders of Reddit. When Reddit was bought out, Aaron made a good bit of money. However, he really didn’t fit in with corporate culture (he complained that he wanted to work) and famously got himself fired. He didn’t fit in within the Silicon Valley culture. He felt that people said that they wanted to change the world and make things better, but didn’t do anything tangible to make that happen. He left Silicon Valley and went back to New York. He became more and more interested in social activism.

One of his fundamental beliefs was that the information should be free. He, along with some others but posted under his name, wrote the Guerilla Open Access Manifesto. This talks about how information is power and that power should not be “locked up” and held by the few.

Fast forward. Aaron works with group that is “recycling” court documents that people pay to access. The government has set up PACER which is supposed to provide the citizenry with access to official court records. However, they charge for that access. The group, and Aaron, felt that access to the law should be a fundamental right for all. That it shouldn’t be restricted to just those who can afford to pay for it. This was a major theme for Aaron. Information should be free.

Aaron ended up downloading Gigabytes of information from JSTOR through a laptop placed at MIT. The Federal government charged Aaron with several felonies. Since Aaron wanted to work in Congress or the White House, he had grown to understand where he could make true changes, and knew that a felony would mean that he couldn’t do that, he fought the charges.

The fight went on for a couple of years. JSTOR issued a notice saying that they were not interested in pursuing charges. MIT refused to publicly support dropping the charges. Still the Federal government pursued the charges. In fact, they added more felony charges under the Computer Fraud and Abuse Act of 1986. The only thing that the government told Aaron’s dad is that they wanted to make Aaron an “example”. What was never clear was the example of what.

During the time of the prosecution, Aaron was also fighting for social justice. He was one of the largest voices and organizers of the movement to stop SOPA. When SOPA was first proposed, it looked like a slam dunk. That Aaron and others were able to effectively organize and defeat SOPA was a key to showing Aaron how influential and effective social organizing could be.

Aaron ended up committing suicide.

My Take

One of the things that struck me the most is how much things have changed and how unsettled those changes are for us as a society. Aaron seemed to be caught up in that. He wanted to make changes. He saw things a little differently He fought for knowledge to be free. Fought is the operative word here. There are many who are invested in the way things are (or used to be). Aaron saw that things could be different and worked to make changes.

Another theme was the lack of Congressional understanding. Congress started investing the issue after the public outcry. On of the lines repeated in the movie is “Bring in the nerds” from several Congressmen. The message was simple. The members of Congress were saying “we don’t understand this” and need someone to explain the issues to us. (Congress used to have an Office of Technology Assessment that any member could go to for help understanding these issues, but Newt Gingrich disbanded that group in 1995). Thus, we’ve ended up with Congress making laws for things that they clearly don’t understand. Not only do they not understand, but seem comfortable in in acknowledging that they don’t know (hence the jokes about “bringing in the nerds”). Add in the idea that the Federal prosecutor was using the Computer Fraud and Abuse Act of 1986. This law has changed very little since 1986. Aaron was charged with “recklessly damaging a protected computer” among other things. The law makes no distinction between what happens with the information. Did it make a difference if he was going to use the information to learn something versus making the information free to anyone versus selling the information? (Interestingly, Aaron had downloaded a large amount of information in the past and used the information to analyze what was going on. He did not make the information public).

Public education is in a similar position as Congress. Things have changed. The world has changed. It is time for us to evaluate where we are, what we need to differently, what changes that we need to make. We won’t get it perfect, but we need to continue the dialogue.