In its ongoing effort to set up a single accountability system that's coherent at the local, state and federal level, the State Board of Education is currently seeking applications to fill up to 15 positions on a newly constituted advisory committee.
The group will be called the California Practitioners Advisory Group and will provide input to the State Board of Education on ongoing efforts to establish said accountability system. The advisory committee will also serve as the state’s committee of practitioners under federal Title I requirements.
For more information about the application, please click here or , or contact the State Board by email at cpag@cde.ca.gov or by telephone at 916-319-0393. Complete applications must be received by 5:00 p.m. on Friday, February 26, 2016.
Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts
Wednesday, February 10, 2016
Tuesday, May 20, 2014
Should California delay implementation of statewide testing?
If you read my story about K-12 testing now underway at California schools, you probably remember expert Doug McRae saying the system will not likely be ready until September 2015. If you missed that fabulous piece of journalism, you can read it here.
McRae is now going a step further, saying the testing should be delayed until the spring of 2017, or even 2018.
His reasons? Common Core is not yet implemented at most districts, and it will probably won't happen in 2014-15 either, he writes in this op-ed piece he wrote for EdSource. He explains it more eloquently than I ever could, so check it out. And let me know what you think.
McRae is now going a step further, saying the testing should be delayed until the spring of 2017, or even 2018.
His reasons? Common Core is not yet implemented at most districts, and it will probably won't happen in 2014-15 either, he writes in this op-ed piece he wrote for EdSource. He explains it more eloquently than I ever could, so check it out. And let me know what you think.
Tuesday, May 13, 2014
Monterey's testing expert Doug McRae's rankling feathers at CDE
Testing expert Doug McRae, a former top executive with a local testing company, was busy two weeks ago visiting schools in Monterey County to see how the new testing system is going. I accompanied him to a couple of the schools, and even though I did not include any of my observations in my story in today's paper, the visits informed it a great deal.
McRae put together his comments in a meaty report to the California Board of Education, which apparently did not sit well with California Department of Education administrators. They immediately called at least one of the schools he visited to find out whether he had signed a confidentiality agreement.
Yes, CDE administrators. Mr. McRae is as "by-the-book" as they get, and he signed an affidavit at every single school he visited. He made me sign two of them, one at each school I accompanied him to. Frankly, I though that was overkill, but now I know why.
Plus, we were not in the room when the actual test began. Mr. McRae pushed me out the moment the test proctor read out loud "you may now begin the test."
We did not see ANY of the questions students had to answer, which is apparently a bigger secret than anti-aging cream formula.
Scientist that he is, McRae wrote a six-page analysis of his methodical observations. He chose different schools with different scholarly achievement, and ranked his commentaries accordingly. Here they are, for your enjoyment.
McRae put together his comments in a meaty report to the California Board of Education, which apparently did not sit well with California Department of Education administrators. They immediately called at least one of the schools he visited to find out whether he had signed a confidentiality agreement.
Yes, CDE administrators. Mr. McRae is as "by-the-book" as they get, and he signed an affidavit at every single school he visited. He made me sign two of them, one at each school I accompanied him to. Frankly, I though that was overkill, but now I know why.
Plus, we were not in the room when the actual test began. Mr. McRae pushed me out the moment the test proctor read out loud "you may now begin the test."
We did not see ANY of the questions students had to answer, which is apparently a bigger secret than anti-aging cream formula.
Scientist that he is, McRae wrote a six-page analysis of his methodical observations. He chose different schools with different scholarly achievement, and ranked his commentaries accordingly. Here they are, for your enjoyment.
Tuesday, December 3, 2013
Educational world abuzz about PISA tests: US students remain stagnant
Educators and pundits all over the United States are abuzz about the latest results of the Program for International Student Assessment (cuddly known as PISA) which scores 15-year-old students in 65 countries in math, reading, science and other topics. The students are chosen at random, and the tests are unique because they're not tied to any curriculum. In other words, it's what children really know and how they can apply their knowledge to the real world.
For more information about the test, click here.
The United States ranked 26th in math, 21st in science and 17th in reading. The rankings have not budged much in a decade.
Students in Shanghai-China, Hong Kong, Singapore, Japan and South Korea scored the highest in all three subjects. Switzerland and the Netherland also ranked near the top.
The results have already prompted a wave of commentaries from education analysts and lobbyists. We must spend more in early childhood education if the United States is going to move forward, some say. Others praise the advent of the Common Core Standards as a possible way for the United States to break out of the pack and move up in the educational ladder.
I'd like to point to a nifty graph I found from Rutgers Professor Bruce Baker, who concludes that, given the level of poverty in the United States, we're actually not doing that badly. You can find the graph here.
But Baker's graph also reminded me of a recent column by Castroville teacher and Herald Columnist Paul Karrer. In his column, Karrer takes issue with a New York Times writer Frank Bruni, who says students in the United States are coddled and that's the reason why they perform poorly compared to other countries. Bruni says the Common Core Standards will mean nothing if students are not challenged to do more.
Karrer responds with the realities of being in a classroom and teaching children with a myriad of problems that affect their learning: broken homes, medical conditions, you name it. Education happens best when children live in environments conductive to learning. And if they come to school with a hungry stomach or worried about whether their families will have a place to live next month, chances are their minds will not be ready to do math.
About one fifth of children in the United States live below 50 percent of median income. Only Turkey, Chile and Mexico have higher share of children living in poverty among ranked countries. Ironically, the United States is the sixth largest economy in the world, based on per capita income. But that income is not being used to reach everyone -- didn't lawmakers just last week cut foodstamps program? Half of the people who receive foodstamps are children, by the way.
People in positions of power refuse to invest in human capital, and as long as investment remains stagnant, education pundits should not expect U.S. students to improve achievement in international rankings, no matter how many educational reforms we undergo.
For more information about the test, click here.
The United States ranked 26th in math, 21st in science and 17th in reading. The rankings have not budged much in a decade.
Students in Shanghai-China, Hong Kong, Singapore, Japan and South Korea scored the highest in all three subjects. Switzerland and the Netherland also ranked near the top.
The results have already prompted a wave of commentaries from education analysts and lobbyists. We must spend more in early childhood education if the United States is going to move forward, some say. Others praise the advent of the Common Core Standards as a possible way for the United States to break out of the pack and move up in the educational ladder.
I'd like to point to a nifty graph I found from Rutgers Professor Bruce Baker, who concludes that, given the level of poverty in the United States, we're actually not doing that badly. You can find the graph here.
But Baker's graph also reminded me of a recent column by Castroville teacher and Herald Columnist Paul Karrer. In his column, Karrer takes issue with a New York Times writer Frank Bruni, who says students in the United States are coddled and that's the reason why they perform poorly compared to other countries. Bruni says the Common Core Standards will mean nothing if students are not challenged to do more.
Karrer responds with the realities of being in a classroom and teaching children with a myriad of problems that affect their learning: broken homes, medical conditions, you name it. Education happens best when children live in environments conductive to learning. And if they come to school with a hungry stomach or worried about whether their families will have a place to live next month, chances are their minds will not be ready to do math.
About one fifth of children in the United States live below 50 percent of median income. Only Turkey, Chile and Mexico have higher share of children living in poverty among ranked countries. Ironically, the United States is the sixth largest economy in the world, based on per capita income. But that income is not being used to reach everyone -- didn't lawmakers just last week cut foodstamps program? Half of the people who receive foodstamps are children, by the way.
People in positions of power refuse to invest in human capital, and as long as investment remains stagnant, education pundits should not expect U.S. students to improve achievement in international rankings, no matter how many educational reforms we undergo.
Wednesday, January 9, 2013
Which tests would California suspend under proposed changes?
Superintendent of California Schools Tom Torlakson proposed on Tuesday to revamp how students in K-12 get tested. Among the recommendations, he proposed all non-federally mandated tests be suspended beginning this school year. Many of us were scratching our heads: which tests would fall into that category?
Well, scratch your heads no more. Here's the list, provided by the California Department of Education. Enjoy!
Standardized Testing and Reporting (STAR) Program
Assessments That Would be Suspended During the 2013-14 School Year
(Not federally mandated or required for early assessment)
Grade-Level Assessments
Grade 2 ELA and Mathematics (CST and CAPA)
Grade 8 History–Social Science (CST)
Grade 9 ELA (CST, CMA, and CAPA)
Grade 9 Math (CAPA)
Grade 10 ELA (CST and CMA)
Grade 11 U.S. History (CST)
Grade 11 ELA (CMA and CAPA)
Grade 11 Math (CAPA)
Grade 2 - 11 Reading/Language Arts (STS)
Grade 2 – 7 Mathematics (STS)
End-of-Course Assessments
Algebra I (CST and CMA) (for grades 9-11)
Algebra II (CST) (for grades 9-10)
General Mathematics (CST) (grade 9)
High School Summative Math (CST) (for grades 9-10)
Geometry (CST and CMA) (for grades 9-11)
Integrated Mathematics 1 (CST) (for grades 9-11)
Integrated Mathematics 2 (CST) (for grades 9-11)
Integrated Mathematics 3 (CST) (for grades 9-11)
Biology (CST)
Chemistry (CST)
Earth Science (CST)
Physics (CST)
Integrated/Coordinated Science 1 (CST)
Integrated/Coordinated Science 2 (CST)
Integrated/Coordinated Science 3 (CST)
Integrated/Coordinated Science 4 (CST)
World History (CST)
Algebra I (STS)
Geometry (STS)
Notes:
EAP: Early Assessment Program
ELA: English-Language Arts
CAPA: California Alternate Performance Assessment
CMA: California Modified Assessment
CST: California Standards Test
STS: Standards-based Tests in Spanish
Well, scratch your heads no more. Here's the list, provided by the California Department of Education. Enjoy!
Standardized Testing and Reporting (STAR) Program
Assessments That Would be Suspended During the 2013-14 School Year
(Not federally mandated or required for early assessment)
Grade-Level Assessments
Grade 2 ELA and Mathematics (CST and CAPA)
Grade 8 History–Social Science (CST)
Grade 9 ELA (CST, CMA, and CAPA)
Grade 9 Math (CAPA)
Grade 10 ELA (CST and CMA)
Grade 11 U.S. History (CST)
Grade 11 ELA (CMA and CAPA)
Grade 11 Math (CAPA)
Grade 2 - 11 Reading/Language Arts (STS)
Grade 2 – 7 Mathematics (STS)
End-of-Course Assessments
Algebra I (CST and CMA) (for grades 9-11)
Algebra II (CST) (for grades 9-10)
General Mathematics (CST) (grade 9)
High School Summative Math (CST) (for grades 9-10)
Geometry (CST and CMA) (for grades 9-11)
Integrated Mathematics 1 (CST) (for grades 9-11)
Integrated Mathematics 2 (CST) (for grades 9-11)
Integrated Mathematics 3 (CST) (for grades 9-11)
Biology (CST)
Chemistry (CST)
Earth Science (CST)
Physics (CST)
Integrated/Coordinated Science 1 (CST)
Integrated/Coordinated Science 2 (CST)
Integrated/Coordinated Science 3 (CST)
Integrated/Coordinated Science 4 (CST)
World History (CST)
Algebra I (STS)
Geometry (STS)
Notes:
EAP: Early Assessment Program
ELA: English-Language Arts
CAPA: California Alternate Performance Assessment
CMA: California Modified Assessment
CST: California Standards Test
STS: Standards-based Tests in Spanish
Guest writer Doug McRae on the proposed revamping of testing
California Superintendent of Schools Tom Torlakson yesterday unveiled a proposal to revamp testing system for California schools. Here's my story on the topic.
As often is the case with topics on testing, I rely on the expertise of Doug McRae, statistician extraordinaire and former top executive with one of the leading testing companies in the country. He has a lot to say about the proposal, so here's what he wrote in response to Torlakson's detailed proposal. (You can also find that in the story).
By Doug McRae
General Comments
The purpose for a K-12 assessment system is the most important factor for the design of a new assessment system. The primary purpose defined by the report is found on page 4 when the report says the recommended system “has as its primary purpose to model and promote high-quality teaching and student learning activities.” This statement of purpose deviates from the usual purpose for a K-12 statewide assessment system, which is to measure the results of instruction. CA’s previous STAR system had measuring the results of instruction as its primary purpose, so the SPI is recommending a radical departure from the previous statewide assessments.
Based on many many years designing and developing large scale K-12 testing systems, I can say that the two purposes cited above are mutually incompatible. Systems designed for purposes of instruction do not serve the needs for measuring the results of instruction, and systems designed for purposes of measuring the results of instruction do not serve instructional needs well. The report itself says this in a one-sided way on page 5 when it says that STAR is valid for comparing school and district performance (i.e., measuring the results of instruction), but it was not designed to support instructional uses. What the report neglects to say is that assessment systems designed for instructional purposes are not designed to serve the needs of measuring the results of instruction (i.e., accountability uses) well.
The theme of the report focuses on use of assessment data for instruction. Now, data for instructional purposes is a vital ingredient for a comprehensive instructional system, but such data need to reside within the instructional system. Such an instructional system should have strong local school and district control to be aligned with other components of an instructional system, such as instructional materials and professional development. A statewide assessment system is centrally controlled with strict standardization and testing windows and test security that limit utility for instruction. In effect, the SPI recommendation replaces accountability testing by an approach to statewide testing motivated by helping instructional systems. As such, the report has a strong anti-accountability theme. Indeed, over the past 20 years, one of the constants in the debates over statewide assessment systems in the US has been the conflict between advocates of accountability and detractors of accountability for our K-12 schools, and advocating for instructional tests rather than tests that measure the results of instruction is in effect an anti-accountability Trojan Horse strategy that focuses on the benefits of instructional tests to hide an intent to minimize the use of accountability tests.
Rather than abandon the accountability purpose for CA’s statewide testing system, a far better approach would be to advocate for instructional tests as part of CA local control instructional systems, and reserve the mandated statewide assessment system efforts for measuring the results of instruction, or the accountability purpose.
The report provides very little information on three other aspects of its recommendations:
• Tests for instruction will take increased testing time – this is acknowledged generally in the report but no specific information is provided for how much increased time. From the information available from the Smarter Balanced consortium, the summative portion of the tests they plan to provide will increase CA’s statewide assessment time anywhere from 50 percent to 300 percent.
• Tests for instruction will require increased financial resources – again, this is generally acknowledged in the report but no specific information is provided for how many additional dollars will be needed. Based on information currently available on the Smarter Balanced tests that the SPI anticipates will serve as the core for a future statewide assessment system, the costs will increase anywhere from 50 percent to 300 percent.
• The prospects for California being able to implement a computer-adaptive testing system by 2014-15 are not addressed at all in the report. California will have to upgrade its technology hardware and bandwidth significantly simply to administer computer-adaptive tests, just to permit schools to conduct test administrations. In addition, whether or not students will be ready to take computerized tests by 2014-15 is an unknown that has yet to be explored at all. Frankly, the chances that California will have the hardware capability or the student experience needed to implement a statewide computerized test by 2014-15 are slim and none. [More on this under detail recommendation (2) below.]
Detailed Recommendations
The report provides 12 detailed recommendations on pp 41-47, with a summary table on p 48. I’ll now provide initial observations on these 12 recommendations.
(1) Suspend Portions of STAR Beginning Spring 2014
This recommendation is to suspend all parts of STAR not used for federal accountability reporting starting 2014. No rationale is given for this recommendation other than “suspending assessments . . . . will allow staff and stakeholders to focus attention, efforts, and resources on building a new assessment and accountability system.” The only logical way to interpret this recommendation is that the SPI and CDE staff are anti-accountability testing and they want to use their time to design a statewide instructional assessment system. The recommendation ignores the benefits of a substantial amount of data generated by the current STAR program, and the achievement data trends it supplies for schools and districts as well as the state. A far more nuanced recommendation would be to suspend the portions of STAR that generate duplicative and/or unneeded information; there are some major portions of our current statewide assessment system (STAR as well as CAHSEE) that can be modified without losing valuable specific information and/or trend information.
(2) Fully Implement Smarter Balanced E/LA and Math Tests in 2014-15
We have no idea whether we can implement a computerized test for 3-4 million kids by 2014-15 yet. As indicated above, hardware and bandwidth upgrades of unknown quantities will be needed, and students have not been prepped to be able to generate valid and reliable scores from computerized assessments yet. The report notes under this recommendation that for students unable to access a computer, a paper-and-pencil version of the SBAC assessment will be provided for up to 3 years. However, there is no discussion whether the two versions of the SBAC assessments will yield comparable scores that would allow for local school, local district, or statewide aggregation; the technical considerations for generating comparable scores particularly for variable form computer-adaptive tests and fixed form paper-and-pencil tests are substantial, and indeed the state-of-the-art for providing such comparisons is in its infancy. The “phase-in over 3 years” recommendation is shaky, much more so for computer-adaptive tests than for fixed form computer-administered tests and counterpart fixed-form paper-and-pencil tests.
(3) Use Grade 11 SBAC E/LA and Mathematics Tests as Indicators of College Readiness Other than college admission tests such as the College Board SAT and the ACT, the idea of measuring college readiness and/or career readiness across the full spectrum of grade 11 enrollment is a relatively new idea, and frankly assessment system designers do not have much experience with this concept. Initial progress by both consortia (Smarter Balanced and PARCC) now developing “next generation” assessments has been slow. The idea that we will have a fully functional college and/or career readiness assessment instrument by 2014-15 is fanciful thinking.
(4) Develop and Administer Science Assessments Aligned to New Science Standards No standards yet, no details yet, no timeline yet, other than these new assessments will be like the E/LA and Math assessments in terms of their instructional qualities. A pretty empty recommendation right now.
(5) Use Multistate Consortia Alternate Assessments to Replace CAPA To date, virtually no information has been disseminated on CA’s involvement in the NCSC consortium of which CA is a member, nor to my knowledge has the consortium itself disseminated information on its progress.
(6) Determine the Need for Assessments in Other Languages than English Primary language assessments are needed to provide evaluation of the effectiveness of various bilingual education programs (i.e., to measure the results of instruction for these programs). California has had an ineffective primary language program for years since the current test was not designed to provide valid accountability information.
(7) Assess the Full Curriculum with Model Instructional Tests
As indicated under General Comments above, this recommendation is a fine contribution to a local control comprehensive instructional system, but does not belong in a large scale standardized secure centrally controlled statewide assessment system.
(8) Invest in Interim, Diagnostic, and Formative Testing Tools
Again, as indicated under General Comments above, these tools are fine for a locally controlled comprehensive instructional system, but not for a centrally controlled statewide assessment system. There is a significant danger that including these components in a statewide assessment system will result in these components simply being used to game a statewide summative assessment system designed to measure the results of instruction, or in effect as state encouraged institutional tools to “teach to the test” in ways more pervasive than current “teaching to the test” efforts. Teaching to the test efforts degrade both the validity of statewide assessment results and good instructional practices.
(9) Consider Alternatives to CAHSEE
CAHSEE is essentially a redundant test for a substantial number of 10th graders in CA. We can use STAR results from earlier grades as “early qualification” tests for the CAHSEE high school graduation requirement. This efficiency for our statewide assessment system has been ignored for years by Sacramento policymakers, and is a major contributor to over testing our high school students.
(10) Explore Matriculation Exams
Finally, a recommendation I can agree with . . . . .
(11) Conduct Comparability Studies
These studies are absolutely necessary, but they will impose a substantial time requirement for students to take both tests. The representative samples needed may be problematic if any phased implementations over three years recommended above takes place if indeed certain subgroups of students (i.e., our underserved subgroups) all take paper-and-pencil tests while other subgroups (non-minorities, high SES) take computer-adaptive tests. Achieving adequate study designs for these studies will not be trivial.
(12) Maintain a Continuous Cycle of Improvement
Always a good recommendation for any “future” report . . . . . fills out a nice round number for recommendations.
As often is the case with topics on testing, I rely on the expertise of Doug McRae, statistician extraordinaire and former top executive with one of the leading testing companies in the country. He has a lot to say about the proposal, so here's what he wrote in response to Torlakson's detailed proposal. (You can also find that in the story).
By Doug McRae
General Comments
The purpose for a K-12 assessment system is the most important factor for the design of a new assessment system. The primary purpose defined by the report is found on page 4 when the report says the recommended system “has as its primary purpose to model and promote high-quality teaching and student learning activities.” This statement of purpose deviates from the usual purpose for a K-12 statewide assessment system, which is to measure the results of instruction. CA’s previous STAR system had measuring the results of instruction as its primary purpose, so the SPI is recommending a radical departure from the previous statewide assessments.
Based on many many years designing and developing large scale K-12 testing systems, I can say that the two purposes cited above are mutually incompatible. Systems designed for purposes of instruction do not serve the needs for measuring the results of instruction, and systems designed for purposes of measuring the results of instruction do not serve instructional needs well. The report itself says this in a one-sided way on page 5 when it says that STAR is valid for comparing school and district performance (i.e., measuring the results of instruction), but it was not designed to support instructional uses. What the report neglects to say is that assessment systems designed for instructional purposes are not designed to serve the needs of measuring the results of instruction (i.e., accountability uses) well.
The theme of the report focuses on use of assessment data for instruction. Now, data for instructional purposes is a vital ingredient for a comprehensive instructional system, but such data need to reside within the instructional system. Such an instructional system should have strong local school and district control to be aligned with other components of an instructional system, such as instructional materials and professional development. A statewide assessment system is centrally controlled with strict standardization and testing windows and test security that limit utility for instruction. In effect, the SPI recommendation replaces accountability testing by an approach to statewide testing motivated by helping instructional systems. As such, the report has a strong anti-accountability theme. Indeed, over the past 20 years, one of the constants in the debates over statewide assessment systems in the US has been the conflict between advocates of accountability and detractors of accountability for our K-12 schools, and advocating for instructional tests rather than tests that measure the results of instruction is in effect an anti-accountability Trojan Horse strategy that focuses on the benefits of instructional tests to hide an intent to minimize the use of accountability tests.
Rather than abandon the accountability purpose for CA’s statewide testing system, a far better approach would be to advocate for instructional tests as part of CA local control instructional systems, and reserve the mandated statewide assessment system efforts for measuring the results of instruction, or the accountability purpose.
The report provides very little information on three other aspects of its recommendations:
• Tests for instruction will take increased testing time – this is acknowledged generally in the report but no specific information is provided for how much increased time. From the information available from the Smarter Balanced consortium, the summative portion of the tests they plan to provide will increase CA’s statewide assessment time anywhere from 50 percent to 300 percent.
• Tests for instruction will require increased financial resources – again, this is generally acknowledged in the report but no specific information is provided for how many additional dollars will be needed. Based on information currently available on the Smarter Balanced tests that the SPI anticipates will serve as the core for a future statewide assessment system, the costs will increase anywhere from 50 percent to 300 percent.
• The prospects for California being able to implement a computer-adaptive testing system by 2014-15 are not addressed at all in the report. California will have to upgrade its technology hardware and bandwidth significantly simply to administer computer-adaptive tests, just to permit schools to conduct test administrations. In addition, whether or not students will be ready to take computerized tests by 2014-15 is an unknown that has yet to be explored at all. Frankly, the chances that California will have the hardware capability or the student experience needed to implement a statewide computerized test by 2014-15 are slim and none. [More on this under detail recommendation (2) below.]
Detailed Recommendations
The report provides 12 detailed recommendations on pp 41-47, with a summary table on p 48. I’ll now provide initial observations on these 12 recommendations.
(1) Suspend Portions of STAR Beginning Spring 2014
This recommendation is to suspend all parts of STAR not used for federal accountability reporting starting 2014. No rationale is given for this recommendation other than “suspending assessments . . . . will allow staff and stakeholders to focus attention, efforts, and resources on building a new assessment and accountability system.” The only logical way to interpret this recommendation is that the SPI and CDE staff are anti-accountability testing and they want to use their time to design a statewide instructional assessment system. The recommendation ignores the benefits of a substantial amount of data generated by the current STAR program, and the achievement data trends it supplies for schools and districts as well as the state. A far more nuanced recommendation would be to suspend the portions of STAR that generate duplicative and/or unneeded information; there are some major portions of our current statewide assessment system (STAR as well as CAHSEE) that can be modified without losing valuable specific information and/or trend information.
(2) Fully Implement Smarter Balanced E/LA and Math Tests in 2014-15
We have no idea whether we can implement a computerized test for 3-4 million kids by 2014-15 yet. As indicated above, hardware and bandwidth upgrades of unknown quantities will be needed, and students have not been prepped to be able to generate valid and reliable scores from computerized assessments yet. The report notes under this recommendation that for students unable to access a computer, a paper-and-pencil version of the SBAC assessment will be provided for up to 3 years. However, there is no discussion whether the two versions of the SBAC assessments will yield comparable scores that would allow for local school, local district, or statewide aggregation; the technical considerations for generating comparable scores particularly for variable form computer-adaptive tests and fixed form paper-and-pencil tests are substantial, and indeed the state-of-the-art for providing such comparisons is in its infancy. The “phase-in over 3 years” recommendation is shaky, much more so for computer-adaptive tests than for fixed form computer-administered tests and counterpart fixed-form paper-and-pencil tests.
(3) Use Grade 11 SBAC E/LA and Mathematics Tests as Indicators of College Readiness Other than college admission tests such as the College Board SAT and the ACT, the idea of measuring college readiness and/or career readiness across the full spectrum of grade 11 enrollment is a relatively new idea, and frankly assessment system designers do not have much experience with this concept. Initial progress by both consortia (Smarter Balanced and PARCC) now developing “next generation” assessments has been slow. The idea that we will have a fully functional college and/or career readiness assessment instrument by 2014-15 is fanciful thinking.
(4) Develop and Administer Science Assessments Aligned to New Science Standards No standards yet, no details yet, no timeline yet, other than these new assessments will be like the E/LA and Math assessments in terms of their instructional qualities. A pretty empty recommendation right now.
(5) Use Multistate Consortia Alternate Assessments to Replace CAPA To date, virtually no information has been disseminated on CA’s involvement in the NCSC consortium of which CA is a member, nor to my knowledge has the consortium itself disseminated information on its progress.
(6) Determine the Need for Assessments in Other Languages than English Primary language assessments are needed to provide evaluation of the effectiveness of various bilingual education programs (i.e., to measure the results of instruction for these programs). California has had an ineffective primary language program for years since the current test was not designed to provide valid accountability information.
(7) Assess the Full Curriculum with Model Instructional Tests
As indicated under General Comments above, this recommendation is a fine contribution to a local control comprehensive instructional system, but does not belong in a large scale standardized secure centrally controlled statewide assessment system.
(8) Invest in Interim, Diagnostic, and Formative Testing Tools
Again, as indicated under General Comments above, these tools are fine for a locally controlled comprehensive instructional system, but not for a centrally controlled statewide assessment system. There is a significant danger that including these components in a statewide assessment system will result in these components simply being used to game a statewide summative assessment system designed to measure the results of instruction, or in effect as state encouraged institutional tools to “teach to the test” in ways more pervasive than current “teaching to the test” efforts. Teaching to the test efforts degrade both the validity of statewide assessment results and good instructional practices.
(9) Consider Alternatives to CAHSEE
CAHSEE is essentially a redundant test for a substantial number of 10th graders in CA. We can use STAR results from earlier grades as “early qualification” tests for the CAHSEE high school graduation requirement. This efficiency for our statewide assessment system has been ignored for years by Sacramento policymakers, and is a major contributor to over testing our high school students.
(10) Explore Matriculation Exams
Finally, a recommendation I can agree with . . . . .
(11) Conduct Comparability Studies
These studies are absolutely necessary, but they will impose a substantial time requirement for students to take both tests. The representative samples needed may be problematic if any phased implementations over three years recommended above takes place if indeed certain subgroups of students (i.e., our underserved subgroups) all take paper-and-pencil tests while other subgroups (non-minorities, high SES) take computer-adaptive tests. Achieving adequate study designs for these studies will not be trivial.
(12) Maintain a Continuous Cycle of Improvement
Always a good recommendation for any “future” report . . . . . fills out a nice round number for recommendations.
Monday, October 15, 2012
So many schools in Monterey County, so little time
California officials handed score cards to all state schools last week, as you probably remember from this story. Many schools went up, some went down, and overall, academic performance at Monterey County public schools declined for the third year in a row.
This trend deserves further examination, something I hope to get around to in the upcoming weeks. Also, another trend that's been brought to my attention and deserves further scrutiny is the increased use of "alternative" testing for special education children -- something that data crunching guru Doug McRae believes is "inflating" test results statewide. Here's his story on EdSource, in case you're curious.
At the local level, there are several local districts that are using high proportion of alternative testings. I'll be looking into that sometime in the near future.
But I'd also love to look into high performing schools. Bay View Academy, for instance, hit its scores off the park by scoring 849 in its inaugural year -- three points ahead of Monte Vista Elementary, which, given Monterey demographics, is actually not very surprising. I'm sure folks at both schools are very pleased, as they should.
Also noteworthy is the Monterey Bay Academy, which also went up by 21 points in the API scale. I'd love to explore what the school's doing, how they're tackling the challenges other schools have with low income, English learners, and other "subgroups." Stay tuned.
Finally, two schools of the Greenfield Union School District made impressive gains. Mary Chapa went up by 108 points. Its sister campus, El Camino Technology Academy, went up 73 points. And nearby Cesar Chavez went down by 33 points. Why the discrepancy?
So many stories...
This trend deserves further examination, something I hope to get around to in the upcoming weeks. Also, another trend that's been brought to my attention and deserves further scrutiny is the increased use of "alternative" testing for special education children -- something that data crunching guru Doug McRae believes is "inflating" test results statewide. Here's his story on EdSource, in case you're curious.
At the local level, there are several local districts that are using high proportion of alternative testings. I'll be looking into that sometime in the near future.
But I'd also love to look into high performing schools. Bay View Academy, for instance, hit its scores off the park by scoring 849 in its inaugural year -- three points ahead of Monte Vista Elementary, which, given Monterey demographics, is actually not very surprising. I'm sure folks at both schools are very pleased, as they should.
Also noteworthy is the Monterey Bay Academy, which also went up by 21 points in the API scale. I'd love to explore what the school's doing, how they're tackling the challenges other schools have with low income, English learners, and other "subgroups." Stay tuned.
Finally, two schools of the Greenfield Union School District made impressive gains. Mary Chapa went up by 108 points. Its sister campus, El Camino Technology Academy, went up 73 points. And nearby Cesar Chavez went down by 33 points. Why the discrepancy?
So many stories...
Thursday, June 14, 2012
Of API, school rankings, and minor annoyances
Everyone has complaints about test scores, so I'm going to jump on the bandwagon.
The state released school ranks today -- Thursday (my head is spinning from all those numbers)
They also release the number on the academic performance index -- the blessed API -- schools must reach in order to be considered to be making progress.
I know this is a number the schools can calculate themselves, so why bother? Plus, the test results are going to be announced in a few weeks -- nobody is getting any time to know what to reach for. The students already took the tests in April.
Why bother?
The state released school ranks today -- Thursday (my head is spinning from all those numbers)
They also release the number on the academic performance index -- the blessed API -- schools must reach in order to be considered to be making progress.
I know this is a number the schools can calculate themselves, so why bother? Plus, the test results are going to be announced in a few weeks -- nobody is getting any time to know what to reach for. The students already took the tests in April.
Why bother?
Subscribe to:
Posts (Atom)
Posts
