Description of this paper


Comprehensive Doctoral Review Examination Presented in Partial-(Answered)


Instant Solution ? Click "Buy button" to Download the solution File


I have completed an assignment for comprehensive exam and all I need is for someone to check how I did with the explanation of the data analysis. How much will this cost? I have the sections that need to be checked highlighted everything else was checked already.

Comprehensive Doctoral Review Examination


Presented in Partial Fulfillment


of the Requirements for the Degree


Doctor of Philosophy






Sally A. Camacho


English as a Second Language


School of Education


Northcentral University


April 17, 2016





Question 3: Research


For question 3, you will need to demonstrate the ability to employ research methods used within


your specialization. You are expected to critique research methodologies used by scholarpractitioners and compose responses identifying best practices in educational research.


Select five empirical articles from peer-reviewed journals you consider critical to your


understanding of your specialization and:


1. Describe each study, including:









The research problem, questions, or hypotheses


The research purpose


The type of design and elements of the design (e.g., sample, data analysis,


operationalization of constructs)


Threats to validity and if and how addressed


The findings and implications


Note if the article makes a compelling case for the meaning and significance of the





2. Select one of the five articles above. Using the article, assess how you might conduct the


research study differently. Address:






The research questions


The research purpose


Type of design and elements of the design (e.g., sample, the type of data you need to


collect and how you will collect it, data analysis)


The strengths and weaknesses of your envisioned design and methods


Quantitative: threats to validity and how your design will address them


Quantitative: the constructs you will measure and what you will do in order to determine


how to operationalize them (you do not need to identify specific measures)


Qualitative: your means of ensuring the quality of your findings


Justification for why your chosen design and methods are more appropriate for your


research question than alternatives you have considered


Your methods of data analysis


How the data you collect will enable you to answer your research question and


contribute to theory


Learning Outcomes


3. Apply relevant theory and research from the specialization coursework to real life situations to


solve specific problems and discusses implications.






Part I


Research Methods used within the Education of English Language Learners and the


Challenges they face with High-Stakes Tests


The methods considered were based on the education of English Language Learners


(ELL) and the challenges they face with high-stakes tests. The five articles explore the changes


in the United States educational, standards, accountability measures and regulations during


testing. The new educational initiatives challenge content area teachers teaching ELL students to


meet the goals set for content assessment during the end of the year standard testing. The ELL


population continues to grow and continues to confront disadvantages in linguistic and cultural


diversity. Teachers are charged with the difficult task of demonstrating content mastery of all


students including non-ELL students and ELL students. School districts must find a way to


address standards while supporting the needs of ELL without watering down content. The


articles provide much insight on what has been considered for the improvement in the decision


making process of accommodations, preparation for testing, and testing.


Selected Empirical, Peer Reviewed Articles


Article One: Do decision rules matter? A descriptive study of English language proficiency


assessment classifications for English language learners and native English speakers in fifth




The research study?s problem is the lack of guiding principles used for classifying and or


reclassifying ELL students on language proficiency assessments. The English Language


Proficiency (ELP) performance standard is used for accountability allowing for the federal


government to report on student academic achievement, but if the students? are not classified


correctly, the report will not be accurate. The research questions focus on to what degree the


decision rules vary in the total amount of students who may be classified as non-proficient. The





second question focuses on to what degree decision rules classify the high-achieving students as


non-proficient and the last question centers around to what degree decision rules vary in the total


amount of students who may be classified as unqualified for redesignation. The purpose of this


research study is to operationalize and test the ELP performance standards and the decision


making of the students? classification and eligibility. The procedures began with the


administration of ELL students with the English language proficiency assessment (ELPA). A


mandated assessment is given annually to ELL students. Another group of non-ELL students


were also tested for research purposes only during the scheduled testing window.


The design of this study was descriptive focusing on operationalizing and testing English


Language Proficiency (ELP) performance standards for student-level decision making. The study


collected data from State A Department of Education in the course of the annual testing of


formerly identified ELL students, spring of 2010. The sample was chosen from two clusters, K12 ELL students, and K-12 non-ELL students. From the study, one grade level pulled out for


analysis, fifth grade. The measurement chosen was the State?s English Language Proficiency


Assessment (ELPA) that targets listening, speaking, reading and writing and is clustered by grade


levels. The second measurement used was the Standards-Based Achievement Assessment


(SBAA) which is computer-administered, non-adaptive, and measures academic content:


Language Arts, reading, language usage, Math, and Science.


The data analysis was based on four decision rules created for non-proficient


classifications of ELL students. The four decision rules were as follows; conjunctive geared to all


indicators at the performance standards, compensatory geared to below performance standards,


mixed allowing for two models to be combined, and complementary which takes into


consideration one or two possible performance indicators. For the classification of high-





achievers to be examined, a different set of rules was set. Researchers hypothesized that the


conjunctive model would allow for more non-proficient classifications than of the mixed model.


Then, to examine the four decision rules on the students who unqualified for redesignation,


researchers would calculate the number and percentage of students were meeting the goals set of




The research study argues that unless test developers or states under peer-review are in


agreement with the U. S. Department of Education and the decision rules and model were chosen


then there would be challenges to validity. This particular study meets the criterion for


convergent validity because of standards-based achievement performances of the ELL and nonELL students. The findings for the decision rules of non-existent high-achievers proved to be too


difficult for the present monolingual students in this study leaving questions about what


constitutes non-proficient as opposed to proficient because of the false positive in the


conjunctive decision rule. The significance of the study is for future research to ensure a more


valid and reliable process with the measurement and assessment systems that would allow for


more efficient and equitable testing.


Article Two: Classroom Assessment and English Language Learners: Teachers?


Accommodations Implementation on Routine Math and Science Tests


The article was chosen based on the challenges of ELL students with high-stakes tests.


The research explores the problem with the surge of new educational accountability measures


and initiatives. The measures brought concerns to content area teachers of ELL students because


of the evaluation of the content material taught in classrooms and on high-stakes tests. The


problem that teachers face with assessing knowledge but not changing the intended construct of


the test is of primary significance. The problem with the policy for how math and science





classroom-based assessment practices and accommodations are established are investigated in


this mixed study. The recent study strives to answer the following questions: What are the


reported accommodations implemented by ELL elementary school teachers on classroom math


and science tests? Do teachers report changes in accommodations implementation for ELLs at


the different proficiency levels? Do teachers report differently for ELLs with special needs than


for other ELLs when they implement accommodations?


The methods used will take into consideration the testing accommodations, testing


environments, and the criteria used for scoring, to present the effects of the construct-irrelevant


variance once the ELL students? linguistic barriers are removed given the same assessment


constructs of non-ELL classmates. The participants chosen were from Pennsylvania public


schools because of their high rate of ELL student population. K-6 elementary teachers across 10


Pennsylvanian school districts took a survey. Then, ten fourth grade teachers were chosen for


classroom observations and another group completed one-on-one interviews. The data collection


and analysis was based on the online surveys, semi-structured individual interviews with the ten


fourth grade teachers, and the classroom observations. For the quantitative portion of the design,


the online adaptive survey of 51 items was analyzed to learn more about the teachers? assessment


practices with the ELL students in the areas of math and science. The themes considered were


grading and teacher accommodations of ELLs resulting in how teachers feel about classroom


math and science assessment. For the qualitative portion of the design a 45 min. Semi-Structured


individual interviews of about 15 questions were led by ten fourth grade teachers. The 15


questions were divided into four sessions. The analysis was based on the assessment practices,


teacher knowledge, teacher understanding of test and accommodations, and the principles behind


the decisions made for ELL students. The data was coded to find the degree to which





professional development could help teacher beliefs, student ability to learn, academic language,


the role of language development, culture as a part of instruction, and teacher self-efficacy, and


teacher modifications in teaching practices. The interviews, coding, and analysis were digitally


recorded and transcribed as soon as they were conducted.


The findings of the research questions are as follows; for the question, if ELL students


are required to take math and science tests, most surveyed teachers always reported giving ELL


students in regular math and science classrooms the same test without taking into consideration


their proficiency levels. Other teachers, tested beginning proficiency level ELL students test less


frequently than the intermediate level ELL students, and another set of teachers exempted


beginning proficiency level ELL students from math and science test. For the question, if


changes are made to tests or test administrations to assist the ELL students, the answers were,


many teachers reported to improve tests depending on ELL students proficiency level. When the


question was asked how much additional time is given for ELL students in math and science test,


most teachers always reported providing additional time to ELL students. Teacher assistance was


also defined by the research resulting in 30% of the teachers provide support to ELL students


during tests. Accommodations led to no teachers providing interpreters or translators during


classroom-based assessments, bilingual dictionary use was limited or never reported, and or test


written in two languages never. The overall findings indicate that ELL students with special


needs classroom accommodations implementation is the custom but needs to be the custom for


ELL students in general. The implications involve more teacher professional development on


assessments, greater teacher awareness of language proficiency levels and the lack of


accommodation implementations in classrooms. The significance of the findings indicate that the


assumptions states made on classroom accommodations must be re-examined to facilitate ELL





students on their performance on high-stakes tests. The study does not mention any threats even


though, in the field of language assessment, validity is of concern when it comes to forms of


measurement, score interpretation, and when subsequent decisions have to be made based on the


assessment. The intent of the test construct was not changed though some of the linguistic


barriers were removed to assess the same constructs as the non-ELL students.


Article Three: Reading, Writing, and Learning English in an American High School




The research problem in this microenthographic study examines the struggles teachers


face with state?s standardized curriculum to support multilingual students within an academic


school year. Teachers? perception is that standardized curriculum only focuses on standardized


tests and not on the individual needs of the multilingual students. Multilingual students in middle


schools focus on genres that are directly aligned with the standardized test and are normally


shorter passages than what is expected in high school. The research questions targeted in this


study focuses on how do high school English language arts classrooms with multilingual learners


in reading and writing instruction use the text-based standards-driven curriculum. The second


question focuses on teachers? interpretation of the standards-driven curriculum requirements and


if multilingual students are supported or limited to opportunities for learning academic reading


and writing. The purpose of the study is to examine effects the educational policies on reading


and writing instruction for 9th-grade multilingual learners and how the intended policies


determine the ideologies and interpretations of teacher practices.


The data included 26 days of field notes and observations through an audio-recorded


interview with the teacher, printed data sources from the primary textbook, Holt English 9


curriculum, pacing guide, other resource and supplemental materials used along with the school


district?s teacher handbook for English 9th grade and benchmark assignments.





The data was analyzed by using the micro-ethnographic discourse analysis (MEDA) in


three sessions. The analysis alternated between close examination of spoken and written


discourse by teachers and students in the classroom and the discourse created through larger


events that make up a particular lesson. MEDA identifies multiple levels of analysis. On the


other hand, another analysis was conducted through the transcribing of audio recordings and


indexing key events in the classroom. Frequency tables demonstrated the much use of the


different textbooks, some use, and little use of the textbooks. Threats to validity were not directly


stated but because it was a microethnographic study and only one participant was observed the


objectivity could be threatened. On the other hand, it is a study that can be replicated on a larger


scale, so the assumption is that the threats are minimal.


The findings revealed that policy requirements forced multilingual students in reading


and writing 9th-grade instruction to focus primarily on a task that prepared students for highstakes tests. The implications of this study are whether or not students learned enough to meet


the benchmark assignments to demonstrate then proficiency on the year-end goals of the


standardized assessment. The assumption is that skilled teachers may determine how to teach


curricular programs to multilingual students. The recommendation for future research is to


analyze the new standards and curricular materials and how teachers are using, adapting, and


differentiating instruction for multilingual students.


Article Four: Formative, Informative, and Summative Assessment: The Relationship


Among Curriculum-Based Measurement of Reading, Classroom Engagement, and Reading




In Quebec, classroom teachers are required to administer and correct high school exit


exams. The high school exit exams are developed by the New Ministry of Education, Leisure and


Sports (MELS) program. Classroom teachers are also put in the position of participating in the





development and validation process of aligning classroom-based assessments along with a


writing exam. The problem is classroom teachers, and the large-scale of test developers confront


external standards and requirement issues of validity. For teachers to develop classroom-based


assessments, teachers must know their students and focus the assessment on what has been


taught in the classroom. Working with a large-scale test development and with the classroom


context to consider, the factors caused uncontrolled variables to affect the outcomes of the


assessments. The purpose of this research study is to align context and pedagogy to high-stakes


English as a Second Language testing. The questions considered refer to how the rater training


affect the teachers? use of the marking rubric and how the teachers participation in the rater


training perceive its benefits to teaching, learning, and assessing students in the classroom. In


Quebec, 50% of the total grade is taken from the exit exams, and the other 50% is taken from the


classroom-based assessments.


The study employed a convergent mixed method design to examine how secondary


teachers? rate training facilitated the alignment of the MELS-expected standards. The data was


collected through four phases. The first phase was marking and recording students? scores. The


second phase was sharing students? papers. In phase three, teacher rater training took place at the


same time the quantitative and qualitative data was collected. During phase four, the rater


training on scoring behaviors of both testing periods took center stage. The results for the first


question revealed rater behavior across the year 2010 and 2011 led to changes in the rubric.


Descriptive statistics and correlations were used by looking at means and standard deviations for


the correlations between the variables of the engagement-student report, the engagement-teacher


report, reading competence, and reading performance. A Chi-square analysis and Factor Analysis





M-Plus was conducted to evaluate if the missing data differed among demographic




The results for the second question indicate that the classroom teachers training had a


positive impact in their classrooms and their understanding of writing exams and rubric


expectations improved. The findings offered a clear perspective on the importance of


communication and cooperation among test developers, teacher raters and classroom teachers in


understanding the expectations of the new writing assessments and rubrics. The significance of


the study was to help improve the rigor, alignment, and qualities of the rubric descriptors. The


underlying principles allowed for test validity, benefiting teachers and students in teaching and




Article Five: The Impact of Communication and Collaboration Between Test Developers


and Teachers on a High-Stakes ESL Exam: Aligning External Assessment and Classroom




The research on formative assessment and student reading achievement has been


investigated for much time now, with the difference that this study examines the problem of


curriculum-based measurement and reading (CBM-R) associated with social, behavioral, and or


emotional aspects because it has not been fully explored. The purpose is to build a relationship


between CBM-R, student engagement, reading competence, and performance in the general


education classrooms of third to fifth-grade students. The research questions focus on which


formative assessments of reading facilitates learning through engagement and which tools can


assist in the decision making of appropriate instruction and intervention.


Teachers and students were selected from six schools in high-risk communities, and four


were Title I schools. The students were given parental consent forms to participate. The data


analysis plan only included information for the students? whose teachers contributed to the





survey from the fall and spring data. The instrumentation chosen was teacher report of student


engagement, student report of engagement, reading competence using CBM-R measurements,


and reading performance from state criterion referenced tests. The data was analyzed using


several descriptive statistics to examine central tendencies, variability, and distributions of the


study variables. A range of multivariate analyses was conducted to examine the effects of topic


choice on writing task scores, the Rasch analyses were performed using the FACETS program,


and the MELS was used to evaluate the ratings and the content mainly involving the coding for


theme development.


The results indicate that students with lower reading competence who were engaged


during reading assessments performed better on the summative assessments. On the contrary,


students with higher reading ability were not influenced by the engagement. The findings reveal


motivation as an essential component of student reading achievement in CBM-R assessments. Its


limitations, however are, based on the threats of validity found in the technical qualities of the


CBM-R measurement equivalence related to the different subgroups, left unanswered questions.


The implications and further research support a growing need for the literature of CBM-R for


school use and how it relates to extra-academic factors for student achievement. The study?s


significance is the continued investigation of whether or how engagement can be adjusted or


influenced as an indicator of recent or distal performance.





Part II


Article Five: The Impact of Communication and Collaboration Between Test Developers


and Teachers on a High-Stakes ESL Exam: Aligning External Assessment and Classroom




Research Redefined


The redefined research study considered is based on the need of English Language


Learners in St. Croix, United States Virgin Islands (USVI) taking and meeting the standards on


high-stakes tests. The impact of communication and collaboration among all test developers and


teachers alike may assist with the alignment of assessments.


Revised Research Questions


The study would ask the following research questions concerning the need to link context


pedagogy in lesson planning and design in preparation for ESL students taking high-stakes tests.


Question 1. What literacy assignments did the ESL and content area teachers use in the


classroom, and how were they linked to the standard-based assessments? Question 2. What type


of literacy test preparation materials did the ESL and content area teachers choose, adjust, or


construct for classroom use in order to prepare ESL students for the test? Question 3. What were


the benefits of using the literacy test preparation materials to the teaching, learning, and


assessments process of the ESL and content area classrooms?


Revised Research Purpose


The revised purpose focused around secondary school ESL and content area teachers?


alignment of classroom-based literacy assessments and materials and high-stakes tests. The study


is as a result of concerns with ESL students not meeting the standards set forth by high-stakes


tests. The purpose is to examine how teacher training on the alignment of classroom-based


assessments and materials and high-stakes tests.


Revised Research Design





The USVI public schools, where this revised research study would possibly to take place,


has an increase of Hispanics/ESL population. Many of the immigrants are coming from the


Dominican Republic. There is a need to robust the ESL program and materials for the ESL


students to meet the standards set forth by the high-stakes tests at the end of the year. The schools


included in this revised study are the public St. Croix schools with a high ESL population,


schools with free and reduce lunch, and schools that did not attain reached yearly progress


standard goals. Surveys would be given to school administrators to determine eligibility. The


data will be analyzed using several descriptive statistics to examine the constructs of external


assessment development contexts, ELL test-takers, and classroom context. They will be


measured through questions, survey, interview, observations and pre and post testing data. To


determine how to operationalize the constructs, I will define variables into measurable factors


and develop specific procedures.


The research design chosen is a mixed method design because it will best answer the


research questions. The quantitative and qualitative data synthesis would take place at the end of


the data analysis process after all testing and training have...


Paper#9210853 | Written in 27-Jul-2016

Price : $22