During the course of Evidence based Leading for Learning @ACUeducandarts #EDLE685 students have been invited to design a problem based project. The problem or concern is situated In their learning communities. Their first step is to identify a problem and provide evidence it is a problem! The problem needs to relate to student learning. Given this is a leadership unit students also need to frame the problem from a leadership (L) perspective. The L perspective is in the planning, implementation and evaluation of the project. In this blog students are invited to share their evaluation techniques that they will employ.
Students: In the comment section post your framework/model/figure (FMF) to demonstrate how you will evaluate your evidence based project. Name the FMF and also you are invited to write a few sentences about it. Remember to acknowledge sources.
Leadership Threads 
Program Evaluation Tool from Claire Nailon
Source: Bernhardt, Victoria L. (2018). Data analysis for continuous school improvement (4th Edition). Routledge: New York, p. 256.
The table that encapsulates Bernhardt’s program evaluation tool is a great way of thinking through a program that is being used to solve a problem. Sadly I cannot work out how to post it here! I like the emphasis on integrity and fidelity referred to in the implementation section. The only thing not made explicit in this tool is the importance of conversations about the findings. Earl and Timperley (2008, p. 126) explain that “inquiry-based conversations are pivotal to creating the shared meanings that form the basis of these actions.” Furthermore, Marsh and Farrell (2015, p. 278) outline core practices that will assist in building teacher capacity to use data for instructional improvement and therefore assist with evaluation of data. These are assessing teacher needs, modelling, observing, providing feedback and sharing expertise, dialogue and questioning and brokering. Again, the importance of dialogue and questioning in conversations about data is emphasised.
References
Bernhardt, Victoria L. (2018). Data analysis for continuous school improvement (4th Edition). Routledge: New York.
Marsh, J., & Farrell, C. (2015). How leaders can support teachers with data-driven decision making. Educational Management Administration & Leadership, 43(2), 269-289. Accessed on 15/10/18 at http://ezproxy.acu.edu.au/login?url=http://journals.sagepub.com/doi/abs/10.1177/1741143214537229
Earl, Lorna M, & Timperley, Helen. (2008). Professional Learning Conversations Challenges in Using Evidence for Improvement Professional Learning and Development in Schools and Higher Education, 1. Dordrecht: Springer. Accessed on 15/10/18 at http://ezproxy.acu.edu.au/login?url=https://ebookcentral.proquest.com/lib/acu/detail.action?docID=372585#
LikeLiked by 1 person
Thank you Claire for this comprehensive post. You picked up the importance of conversations about data. In some ways conversations are integral to moving data to evidence.
LikeLike
The learning issue that I addressed in Assignment 2 required teachers and leaders to work towards a common goal and adopt a shared responsibility for improving student reading comprehension skills. The effective implementation of this plan would require collaboration, observations and honest feedback. Therefore, the action plan that I created must encourage an atmosphere of collaboration, shared responsibility, common goals, honesty and trust.
In this light, the questions I developed for evaluation are based on those in the Decision Making Index for Principals and Teachers as explained by Mulford et al. (2008).
Evaluation Questions for the Evidence-Based Analysis and Action Plan
• How does this action plan promote a shared and coherent sense of vision?
• What collaborative practices are evident in the decision making process?
• How does this plan distribute leadership among staff?
• What structures does this plan put in place to support teacher initiative, experimentation and change?
References:
Mulford, B, Kendall, L, Kendall, D, Edmunds, B, Ewington, J, & Silins, H. (2008). Successful School Principalship and Decision Making. Leading and Managing, 14(1), 60-71.
LikeLiked by 1 person
Susannah, it is great how you have adapted your evaluative tools from Mulford et al (2008).
LikeLike
Lovely posts thank you
LikeLike
The problem I addressed in my Assignment was that of Professional Learning for staff. We as a Literacy team have started the journey of reimagining the whole staff approach to Teacher Learning meetings and are moving toward a PLC approach. I used Lynn Sharratt’s 14 parameters as a guide for this leadership initiative. (I have tried to copy and paste my table and framework model but unfortunately it won’t allow.) I have a self created framework where Relational trust, Culture of Collaboration and Individualised learning feed into …teacher efficacy. The premise of this was derived from Vivian Robinson (2011).
Robinson, V. (2011). Student-centered leadership (Vol. 15). John Wiley & Sons.
Sharratt, L., & Harild, G. (2014). Good to Great to Innovate: Recalculating the route to career readiness, K-12+. Corwin Press.
LikeLiked by 1 person
Julianne I am impressed to read how you created your own framework for your school’s PLC approach. The framework being steeped with guides (and evidence based studies) of Sharrat and Robinson will really ground the framework. Thanks for this post
LikeLiked by 1 person
In Assessment Two, I analysed a problem using a real context in our learning community. The concern identified is the lack of teacher engagement with data, to maximise their students outcomes and improve their teaching practices so that their students are learning. For instance, using NAPLAN data to guide their teaching practice to improve student’s writing skills. In addition, alarmingly there is a noticeable downward trend in the NAPLAN data specifically in literacy, in both narrative and persuasive writing skills for the past three years, yet there is no evidence that this an area of that
Furthermore, the collaborative inquiry cycle will be utilised to guide the evidence-based action and identity the tools and methods required to find out as much as possible about this concern
Sharratt, L. & Fullan, M. (2012). Putting Faces on the Data. What great leaders do! United States of America: Corwin.
LikeLiked by 1 person
Sarah,
I am interested when you identify the tools to evaluate this project. I note that several students are employing scaffolds that they already know – such as the inquiry cycle.
Sharrat’s putting faces on data tells as a good deal about what teachers value in schools in their relationship with students.
LikeLiked by 1 person
Hi Sarah,
I agree that teachers struggle with analysing data so readily available despite it being so powerful. As Sharrat explains, this would certainly help teachers to visualise the data in terms of student growth over the years. I believe the teacher’s day is often overloaded and unless we specifically timetable a data period, use of this valuable information which could inform teaching practice is last on the ‘to do’ list and possibly too late.
LikeLike
Students are not engaging with the feedback that is being provided by their teacher (Donovan, 2014). The mediating factors for feedback engagement are the feedback practice of the teacher and the personal attributes of the student (Brookhart, 2008; Pitt & Norton, 2017). Further, an accompanying grade or score – very low or very high – could severely lower the likelihood that the student will engage with feedback (Black, & Wiliam, 1998; Vancouver, Thompson, Tischner, Putka, & Murphy, Kevin, 2002; Sendziuk, 2010). Although it has been significantly researched that the removal of grades would increase motivation, engagement with feedback, social collaboration and academic attainment, the practice of grading accompanying feedback remains (Butler, & Nisan, 1986; Black, & Wiliam, 1998; Lipnevich, & Smith, 2008; Hayek, Toma, Oberlé, & Butera, 2015). Overall, I propose the complete removal of grades and marks from the education system.
To evaluate the removal of grades and marks, multiple data sources should be utilised. A questionnaire to students, parents and teachers in addition to interviews would be useful. Further, quantitative data comparing classes with academic achievement over a number of years would also be useful.
(if you want the resources just let me know :))
LikeLiked by 1 person
In my assessment I decided to focus on the problem of meaningful technology integration into all learning areas. The reason I chose to focus on this is that it is a problematic topic in my learning community and deals with the dilemma faced by leadership of meeting the needs of a highly technological workplace, whilst still meeting expected levels of results in standardised testing.
Based on this to evaluate the actions of leadership the following questions were developed –
1 – What evidence is there to suggest that technology driven innovation is not occuring in classrooms?
2 – Why do some staff members not utilise the technology in their classrooms?
3 – How do students feel about their learning incorporating the iPads and related technologies?
4 – If there is a staff resistance then what needs are not being addressed?
5 – How does this resistance affect student learning experiences?
6 – What are the key things staff need more assistance with?
7 – How can staff be assisted in the most effective manners?
8 – Will any changes in the learning style affect students outcomes?
9 – How can skills be meaningfully taught to be relevant for all subject areas?
10 – What is the likelihood that significant change will occur across all classrooms?
11 – Has adequate time been allocated for sustained changes to occur?
The importance of collaboration and communication were highlighted as paramount in the success of any changes.
LikeLiked by 1 person
In using a tool or process to discern about a learning problem, data is always important. Sometimes we believe we have a problem in an particular area, but when we examine the issue using data and a critical mindset we discover that the issue is actually different or something else. A process to use is:
1. Definition of the problem – what is the problem and how do you know that its a problem?
2. What does the data tell us about the issue. The type of data collected will depend on the problem.
3. Who are the stakeholder and are they involved in the examination and the solutions for the problem?
4. What is desired as an outcome? What do we want the problem to dress sot hat it is no longer an issue?
5. Collaboration between all stakeholders to create a program to address the issue ( what is the problem, what are we going to do about it, when and for how long will the program run, and what evidence will exist that we have been successful?
6. Implementation and maintenance of the program.
7. Evaluation of the results from the program.
Mulford, B, Kendall, L, Kendall, D, Edmunds, B, Ewington, J, & Silins, H. (2008). Successful School Principalship and Decision Making. Leading and Managing, 14(1), 60-71.
LikeLiked by 1 person
In Assignment 2, I focussed on an issue relevant to my context, teaching and leading in the Norther Territory, that is the challenge of identifying and supporting EAL/D learners. There has been system wide (Catholic Education) recognition of a failure to adequately identify EAL/D learners through processes such as enrolment. Within my own school, teachers and administration acknowledge that our data is not comprehensive. Teachers have also expressed limited ability in identifying and providing appropriate instructional support to EAL/D learners.
The framework I applied to addressing the problem was the Whole-of-school Approach to ESL Instruction (Department of Education Northern Territory). I chose this framework over other school improvement frameworks as it relates specifically to improving ESL instruction in schools. The framework moves from a school organisational level to the classroom, professional development in ESL pedagogy and finally monitoring of the effectiveness of interventions.
Department of Education (2015). Guidelines: English as a second language.
Retrieved from https://education.nt.gov.au/__data/assets/pdf_file/0009/274671/English-as-a-Second-Language-guidelines.pdf
LikeLiked by 1 person
The problem i addressed in assessment 2 was the poor level of communication in secondary students who leave school via the VCAL pathway. This study analysed students entering VCAL and strategies/professional development that enabled leaders to better use evidence to have a greater effect within the class. During my research i utilised data from Emmanuel College, VCAL and vocational industry as these are all linked to a secondary student completing their VCAL qualification.
From the research, Hattie’s (2013) effect size confirmed the difficult environment that leaders and teachers have to work within VCAL. With reports noting that students were;
– disengaged
– well-being concerns
– negative about school
This issues, all had a negative effect size response via Hattie vs Cohen notation of effect size SD. This lead to the professional development and tools introduced to leaders at Emmanuel College created by Agile Sprints. These tools enabled leaders to assist their team in collecting evidence and work collaboratively to achieve goals set in teaching and learning. Once evidence was collected, this was discussed with the team to allow the teacher to review what they did, reflect on its success and review the actions if change is needed (Breakspear 2018).
REFERENCES
Breakspear, S. (2018, May 6th). http://www.agileschools.com/learning-sprints/. Retrieved from Agile Sprints: http://www.agileschools.com/learning-sprints/
Hattie, J. (2013). Visible Learning for Teachers: Maximizing Impact on Learning. London: Sage Publications.
LikeLiked by 1 person
The issue identified as a priority from leadership and staff centres around implementing a whole school behaviour management framework that support student learning and staff well-being. In light of the complex nature of behaviour and culture within a school the action plan is to work on collaboration, strategic planning, professional development and a whole school and community (parents and students) working toward improved results in behaviour. This action plan will be based on positive behaviour learning (PBL) and positive behaviour interventions and support (PBIS) goals and strategies to create an autonomous and evidence-based practice in relation to behaviour frameworks.
Evaluating the effectiveness of the action plan according to Marsh and Farrell (2015) revolves around assessing needs, modelling, observing, providing feedback and sharing expertise. Mulford et al. (2008) also states in evaluating an action plan one needs to consider management styles, shared vision, flexibility and teacher empowerment as some of the important elements to reflect an effective plan or implementation of an issue or concern.
Based on the studies of evaluation on action plan I would ask the following questions:
1. How does this action plan engage all staff to a shared goal?
2. What support is provided to staff to initiate, experiment and adapt framework?
3. How does leadership plan to empower other staff and facilitate collaborative practice?
Marsh, J., & Farrell, C. (2015). How leaders can support teachers with data-driven decision making. Educational Management Administration & Leadership, 43(2), 269-289. Accessed on 15/10/18 at http://ezproxy.acu.edu.au/login?url=http://journals.sagepub.com/doi/abs/10.1177/1741143214537229
Mulford, B, Kendall, L, Kendall, D, Edmunds, B, Ewington, J, & Silins, H. (2008). Successful School Principalship and Decision Making. Leading and Managing, 14(1), 60-71.
LikeLiked by 1 person
Assessment of actions derived from evidence
In regards to using a framework or model to assist in evaluating the implementation of a learning and teaching program, Mulford et al’s Decision Making Index (DMI) could be a very useful tool for measuring the quality of evidence-based improvements and initiatives.
The DMI focuses on the key areas around the effectiveness of evidence-based improvements, that I feel are beneficial for implementing effective change. The index assesses the effectiveness of the collegial, collaborative, cooperative and consultative aspects of evidence-based action.
In gaining feedback from my leadership colleagues on the analysis and proposed action in regards to the review of the structure of our Year 9 curriculum, I developed the following evaluating questions:
Did the methods of data collection and evidence, guide the decision-making process?
Were all members of the school community consulted in the review of this decision-making process?
Was this process reflective of shared/distributed leadership?
What structures have been established through the leadership of this process to support this change?
Mulford, B, Kendall, L, Kendall, D, Edmunds, B, Ewington, J, & Silins, H. (2008). Successful School Principalship and Decision Making. Leading and Managing, 14(1), 60-71.
LikeLiked by 1 person
In my analysis I’ve been looking at how leaders can engage staff when change is being imposed at a systemic level. I’ve developed some evaluation questions based on Caldwell and Spinks 1992 refined model of self management (in Caldwell 2006, p20). The model can be seen here: https://imgur.com/a/00TKOUc
I see a connection with my issue in that the ‘top level’ of the model in setting charter, policies, and the 3-5 year plan can be analogous to the priority setting by the system. So I’ve developed some guiding questions that can be used at a school level to cover the cycle in the lower level of the diagram.
Does this plan meet school priorities?
How will this plan impact curriculum design?
How will this plan impact teacher practice?
Is resource allocation appropriate? Are changes needed?
What will be the impact on learning and teaching in classrooms?
Evaluation and review phase: Are we happy with the proposed changes, or do we need to revisit and come up with a different approach?
Reference:
Caldwell, B. (2006) Australian Council for Educational Research, & Caldwell, Brian J. (2006). Re-imagining educational leadership. Camberwell, Vic.: ACER Press.
LikeLiked by 1 person
In my assignment I will be addressing the concern of Student Well-being. Some students have voiced that they don’t feel like they belong at the school and that they arent enjoying school. I have proposed that the school use the PERMA approach to help build well-being across the school community.
While researching I found that there is a well-being measurement tool available which has a 8-questions scale. This Flourishing Scale measures the perceived success in relationships, self-esteem, purpose and optimism.
Also, when the following questions are asked, the projected insights would be received around the strength and validity of the proposed action towards improving well-being in the school community.
Are there enough forms of evidence collected?
Does the evidence-based practice meet the needs of the School?
Are the recommendations strong enough to minimise the risk of implications and their consequences?
Diener, E., Wirtz, D., Tov, W., Kim-Prieto, C., Choi, D., Oishi, S., & Biswas-Diener, R. (2009). New measures of well-being: Flourishing and positive and negative feelings. Social Indicators Research, 39, 247-266.
Seligman, M. (2011). Flourish: A new understanding of happiness and wellbeing – And how to achieve them. London: Nicholas Brealey Publishing.
LikeLike