The Road to RAMP: Data Mining

This is the second in a series of reflections on the RAMP process.

A few weeks ago I was teaching a class of new counselors and one of the topics was "data."  What are the different types of data that school counselors use?  Why do school counselors need to use data in their practice?  Where do you start?

I've learned over the years that school counselors, new or otherwise, are all at very different places with this topic.  Our comfort level with finding, using, and analyzing data seems to be dependent on a wide variety of factors, such as when you were in graduate school, the specific graduate program that you were in, the comfort level of your professors with data-driven practice, and the professional development that you received after graduation supporting the use of data in your counseling program.  Yet, if you are at all interested in working towards RAMP (Recognized ASCA Model Program) certification, it is a world with which you will need to become familiar.  Quickly.

As I wrote about in a prior post, at my school we have really been working towards RAMP for several years.  A large part of that process has been spent in developing a comfort level, as a team, with data.  What I am finding, though, with the RAMP process, is that we are really being forced to go deeper into our data, both to justify our interventions and define our goals (a topic for a later post), as well as to analyze the effectiveness of our programming.  This process is really helping us to tie together our mission, vision, and beliefs with what we do and how we do it, as well as making us ask the question as to what impact all of our programs are having on academic achievement, post-secondary outcomes, attendance, and school safety.  Where, though, do you begin?

First, it is important to know the three types of data that we will work with:

• Process Data:  Process data literally tells us two things--who is effected by the intervention and what did they do?  Thus, 97% of 9th graders wrote a post-secondary goal.  320 seniors took part in a lesson about dating violence.  56 parents participated in an evening program that focused on supporting students' study-skills at home.  This type of data is a place to start--it is helpful in that it can help to show people that you are taking an active role in the school community and are working with students and parents, oftentimes in large groups.  If you are in a situation where you have concerns that your community is unsure of what it is that you do or have the impression that you don't do much, this type of data can help to inform them of your role.  However, it is lacking in that it does not demonstrate the change or impact that your interventions may have.
• Perception Data:  Perception data comes in a variety of shapes and sizes.  Basically, you are getting a snapshot of people's thoughts, or "perceptions," on a program at a given time.  One way that you might use perception data before a program is to do a needs assessment to see what the perception of your students, parents, teachers, administration, or any combination thereof might be with regards to what they need from you.  For example, if you are trying to determine what groups might be the best use of your time in your elementary school, you can develop a brief questionnaire that asks students to pick their top two out of several options, and maybe follow up with a questionnaire to the teachers and parents, as well.  In this way, you are making decisions about how to utilize your time and resources that reflect what your school community needs, not just what you think they need or what you like to do.  Thus, you may learn that 70% of your students want groups on study-skills and that this is supported with 80% of teachers and parents, versus only 2% of students, teachers, and parents that feel a group on social-skills would be beneficial.  Now, this does not necessarily mean that  you will throw a social-skills group out the window, but it would warrant looking into it further and finding additional sources of data that might support the need.  We also use perception data quite often to see what change has occurred in students feelings, thoughts, or knowledge as a result of our interventions.  Often this is done through either a pre and post test combination, or simply a post test.  These can come in many forms, but asked in developmentally appropriate ways (a choice of two pictures, such as a smiley face and a frowny face for younger students versus a five-point Likert scales for older students) how someone believes they have changed because of your lesson, group, or program.  Thus, there was a 50% increase in 10th graders knowledge of career options as a result of your lesson.  After your lesson on bullying and harassment, 98% of 8th graders could correctly identify the bully, the victim, and the bystander and their roles in the bullying cycle.  Thus, perception data is more powerful than process data in that it can demonstrate change and growth as a result of your program.  However, it is subject to self-reporting, which is not always the most reliable, and does not address the direct impact that the program  has on the academic process.
• Outcome Data: This data was previously known as "results" data, so if you see that term used in prior publications, articles, or entries, realize that they are referring to the same general idea.  Outcome data refers to academic information such as grades and test scores, but also enrollment in rigorous courses and post-secondary planning outcomes.  It also refers to attendance and school-safety.  This is the most powerful data that we can use, as these are the hard numbers that administrators, communities, and state and national entities are tracking.  If you can show that through your intervention grades have improved, test-scores have gone up, enrollment in advanced coursework has increased, more students are applying and being admitted to two and four year colleges, students are coming to school more, or disciplinary incidents have decreased, you are demonstrating that your program is having a direct effect on the academic process, success, and "outcome" of the students in your community.  80% of students who participated in the year-long "Student Success" program saw an increase of .5 or better in their GPA.  Students with five or more unexcused absences that participated in the student-parent program targeting school engagement saw a 65% drop in truancy after two months.  As a result of a school-wide, comprehensive bullying and harassment campaign led by the school-counselors, reported incidents of bullying and harassment were down by 50% from the previous year.  Ultimately, we want as many of our programs to show a change in this data category, as it speak to the true power that school counselors can have on the success of our students.

For more in depth explanation of this information, refer to the ASCA National Model, 3rd Edition.  

So, why do school counselors need to speak this language and use data?  I have been involved in many conversations over the years with practitioners who state that finding data, developing instruments to measure interventions, and analyzing and presenting data takes time away from our direct work with students, which is what we have been trained to do and which is what is most beneficial to students and communities.  There are those that feel that the data-driven movement has taken the humanity out of our profession.  My counter to this is that first, we need data to better make use of our limited time. We all have high case-loads, and some of us in various parts of the country have extremely high case-loads.  By looking at information such as grades and test-scores, broken down by subgroups, as well as needs assessments, we can better target our time with those who may benefit the most from more direct interventions.  Secondly, we need to invest our time and resources on programs that have an impact.  This scenario happens all the time:  You have a speaker come in.  You think the speaker is wonderful.  Other adults think the speaker is wonderful.  No one ever asks the kids what they learned from the speaker, and no one ever follows up to look at academic, attendance, or school-safety data to see if there was any strong, long term impact from the speaker.  Are you really getting the bang for your buck with this intervention, especially if you have taken away from instructional time and spent money to bring this speaker in?  By using data to measure the effectiveness of programs, either with perception data, outcome data, or both, you will quickly determine which programs work and which do not.  This empowers you to justify cutting some things, changing some programs, and developing others based on the true needs of your school.  It also will allow you to ultimately spend more time with students if you are able to do away with some things that have no impact whatsoever on your school community.  Lastly, we need to collect and use data in order to promote, defend, and expand our profession.  As the latest 2012 National Survey of School Counselors from the National Office of School Counselor Advocacy (NOSCA) tells us, both school counselors and school principals believe that school counselors are ready to take the lead in helping develop students who are college and career ready and chart them on a path to strong post-secondary options.  However, for better or for worse, we need to show our local, state, and national communities just how strong an impact that we are able to have on the academic achievement and outcomes of our students in order to receive the necessary resources and reasonable case-loads that will allow us to do this important work.

So, where do you start?

• Examine data sources that are already available.  Most schools across the country have public "report cards" that breakdown testing data by subgroups.  This can give you an idea pretty quickly of where students are and are not doing well, academically.  You can take this information to your team, administration, or advisory council to begin to develop a program that can address this.  Additionally, look at student grades, graduation rates, attendance, and school-safety data.  This information alone should help to give you some ideas of where there are needs for your services within your building.
• Consider doing a needs assessment.  Take a deep breath--this does not mean do a needs assessment of every student, teacher, or parent in your building.  Target a small population--pick a grade level, do a random sampling, or look at a subgroup that you've found based on information you've gleaned from the previous bullet-point.  In this way, before you develop a program, you can get a better handle on what those students need most.  You may think that your group targeting 7th grade attendance needs help with study-skills, but your needs assessment of them prior to starting may point you in a different direction.
• Find ways to collect data on your programs.  If this is overwhelming, start with one or two, and over the course of a few years add more until you are examining your entire program.  It is important to begin to ask yourself and your team what changes in perception and student outcomes are occurring as a result of your interventions.  You can use paper-pencil pre and post tests or you can use online tools like Survey Monkey and Naviance to gather perception data, and you can go into your school's database to gather the outcome data.  Once you start taking a look at some of the programs that may have been in place for years, you may be surprised by the results.  This can begin some important conversations within your school as to how to best utilize your time and resources.  Some of them may be difficult conversations, but with data you are always able to bring the discussion back to student success and outcomes.
• Share your data.  Regardless of what the data shows, whether effective or ineffective, share your data with stakeholders.  This can be a great thing to do during your advisory council meetings, but you should also feel free to share it with teachers, administrators, and parents as appropriate.  This can be done informally through e-mails, presentations, or newsletters, or you can develop more formalized reports.

If these steps seem like they are just scratching the surface and you are looking for more information on how to compile data and make more formalized DATA reports, I would highly recommend taking a look at Making DATA Work by Dr. Anita Young and Dr. Carol Kaffenberger and available through the ASCA Bookstore.  This book will go into much more detail and give you step-by-step instructions on compiling data, developing instruments to measure programs, and writing a DATA report to share with your stakeholders.  It also includes samples for you to look at that can serve as models.  

Once you begin this process and develop a comfort level with data, you will not only be on your way to RAMP, but you will have begun to find ways to maximize your time with students as well as demonstrate just how powerful the role of a school counselor can be within a school and community.