Completing Ratings Frequently Asked Questions
Table of Contents
-
Q: How can I access ratings for a previously removed student?
-
Q: I accidentally selected a wrong answer and submitted the rating. How can I go back and fix it?
-
Q: Do the various DESSA response options on the Likert scale need to be defined?
-
Q: Program Admin - How can I edit the name of a rating window?
-
Q: What is the recommended amount of time between pre- and post-assessment ratings?
Overview
This article contains a list of frequently asked questions about completing ratings in the DESSA System.
Questions
Q: How can I access ratings for a previously removed student?
A: If a student has been removed from the system, educators will no longer be able to access the student's profile or past ratings. Site Leaders and Program Admins can access the student's past data through the Rating Export, if the student had a rating that fell within that specific time period. If a student needs to be re-activated, a Site Leader or Program Admin can add the student back into the system using the same Student ID that was used for the student in the past.
For more information on running the Rating Exports, please see the following article: How can I access ratings for a previously removed student?
Q: Can I skip questions when rating students?
A: A limited number of questions can be skipped on some forms, but only as follows:
- DESSA: There can be no more than three (3) items left blank on the entire DESSA. There can be no more than one (1) item left blank on any individual scale.
- DESSA-SSE: There can be no more than two (2) items left blank on the entire DESSA-SSE: There can be no more than one (1) item left blank on any individual scale.
- DESSA-HSE: There can be no more than two (2) items left blank on the entire DESSA-HSE: There can be no more than one (1) item left blank on any individual scale.
- DESSA-HSE SSR: unable to skip
- DESSA-mini forms: unable to skip
See this article for suggestions about what to do if you feel you are unable to answer all of the questions.
Q: I accidentally selected a wrong answer and submitted the rating. How can I go back and fix it?
A: Unfortunately, educators do not have the ability to correct or edit any past ratings. However, they may contact their Site Leader or Program Administrator, who can delete the incorrect rating. Then, the educator can submit a correct rating for the student.
Q: Do the various DESSA response options on the Likert scale need to be defined?
A: The various options on the Likert scale do not need to be defined for the rater. When individuals complete the assessments, they are reflecting back on a child’s behavior over the past four weeks and selecting the option based on their general impression. Providing a definition of the options would not necessarily make the ratings any more accurate, given that raters will ultimately base their rating on their recollections and general impression.
For more information on how to use the various DESSA response options on the Likert scale, see this article: Can you define the DESSA response options (e.g., Rarely, Frequently) for raters?
Q: I cannot answer some of the items. What should I do?
A: Here are recommendations when raters are unable to answer some items:
- Provide raters additional time to observe the student(s).
- Provide raters with the PDF version of the rating form so they are familiar with the behaviors. This may cue them to look for those specific behaviors when working with students.
- Create opportunities to observe behaviors. For example, a math teacher who typically instructs through lectures may want to create a group assignment to observe relationship skills.
- In a group with other educators, discuss what the different behaviors might look like in their classes.
Q: Can we fill out the 72-item full-DESSA as a grade-level team together or would this impact the validity of the assessment?
A: It is not recommended that your team fill out the DESSA together. Instead, we recommend having the person who knows the child best complete the DESSA. Another alternative would be to have a few team members who know the child well complete the DESSA individually and then discuss the results. The reason for this is that there is great value in getting the unique perspective of each rater. It is possible that the child’s behavior may be quite different in the presence of different people, so their individual ratings would reflect these differences and provide richer and more useful information to the team. Additionally, it’s possible that if the DESSA is filled out as a group, one person who sees the child’s behavior differently may be likely to go along with the rest of group, rather than voice their unique perspective.
Q: What programs have the best capacity to incorporate the DESSA when considering retention and program dosage?
A: Many factors play a role in a program’s ability to incorporate the DESSA as a planning tool. First, a program should have manageable staff-child ratios. Second, it will be easier to incorporate the DESSA in programs that provide a variety of opportunities for children to demonstrate social and emotional skills. Third, programs that have experienced and trained staff members that are already mindful of children’s skills and interactions may have a greater capacity to incorporate the DESSA. In terms of retention, programs with lower staff turnover would be better able to provide ongoing support to children planned for with the DESSA. Finally, we believe that program dosage would not impact a program’s ability to incorporate the DESSA; however, dosage may impact the outcome of any strategies or interventions planned.
Q: How well do you need to know the child to complete the DESSA or the DESSA-mini? Does this differ among various types of programs?
A: A rater must have sufficient opportunity to observe the child’s behavior over the four weeks prior to completing the DESSA. Therefore, we recommend that raters should have contact with the child for two or more hours for at least three days per week for a four-week period. This translates to approximately 24 hours of exposure to the child. This guideline was determined through feedback from teachers during the development of the DESSA.
- It is important to keep in mind that although this is the recommendation, it is only an estimate. Especially in after-school settings, one must be aware of the many factors that play a role in a rater’s exposure to a child, such as staff-child ratios, types of interactions in the program (e.g., help with homework, large or small group activities, free play, etc.). Therefore, a rater who has less exposure than recommended in the manual may still know a child well enough to complete the DESSA and can do so accurately, depending on these various factors.
- Because the scores are based on the number of times specific behaviors have been noted, a rater’s insufficient opportunity to observe the child could lead to an erroneously low rating.
Q: What factors impact the integrity of the data? For example, if a staff member or parent is having a difficult day with the child, does this impact the rating?
A: Because the DESSA and other parent, teacher, or staff report measures are based on the perspective of the rater, it is expected that responses may be impacted by the experiences a rater has with a child. To reduce this rater influence, it is important to carefully explain the purpose of the rating and how it will be used. It is also important to review the general administrative guidelines with raters.
Q: Should teachers and staff administer the DESSA or DESSA-mini to all students 4 times per year? Or should they administer the DESSA-mini to students at the beginning of the year to determine who needs the full DESSA?
A: Generally, the DESSA-mini would be administered to all students at the beginning and at the end of the year. Then, the full DESSA would be administered to those students who are identified as requiring targeted strategies. DESSA-minis are typically used at times 2 and 3 only with those students receiving targeted strategies to monitor their progress in acquiring social and emotional competency.
Some schools and programs have chosen to use the full DESSA with every student at the beginning of the year rather than the DESSA-mini. Often the reason is that the teachers value the wealth of information provided by the classroom profile and feel that it is worth the extra time required to complete the full DESSA.
Q: Are there concerns regarding quality control? Will agency staff be more likely to reflect youth more positively because the information is reported to a program funder?
A: It is important to communicate with staff both the purpose of completing an assessment and how the information from the assessment will be used. If staff members incorrectly view the assessment as a reflection on their performance, they may answer inaccurately. Therefore, it is important to clearly articulate the purpose and use of the assessment prior to completing the ratings.
Q: Is it necessary to define the DESSA or DESSA-mini items for the raters? Would it be appropriate to develop a local rubric to help staff understand how the items might look different for a first grader versus an eighth grader?
A: It is not necessary to define the items of the DESSA or DESSA-mini for raters or to develop a local rubric. During the development of the assessments, we ensured that each of the items could be answered by teachers, after-school staff, and parents of children in grades K-8. Additionally, we determined that raters typically understand the developmental level of the child for which they are completing the assessment and keep this in mind as they are completing a rating.
Q: When is it appropriate to transition from using the DESSA K-8 tools to using the DESSA-HSE tools with students transitioning from eighth grade to ninth grade?
A: Programs can begin using the DESSA-HSE tools with students once they have finished 8th grade. This includes rising 9th graders, or students who have finished 8th grade but have not yet started 9th grade. These rising 9th graders were considered high school students for the purposes of standardizing the DESSA-HSE tools and were not excluded from standardization studies on the basis of their age or grade.
Q: How can I edit or delete a rating?
A: Unfortunately, users cannot edit a rating once it has been submitted. The rating would need to be deleted by a Program or Site Administrator. Then, it may be resubmitted by the rater (if the rating window is currently open).
For more details on deleting ratings, see the following article: How can I edit or delete a rating?
Q: Program Admin - How can I edit the name of a rating window?
A: In the DESSA System, Ratings Windows have a name assigned by the System. The name is assigned depending on how many times (up to four times) your organization will be rating during the school year, as indicated in the Yearly Setup.
Q: What is the recommended amount of time between pre- and post-assessment ratings?
A: The minimum amount of time between ratings is four weeks because the rating is based on the previous four weeks, and you would not want to have overlapping weeks included in both the pre and the post. Preferably, 90-120 day intervals are recommended.
Q: Are there concerns if the pre- and post-assessment ratings are completed by two different staff members?
A: Ideally the same rater should complete both pre- and post-assessment ratings. However, if this is not possible, it is expected that there could be some variation in scores when two different staff members provide ratings. If the ratings are providing information on an individual child’s progress, it is important to consider the possible rater differences when interpreting results. If the ratings are being used for program evaluation purposes, it is expected that any fluctuations in scores due to different raters would be canceled out when combined with the ratings from an entire program. Also, as mentioned above, reviewing the purpose of the assessment and the general administration guidelines will result in more consistent use across raters.
Q: Most of the students in a class scored in the "Typical" range. Now, the classroom teacher is noticing a large shift in behavior for many of them. Should the students be re-assessed?
A: Our national standardization data indicates that on average 16% of kids will fall in the need range and 16% in the strength range. However, not every classroom has that number of students with needs or strengths. It could well be that in that particular class every student was demonstrating typical social and emotional competence at the time of the screening.
Regarding students now showing needs – this can certainly happen. Either risk factors are piling up in some kids’ lives and beginning to overwhelm them (e.g. parents divorcing, bullying at school, poor initial academic performance), or sometimes kids are on their best behavior for the first few weeks until they figure out the power dynamics in the classroom or school.
Q: Can I retrieve a deleted rating?
A: Unfortunately, a deleted rating is removed from the database completely and may not be retrieved.
Q: We may have some students who come into the program later on in the year, outside of what should be rating period 1, but who still should be getting a “pre” DESSA. What can we do to ensure they are assigned to the "pre" DESSA?
A: All DESSA Assignments must fall within the designated Rating Period open and close dates. After a rater gets to know a new student for at least 4 weeks as recommended by R&D, the student can be rated before the next Rating Window opens. If the mid Rating Window has already started, the Rating results will fall under the current open Rating Window.
Comments
0 comments
Please sign in to leave a comment.