Research and Practice: An Applied Examination of the Parental Perceptions of Developmental Outcomes Scale
DOI:
https://doi.org/10.18666/JOREL-2019-V11-I1-9070Keywords:
camp, youth development, applied multivariate research, parents, Parental Perceptions of Developmental Outcomes ScaleAbstract
To support practitioners’ need for reliable and valid measures of the youth camp experience, this study compares the results of two analytic approaches—(1) composite-based paired samples t tests and (2) a latent structural equation model approach—using the Parental Perceptions of Developmental Outcomes (PPDO) scale in a sample of 930 parents of youth attending residential summer camps. Through a retrospective pretest (RPT) design, parents reported statistically significant growth across the five dimensions of the PPDO (responsibility, exploration, self-regulation, attitude, and communication) in both analytic approaches. The study results indicated similar effect sizes and conclusions were reached using both analytical approaches, suggesting a more practitioner-friendly approach (composite) may tell the same story as a more complex and research-oriented (latent) approach. This finding, as well as the revalidation of the PPDO in this sample, indicates the PPDO's usefulness in applied and research contexts.
References
Allen, J. M., & Nimon, K. (2007). Retrospective pretest: A practical technique for professional development evaluation. Journal of Industrial Teacher Education, 44(3), 1-7.
American Camp Association (2005). Directions: Youth development outcomes of the camp experience. Retrieved from https://www.acacamps.org/sites/default/files/resource_library/report-directions-youth-development-outcomes.pdf
American Camp Association. (2018). ACA youth outcomes battery. Retrieved from: https://www.acacamps.org/resource-library/research/aca-youth-outcomes-battery
Appelbaum, M., Cooper, H., Kline, R. B., May-Wilson, E., Nezu, A. M., Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. American Psychologist, 73(1), 3-25. http://dx.doi.org/10.1037/amp0000191
Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, 452-454. http://dx.doi.org/10.1038/533452a
Ball, A., Bates, S., Amorose, A., & Anderson-Butcher, D. (2017). The parent perceptions of overall school experiences scale: Initial development and validation. Journal of Psychoeducational Assessment, 1-12. http://dx.doi.org/10.1177/0734282917742310
Bashkov, B. M., Finney, S. J. (2013). Applying longitudinal mean and covariance structures (LMACS) analysis to assess construct stability over two time points. Measurement and Evaluation in Counseling and Development, 46(4), 289-314.
http://dx.doi.org/10.1177/0748175613497038
Baughman, S., Garst, B., & Fuhrman, N.E. (2009). Consistency of developmental outcomes of 4-H camp experiences over time and across sites. Journal of Youth Development, 4(2), 17-28. http://dx.doi.org/10.5195/JYD.2009.261
Bhattacharyya, S., Mead, T. P., & Nathaniel, R. (2011). The influence of science summer camp on African‐American high school students' career choices. School Science and Mathematics, 111(7), 345-353. http://dx.doi.org/10.1111/j.1949-8594.2011.00097.x
Bialeschki, M. D., & Sibthorp, J. (2011). Celebrating the camp experience through eighty years of camp research. Taproot, 20(2), 13-24.
Bobzien, J., & Judge, S. (2014). Characteristics of peer models at a summer camp for children with autism. Journal for Multicultural Education, 8(4), 237-248. http://dx.doi.org/10.1108/JME-04-2014-0017
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: The Guilford Press.
Browne, L. (2018). Let’s get retrospective. American Camp Association. Retrieved from
https://www.acacamps.org/news-publications/blogs/research-360/lets-get-retrospective
Bultas, M. W., Steurer, L. M., Balakas, K., Brooks, C., & Fields, H. (2015). Psychosocial outcomes of a summer overnight recreational experience for children with heart disease. Journal of Child Health Care, 19(4), 542-549.
http://dx.doi.org/10.1177/1367493514540350
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T., Huber, J., Johannesson, M.,…Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature: Human Behavior, 2, 637-644.
http://dx.doi.org/10.1038/s41562-018-0399-z
Campbell, N., Ali, F., Finlay, A. Y., & Salek, S. S. (2015). Equivalence of electronic and paper- based patient-reported outcome measures. Quality of Life Research, 24(8), 1949-1961.
http://dx.doi.org/10.1007/s11136-015-0937-3
Chang, R., & Little, T. D. (2018). Innovations for evaluation research: Multiform protocols, visual analog scaling, and the retrospective pretest–posttest design. Evaluation & the Health Professions, 41(2), 246-269. http://dx.doi.org/10.1177/0163278718759396
Chen, B., Vansteenkiste, M., Beyers, W., Boone, L., Deci, E. L., Van der Kaap-Deeder, J., ... & Ryan, R. M. (2015). Basic psychological need satisfaction, need frustration, and need strength across four cultures. Motivation and Emotion, 39(2), 216-236. http://dx.doi.org/10.1007/s11031-014-9450-1
Clary, C. D., & Ferrari, T. M. (2015). Communication, coping, and connections: Campers’ and parents’ perspectives of self-efficacy and benefits of participation in deployment support camps. Journal of Youth Development, 10(2), 31-54. http://dx.doi.org/10.5195/JYD.2015.407
Cohen, J., Cohen, P., West, S. G. & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). New York, NY: Routledge.
Collado, S., Staats, H., & Corraliza, J. A. (2013). Experiencing nature in children's summer camps: Affective, cognitive and behavioural consequences. Journal of Environmental Psychology, 33, 37-44. http://dx.doi.org/10.1016/j.jenvp.2012.08.002
Coman, E. N., Picho, K., McArdle, J. J., Villagra, V., Dierker, L., & Lordache, E. (2013). The paired t-test as a simple latent change score model. Frontiers in Psychology 4(738), 1-3.
http://dx.doi.org/10.3389/fpsyg.2013.00738
Dawes, J. (2008). Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50(1), 61-104. http://dx.doi.org/10.1177/147078530805000106
Ellis, G., Lee, K. J., & Satchabut, T. (2016). Experimental designs in leisure studies. In G. J. Walker, D. Scott, & M. Stodolska (Eds.), Leisure matters: The state and future of leisure studies. (pp. 332-343) State College, PA: Venture Publishing.
Enders, C. K. (2010). Applied missing data analysis. New York, NY: The Guilford Press.
Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). Thousand Oaks, CA: SAGE Publications.
Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error, Journal of Marketing Research, 18, 39-50.
http://dx.doi.org/10.2307/3151312
Freire, T., & Caldwell, L. L. (2013). Afterthoughts on leisure and future research directions. In T. Freire (Ed.), Positive leisure science: From subjective experience to social contexts (pp. 213–223). New York, NY: Springer. https://doi.org/10.1007/978-94-007-5058-6_12
Furr, R. M. (2018). Psychometrics: An introduction (3rd ed.). Thousand Oaks, CA: SAGE.
Gagnon, R. J., & Bumpus, M. F. (2016). Fidelity and its importance to experiential and outdoor education. Journal of Outdoor Recreation, Education, and Leadership, 8(1), 10-25.
http://dx.doi.org/10.18666/JOREL-2016-V8-I1-727
Gagnon, R. J., Stone, G., & Garst, B. (2017). Beyond composite scores and Cronbach’s alpha: Advancing methodological rigor in recreation research. Journal of Outdoor Recreation, Education, and Leadership, 9(4), 383-402. http://dx.doi.org/10.18666/JOREL-2017-V9-I4-7959
Garst, B., Baughman, S., Franz, N., & Seidel, R. (2013). Strengthening families: Exploring the impacts of family camp experiences on family functioning and parenting. Journal of Experiential Education, 36(1), 65-77. http://dx.doi.org/10.1177/1053825913481582
Garst, B. A., & Gagnon, R. J. (2016). A structural model of camp director practices and outcomes: Does intention toward program outcomes matter? Journal of Park and Recreation Administration, 34(4), 51-65. http://dx.doi.org/10.18666/JPRA-2016-V34-I4-
Garst, B., Gagnon, R., & Bennett, T. (2016). Parent anxiety causes and consequences: Perspectives from camp program providers. LARNet: The Cyber Journal of Applied Leisure and Recreation Research, 18(1), 21-39.
Gillard, A., & Watts, C. E. (2013). Program features and developmental experiences at a camp for youth with cancer. Children and Youth Services Review, 35(5), 890-898.
http://dx.doi.org/10.1016/j.childyouth.2013.02.017
Gould, J., Moore, D., McGuire, F., & Stebbins, R. (2008). Development of the serious leisure inventory and measure. Journal of Leisure Research, 40(1), 47-68. http://dx.doi.org/10.1080/00222216.2008.11950132
Heiervang, E., & Goodman, R. (2011). Advantages and limitations of web-based surveys: Evidence from a child mental health survey. Social Psychiatry and Psychiatric Epidemiology, 46(1), 69-76. http://dx.doi.org/10.1007/s00127-009-0171-9
Henderson, K. A., Thurber, C. A., Whitaker, L. S., Bialeschki, M. D., & Scanlin, M. M. (2006). Development and application of a camper growth index for youth. Journal of Experiential Education, 29(1), 1-17. http://dx.doi.org/10.1177/105382590602900103
Henderson, K. A., Bialeschki, M. D., Scanlin, M. M., Thurber, C., Whitaker, L. S., & Marsh, P. E. (2007a). Components of camp experiences for positive youth development. Journal of Youth Development, 1(3), 1–12. http://dx.doi.org/10.5195/JYD.2007.371
Henderson, K. A., Whitaker, L. S., Bialeschki, M. D., Scanlin, M. M., & Thurber, C. (2007b). Summer camp experiences: Parental perceptions of youth development outcomes. Journal of Family Issues, 28(8), 987-1007. http://dx.doi.org/10.1177/0192513X07301428
Hickerson, B., & Henderson, K. A. (2013). Opportunities for promoting youth physical activity: An examination of youth summer camps. Journal of Physical Activity and Health, 11, 199–205. http://dx.doi.org/10.1123/jpah.2011-0263
Hill, E., Gagnon, R., Ramsing, R., Goff, J., Kennedy, B., & Hooker, T. (2015). Measuring the impact of a medical specialty camp. Therapeutic Recreation Journal, 49(4), 310-325. http://dx.doi.org/10.18666/TRJ-2015-V49-I4-6308
Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501-517. http://dx.doi.org/10.1177/1098214005281356
Kendellen, K., Camiré, M., Bean, C. N., & Forneris, T. (2016). Facilitators and barriers to leadership development at a Canadian residential summer camp. Journal of Park & Recreation Administration, 34(4), 36-50. http://dx.doi.org/10.18666/JPRA-2016-V34-I4-6514
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd Ed.). New York, NY: The Guilford Press.
Lewis, K. M., Bird, M., Wilkins, T., Borba, J., Nathaniel, K., & Schoenfelder, E. (2018). Developing a common evaluation tool for camps. Journal of Youth Development, 13(1-2), 306-315. http://dx.doi.org/10.5195/JYD.2018.539
Lillibacka, R. (2006). Measuring social capital: Assessing construct stability of various operationalizations of social capital in a Finnish sample. ACTA Sociologica, 49(2), 201-220. http://dx.doi.org/10.1177/0001699306064774
Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198-1202.
http://dx.doi.org/10.1080/01621459.1988.10478722
Marsh, H., Hau, K., & Wen, Z. (2004). In search of golden rules: Comment on hypothesistesting approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341.
http://dx.doi.org/10.1207/s15328007sem1103_2
Marshall, J. P., Higginbotham, B. J., Harris, V. W., & Lee, T. R. (2007). Assessing program outcomes: Rationale and benefits of posttest-then-retrospective-pretest designs. Journal of Youth Development, 2(1), 118-123. http://dx.doi.org/10.5195/JYD.2007.366
Merryman, M., Mezei, A., Bush, J. A., & Weinstein, M. (2012). The effects of a summer camp experience on factors of resilience in at-risk youth. The Open Journal of Occupational Therapy, 1(1), Article 3. http://dx.doi.org/10.15453/2168-6408.1016
Michalski, J. M., Mishna, F., Worthington, C., & Cummings, R. (2003). A multi-method impact evaluation of a therapeutic summer camp program. Child and Adolescent Social Work Journal, 20(1), 53-76. https://doi-org.libproxy.clemson.edu/10.1023/A:1021467503588
Patton, M. Q. (2008). Utilization Focused Evaluation (4th ed.). Thousand Oaks, CA: SAGE.
Popper, K. (1968). The logic of scientific discovery. New York, NY: Routledge.
Povilaitis, V., & Tamminen, K. A. (2017). Delivering positive youth development at a residential summer sport camp. Journal of Adolescent Research, 1-26.
http://dx.doi.org/10.1177/0743558417702478
Powers, J., Maley, M., Purington, A., Schantz, K., & Dotterweich, J. (2015). Implementing evidence-based programs: Lessons learned from the field. Applied Developmental Science, 19(2), 108-116. http://dx.doi.org/10.1080/10888691.2015.1020155
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341-349.
http://dx.doi.org/10.1177/109821400002100305
Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4). 443-459.
http://dx.doi.org/10.1177/1098214008324182
Raidl, M., Johnson, S., Gardiner, K., Denham, M., Spain, K., Lanting, R., ... & Barron, K. (2004). Use retrospective surveys to obtain complete data sets and measure impact in extension programs. Journal of Extension, 42(2). Retrieved from https://www.joe.org/joe/2004april/rb2.php
Roth, J. L., & Brooks-Gunn, J. (2016). Evaluating youth development programs: Progress and promise. Applied Developmental Science, 20(3), 188-202.
http://dx.doi.org/10.1080/10888691.2015.1113879
Schueler, B. E., Capotosto, L., Bahena, S., McIntyre, J., & Gehlbach, H. (2014). Measuring parent perceptions of school climate. Psychological Assessment, 26(1), 314-320. http://dx.doi.org/10.1037/a0034830
Schelbe, L., Hansen, M. E. D., France, V. L., Rony, M., & Twichell, K. E. (2018). Does camp make a difference?: Camp counselors' perceptions of how camp impacted youth. Children and Youth Services Review. http://dx.doi.org/10.1016/j.childyouth.2018.08.022
Sibthorp, J., Bialeschki, M. D., Morgan, C., & Browne, L. (2013). Validating, norming, and utility of a youth outcomes battery for recreation programs and camps. Journal of Leisure Research, 45(4), 514-536. http://dx.doi.org/10.18666/jlr-2013-v45-i4-3897
Sibthorp, J., Paisley, K., Gookin, J., & Ward, P. (2007). Addressing response-shift bias: Retrospective pretests in recreation research and evaluation. Journal of Leisure Research, 39(2), 295-315. http://dx.doi.org/10.1080/00222216.2007.11950109
Thurber, C. A., Scanlin, M. M., Scheuler, L., & Henderson, K. A. (2007). Youth development outcomes of the camp experience: Evidence for multidimensional growth. Journal of Youth and Adolescence, 36(3), 241–254. http://dx.doi.org/10.1007/s10964-006-9142-6
Williams, D. C., Ma, Y., Prejean, L., Ford, M. J., & Lai, G. (2007). Acquisition of physics content knowledge and scientific inquiry skills in a robotics summer camp. Journal of Research on Technology in Education, 40(2), 201-216. http://dx.doi.org/10.1080/15391523.2007.10782505
Woods, K., Mayes, S., Bartley, E., Fedele, D., & Ryan, J. (2013). An evaluation of psychosocial outcomes for children and adolescents attending a summer camp for youth with chronic illness. Children's Health Care, 42(1), 85-98. http://dx.doi.org/10.1080/02739615.2013.753822
Downloads
Additional Files
Published
Issue
Section
License
Sagamore Publishing LLC (hereinafter the “Copyright Owner”)
Journal Publishing Copyright Agreement for Authors
PLEASE REVIEW OUR POLICIES AND THE PUBLISHING AGREEMENT, AND INDICATE YOUR ACCEPTANCE OF THE TERMS BY CHECKING THE ‘AGREE TO THE TERMS OF THIS COPYRIGHT NOTICE’ CHECKBOX BELOW.
I understand that by submitting an article to Journal of Outdoor Recreation, Education, and Leadership, I am granting the copyright to the article submitted for consideration for publication in Journal of Outdoor Recreation, Education, and Leadership to the Copyright Owner. If after consideration of the Editor of the Journal of Outdoor Recreation, Education, and Leadership, the article is not accepted for publication, all copyright covered under this agreement will be automatically returned to the Author(s).
THE PUBLISHING AGREEMENT
Assignment of Copyright
I hereby assign to the Copyright Owner the copyright in the manuscript I am submitting in this online procedure and any tables, illustrations or other material submitted for publication as part of the manuscript in all forms and media (whether now known or later developed), throughout the world, in all languages, for the full term of copyright, effective when the article is accepted for publication.
Reversion of Rights
Articles may sometimes be accepted for publication but later be rejected in the publication process, even in some cases after public posting in “Articles in Press” form, in which case all rights will revert to the Author.
Retention of Rights for Scholarly Purposes
I understand that I retain or am hereby granted the Retained Rights. The Retained Rights include the right to use the Preprint, Accepted Manuscript, and the Published Journal Article for Personal Use and Internal Institutional Use.
All journal material is under a 12 month embargo. Authors who would like to have their articles available as open access should contact Sagamore-Venture for further information.
In the case of the Accepted Manuscript and the Published Journal Article, the Retained Rights exclude Commercial Use, other than use by the author in a subsequent compilation of the author’s works or to extend the Article to book length form or re-use by the author of portions or excerpts in other works.
Published Journal Article: the author may share a link to the formal publication through the relevant DOI.
Author Representations
- The Article I have submitted to the journal for review is original, has been written by the stated author(s) and has not been published elsewhere.
- The Article was not submitted for review to another journal while under review by this journal and will not be submitted to any other journal.
- The Article contains no libelous or other unlawful statements and does not contain any materials that violate any personal or proprietary rights of any other person or entity.
- I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in the Article.
- If the Article was prepared jointly with other authors, I have informed the co-author(s) of the terms of this Journal Publishing Agreement and that I am signing on their behalf as their agent, and I am authorized to do so.