Open Access Open Access  Restricted Access Subscription Access

Research and Practice: An Applied Examination of the Parental Perceptions of Developmental Outcomes Scale

Ryan Joseph Gagnon, Barry Garst

Abstract


To support practitioners’ need for reliable and valid measures of the youth camp experience, this study compares the results of two analytic approaches—(1) composite-based paired samples t tests and (2) a latent structural equation model approach—using the Parental Perceptions of Developmental Outcomes (PPDO) scale in a sample of 930 parents of youth attending residential summer camps. Through a retrospective pretest (RPT) design, parents reported statistically significant growth across the five dimensions of the PPDO (responsibility, exploration, self-regulation, attitude, and communication) in both analytic approaches. The study results indicated similar effect sizes and conclusions were reached using both analytical approaches, suggesting a more practitioner-friendly approach (composite) may tell the same story as a more complex and research-oriented (latent) approach. This finding, as well as the revalidation of the PPDO in this sample, indicates the PPDO's usefulness in applied and research contexts.

Subscribe to JOREL


Keywords


camp; youth development; applied multivariate research; parents; Parental Perceptions of Developmental Outcomes Scale

Full Text:

PDF

References


Allen, J. M., & Nimon, K. (2007). Retrospective pretest: A practical technique for professional development evaluation. Journal of Industrial Teacher Education, 44(3), 1-7.

American Camp Association (2005). Directions: Youth development outcomes of the camp experience. Retrieved from https://www.acacamps.org/sites/default/files/resource_library/report-directions-youth-development-outcomes.pdf

American Camp Association. (2018). ACA youth outcomes battery. Retrieved from: https://www.acacamps.org/resource-library/research/aca-youth-outcomes-battery

Appelbaum, M., Cooper, H., Kline, R. B., May-Wilson, E., Nezu, A. M., Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. American Psychologist, 73(1), 3-25. http://dx.doi.org/10.1037/amp0000191

Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, 452-454. http://dx.doi.org/10.1038/533452a

Ball, A., Bates, S., Amorose, A., & Anderson-Butcher, D. (2017). The parent perceptions of overall school experiences scale: Initial development and validation. Journal of Psychoeducational Assessment, 1-12. http://dx.doi.org/10.1177/0734282917742310

Bashkov, B. M., Finney, S. J. (2013). Applying longitudinal mean and covariance structures (LMACS) analysis to assess construct stability over two time points. Measurement and Evaluation in Counseling and Development, 46(4), 289-314.

http://dx.doi.org/10.1177/0748175613497038

Baughman, S., Garst, B., & Fuhrman, N.E. (2009). Consistency of developmental outcomes of 4-H camp experiences over time and across sites. Journal of Youth Development, 4(2), 17-28. http://dx.doi.org/10.5195/JYD.2009.261

Bhattacharyya, S., Mead, T. P., & Nathaniel, R. (2011). The influence of science summer camp on African‐American high school students' career choices. School Science and Mathematics, 111(7), 345-353. http://dx.doi.org/10.1111/j.1949-8594.2011.00097.x

Bialeschki, M. D., & Sibthorp, J. (2011). Celebrating the camp experience through eighty years of camp research. Taproot, 20(2), 13-24.

Bobzien, J., & Judge, S. (2014). Characteristics of peer models at a summer camp for children with autism. Journal for Multicultural Education, 8(4), 237-248. http://dx.doi.org/10.1108/JME-04-2014-0017

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: The Guilford Press.

Browne, L. (2018). Let’s get retrospective. American Camp Association. Retrieved from

https://www.acacamps.org/news-publications/blogs/research-360/lets-get-retrospective

Bultas, M. W., Steurer, L. M., Balakas, K., Brooks, C., & Fields, H. (2015). Psychosocial outcomes of a summer overnight recreational experience for children with heart disease. Journal of Child Health Care, 19(4), 542-549.

http://dx.doi.org/10.1177/1367493514540350

Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T., Huber, J., Johannesson, M.,…Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature: Human Behavior, 2, 637-644.

http://dx.doi.org/10.1038/s41562-018-0399-z

Campbell, N., Ali, F., Finlay, A. Y., & Salek, S. S. (2015). Equivalence of electronic and paper- based patient-reported outcome measures. Quality of Life Research, 24(8), 1949-1961.

http://dx.doi.org/10.1007/s11136-015-0937-3

Chang, R., & Little, T. D. (2018). Innovations for evaluation research: Multiform protocols, visual analog scaling, and the retrospective pretest–posttest design. Evaluation & the Health Professions, 41(2), 246-269. http://dx.doi.org/10.1177/0163278718759396

Chen, B., Vansteenkiste, M., Beyers, W., Boone, L., Deci, E. L., Van der Kaap-Deeder, J., ... & Ryan, R. M. (2015). Basic psychological need satisfaction, need frustration, and need strength across four cultures. Motivation and Emotion, 39(2), 216-236. http://dx.doi.org/10.1007/s11031-014-9450-1

Clary, C. D., & Ferrari, T. M. (2015). Communication, coping, and connections: Campers’ and parents’ perspectives of self-efficacy and benefits of participation in deployment support camps. Journal of Youth Development, 10(2), 31-54. http://dx.doi.org/10.5195/JYD.2015.407

Cohen, J., Cohen, P., West, S. G. & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). New York, NY: Routledge.

Collado, S., Staats, H., & Corraliza, J. A. (2013). Experiencing nature in children's summer camps: Affective, cognitive and behavioural consequences. Journal of Environmental Psychology, 33, 37-44. http://dx.doi.org/10.1016/j.jenvp.2012.08.002

Coman, E. N., Picho, K., McArdle, J. J., Villagra, V., Dierker, L., & Lordache, E. (2013). The paired t-test as a simple latent change score model. Frontiers in Psychology 4(738), 1-3.

http://dx.doi.org/10.3389/fpsyg.2013.00738

Dawes, J. (2008). Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50(1), 61-104. http://dx.doi.org/10.1177/147078530805000106

Ellis, G., Lee, K. J., & Satchabut, T. (2016). Experimental designs in leisure studies. In G. J. Walker, D. Scott, & M. Stodolska (Eds.), Leisure matters: The state and future of leisure studies. (pp. 332-343) State College, PA: Venture Publishing.

Enders, C. K. (2010). Applied missing data analysis. New York, NY: The Guilford Press.

Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). Thousand Oaks, CA: SAGE Publications.

Fornell, C., & Larcker, D. (1981). Evaluating structural equation models with unobservable variables and measurement error, Journal of Marketing Research, 18, 39-50.

http://dx.doi.org/10.2307/3151312

Freire, T., & Caldwell, L. L. (2013). Afterthoughts on leisure and future research directions. In T. Freire (Ed.), Positive leisure science: From subjective experience to social contexts (pp. 213–223). New York, NY: Springer. https://doi.org/10.1007/978-94-007-5058-6_12

Furr, R. M. (2018). Psychometrics: An introduction (3rd ed.). Thousand Oaks, CA: SAGE.

Gagnon, R. J., & Bumpus, M. F. (2016). Fidelity and its importance to experiential and outdoor education. Journal of Outdoor Recreation, Education, and Leadership, 8(1), 10-25.

http://dx.doi.org/10.18666/JOREL-2016-V8-I1-727

Gagnon, R. J., Stone, G., & Garst, B. (2017). Beyond composite scores and Cronbach’s alpha: Advancing methodological rigor in recreation research. Journal of Outdoor Recreation, Education, and Leadership, 9(4), 383-402. http://dx.doi.org/10.18666/JOREL-2017-V9-I4-7959

Garst, B., Baughman, S., Franz, N., & Seidel, R. (2013). Strengthening families: Exploring the impacts of family camp experiences on family functioning and parenting. Journal of Experiential Education, 36(1), 65-77. http://dx.doi.org/10.1177/1053825913481582

Garst, B. A., & Gagnon, R. J. (2016). A structural model of camp director practices and outcomes: Does intention toward program outcomes matter? Journal of Park and Recreation Administration, 34(4), 51-65. http://dx.doi.org/10.18666/JPRA-2016-V34-I4-

Garst, B., Gagnon, R., & Bennett, T. (2016). Parent anxiety causes and consequences: Perspectives from camp program providers. LARNet: The Cyber Journal of Applied Leisure and Recreation Research, 18(1), 21-39.

Gillard, A., & Watts, C. E. (2013). Program features and developmental experiences at a camp for youth with cancer. Children and Youth Services Review, 35(5), 890-898.

http://dx.doi.org/10.1016/j.childyouth.2013.02.017

Gould, J., Moore, D., McGuire, F., & Stebbins, R. (2008). Development of the serious leisure inventory and measure. Journal of Leisure Research, 40(1), 47-68. http://dx.doi.org/10.1080/00222216.2008.11950132

Heiervang, E., & Goodman, R. (2011). Advantages and limitations of web-based surveys: Evidence from a child mental health survey. Social Psychiatry and Psychiatric Epidemiology, 46(1), 69-76. http://dx.doi.org/10.1007/s00127-009-0171-9

Henderson, K. A., Thurber, C. A., Whitaker, L. S., Bialeschki, M. D., & Scanlin, M. M. (2006). Development and application of a camper growth index for youth. Journal of Experiential Education, 29(1), 1-17. http://dx.doi.org/10.1177/105382590602900103

Henderson, K. A., Bialeschki, M. D., Scanlin, M. M., Thurber, C., Whitaker, L. S., & Marsh, P. E. (2007a). Components of camp experiences for positive youth development. Journal of Youth Development, 1(3), 1–12. http://dx.doi.org/10.5195/JYD.2007.371

Henderson, K. A., Whitaker, L. S., Bialeschki, M. D., Scanlin, M. M., & Thurber, C. (2007b). Summer camp experiences: Parental perceptions of youth development outcomes. Journal of Family Issues, 28(8), 987-1007. http://dx.doi.org/10.1177/0192513X07301428

Hickerson, B., & Henderson, K. A. (2013). Opportunities for promoting youth physical activity: An examination of youth summer camps. Journal of Physical Activity and Health, 11, 199–205. http://dx.doi.org/10.1123/jpah.2011-0263

Hill, E., Gagnon, R., Ramsing, R., Goff, J., Kennedy, B., & Hooker, T. (2015). Measuring the impact of a medical specialty camp. Therapeutic Recreation Journal, 49(4), 310-325. http://dx.doi.org/10.18666/TRJ-2015-V49-I4-6308

Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501-517. http://dx.doi.org/10.1177/1098214005281356

Kendellen, K., Camiré, M., Bean, C. N., & Forneris, T. (2016). Facilitators and barriers to leadership development at a Canadian residential summer camp. Journal of Park & Recreation Administration, 34(4), 36-50. http://dx.doi.org/10.18666/JPRA-2016-V34-I4-6514

Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd Ed.). New York, NY: The Guilford Press.

Lewis, K. M., Bird, M., Wilkins, T., Borba, J., Nathaniel, K., & Schoenfelder, E. (2018). Developing a common evaluation tool for camps. Journal of Youth Development, 13(1-2), 306-315. http://dx.doi.org/10.5195/JYD.2018.539

Lillibacka, R. (2006). Measuring social capital: Assessing construct stability of various operationalizations of social capital in a Finnish sample. ACTA Sociologica, 49(2), 201-220. http://dx.doi.org/10.1177/0001699306064774

Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83(404), 1198-1202.

http://dx.doi.org/10.1080/01621459.1988.10478722

Marsh, H., Hau, K., & Wen, Z. (2004). In search of golden rules: Comment on hypothesistesting approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341.

http://dx.doi.org/10.1207/s15328007sem1103_2

Marshall, J. P., Higginbotham, B. J., Harris, V. W., & Lee, T. R. (2007). Assessing program outcomes: Rationale and benefits of posttest-then-retrospective-pretest designs. Journal of Youth Development, 2(1), 118-123. http://dx.doi.org/10.5195/JYD.2007.366

Merryman, M., Mezei, A., Bush, J. A., & Weinstein, M. (2012). The effects of a summer camp experience on factors of resilience in at-risk youth. The Open Journal of Occupational Therapy, 1(1), Article 3. http://dx.doi.org/10.15453/2168-6408.1016

Michalski, J. M., Mishna, F., Worthington, C., & Cummings, R. (2003). A multi-method impact evaluation of a therapeutic summer camp program. Child and Adolescent Social Work Journal, 20(1), 53-76. https://doi-org.libproxy.clemson.edu/10.1023/A:1021467503588

Patton, M. Q. (2008). Utilization Focused Evaluation (4th ed.). Thousand Oaks, CA: SAGE.

Popper, K. (1968). The logic of scientific discovery. New York, NY: Routledge.

Povilaitis, V., & Tamminen, K. A. (2017). Delivering positive youth development at a residential summer sport camp. Journal of Adolescent Research, 1-26.

http://dx.doi.org/10.1177/0743558417702478

Powers, J., Maley, M., Purington, A., Schantz, K., & Dotterweich, J. (2015). Implementing evidence-based programs: Lessons learned from the field. Applied Developmental Science, 19(2), 108-116. http://dx.doi.org/10.1080/10888691.2015.1020155

Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341-349.

http://dx.doi.org/10.1177/109821400002100305

Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4). 443-459.

http://dx.doi.org/10.1177/1098214008324182

Raidl, M., Johnson, S., Gardiner, K., Denham, M., Spain, K., Lanting, R., ... & Barron, K. (2004). Use retrospective surveys to obtain complete data sets and measure impact in extension programs. Journal of Extension, 42(2). Retrieved from https://www.joe.org/joe/2004april/rb2.php

Roth, J. L., & Brooks-Gunn, J. (2016). Evaluating youth development programs: Progress and promise. Applied Developmental Science, 20(3), 188-202.

http://dx.doi.org/10.1080/10888691.2015.1113879

Schueler, B. E., Capotosto, L., Bahena, S., McIntyre, J., & Gehlbach, H. (2014). Measuring parent perceptions of school climate. Psychological Assessment, 26(1), 314-320. http://dx.doi.org/10.1037/a0034830

Schelbe, L., Hansen, M. E. D., France, V. L., Rony, M., & Twichell, K. E. (2018). Does camp make a difference?: Camp counselors' perceptions of how camp impacted youth. Children and Youth Services Review. http://dx.doi.org/10.1016/j.childyouth.2018.08.022

Sibthorp, J., Bialeschki, M. D., Morgan, C., & Browne, L. (2013). Validating, norming, and utility of a youth outcomes battery for recreation programs and camps. Journal of Leisure Research, 45(4), 514-536. http://dx.doi.org/10.18666/jlr-2013-v45-i4-3897

Sibthorp, J., Paisley, K., Gookin, J., & Ward, P. (2007). Addressing response-shift bias: Retrospective pretests in recreation research and evaluation. Journal of Leisure Research, 39(2), 295-315. http://dx.doi.org/10.1080/00222216.2007.11950109

Thurber, C. A., Scanlin, M. M., Scheuler, L., & Henderson, K. A. (2007). Youth development outcomes of the camp experience: Evidence for multidimensional growth. Journal of Youth and Adolescence, 36(3), 241–254. http://dx.doi.org/10.1007/s10964-006-9142-6

Williams, D. C., Ma, Y., Prejean, L., Ford, M. J., & Lai, G. (2007). Acquisition of physics content knowledge and scientific inquiry skills in a robotics summer camp. Journal of Research on Technology in Education, 40(2), 201-216. http://dx.doi.org/10.1080/15391523.2007.10782505

Woods, K., Mayes, S., Bartley, E., Fedele, D., & Ryan, J. (2013). An evaluation of psychosocial outcomes for children and adolescents attending a summer camp for youth with chronic illness. Children's Health Care, 42(1), 85-98. http://dx.doi.org/10.1080/02739615.2013.753822




DOI: https://doi.org/10.18666/JOREL-2019-V11-I1-9070

Refbacks

  • There are currently no refbacks.


Copyright (c) 2018 Sagamore Publishing LLC