[Ailist] How to makes sense of already-collected data?

[Gary Robbins]

Colleagues:

I'm currently doing some research, and was hoping that someone could 
point me to some literature or ideas that I may be missing.

The evaluation literature seems to be geared largely toward implementing 
evaluation models and methods in 'ideal' situations, where the 
evaluators are able to influence the evaluation procedures from the 
beginning of program implementation, so that the formative and summative 
evaluation components are built in to the program at the onset.

I'm wondering if there's a model of working with a program that has been 
running for some time and that has been collecting data in a haphazard, 
we-think-this-might-be-interesting-to-know-someday sort of way.  In 
particular, I'm thinking of a smaller psychoeducational program that did 
not have any particular evaluation guidance at the beginning, so put 
together a trial pre/post instrument, and has been collecting 
information for a few years now, but the data is just accumulating in a 
storage room at the facility.  The program wants to try to wrangle the 
information that they've collected to see if any of it is informative, 
or can be used to provide a sense of how the program has done in the 
past, but there's so much data, and any or all of it may or may not be 
usable, that they are unsure how to begin wading through it all.

So, does anyone know of any kind of step-by-step process, or list of 
strategies or rules or best practices for wading through evaluation data 
in a less-than-ideal evaluation situation like this, where the evaluator 
is asked to come in and see what they can do with the information that 
the program has already been collecting?

Thanks so much for any insight into this issue.
Gary Robbins
Loma Linda University