MEMORANDUM FOR:             RAYMOND UHALDE
                                                      Acting Assistant Secretary
                                                           for Employment and Training
 
 

FROM:                                         JOHN J. GETEK
                                                      Assistant Inspector General
                                                          for Audit

SUBJECT:                                  Adopting Best Practices Can Improve Placement Services to
                                                       Students Terminating From the Job Corps Program
                                                       Final Report No. 03-98-006-03-370

Attached is the subject final report.  Because you have agreed to implement our recommendations, the report is resolved and no further response is necessary.  Your response has been attached in its entirety as appendix D in the report.

We accept your invitation and are looking forward to participating in the workgroup scheduled for
April 1998.  Also, we wish to thank the Office of Job Corps for the cooperation and assistance they provided throughout the audit.

If you have any questions concerning this report, please contact Roger Langsdale, Regional Inspector General for Audit, in Philadelphia at (215) 596-6331.

Attachment

TABLE OF CONTENTS                                                                                                                    i

EXECUTIVE SUMMARY                                                                                                                  1

FINDINGS AND RECOMMENDATIONS                                                                                       7

ADOPTING BEST PRACTICES CAN IMPROVE PLACEMENT SERVICES
TO STUDENTS TERMINATING FROM THE JOB CORPS PROGRAM                       7

Contact                                                                                                                                      9

Best and Promising Practices                                                                                     11
Recommendations                                                                                                        13

Assessment                                                                                                                            14

Practices                                                                                                                         16
Recommendations                                                                                                         17

Job Development                                                                                                                  18

Best and Promising Practices                                                                                      19
Recommendations                                                                                                         20

Followup After Placement                                                                                                   21

Best Practices                                                                                                                23
Recommendations                                                                                                         24

CHANGES IN JOB CORPS' MONITORING PRACTICES CAN ENSURE
IMPROVEMENTS IN THE QUALITY OF PLACEMENT SERVICES                             25

Recommendations                                                                                                         27

A.  BACKGROUND, OBJECTIVES, SCOPE AND METHODOLOGY                          28

B.  ASSESSMENT OF ADEQUACY OF PLACEMENT SERVICES                             34

C.  LIST OF PLACEMENT CONTRACTORS VISITED                                                   36

D.  MANAGEMENT'S RESPONSE                                                                                     37
 
 

Job Corps, with 113 facilities located throughout the United States and Puerto Rico and an annual budget in excess of $1 billion, is the nation's largest Federal youth employment and training program. The primary expected result of the Job Corps program is for its students to be placed in jobs with promising prospects for long-term employment.

Background

In recent years, Job Corps has focused much management attention on increasing the number of students who find employment after leaving the program. As a result, reported placements have improved steadily over the past 4 years (Program Year (PY) 1993 through PY 1996). For example, 80 percent of Job Corps students were placed during PY 1996, compared to 64 percent in PY 1993.

Placement contractors play a critical role in Job Corps' efforts to find jobs for its students. Job Corps spent over $20 million in PY 1996 on placement assistance contracts with private companies and State Employment Security Agencies. These contractors are required to provide placement assistance to students for up to 6 months after they terminate from the Job Corps program.

Audit Results

As part of Job Corps' efforts to continue to improve the quality of placement outcomes achieved by its students, the Office of Inspector General (OIG), with assistance from Job Corps, conducted an audit of the services being provided by a sample of placement contractors. Job Corps assigned program staff to assist OIG in planning the audit and developing the audit program. Job Corps also participated actively in team briefings. Additionally, Job Corps paid for the services of four retired Job Corps employees to work as part of the audit teams during fieldwork. The four former Job Corps employees provided valuable program insight and expertise to the audit effort.

Our objectives were to evaluate the nature and extent of placement services provided to Job Corps students during PY 1996 and, more importantly, to identify best practices being used by placement contractors. OIG and Job Corps believe that implementation of the best practices currently being used in the placement community will ultimately result in students getting better and longer lasting jobs.
 

• making initial contact after termination,
• assessing students' capabilities and developing an employability plan and resume,
• contacting employers to develop suitable job leads and referring students to those jobs, and
• following up with students periodically during the 6 months after termination to determine whether additional placement services need to be provided.

Best Practices for Contacting Students

Early contact with students is essential to providing needed placement services. Best practices for making early contact include:

• becoming involved with students during the time they are in training at the center,
• receiving notification of students who will be terminating in the near future,
• promptly obtaining and using daily center termination reports to identify students who have left the program,
• beginning contact attempts immediately upon becoming aware of a student's termination,
• obtaining various alternate contacts from students as part of the recruitment process,
• providing to students before they leave the center, an information package regarding the placement services available to them,
• obtaining assistance from current students who may know how to contact difficult to locate terminees,
• establishing an "800" telephone line or accepting collect calls from students,
• using an automated system to telephone students that their adjustment checks and placement services are available if they return the call, and
• sending a non-negotiable copy of the readjustment check to students to let them know they will receive a genuine check if they contact the placement contractor.

 

Placement contractors are required to conduct an assessment of each student's training, job readiness skills and work history through individual interviews and review of student records provided by the center. This assessment is critical to the placement contractors' efforts to successfully place students in jobs commensurate with their training and skills.

All of the placement contractors we reviewed stated that they conduct assessments for every student; however, our audit of placement files found that these assessments are seldom documented (20 percent of the placement files reviewed contained a documented assessment).  Documentation of the assessment is essential to providing quality placement services.  A documented assessment shows what services were offered to the student, provides a reference when there is a turnover in placement staff, and provides a basis for followup after initial placement.

The best practice for ensuring that comprehensive assessments are conducted and documented is the use of a model assessment form which standardizes the required assessment elements and prompts placement specialists to ensure that all required elements are addressed.

Best Practices for Job Development

To enhance placement opportunities for students when they leave Job Corps, placement contractors are required to seek employment opportunities in the local job market that match the skills the students have acquired through Job Corps.

Best practices for job development noted during our audit included:

• identifying demand occupations in the area serviced by the placement contractor,
• making and maintaining contact with employers who can utilize the skills which students are learning in Job Corps,
• involving employers more directly in student training, and
• working with vocational and work experience instructors to identify job leads.

Placement contractors are expected to provide assistance to Job Corps' students for up to 180 days after they terminate from Job Corps. Job Corps students remain eligible for continued placement assistance even after becoming employed. Placement contractors should

Best practices for follow up after placement identified during our audit included:

• establishing policies requiring that followup contacts be made on a periodic basis and clearly communicating that policy to placement specialists; and
• implementing tracking systems (manual or automated) to notify placement specialists when followup contacts need to be made.

 

Job Corps needs to ensure that placement contractors improve the quality of placement services. Job Corps' monitoring of placement contractors, carried out by its Regional Office staff, has focused on ensuring that contractors achieved their placement goals. This emphasis has contributed to the improved placement results that Job Corps has achieved over the past several years. To follow up on the implementation of the best practices identified in this report, future monitoring efforts should focus on assessing the quality of placement services, as well as the achievement of placement goals.

Recommendations

The goal of this joint OIG-Job Corps audit was to identify ways to improve the quality of placement services provided to Job Corps students. Based on our assessment of the services provided during PY 1996 and the best practices identified during the course of our audit, we recommend that the Assistant Secretary for Employment and Training require the Office of Job Corps to take the following actions:

-- communicate the best practices identified in this report to all placement contractors and encourage them to implement these practices wherever possible,

-- define the number of days after termination by which placement contractors should have started their efforts to contact assigned students,

-- require placement contractors to document their attempts to initially contact students after they leave Job Corps,

-- establish goals or benchmarks for making actual contact with assigned students, both in terms of elapsed days and percentage of students contacted,

-- require placement contractors to conduct assessments for all assigned students and that the results of these assessments be documented in a placement plan developed jointly with each student,

-- develop and disseminate to placement contractors a model assessment form that contains all the elements of a complete, documented assessment,

-- review the documentation and adequacy of assessments during on-site monitoring of placement contractors,

-- encourage placement contractors to develop linkages with Job Corps centers and large employers in their assigned geographic areas,

-- require placement contractors to document individual job development efforts to place students in jobs that match their training and skills,

-- establish goals or benchmarks for job development efforts and direct referrals of students to employers,

-- require placement contractors to contact students monthly after placement,

-- require placement contractors to develop and implement an internal system that tracks student placement status throughout the 6-month placement period to determine the need for additional placement services,

-- require placement contractors to incorporate followup contacts after placement into their placement specialists' performance standards,

-- monitor the services provided by placement contractors to ensure that contract requirements are being met and students are receiving quality placement services

-- develop a monitoring guide for Regional Office staff to use when reviewing placement contractor performance, and

-- contact a representative number of placed and non-placed students on a periodic basis to ascertain their level of satisfaction with the placement services they received.

The Assistant Secretary for Employment and Training agreed with the findings and recommendations as presented in this report. The Office of Job Corps is forming a workgroup to develop the necessary policy statements and related products to implement the recommendations.

OIG Conclusion

Because the Assistant Secretary for Employment and Training has agreed to implement our recommendations, the report is resolved. OIG has been invited by the Office of Job Corps to participate on its workgroup and we look forward to assisting in this effort to improve placement services to Job Corps students.

Placement contractors can improve the quality of the placement services provided to Job Corps students by ensuring that each student receives all the services to which he or she is entitled, as specified in the contract between the placement contractor and Job Corps. Adopting the best practices identified during our audit can assist placement contractors and Job Corps in making the needed improvements.

Once assigned students, placement contractors are required to contact students timely, assess students' social and career needs, identify employers in their local job markets that need the skills and abilities of returning students, and follow up after the initial placement has been made to determine if students are in need of additional services. However, we found that placement contractors spend much of their efforts ensuring students received their readjustment allowance checks, providing general job referrals, and verifying and reporting placements. Job Corps' monitoring practices did not assess the level and quality of placement services being provided (see report section titled "Changes in Job Corps' Monitoring Practices Can Ensure Improvement in the Quality of Placement Services"). We believe this contributed to contractors' misplaced emphasis on reporting placement activity rather than providing placement assistance. As a result, in our opinion, placement services were not adequate for 49 percent of the students in our sample who were in need of the services⁽¹⁾.

^1. Of the 1,232 students in our sample, we identified 322 (26 percent) that were not in need of all the required placement services because they were already in placements that were adequate, or their personal situation (i.e., pregnant, deceased, did not want assistance, ect.) dictated that placement was not appropriate. We also found there were no placement files or any other documentation for an additional 110 students (9 percent). Thus, we could not identify and evaluate the placement services that were provided to these students. This left 800 students in our sample who needed placement services and their placement files contained sufficient documentation for us to form an opinion on the adequacy of the services. We determined that the placement services were inadequate for 49 percent of these students. See Appendix B for details on our determination of the adequacy of placement services.

The implementation of "best practices" will not guarantee improvement in the quality of placement services, because, like all operations and processes, the caliber and dedication of the placement contractors' management and staff make a significant difference in performance.

The contract requirements and Job Corps' Policy and Requirements Handbook (PRH), dated July 1995, with subsequent amendments, require placement contractors to provide services in four key areas:

• making initial contact after students have terminated from Job Corps,

 
• assessing students' capabilities and developing employability plans and resumes,

 
• contacting employers to develop suitable job leads and referring students to those jobs, and

 
• following up with students periodically during the 6 months following termination from Job Corps to determine whether additional placement services need to be provided.

 

Prior to PY 1995, placement assistance was available to Job Corps students for the first 6 months after they left the program.  However, following initial placement, most students were not actively contacted for placement assistance services, even if that placement occurred just 1 day after leaving the program.

To maximize the amount of time available to provide placement services and find employment opportunities for assigned students, the PRH requires placement contractors to immediately attempt to contact terminating students. The placement contract requires service providers to exert maximum effort in obtaining full-time placement as soon after the students' termination dates as possible. As stated in the PRH, the sooner the contact is made, the less chance there is that the student will not be contacted and thus not be placed. At a work group meeting before the start of our audit, placement contractors told us that attempts to contact students should begin within 7 days after being assigned.

The process of formally contacting students begins when placement contractors are notified of assigned students by daily transmissions of data from Job Corps' Student Pay, Allotment and Management Information System (SPAMIS). The placement contractors use the data to generate the "Placement Assistance Record," Form 6-78. The PRH requires the center to input the termination data into SPAMIS within two working days of students' terminations. Placement contractors have only one SPAMIS terminal to receive the transmissions and generate the Forms 6-78, and it is normally located at their main office. Therefore, it is the placement contractors' responsibility to forward the Forms 6-78 to the appropriate suboffices.

Results

To review student contact, we evaluated the length of time it took for the placement contractors to actually contact assigned students after they terminated from Job Corps. There were 177 students in our sample of 1,232 for whom we could not determine when or if they were contacted because of insufficient evidence. Thus, our analysis is based on the remaining 1,055 students.
 

Placement agencies are required to exert maximum effort in obtaining full-time placement for assigned students, as soon as possible after the termination date. As illustrated above, only 37 percent of the students in our analysis were contacted within 14 days after leaving Job Corps.

Success in contacting students is directly related to placement contractors making a timely attempt to contact. In 62 percent of the placement files we analyzed that had evidence of a date for both the initial attempt and actual contact, successful contact with students occurred within the first week from the date of the first attempt to contact. Therefore, if efforts to contact assigned students are made within the first week following termination, it is more likely that the actual contact will be achieved within the first two weeks after termination.

We reviewed placement files to determine when the first attempt to contact the student was made. There were 165 students out of our sample of 1,232 for whom we could not make a determination of when or if the initial contact attempt was made because of insufficient evidence. Thus, our analysis is based on the remaining 1,067 students. Placement contractors made their initial attempts to contact within the first week after termination for 42 percent of the students (447 of 1,067). Conversely, for the remaining 58 percent, the initial attempt to contact the students was not made until 1 or more weeks after the students had terminated from Job Corps.

Making contact with assigned students is the obvious first step in the placement assistance process. Given the young and mobile population served by Job Corps, we recognize that this task is not as simple as it sounds. The best practices summarized below can assist placement contractors in achieving this task in a more timely manner.

Placement contractors told us that it is important to contact students immediately after they terminate from the Job Corps program. We identified five placement contractors that made the initial attempt to contact the students within 7 days after termination for at least 70 percent of their students in our sample. In actually contacting students, we identified six placement contractors (including two placement contractors previously cited) that contacted the largest percentage of their students in our sample within 7 days after leaving the center. These nine placement contractors told us they emphasized the importance of placement to the students before they left the center so they knew where to go and who to contact. These placement contractors used one or more of the following practices to contact students after termination.
 

At the time of admission into the Job Corps program, placement contractors:
• Obtained various alternate contacts from the students when they were recruited into the Job Corps program. This provided the placement contractors several leads if they could not locate the students on the initial attempt after they left the Job Corps center.
 
While students were at the Job Corps center, placement contractors:
• Made sure that their placement specialists at the center were involved with the students throughout their stay. This ongoing interaction emphasized the importance of placement to the students. Thus, the students were aware of the placement services available and knew where to go to receive them. Both center and non-center-based placement contractors practiced this method.  Non-center-based placement contractors either located placement specialists at the Job Corps centers they served, or had them visit the centers on a regular basis.

• Reviewed daily center termination reports before the students were formally assigned so that attempts to contact them could begin immediately.
 
• Made arrangements to be notified before students graduated so the placement specialist could meet with the students before they left the center. The purpose of the meeting was to interview the students and provide information on where and how to obtain placement services.
 
• Provided students an information package about placement services. This package told students who their placement specialist was and how to contact him/her.

 

After the students left the Job Corps center, placement contractors:

• Posted notices throughout the center regarding students they had been unable to locate and solicited assistance from students who may know where to locate them.
 

We also identified additional practices used by placement contractors who were not as successful as the seven placement contractors cited above. We believe these are promising practices for contacting students as soon as possible after the students leave the centers.
 

At the time of admission into the Job Corps program, placement contractors:

• Used the same staff for both admission counselors and placement specialists so there was ongoing interaction with students from the time they applied for the program to when they were ready to be placed.
 
After students left the Job Corps center, placement contractors:
• Committed resources to provide termination notices to placement specialists as soon as possible. This is critical for placement contractors that have multiple sites. The termination notices were generated at the main office and then faxed or sent by overnight mail to the responsible placement specialist.

• Established an 800 telephone line or accepted collect calls to allow terminating students to reach the placement specialists.
 
• Used automated telephone systems that called students to notify them that their readjustment check had arrived and placement services were available if they return the call.
 
• Sent non-negotiable copies of readjustment checks to students to inform them they will get a real check if they contact their placement specialists.

To help improve placement contractors' timeliness in making their initial contact with assigned Job Corps' students, we recommend that the Assistant Secretary for Employment and Training require the Office of Job Corps to:

-- define in the PRH the maximum number of days the placement contractors should start their efforts to contact students after they leave the centers;

-- require placement contractors to document their attempts to initially contact students after they leave the Job Corps; and

-- establish goals or benchmarks for making contact with assigned students, both in terms of elapsed days and percentage of students contacted.

As specified in the placement contract, an assessment involves the following four elements:

1.   Student Center Records.   As part of the basis for analyzing each student's potential for placement, the placement contractor is required to review records from the Job Corps center that document the student's vocational and academic training, job readiness skills, and work experience. At a group planning session before the start of our audit, placement contractors told us that it is unrealistic to expect a good assessment of a student's accomplishments if a complete set of vocational records from the center is not available.

2.   Skills Assessment.   Placement contractors are required to assess each student's vocational and academic training, job readiness skills, and prior work history to determine the best placement options for the student. This assessment should be based on review of center records and interviews with the student.

3.   Social Needs Assessment.   The placement contractor is required to assess each student's social needs to identify potential barriers to employment such as a need for child care or transportation problems. The placement contractor should refer the student to agencies that can assist the student in meeting these needs.

4.   Student Resume.   According to the PRH, students need desirable resumes which provide the best possible presentation of their skills and experiences in order to compete successfully in today's job market. For those students who do not have a resume, the contract requires the placement contractor to assist the student in developing one.

The following chart presents the overall results of our review of the student placement files for each of the four required assessment elements. These results are based on the 910 students who were in need of placement services.
 

 

Assessment Element	Percent of Placement Files Reviewed That Contained Evidence That The Required Assessment Was Performed.
Evidence That Center Records Were Obtained	46%
Evidence That Skills Assessment Was Performed	20%
Evidence That Social Needs Assessment Was Performed	12%
Evidence That Student Had a Resume	15%

 

We also evaluated the extent to which each of the 21 placement contractors in our sample conducted and documented assessments. The following table presents the results of our evaluation.
 
 

Assessment  Element	Number of Placement Contractors In Which There Was Evidence That The Assessment Was Performed For The Following Percentage Of Students In Our Sample Who Needed Services
	0 to 24 %	25 to 49%	50 to 69%	70% or More
Student Records Were Obtained	4	5	8	4
Skills Assessment Was Performed	17	2	1	1
Social Needs Assessment Was Performed	18	3	0	0
Student Has a Resume	16	3	2	0

 

As the data in the table show, placement contractors are either not doing assessments for all students, or they are doing assessments but not documenting them. For 110 students (9 percent) in our sample, we found no placement file or any other documentation to show what placement services had been provided. Two placement contractors were responsible for 86 percent of these students.

We also found that 120 students were referred to social agencies and/or provided job readiness skills training. However, because of the small percentage of assessments in the placement files, we could not make a determination of whether these services were provided to all students who needed them.

All the placement contractors told us they did comprehensive assessments for all students, but only four said that the assessments should be documented. Eight placement contractors had developed a standard form to document assessments but they were not always used. Placement contractors did not monitor student placement files to ensure assessments were being done and documented.

Best Practices for Assessments

The use of a standard assessment form is a best practice for ensuring that comprehensive assessments are done and documented for all assigned students.  Standard assessment forms help to (1) standardize the assessment process, (2) ensure that all elements of the assessment are covered, and (3) reduce the amount of time required to process the paperwork.  Standard assessment forms can also assist in managing the caseload in the event there is a turnover of placement staff.

If consistently used by the placement specialists, standard assessment forms can ensure that each student's placement file contains:

• all records concerning the training received while in Job Corps,
• inventory of skills,
• evaluation of job readiness skills to identify any weaknesses that need to be addressed,
• proper resume or work training history,
• listing of the student's career goals and how to achieve them,
• listing of support services needed and referrals provided, and
• agreement with the student on placement goals.

-- specify in the PRH that placement contractors conduct assessments for all assigned students and the results of these assessments be documented in placement plans jointly developed with the students;

-- develop and disseminate to placement contractors a model assessment form that contains all the elements of a complete, documented assessment; and

-- review the documentation and adequacy of assessments during onsite monitoring of placement contractors. (See the report section entitled "Changes in Job Corps' Monitoring Practices Can Ensure Improvement in the Quality of Placement Services.")

The contract requires placement contractors to seek, in a proactive manner, employment opportunities in the local job market to match the skills that students gained through the Job Corps program. Visits are to be made to employers, academic institutions, military recruiters and local employment services.

Results

We found that all the placement contractors in our sample conducted job development activities. These activities included:

• using public job listings from state employment security agencies, various internet sites, unions and newspapers,

 
• attending job fairs and local Chamber of Commerce meetings,

 
• contacting employers who have previously hired Job Corps students, and

 
• making new employer contacts on a periodic basis.

The vocational completers in our sample were not always provided referrals to jobs that matched their training and skills. Of the 492 vocational completers in our sample who were in need of placement services, 181 (37 percent) were referred to jobs related to their training. Of these, 86 (48 percent) found a job with the referred employer.

As part of our assessment of job development activities, we sent letters to 233 potential employers of Job Corps' students. These 233 employers were selected from the employer listings maintained by the placement contractors we visited. We received responses from
 

Best Practices For Job Development

The best practices for job development involved working with area employers to identify the skills those employers need and convincing center operators to provide training that equips students with the skills in demand by employers.

Our audit identified two examples of best practices for job development:

• One placement contractor arranged for a large hotel operator to participate in the training provided at a Job Corps center. The hotel operator agreed to hire students who successfully completed the training.

 
• Another placement contractor, with assistance from the Job Corps Regional Office, worked with the Job Corps center they served and a large employer in need of welders to tailor the Job Corps training to meet the employer's needs. Again, the employer agreed to hire students who successfully competed the training.

Promising Practices

We also identified some practices that were innovative in developing job referrals for returning students. For promising practices, placement contractors:

• Required that their placement specialist regularly visit employers in their area to identify job openings and sell Job Corps.

• Established a working relationship with local military recruiters.

 
• Regularly contacted employers who hired Job Corps students to establish ongoing relationships that may result in those employers hiring more students.

 
• Established a system whereby employers FAXed job opening notices to the placement office.

 
• Implemented automated systems that match job openings to students' training.

 
• Contacted employers in the area where students planned to live that the students themselves had identified as employers for whom they would like to work.

 

To increase the effectiveness of placement contractors' job development efforts, we recommend that the Assistant Secretary for Employment and Training require the Office of Job Corps to:

-- encourage placement contractors to develop linkages with Job Corps centers and large employers in their job market so that students' skills are matched to employers' needs;

-- require placement contractors to document individual job development efforts to place students in jobs that match their training and skills; and

-- establish goals or benchmarks for job development efforts and direct referrals of students to employers.

The primary reasons for followup after placement are to:

• assist students who have become unemployed to find other employment,

 
• upgrade the initial placement to meet Job Corps' definition of a quality placement, and

 
• provide additional services that students may need to achieve Job Corps' expectation of permanent employment.

 

Of the 1,232 students in our sample, 1,068 (87 percent) were placed. We found that 31 percent of these placed students were contacted after placement to find out if they were still employed and/or needed additional placement assistance.
 

The following chart summarizes placement contractors' performance in making followup contact with students after placement has occurred.
 
 

Summary of Contacts Made After Placement
Percent of Sampled Files Which Contained Evidence of Follow-up Contacts	0-24 Percent	25-49 Percent	50-69 Percent	More than 70 Percent
Number of Contractors	11	4	1	5

 

For 84 of the 1,068 placed students, we found evidence in the placement files that the placement contractors were aware of the students' need for additional services but did not provide the needed assistance. Specifically, we found:

Fifty-one students placed themselves before initial contact. However, their placements were inadequate and the placement contractors did not attempt to upgrade them.   For example, Job Corps assigned an advanced completer in the home-health aid vocation to the placement contractor on January 17, 1997. On February 11, 1997, the placement specialist found out that the student was working as a file clerk. The placement specialist verified the placement but there was no attempt to try to upgrade the job. In another example, Job Corps assigned a student, a completer in welding, to the placement contractor on May 6, 1997. The placement specialist contacted the student on May 19, 1997, and found that the student was employed at a food store as a bagger. The only action taken by the placement specialist was to confirm the student's address so he could send the student the readjustment check. There was no attempt to upgrade the job.

We also identified 33 students who the placement contractors became aware that they were no longer placed and did not provide them any additional services.   For example, Job Corps assigned a student to the placement contractor on August 28, 1996. The placement specialist provided the student job referrals and on October 4, 1996, found that the student had a job. The placement specialist contacted the student on December 24, 1996, and found the student was no longer employed. The last note the placement specialist made in the file was "will refer to other employers in the area." There was no evidence that the placement specialist subsequently referred the student to other employers. In another example, Job Corps assigned a student to the placement contractor on November 18, 1996.  However, the placement specialist did not contact the student until January 14, 1997.  The student told the placement specialist that he worked for

about a month. The last note the placement specialist made was on January 16, 1997, which stated "Sent employer verification and received it back. Put placement through."   There was no evidence that the placement specialist provided further assistance.

The following section describes the best practices for followup currently being used by some placement contractors. Implementation of these practices should improve the quality of followup services provided to Job Corps students.

Best Practices for Followup After Placement

There were five placement contractors who made followup contact with 70 percent or more of the students in our sample. Following are the practices used by these placement contractors:

• Two placement contractors had established procedures that required their placement specialists to contact placed students 9 weeks after placement. The purpose of the contact is to determine the students' placement status, whether upgrades in placement were needed, and whether continued placement services were required. These placement contractors had implemented manual tracking systems that notified their placement specialists when a student's 9-week period approached.

 
• Another placement contractor established procedures that required their placement specialists to contact placed students at least once a month throughout the 6-month placement period. This contractor implemented a comprehensive automated caseload management system that tracked all activity between the placement specialists and the students. The tracking system also notified the placement specialists when the 30 day follow-up contact was due.

• Two of the placement contractors had a policy which required their placement specialists to contact students at least once every 30 days after the initial placement was reported, but had no tracking system.

To ensure that students receive placement assistance for 6 months after leaving Job Corps, we recommend that the Assistant Secretary for Employment and Training require the Office of Job Corps to:

-- amend the PRH to require monthly contact with students after placement;

-- require placement contractors to develop and implement an internal system that tracks student placement status throughout the 6-month placement period to determine the need for additional placement services; and

-- require placement contractors to incorporate followup contacts after placement into their placement specialists' performance standards.

 
 While Job Corps is concerned about the quality of the placement services being provided to students, they do not have a monitoring process that fully assesses the extent and quality of these placement services. The monitoring requirements and performance standards used to evaluate the placement contractors focus on reporting and verifying placements. The basis of Job Corps' monitoring efforts is the assumption that quality placements are the result of quality placement services. Without a comprehensive monitoring process that includes an assessment of the level and quality of placement services, Job Corps would have difficulty in identifying the extent of the problems that were found in this audit and the action needed to correct them.

According to the PRH, Job Corps' Regional Offices exercise operational supervision over all placement activities in the region, communicating directly with centers, placement agencies, and placement support organizations. The specific Regional Office requirements in the PRH that concern the monitoring of placement contractors are:

• incorporate into all placement contracts, the performance standards issued by the Director of Job Corps,

 
• monitor and supervise all placement activities within the region by providing technical assistance and initiating corrective action for deficient performance,

 
• ensure complete understanding of placement policies and full compliance with established procedures by placement organizations, and

 
• conduct regular onsite monitoring of the placement verification process of the support contractors and placement contractors and ensure that placement activities continue for students until the end of the 6-month period.

• Place at least 70 percent of all students assigned

 
• At least 80 percent of all placements be in full-time jobs

• The average wage for job placements be equal or greater than the level specified by Job Corps

• At least 50 percent of all job placements be in occupations related to the students' training experience

During onsite reviews at placement contractors, the Regional Offices offered technical assistance and evaluated placement operations. However, the Regional Offices did not review individual placement files to determine the level and quality of the placement services being provided. Following describes the monitoring practices used by the Regional Offices:

• At one Regional Office, monitoring consisted of desk reviews of monthly SPAMIS reports. These reports provided statistics showing placement contractors' achievements in meeting each of the four performance standards, a listing of students who had not been placed and the time remaining on their placement period. According to the Regional Office, placement contractor performance is adequate if the performance standards are met. The Regional Director told us that he did not have the resources to do onsite reviews.

 
• Another Regional Office had not done any monitoring visits since April 1997, because of staff and budget shortages. Thus, monitoring was limited to reviewing the monthly SPAMIS reports on placement contractor performance. Placement contractor performance is adequate if the performance standards are met. The Regional Director told us that a staff person had recently been hired to monitor placement contractors and they are planning on developing a review guide which will cover contract requirements.

 
• Two Regional Offices did both monitoring visits and reviews of monthly SPAMIS reports. The monitoring visits were for the entire outreach, admissions, and placement contract. These visits to the placement

contractors' main offices were done at least twice a year. If the Job Corps centers had a placement component, the visits were done quarterly as part of the center reviews. The placement portion of the reviews consisted of discussions of the placement contractors' performance in relation to the performance standards and what can be done towards better reaching the goals. One of these regions developed several review guides. However, interviews rather than placement file reviews was the methodology used to evaluate the placement contractors.

To ensure that placement contractors are providing the required placement services and that students are receiving quality placement assistance, we recommend that the Assistant Secretary for Employment and Training require the Office of Job Corps to:
 

-- monitor the services provided by placement contractors to ensure that contract requirements are being met and students are receiving quality placement services;

-- develop a monitoring guide for Regional Office staff to use when reviewing placement contractor performance;

-- contact a representative number of placed and non-placed students on a periodic basis to ascertain their level of satisfaction with the placement services they received; and

-- communicate the best and promising practices in this report to all placement contractors and encourage them to implement these practices wherever possible.
 
 

The Job Corps program was established in 1964 and is presently authorized under Title IV, Part B of the Job Training Partnership Act, Public Law 97-300, dated October 1982. Job Corps was designed as a national employment and training program for economically disadvantaged youth ages 16 to 24. With annual funding of over $1 billion, Job Corps is the largest Federal youth employment and training program. The operations of the program are carried out at mostly residential centers where students participate in intensive programs of academic training, vocational training, work experience, and counseling.

There are 113 Job Corps centers located throughout the United States and Puerto Rico. Private corporations and nonprofit organizations, selected through competitive procurement, operate 85 of the centers. The remaining 28 centers, known as Civilian Conservation Centers, are operated either by the U.S. Department of Agriculture or various agencies of the U.S. Department of Interior.

The Job Corps National Office provides overall guidance to Job Corps regions, centers, and placement contractors in its PRH. Job Corps' nine regional offices are responsible for administering all contracts and centers within each region's geographic jurisdiction.

The overall purpose of the program is to provide economically disadvantaged youth with the opportunity to become more responsible, employable citizens. The primary expected results of the Job Corps program are to have the students leave the center and be placed in jobs with promising prospects for long-term employment. In addition to employment, a successful placement can also include enlistment in the armed forces or enrollment in school full-time.

Rather than relying on existing job placement services provided by Federal and state programs which are available to the general public, Job Corps awards contracts to placement agencies to place students because the typical student comes from an economically disadvantaged background and is in need of a higher level of service. For program year (PY) 1996, Job Corps spent over $20 million for placement services.
 
 

There are three primary types of placement service providers utilized by Job Corps.  Center placement contractors are typically responsible for students attending their own centers. T he advantage of a center placement agency is that placement specialists have easy access to students and they can be involved with students from the time they enter the center until they are placed.  Private placement agencies are responsible for students from centers which do not have a placement component. Contracts with private placement agencies are for a defined geographic area.  The advantage of a private placement agency is that they can cover a larger geographic area than a center placement agency.  State employment service agencies operate similar to private placement agencies.  The principal advantage of state employment service agencies is that they have an extensive network of offices located throughout the state.

Performance Standards

Job Corps has established performance standards which placement contractors are expected to achieve and which are used as a basis for determining continued participation in the program. Placement contractors are responsible for providing assistance to all students regardless of how long they were enrolled at the Job Corps center. The exceptions to this are those students who terminated within the first 30 days for violating the program's zero tolerance policy for drugs or violence and those found to be ineligible after enrollment. For PY 1996, Job Corps established the following performance standards for placement contractors:

• Place at least 70 percent of all students assigned.

 
• At least 80 percent of all placements be in full-time jobs.

 
• The average wage for job placements be equal or greater than the level specified by Job Corps.

 
• At least 50 percent of all job placements be in occupations related to the terminating student's training experience.

 

Terms of the placement contracts require that, once assigned students, the contractor must immediately make every effort to place students in jobs with promising prospects for long-term employment.
 
 

• CONTACT - Once assigned a student, the placement contractor must make all efforts to contact the student immediately.

• ASSESSMENT - After contact, the placement contractor should assess the student's capabilities and determine the best suitable placement. This process should be done in conjunction with the student so that placement contractors know and understand the student.

• JOB DEVELOPMENT - In order to provide job referrals to the student after the assessment process is complete, placement contractors should proactively seek employment opportunities in the local job market that match the skills of the student who is returning to the area from the centers.

 
• FOLLOWUP - After placement, contractors are required to follow up to determine if the student is still employed, enrolled in full-time school, or enlisted in the Armed Forces. If the student has not been, or no longer is placed, then additional placement assistance is to be provided.

 

OBJECTIVES

The specific objectives of this audit were to (1) evaluate the nature and extent of placement services that are being provided to Job Corps students after they terminate from the program, and (2) identify "best practices" employed by various placement contractors which, if implemented nationwide, would improve the overall quality of placement services and, therefore, improve the likelihood of quality placements for students who are not successful in finding suitable employment on their own.

To help achieve our objectives, Job Corps assigned program staff to assist in planning the audit and developing the audit program. Job Corps also participated actively in team briefings. Job Corps also paid for the services of four retired Job Corps employees to work as part of the audit teams during field work. The four former Job Corps employees provided valuable program insight and expertise to the audit effort.

SCOPE AND METHODOLOGY

Our audit scope included all placement contractors that were under contract with the Job Corps program for PY 1996 (July 1996 - June 1997). We excluded the Instructional Work Experience Program contractor from the universe because the nature of their operation is significantly different than the placement contractors.

There were 73 placement agency contracts in effect for PY 1996. However, some of these contracts covered more than one state. To reduce the logistics of our visits, we decided to consider each state as a separate contract which resulted in a total of 81 contract service areas.

The universe of students consisted of those students who terminated during PY 1996 and were either placed or were not placed within their 6-month placement period. This omitted students who terminated in PY 1995 but were eligible for placement services during PY 1996 and students who were not yet placed but their 6-month placement period had not expired. The reason for omitting these students was to obtain a current representation of students who most recently terminated from Job Corps and students who had an outcome (placed or not placed within their 6-month placement period). This resulted in a universe of 42,903 students.
 
 

The universe of placement contractors was segregated into three strata - small (less than 500 terminees), medium (more than 500 but less than 1,000 terminees), and large (1,000 or more terminees).  We also segregated the universe by type of contract - center based, non-center based, and state employment service agencies. In selecting placement contractors for audit, we used proportional sampling to obtain a representation of the universe of placement contracts.  Our sampling plan selected 17 placement contracts for site visits.  An additional four placement contracts were judgmentally selected based on recommendations from Job Corps as being good performers.

We reviewed 1,232 student placement files. We used random statistical sampling to select student placement files for review for each placement contractor. The number of student placement files reviewed at each site depended on the size and type of the placement contract. We also visited four Job Corps Regional offices that were judgmentally selected based on our sample of contracts and recommendations from the Job Corps National Office.

See appendix C for a schedule of the placement contracts selected and the population of assigned students who terminated in PY 1996.

Overall Approach

Our audit of placement services focused on four areas: contact, assessment, job development and followup. Contractors' responsibilities for these four areas are detailed in Job Corps' PRH and the statement of work section of the contract.

To accomplish our objective of identifying practices that would improve the overall quality of placement services, we first interviewed the placement contractors' managers and placement specialists. We then reviewed the placement files for the students in our sample to determine if these practices were used and were effective. However, because the placement files did not always contain documentation of the required placement services that were supposed to be provided, we could not accomplish our objective of whether the identified "best practice" specifically resulted in a quality placement for the student. For example, in the area of contact, if the identified "best practice" was to arrange a meeting with the students before they left the center so they knew who to contact and where to go for placement services, we could not find this documented in the student placement file to evaluate whether it resulted in timely contact after termination or led to a quality placement.

As an alternative, we determined which placement contractors performed the required services for 70 percent or more of their students in our sample.  We refer to the practices used by these placement contractors as "best practices." We refer to the practices used by placement
 
 

What We Did At Each Site

The audit teams were made up of both OIG auditors and Job Corps consultants.  At the contractors, the project manager and senior placement specialists were interviewed to identify their methods for providing placement assistance and job development to Job Corps students. Individual student placement files and any other applicable documentation were reviewed to enable us to identify the services and practices that were used to help place the students. The work was performed at the placement contractors' main offices in the states. Also, at least one suboffice in each state was visited to ascertain whether the services provided at the suboffices differed from the services provided at the main offices. Additionally, a sample of current placement files (PY 1997) were reviewed to identify current practices that may be different than practices utilized in PY 1996.

At the Job Corps Regional Offices, management and personnel responsible for administering and monitoring placement contracts were interviewed to gain an understanding of their oversight responsibilities and information on any initiatives or pilot projects being tested to strengthen the placement process.

We sent questionnaires to the students in our sample to verify information in the placement file and determine whether students were satisfied with the services provided by their placement contractors. The response rate to our questionnaires was not sufficient to provide any reportable results.

Also, questionnaires were sent to a sample of employers from the placement contractors' employer listings to determine whether or not Job Corps students were being hired.

We conducted our audit work from August 1997 through January 1998, in accordance with Government Auditing Standards, as necessary to achieve the above objectives.
 
 
 

Overall, in our opinion, placement services were not adequate for 49 percent of the students in our sample who were in need of the services. Of the 1,232 students in our sample, we identified 322 (26 percent) that were not in need of services, except to be given their readjustment checks and placement bonus after the placement was verified. These students were already in placements that were adequate, or their personal situations (i.e., pregnant, deceased, did not want assistance, ect.) dictated that placement services were not needed. We also found there were no placement files or any other documentation for an additional 110 students. Two placement contractors were responsible for 86 percent of these students with no documentation. Thus, we could not identify and evaluate the placement services that were provided to these students. This left us with 800 students in our sample who needed placement services when contacted by the placement contractor and their placement files contained sufficient documentation for us to form an opinion on the adequacy of the services provided.

The following table summarizes how we determined the 800 students to review.
 
 

In our opinion, placement services to 398 (49 percent) of the remaining 800 students were not adequate. The table on the following page summarizes our reasons for concluding that placement services were inadequate.