Section 7: Project Evaluation

Introduction

Project evaluation is an ongoing process conducted throughout the grant agreement period. This evaluation process is performed at different levels of administration. The following table shows the phases of project evaluation and who performs them.
Performance of Project Evaluation
Type of Evaluation
Performed by
Periodic project monitoring (described later in this section)
Project Managers
Review of Performance Reports
Project Managers
Risk Assessment
TRF-BTS Director through eGrants
eGrants project grading
Project Managers
Final project evaluation
Project and Program Managers
Annual Report (covered in Section 8 of this chapter)
TRF-BTS Planner

Periodic Project Monitoring

The project monitoring process (covered in Sections 2 and 3 of this Chapter) is an important part of evaluation of the project.
Project Performance Reports and Requests for Reimbursement (RFRs) normally give an evaluator some sense of whether or not the project is proceeding according to the project’s objectives in terms of activities reported and expenditures included. Anything that appears out of the ordinary can then be followed up with an on-site monitoring visit. In addition, on-site monitoring visits provide an opportunity to determine progress and detect problems. In this way, the evaluator might discover a minor problem early and recommend an early correction in activity or tighter fiscal control to avoid a major problem later.
In addition, the reports should also indicate whether substantial overruns or underruns are occurring, and if so, they should be explained in the report. In the event of substantial underruns, the Program Manager might recommend reallocation of these resources to other areas where additional funds can be better utilized. For overruns, the Program Manager should ensure that all costs are expended according to the approved budget and that the grant can sustain itself throughout the effective term period. Otherwise, an amendment to the grant may be required. Project and program managers should submit these recommendations to the TxDOT TRF-BTS Director for approval prior to implementation.

Risk Assessment

Beginning with FY 2016 grants, TxDOT started conducting a risk evaluation for each subgrantee receiving National Highway Traffic Safety Administration (NHTSA) funds. Under the Uniform Grant Guidance, TxDOT is required to evaluate each subgrantee’s risk of noncompliance with federal statutes, regulations and terms and conditions of the grant for purposes of determining the appropriate level of subgrantee monitoring. TxDOT will conduct the risk assessment annually and will document the outcome of the risk assessment process.

eGrants Project Grading

2 C.F.R. §200.318(h) states:
“The non-Federal entity must award contracts only to responsible contractors possessing the ability to perform successfully under the terms and conditions of the proposed procurement. Consideration will be given to such matters as contract integrity, compliance with public policy, record of past performance, and financial and technical resources.”
TRF-BTS has developed the eGrants Project Grading System in order to comply with the above and to assist TRF-BTS in reviewing project accomplishments (performance measures completed, targets achieved), to determine whether or not the grant provided a beneficial service to the Traffic Safety Program, and to determine the continued need for the project.
Each subgrantee receives a grade of A through F at the end of the grant year based on a possible 100 points. Subgrantees start with 0 points and earn points throughout the grant year. Points are earned by the timely submission of Performance Reports, RFRs, subgrantee’s performance (per reporting period) and adherence to the project budget (per reporting period).
Points are awarded by the review and grading of Performance Reports and RFRs by the TxDOT Project Manager. Both the STEP and general grants are graded on 10 criteria. eGrants automatically calculates and grades five criteria for STEP grants and three criteria for general grants. Project managers review and grade the remaining criteria by reviewing Performance Reports and RFRs and completing the Review and Comment pages of these reports. These criteria include:
  • STEP Indicator, based on grant target number (for STEP projects only).
  • Matching contribution.
  • Performance Report submission dates:
    • On time (met the submission deadline).
    • Missed the submission deadline.
  • RFR submission dates:
    • On time (met the submission deadline).
    • Missed the submission deadline.
  • Public Information and Education (PI&E) activity objectives (for STEP Projects only).
  • Project manager review of Performance Reports.
  • Citations/arrests for STEP grants/objectives, applicable activities, and performance measures for general grants.
    • Less than target – Justified.
    • Less than target – Not justified.
    • Meets.
    • Exceeds.
  • Risk assessment:
    • Termination.
    • “High Risk” suspension.
    • Below.
    • Meets.
    • Exceeds.
  • Overruns/Underruns:
    • Yes.
    • No.
Subgrantees that meet all grant objectives, including submitting all reports within the 30-day time frame, meeting all grant goals and target numbers, and staying within the grant budget by 10% through the grant year, will receive a score of 90, or an A.
The grading process is cumulative and can be viewed at any time by the subgrantee or the Project Manager by clicking “Project Grading,” found on the bottom right side of the grant.
TxDOT review includes the following:
  • The Project Manager reviews Performance Reports and completes the Review Comments page of the Performance Report. This page contains questions concerning the subgrantee’s performance and the Project Manager’s risk assessment of the subgrantee to date. A comments box is also included on the page to allow the Project Manager the opportunity to provide any comments.
  • The Project Manager reviews RFRs and completes the Review Comments page of the RFR. There are three questions on this page that the Project Manager must answer concerning budget and supporting documentation submitted by the subgrantee. A comments box is also included on the page to allow the Project Manager the opportunity to provide any comments.
  • TRF-BTS project managers must complete the questions on the Review/Comments pages of the reports to the best of their ability. Each question is answered by selecting the appropriate radio button that best represents the subgrantee’s performance for the reporting period. Therefore, careful review of information contained in the Performance Reports and RFRs, including supporting documentation, is required.
Each Review/Comment page includes a comment box, which is a mandatory field. This is an excellent place to document any concerns or reminders, or praise a subgrantee for excellent work. The comments box should be used in conjunction with the Project Manager’s assessment and grading of the performance period through the use of the radio button selections. The file of record, including the subgrantee’s performance, must be thoroughly documented (especially for a subgrantee that is struggling to meet performance measures/target numbers) to include evidence of continued communication between the Project Manager and the subgrantee.
“Justified” means the subgrantee has included adequate information in the Performance Report Narrative Page explaining difficulties/obstacles encountered by the subgrantee that directly impacted their ability to meet the objectives and/or activities of the grant for the reporting period. Project managers must use their best judgment, based on the available information, that a valid reason existed that caused the subgrantee to fail to meet one or more objectives or target numbers for the reporting period in order to be considered justified. Valid reasons would have to be considered exceptional and beyond the subgrantee’s control.
“Not justified” means insufficient information is included on the Performance Report Narrative Page to substantiate the reason(s) for failing to meet the grant’s goals and objectives. Therefore, the Project Manager must choose the “Not Meeting Target - Not Justified” radio button on the Comments/Review Page.
The eGrants Project Grading Page can be accessed through the eGrants Project Grading link located on the eGrants Help Page. The page provides the latest information concerning the automated grant grading system, including the values of the above criteria.

End-of-Grant Performance Report

Within 30 days of the end of the project period, the subgrantee is required to submit the end-of-grant Performance Report to the Project Manager (See Chapter 5, Section 2, “Performance Report.”). This report is intended to be an overall and brief synopsis of the project performance for the year. The final end-of-grant Performance Report:
  • Tells whether or not the project objectives were met, and
  • Lists accomplishments (performance measures completed, targets achieved).
After review by the Project Manager, end-of-grant performance reports are used by TRF-BTS for compilation and use in developing the annual report.

Final Requests for Reimbursement

Within 45 days of the end of the project, subgrantees must submit all final RFRs to TxDOT. The final RFR is critical to the final closeout of the fiscal year budget because it allows any residual monies to be carried forward to fund the next year's projects.