Transformation Stories

HUD Transforms Risk Management with Data Science

HUD Chief Risk Officer Larry Koskinen and Pei Hamsik, senior advisor to HUD's CFO.

Data is critical to efficiency, decision-making, and increased accountability in risk management. In partnership with the Department of the Treasury, the Department of Housing and Urban Development (HUD) is transforming its enterprise risk management with data. Here's how.

HUD's Office of Community Planning and Development (CPD) provides grants to thousands of individuals, groups, municipalities, and states. The grants are used to improve communities by providing decent housing and expanding economic opportunities for low and moderate-income persons. CPD's grants benefit veterans, those affected by natural disasters, and the homeless.

Oversight of its grants is imperative to CPD, said HUD Chief Risk Officer Larry Koskinen.

Grantees are regularly audited to ensure that their records are accurate and that grant recipients are delivering what they’ve committed to do.

Over the years, the audits have generated hundreds of thousands of single audit reports, rich with data. Yet analyzing the data contained in the single audit reports in order to differentiate grantees is challenging.

Using Data in Decision-Making

Audit reports are captured in PDF files. Teasing out clear insights based on mountains of unstructured text is time-consuming and inefficient.

HUD needs data so it can make objective fact-based decisions to allocate resources and discover cross-cutting issues. Which programs are working? And which are in trouble? Where should investigators focus?

"It's easy to find failed programs," said Mr. Koskinen. "Instead, we needed a way to prevent failure by identifying the red flags earlier in the process and mitigating risk. Our interest is to help grantees succeed."

Streamlining Data-Rich Risk Assessment

Working with Treasury's Office of Financial Innovation and Transformation, HUD is developing an enhanced Streamlined Risk Analysis. The initiative is building on the work at HUD, which has digitized the textual data contained in the single audit reports and created a platform that allows stakeholders to search, explore, and drill down on grantee risk profiles.

Through one interface, stakeholders can access single audit reports. From there, users can perform a keyword search of all text from the audits, and quickly access all findings sections.

Using Visualization

For example, an analyst could review a specific state or region to see if there are clusters of high-risk programs. The analyst could then drill down to find out if there's a common problem in the region. This allows HUD to determine where investigative resources are needed.

"The Streamlined Risk Analysis is already at work, helping HUD determine its investigative priorities in the coming year," said Mr. Koskinen.

The "Secret Sauce"

Using data, HUD created a linguistic sentiment-based risk score for each grantee

Working with data engineers, subject matter experts (SMEs) created a taxonomy of terms to be used in key word searches and analysis. Terms were weighted to associate specific terms with audit risk. Data scientists built an algorithm to create the sentiment-based risk scores for grantees. A result is illustrated below.

In comparing the sentiment risk scores against SME risk scores, the results were accurate and also revealed some unexpected results. Analysis showed that some grants were higher risk than expected – and some were lower. These outliers resulted in ongoing conversations about risk, and refinement of the algorithm.

"Working with SMEs was our secret sauce," said Mr. Koskinen. "Experts in housing grants know what to look for in audits. We applied their knowledge and experience to the algorithm by applying weights to specific terms in the taxonomy."

Sharing the Knowledge

HUD envisions providing its Streamlined Risk Analysis through HUD’s shared services partnership with the Treasury Administrative Resource Center.

"We would like to marry up transactional data with other data fields, such as procurement and staffing data," said Pei Hamsik, senior advisor to HUD's CFO. More data can provide a richer picture of grant performance and how we are managing our resources.

"The Streamlined Risk Analysis is government owned, and it's portable," said Mr. Koskinen. "Our long term vision is to share this tool with others. Other analysts could use the same data and infrastructure to apply our knowledge to analyze their programs, and vice versa."

At HUD, being data-driven allows for faster, better fact-based decisions – a win-win for the agency, taxpayers, and the vulnerable American families that HUD serves.

Last modified 03/23/20