Knowledge Is Good
Introduction
The opening shots of the 1978 film “Animal House” show idyllic scenes of the fictional Faber College. At one point, the camera zooms in on a statue of Emil Faber, the founder of the college. The inscription on the base of the statue displays Faber’s motto: Knowledge Is Good.
While Faber’s motto is vacuous — and a tip-off for what is to come in the rest of the movie — knowledge is not only good but it is essential when making complex financial decisions. And deciding whether or not to purchase a home is one of the most complex financial decisions most people make. First-time homebuyers, in particular, may be overwhelmed by the sheer number of things to learn. How much house can they afford? Will they qualify for a mortgage? What type of mortgage is best for them? What about mortgage insurance, title insurance, appraisals, flood certifications, and on and on? Would they be better off continuing to rent?
Where can potential homebuyers find help? Of course, there are many experienced professionals involved in the purchase to whom they can turn. However, as we saw in the run-up to the housing crisis, many borrowers did not receive or did not act on the information they needed. What buyers need is unbiased and authoritative information about the homebuying and — just as important — loan-paying processes in order to make informed decisions.
Many organizations offer this type of information. For example, My Home by Freddie Mac® provides information on renting, owning a home, refinancing a mortgage, and dealing with the possibility of foreclosure. In fact, Freddie Mac believes this type of objective information is so important that it requires first-time homebuyers taking advantage of Home Possible Advantage®, our low-down-payment product, to participate in a financial literacy education program.
How much does borrower education and counseling help potential homebuyers? It has been surprisingly difficult to answer this question. During the housing crisis, it became apparent that many borrowers had ended up with debt burdens and exotic mortgages that led them into financial distress. There is widespread agreement in the industry that homebuyer counseling can help prevent some of the mistakes made during the housing boom. However, early studies of the impact of counseling produced sometimes conflicting or inconclusive results and raised questions about the effectiveness of borrower education and counseling.
Measuring the effectiveness of counseling
Over the years, there have been several studies of the effectiveness of pre-purchase homeownership counseling. For example, Hirad and Zorn reviewed data on 40,000 participants in Freddie Mac’s Affordable Gold Loans program and concluded that borrowers who received classroom and home study counseling had reductions in their subsequent rates of serious delinquency of 26 percent and 21 percent, respectively. Borrowers who received individual counseling averaged a 34 percent reduction in their rate of serious delinquency.
In contrast, a study by Quercia and Spader, which is based on a different program that required borrowers to participate in a homeownership education and counseling (HEC) component found“no evidence that HEC completion reduces default.” And studies by Agarwal, et al., and by Birkenmaier and Tyuse came to mixed conclusions.
This lack of a definitive consensus is a common problem in social research. Social scientists must rely for the most part on observational data, that is, data available from observations of uncontrolled, ordinary activity. For instance, researchers might estimate the impact of pre-purchase counseling by comparing the delinquency rates of a group of borrowers that received counseling to a group that didn’t receive counseling. While the counseled borrowers may have lower average delinquency rates than the borrowers in the uncounseled group, it’s not immediately clear that the counseling accounts for the entire difference in delinquency rates. Perhaps the borrowers who received counseling also were more highly-educated than the borrowers in the other group. Maybe they had a greater disposition or ability to apply the information provided by the education course. Maybe they had higher credit scores than the other borrowers.
All the researchers in the studies cited above were aware of the challenges to basing conclusions on observational data, and they went to great lengths to insulate their inferences from these types of confounding factors. Nonetheless, there are limits to what can be concluded from observational data. One way to overcome these limitations is to use experimental data rather than observational data.
Experimental data divides participants into a treatment group and a control (that is, untreated) group in such a way that any difference in outcomes between groups is most likely to be a result of the treatment rather than some other, uncontrolled difference in the characteristics of the two groups. Individuals can be assigned randomly to each group, reducing the chance that, for example, people with higher credit scores are likelier to receive counseling. Alternatively, the treatment and control groups can be constructed to be as similar as possible — the same shares of college graduates, the same shares of men, the same shares of high-FICOs, etc.
The advantages to the researcher of experimental data are obvious, but it’s not often available in social research
for a variety of reasons:
- Experiments are expensive. Potential subjects have to be located and induced to participate in the experiment, sometimes by paying them. Subjects often have to be monitored over long periods of time. For example, since mortgage delinquencies typically don’t begin to appear until several years into the life of a loan, it takes many years to assess the impact of counseling;
- People willing to participate in an experiment may be different from those who are unwilling to participate. And some participants drop out of the experiment before its conclusion. Both factors reintroduce some of the problems of observational data;
- In some cases, it may be unethical to offer a beneficial treatment to some participants but not to others.
This problem is more frequent in medical research, but it does occur in some social research.
A final problem that affects both observational and experimental studies is clearly defining the effect. For example, how can we assess the effectiveness of, say, a smartphone app that promises to increase your intelligence? We can’t observe intelligence directly. We can measure changes in a variety of IQ and related tests. We can measure changes in performance on specific tasks, like remembering random number sequences. But it’s not clear that we’re observing changes in intelligence.
Similarly, it can be difficult to clearly define the expected benefit of pre-purchase homeownership counseling.
Do we expect that potential homeowners who receive counseling will be more likely to purchase a home or take on debt? Or do we expect that they will rent for a longer period to build up sufficient cash reserves? Do we expect their credit scores to increase as they manage their credit more effectively? It’s tough to say.
A controlled experiment
In 2014, the Federal Reserve Bank of Philadelphia published the results of a five-year study of the effectiveness of pre-purchase homeownership counseling and financial management skills. In contrast to prior studies, this effort employed an experimental design to overcome the challenges facing the earlier studies that relied on observational data. Only first-time homebuyers were included in the study, and the participants could not previously have applied for a mortgage, received pre-purchase homeownership counseling, have a contract to purchase a home, or already be in a program that required pre-purchase counseling. Participants were randomly assigned to either a treatment group or a control group.
Both the treatment and the control group received a two-hour pre-purchase workshop. The treatment group also received additional one-on-one counseling. The control group received no additional counseling or education services.
The two-hour workshop included information on
- Preparing for homeownership (advantages/disadvantages, affordability),
- Shopping for a house,
- Shopping for a mortgage,
- Applying for a mortgage, and
- Closing and settlement.
In addition, workshop participants received a workbook that contained additional information.
The treatment group received individual guidance on budgeting and their homebuying effort plus any other services offered by the counseling agency, as needed. Twenty-nine percent of the participants in the treatment group opted to use some of these extra services.
All the counselors were required to attend training designed to ensure that the workshops and individual counseling provided consistent information to all participants.
Participants in both groups were tracked for four years after their initial assistance. Credit reports and scores for each participant were obtained annually, and annual follow-up surveys were conducted to track a variety of changes in the participants’ situations.
Results of the experiment
As we noted above, it’s not immediately clear how best to measure the impact of pre-purchase homeownership education and counseling. The researchers in the Philadelphia Fed study chose to measure potential impacts on credit scores; total debt balance on trade lines (that is, auto loan balances, credit card balances, and similar non-mortgage debt balances); and delinquent payments on financial obligations.
Exhibit 1 compares the change in average credit score of the control group — the group that received just the two-hour workshop — to the change in the average score of the treatment group — the group that received individual counseling in addition to the two-hour workshop. The control group had an average increase of 8.5 points in their credit score. The treatment group enjoyed an even-larger 16.2 point increase in average credit score. Both of these increases are statistically significant. However, the 7.7 point difference in the credit score increase in the two groups is not statistically significant. The results in Exhibit 1 suggest that the two-hour workshop had a beneficial impact on all participants’ management of their credit. Individual counseling may have provided an incremental benefit, but the evidence of benefit is strongest for the two-hour workshop.
The lack of a statistically significant difference between the treatment and control groups may simply reflect the limited size of the sample. The high cost of experimental studies tends to limit the number of participants compared to studies of observational data. As it happens, the Philadelphia Fed study included fewer than 1,000 participants. Compare that to the 40,000 participants in the Hirad and Zorn study.
Over the course of the five-year experiment, some, but not all, participants in both groups purchased homes. Exhibit 2 compares the changes in credit score, total non-mortgage debt, and delinquency separately for non-homeowners and homeowners. The purchase of a home may signal some unobserved difference in the financial situation, financial sophistication, or risk tolerance of the participants. Separate comparisons of non-homeowners and homeowners guard against the influence of these types of unobserved differences.
Statistically significant impacts and differences are highlighted in Exhibit 2. For instance, the changes in average credit score are statistically significant for the treatment group (16.3 points for non-homeowners, 16.1 points for homeowners). However the impacts on the control group and the differences in impact across the groups are not statistically significant. This pattern of results may seem a little confusing. After all, Exhibit 1 shows the same pattern of impacts on average credit score, but in Exhibit 1 the impacts are statistically significant for both the control and treatment groups. However dividing the experiment participants into non-homeowners and homeowners in Exhibit 2 decreases the size of the groups, and, as we noted above, it is more difficult to identify statistically significant impacts and differences in smaller groups.
The changes in total non-mortgage debt display an intriguing pattern. Non-homeowners increased their total debt over the course of the experiment, and the treatment group increased their debt more than the control group. In contrast, homeowners decreased their total debt, and the treatment group decreased their debt more than the control group. (Only the $3,109 decrease in debt by homeowners in the treatment group is statistically significant.) Perhaps non-homeowners felt able to increase their non-mortgage debt because they didn’t face the debt burden of a mortgage. Conversely, homeowners may have pared back non-mortgage debt in anticipation of buying a home and taking on a mortgage.
There is some evidence that the individual counseling produced a statistically significant reduction in future delinquency especially among homeowners.
Conclusion
Freddie Mac believes objective, unbiased homebuyer education and counseling can improve the ability of borrowers to make prudent homeownership and home financing choices. The benefit is likely to be greatest for first-time homebuyers, and, as a result, Freddie Mac requires financial literacy education for first-time homebuyers who take advantage of Freddie Mac’s low-down-payment program, Home Possible Advantage.
The Philadelphia Fed’s five-year experiment supports Freddie Mac’s belief in the benefits of pre-purchase homeownership counseling. The two-hour workshop provided to all participants produced statistically significant increases in credit scores. Both the workshop and the individual counseling provided to the treatment group reduced future delinquencies, especially among homeowners. The experimental design employed by the Philadelphia Fed addresses some of the challenges faced by non-experimental studies and increases confidence in earlier research that documented the benefits of both homeownership education and counseling.