People are creatures of habit. We tend to buy the same brand of toothpaste, the same make of car and shop at the same stores that we have in the past. And when it's time to get a mortgage to buy or refinance a home, we often go straight to our regular bank.

Which is a big mistake.

A recent report by the Consumer Financial Protection Bureau found that nearly half of all mortgage borrowers surveyed seriously considered only one lender before applying for a mortgage. Over three-quarters submitted an application to only a single lender when obtaining their loan.

Why is that a problem? Because the same study found that mortgage rates on a conventional 30-year fixed-rate mortgage can vary by more than half a percentage point among lenders. That can mean a difference of more than $70 a month on the payments for a $250,000 mortgage, or as much as $36,000 over the life of the loan.

So yes, it pays to shop around when you're thinking about a mortgage.

Big variations among lenders

To be sure, there's nothing necessarily wrong with getting a mortgage from your regular bank. It could turn out that they're offering the best terms for someone with your credit and financial profile on the type of mortgage you're looking for. But the odds are against it - there's so much variety in the rates and terms offered by different mortgage lenders that it would take a real stroke of luck for your regular bank to turn out to be the one that just happens to have the best deal for you.

For most consumers, a home is the most expensive thing they'll ever buy. The amount of money and the length of time involved in a typical mortgage mean that very small differences in the terms of two competing loan offers can add up to some serious cash. When you take that into account, it's surprising that people aren't more aggressive about shopping for a mortgage than they are.

Differences in mortgage rates are only one factor to consider when shopping for a mortgage, although those can have a very big impact. The following are some other things to consider when comparing loans offered by different lenders - and which may differ significantly from what your regular bank is offering.

Fee structure

All mortgages come with fees, also known as closing costs. These are additional charges the lender adds on in part to cover expenses associated with the loan, such as legal filings and obtaining credit reports, but also as its own fee(s) for originating the loan. That's how they usually make their money (interest typically goes to investors who buy the loan as part of a package of securities).

These fees can vary from lender to lender, not only in the amount charged, but in the fees themselves. One lender may charge for something another does not, or may include three services under a single fee that another lender charges for separately. The names used for various fees and services vary from lender to lender as well, making it hard to make apples to apples comparisons.

No-closing cost loans

Some lenders will offer what are called "no closing cost" mortgages. In reality, the closing costs are either rolled into the loan amount itself, or are covered by charging a slightly higher interest rate to recoup the difference. You're still paying the closing costs in the long run, but doing it this way can make sense for people who want to minimize their out-of-pocket costs up front.

Discount Points

Discount points , often referred to simply as "points," are a particular type of fee that deserve a separate explanation. Most lenders will give you the option of buying a lower interest rate by paying for points. Each point costs 1 percent of the loan amount and will typically reduce your interest rate by one-eighth to one-quarter of a percent.

When comparing closing costs from lender to lender, you first want to compare offers with no points included. That makes it easier to make a comparison of the basic costs and interest rates. Then go ahead and figure in the points, if you choose to do so.

Down payment

Many people don't realize it, but down payment requirements may vary from lender to lender. One lender may look at your profile and the home you wish to purchase and require 10 percent down, while another may allow you to get by with only 5 percent.

One example of this is the new loan programs authorized by Fannie Mae and Freddie Mac that require only 3 percent down on 30-year loans for borrowers with good credit. However, not all lenders currently offer this product. So if you're interested in a minimal down payment without going the FHA route, your regular bank may not offer this particular loan type.

Flexibility

Once you've begun the application process, how much wiggle room do you have if things change? For example, some lenders will allow a one-time adjustment if mortgage rates should fall after you lock in your rate. Some may charge a fee for doing this; others will not. And some banks won't change a locked rate at all. But it's another thing to consider when you're looking at different lenders.

Loan programs

Your regular bank isn't likely to tell you about mortgage options they don't offer. They're going to want your business. But other lenders may offer loan products that are a better fit for your needs.

One example would be a USDA Rural Development Loan. Relatively few lenders offer them - you generally need to contact your local USDA office to get a list. But if you're a first-time homebuyer who falls within the income limits and other qualifications, these no-money-down loans are tough to beat.

Another example would be portfolio loans. These are mortgages that a lender keeps on their own books, or sells them to investors they have a direct relationship with, rather than channeling them through Fannie Mae, Freddie Mac, the FHA or another agency.

As a result, the lender can set its own guidelines rather than following those set by an agency. This can provide greater flexibility for borrowers who cannot or do not want to meet the requirements for agency-backed loans. Such loans are often popular with business owners who have difficulty documenting their earnings or do not wish to open their books to outsiders, but can prove their creditworthiness in other ways. Another example might be someone who went through a recent bankruptcy and has returned to financial health, but can't qualify for a conventional mortgage. Or anyone who is financially capable but can't meet the strict requirements for a conventional loan.

Choosing a lender

When shopping for a mortgage, you want to start out by looking at a variety of lenders - check into large banks, community banks, credit unions, nonbank lenders and mortgage brokers to see what sort of terms they're offering.

Then narrow it down to two or three and ask them to provide detailed quotes and price breakdowns of what they're willing to offer. Put your requests in to all of them on the same morning or afternoon, so you know they're working from the same market rates, which can change every few hours. You might even go ahead and submit an application to each, so that they need to respond with a detailed good faith estimate breaking down all the costs. Then compare the offers side to side and make your choice.

You don't even have to go to a lender with an office in your area. These days, mortgages are commonly shopped and applied for online, with relevant documents and other information submitted electronically. You can even do business with a lender in a different state, as long as they're licensed to do business in yours. When it comes time to close, you meet with an attorney or title agency who's acting on behalf of the lender to sign all the paperwork, meeting in their office or sometimes even in your own home.

It can be time-consuming and may seem like a lot of work, but when you consider the amount of money that's going to be involved, thoroughly investigating your mortgage options is time well spent.